Mark M. answered 09/25/15
Tutor
4.9
(954)
Retired college math professor. Extensive tutoring experience.
When an expression involves "nested" grouping symbols (parentheses, brackets, braces, etc), simplify, starting with the innermost set of grouping symbols and work your way outward. So, in the given expression, deal with ( ) first, then [ ], and then { }.
x - x{x - x[x - x(x+1)]/x3} = x - x{x - x[x - x2 - x]/x3}
= x - x{x - x[-x2]/x3}
= x - x{x + x3/x3}
= x - x{x + 1}
= x - x2 - x = -x2