Hi Atharva,

My first step would be to graph these. Hopefully you've already done that. Second step is to solve the two equations x^{2} - 5 = x and x^{2} - 2 = 5. These give us

x = 1/2 ± √(21/4) ≈ -1.79, 2.79

x = 1/2 ± 3/2 = -1, 2

Therefore, we're expecting solutions in the rough area of -1.79 to -1 and from 2 to 2.79. Now, let's look closely at how our greatest integer function behaves in those areas, starting with the positive. For x values between 2 and 2.999999..., [x] = 2. We see that that horizontal line intercepts y = x^{2}-2 at 2. In order to figure out where the right end of that line intercepts y = x^{2}-5, we solve x^{2}-5 = 2 to get that x = √7. So, we have a solution set on the bound:

2 < x < √7

Now we need to consider what the function does on either side of this. At x = 3, [x] jumps up to 3, but if we evaluate x^{2}-5 at 3, we note that curve is already at 4, so there are no further solutions to the right of this interval.

To the left of x = 2, [x] drops down to 1, so there will be another interval for x < 2. To find the left bound of this interval, we set x^{2}-2 = 1 and get x = √3. So, we have a solution on the bound:

√3 < x < 2

Note that 2 isn't a solution, because then [x] = x^{2}-2, and the initial problem had strict inequalities.

For the negative solutions, we note that between -2 ≤ x < -1, [x] = -2. So, we have a solution set with -1 as the right bound. To find the left bound, we set x^{2}-5 = -2 to get x = ±√3. It's the negative solution that applies here, so we have a solution set on:

-√3 < x < -1

We can note that for x < -√3, [x] stays below the lower curve and that at x = -1, [x] jumps above the upper curve and stays there until the positive intervals we already found. So our complete solution set is:

-√3 < x < -1

√3 < x < 2

2 < x < √7

If you have additional questions on this, feel free to let me know.