Vaibhav and Richard,
The solution given by Richard above shows a correct approach to the problem, but makes one small but crucial misstep. In the line where the modulus of the expression (z + 3/z) is computed, the given formula is not quite correct. Letting |z|=R, and computing instead the square of the modulus, it should read:
|z + 3/z|2 = |Reiθ + 3/Reiθ|2
= (R + 3/R)2cos2(θ) + (R - 3/R)2sin2(θ)
= R2 + (6cos(2θ)) + 9/R2
and upon equating this with 32 = 9, we obtain
R4 + (6cos(2θ) - 9)R2 + 9 = 0.
In the last equation, which is also quadratic in R2, note that the maximal solution will occur when the coefficient of the R2-term is as negative as possible. This occurs at the minimal value of cos(2θ), which occurs at θ = π/2, and which is of course -1. The resulting equation is then
R4 -15R2 + 9 = 0,
whose maximal solution is
R2 = (15 + √189)/2
and hence the maximal value for R = |z| is
√( (15 + √189)/2 ).
By the way, we now also know a complex number z that takes this maximal modulus solution, namely z = i√( (15 + √189)/2 ).