It's hard too know how to answer this problem because I'm not sure where you are in the "Calculus curriculum". If this is the only information given, then the problem is a multi-variable problem requiring Calc 3 type methods. However, we can do this problem using Calc 1 methods if we make a simple assumption. Let's initially assume that a = b.
The equation for the surface is:
SA = ab + 2ac + 2bc and, letting a = b it becomes:
SA = a2 + 2ac + 2ac = a2 + 4ac and since we know SA = 27, we can say:
27 = a2 + 4ac
27 - a2 = 4ac
c = (27 - a2)/4a
The volume is V = abc but since a = b and c = (27 - a2)/4a, we can say:
V(a) = a2(27 - a2)/4a
V(a) = a(27 - a2)/2
V(a) = 27a/2 - a3/2
To maximize, take the derivative and set it equal to zero:
V'(a) = 27/2 - 3a2/2
27/2 - 3a2/2 = 0
27 - 3a2 = 0
9 - a2 = 0
a2 = 9
a = 3
Since we assumed a = b, then b = 3
c = (27 - a2)/4a = (27 - 32)/4•3 = 18/12 = 1.5
So the volume = 3•3•1.5 = 13.5 cm3
Now we can test our assumption by letting a and b NOT be equal.
SA = ab + 2ac + 2bc
27 = ab + 2ac + 2bc
27 - 2bc = ab + 2ac
27 - 2bc = a(b + 2c)
a = (27 - 2bc)/(b + 2c)
So Volume is:
V = abc
V = [(27 - 2bc)/(b + 2c)]bc
Let's let b = 2.9 and then let b = 3.1 but maintain c = 1.5 and see what happens to the volume:
For b = 2.9:
V = [(27 - 2•2.9•1.5)/(2.9 + 2•1.5)]•2.9•1.5 = 13.4924
For b = 3.1
V = [(27 - 2•3.1•1.5)/(3.1 + 2•1.5)]•3.1•1.5 = 13.4926
So, we indeed see that the volume of 13.5 cm3 is a maximum.
You can repeat the same thing by testing different values of c (rather than maintaining it at 1.5) and you get similar results.
Using multi-variable calculus, you would take partial derivatives but it gets a lot more complicated. If that's what you are working on, my apologies.