Tom K. answered 08/06/21
Knowledgeable and Friendly Math and Statistics Tutor
You don't have to use calculus for this problem. (3x-y)^2 = 0 on y = 3x, and this is the minimum. The function is constant on any line with slope 3 and is otherwise strictly convex along any line. Thus, the maxima will be at the vertices (the slope does not equal 3 on any boundaries). Thus, plugging in the three vertices, the maximum is at (2, 0) and is 36.
The answer is a, though no reason to specifically mention (1, 3) unless we also want to point out the other endpoint of the line segment, which is (1/2, 3/2) - this is where y = 3x meets x + y = 2, one of the boundaries.