Raymond B. answered 08/31/22
g(x) has no inverse function, until you take account of the restriction x>1
y=x^2-2x
switch x and y
x=y^2 -2x
solve for the new y
x+1 = y^2-2x+1
x+1 = (y-1)^2
y-1 =+/-sqr(x+1)
y = 1+/-sqr(x+1)
but ignore the positive square root, due to the restriction x>1 for g(x)
y =g^-1(x) = 1+sqr(x+1)
it may help to draw a rough sketch.
g(x) is the right half an upward opening parabola if x>1
g^-1(x) is the bottom half a rightward opening parabola, if x>1 for g(x)
g^-1(g(x)) = g^-1(x^2-2x+1)
= 1+sqr(x-1)^2
= 1+x-1= x
g(g^-1(x)) also = x
the composite function of inverse functions of x = x