Mark M. answered 03/31/20
Retired Math prof with teaching and tutoring experience in trig.
In quadrant 2, cosine is negative and sine is positive
sina = √[1-cos2a] = 5/13 and cosB = -√[1-sin2B] = -√3/2
So, cos(a-B) = (cosa)(cosB) + (sina)(sinB) = (-12/13)(-√3/2) + (5/13)(1/2) = (12√3 + 5)/26