Perform a rotation of axes to eliminate the xy-term.
(Use x2 and y2 for the
rotated coordinates.)
7x2 − 6 sqrt(3) xy + 13y2 − 64 = 0