Answer:
[tex]A'=PAP^{-1}[/tex]
Step-by-step explanation:
The first that you must ask is whether or not the matrix [tex]A[/tex] is a diagonalizable matrix. If so, you have to find a basis of [tex]\mathbf{R}^{3}[/tex] consisting of eigenvectors of the matrix [tex]A[/tex]. To do that first find the eigenvalues of the matrix and the the corresponding eigenspaces to that eigenvalues. This will give you the basis of [tex]\mathbb{R}^{3}[/tex] consisting of eigenvectors of A. Next apply the change of coordinates using the matrix of change of coordinates [tex]P[/tex] whose columns are the vectors on the new basis. So we have that
[tex]D=A'=PAP^{-1}[/tex]
is the matrix of T respect to new basis consisting of eigenvectors of T, and it is a diagonal matrix, and its elements on the diagonal are the eigenvalues.