Krdytkyu

If matrices $A$ and $B$ satisfy the definition of similar matrix, i.e. $B=PAP^{-1}$, then we can interpret $A$ and $B$ are the same linear transformation under different basis. But my question is how to "grok" this interpretation by just look at the definition of similar matrix?

Sure. $P$ is a linear transformation which takes things to a new basis. after applying $B$, you bring your (transformed) basis back to where it used to be (transformed). If this is the same thing as your original transformation ($A$), then $A$ and $B$ must fundamentally be the same transform, in the different basis. I visualize it with a rotation example: rotate to a new basis ($P$), do your rotation $B$, and rotate back $(P^{-1})$. If this is the same as one rotation $A$, then they were the same.

$Startrightarrow^P Rotated$

$downarrow Ahspace{40 pt}downarrow B$

$End leftarrow^{P^{-1}} Twice$

If you end at the same place then $A$ and $B$ must be the same.

I always understood that the matrix $P$ is a change of basis matrix from the domain to itself. So as the codomain and the domain are the same vector space, the definition is saying $A$ and $B$ are similar if transforming by $A$ is the same as changing basis, then transforming via $B$, then changing back again.
Hope it helps.

I'm not sure if this helps, but...

The way I think of it is that if $pi in mathbb{R}^n$ is the representation of the vector $v$ in the basis formed from the columns of $P$ (ie, $v = sum pi_i p_i$), then $Ppi$ is the representation of $v$ in the basis $e_1,...e_n$ (since $v = sum pi_i p_i = sum pi_i P e_i = P sum pi_i e_i = sum [P pi]_i e_i$).

Specifically, this means that $P$ is the identity mapping with different bases for the domain and range ($v$ in basis $p_1,...,p_n$ maps to $v$ in basis $e_1,...,e_n$).

Similarly, $P^{-1}$ is the identity mapping with the domain and range bases swapped.

So, I think of the $P$ and $P^{-1}$ in the expression $P^{-1}AP$ as identity mappings, and both $A$ and $B$ represent the same linear operator in some 'coordinate-free' space.

My current understanding, not sure this helps:

$B=PAP^{-1}$ means $BP=PA$, so if $A$ is a diagonal, $BP=PA$ is just the definition of eigenvetors, which is easy to understand. Because we know a matrix and its diagonal form are the same transformation under different basis, so to extrapolate to non-diagonal cases, we can define any square matrix of the form $B'P=PA'$ or $B'=PA'P^{-1}$ representing the same transformation under different basis, called similar matrix.

Edit: $B=PAP^{-1}$ means $A=P^{-1}BP$, and think of $P$ as the change of basis matrix, then $P^{-1}BP$ means change of basis first, then do the linear transformation, then change the basis back, so $A$ and $B$ represent the same linear transformation under different bases. See here: https://youtu.be/P2LTAUO1TdA?t=9m12s