Solution: By using
Cayley-Hamilton theorem, we know that $M$ satisfies its characteristic equation and hence
\[
M ^3 + \alpha M ^2 + \beta M - I = 0,
\]
where $I$ is the identity matrix of size $3$ and $0$ is the zero matrix of size $3$.
We will analyze each option to see which of them is not true.
Option A
The given relation is $M(I - \beta M) = M^{-1} (M + \alpha I)$ which implies,
\begin{align*}
M(I - \beta M) = M^{-1} (M + \alpha I) & \implies M - \beta M^2 = I + \alpha M^{-1} \\
& \implies M^2 - \beta M^3 - M - \alpha I = 0,
\end{align*}
which can not be true.
Option B
The relation is $M(I + \beta M) = M^{-1} (M-\alpha I)$. Multiplyng with $M$, we get
\begin{align*}
M^2(I + \beta M) = (M-\alpha I) & \implies M^2 + \beta M^3 - M + \alpha I = 0,
\end{align*}
which is again not true.
Option C
The given option is $M^{-1} (M^{-1} + \beta I) = M - \alpha I$. Multiplying with $M^2$ on both sides we get,
\begin{align*}
M (M^{-1} + \beta I) = M^3 - \alpha M^2 & \implies I + \beta M - M^3 + \alpha M^2,
\end{align*}
which is not true.
Option D
Finally, the last option is $M^{-1} (M^{-1} - \beta I) = M + \alpha I$. Let us multiply by $M^2$, we get
\begin{align*}
M (M^{-1} - \beta I) = M^2 + \alpha M^2 & \implies M^3 + \alpha M^2 + \beta M - I = 0
\end{align*}
Hence, Option (D) is correct.