The Moorse-Penrose pseudoinverse for orthogonal real matrices is
\( A^+ = (A^T A)^{-1} A^T \)
The normal equation is
\(\widehat\beta = (X^T \Sigma^{-1} X)^{-1} X^T \Sigma^{-1} y\)
The hat matrix is given by
\( X (X^T \Sigma^{-1} X)^{-1} X^T \Sigma^{-1} \)
Singular Value decomposition: \(A = U D V^T\) where
LU decomposition: \(A = LU\) where
$$\mathcal{N}(\mathbf{x} | \mathbf{\mu}, \Sigma) = \exp\left( - \frac{1}{2} \mathbf{r}^T \Sigma^{-1} \mathbf{r} \right) , |2 \pi \Sigma|^{-\frac{1}{2}}$$
where \(\mathbf{r} = \mathbf{x} - \mathbf{\mu}\). Note that the \(2 \pi\) is inside the determinant.
linear operations on Gaussian random variables: If \( x \sim \mathcal{N}(\mu_x, \Sigma_X)\), and \(y \sim \mathcal{N}(\mu_y, \Sigma_y)\), then
Product of Gaussian PDFs: \(\mathcal{N}(\mathbf{x} \vert \alpha, \Sigma) \mathcal{N}(\mathbf{x} \vert \beta, \Omega) = \eta \mathcal{N}(\mathbf{x} | \mathbf{m}, C)\), where
Refactoring the product of heirarchical Gaussian PDFs: \(\mathcal{N}(\mathbf{x} \vert M \theta, C) \mathcal{N}(\theta \vert \mu, \Lambda) = \mathcal{N}(\theta \vert \mathbf{a}, A) \mathcal{N}(\mathrm{x} \vert \mathrm{b}, B)\), where
Sources: The Matrix Cookbook, Hogg+ 2020