Variational Auto-Encoder

Variational Auto-Encoder (VAE)

• AE encoder directly produces a latent vector $z$ (single value for each attribute).
Then, AE decoder takes these values to reconstruct the original input.

Goal is getting a compressed latent vector of the input.
• VAE encoder produces 2 latent vectors mean µ and standard deviation σ of Gaussian N(µ, σ2) approximating the posterior distribution p(z| x) of each attribute (Variational inference).

Then, using sampled latent vectors from the distributions, the decoder reconstructs the input.
Goal is generating many variations of the input (probabilistic encoder + generative decoder).

VAE encoder would like to produce the posterior $p(z|x)$ instead of a single point $z$, but because of the intractability of $p(z|x)$, we approximate it by variational posterior $q_θ(z|x)$ which is Gaussian $N(µ_x, σ^2_x)$ so that the encoder network $q_θ$ is enforced to produce $µ_x$ and σx.

Reparametrization trick