Generative Adversarial Nets

개 발광 어·2023년 5월 18일
0

Generative_Model

목록 보기
1/5
post-thumbnail

Adversarial nets

  • Generator : generates samples by passing random noise through a multilayer perceptron
  • Discriminator : learns to determine whether a sample is from the model distribution or the data distribution

Training

Value function V(G,D)V(G,D)

D and G play two-player minmax game with value function :

minGmaxDV(D,G)=Expdata(x)[logD(x)]+Ezpz(z)[log(1D(G(z)))]\min_G \max_D V(D, G) = \mathbb{E}_{\boldsymbol{x} \sim p_{data}(\boldsymbol{x})}[\log D(\boldsymbol{x})] + \mathbb{E}_{\boldsymbol{z} \sim p_{z}(\boldsymbol{z})}[\log(1 - D(G(\boldsymbol{z})))]

  • Discriminator D\mathcal{D} tries to make D(x)=1D(\boldsymbol{x})=1, and D(G(z)))=0D(G(\boldsymbol{z})))=0
    • classify real data to 1, fake one to 0
  • Generator GG tries to make D(G(z)))=1D(G(\boldsymbol{z})))=1
    • "deceive" the discriminator to classify fake data to 1

Algorithm of Training GAN

  • optimizing the DD to completion on finite datasets is prohibitive
    • computationally expensive
    • lead to overfitting
  • k steps of optimizing DD, while only one step of optimizing GG

Experiments

  • trained based on MNIST, the Toronto Face Database (TFD), and CIFAR-10
  • generator : mixture of rectifier linear activations
  • discriminator
    • maxout activations
    • dropout applied when training
  • input noise z to only the bottom layer

Advantages and Disadvantages

Advantages

  • Generator can be updated without data examples, and only with discriminator's gradient flow
  • can represent sharp, even degenerate distributions

Disadvantages

  • no explicit representation of pg(x)p_g(x)
  • D should be well synced with the generator, or else can lead to mode collapse
profile
개발하는 광어입니다.

0개의 댓글