1.강의 내용

[DL Basic]Generative Models 1 & 2

• Learning Generative Model

  Suppose we are given images of dogs.
  We want to learn a probability distribution p(x) such that
  Generation: If we sample xnew ∼ p(x), xnew should look like a dog (sampling).
  Density estimation: p(x) should be high if x looks like a dog, and low otherwise (anomaly detection).
  Also known as, explicit models.
  Unsupervised representation learning: We should be able to learn what these images have in common, e.g., ears, tail, etc (feature learning).

• Basic Discrete Distributions

  Bernoulli distribution: (biased) coin flip
  Categorical distribution: (biased) m-sided dice

• Structure through Independence

  What if X1, ..., Xn are independent, then p(x1, ..., xn) = p(x1)p(x2)⋯p(xn)
  How many possible states? 2^n
  How many parameters to specify p(x1, ..., xn)? n
  2^n entries can be described by just n numbers! But this independence assumption is too strong to model useful distributions.

• Auto_regressive model

  uppose we have 28 × 28 binary pixels.
  Our goal is to learn p(x) = p(x1, ..., x784) over x ∈ {0,1}^784.
  How can we parametrize p(x)?
  Let's use the chain rule to factor the joint distribution.
  p(x1:784) = p(x1)p(x2 | x1)p(x3 | x1:2)⋯
  This is called an autoregressive model.
  Note that we need an ordering of all random variables.

2.과제 수행 과정/결과물 정리

완료

3.피어 세션

학습 내용 공유

1.과제 코드 리뷰

• 진행 X

1.강의 내용 및 심화내용 토론

[DL Basic]Generative Models 1
[DL Basic]Generative Models 2

3.논문 리뷰

1.DenseNet

4.학습회고

금일 있었던 논문 리뷰는 발표자 분께서 직접 구현까지 하셔서 보여주었습니다. 곧 있을 대회 전 주요 cnn 모델들은 스스로 구현을 해봐야할 것 같습니다. 이번주 배웠던 내용들 중 상당수가 많은 내용을 담기 위해 간결하게 설명되다 보니 주말을 이용하여 좀더 살펴봐야할 것 같습니다. 특히 Transforemr 부분은 장기간에 걸쳐 빈번하게 살펴봐야할 것 같습니다.