🔎 Stanford CS231n 강의를 듣고 기록용도로 정리하는 게시글입니다.

Field of CV

• an interdisciplinary field
• ex)physics - to understand image formation
• ex)biology - to understand how animal brains actually see and understand things

history of vision

• Back to 543 million years ago, there were only few species of animals
• zoologists found out that the number of animal species exploded (= evolution's big bang)
  -> what caused this?
• most convincing theory
  • about 540 million years ago, first animals developed eyes
  • and the onset of the vision started this exposive speciation phase => animals can suddenly see => life becomes more proactive (escaping from the predators, attack... etc
  • started evolutionary arms race, and to survive they had to evolve quickly
• visual = biggest sensory system that enables us to communicate, entertain ... etc.
  => vision is important to animals (especially intelligent animals)

camera obscura

• 초기 카메라 obscura = 초기 동물들에게서 발전했던 눈과 굉장히 비슷함

hubel and wiesel, 1959

• "what was the visual processing mechanism like in primates, in mammals?" 라는 질문에서 시작
• study cat brain (more less similar to humans)
• 시각 피질이 있는 뒷쪽에 전극
  • many types of cells in 시각 피질
  • but simple cells respond to oriented edges when they move in certain directions

=> visual processing starts with simple structure of the visual world, oriented edges, and the information moves along, the brain builds up the complexity of the visual information until it can recognize the complex visual world

block world

• larry roberts - first PhD thesis of computer vision.
• the famous MIT summer project "The Summer Vision Project" - goal is to work out the bulk of the visual system
• field of cv - not yet solved vision but one of the most important and fastest growing ai field

david marr

• MIT vision scientist
• several processes to arrive at a final holistic full 3d representation of the visual world

1. primal sketch
   • mostly the edges, bars, ends, virtual lines, curves, boundaries...
   • early processing has a lot to do with simple structures like edges
2. 2 1/2-D sketch
   • piece together the surfaces, depth information, layers, discontinuities, ...
3. 3D model
   • put everything together
   • hierarchically organized

=> idealized thought process of what vision is, and this way of thinking dominated cv for several decades..

recognition via parts

• "how can we move beyond the simple block world and represent real world"

• "generalized cylinder", "pictorial structure"

  • every object is composed of simple geometric primitives (ex. 사람의 형태를 실린더 여러 개로 표현)

=> representation is a way to reduce the complex structure of a object into a collection of simpler shapes and their geometric configuration

david lowe

• how to reconstruct or recognize the visual world from simple world structures

---

Image Segmentaion

• "if object recognition is too hard, first do object segmentation"
  • taking an image and group the pixels into meaningful areas
• extract out all the pixels that belong to the pixels from its background = image segmentation
  

---

Face Detection

• AdaBoost algorithm -> real-time face detection
• rapid transfer from basic science research to real world application

---

Feature-based Object Recognition (SIFT feature)

• late 90s ~ 2000
• SIFT feature
• to match and the entire object (ex. stop sign) to another stop sign
  • there could be changes due to camera angles, lighting, viewpoint....
  • but observe that there are some features that tend to remain diagnositc and invariant to changes

=> object recognition began with identifying these critical features on the object and then match the features to a similar object

spatial pyramid matching

• feature in the images that gives us clues about which type of scene it is
• features from different parts of the image -> put together in feature descriptor and SVM algorithm on top of that

early 2000s

• in terms of recognition, object recognition is an important problem to solve
• began to have a benchmark data set, to measure the progress of object recognition
• data set = PASCAL visual object challenge
  • dataset composed of 20 object classes

-> 2006~2012 - performance of detecting objects in the 20 objects -> increased! => a lot of pregress made

---

Overfit problems

• most of the ML algorithms are very likely to overfit in the training process
• and part of the problem is that visual data is very complex
  -> models tend to have high dimension of input, have a lot of parameters...
  -> but not enough training data
  => overfitting happens very fast

ImageNet project (Stanford)

• to recognize whole objects + overcome the overfitting problem

• biggest dataset produced in the field of AI at that time

• began to push forward the algorithm dev of object recog into another face

• benchmark the progress.. - team rolled out an international challenge called ImageNet Large-Scale Visual Recognition Challenge

• more test set to test the image classification rec results for the cv algorithms

• if algorithm can ouput 5 labels and the top 5 includes correct object -> = success

• error rate is steadily decreasing, but 2012, error rate is so low that it's on par with what humans can do (human = 1 standford phd student)

• 2012 -> error rate was dropped almost 10 percent to 16 percent => significant drop, and the winning algorithm of that year = convolution nerual network model

• 이 수업의 목표 = to have a deep dive into what convolution nerual network model is(=deep learning)

---

CS231n overview

• primary focus - image classification problem
• your algorihtm looks at an image -> picks among a fixed set of categories and classify that image
• other problems like object detection, image captioning...
  - object detection - not classifying an entire iamge, but draw a box and say that there is a dog here, cat here ,,, etc
  - image captioning - given an image and produce a natural language sentence describing the image

CNN

• 2011 - multiple layers (features -> local invariances -> feed their resulting descriptor to linear svm) - still hierarchical (= still detecting edges)
• 2012 - 7 layer cnn = alexnet = supervision
• trend = neurals became deeper
• 2015 - googlenet, 19 layers
• 2015.- residual network - 152 layers

Why CNN became popular recently

• CNN was not invented in 2012

1. computation

   • faster computers every year (thx to Moore's law)
   • number of transistors on chips increased
   • advent of GPUS (super parallelizable, perfect tool for computationally intensive cnn models)

2. data

   • these algorithms were hungry for data (a lot of labeled images and labeled pixels), but not enough data available
   • it was before the Internet was widely used
   • difficult to collect large datasets

• these algorithms existed for quite a long time..

Human visual system

• People do a lot of things using their visual system
• So we continue to develop our algorithms to do even better
• ex) semantic segmentation - understand every pixel in the image (what is it doing, what does it mean.. )
• ex) 3d understanding - reconstructing the entire world
• there are tons of other tasks left to be solved

• 이 사진을 1초 보여주고 그 사람에게 사진에 대해 설명하라고 한다면 -> 오른쪽과 같이 이 상황 전체에 대해 유추를 하고, 어디인지, 무엇을 하고 있는 것 같은지 등에 대해 한 단락을 설명한다. => 인간은 이미지를 보고 전체적인 맥락 파악이 가능함

• 이 사진을 보고 인간들이 웃을 수 있는 이유?

• 사람들이 본인의 몸무게에 대해서 약간은 conscious함을 가진다는 것, scale에 발을 올리면 더 무거워진다는 것을 이미 알고 있음 -> 뒤에 있는 사람이 발을 올려서 무게가 더 나가게 하고 있음 -> 사실 그 사람은 버락 오바마로 당시 대통령이고, 우리가 아는 바로 대통령은 보통 respectable 하게, making jokes 하지 않는 사람들임 -> 이 모든 정보를 알고 있으므로 우리는 이 사진을 보고 웃을 수 있음

• 이미지에 내용이 엄청 많음 -> cv 알고리즘은 아직 이 이미지에 대해 진정한 이해를 하기엔 많은 여정이 남았다.