Day1

🟢교재 기반 복습

https://github.com/onlybooks/pytorch.git

• jax ⇒ low code로 설계 가능. 속도가 더 빠르고, 높은 성능을 기대할 수 있다.
• 면접에서 딥러닝 프레임워크 뭐 사용해봤는지 물어보면 pytorch라고 답변
• 머신러닝 프레임워크 - 사이킷런
• 면접에서 웹 프레임워크 뭐 사용해봤는지 물어보면 fastapi,django라고 답변

머신러닝 배포

• fastapi, mlflow 배포

GPT → Generative pre-trained transformer

잔차 신경망 - Residual network ⇒ Resnet

• gradient 이슈를 피하고 깊게 학습시킬 수 있다.

층이 깊어지면 gradient vanishing 문제,

혹은 gradient exploding (overflow error) 발생 가능

🟢torchvision에서 모델 로드

• 이미지 처리 관련 pretrained model 등록

from torchvision import models

모델 호출

model = models.AlexNet()
model

AlexNet(
  (features): Sequential(
    (0): Conv2d(3, 64, kernel_size=(11, 11), stride=(4, 4), padding=(2, 2))
    (1): ReLU(inplace=True)
    (2): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)
    (3): Conv2d(64, 192, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2))
    (4): ReLU(inplace=True)
    (5): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)
    (6): Conv2d(192, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (7): ReLU(inplace=True)
    (8): Conv2d(384, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (9): ReLU(inplace=True)
    (10): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (11): ReLU(inplace=True)
    (12): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)
  )
  (avgpool): AdaptiveAvgPool2d(output_size=(6, 6))
  (classifier): Sequential(
    (0): Dropout(p=0.5, inplace=False)
    (1): Linear(in_features=9216, out_features=4096, bias=True)
    (2): ReLU(inplace=True)
    (3): Dropout(p=0.5, inplace=False)
    (4): Linear(in_features=4096, out_features=4096, bias=True)
    (5): ReLU(inplace=True)
    (6): Linear(in_features=4096, out_features=1000, bias=True)
  )
)

• cnn은 커널에 weight 존재

🟢딥러닝 모델 소개

🔵사전학습된 모델 활용 방법

• transfer learning - 전이학습
  • 미리 학습된 모델의 지식을 가져와서 내 문제에 적용
  • 훈련 가능한 파라미터와 불가능한 파라미터가 다르다.
  • 뒷부분 쪽 층만 변형시키고 싶으면 transfer learning
• fine-tuning
  • 가져온 모델의 가중치를 내 데이터에 맞게 미세하게 조정
  • 앞쪽 층까지 변형

시간과 돈 절약 → transfer learning

🔵GAN

generative adversarial nets

https://baechu-story.tistory.com/12

• generator와 discriminator 둘이 생성 위조 지폐에 대해 평가하고 생성하고를 반복함
• 구분이 불가한 수준이 되면 위조지폐를 생산함

⇒ 악용 사례 : deep fake.

🔵stable diffusion 모델

• GAN보다 대세..

🔵딥러닝 양자화 기술의 기반

• 소수점 이진수 변환

• 컴퓨터에 9.1 저장되는 방식
• floating point - IEEE754 format

+)Edge AI

• 엣지 단말기에서 고성능 컴퓨팅 기능을 제공하여 사용

• 갤럭시 온디바이스 ai

  • 인터넷이 없는 환경에서도 번역 등의 태스크 수행 가능

• Q4

  • 16비트에서 4비트까지 전환하는 기술

• unsqueeze()

  • CHW ⇒ HWC 순서로 변경 때 사용

🔵역전파

• 역전파 할 때 loss function으로 값 조절
• 얼마나 값이 다른지를 판단하여 가중치 값을 조절

output = model(image) ⇒ inference

loss= criterion(output, label…)

backward() → 오차를 통한 역전파 진행

optimizer.step() ⇒ 역전파를 통해 얻은 값으로 업데이트

• sgd ⇒ 딥러닝 관련 포지션 지원시 면접 질문 가능

🔵minimum에 관하여

local minimum, global minimum

https://en.wikipedia.org/wiki/Maximum_and_minimum

🔵optimizer

1차 미분: gradient

• SGD
• Momentum

2차 미분: 모멘텀

• adagrad
  • 자주 업데이트 되는 파라미터는 작게 ,드문드문 되는 파라미터는 작게
• rmsprop
  • EMA 사용하여 AdaGrad의 문제점 해결
• adam
  • rmsprop + momontum

로젠브록 함수

https://ko.wikipedia.org/wiki/%EB%A1%9C%EC%A0%A0%EB%B8%8C%EB%A1%9D_%ED%95%A8%EC%88%98

import numpy as np
import matplotlib as plt
def rosenblock(x,y):
    return (1-x)**2 +100 *(y-x**2)**2

• 최적화 검사용 함수

미분 함수

def grad_rosenbrock(x, y):
    dx = -2 * (1 - x) - 400 * x * (y - x**2)
    dy = 200 * (y - x**2)
    return np.array([dx, dy])

옵티마이저 선언

• sgd 수식

def optimizer(optimizer_name, start_pos, lr, epochs, **kwargs):
    x,y = start_pos
    path=[(x,y)]

    for i in range(1, epochs+1):
        grad = grad_rosenbrock(x,y)

        update = -lr *grad
        x+=update[0]
        y+=update[1]
        path.append((x,y))
        
    return np.array(path)

로슨블록 함수 값 생성

x_range=np.linspace(-2,2,300)
y_range=np.linspace(-1,3,250)

X,y = np.meshgrid(x_range, y_range)
z= rosenbrock(X,y)

SGD 실행

start_pos=(-1.5,1.5)
lr= 0.0005
results = optimizer('SGD',start_pos, lr, epochs =2000)

results

array([[-1.5       ,  1.5       ],
       [-1.47725   ,  1.5075    ],
       [-1.45706627,  1.51424768],
       ...,
       [-1.19362667,  1.43256133],
       [-1.19359391,  1.43248316],
       [-1.19356115,  1.432405  ]])

확률적 경사하강법 시각화

import matplotlib.pyplot as plt

plt.figure(figsize=(12, 8))
plt.contour(X,y, z, levels=np.logspace(-0.5, 3.5, 20), cmap='jet', alpha=0.3)
plt.plot(results[:, 0], results[:, 1], color='black', linewidth=2)

plt.show()

모멘텀 옵티마이저 추가

• 가변 변수 꺼낼 때 딕셔너리로 담기므로 get으로 호출해오기

def optimizer(optimizer_name, start_pos, lr, epochs, **kwargs):
    x,y = start_pos
    path=[(x,y)]
    v= np.zeros(2)

    for i in range(1, epochs+1):
        grad = grad_rosenbrock(x,y)

        if optimizer_name =='SGD':
            update = -lr *grad
        elif optimizer_name == 'Momentum':
            beta = kwargs.get('beta')
            v= beta *v +grad #sgd와 다른 점 : beta 추가
            update = -lr *v

        x+=update[0]
        y+=update[1]
        path.append((x,y))

    return np.array(path)

시각화

import matplotlib.pyplot as plt

plt.figure(figsize=(12, 8))
plt.contour(X,y, z, levels=np.logspace(-0.5, 3.5, 20), cmap='jet', alpha=0.3)
plt.plot(momentum[:, 0], momentum[:, 1], color='red', linewidth=2,label = "Momentum")
plt.plot(results[:, 0], results[:, 1], color='black', linewidth=2,label = "SGD")

plt.legend()
plt.show()

• tanh → -1~1 범위의 활성화 함수
• softmax에서 로그 붙였더니 식이 분해가 됨 → log - softmax
• cross entropy loss 사용 때 softmax 사용이 2번으로 겹칠 수 있음
  • output이 linear로 출력

VGGnet 은 깊이가 점점 깊어진다.

깊이가 깊어질 때 일어날 수 있는 부분.

resnet은 잔차 학습을 하여 이 부분이 해결된다.

🟢transfer learning

🔵강아지 품종 분류 데이터를 분석한다

import matplotlib.pyplot as plt 
from PIL import Image
import scipy.ndimage as spi
plt.imshow(plt.imread("/kaggle/input/dog-breed-identification/train/000bec180eb18c7604dcecc8fe0dba07.jpg"))

import pandas as pd 
label = pd.read_csv("/kaggle/input/dog-breed-identification/labels.csv")
display(label)

import os 

print(f'train -> {len(os.listdir("/kaggle/input/dog-breed-identification/train"))}' )
print(f'test -> {len(os.listdir("/kaggle/input/dog-breed-identification/test"))}' )

train -> 10222
test -> 10357

EDA 및 전처리.

import torch
from torch.utils.data import Dataset, DataLoader
from torchvision import transforms

print(label.breed.unique().size)
print(label.breed.value_counts().describe())

unique_breeds = sorted(label.breed.unique())

120
count    120.000000
mean      85.183333
std       13.298122
min       66.000000
25%       75.000000
50%       82.000000
75%       91.250000
max      126.000000
Name: count, dtype: float64

품종 인덱스 처리

breed_to_idx = {i:breed for i, breed in enumerate(unique_breeds)}
b_to_idx = {val : key for  key, val in breed_to_idx.items()}
label['target'] = label.breed.map(b_to_idx)

데이터 쪼개기

from sklearn.model_selection import train_test_split

y= label.pop('target')
X_train, X_test, y_train, y_test = train_test_split(label, y, test_size=0.2, random_state=42, stratify=y)

Dataset, Dataloader

class DogBreedDataset(Dataset):
    def __init__(self, df, y, root_dir, transform=None):
        self.df = df 
        self.y = y
        self.root_dir = root_dir
        self.transform = transform

    def __len__(self):
        return len(self.df)

    def __getitem__(self, idx):
        if torch.is_tensor(idx):
            idx = idx.tolist()

        # 사진이 있는 경로 
        img_id = self.df.iloc[idx,0]
        img_name = os.path.join(self.root_dir,img_id ) + ".jpg"

        image = Image.open(img_name).convert('RGB')
        label = self.y.iloc[idx]

        return image, label     
        

이미지 데이터 호출하기

import numpy as np
tmp_iter =iter(tmp)
data  = next(tmp_iter)
print(np.array(data[0]).shape)

trans = transforms.Compose([
    transforms.ToTensor()## 순서 변경
])

print(trans(data[0]).shape)

(440, 280, 3)
torch.Size([3, 440, 280])

• toTensor → 순서 변경

크기변경 추가

import numpy as np
tmp_iter =iter(tmp)
data  = next(tmp_iter)
print(np.array(data[0]).shape)

trans = transforms.Compose([
    transforms.Resize((224,224)),
    transforms.ToTensor()## 순서 변경
])

print(trans(data[0]).shape)

(440, 280, 3)
torch.Size([3, 224, 224])

위 내용을 Dataset 클래스에 추가

class DogBreedDataset(Dataset):
    def __init__(self, df, y, root_dir, transform=None):
        self.df = df 
        self.y = y
        self.root_dir = root_dir
        self.transform = transform

    def __len__(self):
        return len(self.df)

    def __getitem__(self, idx):
        if torch.is_tensor(idx):
            idx = idx.tolist()

        # 사진이 있는 경로 
        img_id = self.df.iloc[idx,0]
        img_name = os.path.join(self.root_dir,img_id ) + ".jpg"

        image = Image.open(img_name).convert('RGB')
        label = self.y.iloc[idx]

        if self.transform == True:
            trans = transforms.Compose([
                            transforms.Resize((224,224)),
                            transforms.ToTensor()
                        ])
            image = trans(image)

        return image, label     
        

값 확인

tmp = DogBreedDataset(X_train, y_train, "/kaggle/input/dog-breed-identification/train", transform=True)
        
tmp_iter = iter(tmp)

data  = next(tmp_iter)

data[0]

tensor([[[0.8588, 0.8471, 0.8431,  ..., 0.4353, 0.4353, 0.4314],
         [0.8667, 0.8549, 0.8549,  ..., 0.4431, 0.4392, 0.4392],
         [0.8745, 0.8667, 0.8627,  ..., 0.4471, 0.4431, 0.4431],
         ...,
         [0.3686, 0.3451, 0.3255,  ..., 0.3961, 0.3843, 0.3882],
         [0.3725, 0.3529, 0.3412,  ..., 0.3961, 0.3882, 0.3843],
         [0.3647, 0.3569, 0.3529,  ..., 0.3804, 0.3725, 0.3647]],

vgg 모델은 데이터셋 imagenet 사용.

• 사전학습된 모델을 활용할 것인데, 예측값을 높이기 위해 pre-trained 데이터셋이 가지고 있는 값의 특성 가지도록 조절(mean, std)

  데이터셋 클래스 최종 선언

• 수정 사항 파이프라인으로 추가됨

class DogBreedDataset(Dataset):
    def __init__(self, df, y, root_dir, transform=None):
        self.df = df 
        self.y = y
        self.root_dir = root_dir
        self.transform = transform

    def __len__(self):
        return len(self.df)

    def __getitem__(self, idx):
        if torch.is_tensor(idx):
            idx = idx.tolist()

        # 사진이 있는 경로 
        img_id = self.df.iloc[idx,0]
        img_name = os.path.join(self.root_dir,img_id ) + ".jpg"

        image = Image.open(img_name).convert('RGB')
        label = self.y.iloc[idx]

        if self.transform == True:
            trans = transforms.Compose([
                            transforms.Resize((224,224)),
                            transforms.ToTensor(),
                            transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
                        ])
            image = trans(image)

        return image, label     

각각 만들기

train_dataset = DogBreedDataset(X_train, y_train, "/kaggle/input/dog-breed-identification/train", transform=True)
train_loder = DataLoader(train_dataset, batch_size=32, shuffle=True)

test_dataset = DogBreedDataset(X_test, y_test, "/kaggle/input/dog-breed-identification/test", transform=True)
test_loder = DataLoader(test_dataset, batch_size=32, shuffle=True)

print(len(train_dataset), len(test_dataset))

8177 2045

모델 호출

from torchvision.models import VGG16_Weights
from torchvision import models
vgg = models.vgg16(weights=VGG16_Weights.IMAGENET1K_V1)

VGG(
  (features): Sequential(
    (0): Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): ReLU(inplace=True)
    (2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (3): ReLU(inplace=True)
    (4): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (5): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (6): ReLU(inplace=True)
    (7): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (8): ReLU(inplace=True)
    (9): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (10): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (11): ReLU(inplace=True)
    (12): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (13): ReLU(inplace=True)
    (14): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (15): ReLU(inplace=True)
    (16): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (17): Conv2d(256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (18): ReLU(inplace=True)
    (19): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (20): ReLU(inplace=True)
    (21): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (22): ReLU(inplace=True)
    (23): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (24): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (25): ReLU(inplace=True)
    (26): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (27): ReLU(inplace=True)
    (28): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (29): ReLU(inplace=True)
    (30): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  )
  (avgpool): AdaptiveAvgPool2d(output_size=(7, 7))
  (classifier): Sequential(
    (0): Linear(in_features=25088, out_features=4096, bias=True)
    (1): ReLU(inplace=True)
    (2): Dropout(p=0.5, inplace=False)
    (3): Linear(in_features=4096, out_features=4096, bias=True)
    (4): ReLU(inplace=True)
    (5): Dropout(p=0.5, inplace=False)
    (6): Linear(in_features=4096, out_features=1000, bias=True)
  )
)

• dropout은 훈련 때만 사용, 추론 때는 안 사용함.

https://ollama.com/

오픈 모델 공유 사이트.

원래는 32bit 짜리를 fp16 → 절반 줄임

오픈클로에서 사용하는 모델이 양자화되는 모델

q4_K_M → 4비트로 줄이고, 캐시 사용

transformers ⇒ 파이토치 없이 파인튜닝 가능

모델 피쳐 꺼내보기

• 제너레이터 방식이라 for문을 붙일 수 있음

cnn까지는 학습하지 않도록 동결시킴

from torchinfo import summary
summary(vgg)

for param in vgg.features.parameters():
    param.requires_grad  = False

summary(vgg)

=================================================================
Total params: 138,357,544
Trainable params: 123,642,856
Non-trainable params: 14,714,688
=================================================================

fc로 되어있는 부분

vgg.classifier

Sequential(
  (0): Linear(in_features=25088, out_features=4096, bias=True)
  (1): ReLU(inplace=True)
  (2): Dropout(p=0.5, inplace=False)
  (3): Linear(in_features=4096, out_features=4096, bias=True)
  (4): ReLU(inplace=True)
  (5): Dropout(p=0.5, inplace=False)
  (6): Linear(in_features=4096, out_features=1000, bias=True)
)

in_features = vgg.classifier[0].in_features

25088

전이학습

in_features = vgg.classifier[0].in_features

import torch.nn as nn
vgg.classifer= nn.Sequential(
    nn.Linear(in_features,2048),
    nn.ReLU(True),
    nn.Dropout(0.5),
    nn.Linear(2048,120)
)
summary(vgg)

├─Sequential: 1-3                        --
│    └─Linear: 2-32                      102,764,544
│    └─ReLU: 2-33                        --
│    └─Dropout: 2-34                     --
│    └─Linear: 2-35                      16,781,312
│    └─ReLU: 2-36                        --
│    └─Dropout: 2-37                     --
│    └─Linear: 2-38                      4,097,000
├─Sequential: 1-4                        --
│    └─Linear: 2-39                      51,382,272
│    └─ReLU: 2-40                        --
│    └─Dropout: 2-41                     --
│    └─Linear: 2-42                      245,880
=================================================================
Total params: 189,985,696
Trainable params: 175,271,008
Non-trainable params: 14,714,688
=================================================================

Day2

🟢딥러닝 성능에 영향을 주는 정규화

🔵 batch normalization

• 내부 공변량에 변화를 대처할 수 있다.
• 내부적으로 공변량이 변하기 때문에 정규화해서 이 성능을 올려보고자 함(1)
• 손실함수를 smoothing 하여 gradient 예측을 쉽게 한다.(논문 기반)

mini batch 데이터는 랜덤이다

위스키

• 보리
  • 수확 시기, 보관 상태에 따라 당분 함량, 수분기 다를 수 있음
  • 매번 다른 보리 상태 → 효모 → 발효
    • T분포를 활용하여 소량의 샘플을 통해 대량의 원료 품질을 관리할 수 있을지에 대한 고민
    • 내부 공변량 변화,internal covariate shift (효모의 양을 조절)
  • 표준화 작업이 진행됨, (표준 당도와 온도를 유지하고자 함)

파이토치에서 batch normalization 제공

🟢전이학습과 파인튜닝을 시도한다

🔵강아지 이미지 뜨는지 확인

import matplotlib.pyplot as plt 
from PIL import Image
import scipy.ndimage as spi
import os
import zipfile
from tqdm import tqdm
import matplotlib.pyplot as plt 
from PIL import Image
plt.imshow(plt.imread("/kaggle/input/dog-breed-identification/train/000bec180eb18c7604dcecc8fe0dba07.jpg"))

🔵모델 선언 및 라벨 선언

import pandas as pd 
label = pd.read_csv("/kaggle/input/dog-breed-identification/labels.csv")
unique_breeds = sorted(label.breed.unique())
breed_to_idx = {i:breed for i, breed in enumerate(unique_breeds)}
b_to_idx = {val : key for  key, val in breed_to_idx.items()}
label['target'] = label.breed.map(b_to_idx)
from sklearn.model_selection import train_test_split
y = label.pop('target')
X_train, X_test, y_train, y_test = train_test_split(label, y, test_size=0.2, random_state=42, stratify=y)
import torch
from torch.utils.data import Dataset, DataLoader
from torchvision import transforms
class DogBreedDataset(Dataset):
    def __init__(self, df, y, root_dir, transform=None):
        self.df = df 
        self.y = y
        self.root_dir = root_dir
        self.transform = transform
        self.transforms = transforms.Compose([
                            transforms.Resize((224,224)),
                            transforms.ToTensor(),
                            transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
                        ])

    def __len__(self):
        return len(self.df)

    def __getitem__(self, idx):
        if torch.is_tensor(idx):
            idx = idx.tolist()

        # 사진이 있는 경로 
        img_id = self.df.iloc[idx,0]
        img_name = os.path.join(self.root_dir,img_id ) + ".jpg"

        image = Image.open(img_name).convert('RGB')
        label = self.y.iloc[idx]

        if self.transform == True:
            image = self.transforms(image)

        return image, label     
        

🔵데이터셋과 데이터 로더

train_dataset = DogBreedDataset(X_train, y_train, "./input/train", transform=True)
train_loader = DataLoader(train_dataset, batch_size=32, shuffle=True)
test_dataset = DogBreedDataset(X_test, y_test, "./input/train", transform=True)
test_loader = DataLoader(test_dataset, batch_size=32, shuffle=True)

🔵모델 선언 및 라벨 선언

from torchvision.models import VGG16_Weights
from torchvision import models
import torch.nn as nn
class VGG16TransferLearning(nn.Module):
    def __init__(self, num_classes: int, mode: str = 'feature_extraction'):
        super(VGG16TransferLearning, self).__init__()

        self.backbone = models.vgg16(weights=VGG16_Weights.IMAGENET1K_V1)

        # 2. Feature Extractor (conv layers) 동결 설정
        if mode == 'feature_extraction':
            # 모든 conv layer 파라미터 동결 (gradient 계산 중단)
            for param in self.backbone.features.parameters():
                param.requires_grad = False  
               
        elif mode == 'fine_tuning':
            # 마지막 Conv Block (block 4, 5)만 학습 가능하도록 설정
            # VGG16의 features는 [0-30] 인덱스까지 (31개 layer)
            freeze_until = 24  # block 3까지 동결 (인덱스 0-23)
            for idx, param in enumerate(self.backbone.features.parameters()):
                if idx < freeze_until:
                    param.requires_grad = False
                else:
                    param.requires_grad = True
                    print(f"Fine-tuning enabled at layer index: {idx}")

        in_features = self.backbone.classifier[6].in_features
        self.backbone.classifier[6] = nn.Linear(in_features, num_classes)
     

        self._initialize_weights()

    def _initialize_weights(self):
        """새로 추가된 FC 레이어 가중치 초기화"""
        for m in self.backbone.classifier.modules():
            if isinstance(m, nn.Linear):
                nn.init.kaiming_normal_(m.weight, mode='fan_out', nonlinearity='relu')
                if m.bias is not None:
                    nn.init.constant_(m.bias, 0)

    def forward(self, x):
        return self.backbone(x)
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
device
NUM_CLASSES = 120  # 예: CIFAR-10 또는 커스텀 데이터셋
BATCH_SIZE = 32   # VGG16은 메모리를 많이 사용하므로 16-32 권장
NUM_EPOCHS = 30
MODE = 'feature_extraction'  # 또는 'feature_extraction'
model = VGG16TransferLearning(num_classes=NUM_CLASSES, mode=MODE)
model = model.to(device)
from torchinfo import summary
summary(model)
criterion = nn.CrossEntropyLoss()
import torch.optim as optim
if MODE == 'feature_extraction':
    # FC layer만 학습하므로 일반적인 LR 사용 가능
    optimizer = optim.Adam(
        filter(lambda p: p.requires_grad, model.parameters()),
        lr=1e-4,
        weight_decay=1e-4  # L2 정규화 (과적합 방지)
    )
else:  # fine_tuning
    # 전체 네트워크 미세 조정 시 매우 작은 LR 필요
    # VGG16은 깊은 네트워크이므로 SGD + Momentum이 더 안정적
    optimizer = optim.SGD(
        filter(lambda p: p.requires_grad, model.parameters()),
        lr=1e-4,          # ImageNet 학습률의 1/10 이하
        momentum=0.9,     # 관성항으로 안정적 수렴
        weight_decay=5e-4
    )
def train_model(model, dataloaders, criterion, optimizer,
                num_epochs=25, device='cuda'):
    """
    전이학습 학습 루프

    Args:
        model: VGG16TransferLearning 인스턴스
        dataloaders: {'train': DataLoader, 'val': DataLoader}
        criterion: 손실 함수 (CrossEntropyLoss)
        optimizer: optimizer (SGD with momentum 권장)
    """
    best_acc = 0.0
    history = {'train_loss': [], 'train_acc': [], 'val_loss': [], 'val_acc': []}

    for epoch in range(num_epochs):
        print(f'\nEpoch {epoch+1}/{num_epochs}')
        print('-' * 60)

        # 각 epoch마다 train -> val phase 순환
        for phase in ['train', 'val']:
            if phase == 'train':
                model.train()  # Dropout 활성화, BN 학습 모드
            else:
                model.eval()   # Dropout 비활성화, BN 추론 모드

            running_loss = 0.0
            running_corrects = 0

            # 진행률 표시
            pbar = tqdm(dataloaders[phase], desc=phase)

            for inputs, labels in pbar:
                inputs = inputs.to(device, non_blocking=True)
                labels = labels.to(device, non_blocking=True)

                # gradient 누적 초기화
                optimizer.zero_grad()

                # Forward pass
                with torch.set_grad_enabled(phase == 'train'):
                    outputs = model(inputs)
                    _, preds = torch.max(outputs, 1)  # 예측 클래스
                    loss = criterion(outputs, labels)

                    # Backward + Optimize (train phase only)
                    if phase == 'train':
                        loss.backward()

                        # Gradient Clipping (VGG16은 깊어서 안정성을 위해 권장)
                        torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)

                        optimizer.step()

                # 통계 계산
                running_loss += loss.item() * inputs.size(0)
                running_corrects += torch.sum(preds == labels.data)

                # tqdm 업데이트
                pbar.set_postfix({'loss': loss.item()})

            # Epoch 통계
            epoch_loss = running_loss / len(dataloaders[phase].dataset)
            epoch_acc = running_corrects.double() / len(dataloaders[phase].dataset)

            history[f'{phase}_loss'].append(epoch_loss)
            history[f'{phase}_acc'].append(epoch_acc.item())

            print(f'{phase} Loss: {epoch_loss:.4f} Acc: {epoch_acc:.4f}')

            # 최적 모델 저장 (validation 기준)
            if phase == 'val' and epoch_acc > best_acc:
                best_acc = epoch_acc
                best_model_wts = model.state_dict().copy()
                torch.save(best_model_wts, 'best_vgg16_transfer.pth')

        # 학습률 업데이트
        # if scheduler is not None:
        #     if isinstance(scheduler, optim.lr_scheduler.ReduceLROnPlateau):
        #         scheduler.step(epoch_loss)  # val loss 기준
        #     else:
        #         scheduler.step()

    print(f'\nBest val Acc: {best_acc:.4f}')
    model.load_state_dict(best_model_wts)
    return model, history

dataloaders = {'train' : train_loader , 'val' : test_loader}
model, history = train_model(
        model, dataloaders, criterion, optimizer,
        num_epochs=NUM_EPOCHS, device=device
    )

K-ICT 딥러닝 GPU 대여

🟢잔차 학습

• 깊이가 깊어지면 보통 성능이 오르지만, 특정 구간이 지나면 성능이 떨어지는 이상현상 발견
  • 작은 값이면 기울기 소실, 큰 값이면 기울기 폭주
• 이에 잔차 학습을 도입한다.
• 수학적으로 항등원이라고 부른다.
• 잔차 학습은 미분해도 x가 1로 변하기 때문에 기울기가 아예 소실되는 것을 방지할 수 있다.
• transformer 모델에도 이 기능이 포함되어 있음
• f(x)가 잔차이다.

항목	역할
x	identity / shortcut
F(x)	residual (학습 대상)
x + F(x)	최종 출력

• 과적합을 확인할 수 있는 것이 validation loss와 train loss의 의미

🔵모델 학습 결과값

0.6753

🔵loss값 시각화

epochs = range(1, len(history['train_loss']) + 1)

# 그래프 그리기 시작
plt.figure(figsize=(14, 5))

# 1. Loss 그래프
plt.subplot(1, 2, 1)
plt.plot(epochs, history['train_loss'], 'b-', label='Train Loss')
plt.plot(epochs, history['val_loss'], 'r-', label='Validation Loss')
plt.title('Training and Validation Loss')
plt.xlabel('Epochs')
plt.ylabel('Loss')
plt.legend()
plt.grid(True)

# 2. Accuracy 그래프
plt.subplot(1, 2, 2)
plt.plot(epochs, history['train_acc'], 'b-', label='Train Acc')
plt.plot(epochs, history['val_acc'], 'r-', label='Validation Acc')
plt.title('Training and Validation Accuracy')
plt.xlabel('Epochs')
plt.ylabel('Accuracy')
plt.legend()
plt.grid(True)

plt.tight_layout()
plt.show()

• epoch 5 부터 과적합 양상을 확인할 수 있음

🔵데이터 보강 및 파라미터 추가

import torch
from torch.utils.data import Dataset, DataLoader
from torchvision import transforms
class DogBreedDataset(Dataset):
    def __init__(self, df, y, root_dir, transform=None):
        self.df = df 
        self.y = y
        self.root_dir = root_dir
        self.transform = transform

    def __len__(self):
        return len(self.df)

    def __getitem__(self, idx):
        if torch.is_tensor(idx):
            idx = idx.tolist()

        # 사진이 있는 경로 
        img_id = self.df.iloc[idx,0]
        img_name = os.path.join(self.root_dir,img_id ) + ".jpg"

        image = Image.open(img_name).convert('RGB')
        label = self.y.iloc[idx]

        if self.transform:
            image = self.transform(image)

        return image, label     
        

train_transforms = transforms.Compose([
    transforms.RandomResizedCrop(224),
    transforms.RandomHorizontalFlip(),
    transforms.RandomRotation(15),
    transforms.ColorJitter(brightness=0.1, contrast=0.1),
    transforms.ToTensor(),
    transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
])

val_transforms = transforms.Compose([
    transforms.Resize(256),
    transforms.CenterCrop(224),
    transforms.ToTensor(),
    transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
])

train_dataset = DogBreedDataset(X_train, y_train, "/kaggle/input/dog-breed-identification/train", transform=train_transforms)
train_loader = DataLoader(train_dataset, batch_size=32, shuffle=True)
test_dataset = DogBreedDataset(X_test, y_test, "/kaggle/input/dog-breed-identification/train", transform=val_transforms)
test_loader = DataLoader(test_dataset, batch_size=32, shuffle=True)

model = VGG16TransferLearning(num_classes=NUM_CLASSES, mode=MODE)
model = model.to(device)

criterion = nn.CrossEntropyLoss() 
import torch.optim as optim
if MODE == 'feature_extraction':
    # FC layer만 학습하므로 일반적인 LR 사용 가능
    optimizer = optim.Adam(
        filter(lambda p: p.requires_grad, model.parameters()),
        lr=1e-4,
        weight_decay=1e-4  # L2 정규화 (과적합 방지)
    )
else:  # fine_tuning
    # 전체 네트워크 미세 조정 시 매우 작은 LR 필요
    # VGG16은 깊은 네트워크이므로 SGD + Momentum이 더 안정적
    optimizer = optim.SGD(
        filter(lambda p: p.requires_grad, model.parameters()),
        lr=1e-4,          # ImageNet 학습률의 1/10 이하
        momentum=0.9,     # 관성항으로 안정적 수렴
        weight_decay=5e-4
    )

dataloaders = {'train' : train_loader , 'val' : test_loader}
model, history = train_model(
        model, dataloaders, criterion, optimizer, 
        num_epochs=NUM_EPOCHS, device=device
    )

🔵파인튜닝할게………………

# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using "Save & Run All"
# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session
import os
import zipfile
from tqdm import tqdm
import matplotlib.pyplot as plt
import pandas as pd
label = pd.read_csv("/kaggle/input/dog-breed-identification/labels.csv")
unique_breeds = sorted(label.breed.unique())
breed_to_idx = {i:breed for i, breed in enumerate(unique_breeds)}
b_to_idx = {val : key for  key, val in breed_to_idx.items()}
label['target'] = label.breed.map(b_to_idx)
from sklearn.model_selection import train_test_split
y = label.pop('target')
X_train, X_test, y_train, y_test = train_test_split(label, y, test_size=0.2, random_state=42, stratify=y)
import torch
from torch.utils.data import Dataset, DataLoader
from torchvision import transforms

import torch
from torch.utils.data import Dataset, DataLoader
from torchvision import transforms

class DogBreedDataset(Dataset):
    def __init__(self, df, y, root_dir, transform=None):
        self.df = df
        self.y = y
        self.root_dir = root_dir
        self.transform = transform

    def __len__(self):
        return len(self.df)

    def __getitem__(self, idx):
        if torch.is_tensor(idx):
            idx = idx.tolist()

        # 사진이 있는 경로
        img_id = self.df.iloc[idx,0]
        img_name = os.path.join(self.root_dir,img_id ) + ".jpg"

        image = Image.open(img_name).convert('RGB')
        label = self.y.iloc[idx]

        if self.transform:
            image = self.transform(image)

        return image, label    
       

train_transforms = transforms.Compose([
    transforms.RandomResizedCrop(224),
    transforms.RandomHorizontalFlip(),
    transforms.RandomRotation(15),
    transforms.ColorJitter(brightness=0.1, contrast=0.1),
    transforms.ToTensor(),
    transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
])

val_transforms = transforms.Compose([
    transforms.Resize(256),
    transforms.CenterCrop(224),
    transforms.ToTensor(),
    transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
])

train_dataset = DogBreedDataset(X_train, y_train, "/kaggle/input/dog-breed-identification/train", transform=train_transforms)
train_loader = DataLoader(train_dataset, batch_size=32, shuffle=True)
test_dataset = DogBreedDataset(X_test, y_test, "/kaggle/input/dog-breed-identification/train", transform=val_transforms)
test_loader = DataLoader(test_dataset, batch_size=32, shuffle=True)

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
device
NUM_CLASSES = 120  # 예: CIFAR-10 또는 커스텀 데이터셋
BATCH_SIZE = 32   # VGG16은 메모리를 많이 사용하므로 16-32 권장
NUM_EPOCHS = 60
MODE = 'fine_tuning'  # 또는 'feature_extraction'

from torchvision.models import VGG16_Weights
from torchvision import models
import torch.nn as nn

#====================================================================================
# vgg 전이학습 
#====================================================================================
class VGG16TransferLearning(nn.Module):
    def __init__(self, num_classes: int, mode: str = 'feature_extraction'):
        super(VGG16TransferLearning, self).__init__()

        self.backbone = models.vgg16(weights=VGG16_Weights.IMAGENET1K_V1)

        # 2. Feature Extractor (conv layers) 동결 설정
        if mode == 'feature_extraction':
            # 모든 conv layer 파라미터 동결 (gradient 계산 중단)
            for param in self.backbone.features.parameters():
                param.requires_grad = False  
               
        elif mode == 'fine_tuning':
            # 마지막 Conv Block (block 4, 5)만 학습 가능하도록 설정
            # VGG16의 features는 [0-30] 인덱스까지 (31개 layer)
            freeze_until = 21  # block 3까지 동결 (인덱스 0-23)
            for idx, param in enumerate(self.backbone.features.parameters()):
                if idx < freeze_until:
                    param.requires_grad = False
                else:
                    param.requires_grad = True
                    print(f"Fine-tuning enabled at layer index: {idx}")

        in_features = self.backbone.classifier[6].in_features
        self.backbone.classifier[6] = nn.Linear(in_features, num_classes)
     

        #self._initialize_weights()

    def _initialize_weights(self):
        """새로 추가된 FC 레이어 가중치 초기화"""
        for m in self.backbone.classifier.modules():
            if isinstance(m, nn.Linear):
                nn.init.kaiming_normal_(m.weight, mode='fan_out', nonlinearity='relu')
                if m.bias is not None:
                    nn.init.constant_(m.bias, 0)

    def forward(self, x):
        return self.backbone(x)
#====================================================================================
# 훈련 함수
#====================================================================================

def train_model(model, dataloaders, criterion, optimizer,
                num_epochs=25, device='cuda'):
    """
    전이학습 학습 루프
    Args:
        model: VGG16TransferLearning 인스턴스
        dataloaders: {'train': DataLoader, 'val': DataLoader}
        criterion: 손실 함수 (CrossEntropyLoss)
        optimizer: optimizer (SGD with momentum 권장)
    """
    best_acc = 0.0
    history = {'train_loss': [], 'train_acc': [], 'val_loss': [], 'val_acc': []}

    for epoch in range(num_epochs):
        print(f'\nEpoch {epoch+1}/{num_epochs}')
        print('-' * 60)

        # 각 epoch마다 train -> val phase 순환
        for phase in ['train', 'val']:
            if phase == 'train':
                model.train()  # Dropout 활성화, BN 학습 모드
            else:
                model.eval()   # Dropout 비활성화, BN 추론 모드

            running_loss = 0.0
            running_corrects = 0

            # 진행률 표시
            pbar = tqdm(dataloaders[phase], desc=phase)

            for inputs, labels in pbar:
                inputs = inputs.to(device, non_blocking=True)
                labels = labels.to(device, non_blocking=True)
                
                # gradient 누적 초기화
                optimizer.zero_grad()

                # Forward pass
                with torch.set_grad_enabled(phase == 'train'):
                    outputs = model(inputs)
                    _, preds = torch.max(outputs, 1)  # 예측 클래스
                    loss = criterion(outputs, labels)

                    # Backward + Optimize (train phase only)
                    if phase == 'train':
                        loss.backward()

                        # Gradient Clipping (VGG16은 깊어서 안정성을 위해 권장)
                        torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)

                        optimizer.step()

                
                # 통계 계산
                running_loss += loss.item() * inputs.size(0)
                running_corrects += torch.sum(preds == labels.data)

                # tqdm 업데이트
                pbar.set_postfix({'loss': loss.item()})

            # Epoch 통계
            epoch_loss = running_loss / len(dataloaders[phase].dataset)
            epoch_acc = running_corrects.double() / len(dataloaders[phase].dataset)

            history[f'{phase}_loss'].append(epoch_loss)
            history[f'{phase}_acc'].append(epoch_acc.item())

            print(f'{phase} Loss: {epoch_loss:.4f} Acc: {epoch_acc:.4f}')

            
            #=========================================================
            # 모델 저장하는 영역 /최적 모델 저장 (validation 기준)
            #=========================================================
            
            if phase == 'val' and epoch_acc > best_acc:
                best_acc = epoch_acc
                best_model_wts = model.state_dict().copy()
                torch.save(best_model_wts, 'best_vgg16_transfer.pth')

        # 학습률 업데이트
        # if scheduler is not None:
        #     if isinstance(scheduler, optim.lr_scheduler.ReduceLROnPlateau):
        #         scheduler.step(epoch_loss)  # val loss 기준
        #     else:
        #         scheduler.step()

    
    print(f'\nBest val Acc: {best_acc:.4f}')
    model.load_state_dict(best_model_wts)
    return model, history
#====================================================================================
# 실행을 위한 선언 - fine_tuning
#====================================================================================

model = VGG16TransferLearning(num_classes=NUM_CLASSES, mode='fine_tuning')
model = model.to(device)
criterion = nn.CrossEntropyLoss()
import torch.optim as optim
if MODE == 'feature_extraction':
    # FC layer만 학습하므로 일반적인 LR 사용 가능
    optimizer = optim.Adam(
        filter(lambda p: p.requires_grad, model.parameters()),
        lr=1e-4,
        weight_decay=1e-4  # L2 정규화 (과적합 방지)
    )
else:  # fine_tuning
    # 전체 네트워크 미세 조정 시 매우 작은 LR 필요
    # VGG16은 깊은 네트워크이므로 SGD + Momentum이 더 안정적
    optimizer = optim.SGD(
        filter(lambda p: p.requires_grad, model.parameters()),
        lr=1e-2,          # ImageNet 학습률의 1/10 이하
        momentum=0.9,     # 관성항으로 안정적 수렴
        weight_decay=5e-4
    )
#====================================================================================
# 실행을 위한 선언 - 모델 돌리기
#====================================================================================
dataloaders = {'train' : train_loader , 'val' : test_loader}
model, history = train_model(
        model, dataloaders, criterion, optimizer,
        num_epochs=NUM_EPOCHS, device=device
    )

• mode 바꾸기.
• epoch 수 추가해보기
• 가중치 초기화는 함수를 잘못 설계한 경향이 있으므로 주석처리를 진행

🟢로컬에서 모델을 돌려보자

🔵환경 설정

• streamlit, torchvision 설치

🔵노트북 불러오기

import torch
weights = torch.load("./best_vgg16_transfer.pth", map_location=torch.device('cpu'))
weights.keys()

odict_keys(['backbone.features.0.weight', 'backbone.features.0.bias', 'backbone.features.2.weight', 'backbone.features.2.bias', 'backbone.features.5.weight', 'backbone.features.5.bias', 'backbone.features.7.weight', 'backbone.features.7.bias', 'backbone.features.10.weight', 'backbone.features.10.bias', 'backbone.features.12.weight', 'backbone.features.12.bias', 'backbone.features.14.weight', 'backbone.features.14.bias', 'backbone.features.17.weight', 'backbone.features.17.bias', 'backbone.features.19.weight', 'backbone.features.19.bias', 'backbone.features.21.weight', 'backbone.features.21.bias', 'backbone.features.24.weight', 'backbone.features.24.bias', 'backbone.features.26.weight', 'backbone.features.26.bias', 'backbone.features.28.weight', 'backbone.features.28.bias', 'backbone.classifier.0.weight', 'backbone.classifier.0.bias', 'backbone.classifier.3.weight', 'backbone.classifier.3.bias', 'backbone.classifier.6.weight', 'backbone.classifier.6.bias'])

가중치 찍어보기

weights['backbone.features.0.weight']

tensor([[[[-5.5373e-01, 1.4270e-01, 5.2896e-01],
[-5.8312e-01, 3.5655e-01, 7.6566e-01],
[-6.9022e-01, -4.8019e-02, 4.8409e-01]],

     [[ 1.7548e-01,  9.8630e-03, -8.1413e-02],
      [ 4.4089e-02, -7.0323e-02, -2.6035e-01],
      [ 1.3239e-01, -1.7279e-01, -1.3226e-01]],

모델 구조 선언

from torchvision.models import VGG16_Weights
from torchvision import models
import torch.nn as nn
class VGG16TransferLearning(nn.Module):
    def __init__(self, num_classes: int, mode: str = 'feature_extraction'):
        super(VGG16TransferLearning, self).__init__()

        self.backbone = models.vgg16(weights=VGG16_Weights.IMAGENET1K_V1)

        # 2. Feature Extractor (conv layers) 동결 설정
        if mode == 'feature_extraction':
            # 모든 conv layer 파라미터 동결 (gradient 계산 중단)
            for param in self.backbone.features.parameters():
                param.requires_grad = False  
               
        elif mode == 'fine_tuning':
            # 마지막 Conv Block (block 4, 5)만 학습 가능하도록 설정
            # VGG16의 features는 [0-30] 인덱스까지 (31개 layer)
            freeze_until = 21  # block 3까지 동결 (인덱스 0-23)
            for idx, param in enumerate(self.backbone.features.parameters()):
                if idx < freeze_until:
                    param.requires_grad = False
                else:
                    param.requires_grad = True
                    print(f"Fine-tuning enabled at layer index: {idx}")
        in_features = self.backbone.classifier[6].in_features
        self.backbone.classifier[6] = nn.Linear(in_features, num_classes)
     

NUM_CLASSES = 120
model = VGG16TransferLearning(num_classes=NUM_CLASSES, mode='fine_tuning')

Downloading: "https://download.pytorch.org/models/vgg16-397923af.pth" to C:\Users\rosie/.cache\torch\hub\checkpoints\vgg16-397923af.pth
100%|██████████| 528M/528M [00:39<00:00, 13.9MB/s]
Fine-tuning enabled at layer index: 21
Fine-tuning enabled at layer index: 22
Fine-tuning enabled at layer index: 23
Fine-tuning enabled at layer index: 24
Fine-tuning enabled at layer index: 25

가중치 불러와지는지 확인

model.load_state_dict(weights)

cpu로 등록

model=model.to('cpu')

추론 모델 설정

model.eval()

VGG16TransferLearning(
(backbone): VGG(
(features): Sequential(
(0): Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(1): ReLU(inplace=True)
(2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(3): ReLU(inplace=True)
(4): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(5): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(6): ReLU(inplace=True)
(7): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(8): ReLU(inplace=True)
(9): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(10): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(11): ReLU(inplace=True)
(12): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(13): ReLU(inplace=True)
(14): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(15): ReLU(inplace=True)
(16): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(17): Conv2d(256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(18): ReLU(inplace=True)
(19): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(20): ReLU(inplace=True)
(21): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
...
(5): Dropout(p=0.5, inplace=False)
(6): Linear(in_features=4096, out_features=120, bias=True)
)
)
)

강아지 인덱스 설정

dog_dict={0: 'affenpinscher',
 1: 'afghan_hound',
 2: 'african_hunting_dog',
 3: 'airedale',

val_transform을 함수로 선언하기

from torchvision import transforms
def process_image(imgage):
    transform = transforms.Compose([
        transforms.Resize(256),
        transforms.CenterCrop(224),
        transforms.ToTensor(),
        transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
    ])
    return transform(image).unsqueeze(0) # 차원 올리기 
****

강아지 이미지 확인

from PIL import Image
img = Image.open("./papilion.webp")
img

tensor로 이미지 변경

img_tensor=process_image(img).to('cpu')
img_tensor.shape

torch.Size([1, 3, 224, 224])

추론 시작

with torch.no_grad():
    output =model(img_tensor)
output.shape

torch.Size([1, 120])

결과값 확률로 보기 위해 softmax 에 넣기

import torch.nn as nn
pred= nn.functional.softmax(output[0],dim=0)
pred.sum()

tensor(1.0000)

• softmax에 태웠으므로 값이 나옴

값 확인

neo4j 공부 필요

🔵진짜 로컬에 옮겨보자

import streamlit as st
import torch
import torch.nn as nn
from torchvision import models, transforms
from PIL import Image
import pandas as pd
import numpy as np
from torchvision.models import VGG16_Weights
import pickle

with open("./label.pkl", "rb") as f:
    data = pickle.load(f)

class VGG16TransferLearning(nn.Module):
    def __init__(self, num_classes: int, mode: str = 'feature_extraction'):
        super(VGG16TransferLearning, self).__init__()

        self.backbone = models.vgg16(weights=VGG16_Weights.IMAGENET1K_V1)

        # 2. Feature Extractor (conv layers) 동결 설정
        if mode == 'feature_extraction':
            # 모든 conv layer 파라미터 동결 (gradient 계산 중단)
            for param in self.backbone.features.parameters():
                param.requires_grad = False  
               
        elif mode == 'fine_tuning':
            # 마지막 Conv Block (block 4, 5)만 학습 가능하도록 설정
            # VGG16의 features는 [0-30] 인덱스까지 (31개 layer)
            freeze_until = 21  # block 3까지 동결 (인덱스 0-23)
            for idx, param in enumerate(self.backbone.features.parameters()):
                if idx < freeze_until:
                    param.requires_grad = False
                else:
                    param.requires_grad = True
                    print(f"Fine-tuning enabled at layer index: {idx}")
        in_features = self.backbone.classifier[6].in_features
        self.backbone.classifier[6] = nn.Linear(in_features, num_classes)
     
    def forward(self, x):
        return self.backbone(x)

def process_image(image):
    transform = transforms.Compose([
        transforms.Resize(256),
        transforms.CenterCrop(224),
        transforms.ToTensor(),
        transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
    ])
    return transform(image).unsqueeze(0) # 차원 올리기 

weights = torch.load("./best_vgg16_transfer.pth", map_location=torch.device('cpu'))

NUM_CLASSES = 120
model = VGG16TransferLearning(num_classes=NUM_CLASSES, mode='fine_tuning')
model.load_state_dict(weights)
model = model.to('cpu')

st.title("강아지 품종 분류 서비스")
st.write("사진 업로드 하시면 강아지의 품종을 알려드립니다.")

upload_file = st.file_uploader("강아지 사진 선택하기", type=['jpg', 'png', 'webp'])

if upload_file is not None:
    image = Image.open(upload_file).convert('RGB')
    st.image(image, caption='upload image')

    if st.button("품종확인"):
        with st.spinner("분석중...."):
            img_tensor = process_image(image).to('cpu')

            with torch.no_grad():
                output = model(img_tensor)
                pred = nn.functional.softmax(output[0], dim=0)
                pred_idx = int(torch.topk(pred, 1)[-1][0])
                breed = data['label'][pred_idx]
        st.success("분석 완료")
        st.markdown(f"### 강이지 품종 : {breed}")

강사님 포트에 접근

Day3

🟢yolo를 써보자

🔵환경 설정

from ultralytics import YOLO
model = YOLO("yolov8m-pose.pt")

results = model('https://imgnews.pstatic.net/image/139/2026/02/11/0002242158_001_20260211074307959.jpg?type=w647')

results[0].show()

🔵각 요소 확인해보기

for x in results:
    boxes = x.boxes
    #print(boxes)

    print(f"boxes =>{boxes.xyxy}")
    print(f"conf =>{boxes.conf}")
    print(f"class =>{boxes.cls}")
    
    

boxes =>tensor([[7.7162e+01, 4.1559e+01, 2.9320e+02, 3.3366e+02],
[2.3542e+02, 1.3726e+01, 4.6752e+02, 3.3299e+02],
[4.7876e-01, 5.1264e+01, 1.0486e+02, 2.6688e+02],
[5.2438e+02, 7.3492e+01, 6.0000e+02, 3.3457e+02],
[5.2321e+02, 1.9812e+02, 6.0000e+02, 3.3621e+02]])
conf =>tensor([0.9364, 0.9327, 0.5135, 0.4057, 0.3233])
class =>tensor([0., 0., 0., 0., 0.])

🔵다른 모델 활용

seg_model = YOLO('yolov8m-seg.pt')
results = seg_model('https://imgnews.pstatic.net/image/139/2026/02/11/0002242158_001_20260211074307959.jpg?type=w647')
results[0].show()

🔵다른 모델 활용

pose_model = YOLO('yolov8m-pose.pt')
results=pose_model("./kick.webp")
results[0].show()

🔵유튜브 데이터를 다운받아보자

import yt_dlp
url = "https://www.youtube.com/watch?v=akLRbdTtD7Y"

ydl_opts = {
    'format': 'best[height<=720]',
    'outtmpl': '%(title)s.%(ext)s',}

with yt_dlp.YoutubeDL(ydl_opts) as ydl:
    ydl.download([url])

import cv2
capture = cv2.VideoCapture("./이것이 스쿼트 자세다! - 1분만 들어봐!.mp4")
capture.get(cv2.CAP_PROP_FRAME_COUNT)

2360.0

def predict(frame, iou=0.7, conf=0.25):
    results = model(source=frame,
            device='cpu',
            iou=iou ,
            conf=conf ,
            verbose=False,
            )
    return results[0]

def draw_boxes(result, frame):
    for boxes in result.boxes:
        x1, y1, x2, y2, score, classes = boxes.data.squeeze().cpu().numpy()
        cv2.rectangle(frame, (int(x1), int(y1)), (int(x2), int(y2)), (0, 0, 255), 1)
    return frame

from ultralytics.utils.plotting import Annotator
def draw_keypoints(result, frame):
    annotator = Annotator(frame, line_width=1)
    for kps in result.keypoints:
        kps = kps.data.squeeze()
        annotator.kpts(kps)
       
        nkps = kps.cpu().numpy()
        # nkps[:,2] = 1
        # annotator.kpts(nkps)
        for idx, (x, y, score) in enumerate(nkps):
            if score > 0.5:
                cv2.circle(frame, (int(x), int(y)), 3, (0, 0, 255), cv2.FILLED)
                cv2.putText(frame, str(idx), (int(x), int(y)), cv2.FONT_HERSHEY_COMPLEX, 1, (0, 0, 255), 1)
       
    return frame

🔵 예측 및 키포인트 그리기 함수

def predict(frame, iou=0.7, conf=0.25):
    results = model(source=frame,
            device='cpu',
            iou=iou ,
            conf=conf ,
            verbose=False,
            )
    return results[0]

def draw_boxes(result, frame):
    for boxes in result.boxes:
        x1, y1, x2, y2, score, classes = boxes.data.squeeze().cpu().numpy()
        cv2.rectangle(frame, (int(x1), int(y1)), (int(x2), int(y2)), (0, 0, 255), 1)
    return frame

from ultralytics.utils.plotting import Annotator
def draw_keypoints(result, frame):
    annotator = Annotator(frame, line_width=1)
    for kps in result.keypoints:
        kps = kps.data.squeeze()
        annotator.kpts(kps)
       
        nkps = kps.cpu().numpy()
        # nkps[:,2] = 1
        # annotator.kpts(nkps)
        for idx, (x, y, score) in enumerate(nkps):
            if score > 0.5:
                cv2.circle(frame, (int(x), int(y)), 3, (0, 0, 255), cv2.FILLED)
                cv2.putText(frame, str(idx), (int(x), int(y)), cv2.FONT_HERSHEY_COMPLEX, 1, (0, 0, 255), 1)
       
    return frame

🔵카메라 켜기

capture = cv2.VideoCapture(0)
capture.set(cv2.CAP_PROP_FRAME_WIDTH, 640)
capture.set(cv2.CAP_PROP_FRAME_HEIGHT, 480)
while True:
    ret, frame = capture.read()
    result = predict(frame)
    frame = draw_boxes(result, frame)
    frame = draw_keypoints(result, frame)
    frame = cv2.flip(frame, 1)

    # cv2.putText(frame, text, position, font, scale, color, thickness)
    if not ret:
        print("카메라 오류")Q
        break
    # print(type(frame))
    cv2.imshow("VideoFrame", frame)

    if cv2.waitKey(10) & 0xFF == ord('q'):
        capture.release()
        cv2.destroyAllWindows()
        break

🔵 영상 기반으로 추출해보기

import cv2
from tqdm import tqdm

import cv2
capture = cv2.VideoCapture("./이것이 스쿼트 자세다! - 1분만 들어봐!.mp4")
frame_cnt = capture.get(cv2.CAP_PROP_FRAME_COUNT)
from tqdm import tqdm 
output_filename = 'result_video2.mp4'
fourcc = cv2.VideoWriter_fourcc(*'mp4v')
width = int(capture.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(capture.get(cv2.CAP_PROP_FRAME_HEIGHT))
fps = capture.get(cv2.CAP_PROP_FPS)

writer = cv2.VideoWriter(output_filename, fourcc, fps, (width, height))
for x in tqdm(range(int(frame_cnt))):
    ret, frame = capture.read()
    if not ret:
        print('영상오류')
        cv2.destroyAllWindows()
        break
    state_frame = capture.get(cv2.CAP_PROP_POS_FRAMES)
    if state_frame % 5 == 0:
        result = predict(frame)

    frame = draw_boxes(result, frame)
    frame = draw_keypoints(result, frame)
    # cv2.imshow("Video Display", frame)
    writer.write(frame)
    
    if cv2.waitKey(1) & 0xFF == ord('q'):
        print("사용자에 의한 종료")
        capture.release()
        cv2.destroyAllWindows()
        break
capture.release()
writer.release()
cv2.destroyAllWindows()

🟢resnet

• 병목현상 도입 이유 ⇒ 레이어를 추가하여 연산량 확장.
• 대량의 통에서 위스키를 만들다 보면 매 번 들어가는 원료의 품질에 따라 컨디션이 다를 수 있음.
  • 이 부분을 조절해야한다는 니즈가 존재.(내부 공변량 변화 문제) ICS 문제

🔵미니배치 대상 정규화

• 배치 정규화
• 스케일 과 시프트 변환.
• 입실론은 값을 0으로 만들지 않기 위함.

🔵배치 전체 대상 정규화

• 레이어 정규화

🟢모델 파라미터 초기화

🔵왜 초기화가 중요한가?

• 딥러닝 모델의학습은 경사 하강법을 통해 이루어지며, 초기 파라미터 값은 학습의 수렴 속도와 성능에 큰 영향을 미친다.

🔵발생 가능 문제점

• 기울기 소실
• 기울기 폭발
• 대칭성 문제

Day4

🟢Resnet

🔵환경설정

from torchvision.models import ResNet50_Weights
from torchvision import models, transforms, datasets

resnet=models.resnet50(weights = ResNet50_Weights.DEFAULT)

🔵모델 요약 확인

from torchinfo import summary
summary(resnet)

=================================================================
 Layer (type:depth-idx)                   Param #
=================================================================
ResNet                                   --
├─Conv2d: 1-1                            9,408
├─BatchNorm2d: 1-2                       128
├─ReLU: 1-3                              --
├─MaxPool2d: 1-4                         --
├─Sequential: 1-5                        --
│    └─Bottleneck: 2-1                   --

🔵모델 구조 확인

degradation problem

딥러닝 모델의 레이어가 깊어졌을 때 모델이 수렴했음에도 불구하고 오히려 레이어 개수가 적을 때보다 보델의 성능이 나빠진 문제

• 병목현상 도입으로 길게 학습하더라도 효과를 볼 수 있게 함
• attention 알고지금으로 gpu 연산 기준 rnn 대비 10배 이상 효율 향상

🔵모델 클래스 선언

import torch.nn as nn
class ResNet50TransferLearning(nn.Module):
    def __init__(self, num_classes: int, mode: str = 'feature_extraction'):
        # 부모 초기화 함수 호출 
        super(ResNet50TransferLearning, self).__init__()
        self.backbone = models.resnet50(weights=ResNet50_Weights.DEFAULT)

        # 조건에 따라 전이학습과 파인튜닝 선택 
        if mode == 'feature_extraction':
            for param in self.backbone.parameters():
                param.requires_grad = False
        # find tuning시 layer4만 파라미터 업데이트 
        elif mode == 'fine_tuning':
            for name, param in self.backbone.named_parameters():
                if "layer4" in name:
                    param.requires_grad = True
                else:
                    param.requires_grad = False
        
        in_features = self.backbone.fc.in_features 
        # 우리가 분류할려고 하는 클래스로 출력 변경
        # 512 node의 layer 추가 
        self.backbone.fc = nn.Sequential(
            nn.Linear(in_features, 512),
            nn.ReLU(True),
            nn.Dropout(p=0.4),
            nn.Linear(512, num_classes)
        )

        self._initialize_weights()
        
    def _initialize_weights(self):
        for m in self.backbone.fc.modules():
            if isinstance(m, nn.Linear):
                nn.init.kaiming_normal_(m.weight, mode='fan_out', nonlinearity='relu')
                if m.bias is not None:
                    nn.init.constant_(m.bias, 0)

    def forward(self, x):
        return self.backbone(x)

🔵경로 정리

import os
import pandas as pd
import shutil

target = "/kaggle/input/datasets/nerffia/korea-food"
file_list = []
for roots, dirs, files in os.walk(target):
    for file in files:
        #print(f"{roots}/{file}")
        file_list.append(file)
df = pd.DataFrame( {'img_src' : file_list})

🔵경로 정리

df.loc[:,'label'] = df.img_src.apply(lambda x : x.split("_")[1])

df = df[~((df.label == "url.csv") | (df.label == "area.properties"))].copy()

df.label.unique()

master = { '002' : 0 ,
  '131' : 1, 
  '135' : 2, 
  '098' : 3,
  '069' : 4,
  '023' : 5} 

df.loc[:, 'y'] = df.label.apply(lambda x : master[x])

df.y.value_counts()

for roots, dirs, files in os.walk(target):
    for file in files:
        # print(f"{roots}/{file}")
        shutil.copy(f"{roots}/{file}", f'/kaggle/working/data/{file}')

🔵권한 확인

import subprocess
result = subprocess.run(['ls','-al'],capture_output =True, text = True)
print(result.stdout)

total 164
drwxr-xr-x 4 root root   4096 Feb 12 01:44 .
drwxr-xr-x 5 root root   4096 Feb 12 01:42 ..
drwxr-xr-x 2 root root 155648 Feb 12 01:45 data
drwxr-xr-x 2 root root   4096 Feb 12 01:43 .virtual_documents

🔵훈련 평가 데이터 생성

data_dir = "/kaggle/working/data"
from sklearn.model_selection import train_test_split
X_train , X_test, y_train, y_test  =train_test_split(df.drop('y', axis=1), df.y, random_state=42, 
                stratify=df.y)

y_train.value_counts(normalize=True)

y_test.value_counts(normalize=True)

🔵데이터 클래스 제작

from torch.utils.data import Dataset, DataLoader
from PIL import Image
class FoodDataset(Dataset):
    def __init__(self, X, y, target, transform=None):
        self.X = X
        self.y = y
        self.taget = target
        self.transform = transform

    def __len__(self):
        return len(self.X)

    def __getitem__(self, idx):
        file_name = self.X.iloc[idx, 0]
        img_path = os.path.join(self.target, file_name)

        image = Image.open(img_path)

        if self.transform:
            image = self.transform(image)
        return image, self.y.iloc[idx]

🔵이미지 확인

tmp = FoodDataset(X_train,y_train, '/kaggle/working/data')
a = iter(tmp)
tmp_data = next(a)[0]

tmp_data[0]

+) Laxy Evaluation - 게으른 연산

a에 인덱스 0에는 사진, 1에는 라벨 담김

🔵transform 정의

train_transforms = transforms.Compose([
    transforms.RandomResizedCrop(224),
    transforms.RandomHorizontalFlip(),
    transforms.RandomRotation(15),
    transforms.ColorJitter(brightness=0.1, contrast=0.1),
    transforms.ToTensor(),
    transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
])

val_transforms = transforms.Compose([
    transforms.Resize(256),
    transforms.CenterCrop(224),
    transforms.ToTensor(),
    transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
])

train_dataset = FoodDataset(X_train, y_train, "/kaggle/working/data", transform=train_transforms)
train_loader = DataLoader(train_dataset, batch_size=32, shuffle=True)
test_dataset = FoodDataset(X_test, y_test, "/kaggle/working/data", transform=val_transforms)
test_loader = DataLoader(test_dataset, batch_size=32, shuffle=True)

데이터 확인

next(iter(train_loader))

[tensor([[[[-2.1179, -2.1179, -2.1179,  ..., -2.1179, -2.1179, -2.1179],
           [-2.1179, -2.1179, -2.1179,  ..., -2.1179, -2.1179, -2.1179],
           [-2.1179, -2.1179, -2.1179,  ..., -2.1179, -2.1179, -2.1179],
           ...,
           [-2.1179, -2.1179, -2.1179,  ..., -2.1179, -2.1179, -2.1179],
           [-2.1179, -2.1179, -2.1179,  ..., -2.1179, -2.1179, -2.1179],
           [-2.1179, -2.1179, -2.1179,  ..., -2.1179, -2.1179, -2.1179]],

훈련 모델 코드 선언

import torch
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
def train_model(model, dataloaders, criterion, optimizer,
                num_epochs=25, device='cuda'):

    best_acc = 0.0
    history = {'train_loss': [], 'train_acc': [], 'val_loss': [], 'val_acc': []}

    for epoch in range(num_epochs):
        print(f'\nEpoch {epoch+1}/{num_epochs}')
        print('-' * 60)

        # 각 epoch마다 train -> val phase 순환
        for phase in ['train', 'val']:
            if phase == 'train':
                model.train()  # Dropout 활성화, BN 학습 모드
            else:
                model.eval()   # Dropout 비활성화, BN 추론 모드

            running_loss = 0.0
            running_corrects = 0

            # 진행률 표시
            pbar = tqdm(dataloaders[phase], desc=phase)

            for inputs, labels in pbar:
                inputs = inputs.to(device, non_blocking=True)
                labels = labels.to(device, non_blocking=True)

                # gradient 누적 초기화
                optimizer.zero_grad()

                # Forward pass
                with torch.set_grad_enabled(phase == 'train'):
                    outputs = model(inputs)
                    _, preds = torch.max(outputs, 1)  # 예측 클래스
                    loss = criterion(outputs, labels)

                    # Backward + Optimize (train phase only)
                    if phase == 'train':
                        loss.backward()

                        # Gradient Clipping (VGG16은 깊어서 안정성을 위해 권장)
                        torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)

                        optimizer.step()

                # 통계 계산
                running_loss += loss.item() * inputs.size(0)
                running_corrects += torch.sum(preds == labels.data)

                # tqdm 업데이트
                pbar.set_postfix({'loss': loss.item()})

            # Epoch 통계
            epoch_loss = running_loss / len(dataloaders[phase].dataset)
            epoch_acc = running_corrects.double() / len(dataloaders[phase].dataset)

            history[f'{phase}_loss'].append(epoch_loss)
            history[f'{phase}_acc'].append(epoch_acc.item())

            print(f'{phase} Loss: {epoch_loss:.4f} Acc: {epoch_acc:.4f}')

            # 최적 모델 저장 (validation 기준)
            if phase == 'val' and epoch_acc > best_acc:
                best_acc = epoch_acc
                best_model_wts = model.state_dict().copy()
                torch.save(best_model_wts, 'best_vgg16_transfer.pth')

        # 학습률 업데이트
        # if scheduler is not None:
        #     if isinstance(scheduler, optim.lr_scheduler.ReduceLROnPlateau):
        #         scheduler.step(epoch_loss)  # val loss 기준
        #     else:
        #         scheduler.step()

    print(f'\nBest val Acc: {best_acc:.4f}')
    model.load_state_dict(best_model_wts)
    return model, history

model = ResNet50TransferLearning(6)
model = model.to(device)
criterion = nn.CrossEntropyLoss()
MODE = 'feature_extraction'
import torch.optim as optim
if MODE == 'feature_extraction':
    # FC layer만 학습하므로 일반적인 LR 사용 가능
    optimizer = optim.Adam(
        filter(lambda p: p.requires_grad, model.parameters()),
        lr=1e-4,
        weight_decay=1e-4  # L2 정규화 (과적합 방지)
    )
else:  # fine_tuning
    # 전체 네트워크 미세 조정 시 매우 작은 LR 필요
    # VGG16은 깊은 네트워크이므로 SGD + Momentum이 더 안정적
    optimizer = optim.SGD(
        filter(lambda p: p.requires_grad, model.parameters()),
        lr=1e-2,          # ImageNet 학습률의 1/10 이하
        momentum=0.9,     # 관성항으로 안정적 수렴
        weight_decay=5e-4
    )

from tqdm import tqdm
NUM_EPOCHS = 30
dataloaders = {'train' : train_loader , 'val' : test_loader}
model, history = train_model(
        model, dataloaders, criterion, optimizer, 
        num_epochs=NUM_EPOCHS, device=device
    )

🟢Attention

• transformer 알고리즘 소개

• layer normalization → 시계열에서 주로 활용

• 딥러닝 쪽 면접 준비 때 sota 모델 논문 확인할 필요 있음

🟢강화학습

import matplotlib.animation
import matplotlib.pyplot as plt

plt.rc('font', size=14)
plt.rc('axes', labelsize=14, titlesize=14)
plt.rc('legend', fontsize=14)
plt.rc('xtick', labelsize=10)
plt.rc('ytick', labelsize=10)
plt.rc('animation', html='jshtml')

import gymnasium as gym
import torch
import torch.nn as nn
import torch.optim as optim
import torch.nn.functional as F
import numpy as np
import matplotlib.pyplot as plt
from collections import deque
import random

# GPU 설정
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

from pathlib import Path

IMAGES_PATH = Path() / "images" / "rl"
IMAGES_PATH.mkdir(parents=True, exist_ok=True)

def save_fig(fig_id, tight_layout=True, fig_extension="png", resolution=300):
    path = IMAGES_PATH / f"{fig_id}.{fig_extension}"
    if tight_layout:
        plt.tight_layout()
    plt.savefig(path, format=
fig_extension, dpi=resolution)

import gymnasium as gym

env = gym.make("CartPole-v1", render_mode="rgb_array")
envs = gym.envs.registry
envs["CartPole-v1"]
obs, info = env.reset(seed=42)

img = env.render()
plt.imshow(img)

def plot_environment(env, figsize=(5, 4)):
    plt.figure(figsize=figsize)
    img = env.render()
    plt.imshow(img)
    plt.axis("off")
    return img

plot_environment(env)
plt.show()

right_action = 1
env.step(right_action)

def basic_policy(obs):
    angle = obs[2]
    return 0 if angle < 0 else 1

totals = []
for episode in  range(500):
    episode_rewards = 0
    obs, info = env.reset(seed=episode)
    for step in range(200):
        obs, reward, done, truncated, info = env.step(basic_policy(obs))
        episode_rewards += reward
        if  done or truncated:
            break
    totals.append(episode_rewards)

len(totals)

500

import numpy as np
print(np.mean(totals),np.std(totals),np.min(totals),np.max(totals))

41.698 8.389445512070509 24.0 63.0

def update_scene(num, frames, patch):
    patch.set_data(frames[num])
    return patch,

def plot_animation(frames, repeat=False, interval=40):
    fig = plt.figure()
    patch = plt.imshow(frames[0])
    plt.axis('off')
    anim = matplotlib.animation.FuncAnimation(
        fig, update_scene, fargs=(frames, patch),
        frames=len(frames), repeat=repeat, interval=interval)
    plt.close()
    return anim

def show_one_episode(policy, n_max_steps=200, seed=42):
    frames = []
    env = gym.make("CartPole-v1", render_mode="rgb_array")
    np.random.seed(seed)
    obs, info = env.reset(seed=seed)
    for step in range(n_max_steps):
        frames.append(env.render())
        action = policy(obs)
        obs, reward, done, truncated, info = env.step(action)
        if done or truncated:
            break
    env.close()
    return plot_animation(frames)

show_one_episode(basic_policy)

• 동적 시각화 출력됨

딥러닝 설계

• 기존 정책을 따르는 거 : exploiting
• 새로운 행동을 하는 거 : exploring
  • 최적의 값을 찾기 위함.
  • 다항분포 샘플링에 적용됨.

import torch.nn as nn 
class BasicPolicyNet(nn.Module):
    def __init__(self, obs_size, n_hidden, n_outputs):
        super(BasicPolicyNet, self).__init__()
        self.fc1 = nn.Linear(obs_size, n_hidden)
        self.fc2 = nn.Linear(n_hidden, n_outputs)
    def forward(self, x):
        x = F.relu(self.fc1(x))
        x = F.softmax(self.fc2(x), dim=-1)
        return x

환경 초기화

env = gym.make('CartPole-v1')
obs, info = env.reset()
model = BasicPolicyNet(env.observation_space.shape[0], 5, 2).to(device) #2 ->왼쪽과 오른쪽

확률 예측

obs_tensor=torch.tensor(obs).unsqueeze(0).to(device)
probs= model(obs_tensor)
probs

tensor([[0.3420, 0.6580]], grad_fn=)

action = torch.multinomial(probs, num_samples=1).item()

def play_one_step(env, obs, model, loss_fn):
    obs_tensor = torch.from_numpy(obs).float().unsqueeze(0).to(device)
    probs = model(obs_tensor)
    m = torch.distributions.Categorical(probs)
    action = m.sample()
    log_prob = m.log_prob(action)
    next_obs, reward, done, truncated, info = env.step(action.item())
    return next_obs, reward, done, truncated, log_prob

def play_multiple_episodes(env, n_episodes, n_max_steps, model, loss_fn):
    all_rewards = []
    all_log_probs = []
    for episode in range(n_episodes):
        current_rewards = []
        current_log_probs = []
        obs, info = env.reset()
        for step in range(n_max_steps):
            obs, reward, done, truncated, log_prob = play_one_step(env, obs, model, loss_fn)
            current_rewards.append(reward)
            current_log_probs.append(log_prob)
            if done or truncated:
                break
        all_rewards.append(current_rewards)
        all_log_probs.append(current_log_probs)
    return all_rewards, all_log_probs

def discount_rewards(rewards, discount_rate):
    discounted = np.array(rewards)
    for step in range(len(rewards) - 2, -1, -1):
        discounted[step] += discounted[step + 1] * discount_rate
    return discounted

def discount_and_normalize_rewards(all_rewards, discount_rate):
    all_discounted_rewards = [discount_rewards(rewards, discount_rate) for rewards in all_rewards]
    flat_rewards = np.concatenate(all_discounted_rewards)
    reward_mean = flat_rewards.mean()
    reward_std = flat_rewards.std()
    return [(discounted_rewards - reward_mean) / reward_std for discounted_rewards in all_discounted_rewards]

n_iterations = 150
n_episodes_per_update = 10
n_max_steps = 200
discount_rate = 0.95
learning_rate = 0.01

model = BasicPolicyNet(4, 5, 2).to(device)
optimizer = optim.Adam(model.parameters(), lr=learning_rate)

for iteration in range(n_iterations):
    all_rewards, all_log_probs = play_multiple_episodes(env, n_episodes_per_update, n_max_steps, model, None)
    
    all_final_rewards = discount_and_normalize_rewards(all_rewards, discount_rate)
    
    optimizer.zero_grad()
    policy_loss = []
    for log_probs, final_rewards in zip(all_log_probs, all_final_rewards):
        for log_prob, reward in zip(log_probs, final_rewards):
            # Policy Gradient Loss: -log(prob) * return
            policy_loss.append(-log_prob * reward)
            
    loss = torch.stack(policy_loss).sum()
    loss.backward()
    optimizer.step()
    
    if iteration % 10 == 0:
        avg_reward = sum(map(sum, all_rewards)) / n_episodes_per_update
        print(f'Iteration: {iteration}, Avg Reward: {avg_reward:.2f}')

Iteration: 0, Avg Reward: 19.80
Iteration: 10, Avg Reward: 27.20
Iteration: 20, Avg Reward: 31.20
Iteration: 30, Avg Reward: 50.50
Iteration: 40, Avg Reward: 43.40
Iteration: 50, Avg Reward: 61.00
Iteration: 60, Avg Reward: 54.70
Iteration: 70, Avg Reward: 68.10
Iteration: 80, Avg Reward: 98.20
Iteration: 90, Avg Reward: 175.80
Iteration: 100, Avg Reward: 170.00
Iteration: 110, Avg Reward: 195.70
Iteration: 120, Avg Reward: 173.30
Iteration: 130, Avg Reward: 189.40
Iteration: 140, Avg Reward: 179.00

from matplotlib import animation
import matplotlib.pyplot as plt
from IPython.display import HTML

def update_scene(num, frames, patch):
    patch.set_data(frames[num])
    return patch,

def plot_animation(frames, repeat=False, interval=40):
    fig = plt.figure()
    patch = plt.imshow(frames[0])
    plt.axis('off')
    anim = animation.FuncAnimation(
        fig, update_scene, fargs=(frames, patch),
        frames=len(frames), repeat=repeat, interval=interval
    )
    plt.close()
    return anim

def render_policy_net(model, n_max_steps=200, seed=42):
    frames = []
    # 렌더링을 위해 render_mode='rgb_array' 설정 필수
    env = gym.make('CartPole-v1', render_mode='rgb_array')
    obs, info = env.reset(seed=seed)
    
    for step in range(n_max_steps):
        frames.append(env.render())
        
        # PyTorch 모델로 행동 결정
        obs_tensor = torch.tensor(obs, dtype=torch.float32).unsqueeze(0).to(device)
        with torch.no_grad():
            # DQN인 경우와 PolicyNet인 경우 분기 처리
            q_values = model(obs_tensor)
            if q_values.shape[-1] > 1: # Softmax 출력인 경우 (Policy Net)
                 action = torch.argmax(q_values).item()
            else: # Q-Value 출력인 경우 (DQN)
                 action = torch.argmax(q_values).item()
        
        obs, reward, done, truncated, info = env.step(action)
        if done or truncated:
            break
    env.close()
    return frames

frames = render_policy_net(model)
anim = plot_animation(frames)
HTML(anim.to_jshtml())

Day5