[PyTorch] Lab04.1 - Multivariable Linear regression

📌 학습 목표

---

• Multivariate Linear Regression
• nn.Module
• nn.functional

Multivariate Linear Regression

기존 Simple Linear Regression은 x데이터 1개에 y데이터 1개이다.

Multivariate Linear Regression은 주어지는 x데이터가 여러개이다.

이번에는 3개의 점수(x1, x2, x3)가 주어지고 이 점수로 마지막 점수(y)를 예측한다고 한다.

# Data
x_train = torch.FloatTensor([[73, 80, 75],
                            [93, 88, 93],
                            [89, 91, 90],
                            [96, 98, 100],
                            [73, 66, 70]])
y_train = torch.FloatTensor([[152], [185], [180], [196], [142]])

Hypothesis는 어떻게 표현할까?

기존에는 $H(x) = Wx + b$ 였다면,

$H(x) = w_1x_1 + w_2x_2 + w_3x_3 + b$ 로 바뀐다.

각 데이터($x_1, x_2, x_3$)에 대한 weight($w_1,w_2,w_3$)도 생긴다.

hypothesis는 matmul을 이용해 간단하게 행렬의 곱으로 나타낼 수 있다.

# Hypothesis
hypothesis = x_train.matmul(W) + b

전체 코드로 살펴보면 다음과 같다.

점점 cost는 감소하고, hypothesis의 값들은 y값에 가까워진다.

import torch

x_train = torch.FloatTensor([[73, 80, 75],
                            [93, 88, 93],
                            [89, 91, 90],
                            [96, 98, 100],
                            [73, 66, 70]])
y_train = torch.FloatTensor([[152], [185], [180], [196], [142]])

W = torch.zeros((3, 1), requires_grad=True)
b = torch.zeros((1), requires_grad=True)

optimizer = torch.optim.SGD([W, b], lr=1e-5)

epochs = 20

for epoch in range(1, epochs+1, 1):
    # H(x)
    hypothesis = x_train.matmul(W) + b
    
    # cost
    cost = torch.mean((hypothesis - y_train) ** 2)
    
    # cost to H(x)
    optimizer.zero_grad()
    cost.backward()
    optimizer.step()
    
    print('Epoch {:4d}/{} hypothesis: {}, Cost: {:.6f}'.format(
        epoch, epochs, hypothesis.squeeze().detach(), cost.item()
    ))

'''
Epoch    1/20 hypothesis: tensor([0., 0., 0., 0., 0.]), Cost: 29661.800781
Epoch    2/20 hypothesis: tensor([67.2578, 80.8397, 79.6523, 86.7394, 61.6605]), Cost: 9298.520508
Epoch    3/20 hypothesis: tensor([104.9128, 126.0990, 124.2466, 135.3015,  96.1821]), Cost: 2915.712646
Epoch    4/20 hypothesis: tensor([125.9942, 151.4381, 149.2133, 162.4896, 115.5097]), Cost: 915.040527
Epoch    5/20 hypothesis: tensor([137.7968, 165.6247, 163.1911, 177.7112, 126.3307]), Cost: 287.936005
Epoch    6/20 hypothesis: tensor([144.4044, 173.5674, 171.0168, 186.2332, 132.3891]), Cost: 91.371017
Epoch    7/20 hypothesis: tensor([148.1035, 178.0144, 175.3980, 191.0042, 135.7812]), Cost: 29.758139
Epoch    8/20 hypothesis: tensor([150.1744, 180.5042, 177.8508, 193.6753, 137.6805]), Cost: 10.445305
Epoch    9/20 hypothesis: tensor([151.3336, 181.8983, 179.2240, 195.1707, 138.7440]), Cost: 4.391228
Epoch   10/20 hypothesis: tensor([151.9824, 182.6789, 179.9928, 196.0079, 139.3396]), Cost: 2.493135
Epoch   11/20 hypothesis: tensor([152.3454, 183.1161, 180.4231, 196.4765, 139.6732]), Cost: 1.897688
Epoch   12/20 hypothesis: tensor([152.5485, 183.3610, 180.6640, 196.7389, 139.8602]), Cost: 1.710541
Epoch   13/20 hypothesis: tensor([152.6620, 183.4982, 180.7988, 196.8857, 139.9651]), Cost: 1.651413
Epoch   14/20 hypothesis: tensor([152.7253, 183.5752, 180.8742, 196.9678, 140.0240]), Cost: 1.632387
Epoch   15/20 hypothesis: tensor([152.7606, 183.6184, 180.9164, 197.0138, 140.0571]), Cost: 1.625923
Epoch   16/20 hypothesis: tensor([152.7802, 183.6427, 180.9399, 197.0395, 140.0759]), Cost: 1.623412
Epoch   17/20 hypothesis: tensor([152.7909, 183.6565, 180.9530, 197.0538, 140.0865]), Cost: 1.622141
Epoch   18/20 hypothesis: tensor([152.7968, 183.6643, 180.9603, 197.0618, 140.0927]), Cost: 1.621253
Epoch   19/20 hypothesis: tensor([152.7999, 183.6688, 180.9644, 197.0662, 140.0963]), Cost: 1.620500
Epoch   20/20 hypothesis: tensor([152.8014, 183.6715, 180.9666, 197.0686, 140.0985]), Cost: 1.619770
'''

nn.Module

• nn.Module을 상속해 모델 생성을 할 수 있다.
• nn.Linear(3, 1)
  • input dim = 3
  • ouput dim = 1
• Hypothesis를 계산하는 것은 forward()를 이용한다.
• Gradient 계산을 PyTorch가 해준다.

import torch.nn as nn

class MultivariateLinearRegressionModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.linear = nn.Linear(3, 1)
        
    def forward(self, x):
        return self.linear(x)
    
model = MultivariateLinearRegressionModel()
hypothesis = model(x_train)

nn.functional

• nn.functional을 통해 cost function을 이용할 수 있다.

import torch.nn.functional as F

cost = F.mse_loss(prediction, y_train)

nn.module 과 nn.functional 을 이용해 다시 코드로 구현하면 다음과 같다.

import torch
import torch.nn as nn
import torch.nn.functional as F

class MultivariateLinearRegressionModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.linear = nn.Linear(3, 1)
        
    def forward(self, x):
        return self.linear(x)

x_train = torch.FloatTensor([[73, 80, 75],
                            [93, 88, 93],
                            [89, 91, 90],
                            [96, 98, 100],
                            [73, 66, 70]])
y_train = torch.FloatTensor([[152], [185], [180], [196], [142]])

model = MultivariateLinearRegressionModel()
optimizer = torch.optim.SGD(model.parameters(), lr=1e-5)

epochs = 20

for epoch in range(1, epochs+1, 1):
    # H(x)
    Hypothesis = model(x_train)
    
    # cost
    cost = F.mse_loss(Hypothesis, y_train)
    
    # cost to H(x)
    optimizer.zero_grad()
    cost.backward()
    optimizer.step()
    
    print('Epoch {:4d}/{} hypothesis: {}, Cost: {:.6f}'.format(
        epoch, epochs, Hypothesis.squeeze().detach(), cost.item()
    ))
'''
Epoch    1/20 hypothesis: tensor([-47.7381, -61.9893, -58.7048, -64.1374, -48.2127]), Cost: 52346.269531
Epoch    2/20 hypothesis: tensor([41.6050, 45.3966, 47.1032, 51.0850, 33.6961]), Cost: 16413.558594
Epoch    3/20 hypothesis: tensor([ 91.6245, 105.5183, 106.3411, 115.5937,  79.5543]), Cost: 5150.562988
Epoch    4/20 hypothesis: tensor([119.6280, 139.1786, 139.5061, 151.7096, 105.2290]), Cost: 1620.208984
Epoch    5/20 hypothesis: tensor([135.3057, 158.0241, 158.0738, 171.9295, 119.6038]), Cost: 513.627136
Epoch    6/20 hypothesis: tensor([144.0826, 168.5753, 168.4690, 183.2497, 127.6521]), Cost: 166.769821
Epoch    7/20 hypothesis: tensor([148.9960, 174.4829, 174.2888, 189.5874, 132.1586]), Cost: 58.045410
Epoch    8/20 hypothesis: tensor([151.7463, 177.7907, 177.5469, 193.1355, 134.6820]), Cost: 23.962969
Epoch    9/20 hypothesis: tensor([153.2856, 179.6429, 179.3708, 195.1219, 136.0952]), Cost: 13.276967
Epoch   10/20 hypothesis: tensor([154.1470, 180.6802, 180.3919, 196.2338, 136.8869]), Cost: 9.924462
Epoch   11/20 hypothesis: tensor([154.6287, 181.2613, 180.9634, 196.8563, 137.3305]), Cost: 8.870623
Epoch   12/20 hypothesis: tensor([154.8979, 181.5870, 181.2832, 197.2046, 137.5794]), Cost: 8.537291
Epoch   13/20 hypothesis: tensor([155.0482, 181.7696, 181.4621, 197.3995, 137.7191]), Cost: 8.429845
Epoch   14/20 hypothesis: tensor([155.1318, 181.8722, 181.5621, 197.5086, 137.7978]), Cost: 8.393110
Epoch   15/20 hypothesis: tensor([155.1782, 181.9300, 181.6179, 197.5695, 137.8423]), Cost: 8.378638
Epoch   16/20 hypothesis: tensor([155.2036, 181.9627, 181.6490, 197.6035, 137.8676]), Cost: 8.371113
Epoch   17/20 hypothesis: tensor([155.2174, 181.9813, 181.6663, 197.6224, 137.8823]), Cost: 8.365725
Epoch   18/20 hypothesis: tensor([155.2246, 181.9921, 181.6759, 197.6329, 137.8909]), Cost: 8.361066
Epoch   19/20 hypothesis: tensor([155.2282, 181.9984, 181.6810, 197.6386, 137.8962]), Cost: 8.356608
Epoch   20/20 hypothesis: tensor([155.2297, 182.0023, 181.6838, 197.6417, 137.8996]), Cost: 8.352201
'''