컨볼루션 인공신경망 Convolutional Neural Network¶
- MNIST µ¥ÀÌÅÍ
- ÇÕ¼º°ö ¿¬»ê(CNN)
- ¸Æ½ºÇ®¸µ(Max Pooling)
- ¿ÏÀü¿¬°á ½Å°æ¸Á(Fully Connected Network)
In [1]:
# ÆÄÀÌÅäÄ¡ ¹× ÅäÄ¡ºñÁ¯ ¼³Ä¡
# ·±Å¸ÀÓÀ» GPU ¸ðµå·Î ¹Ù²ã¼ ½ÇÇàÇϼ¼¿ä
!pip install torch torchvision
In [2]:
import torch
import torch.nn as nn
import torch.optim as optim
import torch.nn.init as init
# https://pytorch.org/docs/stable/torchvision/datasets.html
# ÆÄÀÌÅäÄ¡¿¡¼´Â torchvision.datasets¿¡ MNIST µîÀÇ ´Ù¾çÇÑ µ¥ÀÌÅ͸¦ »ç¿ëÇϱ⠿ëÀÌÇÏ°Ô Á¤¸®Çسù½À´Ï´Ù.
# À̸¦ »ç¿ëÇÏ¸é µ¥ÀÌÅ͸¦ µû·Î ÇнÀ¿¡ ¸Â°Ô Á¤¸®Çϰųª ÇÏÁö ¾Ê¾Æµµ ¹Ù·Î »ç¿ëÀÌ °¡´ÉÇÕ´Ï´Ù.
import torchvision.datasets as dset
# https://pytorch.org/docs/stable/torchvision/transforms.html?highlight=transforms
# torchvision.transforms¿¡´Â À̹ÌÁö µ¥ÀÌÅ͸¦ ÀÚ¸£°Å³ª È®´ë ¹× ´Ù¾çÇÏ°Ô º¯Çü½ÃÅ°´Â ÇÔ¼öµéÀÌ ±¸ÇöµÇ¾î ÀÖ½À´Ï´Ù.
import torchvision.transforms as transforms
# https://pytorch.org/docs/stable/data.html?highlight=dataloader#torch.utils.data.DataLoader
# DataLoader´Â Àü󸮰¡ ³¡³ µ¥ÀÌÅ͵éÀ» ÁöÁ¤ÇÑ ¹èÄ¡ Å©±â¿¡ ¸Â°Ô ¸ð¾Æ¼ Àü´ÞÇØÁÖ´Â ¿ªÇÒÀ» ÇÕ´Ï´Ù.
from torch.utils.data import DataLoader
import numpy as np
import matplotlib.pyplot as plt
print(torch.__version__)
2) 하이퍼파라미터 지정¶
In [0]:
batch_size = 256
learning_rate = 0.0002
num_epoch = 10
In [0]:
# https://pytorch.org/docs/stable/torchvision/datasets.html?highlight=mnist#torchvision.datasets.MNIST
# ù¹ø° ÀÎÀÚ root´Â µ¥ÀÌÅ͸¦ ÀúÀåÇÒ À§Ä¡, trainÀº ÇнÀ¿ë µ¥ÀÌÅÍÀÎÁö Å×½ºÆ®¿ë µ¥ÀÌÅÍÀÎÁöÀÇ ¿©ºÎ¸¦ ÀǹÌÇÕ´Ï´Ù.
# MNIST µ¥ÀÌÅÍ´Â ¼ýÀÚ ¼Õ±Û¾¾ À̹ÌÁö¿Í ÀÌ¿¡ ´ëÇÑ Á¤´ä ½ÖÀ¸·Î ÀÌ·ç¾îÁ® ÀÖ½À´Ï´Ù.
# transformÀº À̹ÌÁö¿¡ ´ëÇÑ º¯Çü, target_transformÀº Á¤´ä ¶óº§¿¡ ´ëÇÑ º¯ÇüÀ» ÀǹÌÇÕ´Ï´Ù.
# transform.ToTensor()´Â PIL À̹ÌÁö³ª Numpy ¹è¿À» ÅäÄ¡ ÅÙ¼·Î ¹Ù²ãÁÝ´Ï´Ù.
# download´Â µ¥ÀÌÅÍ°¡ ÀúÀåÇÒ À§Ä¡¿¡ ¾øÀ» °æ¿ì »õ·Î ´Ù¿î¹ÞÀ»Áö ¿©ºÎÀÔ´Ï´Ù.
mnist_train = dset.MNIST(root="../", train=True, transform=transforms.ToTensor(), target_transform=None, download=True)
mnist_test = dset.MNIST(root="../", train=False, transform=transforms.ToTensor(), target_transform=None, download=True)
2) 데이터셋 체크¶
- getitemÀ» »ç¿ëÇصµ µÇ°í
- À妽ÌÀ¸·Îµµ µ¥ÀÌÅ͸¦ È®ÀÎÇÒ ¼ö ÀÖ½À´Ï´Ù.
In [6]:
print(mnist_train.__getitem__(0)[0].size(), mnist_train.__len__())
print(mnist_test.__getitem__(0)[0].size(), mnist_test.__len__())
print(len(mnist_train),len(mnist_test))
#print(mnist_train[0])
3) DataLoader 설정¶
- »ç¿ëÇÒ µ¥ÀÌÅÍ
- ¹èÄ¡ »çÀÌÁî (batch_size)
- ¼¯À»Áö ¿©ºÎ (shuffle)
- »ç¿ëÇÒ ÇÁ·Î¼¼½º °³¼ö (num_workers)
- ¸¶Áö¸·¿¡ ³²´Â µ¥ÀÌÅÍÀÇ Ã³¸® ¿©ºÎ (drop_last)
In [0]:
# https://pytorch.org/docs/stable/data.html?highlight=dataloader#torch.utils.data.DataLoader
train_loader = DataLoader(mnist_train,batch_size=batch_size, shuffle=True,num_workers=2,drop_last=True)
test_loader = DataLoader(mnist_test,batch_size=batch_size, shuffle=False,num_workers=2,drop_last=True)
In [0]:
# https://pytorch.org/docs/stable/nn.html?highlight=conv2d#torch.nn.Conv2d
# https://pytorch.org/docs/stable/tensors.html?highlight=view#torch.Tensor.view
class CNN(nn.Module):
def __init__(self):
super(CNN,self).__init__()
self.layer = nn.Sequential(
nn.Conv2d(in_channels=1,out_channels=16,kernel_size=5), # [batch_size,1,28,28] -> [batch_size,16,24,24]
nn.ReLU(), # ÇÊÅÍÀÇ °³¼ö´Â 1°³(Èæ¹éÀ̹ÌÁö)¿¡¼ 16°³·Î ´Ã¾î³ªµµ·Ï ÀÓÀÇ·Î ¼³Á¤Çß½À´Ï´Ù.
nn.Conv2d(in_channels=16,out_channels=32,kernel_size=5), # [batch_size,16,24,24] -> [batch_size,32,20,20]
nn.ReLU(),
nn.MaxPool2d(kernel_size=2,stride=2), # [batch_size,32,20,20] -> [batch_size,32,10,10]
nn.Conv2d(in_channels=32, out_channels=64, kernel_size=5), # [batch_size,32,10,10] -> [batch_size,64,6,6]
nn.ReLU(),
nn.MaxPool2d(kernel_size=2,stride=2) # [batch_size,64,6,6] -> [batch_size,64,3,3]
)
self.fc_layer = nn.Sequential(
nn.Linear(64*3*3,100), # [batch_size,64*3*3] -> [batch_size,100]
nn.ReLU(),
nn.Linear(100,10) # [batch_size,100] -> [batch_size,10]
)
def forward(self,x):
out = self.layer(x) # self.layer¿¡ Á¤ÀÇÇÑ SequentialÀÇ ¿¬»êÀ» Â÷·Ê´ë·Î ´Ù ½ÇÇàÇÕ´Ï´Ù.
out = out.view(batch_size,-1) # view ÇÔ¼ö¸¦ ÀÌ¿ëÇØ ÅÙ¼ÀÇ ÇüŸ¦ [batch_size,³ª¸ÓÁö]·Î ¹Ù²ãÁÝ´Ï´Ù.
# ex) 2x3 ÇüÅ¿´´ø ÅÙ¼¸¦ .view(1,-1) ÇØÁÖ¸é 1x6ÀÇ ÇüÅ·Π¹Ù²ò´Ï´Ù. .view(3,-1)À̸é 3x2·Î ¹Ù²ñ.
# ¸¸¾à Àüü ÅÙ¼ÀÇ Å©±â°¡ batch_size·Î ³ª´©¾î ¶³¾îÁöÁö ¾ÊÀ¸¸é ¿À·ù°¡ ³³´Ï´Ù.
out = self.fc_layer(out)
return out
2) 손실함수 & 최적화함수¶
In [9]:
# gpu°¡ »ç¿ë °¡´ÉÇÑ °æ¿ì¿¡´Â device¸¦ gpu·Î ¼³Á¤ÇÏ°í ºÒ°¡´ÉÇϸé cpu·Î ¼³Á¤ÇÕ´Ï´Ù.
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
print(device)
# ¸ðµ¨À» ÁöÁ¤ÇÑ ÀåÄ¡·Î ¿Ã¸³´Ï´Ù.
model = CNN().to(device)
# ¼Õ½ÇÇÔ¼ö·Î´Â Å©·Î½º¿£Æ®·ÎÇǸ¦ »ç¿ëÇÕ´Ï´Ù.
loss_func = nn.CrossEntropyLoss()
# ÃÖÀûÈÇÔ¼ö·Î´Â AdamÀ» »ç¿ëÇÕ´Ï´Ù.
optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)
4. 학습¶
In [10]:
loss_arr =[]
for i in range(num_epoch):
for j,[image,label] in enumerate(train_loader):
x = image.to(device)
y_= label.to(device)
optimizer.zero_grad()
output = model.forward(x)
loss = loss_func(output,y_)
loss.backward()
optimizer.step()
if j % 1000 == 0:
print(loss)
loss_arr.append(loss.cpu().detach().numpy())
In [0]:
#param_list = list(model.parameters())
#print(param_list)
5. 학습시 손실 시각화¶
In [11]:
plt.plot(loss_arr)
plt.show()
6. 테스트 데이터 정확도 측정¶
In [13]:
# ¸ÂÀº °³¼ö, Àüü °³¼ö¸¦ ÀúÀåÇÒ º¯¼ö¸¦ ÁöÁ¤ÇÕ´Ï´Ù.
correct = 0
total = 0
# ÀÎÆÛ·±½º ¸ðµå¸¦ À§ÇØ no_grad ÇØÁÝ´Ï´Ù.
with torch.no_grad():
# Å×½ºÆ®·Î´õ¿¡¼ À̹ÌÁö¿Í Á¤´äÀ» ºÒ·¯¿É´Ï´Ù.
for image,label in test_loader:
# µÎ µ¥ÀÌÅÍ ¸ðµÎ ÀåÄ¡¿¡ ¿Ã¸³´Ï´Ù.
x = image.to(device)
y_= label.to(device)
# ¸ðµ¨¿¡ µ¥ÀÌÅ͸¦ ³Ö°í °á°ú°ªÀ» ¾ò½À´Ï´Ù.
output = model.forward(x)
# https://pytorch.org/docs/stable/torch.html?highlight=max#torch.max
# torch.max¸¦ ÀÌ¿ëÇØ ÃÖ´ë °ª ¹× ÃÖ´ë°ª À妽º¸¦ »Ì¾Æ³À´Ï´Ù.
# ¿©±â¼´Â ÃÖ´ë°ªÀº ÇÊ¿ä¾ø±â ¶§¹®¿¡ À妽º¸¸ »ç¿ëÇÕ´Ï´Ù.
_,output_index = torch.max(output,1)
# Àüü °³¼ö´Â ¶óº§ÀÇ °³¼ö·Î ´õÇØÁÝ´Ï´Ù.
# Àüü °³¼ö¸¦ ¾Ë°í ÀÖÀ½¿¡µµ ÀÌ·¸°Ô ÇÏ´Â ÀÌÀ¯´Â batch_size, drop_lastÀÇ ¿µÇâÀ¸·Î ¸î¸î µ¥ÀÌÅÍ°¡ À߸±¼öµµ Àֱ⠶§¹®ÀÔ´Ï´Ù.
total += label.size(0)
# ¸ðµ¨ÀÇ °á°úÀÇ ÃÖ´ë°ª À妽º¿Í ¶óº§ÀÌ ÀÏÄ¡ÇÏ´Â °³¼ö¸¦ correct¿¡ ´õÇØÁÝ´Ï´Ù.
correct += (output_index == y_).sum().float()
# Å×½ºÆ® µ¥ÀÌÅÍ Àüü¿¡ ´ëÇØ À§ÀÇ ÀÛ¾÷À» ½ÃÇàÇÑ ÈÄ Á¤È®µµ¸¦ ±¸ÇØÁÝ´Ï´Ù.
print("Accuracy of Test Data: {}%".format(100*correct/total))
In [0]: