Diary of Chanjun 데이터 분석가의 다이어리
CNN에 관하여
학습목적
이미지 데이터를 처리하고 학습하는 딥러닝 모델 중 CNN을 공부하고 오픈 이미지 데이터를 가지고 한 스텝 한 스텝 따라가본다.
CNN이란?
Convolutional Neural Network의 약자로 주로 이미지나 영상 데이터를 처리하기 위한 딥러닝 알고리즘 중 하나이다.
Flatten한 데이터를 처리하는 Fully Connected Neural Network와 다르게 이미지의 공간 정보 데이터를 유지하여 이미지를 인식할 수 있도록 만든다.
이를 위해서 Filter, Pooling, Padding, Stride 등의 개념들이 사용되며, 천천히 알아가보도록 하겠습니다.
출처/참고자료 : http://taewan.kim/post/cnn/
간단한 이미지를 참고해서 가져오겠습니다. 출처는 http://taewan.kim/post/cnn/ 입니다.
사람은 간단하게 2라고 보이는 것을 알 수 있지만, 컴퓨터 혹은 딥러닝으로 학습시키기에는 이렇게만 정보가 있어서는 안됩니다.
그래서 위와 같이 숫자로 이루어진 바둑판 모양으로 변환을 하게됩니다.
그리고 각 위치별 특성을 보기 위하여 filter라는 어떠한 판을 만들어 그것을 곱하여 특성으로 사용되게 됩니다.
참고 그림이 잠깐 달라지지만, 결국 아래와 같은 Feature map을 만들 수 있습니다.
여기서는 [[1, 0, 1], [0, 1, 0], [1, 0, 1]] 이렇게 생긴 filter를 사용했네요.
Filter에 대한 값은 Keras layer initializers에서 확인하실 수 있습니다.
다양하게 초기화시킬 수 있지만, 이 파라미터를 크게 바꾸는 경우나 바꿔서 크게 성능이 좋아지는 경우는 아직 확인하지 못했습니다.
이러한 Convolution과 filter의 개념을 사용하여 아래와 같은 그림으로 CNN이 이루어지게 됩니다.
출처/참고자료 : http://taewan.kim/post/cnn/
출처/참고자료 : https://youngq.tistory.com/40
그럼 이제 위에서 나오는 channel, Pooling, padding, Stride 등의 용어에 대해서 정리해보겠습니다.
- Channel
- 채널은 이미지의 차원?이라고 생각하시면 될 것 같습니다. 흑백(Grey scale) 이미지에서는 1개의 채널을 갖지만 보통 우리가 많이 보는 색상이 있는 이미지라면 RGB라는 총 3개의 채널을 갖게 됩니다.
(투명도(alpha) 채널이 추가되는 경우도 있다고 합니다.) - 처음 이미지 채널을 시작으로 Convolution Layer의 채널 수를 늘려가면서 Hidden 층을 깊게 만들어 계산하게 됩니다.
- Pooling 과정을 거치며 이미지 사이즈는 줄어들지만, Channel 수가 늘어나면서 연산량이 늘어나게 됩니다.
- 채널은 이미지의 차원?이라고 생각하시면 될 것 같습니다. 흑백(Grey scale) 이미지에서는 1개의 채널을 갖지만 보통 우리가 많이 보는 색상이 있는 이미지라면 RGB라는 총 3개의 채널을 갖게 됩니다.
- Filter
- Filter는 위에서 말씀드렸듯이 어떠한 Channel에 곱해지는 하나의 랜덤한 판입니다. 연산을 하는 과정에서 Back propagation(역전파)를 통하여 이 filter의 값을 업데이트하면서 최적해를 찾아가게 됩니다.
- Pooling
- Pooling은 계속 연산량만 늘어나는 것을 방지하기 위해서 feature map 상에서 원하는 정보를 추출할 수 있도록 해주는 장치입니다.
- 보통 feature map 상에서 가장 두들어진 부분을 뽑아내기 위하여 max pooling이라는 기법을 활용하고, 상황에 따라 전체적인 부분을 보기 위해서 average pooling과 같은 기법도 사용한다고 합니다.
보통 max pooling을 사용합니다.
이 외에도 loss function, Optimizer, Activate function 등 아직 배워야할 것이 많기 때문에 일단 기본적인 분류 예제를 만든 후 개념에 대해서는 각각 캡터를 만들어보도록 하겠습니다.
Library Import
import os
import sys
import warnings
from glob import glob
from tqdm import tqdm
import numpy as np
import pandas as pd
import tensorflow as tf
import matplotlib as mpl
import matplotlib.pyplot as plt
import cv2
%matplotlib inline
warnings.filterwarnings("ignore")
저는 python 3.8 버젼과 tensorflow 2.5 버젼을 사용하겠습니다.
!python --version
Python 3.8.10
tf.__version__
'2.5.0'
1. 데이터 탐색 및 구축과 이미지 데이터 다루기
오픈 데이터셋을 제공해주는 AI HUB에서 한국 음식 데이터를 받아서 실습을 해보겠습니다.
많은 종류가 있지만 구이 카테고리를 분류해보도록 하겠습니다.
file_list = pd.DataFrame(glob("data/kfood/구이/*/*"), columns = ["dir_name"])
file_list
| dir_name | |
|---|---|
| 0 | data/kfood/구이\갈비구이\crop_area.properties |
| 1 | data/kfood/구이\갈비구이\Img_000_0000.jpg |
| 2 | data/kfood/구이\갈비구이\Img_000_0001.jpg |
| 3 | data/kfood/구이\갈비구이\Img_000_0002.jpg |
| 4 | data/kfood/구이\갈비구이\Img_000_0003.jpg |
| ... | ... |
| 14024 | data/kfood/구이\훈제오리\Img_013_0997.jpg |
| 14025 | data/kfood/구이\훈제오리\Img_013_0998.jpg |
| 14026 | data/kfood/구이\훈제오리\Img_013_0999.jpg |
| 14027 | data/kfood/구이\훈제오리\Img_013_1000.jpg |
| 14028 | data/kfood/구이\훈제오리\org_url.csv |
14029 rows × 1 columns
- 정규식을 이용하여 category, 파일명, 확장자들을 뽑아내고, 그 중에서 jpg 파일만 뽑아내도록 하겠습니다.
file_list.dir_name = file_list.dir_name.str.replace(pat = "\\", repl = "/")
file_list["category"] = file_list.dir_name.str.replace(pat = ".*/.*/.*/(.*)/(.*)(\.*)", repl = "\\1", regex = True)
file_list["file_name"] = file_list.dir_name.str.replace(pat = ".*/.*/.*/(.*)/(.*)(\.*)", repl = "\\2", regex = True)
file_list["ext_name"] = file_list.dir_name.str.replace(pat = ".*/.*/.*/(.*)/(.*)\.(.*)", repl = "\\3", regex = True)
file_list = file_list[file_list.ext_name.isin(["JPG", "jpg"])]
file_list
| dir_name | category | file_name | ext_name | |
|---|---|---|---|---|
| 1 | data/kfood/구이/갈비구이/Img_000_0000.jpg | 갈비구이 | Img_000_0000.jpg | jpg |
| 2 | data/kfood/구이/갈비구이/Img_000_0001.jpg | 갈비구이 | Img_000_0001.jpg | jpg |
| 3 | data/kfood/구이/갈비구이/Img_000_0002.jpg | 갈비구이 | Img_000_0002.jpg | jpg |
| 4 | data/kfood/구이/갈비구이/Img_000_0003.jpg | 갈비구이 | Img_000_0003.jpg | jpg |
| 5 | data/kfood/구이/갈비구이/Img_000_0004.jpg | 갈비구이 | Img_000_0004.jpg | jpg |
| ... | ... | ... | ... | ... |
| 14023 | data/kfood/구이/훈제오리/Img_013_0996.jpg | 훈제오리 | Img_013_0996.jpg | jpg |
| 14024 | data/kfood/구이/훈제오리/Img_013_0997.jpg | 훈제오리 | Img_013_0997.jpg | jpg |
| 14025 | data/kfood/구이/훈제오리/Img_013_0998.jpg | 훈제오리 | Img_013_0998.jpg | jpg |
| 14026 | data/kfood/구이/훈제오리/Img_013_0999.jpg | 훈제오리 | Img_013_0999.jpg | jpg |
| 14027 | data/kfood/구이/훈제오리/Img_013_1000.jpg | 훈제오리 | Img_013_1000.jpg | jpg |
13964 rows × 4 columns
- 파일들은 거의 1000개씩 고루고루 있으므로 이번에도 제 취향대로 [삼겹살, 갈비구이, 곱창구이, 장어구이, 불고기, 조개구이] 정도를 선택하도록 하겠습니다.
file_list.reset_index(drop = True, inplace = True)
file_list.groupby("category").file_name.count()
category
갈비구이 998
갈치구이 998
고등어구이 1000
곱창구이 1000
닭갈비 996
더덕구이 1000
떡갈비 997
불고기 997
삼겹살 992
장어구이 997
조개구이 1000
조기구이 1000
황태구이 997
훈제오리 992
Name: file_name, dtype: int64
file_list = file_list[file_list.category.isin(["삼겹살", "갈비구이", "곱창구이", "장어구이", "불고기", "조개구이"])]
print(len(file_list))
print(file_list.category.unique())
5984
['갈비구이' '곱창구이' '불고기' '삼겹살' '장어구이' '조개구이']
- opencv는 경로에 한글명이 있으면 이미지를 못 불러오기 때문에 함수를 만들어 불러오도록 하고, opencv.imshow 를 이용하면 jupyter와 충돌이 일어나 plt.imshow 를 활용하여 데이터 시각화를 하겠습니다.
한글 경로가 있을 때 opencv가 이미지를 읽지 못하는 현상 발생 시 참고자료: (https://zzdd1558.tistory.com/228)[https://zzdd1558.tistory.com/228]
OPENCV를 jupyter에서 시각화하기 위한 참고자료 : https://stackoverflow.com/questions/46236180/opencv-imshow-will-cause-jupyter-notebook-crash
def cv2_filereader(filePath) :
stream = open(filePath.encode("utf-8"), "rb")
bytes = bytearray(stream.read())
numpyArray = np.asarray(bytes, dtype=np.uint8)
return cv2.imdecode(numpyArray , cv2.IMREAD_COLOR)
img = cv2_filereader(file_list.dir_name[0])
img.shape
(600, 900, 3)
- OPENCV는 RGB채널을 불러올 때 BGR로 불러오므로, 붉은색과 파란색이 반전이 된다. 그러므로 plt.imshow를 이용할 때는 BGR2RGB로 바꾸어주어 시각화해야한다.
plt.imshow(img)
plt.show()
plt.imshow(cv2.cvtColor(img, cv2.COLOR_BGR2RGB))
plt.show()
def show_image(dirs) :
img = cv2_filereader(dirs)
plt.imshow(cv2.cvtColor(img, cv2.COLOR_BGR2RGB))
plt.show()
show_image(file_list.dir_name[0])
- 이미지를 읽어오는데, 몇개가 Nonetype으로 읽어진다. 아마 픽셀값이 누락되거나 완전한 이미지가 아닌 것들이 있는 것 같다.(~~윈도우 상에서는 정상으로 보입니다)
- 사용할 수 있는 데이터만 가져오기 위하여 valid_chk 함수를 만들어 사용할 수 있는 데이터만 사용하도록 하겠습니다.
cv2_filereader("data/kfood/구이/갈비구이/Img_000_0145.jpg").shape[0]
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
<ipython-input-2-cb57bb692161> in <module>
----> 1 cv2_filereader("data/kfood/구이/갈비구이/Img_000_0145.jpg").shape[0]
NameError: name 'cv2_filereader' is not defined
각각의 이미지 크기를 확인해보도록 한다.
def file_valid_chk(x) :
try :
cv2_filereader(x).shape[0]
return 1
except :
return 0
file_list["valid"] = [file_valid_chk(x) for x in tqdm(file_list.dir_name)]
file_list.valid.value_counts()
100%|█████████████████████████████████████████████████████████████████████████| 5984/5984 [00:28<00:00, 207.94it/s]
1 5972
0 12
Name: valid, dtype: int64
- 데이터를 보면 dimension, 즉 모두가 컬러 이미지인 것은 확인할 수 있으나, shape(크기)가 모두 제각각이다.
- Model의 Input은 모두 동일하게 들어가야하므로 모두 통일시키는 작업이 추가되어야한다.
file_list = file_list[file_list.valid == 1]
file_list["shapes"] = [cv2_filereader(x).shape[:2] for x in tqdm(file_list.dir_name)]
file_list["dimensions"] = [cv2_filereader(x).shape[2] for x in tqdm(file_list.dir_name)]
file_list
100%|█████████████████████████████████████████████████████████████████████████| 5972/5972 [00:27<00:00, 219.15it/s]
100%|█████████████████████████████████████████████████████████████████████████| 5972/5972 [00:27<00:00, 215.28it/s]
| dir_name | category | file_name | ext_name | valid | shapes | dimensions | |
|---|---|---|---|---|---|---|---|
| 0 | data/kfood/구이/갈비구이/Img_000_0000.jpg | 갈비구이 | Img_000_0000.jpg | jpg | 1 | (600, 900) | 3 |
| 1 | data/kfood/구이/갈비구이/Img_000_0001.jpg | 갈비구이 | Img_000_0001.jpg | jpg | 1 | (480, 640) | 3 |
| 2 | data/kfood/구이/갈비구이/Img_000_0002.jpg | 갈비구이 | Img_000_0002.jpg | jpg | 1 | (600, 900) | 3 |
| 3 | data/kfood/구이/갈비구이/Img_000_0003.jpg | 갈비구이 | Img_000_0003.jpg | jpg | 1 | (336, 600) | 3 |
| 4 | data/kfood/구이/갈비구이/Img_000_0004.jpg | 갈비구이 | Img_000_0004.jpg | jpg | 1 | (705, 470) | 3 |
| ... | ... | ... | ... | ... | ... | ... | ... |
| 10970 | data/kfood/구이/조개구이/Img_010_0995.jpg | 조개구이 | Img_010_0995.jpg | jpg | 1 | (400, 394) | 3 |
| 10971 | data/kfood/구이/조개구이/Img_010_0996.jpg | 조개구이 | Img_010_0996.jpg | jpg | 1 | (400, 711) | 3 |
| 10972 | data/kfood/구이/조개구이/Img_010_0997.jpg | 조개구이 | Img_010_0997.jpg | jpg | 1 | (400, 525) | 3 |
| 10973 | data/kfood/구이/조개구이/Img_010_0998.jpg | 조개구이 | Img_010_0998.jpg | jpg | 1 | (400, 601) | 3 |
| 10974 | data/kfood/구이/조개구이/Img_010_0999.jpg | 조개구이 | Img_010_0999.jpg | jpg | 1 | (400, 602) | 3 |
5972 rows × 7 columns
file_list.dimensions.value_counts()
3 5972
Name: dimensions, dtype: int64
file_list.shapes.nunique()
936
file_list.category.nunique()
6
file_list.category.value_counts()
조개구이 1000
곱창구이 1000
불고기 995
장어구이 995
갈비구이 994
삼겹살 988
Name: category, dtype: int64
2. 모델 구축
- 이제 실제 모델을 돌릴 차례인데, 이번에는 모델을 짜지 않고 keras application에서 지원해주는 efficientNet이라는 것을 사용해볼 것이다.
from tensorflow.keras.applications import EfficientNetB0
from tensorflow.keras.layers import Dense, Input, Activation
- 이번 실습에서는 전이학습(pre-trained model의 weight를 가져오는 것)을 하지 않는다.
- 추후 포스팅할 것이다.
- include_top = True로 놓아 마지막 fully connect 층까지 가져와 다져오는 것
- 이번 포스팅에는 따로 모델을 커스텀하지 않고 커스텀하여 어떤 것을 할 수 있는지 추후 포스팅
- Input shape을 (124, 124, 3)으로 124 x 124 의 RGB 이미지로 만든다.
- 위에서 언급했듯이, 모든 이미지의 shape을 통일시켜주는 작업이 추가되어야한다.
model = EfficientNetB0(include_top=True, weights=None, input_tensor = Input(shape = (124, 124, 3)), classes = 6)
- EfficientNet 모델의 아래와 같이 시각화할 수 있습니다.
- 효율적이란 이름과 다르게 굉장히 깊은 모델을 만드는 것을 알 수 있다.
- 각 레이어가 어떠한 역할을 하는지는 추후 포스팅
from tensorflow.keras.utils import plot_model
plot_model(model, to_file= "model_png/efficent_net_B0.png")
- 모델을 정리해서 보는 방법 중 summary를 통하여 보는 방법이다.
- 위의 그림과 다르게 파라미터 개수들이 나온다.
- 깊은 층으로 갈수록 계산량이 늘어나는 것을 확인할 수 있습니다.
model.summary()
Model: "efficientnetb0"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_1 (InputLayer) [(None, 124, 124, 3) 0
__________________________________________________________________________________________________
rescaling (Rescaling) (None, 124, 124, 3) 0 input_1[0][0]
__________________________________________________________________________________________________
normalization (Normalization) (None, 124, 124, 3) 7 rescaling[0][0]
__________________________________________________________________________________________________
stem_conv_pad (ZeroPadding2D) (None, 125, 125, 3) 0 normalization[0][0]
__________________________________________________________________________________________________
stem_conv (Conv2D) (None, 62, 62, 32) 864 stem_conv_pad[0][0]
__________________________________________________________________________________________________
stem_bn (BatchNormalization) (None, 62, 62, 32) 128 stem_conv[0][0]
__________________________________________________________________________________________________
stem_activation (Activation) (None, 62, 62, 32) 0 stem_bn[0][0]
__________________________________________________________________________________________________
block1a_dwconv (DepthwiseConv2D (None, 62, 62, 32) 288 stem_activation[0][0]
__________________________________________________________________________________________________
block1a_bn (BatchNormalization) (None, 62, 62, 32) 128 block1a_dwconv[0][0]
__________________________________________________________________________________________________
block1a_activation (Activation) (None, 62, 62, 32) 0 block1a_bn[0][0]
__________________________________________________________________________________________________
block1a_se_squeeze (GlobalAvera (None, 32) 0 block1a_activation[0][0]
__________________________________________________________________________________________________
block1a_se_reshape (Reshape) (None, 1, 1, 32) 0 block1a_se_squeeze[0][0]
__________________________________________________________________________________________________
block1a_se_reduce (Conv2D) (None, 1, 1, 8) 264 block1a_se_reshape[0][0]
__________________________________________________________________________________________________
block1a_se_expand (Conv2D) (None, 1, 1, 32) 288 block1a_se_reduce[0][0]
__________________________________________________________________________________________________
block1a_se_excite (Multiply) (None, 62, 62, 32) 0 block1a_activation[0][0]
block1a_se_expand[0][0]
__________________________________________________________________________________________________
block1a_project_conv (Conv2D) (None, 62, 62, 16) 512 block1a_se_excite[0][0]
__________________________________________________________________________________________________
block1a_project_bn (BatchNormal (None, 62, 62, 16) 64 block1a_project_conv[0][0]
__________________________________________________________________________________________________
block2a_expand_conv (Conv2D) (None, 62, 62, 96) 1536 block1a_project_bn[0][0]
__________________________________________________________________________________________________
block2a_expand_bn (BatchNormali (None, 62, 62, 96) 384 block2a_expand_conv[0][0]
__________________________________________________________________________________________________
block2a_expand_activation (Acti (None, 62, 62, 96) 0 block2a_expand_bn[0][0]
__________________________________________________________________________________________________
block2a_dwconv_pad (ZeroPadding (None, 63, 63, 96) 0 block2a_expand_activation[0][0]
__________________________________________________________________________________________________
block2a_dwconv (DepthwiseConv2D (None, 31, 31, 96) 864 block2a_dwconv_pad[0][0]
__________________________________________________________________________________________________
block2a_bn (BatchNormalization) (None, 31, 31, 96) 384 block2a_dwconv[0][0]
__________________________________________________________________________________________________
block2a_activation (Activation) (None, 31, 31, 96) 0 block2a_bn[0][0]
__________________________________________________________________________________________________
block2a_se_squeeze (GlobalAvera (None, 96) 0 block2a_activation[0][0]
__________________________________________________________________________________________________
block2a_se_reshape (Reshape) (None, 1, 1, 96) 0 block2a_se_squeeze[0][0]
__________________________________________________________________________________________________
block2a_se_reduce (Conv2D) (None, 1, 1, 4) 388 block2a_se_reshape[0][0]
__________________________________________________________________________________________________
block2a_se_expand (Conv2D) (None, 1, 1, 96) 480 block2a_se_reduce[0][0]
__________________________________________________________________________________________________
block2a_se_excite (Multiply) (None, 31, 31, 96) 0 block2a_activation[0][0]
block2a_se_expand[0][0]
__________________________________________________________________________________________________
block2a_project_conv (Conv2D) (None, 31, 31, 24) 2304 block2a_se_excite[0][0]
__________________________________________________________________________________________________
block2a_project_bn (BatchNormal (None, 31, 31, 24) 96 block2a_project_conv[0][0]
__________________________________________________________________________________________________
block2b_expand_conv (Conv2D) (None, 31, 31, 144) 3456 block2a_project_bn[0][0]
__________________________________________________________________________________________________
block2b_expand_bn (BatchNormali (None, 31, 31, 144) 576 block2b_expand_conv[0][0]
__________________________________________________________________________________________________
block2b_expand_activation (Acti (None, 31, 31, 144) 0 block2b_expand_bn[0][0]
__________________________________________________________________________________________________
block2b_dwconv (DepthwiseConv2D (None, 31, 31, 144) 1296 block2b_expand_activation[0][0]
__________________________________________________________________________________________________
block2b_bn (BatchNormalization) (None, 31, 31, 144) 576 block2b_dwconv[0][0]
__________________________________________________________________________________________________
block2b_activation (Activation) (None, 31, 31, 144) 0 block2b_bn[0][0]
__________________________________________________________________________________________________
block2b_se_squeeze (GlobalAvera (None, 144) 0 block2b_activation[0][0]
__________________________________________________________________________________________________
block2b_se_reshape (Reshape) (None, 1, 1, 144) 0 block2b_se_squeeze[0][0]
__________________________________________________________________________________________________
block2b_se_reduce (Conv2D) (None, 1, 1, 6) 870 block2b_se_reshape[0][0]
__________________________________________________________________________________________________
block2b_se_expand (Conv2D) (None, 1, 1, 144) 1008 block2b_se_reduce[0][0]
__________________________________________________________________________________________________
block2b_se_excite (Multiply) (None, 31, 31, 144) 0 block2b_activation[0][0]
block2b_se_expand[0][0]
__________________________________________________________________________________________________
block2b_project_conv (Conv2D) (None, 31, 31, 24) 3456 block2b_se_excite[0][0]
__________________________________________________________________________________________________
block2b_project_bn (BatchNormal (None, 31, 31, 24) 96 block2b_project_conv[0][0]
__________________________________________________________________________________________________
block2b_drop (Dropout) (None, 31, 31, 24) 0 block2b_project_bn[0][0]
__________________________________________________________________________________________________
block2b_add (Add) (None, 31, 31, 24) 0 block2b_drop[0][0]
block2a_project_bn[0][0]
__________________________________________________________________________________________________
block3a_expand_conv (Conv2D) (None, 31, 31, 144) 3456 block2b_add[0][0]
__________________________________________________________________________________________________
block3a_expand_bn (BatchNormali (None, 31, 31, 144) 576 block3a_expand_conv[0][0]
__________________________________________________________________________________________________
block3a_expand_activation (Acti (None, 31, 31, 144) 0 block3a_expand_bn[0][0]
__________________________________________________________________________________________________
block3a_dwconv_pad (ZeroPadding (None, 35, 35, 144) 0 block3a_expand_activation[0][0]
__________________________________________________________________________________________________
block3a_dwconv (DepthwiseConv2D (None, 16, 16, 144) 3600 block3a_dwconv_pad[0][0]
__________________________________________________________________________________________________
block3a_bn (BatchNormalization) (None, 16, 16, 144) 576 block3a_dwconv[0][0]
__________________________________________________________________________________________________
block3a_activation (Activation) (None, 16, 16, 144) 0 block3a_bn[0][0]
__________________________________________________________________________________________________
block3a_se_squeeze (GlobalAvera (None, 144) 0 block3a_activation[0][0]
__________________________________________________________________________________________________
block3a_se_reshape (Reshape) (None, 1, 1, 144) 0 block3a_se_squeeze[0][0]
__________________________________________________________________________________________________
block3a_se_reduce (Conv2D) (None, 1, 1, 6) 870 block3a_se_reshape[0][0]
__________________________________________________________________________________________________
block3a_se_expand (Conv2D) (None, 1, 1, 144) 1008 block3a_se_reduce[0][0]
__________________________________________________________________________________________________
block3a_se_excite (Multiply) (None, 16, 16, 144) 0 block3a_activation[0][0]
block3a_se_expand[0][0]
__________________________________________________________________________________________________
block3a_project_conv (Conv2D) (None, 16, 16, 40) 5760 block3a_se_excite[0][0]
__________________________________________________________________________________________________
block3a_project_bn (BatchNormal (None, 16, 16, 40) 160 block3a_project_conv[0][0]
__________________________________________________________________________________________________
block3b_expand_conv (Conv2D) (None, 16, 16, 240) 9600 block3a_project_bn[0][0]
__________________________________________________________________________________________________
block3b_expand_bn (BatchNormali (None, 16, 16, 240) 960 block3b_expand_conv[0][0]
__________________________________________________________________________________________________
block3b_expand_activation (Acti (None, 16, 16, 240) 0 block3b_expand_bn[0][0]
__________________________________________________________________________________________________
block3b_dwconv (DepthwiseConv2D (None, 16, 16, 240) 6000 block3b_expand_activation[0][0]
__________________________________________________________________________________________________
block3b_bn (BatchNormalization) (None, 16, 16, 240) 960 block3b_dwconv[0][0]
__________________________________________________________________________________________________
block3b_activation (Activation) (None, 16, 16, 240) 0 block3b_bn[0][0]
__________________________________________________________________________________________________
block3b_se_squeeze (GlobalAvera (None, 240) 0 block3b_activation[0][0]
__________________________________________________________________________________________________
block3b_se_reshape (Reshape) (None, 1, 1, 240) 0 block3b_se_squeeze[0][0]
__________________________________________________________________________________________________
block3b_se_reduce (Conv2D) (None, 1, 1, 10) 2410 block3b_se_reshape[0][0]
__________________________________________________________________________________________________
block3b_se_expand (Conv2D) (None, 1, 1, 240) 2640 block3b_se_reduce[0][0]
__________________________________________________________________________________________________
block3b_se_excite (Multiply) (None, 16, 16, 240) 0 block3b_activation[0][0]
block3b_se_expand[0][0]
__________________________________________________________________________________________________
block3b_project_conv (Conv2D) (None, 16, 16, 40) 9600 block3b_se_excite[0][0]
__________________________________________________________________________________________________
block3b_project_bn (BatchNormal (None, 16, 16, 40) 160 block3b_project_conv[0][0]
__________________________________________________________________________________________________
block3b_drop (Dropout) (None, 16, 16, 40) 0 block3b_project_bn[0][0]
__________________________________________________________________________________________________
block3b_add (Add) (None, 16, 16, 40) 0 block3b_drop[0][0]
block3a_project_bn[0][0]
__________________________________________________________________________________________________
block4a_expand_conv (Conv2D) (None, 16, 16, 240) 9600 block3b_add[0][0]
__________________________________________________________________________________________________
block4a_expand_bn (BatchNormali (None, 16, 16, 240) 960 block4a_expand_conv[0][0]
__________________________________________________________________________________________________
block4a_expand_activation (Acti (None, 16, 16, 240) 0 block4a_expand_bn[0][0]
__________________________________________________________________________________________________
block4a_dwconv_pad (ZeroPadding (None, 17, 17, 240) 0 block4a_expand_activation[0][0]
__________________________________________________________________________________________________
block4a_dwconv (DepthwiseConv2D (None, 8, 8, 240) 2160 block4a_dwconv_pad[0][0]
__________________________________________________________________________________________________
block4a_bn (BatchNormalization) (None, 8, 8, 240) 960 block4a_dwconv[0][0]
__________________________________________________________________________________________________
block4a_activation (Activation) (None, 8, 8, 240) 0 block4a_bn[0][0]
__________________________________________________________________________________________________
block4a_se_squeeze (GlobalAvera (None, 240) 0 block4a_activation[0][0]
__________________________________________________________________________________________________
block4a_se_reshape (Reshape) (None, 1, 1, 240) 0 block4a_se_squeeze[0][0]
__________________________________________________________________________________________________
block4a_se_reduce (Conv2D) (None, 1, 1, 10) 2410 block4a_se_reshape[0][0]
__________________________________________________________________________________________________
block4a_se_expand (Conv2D) (None, 1, 1, 240) 2640 block4a_se_reduce[0][0]
__________________________________________________________________________________________________
block4a_se_excite (Multiply) (None, 8, 8, 240) 0 block4a_activation[0][0]
block4a_se_expand[0][0]
__________________________________________________________________________________________________
block4a_project_conv (Conv2D) (None, 8, 8, 80) 19200 block4a_se_excite[0][0]
__________________________________________________________________________________________________
block4a_project_bn (BatchNormal (None, 8, 8, 80) 320 block4a_project_conv[0][0]
__________________________________________________________________________________________________
block4b_expand_conv (Conv2D) (None, 8, 8, 480) 38400 block4a_project_bn[0][0]
__________________________________________________________________________________________________
block4b_expand_bn (BatchNormali (None, 8, 8, 480) 1920 block4b_expand_conv[0][0]
__________________________________________________________________________________________________
block4b_expand_activation (Acti (None, 8, 8, 480) 0 block4b_expand_bn[0][0]
__________________________________________________________________________________________________
block4b_dwconv (DepthwiseConv2D (None, 8, 8, 480) 4320 block4b_expand_activation[0][0]
__________________________________________________________________________________________________
block4b_bn (BatchNormalization) (None, 8, 8, 480) 1920 block4b_dwconv[0][0]
__________________________________________________________________________________________________
block4b_activation (Activation) (None, 8, 8, 480) 0 block4b_bn[0][0]
__________________________________________________________________________________________________
block4b_se_squeeze (GlobalAvera (None, 480) 0 block4b_activation[0][0]
__________________________________________________________________________________________________
block4b_se_reshape (Reshape) (None, 1, 1, 480) 0 block4b_se_squeeze[0][0]
__________________________________________________________________________________________________
block4b_se_reduce (Conv2D) (None, 1, 1, 20) 9620 block4b_se_reshape[0][0]
__________________________________________________________________________________________________
block4b_se_expand (Conv2D) (None, 1, 1, 480) 10080 block4b_se_reduce[0][0]
__________________________________________________________________________________________________
block4b_se_excite (Multiply) (None, 8, 8, 480) 0 block4b_activation[0][0]
block4b_se_expand[0][0]
__________________________________________________________________________________________________
block4b_project_conv (Conv2D) (None, 8, 8, 80) 38400 block4b_se_excite[0][0]
__________________________________________________________________________________________________
block4b_project_bn (BatchNormal (None, 8, 8, 80) 320 block4b_project_conv[0][0]
__________________________________________________________________________________________________
block4b_drop (Dropout) (None, 8, 8, 80) 0 block4b_project_bn[0][0]
__________________________________________________________________________________________________
block4b_add (Add) (None, 8, 8, 80) 0 block4b_drop[0][0]
block4a_project_bn[0][0]
__________________________________________________________________________________________________
block4c_expand_conv (Conv2D) (None, 8, 8, 480) 38400 block4b_add[0][0]
__________________________________________________________________________________________________
block4c_expand_bn (BatchNormali (None, 8, 8, 480) 1920 block4c_expand_conv[0][0]
__________________________________________________________________________________________________
block4c_expand_activation (Acti (None, 8, 8, 480) 0 block4c_expand_bn[0][0]
__________________________________________________________________________________________________
block4c_dwconv (DepthwiseConv2D (None, 8, 8, 480) 4320 block4c_expand_activation[0][0]
__________________________________________________________________________________________________
block4c_bn (BatchNormalization) (None, 8, 8, 480) 1920 block4c_dwconv[0][0]
__________________________________________________________________________________________________
block4c_activation (Activation) (None, 8, 8, 480) 0 block4c_bn[0][0]
__________________________________________________________________________________________________
block4c_se_squeeze (GlobalAvera (None, 480) 0 block4c_activation[0][0]
__________________________________________________________________________________________________
block4c_se_reshape (Reshape) (None, 1, 1, 480) 0 block4c_se_squeeze[0][0]
__________________________________________________________________________________________________
block4c_se_reduce (Conv2D) (None, 1, 1, 20) 9620 block4c_se_reshape[0][0]
__________________________________________________________________________________________________
block4c_se_expand (Conv2D) (None, 1, 1, 480) 10080 block4c_se_reduce[0][0]
__________________________________________________________________________________________________
block4c_se_excite (Multiply) (None, 8, 8, 480) 0 block4c_activation[0][0]
block4c_se_expand[0][0]
__________________________________________________________________________________________________
block4c_project_conv (Conv2D) (None, 8, 8, 80) 38400 block4c_se_excite[0][0]
__________________________________________________________________________________________________
block4c_project_bn (BatchNormal (None, 8, 8, 80) 320 block4c_project_conv[0][0]
__________________________________________________________________________________________________
block4c_drop (Dropout) (None, 8, 8, 80) 0 block4c_project_bn[0][0]
__________________________________________________________________________________________________
block4c_add (Add) (None, 8, 8, 80) 0 block4c_drop[0][0]
block4b_add[0][0]
__________________________________________________________________________________________________
block5a_expand_conv (Conv2D) (None, 8, 8, 480) 38400 block4c_add[0][0]
__________________________________________________________________________________________________
block5a_expand_bn (BatchNormali (None, 8, 8, 480) 1920 block5a_expand_conv[0][0]
__________________________________________________________________________________________________
block5a_expand_activation (Acti (None, 8, 8, 480) 0 block5a_expand_bn[0][0]
__________________________________________________________________________________________________
block5a_dwconv (DepthwiseConv2D (None, 8, 8, 480) 12000 block5a_expand_activation[0][0]
__________________________________________________________________________________________________
block5a_bn (BatchNormalization) (None, 8, 8, 480) 1920 block5a_dwconv[0][0]
__________________________________________________________________________________________________
block5a_activation (Activation) (None, 8, 8, 480) 0 block5a_bn[0][0]
__________________________________________________________________________________________________
block5a_se_squeeze (GlobalAvera (None, 480) 0 block5a_activation[0][0]
__________________________________________________________________________________________________
block5a_se_reshape (Reshape) (None, 1, 1, 480) 0 block5a_se_squeeze[0][0]
__________________________________________________________________________________________________
block5a_se_reduce (Conv2D) (None, 1, 1, 20) 9620 block5a_se_reshape[0][0]
__________________________________________________________________________________________________
block5a_se_expand (Conv2D) (None, 1, 1, 480) 10080 block5a_se_reduce[0][0]
__________________________________________________________________________________________________
block5a_se_excite (Multiply) (None, 8, 8, 480) 0 block5a_activation[0][0]
block5a_se_expand[0][0]
__________________________________________________________________________________________________
block5a_project_conv (Conv2D) (None, 8, 8, 112) 53760 block5a_se_excite[0][0]
__________________________________________________________________________________________________
block5a_project_bn (BatchNormal (None, 8, 8, 112) 448 block5a_project_conv[0][0]
__________________________________________________________________________________________________
block5b_expand_conv (Conv2D) (None, 8, 8, 672) 75264 block5a_project_bn[0][0]
__________________________________________________________________________________________________
block5b_expand_bn (BatchNormali (None, 8, 8, 672) 2688 block5b_expand_conv[0][0]
__________________________________________________________________________________________________
block5b_expand_activation (Acti (None, 8, 8, 672) 0 block5b_expand_bn[0][0]
__________________________________________________________________________________________________
block5b_dwconv (DepthwiseConv2D (None, 8, 8, 672) 16800 block5b_expand_activation[0][0]
__________________________________________________________________________________________________
block5b_bn (BatchNormalization) (None, 8, 8, 672) 2688 block5b_dwconv[0][0]
__________________________________________________________________________________________________
block5b_activation (Activation) (None, 8, 8, 672) 0 block5b_bn[0][0]
__________________________________________________________________________________________________
block5b_se_squeeze (GlobalAvera (None, 672) 0 block5b_activation[0][0]
__________________________________________________________________________________________________
block5b_se_reshape (Reshape) (None, 1, 1, 672) 0 block5b_se_squeeze[0][0]
__________________________________________________________________________________________________
block5b_se_reduce (Conv2D) (None, 1, 1, 28) 18844 block5b_se_reshape[0][0]
__________________________________________________________________________________________________
block5b_se_expand (Conv2D) (None, 1, 1, 672) 19488 block5b_se_reduce[0][0]
__________________________________________________________________________________________________
block5b_se_excite (Multiply) (None, 8, 8, 672) 0 block5b_activation[0][0]
block5b_se_expand[0][0]
__________________________________________________________________________________________________
block5b_project_conv (Conv2D) (None, 8, 8, 112) 75264 block5b_se_excite[0][0]
__________________________________________________________________________________________________
block5b_project_bn (BatchNormal (None, 8, 8, 112) 448 block5b_project_conv[0][0]
__________________________________________________________________________________________________
block5b_drop (Dropout) (None, 8, 8, 112) 0 block5b_project_bn[0][0]
__________________________________________________________________________________________________
block5b_add (Add) (None, 8, 8, 112) 0 block5b_drop[0][0]
block5a_project_bn[0][0]
__________________________________________________________________________________________________
block5c_expand_conv (Conv2D) (None, 8, 8, 672) 75264 block5b_add[0][0]
__________________________________________________________________________________________________
block5c_expand_bn (BatchNormali (None, 8, 8, 672) 2688 block5c_expand_conv[0][0]
__________________________________________________________________________________________________
block5c_expand_activation (Acti (None, 8, 8, 672) 0 block5c_expand_bn[0][0]
__________________________________________________________________________________________________
block5c_dwconv (DepthwiseConv2D (None, 8, 8, 672) 16800 block5c_expand_activation[0][0]
__________________________________________________________________________________________________
block5c_bn (BatchNormalization) (None, 8, 8, 672) 2688 block5c_dwconv[0][0]
__________________________________________________________________________________________________
block5c_activation (Activation) (None, 8, 8, 672) 0 block5c_bn[0][0]
__________________________________________________________________________________________________
block5c_se_squeeze (GlobalAvera (None, 672) 0 block5c_activation[0][0]
__________________________________________________________________________________________________
block5c_se_reshape (Reshape) (None, 1, 1, 672) 0 block5c_se_squeeze[0][0]
__________________________________________________________________________________________________
block5c_se_reduce (Conv2D) (None, 1, 1, 28) 18844 block5c_se_reshape[0][0]
__________________________________________________________________________________________________
block5c_se_expand (Conv2D) (None, 1, 1, 672) 19488 block5c_se_reduce[0][0]
__________________________________________________________________________________________________
block5c_se_excite (Multiply) (None, 8, 8, 672) 0 block5c_activation[0][0]
block5c_se_expand[0][0]
__________________________________________________________________________________________________
block5c_project_conv (Conv2D) (None, 8, 8, 112) 75264 block5c_se_excite[0][0]
__________________________________________________________________________________________________
block5c_project_bn (BatchNormal (None, 8, 8, 112) 448 block5c_project_conv[0][0]
__________________________________________________________________________________________________
block5c_drop (Dropout) (None, 8, 8, 112) 0 block5c_project_bn[0][0]
__________________________________________________________________________________________________
block5c_add (Add) (None, 8, 8, 112) 0 block5c_drop[0][0]
block5b_add[0][0]
__________________________________________________________________________________________________
block6a_expand_conv (Conv2D) (None, 8, 8, 672) 75264 block5c_add[0][0]
__________________________________________________________________________________________________
block6a_expand_bn (BatchNormali (None, 8, 8, 672) 2688 block6a_expand_conv[0][0]
__________________________________________________________________________________________________
block6a_expand_activation (Acti (None, 8, 8, 672) 0 block6a_expand_bn[0][0]
__________________________________________________________________________________________________
block6a_dwconv_pad (ZeroPadding (None, 11, 11, 672) 0 block6a_expand_activation[0][0]
__________________________________________________________________________________________________
block6a_dwconv (DepthwiseConv2D (None, 4, 4, 672) 16800 block6a_dwconv_pad[0][0]
__________________________________________________________________________________________________
block6a_bn (BatchNormalization) (None, 4, 4, 672) 2688 block6a_dwconv[0][0]
__________________________________________________________________________________________________
block6a_activation (Activation) (None, 4, 4, 672) 0 block6a_bn[0][0]
__________________________________________________________________________________________________
block6a_se_squeeze (GlobalAvera (None, 672) 0 block6a_activation[0][0]
__________________________________________________________________________________________________
block6a_se_reshape (Reshape) (None, 1, 1, 672) 0 block6a_se_squeeze[0][0]
__________________________________________________________________________________________________
block6a_se_reduce (Conv2D) (None, 1, 1, 28) 18844 block6a_se_reshape[0][0]
__________________________________________________________________________________________________
block6a_se_expand (Conv2D) (None, 1, 1, 672) 19488 block6a_se_reduce[0][0]
__________________________________________________________________________________________________
block6a_se_excite (Multiply) (None, 4, 4, 672) 0 block6a_activation[0][0]
block6a_se_expand[0][0]
__________________________________________________________________________________________________
block6a_project_conv (Conv2D) (None, 4, 4, 192) 129024 block6a_se_excite[0][0]
__________________________________________________________________________________________________
block6a_project_bn (BatchNormal (None, 4, 4, 192) 768 block6a_project_conv[0][0]
__________________________________________________________________________________________________
block6b_expand_conv (Conv2D) (None, 4, 4, 1152) 221184 block6a_project_bn[0][0]
__________________________________________________________________________________________________
block6b_expand_bn (BatchNormali (None, 4, 4, 1152) 4608 block6b_expand_conv[0][0]
__________________________________________________________________________________________________
block6b_expand_activation (Acti (None, 4, 4, 1152) 0 block6b_expand_bn[0][0]
__________________________________________________________________________________________________
block6b_dwconv (DepthwiseConv2D (None, 4, 4, 1152) 28800 block6b_expand_activation[0][0]
__________________________________________________________________________________________________
block6b_bn (BatchNormalization) (None, 4, 4, 1152) 4608 block6b_dwconv[0][0]
__________________________________________________________________________________________________
block6b_activation (Activation) (None, 4, 4, 1152) 0 block6b_bn[0][0]
__________________________________________________________________________________________________
block6b_se_squeeze (GlobalAvera (None, 1152) 0 block6b_activation[0][0]
__________________________________________________________________________________________________
block6b_se_reshape (Reshape) (None, 1, 1, 1152) 0 block6b_se_squeeze[0][0]
__________________________________________________________________________________________________
block6b_se_reduce (Conv2D) (None, 1, 1, 48) 55344 block6b_se_reshape[0][0]
__________________________________________________________________________________________________
block6b_se_expand (Conv2D) (None, 1, 1, 1152) 56448 block6b_se_reduce[0][0]
__________________________________________________________________________________________________
block6b_se_excite (Multiply) (None, 4, 4, 1152) 0 block6b_activation[0][0]
block6b_se_expand[0][0]
__________________________________________________________________________________________________
block6b_project_conv (Conv2D) (None, 4, 4, 192) 221184 block6b_se_excite[0][0]
__________________________________________________________________________________________________
block6b_project_bn (BatchNormal (None, 4, 4, 192) 768 block6b_project_conv[0][0]
__________________________________________________________________________________________________
block6b_drop (Dropout) (None, 4, 4, 192) 0 block6b_project_bn[0][0]
__________________________________________________________________________________________________
block6b_add (Add) (None, 4, 4, 192) 0 block6b_drop[0][0]
block6a_project_bn[0][0]
__________________________________________________________________________________________________
block6c_expand_conv (Conv2D) (None, 4, 4, 1152) 221184 block6b_add[0][0]
__________________________________________________________________________________________________
block6c_expand_bn (BatchNormali (None, 4, 4, 1152) 4608 block6c_expand_conv[0][0]
__________________________________________________________________________________________________
block6c_expand_activation (Acti (None, 4, 4, 1152) 0 block6c_expand_bn[0][0]
__________________________________________________________________________________________________
block6c_dwconv (DepthwiseConv2D (None, 4, 4, 1152) 28800 block6c_expand_activation[0][0]
__________________________________________________________________________________________________
block6c_bn (BatchNormalization) (None, 4, 4, 1152) 4608 block6c_dwconv[0][0]
__________________________________________________________________________________________________
block6c_activation (Activation) (None, 4, 4, 1152) 0 block6c_bn[0][0]
__________________________________________________________________________________________________
block6c_se_squeeze (GlobalAvera (None, 1152) 0 block6c_activation[0][0]
__________________________________________________________________________________________________
block6c_se_reshape (Reshape) (None, 1, 1, 1152) 0 block6c_se_squeeze[0][0]
__________________________________________________________________________________________________
block6c_se_reduce (Conv2D) (None, 1, 1, 48) 55344 block6c_se_reshape[0][0]
__________________________________________________________________________________________________
block6c_se_expand (Conv2D) (None, 1, 1, 1152) 56448 block6c_se_reduce[0][0]
__________________________________________________________________________________________________
block6c_se_excite (Multiply) (None, 4, 4, 1152) 0 block6c_activation[0][0]
block6c_se_expand[0][0]
__________________________________________________________________________________________________
block6c_project_conv (Conv2D) (None, 4, 4, 192) 221184 block6c_se_excite[0][0]
__________________________________________________________________________________________________
block6c_project_bn (BatchNormal (None, 4, 4, 192) 768 block6c_project_conv[0][0]
__________________________________________________________________________________________________
block6c_drop (Dropout) (None, 4, 4, 192) 0 block6c_project_bn[0][0]
__________________________________________________________________________________________________
block6c_add (Add) (None, 4, 4, 192) 0 block6c_drop[0][0]
block6b_add[0][0]
__________________________________________________________________________________________________
block6d_expand_conv (Conv2D) (None, 4, 4, 1152) 221184 block6c_add[0][0]
__________________________________________________________________________________________________
block6d_expand_bn (BatchNormali (None, 4, 4, 1152) 4608 block6d_expand_conv[0][0]
__________________________________________________________________________________________________
block6d_expand_activation (Acti (None, 4, 4, 1152) 0 block6d_expand_bn[0][0]
__________________________________________________________________________________________________
block6d_dwconv (DepthwiseConv2D (None, 4, 4, 1152) 28800 block6d_expand_activation[0][0]
__________________________________________________________________________________________________
block6d_bn (BatchNormalization) (None, 4, 4, 1152) 4608 block6d_dwconv[0][0]
__________________________________________________________________________________________________
block6d_activation (Activation) (None, 4, 4, 1152) 0 block6d_bn[0][0]
__________________________________________________________________________________________________
block6d_se_squeeze (GlobalAvera (None, 1152) 0 block6d_activation[0][0]
__________________________________________________________________________________________________
block6d_se_reshape (Reshape) (None, 1, 1, 1152) 0 block6d_se_squeeze[0][0]
__________________________________________________________________________________________________
block6d_se_reduce (Conv2D) (None, 1, 1, 48) 55344 block6d_se_reshape[0][0]
__________________________________________________________________________________________________
block6d_se_expand (Conv2D) (None, 1, 1, 1152) 56448 block6d_se_reduce[0][0]
__________________________________________________________________________________________________
block6d_se_excite (Multiply) (None, 4, 4, 1152) 0 block6d_activation[0][0]
block6d_se_expand[0][0]
__________________________________________________________________________________________________
block6d_project_conv (Conv2D) (None, 4, 4, 192) 221184 block6d_se_excite[0][0]
__________________________________________________________________________________________________
block6d_project_bn (BatchNormal (None, 4, 4, 192) 768 block6d_project_conv[0][0]
__________________________________________________________________________________________________
block6d_drop (Dropout) (None, 4, 4, 192) 0 block6d_project_bn[0][0]
__________________________________________________________________________________________________
block6d_add (Add) (None, 4, 4, 192) 0 block6d_drop[0][0]
block6c_add[0][0]
__________________________________________________________________________________________________
block7a_expand_conv (Conv2D) (None, 4, 4, 1152) 221184 block6d_add[0][0]
__________________________________________________________________________________________________
block7a_expand_bn (BatchNormali (None, 4, 4, 1152) 4608 block7a_expand_conv[0][0]
__________________________________________________________________________________________________
block7a_expand_activation (Acti (None, 4, 4, 1152) 0 block7a_expand_bn[0][0]
__________________________________________________________________________________________________
block7a_dwconv (DepthwiseConv2D (None, 4, 4, 1152) 10368 block7a_expand_activation[0][0]
__________________________________________________________________________________________________
block7a_bn (BatchNormalization) (None, 4, 4, 1152) 4608 block7a_dwconv[0][0]
__________________________________________________________________________________________________
block7a_activation (Activation) (None, 4, 4, 1152) 0 block7a_bn[0][0]
__________________________________________________________________________________________________
block7a_se_squeeze (GlobalAvera (None, 1152) 0 block7a_activation[0][0]
__________________________________________________________________________________________________
block7a_se_reshape (Reshape) (None, 1, 1, 1152) 0 block7a_se_squeeze[0][0]
__________________________________________________________________________________________________
block7a_se_reduce (Conv2D) (None, 1, 1, 48) 55344 block7a_se_reshape[0][0]
__________________________________________________________________________________________________
block7a_se_expand (Conv2D) (None, 1, 1, 1152) 56448 block7a_se_reduce[0][0]
__________________________________________________________________________________________________
block7a_se_excite (Multiply) (None, 4, 4, 1152) 0 block7a_activation[0][0]
block7a_se_expand[0][0]
__________________________________________________________________________________________________
block7a_project_conv (Conv2D) (None, 4, 4, 320) 368640 block7a_se_excite[0][0]
__________________________________________________________________________________________________
block7a_project_bn (BatchNormal (None, 4, 4, 320) 1280 block7a_project_conv[0][0]
__________________________________________________________________________________________________
top_conv (Conv2D) (None, 4, 4, 1280) 409600 block7a_project_bn[0][0]
__________________________________________________________________________________________________
top_bn (BatchNormalization) (None, 4, 4, 1280) 5120 top_conv[0][0]
__________________________________________________________________________________________________
top_activation (Activation) (None, 4, 4, 1280) 0 top_bn[0][0]
__________________________________________________________________________________________________
avg_pool (GlobalAveragePooling2 (None, 1280) 0 top_activation[0][0]
__________________________________________________________________________________________________
top_dropout (Dropout) (None, 1280) 0 avg_pool[0][0]
__________________________________________________________________________________________________
predictions (Dense) (None, 6) 7686 top_dropout[0][0]
==================================================================================================
Total params: 4,057,257
Trainable params: 4,015,234
Non-trainable params: 42,023
__________________________________________________________________________________________________
- 다 만들어진 모델을 최종으로 Complie 합니다.
- 이때 들어가는 파라미터인 Optimizer와 loss 에 대해서도 추후 포스팅할 것이다.
model.compile(optimizer='Adam', loss='categorical_crossentropy')
3. Keras 모델에 학습시키기 위한 Train, Test Set 구축하기
- 이번 실습에서는 Keras의 ImageDataGenerator를 활용하여 학습을 시키겠습니다.
- 이미지는 RGB값이 각각 0~254로 총 255의 값을 갖게 되는데, 이 때 225로 나누어 0~1까지 Nomalization을 해줍니다.
- 이는 Global Minima에 효과적으로 다가갈 수 있도록 도와줍니다..
- 이는 Global Minima에 효과적으로 다가갈 수 있도록 도와줍니다..
- 이미지는 RGB값이 각각 0~254로 총 255의 값을 갖게 되는데, 이 때 225로 나누어 0~1까지 Nomalization을 해줍니다.
DataGenerator = tf.keras.preprocessing.image.ImageDataGenerator(rescale = 1. / 255, validation_split = 0.2)
- Image Data Generator 중에 flow_from_dataframe이라는 함수를 활용하여 DataFrame에 있는 정보로 Keras 모델을 학습시키도록 하겠습니다.
- Input size를 통일시켜줍니다.
- Y값을 category화 시켜줍니다.
- 기타 파라미터들을 조절하여 학습을 쉽고 효과적으로 하도록 도와줍니다.
- 딥러닝도 역시 train과 test(혹은 validation)셋을 나누어 학습을 시켜야합니다.
- validation set을 나누어 early stopping 같은 overfitting 방지와 학습 시간을 줄일 수 있다. (추후 포스팅)
- 무언가 잘못됬는지, flow_from_dataframe을 활용하면 랜덤하게 training, validation set을 구축하지 않고 0~80%, 80~100%로 그냥 나누어버리기 때문에, data frame을 랜덤하게 sorting한 뒤 generator를 만들도록 하겠습니다.
- 이번 포스팅에는 Augmentation은 하지 않고 추후에 포스팅하도록 하겠습니다.
file_list = file_list.sample(frac = 1)
train_generator = DataGenerator.flow_from_dataframe(file_list, # 데이터프레임
x_col = 'dir_name', # 파일위치 열이름
y_col = 'category', # 클래스 열이름
target_size = (124,124), # 이미지 사이즈
color_mode= 'rgb', # 이미지 채널수
class_mode= 'categorical', # Y값 변화방법
batch_size= 32, # 배치사이즈
Shuffle = True, # 랜덤 여부
seed = 42, # 랜덤엔 시드
subset = "training"
)
Found 4778 validated image filenames belonging to 6 classes.
val_generator = DataGenerator.flow_from_dataframe(file_list, # 데이터프레임
x_col = 'dir_name', # 파일위치 열이름
y_col = 'category', # 클래스 열이름
target_size = (124,124), # 이미지 사이즈
color_mode= 'rgb', # 이미지 채널수
class_mode= 'categorical', # Y값 변화방법
batch_size= 32, # 배치사이즈
Shuffle = True, # 랜덤 여부
seed = 42, # 랜덤엔 시드
subset = "validation"
)
Found 1194 validated image filenames belonging to 6 classes.
- generator에 있는 이미지가 124, 124의 크기로 바뀌고, Y값이 One-Hot encoding 된 것을 확인할 수 있습니다.
training_y = list()
for i in tqdm(range(len(train_generator))) :
training_y = training_y + list(train_generator[i][1])
training_y = pd.DataFrame(training_y)
training_y["Y"] = training_y.idxmax(axis = "columns")
training_y.Y.value_counts().sort_index()
100%|████████████████████████████████████████████████████████████████████████████| 150/150 [00:14<00:00, 10.69it/s]
0 790
1 788
2 827
3 774
4 807
5 792
Name: Y, dtype: int64
val_y = list()
for i in tqdm(range(len(val_generator))) :
val_y = val_y + list(val_generator[i][1])
val_y = pd.DataFrame(val_y)
val_y["Y"] = val_y.idxmax(axis = "columns")
val_y.Y.value_counts().sort_index()
100%|██████████████████████████████████████████████████████████████████████████████| 38/38 [00:03<00:00, 12.14it/s]
0 204
1 212
2 168
3 214
4 188
5 208
Name: Y, dtype: int64
- 위에서 1/255를 해주었기 때문에 다시 이미지를 복구하기 위하여 * 255를 한 값을 이미지화하여 확인합니다.
chk_ = train_generator.next()
print(chk_[0][0].shape)
print(chk_[1][0])
plt.imshow((chk_[0][0] * 255).astype(int))
plt.show()
(124, 124, 3)
[0. 0. 0. 0. 1. 0.]
chk_ = val_generator.next()
print(chk_[0][0].shape)
print(chk_[1][0])
plt.imshow((chk_[0][0] * 255).astype(int))
plt.show()
(124, 124, 3)
[0. 0. 1. 0. 0. 0.]
4. 학습 및 검증
%time hist = model.fit_generator(train_generator, epochs=50, verbose=1)
Epoch 1/50
150/150 [==============================] - 201s 1s/step - loss: 3.0040
Epoch 2/50
150/150 [==============================] - 197s 1s/step - loss: 2.1460
Epoch 3/50
150/150 [==============================] - 192s 1s/step - loss: 1.9840
Epoch 4/50
150/150 [==============================] - 187s 1s/step - loss: 1.7931
Epoch 5/50
150/150 [==============================] - 189s 1s/step - loss: 1.5907
Epoch 6/50
150/150 [==============================] - 185s 1s/step - loss: 1.3879
Epoch 7/50
150/150 [==============================] - 186s 1s/step - loss: 1.2180
Epoch 8/50
150/150 [==============================] - 202s 1s/step - loss: 1.0472
Epoch 9/50
150/150 [==============================] - 196s 1s/step - loss: 0.8599
Epoch 10/50
150/150 [==============================] - 189s 1s/step - loss: 0.7362
Epoch 11/50
150/150 [==============================] - 195s 1s/step - loss: 0.7367
Epoch 12/50
150/150 [==============================] - 198s 1s/step - loss: 0.6045
Epoch 13/50
150/150 [==============================] - 188s 1s/step - loss: 0.6141
Epoch 14/50
150/150 [==============================] - 187s 1s/step - loss: 0.5978
Epoch 15/50
150/150 [==============================] - 192s 1s/step - loss: 0.5272
Epoch 16/50
150/150 [==============================] - 187s 1s/step - loss: 0.5383
Epoch 17/50
150/150 [==============================] - 192s 1s/step - loss: 0.4453
Epoch 18/50
150/150 [==============================] - 194s 1s/step - loss: 0.3941
Epoch 19/50
150/150 [==============================] - 189s 1s/step - loss: 0.3781
Epoch 20/50
150/150 [==============================] - 199s 1s/step - loss: 0.3523
Epoch 21/50
150/150 [==============================] - 195s 1s/step - loss: 0.3675
Epoch 22/50
150/150 [==============================] - 192s 1s/step - loss: 0.4094
Epoch 23/50
150/150 [==============================] - 199s 1s/step - loss: 0.3186
Epoch 24/50
150/150 [==============================] - 195s 1s/step - loss: 0.2725
Epoch 25/50
150/150 [==============================] - 208s 1s/step - loss: 0.2893
Epoch 26/50
150/150 [==============================] - 198s 1s/step - loss: 0.2878
Epoch 27/50
150/150 [==============================] - 201s 1s/step - loss: 0.2867
Epoch 28/50
150/150 [==============================] - 250s 2s/step - loss: 0.2562
Epoch 29/50
150/150 [==============================] - 231s 2s/step - loss: 0.2884
Epoch 30/50
150/150 [==============================] - 250s 2s/step - loss: 0.3691
Epoch 31/50
150/150 [==============================] - 246s 2s/step - loss: 0.2672
Epoch 32/50
150/150 [==============================] - 248s 2s/step - loss: 0.2765
Epoch 33/50
150/150 [==============================] - 247s 2s/step - loss: 0.2616
Epoch 34/50
150/150 [==============================] - 480s 3s/step - loss: 0.2567
Epoch 35/50
150/150 [==============================] - 204s 1s/step - loss: 0.1908
Epoch 36/50
150/150 [==============================] - 212s 1s/step - loss: 0.4389
Epoch 37/50
150/150 [==============================] - 195s 1s/step - loss: 0.2498
Epoch 38/50
150/150 [==============================] - 192s 1s/step - loss: 0.1942
Epoch 39/50
150/150 [==============================] - 191s 1s/step - loss: 0.1737
Epoch 40/50
150/150 [==============================] - 191s 1s/step - loss: 0.1947
Epoch 41/50
150/150 [==============================] - 192s 1s/step - loss: 0.1782
Epoch 42/50
150/150 [==============================] - 193s 1s/step - loss: 0.1515
Epoch 43/50
150/150 [==============================] - 192s 1s/step - loss: 0.1196
Epoch 44/50
150/150 [==============================] - 191s 1s/step - loss: 0.1325
Epoch 45/50
150/150 [==============================] - 191s 1s/step - loss: 0.1362
Epoch 46/50
150/150 [==============================] - 198s 1s/step - loss: 0.1459
Epoch 47/50
150/150 [==============================] - 204s 1s/step - loss: 0.1424
Epoch 48/50
150/150 [==============================] - 191s 1s/step - loss: 0.1122
Epoch 49/50
150/150 [==============================] - 191s 1s/step - loss: 0.1223
Epoch 50/50
150/150 [==============================] - 194s 1s/step - loss: 0.1044
Wall time: 2h 51min 49s
- 아래 보면 loss가 줄어드는 것처럼 보이나, 실제로는 loss가 최저점이 아닌 곳에서 training이 끝난 것을 볼 수 있습니다.
- 이 점을 예방할 수 있는 점도 추후 포스팅하겠습니다.
plt.plot(hist.history['loss'])
plt.title('Model loss')
plt.ylabel('Loss')
plt.xlabel('Epoch')
# plt.legend('Train', 'Test'], loc='upper left')
plt.show()
추후 다른 모델들과 비교하기 위해서 history를 저장해놓도록 하겠습니다.
hist_ = pd.DataFrame(hist.history["loss"]).reset_index()
hist_.rename(columns = {x : y for x, y in zip(hist_.columns, ["epoch", "loss"])}, inplace = True)
hist_.to_csv("model_metrics/first_classification_loss.csv", index= False)
hist_.head()
| epoch | loss | |
|---|---|---|
| 0 | 0 | 3.003965 |
| 1 | 1 | 2.145966 |
| 2 | 2 | 1.983974 |
| 3 | 3 | 1.793144 |
| 4 | 4 | 1.590744 |
predict_value = val_generator.next()
predict_value[1][0]
array([0., 0., 1., 0., 0., 0.], dtype=float32)
plt.imshow((predict_value[0][0] * 255).astype(int))
plt.show()
predict_value[0][0].shape
(124, 124, 3)
predict_proba = model.predict_generator(val_generator)
predict_proba = pd.DataFrame(predict_proba)
predict_proba.apply(sum, axis = 1)
0 1.0
1 1.0
2 1.0
3 1.0
4 1.0
...
1189 1.0
1190 1.0
1191 1.0
1192 1.0
1193 1.0
Length: 1194, dtype: float64
predict_proba["predict"] = predict_proba.idxmax(axis = "columns")
predict_proba.predict.value_counts().sort_index()
0 176
1 188
2 181
3 144
4 198
5 307
Name: predict, dtype: int64
예측 결과보기
- Training Loss가 낮은 것 같은데도 Confusion matrix나 정확도가 굉장히 낮게 나왔습니다.
- Overfitting이 나왔을 확률도 있을 것 같습니다. 오버피팅을 방지하는 방법에 대해서 추후 포스팅하겠습니다.
- Augmentation을 활용하여 데이터의 양과 다양성을 높이고 데이터를 전처리하는 방법에 대해서도 추후 포스팅하겠습니다.
from sklearn.metrics import confusion_matrix
confusion_matrix(predict_proba["predict"], val_y["Y"])
array([[26, 26, 24, 31, 32, 37],
[36, 31, 29, 31, 22, 39],
[37, 27, 23, 36, 25, 33],
[28, 23, 22, 20, 25, 26],
[28, 40, 29, 37, 32, 32],
[49, 65, 41, 59, 52, 41]], dtype=int64)
# 참담...
(predict_proba["predict"] == val_y["Y"]).mean()
0.14489112227805695
추후 포스팅(순서는 미정입니다.)
- Augmentation
- Loss function
- Optimizer
- Overfitting을 방지하기 위한 방법(Early stopping, BN, Dropout 등)
- Transfer learning
- Model 평가
포스팅을 마치며
- 포스팅해야할 것들이 어마어마하게 많습니다…. (분류가 끝나면 object detection도 해야할텐데…)
- 학습속도가…. gpu가 있긴한데… tensorflow와 cuda 버전을 맞추기가 어렵습니다. 다음부턴 colab을 활용해야겠습니다.
- ImageDataGenerator를 오랜만에 쓰고 랜덤하게 train/validation set이 나뉘는 것이 아니라는 것을 알고 다시 학습시키느라 고생했습니다..
code : https://github.com/Chanjun-kim/Chanjun-kim.github.io/blob/main/_ipynb/2021-06-08-CNN.ipynb
참고 자료 :
- https://www.tensorflow.org/api_docs/python/tf/keras/
- https://youngq.tistory.com/40
- https://subinium.github.io/introduction-to-normalization/
Feel free to share!