CNNãããããããŒãã©ã€ãŒãŒã·ã§ã³ãšãã®ä»²éãã¡ãŸã§ãå³ã§è§£èª¬ïŒ
ãã£ãŒãã©ãŒãã³ã°ãæµè¡ãããã£ãããšãªã£ãåéã¯ç»åèªèãšèšã£ãŠãéèšã§ã¯ãªãã§ãããã
å®éã«ãã£ãŒãã©ãŒãã³ã°æµè¡ã®ç«ä»ã圹ãšãªã£ã2012幎ã®AlexNetã¯ç»ååé¡ã¿ã¹ã¯ã«ãã£ãŒãã©ãŒãã³ã°ãé©çšãããŠé©ç°çãªçµæãåºããŸãããAlexNetã®è«æã¯çŸæç¹(2019幎12æçŸåš)㧠被åŒçšæ° 52,655ãèªããã±ã¢ãè«æ ã«ãªã£ãŠããŸãããã®AlexNet以éã§ ç»åèªèåéã§äœ¿ãããŠããæè¡ãããCNN(Convolutional Neural Network) ãªã®ã§ãã(ãã¡ããæåŒ·ã®ç»åèªèã¢ãã«EfficientNet(æèã®è§£èª¬èšäº)ãCNNããŒã¹ã§ãã)
ãã®CNNã®å±€ãæ·±ãããŠããããšã§ç²ŸåºŠãåäžãããŠããã®ã§ããã
å±€ãæ·±ãããã°ããã»ã©CNNãéã«ããŸãåŠç¿ããŠãããªããªã£ãŠããŸãã®ã§ãã
ããã解決ããæè¡ããããããŒãã©ã€ãŒãŒã·ã§ã³(Batch Normalization) (以äžããããããŒã )ã§ãã(ã¡ãªã¿ã«ãã®è«æã®è¢«åŒçšæ°ã¯2019幎12æçŸåšã§14,761ã§ãã¡ãããã±ã¢ãè«æã)ãã®ãããããŒã ã¯ä»ã§ã¯åœããåã®ããã«äœ¿ãããŠããŸãããããããŒã ã®ä»²éã§ãã ã¬ã€ã€ãŒããŒãã©ã€ãŒãŒã·ã§ã³ãšããã®ã¯Transformer(æèã®è§£èª¬èšäº)ã§ã䜿ãããŠããããããããŒã ç³»ã¯ãã£ãŒãã©ãŒãã³ã°ã«ãšã£ãŠå¿
èŠäžå¯æ¬ ãªæè¡ ã«ãªã£ãŠããŸãã
æ¬çš¿ã§ã¯ CNNã®è§£èª¬ããå ¥ãããããããŒã ããã³ãã®ä»²éãã¡ã培åºè§£èª¬ ããŸããæ¬èšäºã¯ä»¥äžã®æ§æã§ãã
- CNNã£ãŠããããã©ãåããŠããã®ïŒ
- ãããããŒãã©ã€ãŒãŒã·ã§ã³ãšã¯ïŒ
- ãããããŒãã©ã€ãŒãŒã·ã§ã³ã®ä»²éãã¡
- (ãªãã·ã§ã³)ãªãã§ãããããŒãã©ã€ãŒãŒã·ã§ã³ã£ãŠåŒ·ãã®ïŒ
- ãŸãšã
- åè
èªãã§å°ãã§ãäœãåŠã¹ããšæããã ããããã ã ãã³ã¡ã³ãã ããããããšããããã®å±ã¿ã«ãªããŸãïŒãããããé¡ãããŸãïŒ
(ééããªã©ãããããŸãããããææãããããé¡ãããŸãã)
1. CNNã£ãŠããããã©ãåããŠããã®ïŒ
ãŸãCNNãã©ãåããŠãããã«ã€ããŠè§£èª¬ããŸãã
ããããããã®ããã以äžã§ã¯å
šãŠå
¥åãçœé»(ã°ã¬ãŒã¹ã±ãŒã«)ã®ç»åã ãšããŸãã
ã«ã©ãŒ(RGB)ã®å Žåã¯å
¥åç»åãã«ã©ãŒã«å€ããã ãã§ä»ã¯äœãå€ãããªãã®ã§ã
ã°ã¬ãŒã¹ã±ãŒã«ããçè§£ã§ããã°å€§äžå€«ã§ãã
1.1 å ¥åç»å1æããã£ã«ã¿ãŒ1æã®CNN
ãŸãã¯å
¥åç»åã1æããã£ã«ã¿ãŒ1æã®æãèããŸãããã
å·Šããå³ãžãšæµããŠããæãã§ãã

å
¥åç»åã«ãã£ã«ã¿ãŒ(ã«ãŒãã«ãšãèšãããã)ããããããã£ã«ã¿ãŒåŸã®ç»åã«å¯ŸããŠæŽ»æ§å颿°ãé©çšããããšã§åºåã®ç»åãåŸãããŸããããã§ç³ã¿èŸŒã¿å±€(Convolutionå±€)ãåºæ¥äžãããŸãã
ãã£ã«ã¿ãŒããããããã»ã¹ãèŠãŠã¿ãŸãã
å
¥åç»åã®3x3ã®éšåããã£ã«ã¿ãŒã«ãã£ãŠ1ãã¹ã«ãªããŸãããã® ãã£ã«ã¿ãŒããããåŠçããããç³ã¿èŸŒã¿ããšèšãããåŠç ã§ãã
ããããŠ åºæ¥äžãã£ãç»åã¯äžåãå°ãããªã£ãŠããã®ã§ãããã£ã³ã°(padding)ãšèšãããåŠçãæœã ããšã§å
ã®ç»åãµã€ãºãšåãã«ããŸããããã£ã³ã°ã¯åºåãããç»åã®åšãã«ä»»æã®æ°åãåããŠããåŠçã§ã0ã§åããããšãå€ãã§ãã(zero padding)
ãã®ãã£ã«ã¿ãŒã«ãã£ãŠåŸãããç»åã¯ãäŸãã°ããå ã®ç»åããã«ãã®ææãã¯ã£ããããŠããç»åãã ã£ããããå ã®ç»åããå¢çç·ãã¯ã£ããããŠããç»åãã®ããã«ãç»åã®åºæ¬çãªç¹åŸŽãæµ®ãåºãŠããç»åãšãªã£ãŠããŸãããã®ãã£ã«ã¿ãŒãã©ã決ããŠããã®ãããšããçåãæµ®ãã¶ãšæããŸãããã® ãã£ã«ã¿ãŒã¯äººéã§ãã¡ãã¡æ±ºããã®ã¯é£ããããããã¥ãŒã©ã«ãããã¯ãŒã¯ã«èšç·ŽããŒã¿ã«åºã¥ããŠæ±ºããŠããã ã®ã§ãã
ã¡ãªã¿ã«RGBãæã€ã«ã©ãŒã®å Žå(=ãã£ãã«æ°3)ã¯ããã£ã«ã¿ãŒãR,G,Bãã£ãã«ããããã«é©çšããããã®åèšå€ãåºåãããããçµæã¯ä»¥äžå³ã®ããã«ã°ã¬ãŒã¹ã±ãŒã«ãšå€ããã1ã€ã®ãã£ãã«ã«ãªã£ãŠããŸããã€ãŸããå ¥åãã£ãã«æ°ããããã§ãã£ãŠããã£ã«ã¿ãŒïŒã€ã«ãã£ãŠåºåãããç»åã¯ïŒã€ãšããããšã§ãããã®ããã«ã°ã¬ãŒã§ãã«ã©ãŒã§ãåºåã¯å€ãããªãã®ã§ãããããã¯ãŸãå ¥åãã°ã¬ãŒã¹ã±ãŒã«ã«æ»ããŠèããŸãã
1.2 å ¥åç»å1æããã£ã«ã¿ãŒè€æ°æã®CNN
éåžžããã£ã«ã¿ãŒã¯1ã€ã ãã§ã¯ãªãè€æ°å䜿ããŸãã(人éã®ç»åèªèãè¶ ããResNetã§ã¯64åã128åã512åã®ãã£ã«ã¿ãŒã1åã®ç³ã¿èŸŒã¿å±€ã§äœ¿ããŸãã)ãšããããšã§ãããã§ã¯ãã£ã«ã¿ãŒã4ã€äœ¿ã£ãæã«ã©ããªããèŠãŸãããã
1æã®ç»åã«4ã€ã®ãã£ã«ã¿ãŒããããããããã®ã§ãã®çµæãåŸãããŸãã
ã€ãŸãããã® 4ã€ã®ãã£ã«ã¿ãŒã«ãã£ãŠ1æã®ç»åã4ã€ã®ãã£ãã«(channel)ãæã€ ããšã«ãªããŸãããã£ã«ã¿ãŒ1ã€ã«å¯Ÿå¿ããŠç»åã®ãã£ãã«(æ·±ãããã£ã³ãã«ãšãèšããã)ã1ã€åºæ¥äžãããŸãã
1.3 å ¥åç»åè€æ°æããã£ã«ã¿ãŒ1æã®CNN
ãŸããå ¥åç»åã«ã€ããŠã§ãããããéåžž1æã ãã§ãªããè€æ°æãäžæ°ã«åŠçããŸãããã® è€æ°æã®ã«ã¿ããªã®ããšãããããã ãšåŒã³ãéåžžããããµã€ãº(1ã«ã¿ããªã®ææ°)ã¯16æã32æã64æãªã©ã§ãããã·ã³ã®æ§èœãäžããã°äžããã»ã©ãã®ãããããã®å€§ããã¯å€§ããããããšãã§ããŸãããããããã«ãã£ã«ã¿ãŒã1ã€ã ããããŠã¿ãŸãã
åç»åã«ãã£ã«ã¿ãŒããããŠãåç»åã«å¯Ÿå¿ããåºåãåŸãŸãã
ãªããé£ããããšã¯ãããŸãããã
1.4 å ¥åç»åè€æ°æããã£ã«ã¿ãŒè€æ°æã®CNN
ãšããããšã§ãè€æ°ãã£ã«ã¿ãŒããããå ŽåãèããŸãããã
ãã¡ããä»ãŸã§éãã§é£ããã¯ãããŸããã
ããããããã®ããã«åãã£ã«ã¿ãŒãšããã«å¯Ÿå¿ããåºåãèµ€ç·ã§åŒã£åŒµã£ãŠã¿ãŸããã
ããããã®åºåç»åããã£ã«ã¿ãŒã®ææ°ã«å¿ãã4ãã£ãã«ã«ãªã£ãŠããã ãã§ãã
ãã®åºåãæ¬¡ã®å±€ã®å
¥åãšãªããŸãã
2. ãããããŒãã©ã€ãŒãŒã·ã§ã³ãšã¯ïŒ
åé ã§ãç³ã¿èŸŒã¿å±€ãäœæ®µãéããŠæ·±ãããããšã§ç²ŸåºŠãäžããŠããããšãè¿°ã¹ãŸãããå®éã8å±€ãããªãã£ãAlexNet(2012)ã«å¯ŸããŠ2014幎ã«ã¯19å±€ã®VGG-19ãçºè¡šããã2015幎ã«çºè¡šãããResNetã¯ãªããš152å±€ããã®ããã«å±€ãæ·±ããªããšããŸãåŠç¿ãã§ããªããªããŸããããã§å°å
¥ãããã®ããããããŒãã©ã€ãŒãŒã·ã§ã³ãåŠçå
容ã¯ã1ãããããå
ã®å
šããŒã¿ã®åäžãã£ãã«ãå¹³å0,忣1ã«æ£èŠåãã ããšãããããããšããã®ã¯1åã«åŠçããè€æ°ç»åã®ã«ã¿ã㪠ã§ãããã
æ£èŠåãšããã®ã¯ ããŒã¿ããå
šäœã®å¹³åå€ãåŒããŠãå
šäœã®æšæºåå·®ã§å²ãç®ããããš ã§ãã
ãããããŒã ãé©çšãããå(玫è²)ããåãã£ãã«ããšã«ãããããŒã ãé©çšãããŠããçµéã衚ããŠããŸããèµ€è²ã«ãªã£ãŠãããšããã§ããããå¹³å忣ãåºããŠæ£èŠåããŸã(ç·è²ãé©çšåŸ)ã
[泚æ]
ãæ£èŠåãããâ ãæ£èŠååžã«ãããã§ãã®ã§ããæ°ãã€ããã ããã
æ£èŠåãšããã®ã¯åçŽã«ããŒã¿ã®ç·åœ¢å€æãªã®ã§ãå
ã®ããŒã¿ãåŸãååžãå€ããããã§ã¯ãªãã®ã§ã泚æãã ããã
ããã§ã¯ãå®éã«åè«æã§æå±ããããããããŒã ã®æé ãèŠãŠã¿ãŸãããã

Ioffe, S. et al. "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift" (2015)
3è¡ç®ãŸã§ã¯åçŽã«æ£èŠåããŠããã ãã§ãã
1è¡ç®ã¯å¹³åã2è¡ç®ã¯åæ£ãåãã3è¡ç®ã§å¹³å0忣1ã«æ£èŠåããŠããŸããããã® $\epsilon$ ã¯åæ£ã0ã®æã«ãŒãé€ç®ããªãããã«èšããããŠããéåžžã«å°ããªå€(=1e-3ãšã)ã§ãã
ãã ãã4è¡ç®ã§æ£èŠåããå€ã« $\gamma$ ããã㊠$\beta$ ãè¶³ããŠããŸãã
ããã¯ã©ãããæå³ãªã®ã§ããããã
å¹³å0忣1ã®ããŒã¿ãã¡ã«å¯Ÿã㊠$\gamma$ ããã㊠$\beta$ ãè¶³ããŠããã®ã§ã ãããããã®ããŒã¿(ã®åãã£ãã«)ã¯å¹³å $\beta$ ã忣 $\gamma$ ãæ°ããæã€ããšã«ãªããŸãããã® $\beta$ ãš $\gamma$ ã¯ããŒã¿ã§åŠç¿ããããã©ã¡ãŒã¿ã§ãããããã®ãã©ã¡ãŒã¿ãèšããããšã§ãããããŒã ã§å¿
ãå¹³å0忣1ã«ããã®ã§ã¯ãªããåŠç¿ã«ãã£ãå¹³åãšåæ£ã«åŸãããã«èªç±åºŠãããããŠãã ã®ã§ãã
ãã®æé ãå³ã«è¡šãããã®ãçšæããã®ã§ãåç
§ãã ããã
ãããããå
ã®äžçªæåã®ãã£ãã«ã«ãããããŒã ãé©çšãããŠããŸãã
ã¹ããŒã¹ã®éœåäžãç¹ã«äžçªäžã®ç»åã«ããããããããŒã é©çšã«ãã©ãŒã«ã¹ããŠããŸãã
3. ãããããŒãã©ã€ãŒãŒã·ã§ã³ã®ä»²éãã¡
3.1 ãããããŒã ã®åŒ±ç¹
ãããããŒã ãšããã®ã¯ããŸãåœããåã®ããã«äœ¿ãããŠãããåªããçµæãåºããŸãã£ãŠããŸãããã ããã©ããªè¡ã«ã匱ç¹ãšãªã穎ã¯å¿ ãããããšæŒ«ç»NARUTOã§ãã¡ã¯ã€ã¿ããæããŠãããããã«ãããããŒã ã«ã匱ç¹ããããŸããããã¯ä»¥äžã®2ã€ã§ãã
-
ããããµã€ãºãå°ãããšäœ¿ããªã
- äŸãã°æ¥µç«¯ãªè©±ããããµã€ãºã1ã ã£ããããã¡ããæ£èŠåãªããŠç¡çã
- ãšããžã³ã³ãã¥ãŒã¿ãªã©ãªãœãŒã¹ãéãããŠãããšãã¯ããããµã€ãºå€§ããã§ããªãããåé¡ã«ãªãã
-
RNNãªã©ã®æç³»åããŒã¿ã«äœ¿ããªã
- RNNãæç³»åããŒã¿ã«å¯ŸããŠåºã䜿ãããŠããã®ãèãããšèŽåœçïŒ
3.2 ãããããŒã ã®ä»²éãã¡
äžè¿°ããåé¡(ç¹ã«1ã€ç®)ã解決ããããã«åºãŠãã代衚çãªææ³ã3ã€ãããŸãã
ãã¡ããäžå³ã«è¡šããã®ã§ç°¡åã«èª¬æããŸãã
äžçªå·Šã«ããããããå·Šãã2çªç®ã«æ¬çš¿ã®äž»äººå
¬ãããããŒãã©ã€ãŒãŒã·ã§ã³ãããŸãããããŠããããã**ã¬ã€ã€ãŒããŒãã©ã€ãŒãŒã·ã§ã³ãã€ã³ã¹ã¿ã³ã¹ããŒãã©ã€ãŒãŒã·ã§ã³ãã°ã«ãŒãããŒãã©ã€ãŒãŒã·ã§ã³** ãšç¶ããŸãããããããããããŒã ãšåŠçæ¹æ³ã¯åãã§ã1åºŠã«æ£èŠåããç¯å²ãç°ãªãã ãã§ãã(ãããããŒã ã¯ããããããå
ã®åäžãã£ãã«ããšã«æ£èŠåã)
ããã§ãããããå
ã®ããããã®ãããã¯ã¯ä»ãŸã§éãç»åããŒã¿ã衚ããŠãããšããŸãã
ãã®äŸã§ã¯ããããµã€ãº3ã§ããŒã¿ã®ãã£ãã«æ°ã4ã§ããèµ€è²ã«ãªã£ãŠããéšåãããŒãã©ã€ãŒãŒã·ã§ã³ãé©çšããéã®å¹³åãšåæ£ãèšç®ããç®æã衚ããŠããŸãã
ãããããããããŒã ãæã€ãããããµã€ãºã«äžé(éåžž16以äž)ãèšããŠããŸããšããæ¬ ç¹ã«å¯ŸããŠææ¡ãããŠãããæŽŸçææ³ã¯ãããããããæ¹åã§ã¯ãªãããã£ãã«æ¹åã«å¯ŸããŠæ£èŠåãããŠããããšãããããŸãããããããŒã ã®ããããµã€ãºãžã®äžéãå
æããããã«ãŸãææ¡ããããã®ã ã¬ã€ã€ãŒããŒã ãš ã€ã³ã¹ã¿ã³ã¹ããŒã ã§ããã¬ã€ã€ãŒããŒã ã¯1ã€ã®ããŒã¿ã®å
šãã£ãã«ã«å¯ŸããŠæ£èŠå ãè¡ããã€ã³ã¹ã¿ã³ã¹ããŒã ã¯1ã€ã®ããŒã¿ã®1ã€ã®ãã£ãã«ã«å¯ŸããŠæ£èŠå ãè¡ããŸãããããŠãããã®äžéã®ãããªãã®ãšããŠçºè¡šããããã®ã ã°ã«ãŒãããŒã ã§ããã㯠1ã€ã®ããŒã¿ã®ä»»æã®ãã£ãã«æ°ã«å¯ŸããŠæ£èŠå ãããã®ã§ãããããŸã§èªãã§ããäžåºŠäžå³ãèŠããšãããããããšæããŸãã
äžå³ã«ãããŠã°ã«ãŒãããŒã ã§1ã°ã«ãŒãã®ãµã€ãºã4ã«èšå®ããã°ãã¬ã€ã€ãŒããŒã ãšåãã§éã«1ã«èšå®ããã°ã€ã³ã¹ã¿ã³ã¹ããŒã ãšåãããšã«ãªããŸãã
3.3 äžäœã©ãã䜿ãã°ãããã ã
çãã¯ç°¡åã§ãã
ãããããŒã äžæ
å®ã¯ãããã®æ¹æ³ããããããŒã ã®æ§èœãè¶ããããšãã§ããŠããªãã®ã§ããã
ãããŸã§ãããããµã€ãºãå°ããæã«ãããããããŒã ãããè¯ã广ãçºæ®ããªãããšããããšã§ãããã ãããããå®çšã§äœ¿ãããŠããäŸããããŸãã®ã§ãã¡ãã玹ä»ããŠãã®ç¯ãç· ããããšæããŸãã
- ã¬ã€ã€ãŒããŒã : RNNç³»ãTransformer
- ã€ã³ã¹ã¿ã³ã¹ããŒã : GANç³»(StyleGAN)
- ã°ã«ãŒãããŒã : ç»åèªèç³»(ã«è¯ããããã(GNã®åè«æ))
4.(ãªãã·ã§ã³)ãªãã§ãããããŒãã©ã€ãŒãŒã·ã§ã³ã£ãŠåŒ·ãã®ïŒ
ãã¥ãŒã©ã«ãããã«ãããåŸé
éäžæ³ãããã¯ãããã²ãŒã·ã§ã³ãªã©ãããçšåºŠçè§£ããŠããããšãåæã«æžããŸããã®ã§ãããããããªããã°èªã¿é£ã°ããŠããã£ãŠæ§ããŸããã
ããã§èšã£ãŠããããšã¯ç°¡åã«ãŸãšãããšã以äžã®2ç¹ã ãã§ãã
- ãããããŒã ãããŸãããçç±ã¯ãŸã ããã£ãŠããªãã
- ã§ãã©ããããããããŒã ã®ãããã§ãã¥ãŒã©ã«ãããã«ãšã£ãŠã¿ã¹ã¯ãç°¡åã«ãªã£ãŠããã®ã ããã
ããããã¯äžèš2ç¹ã«ã€ããŠããå°ãã ãè£è¶³ããŸãã
ãããããŒã ãè¯ãçç±ãšããŠå éšå ±å€éã·ãã(= ICS, Internal Covariance Shift)ã®è§£æ¶ãåè«æã§ã¯èšãããŠããŸããICSãšã¯åçŽã« ãã¥ãŒã©ã«ãããã®å±€ãéããæ¯ã«ãã®å±€ã®ã¢ãŠããããã®ååžãå€ãããããšããããšããã¥ãŒã©ã«ãããã®äžãéãããŒã¿ã®ååžãé »ç¹ã«å€ãã£ãŠããŸã£ãŠã¯åŠç¿ãã¥ããããããšããåé¡ã(ã ãšçè§£ããŠãããééã£ãŠããã°ã³ã¡ã³ããé¡ãããŸãã)ãããããŒã ã¯ãã®ICSã解決ããŠãããã粟床åäžã«å¯äžããŠããããšãã䞻匵ãåè«æã«ãã£ãŠãããŠããããã§ãããã ãå®éã¯ãã®äž»åŒµã¯ééã£ãŠãããšããããšãããã£ãŠããŸããã€ãŸãããããããŒã ã«ãã£ãŠICSã¯è§£æ±ºãããªã ã®ã§ãã
ã§ã¯ãªããããããŒã ã匷ãã®ãããã®çç±ã¯å®ã¯ãŸã ããã£ãŠããŸããã
ãã ãïŒã€ã®å¯èœæ§ãšããŠããããããŒã ã¯ãã§ãã¯ãã€ã³ããšããŠåããŠãã ãšãã説æãIan Goodfellowæ°(GANäœã£ã人ïŒ)ã«ãã£ãŠãããŠããŸãã
ãŸããæŽ»æ§å颿°ãæããªãåçŽãªãã¥ãŒã©ã«ããããèããŸãã
åäžžãå
šçµåå±€ã§ãi åç®ã®ã€ãã¬ãŒã·ã§ã³ãš(i+1) åç®ã®ã€ãã¬ãŒã·ã§ã³ã衚ããŠããŸãã
ã€ãã¬ãŒã·ã§ã³ã®å®çŸ©ã¯ãã£ãŒããã©ã¯ãŒããšããã¯ãããã²ãŒã·ã§ã³ã®1ã»ããã§ãã
ããã§må±€ã«æ³šç®ãããš(i+1) åç®ã®ã€ãã¬ãŒã·ã§ã³ã®éã¿ $w^{i+1}_m$ 㯠å(i åç®)ã®ã€ãã¬ãŒã·ã§ã³ã®Lossã䜿ã£ãп޿°ãããŠãããããi åç®ã«ãããmå±€ãããåå±€ã®éã¿ãã¡(ã€ãŸãã $w_l^{i}, w_k^{i}, ...$ )ã«å¯ŸããŠæé©åãããŠããŸã ããã ãã (i+1) åç®ã®æã®ãã£ãŒããã©ã¯ãŒã㯠$w_l^{i}, w_k^{i}, ...$ ã§ã¯ãªããæŽæ°ããã $w_l^{i+1}, w_k^{i+1}, ...$ ã䜿ã£ãŠäºæž¬å€ãåºããŸããã€ãŸããããéã¿ãæŽæ°ããæã«ã¯ãæ¬æ¥ãã®åã®å±€ã®éã¿ãã¡ãã©ãæŽæ°ãããã®ããšããããšãèæ ®ããªããã°æé©ãªå€ã¯æ±ããããªãã®ã§ãããäžé埮åã ããçšããŠããåŸé éäžæ³ã§ã¯ãã®ãããªè€éãªé¢ä¿æ§ãŸã§èæ ®ã§ããŸããããã®è€éãªé¢ä¿æ§ã®æäžäž»ããããããŒã ã§ãã
ãããããŒã ããããŠã¿ãŸãããã
åã³må±€ã«æ³šç®ããŸãããããããŒã ã«ãããmå±€ãžã®å
¥åå€ã¯æ£èŠåãããŠããŸããm局以åã®å±€ã«ããå€ãã©ãã§ããããšäžæŠãããããŒã ã§èª¿æŽãããŠmå±€ãžãšå
¥åãããã®ã§ããããã¯ã€ãŸããmå±€ã®éã¿ã¯ãã¯ãm局以åã®å
šãŠã®éã¿ãã¡ãèããã®ã§ã¯ãªããçŽåã®ãããããŒã ã®ãã©ã¡ãŒã¿ $\gamma, \beta$ ã ããèããã°è¯ã ã®ã§ãããã®ããšã«ãã倧éã®éã¿ãã¡ã«ããè€éãªé¢ä¿æ§ãèæ
®ããªããŠããããããã¥ãŒã©ã«ããããåŠç¿ãããããªã粟床ãé床ããããããšããããšã§ãããããIan Goodfellowæ°ã«ãããããããŒã ãããŸãããçç±ã§ãã
5.ãŸãšããšææ
ä»ãŸã§ãªããšãªããããããŒã ã䜿ã£ãŠããã®ã§ãããä»å調ã¹ããããã§å¹æã仲éãã¡ãããããŸããããããããŒã ã®ããšãçè§£ããããã«ãéã¿ã®åæåãæŽ»æ§å颿°ãæé©å颿°ã®æçŸ©ãªã©ãä»ãŸã§ããæ·±ãçè§£ããããšãã§ããã®ã§ãã¡ããéèŠãããã°ã¢ãŠããããããŠãããããšæã£ãŠããŸãã®ã§ãããããé¡ãããŸãã
èªãã§å°ãã§ãäœãåŠã¹ããšæããã ããããã ã ãã³ã¡ã³ãã ããããããšããããã®å±ã¿ã«ãªããŸãïŒãããããé¡ãããŸãïŒ
(ééããªã©ãããããŸãããããææãããããé¡ãããŸãã)
6.åè
-
Batch Normalizationãšãã®æŽŸçã®æŽç
Batch Normalizationã®æŠèŠãšãã®æŽŸçãã¡ãç°¡åã«çè§£ã§ãããã ãã¶åèã«ããŠããŸãã -
Normalizing Activations in a Network (C2W3L04)
(è±èª) Andrew Ngæ°ã«ããããããããBN解説åç»ãæ°åŒãšãšãã«è§£èª¬ããŠãããŠããã -
Intro to Optimization in Deep Learning: Busting the Myth About Batch Normalization
(è±èª) BNããªãå¿ èŠã§ãªãæå¹ãªã®ããšããã®ãããããããæããŠãããŠããã -
An Overview of Normalization Methods in Deep Learning
(è±èª) BNã®æŽŸçãã¡ãæããŠãããŠããã -
Understanding Batch Normalization
(è«æ) ãããããŒã ãå éšå ±å€éã·ããã解決ããŠããããã§ã¯ãªããšããæšãæžãããŠããã -
An Intuitive Explanation of Why Batch Normalization Really Works (Normalization in Deep Learning Part 1)
(è±èª) ãããããŒã ããã¥ãŒã©ã«ãããã®Higher-order interactionãç·©åãããŠããããšãæžãããŠããã



