Art

J-GLOBAL ID：202002232294478998   Reference number：20A0249036

Efficient Learning for Distillation of DNN by Self Distillation

自己蒸留によるDNNの蒸留の効率化

Author (2)： TAKAGI JUMPEI ((株) セック) ,  HATTORI MOTONOBU (山梨大学 大学院総合研究部)
Material： 電気学会論文誌 C  (IEEJ Transactions on Electronics, Information and Systems)
Volume： 139  Issue： 12  Page： 1509-1516(J-STAGE)  Publication year： 2019 
JST Material Number： S0810A  ISSN： 0385-4221  Document type： Article
Article type： 原著論文  Country of issue： Japan (JPN)  Language： JAPANESE (JA)

Thesaurus term： *neural network(biomodel) ,  *information extraction ,  *efficiency ,  distillation ,  supervised learning ,  classification ,  time ,  accuracy
Semi thesaurus term： *deep neural network ,  *knowledge extracting ,  *extraction efficiency ,  image classification ,  learning time

Author keywords (6)： 知識蒸留 ,  深層学習 ,  画像分類 ,  knowledge distillation ,  deep learning ,  image classification

JST classification (2)： Artificial intelligence  (JE08000Z) ,  Pattern recognition  (JE07000S)

Reference (18)：

• (1) K. Simonyan and A. Zisserman: “Very deep convolutional networks for large-scale image recognition”, In ICLR (2015)
• (2) K. He, X. Zhang, S. Ren, and J. Sun: “Deep residual learning for image recognition”, In CVPR (2016)
• (3) G. Hinton, O. Vinyals, and J. Dean: “Distilling the knowledge in a neural network”, In NIPS 2014 Deep Learning Workshop (2014)
• (4) R. Anil, G. Pereyra, A. Passos, R. Ormandi, G. E. Dahl, and G. E. Hinton: “Large scale distributed neural network training through online distillation”, In ICLR (2018)
• (5) A. Romero, N. Ballas, S. E. Kahou, A. Chassang, C. Gatta, and Y. Bengio: “Fitnets: Hints for thin deep nets”, In ICLR (2015)

Terms in the title (3)： 自己 ,  蒸留 ,  効率化