13.6.3.1 Knowledge Distillation

Chapter Contents (Back)
Knowledge Distillation. Knowledge. Distillation. Knowledge-Based Vision.
See also Transfer Learning from Other Tasks, Other Classes.

Chen, G.Z.[Guan-Zhou], Zhang, X.D.[Xiao-Dong], Tan, X.L.[Xiao-Liang], Cheng, Y.F.[Yu-Feng], Dai, F.[Fan], Zhu, K.[Kun], Gong, Y.F.[Yuan-Fu], Wang, Q.[Qing],
Training Small Networks for Scene Classification of Remote Sensing Images via Knowledge Distillation,
RS(10), No. 5, 2018, pp. xx-yy.
DOI Link 1806
BibRef

Wu, X.[Xiang], He, R.[Ran], Hu, Y.[Yibo], Sun, Z.N.[Zhe-Nan],
Learning an Evolutionary Embedding via Massive Knowledge Distillation,
IJCV(128), No. 8-9, September 2020, pp. 2089-2106.
Springer DOI 2008
transferring knowledge from a large powerful teacher network to a small compact student one. BibRef

Zaras, A.[Adamantios], Passalis, N.[Nikolaos], Tefas, A.[Anastasios],
Improving knowledge distillation using unified ensembles of specialized teachers,
PRL(146), 2021, pp. 215-221.
Elsevier DOI 2105
68T99, Knowledge distillation, Knowledge transfer, Specialized teachers, Unified ensemble, Unified specialized teachers ensemble BibRef

Bae, J.H.[Ji-Hoon], Yeo, D.[Doyeob], Yim, J.[Junho], Kim, N.S.[Nae-Soo], Pyo, C.S.[Cheol-Sig], Kim, J.[Junmo],
Densely Distilled Flow-Based Knowledge Transfer in Teacher-Student Framework for Image Classification,
IP(29), 2020, pp. 5698-5710.
IEEE DOI 2005
BibRef
Earlier: A2, A1, A5, A3, A4, A6:
Sequential Knowledge Transfer in Teacher-Student Framework Using Densely Distilled Flow-Based Information,
ICIP18(674-678)
IEEE DOI 1809
Knowledge transfer, Training, Computational modeling, Data mining, Optimization, Image classification, Computer architecture, residual network. Training, Data mining, Optimization, Image classification, Knowledge transfer, Computational modeling, Reliability, BibRef

Mazumder, P.[Pratik], Singh, P.[Pravendra], Namboodiri, V.P.[Vinay P.],
GIFSL: Grafting based improved few-shot learning,
IVC(104), 2020, pp. 104006.
Elsevier DOI 2012
Few-shot learning, Grafting, Self-supervision, Distillation, Deep learning, Object recognition BibRef

Li, X.W.[Xue-Wei], Li, S.Y.[Song-Yuan], Omar, B.[Bourahla], Wu, F.[Fei], Li, X.[Xi],
ResKD: Residual-Guided Knowledge Distillation,
IP(30), 2021, pp. 4735-4746.
IEEE DOI 2105
BibRef

Nguyen-Meidine, L.T.[Le Thanh], Belal, A.[Atif], Kiran, M.[Madhu], Dolz, J.[Jose], Blais-Morin, L.A.[Louis-Antoine], Granger, E.[Eric],
Knowledge distillation methods for efficient unsupervised adaptation across multiple domains,
IVC(108), 2021, pp. 104096.
Elsevier DOI 2104
BibRef
And:
Unsupervised Multi-Target Domain Adaptation Through Knowledge Distillation,
WACV21(1338-1346)
IEEE DOI 2106
Deep learning, Convolutional NNs, Knowledge distillation, Unsupervised domain adaptation, CNN acceleration and compression. Adaptation models, Computational modeling, Benchmark testing, Real-time systems BibRef

Remigereau, F.[Félix], Mekhazni, D.[Djebril], Abdoli, S.[Sajjad], Nguyen-Meidine, L.T.[Le Thanh], Cruz, R.M.O.[Rafael M. O.], Granger, E.[Eric],
Knowledge Distillation for Multi-Target Domain Adaptation in Real-Time Person Re-Identification,
ICIP22(3853-3557)
IEEE DOI 2211
Training, Adaptation models, Scalability, Streaming media, Video surveillance, IEEE Standards, Video Surveillance, Knowledge Distillation BibRef

Zhang, H.R.[Hao-Ran], Hu, Z.Z.[Zhen-Zhen], Qin, W.[Wei], Xu, M.L.[Ming-Liang], Wang, M.[Meng],
Adversarial co-distillation learning for image recognition,
PR(111), 2021, pp. 107659.
Elsevier DOI 2012
Knowledge distillation, Data augmentation, Generative adversarial nets, Divergent examples, Image classification BibRef

Gou, J.P.[Jian-Ping], Yu, B.S.[Bao-Sheng], Maybank, S.J.[Stephen J.], Tao, D.C.[Da-Cheng],
Knowledge Distillation: A Survey,
IJCV(129), No. 6, June 2021, pp. 1789-1819.
Springer DOI 2106
Survey, Knowledge Distillation. BibRef

Deng, Y.J.[Yong-Jian], Chen, H.[Hao], Chen, H.Y.[Hui-Ying], Li, Y.F.[You-Fu],
Learning From Images: A Distillation Learning Framework for Event Cameras,
IP(30), 2021, pp. 4919-4931.
IEEE DOI 2106
Task analysis, Feature extraction, Cameras, Data models, Streaming media, Trajectory, Power demand, Event-based vision, optical flow prediction BibRef

Liu, Y.[Yang], Wang, K.[Keze], Li, G.B.[Guan-Bin], Lin, L.[Liang],
Semantics-Aware Adaptive Knowledge Distillation for Sensor-to-Vision Action Recognition,
IP(30), 2021, pp. 5573-5588.
IEEE DOI 2106
Videos, Knowledge engineering, Wearable sensors, Adaptation models, Sensors, Semantics, Image synthesis, Action recognition, transfer learning BibRef

Feng, Z.X.[Zhan-Xiang], Lai, J.H.[Jian-Huang], Xie, X.H.[Xiao-Hua],
Resolution-Aware Knowledge Distillation for Efficient Inference,
IP(30), 2021, pp. 6985-6996.
IEEE DOI 2108
Knowledge engineering, Feature extraction, Image resolution, Computational modeling, Computational complexity, Image coding, adversarial learning BibRef

Liu, Y.Y.[Yu-Yang], Cong, Y.[Yang], Sun, G.[Gan], Zhang, T.[Tao], Dong, J.H.[Jia-Hua], Liu, H.S.[Hong-Sen],
L3DOC: Lifelong 3D Object Classification,
IP(30), 2021, pp. 7486-7498.
IEEE DOI 2109
Task analysis, Solid modeling, Data models, Knowledge engineering, Shape, Robots, task-relevant knowledge distillation BibRef

Bhardwaj, A.[Ayush], Pimpale, S.[Sakshee], Kumar, S.[Saurabh], Banerjee, B.[Biplab],
Empowering Knowledge Distillation via Open Set Recognition for Robust 3D Point Cloud Classification,
PRL(151), 2021, pp. 172-179.
Elsevier DOI 2110
Knowledge Distillation, Open Set Recognition, 3D Object Recognition, Point Cloud Classification BibRef

Shao, B.[Baitan], Chen, Y.[Ying],
Multi-granularity for knowledge distillation,
IVC(115), 2021, pp. 104286.
Elsevier DOI 2110
Knowledge distillation, Model compression, Multi-granularity distillation mechanism, Stable excitation scheme BibRef

Zhang, L.[Libo], Du, D.W.[Da-Wei], Li, C.C.[Cong-Cong], Wu, Y.J.[Yan-Jun], Luo, T.J.[Tie-Jian],
Iterative Knowledge Distillation for Automatic Check-Out,
MultMed(23), 2021, pp. 4158-4170.
IEEE DOI 2112
Testing, Training, Adaptation models, Reliability, Feature extraction, Training data, Task analysis, iterative knowledge distillation BibRef

Qin, D.[Dian], Bu, J.J.[Jia-Jun], Liu, Z.[Zhe], Shen, X.[Xin], Zhou, S.[Sheng], Gu, J.J.[Jing-Jun], Wang, Z.H.[Zhi-Hua], Wu, L.[Lei], Dai, H.F.[Hui-Fen],
Efficient Medical Image Segmentation Based on Knowledge Distillation,
MedImg(40), No. 12, December 2021, pp. 3820-3831.
IEEE DOI 2112
Image segmentation, Biomedical imaging, Semantics, Knowledge engineering, Feature extraction, Tumors, transfer learning BibRef

Tian, L.[Ling], Wang, Z.C.[Zhi-Chao], He, B.[Bokun], He, C.[Chu], Wang, D.W.[Ding-Wen], Li, D.[Deshi],
Knowledge Distillation of Grassmann Manifold Network for Remote Sensing Scene Classification,
RS(13), No. 22, 2021, pp. xx-yy.
DOI Link 2112
BibRef

Yue, J.[Jun], Fang, L.Y.[Le-Yuan], Rahmani, H.[Hossein], Ghamisi, P.[Pedram],
Self-Supervised Learning With Adaptive Distillation for Hyperspectral Image Classification,
GeoRS(60), 2022, pp. 1-13.
IEEE DOI 2112
Feature extraction, Training, Adaptive systems, Mirrors, Knowledge engineering, Hyperspectral imaging, Spectral analysis, spatial-spectral feature extraction BibRef

Chen, J.Z.[Jing-Zhou], Wang, S.H.[Shi-Hao], Chen, L.[Ling], Cai, H.B.[Hai-Bin], Qian, Y.T.[Yun-Tao],
Incremental Detection of Remote Sensing Objects With Feature Pyramid and Knowledge Distillation,
GeoRS(60), 2022, pp. 1-13.
IEEE DOI 2112
Feature extraction, Remote sensing, Training, Object detection, Adaptation models, Proposals, Detectors, Deep learning, remote sensing BibRef

Chen, H.Y.[Hong-Yuan], Pei, Y.T.[Yan-Ting], Zhao, H.W.[Hong-Wei], Huang, Y.P.[Ya-Ping],
Super-resolution guided knowledge distillation for low-resolution image classification,
PRL(155), 2022, pp. 62-68.
Elsevier DOI 2203
Low-resolution image classification, Super-resolution, Knowledge distillation BibRef

Wang, S.L.[Shu-Ling], Hu, M.[Mu], Li, B.[Bin], Gong, X.J.[Xiao-Jin],
Self-Paced Knowledge Distillation for Real-Time Image Guided Depth Completion,
SPLetters(29), No. 2022, pp. 867-871.
IEEE DOI 2204
Knowledge engineering, Predictive models, Training, Task analysis, Real-time systems, Color, Loss measurement, self-paced learning BibRef

Zhang, K.[Kangkai], Zhang, C.H.[Chun-Hui], Li, S.[Shikun], Zeng, D.[Dan], Ge, S.M.[Shi-Ming],
Student Network Learning via Evolutionary Knowledge Distillation,
CirSysVideo(32), No. 4, April 2022, pp. 2251-2263.
IEEE DOI 2204
Training, Knowledge representation, Knowledge transfer, Predictive models, Germanium, Data models, Data mining, deep learning BibRef

Ge, S.M.[Shi-Ming], Liu, B.C.[Bo-Chao], Wang, P.J.[Peng-Ju], Li, Y.[Yong], Zeng, D.[Dan],
Learning Privacy-Preserving Student Networks via Discriminative-Generative Distillation,
IP(32), 2023, pp. 116-127.
IEEE DOI 2301
Data models, Data privacy, Synthetic data, Training, Generators, Knowledge engineering, Privacy, Differentially private learning, knowledge distillation BibRef

Wang, L.[Lin], Yoon, K.J.[Kuk-Jin],
Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlooks,
PAMI(44), No. 6, June 2022, pp. 3048-3068.
IEEE DOI 2205
Training, Measurement, Computational modeling, Visualization, Task analysis, Knowledge transfer, Speech recognition, visual intelligence BibRef

Wang, L.[Lin], Chae, Y.J.[Yu-Jeong], Yoon, S.H.[Sung-Hoon], Kim, T.K.[Tae-Kyun], Yoon, K.J.[Kuk-Jin],
EvDistill: Asynchronous Events to End-task Learning via Bidirectional Reconstruction-guided Cross-modal Knowledge Distillation,
CVPR21(608-619)
IEEE DOI 2111
Training, Knowledge engineering, Semantics, Dynamic range, Cameras, Data models BibRef

Song, J.[Jie], Chen, Y.[Ying], Ye, J.W.[Jing-Wen], Song, M.L.[Ming-Li],
Spot-Adaptive Knowledge Distillation,
IP(31), 2022, pp. 3359-3370.
IEEE DOI 2205
Knowledge engineering, Training, Routing, Data models, Adaptation models, Deep learning, Training data, spot-adaptive distillation BibRef

Zhao, P.S.[Pei-Sen], Xie, L.X.[Ling-Xi], Wang, J.J.[Jia-Jie], Zhang, Y.[Ya], Tian, Q.[Qi],
Progressive privileged knowledge distillation for online action detection,
PR(129), 2022, pp. 108741.
Elsevier DOI 2206
Online action detection, Knowledge distillation, Privileged information, Curriculum learning BibRef

Zhao, H.R.[Hao-Ran], Sun, X.[Xin], Gao, F.[Feng], Dong, J.Y.[Jun-Yu],
Pair-Wise Similarity Knowledge Distillation for RSI Scene Classification,
RS(14), No. 10, 2022, pp. xx-yy.
DOI Link 2206
BibRef

Li, K.[Kunchi], Wan, J.[Jun], Yu, S.[Shan],
CKDF: Cascaded Knowledge Distillation Framework for Robust Incremental Learning,
IP(31), 2022, pp. 3825-3837.
IEEE DOI 2206
Task analysis, Computational modeling, Adaptation models, Data models, Training, Knowledge engineering, Feature extraction, incremental learning BibRef

Yang, D.B.[Dong-Bao], Zhou, Y.[Yu], Zhang, A.[Aoting], Sun, X.R.[Xu-Rui], Wu, D.[Dayan], Wang, W.P.[Wei-Ping], Ye, Q.X.[Qi-Xiang],
Multi-View correlation distillation for incremental object detection,
PR(131), 2022, pp. 108863.
Elsevier DOI 2208
Object detection, Incremental learning, Catastrophic forgetting, Knowledge distillation BibRef

Zhou, H.[Haonan], Du, X.P.[Xiao-Ping], Li, S.[Sen],
Self-Supervision and Self-Distillation with Multilayer Feature Contrast for Supervision Collapse in Few-Shot Remote Sensing Scene Classification,
RS(14), No. 13, 2022, pp. xx-yy.
DOI Link 2208
BibRef

Chi, Q.[Qiang], Lv, G.H.[Guo-Hua], Zhao, G.X.[Gui-Xin], Dong, X.J.[Xiang-Jun],
A Novel Knowledge Distillation Method for Self-Supervised Hyperspectral Image Classification,
RS(14), No. 18, 2022, pp. xx-yy.
DOI Link 2209
BibRef

Wang, G.H.[Guo-Hua], Ge, Y.F.[Yi-Fan], Wu, J.X.[Jian-Xin],
Distilling Knowledge by Mimicking Features,
PAMI(44), No. 11, November 2022, pp. 8183-8195.
IEEE DOI 2210
Hash functions, Training, Standards, Residual neural networks, Radio frequency, Numerical models, Convolutional neural networks, object detection BibRef

He, Y.Y.[Yin-Yin], Wu, J.X.[Jian-Xin], Wei, X.S.[Xiu-Shen],
Distilling Virtual Examples for Long-tailed Recognition,
ICCV21(235-244)
IEEE DOI 2203
Visualization, Predictive models, Benchmark testing, Recognition and classification, BibRef

Hao, Z.W.[Zhi-Wei], Luo, Y.[Yong], Wang, Z.[Zhi], Hu, H.[Han], An, J.P.[Jian-Ping],
CDFKD-MFS: Collaborative Data-Free Knowledge Distillation via Multi-Level Feature Sharing,
MultMed(24), 2022, pp. 4262-4274.
IEEE DOI 2210
Generators, Knowledge engineering, Computational modeling, Aggregates, Predictive models, Collaboration, Model Compression, Attention BibRef

Zhao, Y.[Yibo], Liu, J.J.[Jian-Jun], Yang, J.L.[Jin-Long], Wu, Z.B.[Ze-Bin],
Remote Sensing Image Scene Classification via Self-Supervised Learning and Knowledge Distillation,
RS(14), No. 19, 2022, pp. xx-yy.
DOI Link 2210
BibRef

Tu, Z.G.[Zhi-Gang], Liu, X.J.[Xiang-Jian], Xiao, X.[Xuan],
A General Dynamic Knowledge Distillation Method for Visual Analytics,
IP(31), 2022, pp. 6517-6531.
IEEE DOI 2211
Knowledge engineering, Visualization, Task analysis, Optimization, Knowledge transfer, Image coding, Loss measurement. BibRef

Hou, J.W.[Jing-Wen], Ding, H.H.[Heng-Hui], Lin, W.S.[Wei-Si], Liu, W.[Weide], Fang, Y.M.[Yu-Ming],
Distilling Knowledge From Object Classification to Aesthetics Assessment,
CirSysVideo(32), No. 11, November 2022, pp. 7386-7402.
IEEE DOI 2211
Semantics, Computational modeling, Predictive models, Feature extraction, Training, Pattern matching, Visualization, image aesthetics assessment BibRef

Xing, S.Y.[Shi-Yi], Xing, J.S.[Jin-Sheng], Ju, J.G.[Jian-Guo], Hou, Q.S.[Qing-Shan], Ding, X.[Xiurui],
Collaborative Consistent Knowledge Distillation Framework for Remote Sensing Image Scene Classification Network,
RS(14), No. 20, 2022, pp. xx-yy.
DOI Link 2211
BibRef

Zhang, L.[Li], Wu, X.Q.[Xiang-Qian],
Latent Space Semantic Supervision Based on Knowledge Distillation for Cross-Modal Retrieval,
IP(31), 2022, pp. 7154-7164.
IEEE DOI 2212
Feature extraction, Semantics, Object detection, Correlation, Context modeling, Recurrent neural networks, Multitasking, knowledge distillation BibRef

Xu, H.T.[Hong-Teng], Liu, J.C.[Jia-Chang], Luo, D.[Dixin], Carin, L.[Lawrence],
Representing Graphs via Gromov-Wasserstein Factorization,
PAMI(45), No. 1, January 2023, pp. 999-1016.
IEEE DOI 2212
Kernel, Computational modeling, Task analysis, Message passing, Graph neural networks, Data models, Graph representation, permutation-invariance BibRef

Chen, L.Q.[Li-Qun], Wang, D.[Dong], Gan, Z.[Zhe], Liu, J.J.[Jing-Jing], Henao, R.[Ricardo], Carin, L.[Lawrence],
Wasserstein Contrastive Representation Distillation,
CVPR21(16291-16300)
IEEE DOI 2111
Knowledge engineering, Measurement, Computational modeling, Collaborative work, Robustness, Pattern recognition BibRef

Ye, H.J.[Han-Jia], Lu, S.[Su], Zhan, D.C.[De-Chuan],
Generalized Knowledge Distillation via Relationship Matching,
PAMI(45), No. 2, February 2023, pp. 1817-1834.
IEEE DOI 2301
Task analysis, Training, Neural networks, Knowledge engineering, Deep learning, Standards, Training data, Cross-Task, representation learning BibRef

Borza, D.L.[Diana Laura], Ileni, T.A.[Tudor Alexandru], Marinescu, A.I.[Alexandru Ion], Darabant, S.A.[Sergiu Adrian],
Teacher or supervisor? Effective online knowledge distillation via guided collaborative learning,
CVIU(228), 2023, pp. 103632.
Elsevier DOI 2302
Knowledge distillation, Collaborative learning, Online knowledge distillation, Model compression BibRef

Tang, R.N.[Rui-Ning], Liu, Z.Y.[Zhen-Yu], Li, Y.G.[Yang-Guang], Song, Y.[Yiguo], Liu, H.[Hui], Wang, Q.[Qide], Shao, J.[Jing], Duan, G.F.[Gui-Fang], Tan, J.R.[Jiang-Rong],
Task-balanced distillation for object detection,
PR(137), 2023, pp. 109320.
Elsevier DOI 2302
Object detection, Knowledge distillation BibRef

Xu, G.D.[Guo-Dong], Liu, Z.W.[Zi-Wei], Loy, C.C.[Chen Change],
Computation-Efficient Knowledge Distillation via Uncertainty-Aware Mixup,
PR(138), 2023, pp. 109338.
Elsevier DOI 2303
Knowledge distillation, Training cost BibRef

Zhang, Q.S.[Quan-Shi], Cheng, X.[Xu], Chen, Y.[Yilan], Rao, Z.[Zhefan],
Quantifying the Knowledge in a DNN to Explain Knowledge Distillation for Classification,
PAMI(45), No. 4, April 2023, pp. 5099-5113.
IEEE DOI 2303
Knowledge engineering, Task analysis, Measurement, Optimization, Feature extraction, Birds, Visualization, Knowledge distillation, knowledge points BibRef

Yu, X.[Xinyi], Yan, L.[Ling], Yang, Y.[Yang], Zhou, L.[Libo], Ou, L.L.[Lin-Lin],
Conditional generative data-free knowledge distillation,
IVC(131), 2023, pp. 104627.
Elsevier DOI 2303
Data-free knowledge distillation, Generative adversarial networks, Model compression, Convolutional neural networks BibRef

Su, T.T.[Tong-Tong], Liang, Q.Y.[Qi-Yu], Zhang, J.S.[Jin-Song], Yu, Z.Y.[Zhao-Yang], Xu, Z.[Ziyue], Wang, G.[Gang], Liu, X.G.[Xiao-Guang],
Deep Cross-Layer Collaborative Learning Network for Online Knowledge Distillation,
CirSysVideo(33), No. 5, May 2023, pp. 2075-2087.
IEEE DOI 2305
Knowledge engineering, Cross layer design, Training, Feature extraction, Visualization, Collaboration, model compression BibRef

Liu, Y.F.[Yi-Fan], Shu, C.Y.[Chang-Yong], Wang, J.D.[Jing-Dong], Shen, C.H.[Chun-Hua],
Structured Knowledge Distillation for Dense Prediction,
PAMI(45), No. 6, June 2023, pp. 7035-7049.
IEEE DOI 2305
Task analysis, Semantics, Training, Object detection, Image segmentation, Estimation, Knowledge engineering, dense prediction BibRef

Zhou, W.[Wujie], Sun, F.[Fan], Jiang, Q.P.[Qiu-Ping], Cong, R.[Runmin], Hwang, J.N.[Jenq-Neng],
WaveNet: Wavelet Network With Knowledge Distillation for RGB-T Salient Object Detection,
IP(32), 2023, pp. 3027-3039.
IEEE DOI 2306
Transformers, Feature extraction, Discrete wavelet transforms, Training, Knowledge engineering, Cross layer design, edge-aware module BibRef

Tang, Y.[Yuan], Chen, Y.[Ying], Xie, L.[Linbo],
Self-knowledge distillation based on knowledge transfer from soft to hard examples,
IVC(135), 2023, pp. 104700.
Elsevier DOI 2306
Model compression, Self-knowledge distillation, Hard examples, Class probability consistency, Memory bank BibRef

Gou, J.P.[Jian-Ping], Xiong, X.S.[Xiang-Shuo], Yu, B.S.[Bao-Sheng], Du, L.[Lan], Zhan, Y.B.[Yi-Bing], Tao, D.C.[Da-Cheng],
Multi-target Knowledge Distillation via Student Self-reflection,
IJCV(131), No. 7, July 2023, pp. 1857-1874.
Springer DOI 2307
BibRef

Lee, H.[Hyoje], Park, Y.[Yeachan], Seo, H.[Hyun], Kang, M.[Myungjoo],
Self-knowledge distillation via dropout,
CVIU(233), 2023, pp. 103720.
Elsevier DOI 2307
Deep learning, Knowledge distillation, Self-knowledge distillation, Regularization, Dropout BibRef

Li, Z.H.[Zhi-Hui], Xu, P.F.[Peng-Fei], Chang, X.J.[Xiao-Jun], Yang, L.[Luyao], Zhang, Y.Y.[Yuan-Yuan], Yao, L.[Lina], Chen, X.J.[Xiao-Jiang],
When Object Detection Meets Knowledge Distillation: A Survey,
PAMI(45), No. 8, August 2023, pp. 10555-10579.
IEEE DOI 2307
Survey, Knowledge Distillation. Task analysis, Computational modeling, Analytical models, Image coding, Solid modeling, Object detection, weakly supervised object detection BibRef

Yang, C.G.[Chuan-Guang], An, Z.L.[Zhu-Lin], Zhou, H.[Helong], Zhuang, F.Z.[Fu-Zhen], Xu, Y.J.[Yong-Jun], Zhang, Q.[Qian],
Online Knowledge Distillation via Mutual Contrastive Learning for Visual Recognition,
PAMI(45), No. 8, August 2023, pp. 10212-10227.
IEEE DOI 2307
Visualization, Knowledge engineering, Training, Task analysis, Federated learning, Dogs, Computational modeling, visual recognition BibRef

Fang, H.X.[Hang-Xiang], Long, Y.[Yongwen], Hu, X.[Xinyi], Ou, Y.T.[Yang-Tao], Huang, Y.[Yuanjia], Hu, H.J.[Hao-Ji],
Dual cross knowledge distillation for image super-resolution,
JVCIR(95), 2023, pp. 103858.
Elsevier DOI 2309
Super resolution, Knowledge distillation, Convolutional neural networks BibRef

Yu, L.F.[Li-Fang], Li, Y.W.[Yun-Wei], Weng, S.W.[Shao-Wei], Tian, H.[Huawei], Liu, J.[Jing],
Adaptive multi-teacher softened relational knowledge distillation framework for payload mismatch in image steganalysis,
JVCIR(95), 2023, pp. 103900.
Elsevier DOI 2309
Image steganalysis, PM (payload mismatch), BPDNets, AWA, SRKD BibRef

Yang, Z.[Zhen], Cao, Y.[Ying], Zhou, X.[Xin], Liu, J.[Junya], Zhang, T.[Tao], Ji, J.S.[Jin-Sheng],
Random Shuffling Data for Hyperspectral Image Classification with Siamese and Knowledge Distillation Network,
RS(15), No. 16, 2023, pp. 4078.
DOI Link 2309
BibRef

Cavazza, J.[Jacopo], Murino, V.[Vittorio], Bue, A.D.[Alessio Del],
No Adversaries to Zero-Shot Learning: Distilling an Ensemble of Gaussian Feature Generators,
PAMI(45), No. 10, October 2023, pp. 12167-12178.
IEEE DOI 2310
BibRef

Shao, R.R.[Ren-Rong], Zhang, W.[Wei], Wang, J.[Jun],
Conditional pseudo-supervised contrast for data-Free knowledge distillation,
PR(143), 2023, pp. 109781.
Elsevier DOI 2310
Model compression, Knowledge distillation, Representation learning, Contrastive learning, Privacy protection BibRef

Shao, R.R.[Ren-Rong], Zhang, W.[Wei], Yin, J.H.[Jian-Hua], Wang, J.[Jun],
Data-free Knowledge Distillation for Fine-grained Visual Categorization,
ICCV23(1515-1525)
IEEE DOI Code:
WWW Link. 2401
BibRef

López-Cifuentes, A.[Alejandro], Escudero-Viñolo, M.[Marcos], Bescós, J.[Jesús], Miguel, J.C.S.[Juan C. San],
Attention-Based Knowledge Distillation in Scene Recognition: The Impact of a DCT-Driven Loss,
CirSysVideo(33), No. 9, September 2023, pp. 4769-4783.
IEEE DOI Code:
WWW Link. 2310
BibRef

Ma, W.T.[Wen-Tao], Chen, Q.C.[Qing-Chao], Zhou, T.Q.[Tong-Qing], Zhao, S.[Shan], Cai, Z.P.[Zhi-Ping],
Using Multimodal Contrastive Knowledge Distillation for Video-Text Retrieval,
CirSysVideo(33), No. 10, October 2023, pp. 5486-5497.
IEEE DOI 2310
BibRef

Zhang, L.F.[Lin-Feng], Ma, K.[Kaisheng],
Structured Knowledge Distillation for Accurate and Efficient Object Detection,
PAMI(45), No. 12, December 2023, pp. 15706-15724.
IEEE DOI 2311
BibRef

Zhang, L.F.[Lin-Feng], Dong, R.[Runpei], Tai, H.S.[Hung-Shuo], Ma, K.[Kaisheng],
PointDistiller: Structured Knowledge Distillation Towards Efficient and Compact 3D Detection,
CVPR23(21791-21801)
IEEE DOI 2309
BibRef

Yue, H.[Han], Li, J.D.[Jun-Dong], Liu, H.F.[Hong-Fu],
Second-Order Unsupervised Feature Selection via Knowledge Contrastive Distillation,
PAMI(45), No. 12, December 2023, pp. 15577-15587.
IEEE DOI 2311
BibRef

Yu, X.T.[Xiao-Tong], Sun, S.[Shiding], Tian, Y.J.[Ying-Jie],
Self-distillation and self-supervision for partial label learning,
PR(146), 2024, pp. 110016.
Elsevier DOI 2311
Knowledge distillation, Self-supervised learning, Partial label learning, Machine learning BibRef

Yang, S.Z.[Shun-Zhi], Xu, L.[Liuchi], Zhou, M.[Mengchu], Yang, X.[Xiong], Yang, J.F.[Jin-Feng], Huang, Z.H.[Zhen-Hua],
Skill-Transferring Knowledge Distillation Method,
CirSysVideo(33), No. 11, November 2023, pp. 6487-6502.
IEEE DOI 2311
BibRef

Zhao, Q.[Qi], Lyu, S.C.[Shu-Chang], Chen, L.[Lijiang], Liu, B.[Binghao], Xu, T.B.[Ting-Bing], Cheng, G.L.[Guang-Liang], Feng, W.[Wenquan],
Learn by Oneself: Exploiting Weight-Sharing Potential in Knowledge Distillation Guided Ensemble Network,
CirSysVideo(33), No. 11, November 2023, pp. 6661-6678.
IEEE DOI Code:
WWW Link. 2311
BibRef

Cao, Q.Z.[Qi-Zhi], Zhang, K.[Kaibing], He, X.[Xin], Shen, J.[Junge],
Be an Excellent Student: Review, Preview, and Correction,
SPLetters(30), 2023, pp. 1722-1726.
IEEE DOI 2312
BibRef

Li, X.F.[Xiu-Fang], Sun, Q.G.[Qi-Gong], Jiao, L.C.[Li-Cheng], Liu, F.[Fang], Liu, X.[Xu], Li, L.L.[Ling-Ling], Chen, P.[Puhua], Zuo, Y.[Yi],
D^3K: Dynastic Data-Free Knowledge Distillation,
MultMed(25), 2023, pp. 8358-8371.
IEEE DOI 2312
BibRef

Xu, K.[Kai], Wang, L.C.[Li-Chun], Xin, J.[Jianjia], Li, S.[Shuang], Yin, B.C.[Bao-Cai],
Learning From Teacher's Failure: A Reflective Learning Paradigm for Knowledge Distillation,
CirSysVideo(34), No. 1, January 2024, pp. 384-396.
IEEE DOI 2401
BibRef

Wang, J.H.[Jun-Huang], Zhang, W.W.[Wei-Wei], Guo, Y.F.[Yu-Feng], Liang, P.[Peng], Ji, M.[Ming], Zhen, C.H.[Cheng-Hui], Wang, H.[Hanmeng],
Global key knowledge distillation framework,
CVIU(239), 2024, pp. 103902.
Elsevier DOI 2402
Deep learning, Knowledge distillation, Self-distillation, Convolutional neural network BibRef

Yang, A.[Aijia], Lin, S.[Sihao], Yeh, C.H.[Chung-Hsing], Shu, M.[Minglei], Yang, Y.[Yi], Chang, X.J.[Xiao-Jun],
Context Matters: Distilling Knowledge Graph for Enhanced Object Detection,
MultMed(26), 2024, pp. 487-500.
IEEE DOI 2402
Detectors, Knowledge graphs, Semantics, Object detection, Transformers, Visualization, Image edge detection, knowledge graph BibRef

Yu, H.[Hao], Feng, X.[Xin], Wang, Y.L.[Yun-Long],
Enhancing deep feature representation in self-knowledge distillation via pyramid feature refinement,
PRL(178), 2024, pp. 35-42.
Elsevier DOI Code:
WWW Link. 2402
Self-knowledge distillation, Feature representation, Pyramid structure, Deep neural networks BibRef

Ye, X.[Xin], Jiang, R.X.[Rong-Xin], Tian, X.[Xiang], Zhang, R.[Rui], Chen, Y.W.[Yao-Wu],
Knowledge Distillation via Multi-Teacher Feature Ensemble,
SPLetters(31), 2024, pp. 566-570.
IEEE DOI 2402
Feature extraction, Optimization, Training, Image reconstruction, Transforms, Semantics, Knowledge engineering, Feature ensemble, knowledge distillation BibRef

Bao, Z.Q.[Zhi-Qiang], Chen, Z.H.[Zi-Hao], Wang, C.D.[Chang-Dong], Zheng, W.S.[Wei-Shi], Huang, Z.H.[Zhen-Hua], Chen, Y.[Yunwen],
Post-Distillation via Neural Resuscitation,
MultMed(26), 2024, pp. 3046-3060.
IEEE DOI 2402
Neurons, Computational modeling, Standards, Optimization, Knowledge engineering, Task analysis, Probabilistic logic, transfer learning BibRef

Ma, X.[Xin], Liu, C.[Chang], Xie, C.Y.[Chun-Yu], Ye, L.[Long], Deng, Y.F.[Ya-Feng], Ji, X.Y.[Xiang-Yang],
Disjoint Masking With Joint Distillation for Efficient Masked Image Modeling,
MultMed(26), 2024, pp. 3077-3087.
IEEE DOI 2402
Training, Image reconstruction, Predictive models, Task analysis, Visualization, Convergence, Computational modeling, and training efficiency BibRef

Rao, J.[Jun], Meng, X.[Xv], Ding, L.[Liang], Qi, S.H.[Shu-Han], Liu, X.[Xuebo], Zhang, M.[Min], Tao, D.C.[Da-Cheng],
Parameter-Efficient and Student-Friendly Knowledge Distillation,
MultMed(26), 2024, pp. 4230-4241.
IEEE DOI 2403
Training, Smoothing methods, Knowledge transfer, Data models, Adaptation models, Predictive models, Knowledge engineering, image classification BibRef

Li, Z.[Zheng], Li, X.[Xiang], Yang, L.F.[Ling-Feng], Song, R.J.[Ren-Jie], Yang, J.[Jian], Pan, Z.[Zhigeng],
Dual teachers for self-knowledge distillation,
PR(151), 2024, pp. 110422.
Elsevier DOI 2404
Model compression, Image classification, Self-knowledge distillation, Dual teachers BibRef

Gou, J.P.[Jian-Ping], Chen, Y.[Yu], Yu, B.[Baosheng], Liu, J.H.[Jin-Hua], Du, L.[Lan], Wan, S.H.[Shao-Hua], Yi, Z.[Zhang],
Reciprocal Teacher-Student Learning via Forward and Feedback Knowledge Distillation,
MultMed(26), 2024, pp. 7901-7916.
IEEE DOI 2405
Knowledge engineering, Training, Visualization, Computational modeling, Reviews, Knowledge transfer, Correlation, visual recognition BibRef

Xu, Z.[Zhi], Fu, Z.Y.[Zhen-Yong],
Using Mixture of Experts to accelerate dataset distillation,
JVCIR(100), 2024, pp. 104137.
Elsevier DOI 2405
Dataset distillation, Mixture of experts, Accelerate BibRef

Yang, S.Z.[Shun-Zhi], Yang, J.F.[Jin-Feng], Zhou, M.C.[Meng-Chu], Huang, Z.H.[Zhen-Hua], Zheng, W.S.[Wei-Shi], Yang, X.[Xiong], Ren, J.[Jin],
Learning From Human Educational Wisdom: A Student-Centered Knowledge Distillation Method,
PAMI(46), No. 6, June 2024, pp. 4188-4205.
IEEE DOI 2405
Knowledge engineering, PD control, PI control, Task analysis, Knowledge transfer, Automation, Training, Curriculum learning, student-centered BibRef

Mei, Z.[Zhen], Ye, P.[Peng], Li, B.[Baopu], Chen, T.[Tao], Fan, J.Y.[Jia-Yuan], Ouyang, W.L.[Wan-Li],
DeNKD: Decoupled Non-Target Knowledge Distillation for Complementing Transformer-Based Unsupervised Domain Adaptation,
CirSysVideo(34), No. 5, May 2024, pp. 3220-3231.
IEEE DOI 2405
Transformers, Task analysis, Semantics, Adaptation models, Knowledge transfer, Visualization, Training, Transformer, knowledge distillation BibRef

Liang, G.Q.[Guo-Qiang], Chen, Z.J.[Zhao-Jie], Chen, Z.Q.[Zhao-Qiang], Ji, S.Y.[Shi-Yu], Zhang, Y.N.[Yan-Ning],
New Insights on Relieving Task-Recency Bias for Online Class Incremental Learning,
CirSysVideo(34), No. 5, May 2024, pp. 3451-3464.
IEEE DOI Code:
WWW Link. 2405
Task analysis, Data models, Training, Streaming media, Stability criteria, Predictive models, Circuit stability, virtual knowledge distillation BibRef

Xu, L.[Liuchi], Ren, J.[Jin], Huang, Z.H.[Zhen-Hua], Zheng, W.S.[Wei-Shi], Chen, Y.[Yunwen],
Improving Knowledge Distillation via Head and Tail Categories,
CirSysVideo(34), No. 5, May 2024, pp. 3465-3480.
IEEE DOI 2405
Tail, Head, Knowledge transfer, Task analysis, Knowledge engineering, Image classification, Training, Knowledge distillation, instance segmentation BibRef


Pham, C.[Cuong], Nguyen, V.A.[Van-Anh], Le, T.[Trung], Phung, D.[Dinh], Carneiro, G.[Gustavo], Do, T.T.[Thanh-Toan],
Frequency Attention for Knowledge Distillation,
WACV24(2266-2275)
IEEE DOI 2404
Knowledge engineering, Frequency-domain analysis, Computational modeling, Object detection, Computer architecture, Embedded sensing / real-time techniques BibRef

Lan, Q.Z.[Qi-Zhen], Tian, Q.[Qing],
Gradient-Guided Knowledge Distillation for Object Detectors,
WACV24(423-432)
IEEE DOI Code:
WWW Link. 2404
Deep learning, Codes, Computational modeling, Object detection, Detectors, Feature extraction, Algorithms BibRef

Reddy, N.[Nikhil], Baktashmotlagh, M.[Mahsa], Arora, C.[Chetan],
Domain-Aware Knowledge Distillation for Continual Model Generalization,
WACV24(685-696)
IEEE DOI 2404
Adaptation models, Computational modeling, Prototypes, Artificial neural networks, Predictive models, Synthetic data, Autonomous Driving BibRef

Huang, J.Q.[Jun-Qiang], Guo, Z.[Zichao],
Pixel-Wise Contrastive Distillation,
ICCV23(16313-16323)
IEEE DOI 2401
BibRef

Lebailly, T.[Tim], Stegmüller, T.[Thomas], Bozorgtabar, B.[Behzad], Thiran, J.P.[Jean-Philippe], Tuytelaars, T.[Tinne],
Adaptive Similarity Bootstrapping for Self-Distillation based Representation Learning,
ICCV23(16459-16468)
IEEE DOI Code:
WWW Link. 2401
BibRef

Yang, L.[Longrong], Zhou, X.[Xianpan], Li, X.[Xuewei], Qiao, L.[Liang], Li, Z.[Zheyang], Yang, Z.W.[Zi-Wei], Wang, G.[Gaoang], Li, X.[Xi],
Bridging Cross-task Protocol Inconsistency for Distillation in Dense Object Detection,
ICCV23(17129-17138)
IEEE DOI Code:
WWW Link. 2401
BibRef

Liu, Z.W.[Zi-Wei], Wang, Y.T.[Yong-Tao], Chu, X.J.[Xiao-Jie], Dong, N.[Nan], Qi, S.X.[Sheng-Xiang], Ling, H.B.[Hai-Bin],
A Simple and Generic Framework for Feature Distillation via Channel-wise Transformation,
REDLCV23(1121-1130)
IEEE DOI 2401
BibRef

Lao, S.S.[Shan-Shan], Song, G.[Guanglu], Liu, B.[Boxiao], Liu, Y.[Yu], Yang, Y.[Yujiu],
UniKD: Universal Knowledge Distillation for Mimicking Homogeneous or Heterogeneous Object Detectors,
ICCV23(6339-6349)
IEEE DOI 2401
BibRef

Sun, X.M.[Xi-Meng], Zhang, P.C.[Peng-Chuan], Zhang, P.Z.[Pei-Zhao], Shah, H.[Hardik], Saenko, K.[Kate], Xia, X.[Xide],
DIME-FM: DIstilling Multimodal and Efficient Foundation Models,
ICCV23(15475-15487)
IEEE DOI 2401
BibRef

Radwan, A.[Ahmed], Shehata, M.S.[Mohamed S.],
Distilling Part-whole Hierarchical Knowledge from a Huge Pretrained Class Agnostic Segmentation Framework,
VIPriors23(238-246)
IEEE DOI Code:
WWW Link. 2401
BibRef

Tang, J.L.[Jia-Liang], Chen, S.[Shuo], Niu, G.[Gang], Sugiyama, M.[Masashi], Gong, C.[Chen],
Distribution Shift Matters for Knowledge Distillation with Webly Collected Images,
ICCV23(17424-17434)
IEEE DOI 2401
BibRef

Li, L.[Lujun], Dong, P.[Peijie], Wei, Z.[Zimian], Yang, Y.[Ya],
Automated Knowledge Distillation via Monte Carlo Tree Search,
ICCV23(17367-17378)
IEEE DOI Code:
WWW Link. 2401
BibRef

Choi, J.[Junyong], Cho, H.[Hyeon], Cheung, S.[Seokhwa], Hwang, W.J.[Won-Jun],
ORC: Network Group-based Knowledge Distillation using Online Role Change,
ICCV23(17335-17344)
IEEE DOI 2401
BibRef

Yang, P.H.[Peng-Hui], Xie, M.K.[Ming-Kun], Zong, C.C.[Chen-Chen], Feng, L.[Lei], Niu, G.[Gang], Sugiyama, M.[Masashi], Huang, S.J.[Sheng-Jun],
Multi-Label Knowledge Distillation,
ICCV23(17225-17234)
IEEE DOI Code:
WWW Link. 2401
BibRef

Yang, Z.D.[Zhen-Dong], Zeng, A.[Ailing], Li, Z.[Zhe], Zhang, T.[Tianke], Yuan, C.[Chun], Li, Y.[Yu],
From Knowledge Distillation to Self-Knowledge Distillation: A Unified Approach with Normalized Loss and Customized Soft Labels,
ICCV23(17139-17148)
IEEE DOI Code:
WWW Link. 2401
BibRef

Gu, P.Y.[Pei-Yan], Zhang, C.[Chuyu], Xu, R.J.[Rui-Jie], He, X.M.[Xu-Ming],
Class-relation Knowledge Distillation for Novel Class Discovery,
ICCV23(16428-16437)
IEEE DOI 2401
BibRef

Gu, Z.H.[Zhi-Hao], Liu, L.[Liang], Chen, X.[Xu], Yi, R.[Ran], Zhang, J.N.[Jiang-Ning], Wang, Y.[Yabiao], Wang, C.J.[Cheng-Jie], Shu, A.[Annan], Jiang, G.[Guannan], Ma, L.Z.[Li-Zhuang],
Remembering Normality: Memory-guided Knowledge Distillation for Unsupervised Anomaly Detection,
ICCV23(16355-16363)
IEEE DOI 2401
BibRef

Dong, J.F.[Jian-Feng], Zhang, M.[Minsong], Zhang, Z.[Zheng], Chen, X.[Xianke], Liu, D.[Daizong], Qu, X.Y.[Xiao-Ye], Wang, X.[Xun], Liu, B.[Baolong],
Dual Learning with Dynamic Knowledge Distillation for Partially Relevant Video Retrieval,
ICCV23(11268-11278)
IEEE DOI 2401
BibRef

Zhao, B.[Borui], Cui, Q.[Quan], Song, R.J.[Ren-Jie], Liang, J.J.[Jia-Jun],
DOT: A Distillation-Oriented Trainer,
ICCV23(6166-6175)
IEEE DOI Code:
WWW Link. 2401
BibRef

Zhao, B.[Borui], Song, R.J.[Ren-Jie], Liang, J.J.[Jia-Jun],
Cumulative Spatial Knowledge Distillation for Vision Transformers,
ICCV23(6123-6132)
IEEE DOI Code:
WWW Link. 2401
BibRef

Gao, T.W.[Ting-Wei], Long, R.[Rujiao],
Accumulation Knowledge Distillation for Conditional GAN Compression,
REDLCV23(1294-1303)
IEEE DOI 2401
BibRef

Bender, S.[Sidney], Anders, C.J.[Christopher J.], Chormai, P.[Pattarawat], Marxfeld, H.[Heike], Herrmann, J.[Jan], Montavon, G.[Grégoire],
Towards Fixing Clever-Hans Predictors with Counterfactual Knowledge Distillation,
CVAMD23(2599-2607)
IEEE DOI 2401
BibRef

Wang, Q.[Qi], Liu, L.[Lu], Yu, W.X.[Wen-Xin], Chen, S.Y.[Shi-Yu], Gong, J.[Jun], Chen, P.[Peng],
BCKD: Block-Correlation Knowledge Distillation,
ICIP23(3225-3229)
IEEE DOI 2312
BibRef

Sasaya, T.[Tenta], Watanabe, T.[Takashi], Ida, T.[Takashi], Ono, T.[Toshiyuki],
Simple Self-Distillation Learning for Noisy Image Classification,
ICIP23(795-799)
IEEE DOI 2312
BibRef

Qian, C.[Chengyao], Hayat, M.[Munawar], Harandi, M.[Mehrtash],
Can we Distill Knowledge from Powerful Teachers Directly?,
ICIP23(595-599)
IEEE DOI 2312
BibRef

Zhang, Y.[Yi], Gao, Y.K.[Ying-Ke], Zhang, H.N.[Hao-Nan], Lei, X.Y.[Xin-Yu], Liu, L.J.[Long-Jun],
Cross-Layer Patch Alignment and Intra-and-Inter Patch Relations for Knowledge Distillation,
ICIP23(535-539)
IEEE DOI 2312
BibRef

Wang, C.C.[Chien-Chih], Xu, S.Y.[Shao-Yuan], Fu, J.M.[Jin-Miao], Liu, Y.[Yang], Wang, B.[Bryan],
KD-Fixmatch: Knowledge Distillation Siamese Neural Networks,
ICIP23(341-345)
IEEE DOI 2312
BibRef

Jin, Y.[Ying], Wang, J.Q.[Jia-Qi], Lin, D.[Dahua],
Multi-Level Logit Distillation,
CVPR23(24276-24285)
IEEE DOI 2309
BibRef

Zhang, J.[Jimuyang], Huang, Z.[Zanming], Ohn-Bar, E.[Eshed],
Coaching a Teachable Student,
CVPR23(7805-7815)
IEEE DOI 2309
BibRef

Zhmoginov, A.[Andrey], Sandler, M.[Mark], Miller, N.[Nolan], Kristiansen, G.[Gus], Vladymyrov, M.[Max],
Decentralized Learning with Multi-Headed Distillation,
CVPR23(8053-8063)
IEEE DOI 2309
BibRef

Tastan, N.[Nurbek], Nandakumar, K.[Karthik],
CaPriDe Learning: Confidential and Private Decentralized Learning Based on Encryption-Friendly Distillation Loss,
CVPR23(8084-8092)
IEEE DOI 2309
BibRef

Liu, G.W.[Gao-Wen], Shang, Y.Z.[Yu-Zhang], Yao, Y.G.[Yu-Guang], Kompella, R.[Ramana],
Network Specialization via Feature-level Knowledge Distillation,
VOCVALC23(3368-3375)
IEEE DOI 2309
BibRef

Zhang, T.[Tianli], Xue, M.Q.[Meng-Qi], Zhang, J.T.[Jiang-Tao], Zhang, H.[Haofei], Wang, Y.[Yu], Cheng, L.[Lechao], Song, J.[Jie], Song, M.L.[Ming-Li],
Generalization Matters: Loss Minima Flattening via Parameter Hybridization for Efficient Online Knowledge Distillation,
CVPR23(20176-20185)
IEEE DOI 2309
BibRef

Li, J.Z.[Jing-Zhi], Guo, Z.D.[Zi-Dong], Li, H.[Hui], Han, S.[Seungju], Baek, J.W.[Ji-Won], Yang, M.[Min], Yang, R.[Ran], Suh, S.[Sungjoo],
Rethinking Feature-based Knowledge Distillation for Face Recognition,
CVPR23(20156-20165)
IEEE DOI 2309
BibRef

Lin, H.[Han], Han, G.X.[Guang-Xing], Ma, J.W.[Jia-Wei], Huang, S.Y.[Shi-Yuan], Lin, X.D.[Xu-Dong], Chang, S.F.[Shih-Fu],
Supervised Masked Knowledge Distillation for Few-Shot Transformers,
CVPR23(19649-19659)
IEEE DOI 2309
BibRef

Du, X.[Xuanyi], Wan, W.T.[Wei-Tao], Sun, C.[Chong], Li, C.[Chen],
Weak-shot Object Detection through Mutual Knowledge Transfer,
CVPR23(19671-19680)
IEEE DOI 2309
BibRef

Shen, Y.Q.[Yan-Qing], Zhou, S.P.[San-Ping], Fu, J.W.[Jing-Wen], Wang, R.T.[Ruo-Tong], Chen, S.T.[Shi-Tao], Zheng, N.N.[Nan-Ning],
StructVPR: Distill Structural Knowledge with Weighting Samples for Visual Place Recognition,
CVPR23(11217-11226)
IEEE DOI 2309
BibRef

Xu, Q.[Qi], Li, Y.X.[Ya-Xin], Shen, J.[Jiangrong], Liu, J.K.[Jian K.], Tang, H.[Huajin], Pan, G.[Gang],
Constructing Deep Spiking Neural Networks from Artificial Neural Networks with Knowledge Distillation,
CVPR23(7886-7895)
IEEE DOI 2309
BibRef

Patel, G.[Gaurav], Mopuri, K.R.[Konda Reddy], Qiu, Q.[Qiang],
Learning to Retain while Acquiring: Combating Distribution-Shift in Adversarial Data-Free Knowledge Distillation,
CVPR23(7786-7794)
IEEE DOI 2309
BibRef

Chen, Y.Z.[Yi-Zhuo], Liang, K.[Kaizhao], Zeng, Z.[Zhe], Yao, S.[Shuochao], Shao, H.[Huajie],
A Unified Knowledge Distillation Framework for Deep Directed Graphical Models,
CVPR23(7795-7804)
IEEE DOI 2309
BibRef

Cui, K.W.[Kai-Wen], Yu, Y.C.[Ying-Chen], Zhan, F.N.[Fang-Neng], Liao, S.C.[Sheng-Cai], Lu, S.J.[Shi-Jian], Xing, E.[Eric],
KD-DLGAN: Data Limited Image Generation via Knowledge Distillation,
CVPR23(3872-3882)
IEEE DOI 2309
BibRef

Xu, J.Q.[Jian-Qing], Li, S.[Shen], Deng, A.[Ailin], Xiong, M.[Miao], Wu, J.Y.[Jia-Ying], Wu, J.X.[Jia-Xiang], Ding, S.H.[Shou-Hong], Hooi, B.[Bryan],
Probabilistic Knowledge Distillation of Face Ensembles,
CVPR23(3489-3498)
IEEE DOI 2309
BibRef

Guo, Z.[Ziyao], Yan, H.[Haonan], Li, H.[Hui], Lin, X.D.[Xiao-Dong],
Class Attention Transfer Based Knowledge Distillation,
CVPR23(11868-11877)
IEEE DOI 2309
BibRef

Dong, P.[Peijie], Li, L.[Lujun], Wei, Z.[Zimian],
DisWOT: Student Architecture Search for Distillation WithOut Training,
CVPR23(11898-11908)
IEEE DOI 2309
BibRef

Song, K.[Kaiyou], Zhang, S.[Shan], Luo, Z.[Zimeng], Wang, T.[Tong], Xie, J.[Jin],
Semantics-Consistent Feature Search for Self-Supervised Visual Representation Learning,
ICCV23(16053-16062)
IEEE DOI 2401
BibRef

Song, K.[Kaiyou], Xie, J.[Jin], Zhang, S.[Shan], Luo, Z.[Zimeng],
Multi-Mode Online Knowledge Distillation for Self-Supervised Visual Representation Learning,
CVPR23(11848-11857)
IEEE DOI 2309
BibRef

Yu, S.K.[Shi-Kang], Chen, J.C.[Jia-Chen], Han, H.[Hu], Jiang, S.Q.[Shu-Qiang],
Data-Free Knowledge Distillation via Feature Exchange and Activation Region Constraint,
CVPR23(24266-24275)
IEEE DOI 2309
BibRef

Shrivastava, A.[Aman], Qi, Y.J.[Yan-Jun], Ordonez, V.[Vicente],
Estimating and Maximizing Mutual Information for Knowledge Distillation,
FaDE-TCV23(48-57)
IEEE DOI 2309
BibRef

Gao, L.[Lei], Gao, H.[Hui],
Feature Decoupled Knowledge Distillation via Spatial Pyramid Pooling,
ACCV22(VI:732-745).
Springer DOI 2307
BibRef

Lv, Y.[Yuan], Xu, Y.J.[Ya-Jing], Wang, S.[Shusen], Ma, Y.J.[Ying-Jian], Wang, D.[Dengke],
Continuous Self-Study: Scene Graph Generation with Self-Knowledge Distillation and Spatial Augmentation,
ACCV22(V:297-315).
Springer DOI 2307
BibRef

Liu, Y.F.[Yu-Fan], Cao, J.J.[Jia-Jiong], Li, B.[Bing], Hu, W.M.[Wei-Ming], Ding, J.T.[Jing-Ting], Li, L.[Liang],
Cross-architecture Knowledge Distillation,
ACCV22(V:179-195).
Springer DOI 2307
BibRef

Lee, H.[Hojung], Lee, J.S.[Jong-Seok],
Rethinking Online Knowledge Distillation with Multi-Exits,
ACCV22(VI:408-424).
Springer DOI 2307
BibRef

Wang, L.Y.[Li-Yun], Rhodes, A.[Anthony], Feng, W.C.[Wu-Chi],
Class Specialized Knowledge Distillation,
ACCV22(II:391-408).
Springer DOI 2307
BibRef

Li, W.[Wei], Shao, S.T.[Shi-Tong], Liu, W.Y.[Wei-Yan], Qiu, Z.M.[Zi-Ming], Zhu, Z.H.[Zhi-Hao], Huan, W.[Wei],
What Role Does Data Augmentation Play in Knowledge Distillation?,
ACCV22(II:507-525).
Springer DOI 2307
BibRef

Feng, P.[Ping], Zhang, H.[Hanyun], Sun, Y.Y.[Ying-Ying], Tang, Z.J.[Zhen-Jun],
Lightweight Image Hashing Based on Knowledge Distillation and Optimal Transport for Face Retrieval,
MMMod23(II: 423-434).
Springer DOI 2304
BibRef

Ambekar, S.[Sameer], Tafuro, M.[Matteo], Ankit, A.[Ankit], van der Mast, D.[Diego], Alence, M.[Mark], Athanasiadis, C.[Christos],
Skdcgn: Source-free Knowledge Distillation of Counterfactual Generative Networks Using cgans,
VIPriors22(679-693).
Springer DOI 2304
BibRef

Lebailly, T.[Tim], Tuytelaars, T.[Tinne],
Global-Local Self-Distillation for Visual Representation Learning,
WACV23(1441-1450)
IEEE DOI 2302
Training, Representation learning, Visualization, Codes, Coherence, Task analysis, Algorithms: Machine learning architectures, and algorithms (including transfer) BibRef

Choi, H.J.[Hong-Jun], Jeon, E.S.[Eun Som], Shukla, A.[Ankita], Turaga, P.[Pavan],
Understanding the Role of Mixup in Knowledge Distillation: An Empirical Study,
WACV23(2318-2327)
IEEE DOI 2302
Knowledge engineering, Training, Interpolation, Codes, Transfer learning, Robustness, adversarial attack and defense methods BibRef

Jacob, G.M.[Geethu Miriam], Agarwal, V.[Vishal], Stenger, B.[Björn],
Online Knowledge Distillation for Multi-task Learning,
WACV23(2358-2367)
IEEE DOI 2302
Training, Knowledge engineering, Semantic segmentation, Computational modeling, Estimation, Benchmark testing BibRef

Chen, W.C.[Wei-Chi], Chu, W.T.[Wei-Ta],
SSSD: Self-Supervised Self Distillation,
WACV23(2769-2776)
IEEE DOI 2302
Visualization, Computational modeling, Clustering algorithms, Self-supervised learning, Feature extraction, Data models, visual reasoning BibRef

Mu, M.[Michael], Bhattacharjee, S.D.[Sreyasee Das], Yuan, J.S.[Jun-Song],
Self-Supervised Distilled Learning for Multi-modal Misinformation Identification,
WACV23(2818-2827)
IEEE DOI 2302
Representation learning, Training data, Predictive models, Streaming media, Semisupervised learning, Multitasking, Vision + language and/or other modalities BibRef

Jang, J.[Jiho], Kim, S.[Seonhoon], Yoo, K.[Kiyoon], Kong, C.[Chaerin], Kim, J.[Jangho], Kwak, N.[Nojun],
Self-Distilled Self-supervised Representation Learning,
WACV23(2828-2838)
IEEE DOI 2302
Representation learning, Protocols, Codes, Statistical analysis, Self-supervised learning, Transformers, and algorithms (including transfer) BibRef

Nguyen-Duc, T.[Thanh], Le, T.[Trung], Zhao, H.[He], Cai, J.F.[Jian-Fei], Phung, D.[Dinh],
Adversarial local distribution regularization for knowledge distillation,
WACV23(4670-4679)
IEEE DOI 2302
Perturbation methods, Algorithms: Adversarial learning, adversarial attack and defense methods BibRef

Wu, Y.[Yong], Chanda, S.[Shekhor], Hosseinzadeh, M.[Mehrdad], Liu, Z.[Zhi], Wang, Y.[Yang],
Few-Shot Learning of Compact Models via Task-Specific Meta Distillation,
WACV23(6254-6263)
IEEE DOI 2302
Training, Adaptation models, Computational modeling, Benchmark testing, Servers, visual reasoning BibRef

Iwata, S.[Sachi], Minami, S.[Soma], Hirakawa, T.[Tsubasa], Yamashita, T.[Takayoshi], Fujiyoshi, H.[Hironobu],
Refining Design Spaces in Knowledge Distillation for Deep Collaborative Learning,
ICPR22(2371-2377)
IEEE DOI 2212
Analytical models, Federated learning, Refining, Market research, Task analysis, Knowledge transfer BibRef

Wang, C.F.[Chao-Fei], Zhang, S.W.[Shao-Wei], Song, S.[Shiji], Huang, G.[Gao],
Learn From the Past: Experience Ensemble Knowledge Distillation,
ICPR22(4736-4743)
IEEE DOI 2212
Knowledge engineering, Training, Adaptation models, Costs, Standards, Knowledge transfer BibRef

Tzelepi, M.[Maria], Symeonidis, C.[Charalampos], Nikolaidis, N.[Nikos], Tefas, A.[Anastasios],
Multilayer Online Self-Acquired Knowledge Distillation,
ICPR22(4822-4828)
IEEE DOI 2212
Training, Computational modeling, Pipelines, Estimation, Nonhomogeneous media, Probability distribution BibRef

Xu, Y.F.[Yi-Fan], Shamsolmoali, P.[Pourya], Granger, E.[Eric], Nicodeme, C.[Claire], Gardes, L.[Laurent], Yang, J.[Jie],
TransVLAD: Multi-Scale Attention-Based Global Descriptors for Visual Geo-Localization,
WACV23(2839-2848)
IEEE DOI 2302
Visualization, Codes, Computational modeling, Image retrieval, Self-supervised learning, Transformers, and un-supervised learning) BibRef

Xu, Y.F.[Yi-Fan], Shamsolmoali, P.[Pourya], Yang, J.[Jie],
Weak-supervised Visual Geo-localization via Attention-based Knowledge Distillation,
ICPR22(1815-1821)
IEEE DOI 2212
Knowledge engineering, Training, Visualization, Image matching, Image retrieval, Lighting, Benchmark testing BibRef

Baek, K.[Kyungjune], Lee, S.[Seungho], Shim, H.J.[Hyun-Jung],
Learning from Better Supervision: Self-distillation for Learning with Noisy Labels,
ICPR22(1829-1835)
IEEE DOI 2212
Training, Deep learning, Filtering, Neural networks, Predictive models, Data collection, Benchmark testing BibRef

Chen, D.[Dingyao], Tan, H.[Huibin], Lan, L.[Long], Zhang, X.[Xiang], Liang, T.Y.[Tian-Yi], Luo, Z.G.[Zhi-Gang],
Frustratingly Easy Knowledge Distillation via Attentive Similarity Matching,
ICPR22(2357-2363)
IEEE DOI 2212
Knowledge engineering, Dimensionality reduction, Cross layer design, Semantics, Mobile handsets, Pattern recognition BibRef

Amara, I.[Ibtihel], Ziaeefard, M.[Maryam], Meyer, B.H.[Brett H.], Gross, W.[Warren], Clark, J.J.[James J.],
CES-KD: Curriculum-based Expert Selection for Guided Knowledge Distillation,
ICPR22(1901-1907)
IEEE DOI 2212
Knowledge engineering, Performance evaluation, Bridges, Art, Education BibRef

Yang, Z.[Zhou], Dong, W.S.[Wei-Sheng], Li, X.[Xin], Wu, J.J.[Jin-Jian], Li, L.[Leida], Shi, G.M.[Guang-Ming],
Self-Feature Distillation with Uncertainty Modeling for Degraded Image Recognition,
ECCV22(XXIV:552-569).
Springer DOI 2211
BibRef

Tang, S.[Sanli], Zhang, Z.Y.[Zhong-Yu], Cheng, Z.Z.[Zhan-Zhan], Lu, J.[Jing], Xu, Y.L.[Yun-Lu], Niu, Y.[Yi], He, F.[Fan],
Distilling Object Detectors with Global Knowledge,
ECCV22(IX:422-438).
Springer DOI 2211
BibRef

Shen, Z.Q.[Zhi-Qiang], Xing, E.[Eric],
A Fast Knowledge Distillation Framework for Visual Recognition,
ECCV22(XXIV:673-690).
Springer DOI 2211
BibRef

Yang, C.G.[Chuan-Guang], An, Z.[Zhulin], Zhou, H.[Helong], Cai, L.H.[Lin-Hang], Zhi, X.[Xiang], Wu, J.W.[Ji-Wen], Xu, Y.J.[Yong-Jun], Zhang, Q.[Qian],
MixSKD: Self-Knowledge Distillation from Mixup for Image Recognition,
ECCV22(XXIV:534-551).
Springer DOI 2211
BibRef

Xu, S.[Sheng], Li, Y.J.[Yan-Jing], Zeng, B.[Bohan], Ma, T.[Teli], Zhang, B.C.[Bao-Chang], Cao, X.B.[Xian-Bin], Gao, P.[Peng], Lü, J.[Jinhu],
IDa-Det: An Information Discrepancy-Aware Distillation for 1-Bit Detectors,
ECCV22(XI:346-361).
Springer DOI 2211
BibRef

Shin, S.[Sungho], Lee, J.[Joosoon], Lee, J.[Junseok], Yu, Y.[Yeonguk], Lee, K.[Kyoobin],
Teaching Where to Look: Attention Similarity Knowledge Distillation for Low Resolution Face Recognition,
ECCV22(XII:631-647).
Springer DOI 2211
BibRef

Gao, Y.T.[Yu-Ting], Zhuang, J.X.[Jia-Xin], Lin, S.H.[Shao-Hui], Cheng, H.[Hao], Sun, X.[Xing], Li, K.[Ke], Shen, C.H.[Chun-Hua],
DisCo: Remedying Self-supervised Learning on Lightweight Models with Distilled Contrastive Learning,
ECCV22(XXVI:237-253).
Springer DOI 2211
BibRef

Liu, H.[Hao], Ye, M.[Mang],
Improving Self-supervised Lightweight Model Learning via Hard-Aware Metric Distillation,
ECCV22(XXXI:295-311).
Springer DOI 2211
BibRef

Deng, X.Q.[Xue-Qing], Sun, D.W.[Da-Wei], Newsam, S.[Shawn], Wang, P.[Peng],
DistPro: Searching a Fast Knowledge Distillation Process via Meta Optimization,
ECCV22(XXXIV:218-235).
Springer DOI 2211
BibRef

Deng, X.[Xiang], Zheng, J.[Jian], Zhang, Z.F.[Zhong-Fei],
Personalized Education: Blind Knowledge Distillation,
ECCV22(XXXIV:269-285).
Springer DOI 2211
BibRef

Liu, L.Z.[Lyu-Zhuang], Hirakawa, T.[Tsubasa], Yamashita, T.[Takayoshi], Fujiyoshi, H.[Hironobu],
Class-Wise FM-NMS for Knowledge Distillation of Object Detection,
ICIP22(1641-1645)
IEEE DOI 2211
Computational modeling, Object detection, Feature extraction, Computational efficiency, Object detection, Feature map non-maximum suppression BibRef

Shu, H.Q.[Hong-Qiao],
Two Distillation Perspectives Based on Tanimoto Coefficient,
ICIP22(1311-1315)
IEEE DOI 2211
Training, Length measurement, Task analysis, Knowledge transfer, Knowledge distillation, Tanimoto similarity matrix, Tanimoto coefficient BibRef

Wang, H.L.[Hong-Lie], Xu, J.[Jian], Sun, S.Q.[Shou-Qian],
Distilling DETR-Like Detectors with Instance-Aware Feature,
ICIP22(1186-1190)
IEEE DOI 2211
Training, Head, Fuses, Detectors, Object detection, Transformers, Feature extraction, Object Detection, Knowledge Distillation, Attention Mechanism BibRef

Pei, W.J.[Wen-Jie], Wu, S.[Shuang], Mei, D.[Dianwen], Chen, F.L.[Fang-Lin], Tian, J.[Jiandong], Lu, G.M.[Guang-Ming],
Few-Shot Object Detection by Knowledge Distillation Using Bag-of-Visual-Words Representations,
ECCV22(X:283-299).
Springer DOI 2211
BibRef

Li, C.X.[Chen-Xin], Lin, M.[Mingbao], Ding, Z.Y.[Zhi-Yuan], Lin, N.[Nie], Zhuang, Y.H.[Yi-Hong], Huang, Y.[Yue], Ding, X.H.[Xing-Hao], Cao, L.J.[Liu-Juan],
Knowledge Condensation Distillation,
ECCV22(XI:19-35).
Springer DOI 2211
BibRef

Yang, Z.D.[Zhen-Dong], Li, Z.[Zhe], Shao, M.Q.[Ming-Qi], Shi, D.[Dachuan], Yuan, Z.H.[Ze-Huan], Yuan, C.[Chun],
Masked Generative Distillation,
ECCV22(XI:53-69).
Springer DOI 2211
BibRef

Liang, J.J.[Jia-Jun], Li, L.[Linze], Bing, Z.[Zhaodong], Zhao, B.[Borui], Tang, Y.[Yao], Lin, B.[Bo], Fan, H.Q.[Hao-Qiang],
Efficient One Pass Self-distillation with Zipf's Label Smoothing,
ECCV22(XI:104-119).
Springer DOI 2211
BibRef

Park, J.[Jinhyuk], No, A.[Albert],
Prune Your Model Before Distill It,
ECCV22(XI:120-136).
Springer DOI 2211
BibRef

Qian, B.[Biao], Wang, Y.[Yang], Yin, H.Z.[Hong-Zhi], Hong, R.C.[Ri-Chang], Wang, M.[Meng],
Switchable Online Knowledge Distillation,
ECCV22(XI:449-466).
Springer DOI 2211
BibRef

Okamoto, N.[Naoki], Hirakawa, T.[Tsubasa], Yamashita, T.[Takayoshi], Fujiyoshi, H.[Hironobu],
Deep Ensemble Learning by Diverse Knowledge Distillation for Fine-Grained Object Classification,
ECCV22(XI:502-518).
Springer DOI 2211
BibRef

Jandial, S.[Surgan], Khasbage, Y.[Yash], Pal, A.[Arghya], Balasubramanian, V.N.[Vineeth N.], Krishnamurthy, B.[Balaji],
Distilling the Undistillable: Learning from a Nasty Teacher,
ECCV22(XIII:587-603).
Springer DOI 2211
BibRef

Jang, Y.K.[Young Kyun], Gu, G.[Geonmo], Ko, B.[Byungsoo], Kang, I.[Isaac], Cho, N.I.[Nam Ik],
Deep Hash Distillation for Image Retrieval,
ECCV22(XIV:354-371).
Springer DOI 2211
BibRef

Nguyen, D.[Dang], Gupta, S.I.[Sun-Il], Do, K.[Kien], Venkatesh, S.[Svetha],
Black-Box Few-Shot Knowledge Distillation,
ECCV22(XXI:196-211).
Springer DOI 2211
BibRef

Oh, Y.J.[Yu-Jin], Ye, J.C.[Jong Chul],
CXR Segmentation by AdaIN-Based Domain Adaptation and Knowledge Distillation,
ECCV22(XXI:627-643).
Springer DOI 2211
BibRef

Lee, K.[Kyungmoon], Kim, S.[Sungyeon], Kwak, S.[Suha],
Cross-domain Ensemble Distillation for Domain Generalization,
ECCV22(XXV:1-20).
Springer DOI 2211
BibRef

Li, L.[Lujun],
Self-Regulated Feature Learning via Teacher-free Feature Distillation,
ECCV22(XXVI:347-363).
Springer DOI 2211
BibRef

Zhao, S.[Shiji], Yu, J.[Jie], Sun, Z.L.[Zhen-Long], Zhang, B.[Bo], Wei, X.X.[Xing-Xing],
Enhanced Accuracy and Robustness via Multi-teacher Adversarial Distillation,
ECCV22(IV:585-602).
Springer DOI 2211
BibRef

Li, J.C.[Jun-Cheng], Yang, H.[Hanhui], Yi, Q.[Qiaosi], Fang, F.[Faming], Gao, G.W.[Guang-Wei], Zeng, T.Y.[Tie-Yong], Zhang, G.X.[Gui-Xu],
Multiple Degradation and Reconstruction Network for Single Image Denoising via Knowledge Distillation,
NTIRE22(557-566)
IEEE DOI 2210
Degradation, Knowledge engineering, Computational modeling, Resists, Pattern recognition, Image restoration, Noise measurement BibRef

He, R.F.[Rui-Fei], Sun, S.Y.[Shu-Yang], Yang, J.H.[Ji-Han], Bai, S.[Song], Qi, X.J.[Xiao-Juan],
Knowledge Distillation as Efficient Pre-training: Faster Convergence, Higher Data-efficiency, and Better Transferability,
CVPR22(9151-9161)
IEEE DOI 2210
Training, Codes, Computational modeling, Computer architecture, Data models, Pattern recognition, Efficient learning and inferences BibRef

Beyer, L.[Lucas], Zhai, X.H.[Xiao-Hua], Royer, A.[Amélie], Markeeva, L.[Larisa], Anil, R.[Rohan], Kolesnikov, A.[Alexander],
Knowledge distillation: A good teacher is patient and consistent,
CVPR22(10915-10924)
IEEE DOI 2210
Training, Manifolds, Schedules, Image coding, Computational modeling, Data models, Deep learning architectures and techniques, Representation learning BibRef

Xie, P.T.[Peng-Tao], Du, X.F.[Xue-Feng],
Performance-Aware Mutual Knowledge Distillation for Improving Neural Architecture Search,
CVPR22(11912-11922)
IEEE DOI 2210
Computational modeling, Computer architecture, Pattern recognition, Optimization, Deep learning architectures and techniques BibRef

Chen, D.F.[De-Fang], Mei, J.P.[Jian-Ping], Zhang, H.L.[Hai-Lin], Wang, C.[Can], Feng, Y.[Yan], Chen, C.[Chun],
Knowledge Distillation with the Reused Teacher Classifier,
CVPR22(11923-11932)
IEEE DOI 2210
Costs, Computational modeling, Computer architecture, Feature extraction, Pattern recognition, Deep learning architectures and techniques BibRef

Shen, Y.Q.[Yi-Qing], Xu, L.[Liwu], Yang, Y.Z.[Yu-Zhe], Li, Y.Q.[Ya-Qian], Guo, Y.D.[Yan-Dong],
Self-Distillation from the Last Mini-Batch for Consistency Regularization,
CVPR22(11933-11942)
IEEE DOI 2210
Training, Codes, Computer network reliability, Memory management, Network architecture, Benchmark testing, Machine learning BibRef

Zhao, B.[Borui], Cui, Q.[Quan], Song, R.J.[Ren-Jie], Qiu, Y.[Yiyu], Liang, J.J.[Jia-Jun],
Decoupled Knowledge Distillation,
CVPR22(11943-11952)
IEEE DOI 2210
Training, Deep learning, Codes, Object detection, Computer architecture, Feature extraction, retrieval BibRef

Chen, X.N.[Xia-Ning], Cao, Q.[Qiong], Zhong, Y.J.[Yu-Jie], Zhang, J.[Jing], Gao, S.H.[Sheng-Hua], Tao, D.C.[Da-Cheng],
DearKD: Data-Efficient Early Knowledge Distillation for Vision Transformers,
CVPR22(12042-12052)
IEEE DOI 2210
Training, Deep learning, Computational modeling, Optimization methods, Computer architecture, Transformers, Optimization methods BibRef

Yang, C.G.[Chuan-Guang], Zhou, H.[Helong], An, Z.[Zhulin], Jiang, X.[Xue], Xu, Y.J.[Yong-Jun], Zhang, Q.[Qian],
Cross-Image Relational Knowledge Distillation for Semantic Segmentation,
CVPR22(12309-12318)
IEEE DOI 2210
Image segmentation, Correlation, Codes, Shape, Semantics, Pattern recognition, Efficient learning and inferences, grouping and shape analysis BibRef

Lin, S.[Sihao], Xie, H.W.[Hong-Wei], Wang, B.[Bing], Yu, K.C.[Kai-Cheng], Chang, X.J.[Xiao-Jun], Liang, X.D.[Xiao-Dan], Wang, G.[Gang],
Knowledge Distillation via the Target-aware Transformer,
CVPR22(10905-10914)
IEEE DOI 2210
Knowledge engineering, Deep learning, Codes, Semantics, Neural networks, Computer architecture, retrieval BibRef

Yang, Z.D.[Zhen-Dong], Li, Z.[Zhe], Jiang, X.H.[Xiao-Hu], Gong, Y.[Yuan], Yuan, Z.H.[Ze-Huan], Zhao, D.[Danpei], Yuan, C.[Chun],
Focal and Global Knowledge Distillation for Detectors,
CVPR22(4633-4642)
IEEE DOI 2210
Codes, Detectors, Object detection, Feature extraction, Pattern recognition, Image classification, Efficient learning and inferences BibRef

Mal, Z.Y.[Zong-Yang], Luo, G.[Guan], Gao, J.[Jin], Li, L.[Liang], Chen, Y.X.[Yu-Xin], Wang, S.[Shaoru], Zhang, C.X.[Cong-Xuan], Hu, W.M.[Wei-Ming],
Open-Vocabulary One-Stage Detection with Hierarchical Visual-Language Knowledge Distillation,
CVPR22(14054-14063)
IEEE DOI 2210
Training, Degradation, Vocabulary, Visualization, Semantics, Detectors, Object detection, Recognition: detection, categorization, Vision + language BibRef

Li, T.H.[Tian-Hao], Wang, L.M.[Li-Min], Wu, G.S.[Gang-Shan],
Self Supervision to Distillation for Long-Tailed Visual Recognition,
ICCV21(610-619)
IEEE DOI 2203
Training, Representation learning, Deep learning, Visualization, Image recognition, Head, Semantics, Recognition and classification, Representation learning BibRef

Fang, Z.Y.[Zhi-Yuan], Wang, J.F.[Jian-Feng], Hu, X.W.[Xiao-Wei], Wang, L.J.[Li-Juan], Yang, Y.Z.[Ye-Zhou], Liu, Z.C.[Zi-Cheng],
Compressing Visual-linguistic Model via Knowledge Distillation,
ICCV21(1408-1418)
IEEE DOI 2203
Knowledge engineering, Visualization, Adaptation models, Detectors, Mean square error methods, Transformers, Vision + language, Vision applications and systems BibRef

Yao, L.W.[Le-Wei], Pi, R.J.[Ren-Jie], Xu, H.[Hang], Zhang, W.[Wei], Li, Z.G.[Zhen-Guo], Zhang, T.[Tong],
G-DetKD: Towards General Distillation Framework for Object Detectors via Contrastive and Semantic-Guided Feature Imitation,
ICCV21(3571-3580)
IEEE DOI 2203
Semantics, Pipelines, Detectors, Object detection, Benchmark testing, Feature extraction, Detection and localization in 2D and 3D, BibRef

Chen, Y.X.[Yi-Xin], Chen, P.G.[Peng-Guang], Liu, S.[Shu], Wang, L.W.[Li-Wei], Jia, J.Y.[Jia-Ya],
Deep Structured Instance Graph for Distilling Object Detectors,
ICCV21(4339-4348)
IEEE DOI 2203
Codes, Image edge detection, Semantics, Detectors, Object detection, Knowledge representation, Detection and localization in 2D and 3D BibRef

Zhu, Y.C.[Yi-Chen], Wang, Y.[Yi],
Student Customized Knowledge Distillation: Bridging the Gap Between Student and Teacher,
ICCV21(5037-5046)
IEEE DOI 2203
Knowledge engineering, Training, Visualization, Image segmentation, Semantics, Object detection, BibRef

Kim, Y.[Youmin], Park, J.[Jinbae], Jang, Y.[YounHo], Ali, M.[Muhammad], Oh, T.H.[Tae-Hyun], Bae, S.H.[Sung-Ho],
Distilling Global and Local Logits with Densely Connected Relations,
ICCV21(6270-6280)
IEEE DOI 2203
Image segmentation, Image recognition, Computational modeling, Semantics, Object detection, Task analysis, BibRef

Kim, K.[Kyungyul], Ji, B.[ByeongMoon], Yoon, D.[Doyoung], Hwang, S.[Sangheum],
Self-Knowledge Distillation with Progressive Refinement of Targets,
ICCV21(6547-6556)
IEEE DOI 2203
Training, Knowledge engineering, Adaptation models, Supervised learning, Neural networks, Object detection, Recognition and classification BibRef

Son, W.[Wonchul], Na, J.[Jaemin], Choi, J.[Junyong], Hwang, W.J.[Won-Jun],
Densely Guided Knowledge Distillation using Multiple Teacher Assistants,
ICCV21(9375-9384)
IEEE DOI 2203
Knowledge engineering, Training, Deep learning, Transfer learning, Neural networks, Stochastic processes, Recognition and classification BibRef

Tejankar, A.[Ajinkya], Koohpayegani, S.A.[Soroush Abbasi], Pillai, V.[Vipin], Favaro, P.[Paolo], Pirsiavash, H.[Hamed],
ISD: Self-Supervised Learning by Iterative Similarity Distillation,
ICCV21(9589-9598)
IEEE DOI 2203
Codes, Transfer learning, Iterative methods, Task analysis, Standards, Representation learning, Transfer/Low-shot/Semi/Unsupervised Learning BibRef

Zhou, S.[Sheng], Wang, Y.C.[Yu-Cheng], Chen, D.F.[De-Fang], Chen, J.W.[Jia-Wei], Wang, X.[Xin], Wang, C.[Can], Bu, J.J.[Jia-Jun],
Distilling Holistic Knowledge with Graph Neural Networks,
ICCV21(10367-10376)
IEEE DOI 2203
Knowledge engineering, Correlation, Codes, Knowledge based systems, Benchmark testing, Feature extraction, BibRef

Shang, Y.Z.[Yu-Zhang], Duan, B.[Bin], Zong, Z.L.[Zi-Liang], Nie, L.Q.[Li-Qiang], Yan, Y.[Yan],
Lipschitz Continuity Guided Knowledge Distillation,
ICCV21(10655-10664)
IEEE DOI 2203
Knowledge engineering, Training, Image segmentation, Codes, NP-hard problem, Neural networks, Transfer/Low-shot/Semi/Unsupervised Learning BibRef

Li, Z.[Zheng], Ye, J.W.[Jing-Wen], Song, M.L.[Ming-Li], Huang, Y.[Ying], Pan, Z.[Zhigeng],
Online Knowledge Distillation for Efficient Pose Estimation,
ICCV21(11720-11730)
IEEE DOI 2203
Heating systems, Computational modeling, Pose estimation, Benchmark testing, Complexity theory, Knowledge transfer, Efficient training and inference methods BibRef

Dai, R.[Rui], Das, S.[Srijan], Bremond, F.[François],
Learning an Augmented RGB Representation with Cross-Modal Knowledge Distillation for Action Detection,
ICCV21(13033-13044)
IEEE DOI 2203
Training, Focusing, Streaming media, Real-time systems, Task analysis, Action and behavior recognition, Vision + other modalities BibRef

Xiang, S.[Sitao], Gu, Y.M.[Yu-Ming], Xiang, P.D.[Peng-Da], Chai, M.L.[Meng-Lei], Li, H.[Hao], Zhao, Y.J.[Ya-Jie], He, M.M.[Ming-Ming],
DisUnknown: Distilling Unknown Factors for Disentanglement Learning,
ICCV21(14790-14799)
IEEE DOI 2203
Training, Scalability, Benchmark testing, Generators, Task analysis, Image and video synthesis, Adversarial learning, Neural generative models BibRef

Diomataris, M.[Markos], Gkanatsios, N.[Nikolaos], Pitsikalis, V.[Vassilis], Maragos, P.[Petros],
Grounding Consistency: Distilling Spatial Common Sense for Precise Visual Relationship Detection,
ICCV21(15891-15900)
IEEE DOI 2203
Measurement, Visualization, Grounding, Triples (Data structure), Image edge detection, Predictive models, Visual reasoning and logical representation BibRef

Zi, B.[Bojia], Zhao, S.H.[Shi-Hao], Ma, X.J.[Xing-Jun], Jiang, Y.G.[Yu-Gang],
Revisiting Adversarial Robustness Distillation: Robust Soft Labels Make Student Better,
ICCV21(16423-16432)
IEEE DOI 2203
Training, Deep learning, Codes, Computational modeling, Neural networks, Predictive models, Adversarial learning, Recognition and classification BibRef

Zheng, H.[Heliang], Yang, H.[Huan], Fu, J.L.[Jian-Long], Zha, Z.J.[Zheng-Jun], Luo, J.B.[Jie-Bo],
Learning Conditional Knowledge Distillation for Degraded-Reference Image Quality Assessment,
ICCV21(10222-10231)
IEEE DOI 2203
Measurement, Image quality, Training, Knowledge engineering, Computational modeling, Semantics, Image restoration, Low-level and physics-based vision BibRef

Liu, L.[Li], Huang, Q.[Qingle], Lin, S.[Sihao], Xie, H.W.[Hong-Wei], Wang, B.[Bing], Chang, X.J.[Xiao-Jun], Liang, X.D.[Xiao-Dan],
Exploring Inter-Channel Correlation for Diversity-preserved Knowledge Distillation,
ICCV21(8251-8260)
IEEE DOI 2203
Knowledge engineering, Image segmentation, Correlation, Costs, Semantics, Graphics processing units, grouping and shape BibRef

Wang, H.[Hong], Deng, Y.F.[Yue-Fan], Yoo, S.[Shinjae], Ling, H.B.[Hai-Bin], Lin, Y.W.[Yue-Wei],
AGKD-BML: Defense Against Adversarial Attack by Attention Guided Knowledge Distillation and Bi-directional Metric Learning,
ICCV21(7638-7647)
IEEE DOI 2203
Training, Deep learning, Codes, Computational modeling, Neural networks, Bidirectional control, Adversarial learning, BibRef

Li, C.C.[Cheng-Cheng], Wang, Z.[Zi], Qi, H.R.[Hai-Rong],
Online Knowledge Distillation by Temporal-Spatial Boosting,
WACV22(3482-3491)
IEEE DOI 2202
Training, Knowledge engineering, Benchmark testing, Boosting, Noise measurement, Deep Learning Deep Learning -> Efficient Training and Inference Methods for Networks BibRef

Zheng, Z.Z.[Zhen-Zhu], Peng, X.[Xi],
Self-Guidance: Improve Deep Neural Network Generalization via Knowledge Distillation,
WACV22(3451-3460)
IEEE DOI 2202
Training, Deep learning, Knowledge engineering, Measurement, Visualization, Image recognition, Neural networks, Learning and Optimization BibRef

Zhang, H.[Heng], Fromont, E.[Elisa], Lefevre, S.[Sébastien], Avignon, B.[Bruno],
Low-cost Multispectral Scene Analysis with Modality Distillation,
WACV22(3331-3340)
IEEE DOI 2202
Knowledge engineering, Image analysis, Image resolution, Semantics, Neural networks, Thermal sensors, Predictive models, Vision Systems and Applications BibRef

Vo, D.M.[Duc Minh], Sugimoto, A.[Akihiro], Nakayama, H.[Hideki],
PPCD-GAN: Progressive Pruning and Class-Aware Distillation for Large-Scale Conditional GANs Compression,
WACV22(1422-1430)
IEEE DOI 2202
Training, Image coding, Neural network compression, Computer architecture, GANs BibRef

Kobayashi, T.[Takumi],
Extractive Knowledge Distillation,
WACV22(1350-1359)
IEEE DOI 2202
Temperature distribution, Analytical models, Annotations, Transfer learning, Feature extraction, Task analysis, Deep Learning Object Detection/Recognition/Categorization BibRef

Nguyen, C.H.[Chuong H.], Nguyen, T.C.[Thuy C.], Tang, T.N.[Tuan N.], Phan, N.L.H.[Nam L. H.],
Improving Object Detection by Label Assignment Distillation,
WACV22(1322-1331)
IEEE DOI 2202
Training, Schedules, Costs, Force, Object detection, Detectors, Switches, Object Detection/Recognition/Categorization BibRef

Meng, Z.[Ze], Yao, X.[Xin], Sun, L.F.[Li-Feng],
Multi-Task Distillation: Towards Mitigating the Negative Transfer in Multi-Task Learning,
ICIP21(389-393)
IEEE DOI 2201
Training, Degradation, Image processing, Optimization methods, Benchmark testing, Turning, Multi-task Learning, Multi-objective optimization BibRef

Tang, Q.[Qiankun], Xu, X.G.[Xiao-Gang], Wang, J.[Jun],
Differentiable Dynamic Channel Association for Knowledge Distillation,
ICIP21(414-418)
IEEE DOI 2201
Image coding, Computational modeling, Network architecture, Probabilistic logic, Computational efficiency, Task analysis, weighted distillation BibRef

Tran, V.[Vinh], Wang, Y.[Yang], Zhang, Z.K.[Ze-Kun], Hoai, M.[Minh],
Knowledge Distillation for Human Action Anticipation,
ICIP21(2518-2522)
IEEE DOI 2201
Training, Knowledge engineering, Image processing, Semantics, Neural networks, Training data BibRef

Tran, V.[Vinh], Balasubramanian, N.[Niranjan], Hoai, M.[Minh],
Progressive Knowledge Distillation for Early Action Recognition,
ICIP21(2583-2587)
IEEE DOI 2201
Knowledge engineering, Training, Recurrent neural networks, Image recognition, Training data, Semisupervised learning BibRef

Rotman, M.[Michael], Wolf, L.B.[Lior B.],
Natural Statistics of Network Activations and Implications for Knowledge Distillation,
ICIP21(399-403)
IEEE DOI 2201
Deep learning, Knowledge engineering, Image recognition, Correlation, Semantics, Benchmark testing, Knowledge Distillation, Image Statistics BibRef

Banitalebi-Dehkordi, A.[Amin],
Knowledge Distillation for Low-Power Object Detection: A Simple Technique and Its Extensions for Training Compact Models Using Unlabeled Data,
LPCV21(769-778)
IEEE DOI 2112
Training, Adaptation models, Computational modeling, Object detection, Computer architecture BibRef

Zhu, J.[Jinguo], Tang, S.X.[Shi-Xiang], Chen, D.P.[Da-Peng], Yu, S.J.[Shi-Jie], Liu, Y.K.[Ya-Kun], Rong, M.Z.[Ming-Zhe], Yang, A.[Aijun], Wang, X.H.[Xiao-Hua],
Complementary Relation Contrastive Distillation,
CVPR21(9256-9265)
IEEE DOI 2111
Benchmark testing, Pattern recognition, Mutual information BibRef

Jung, S.[Sangwon], Lee, D.G.[Dong-Gyu], Park, T.[Taeeon], Moon, T.[Taesup],
Fair Feature Distillation for Visual Recognition,
CVPR21(12110-12119)
IEEE DOI 2111
Visualization, Systematics, Computational modeling, Face recognition, Predictive models, Prediction algorithms BibRef

Ghosh, P.[Pallabi], Saini, N.[Nirat], Davis, L.S.[Larry S.], Shrivastava, A.[Abhinav],
Learning Graphs for Knowledge Transfer with Limited Labels,
CVPR21(11146-11156)
IEEE DOI 2111
Training, Visualization, Convolution, Semisupervised learning, Benchmark testing, Pattern recognition BibRef

Huang, Z.[Zhen], Shen, X.[Xu], Xing, J.[Jun], Liu, T.L.[Tong-Liang], Tian, X.M.[Xin-Mei], Li, H.Q.[Hou-Qiang], Deng, B.[Bing], Huang, J.Q.[Jian-Qiang], Hua, X.S.[Xian-Sheng],
Revisiting Knowledge Distillation: An Inheritance and Exploration Framework,
CVPR21(3578-3587)
IEEE DOI 2111
Training, Learning systems, Knowledge engineering, Deep learning, Neural networks, Reinforcement learning BibRef

Chen, P.G.[Peng-Guang], Liu, S.[Shu], Zhao, H.S.[Heng-Shuang], Jia, J.Y.[Jia-Ya],
Distilling Knowledge via Knowledge Review,
CVPR21(5006-5015)
IEEE DOI 2111
Knowledge engineering, Object detection, Pattern recognition, Task analysis BibRef

Ji, M.[Mingi], Shin, S.J.[Seung-Jae], Hwang, S.H.[Seung-Hyun], Park, G.[Gibeom], Moon, I.C.[Il-Chul],
Refine Myself by Teaching Myself: Feature Refinement via Self-Knowledge Distillation,
CVPR21(10659-10668)
IEEE DOI 2111
Knowledge engineering, Training, Codes, Semantics, Neural networks, Object detection BibRef

Salehi, M.[Mohammadreza], Sadjadi, N.[Niousha], Baselizadeh, S.[Soroosh], Rohban, M.H.[Mohammad H.], Rabiee, H.R.[Hamid R.],
Multiresolution Knowledge Distillation for Anomaly Detection,
CVPR21(14897-14907)
IEEE DOI 2111
Training, Location awareness, Knowledge engineering, Image resolution, Pattern recognition, Task analysis BibRef

Haselhoff, A.[Anselm], Kronenberger, J.[Jan], Küppers, F.[Fabian], Schneider, J.[Jonas],
Towards Black-Box Explainability with Gaussian Discriminant Knowledge Distillation,
SAIAD21(21-28)
IEEE DOI 2109
Visualization, Shape, Semantics, Training data, Object detection, Predictive models, Linear programming BibRef

Yang, L.[Lehan], Xu, K.[Kele],
Cross Modality Knowledge Distillation for Multi-modal Aerial View Object Classification,
NTIRE21(382-387)
IEEE DOI 2109
Training, Speckle, Feature extraction, Radar polarimetry, Data models, Robustness, Pattern recognition BibRef

Bhat, P.[Prashant], Arani, E.[Elahe], Zonooz, B.[Bahram],
Distill on the Go: Online knowledge distillation in self-supervised learning,
LLID21(2672-2681)
IEEE DOI 2109
Annotations, Computer architecture, Performance gain, Benchmark testing, Pattern recognition BibRef

Okuno, T.[Tomoyuki], Nakata, Y.[Yohei], Ishii, Y.[Yasunori], Tsukizawa, S.[Sotaro],
Lossless AI: Toward Guaranteeing Consistency between Inferences Before and After Quantization via Knowledge Distillation,
MVA21(1-5)
DOI Link 2109
Training, Quality assurance, Quantization (signal), Object detection, Network architecture, Real-time systems BibRef

Nayak, G.K.[Gaurav Kumar], Mopuri, K.R.[Konda Reddy], Chakraborty, A.[Anirban],
Effectiveness of Arbitrary Transfer Sets for Data-free Knowledge Distillation,
WACV21(1429-1437)
IEEE DOI 2106
Training, Visualization, Sensitivity, Computational modeling, Semantics, Neural networks, Training data BibRef

Lee, J.[Jongmin], Jeong, Y.[Yoonwoo], Kim, S.[Seungwook], Min, J.[Juhong], Cho, M.[Minsu],
Learning to Distill Convolutional Features into Compact Local Descriptors,
WACV21(897-907)
IEEE DOI 2106
Location awareness, Visualization, Image matching, Semantics, Benchmark testing, Feature extraction, Robustness BibRef

Arani, E.[Elahe], Sarfraz, F.[Fahad], Zonooz, B.[Bahram],
Noise as a Resource for Learning in Knowledge Distillation,
WACV21(3128-3137)
IEEE DOI 2106
Training, Uncertainty, Neuroscience, Collaboration, Collaborative work, Brain modeling, Probabilistic logic BibRef

Chawla, A.[Akshay], Yin, H.X.[Hong-Xu], Molchanov, P.[Pavlo], Alvarez, J.[Jose],
Data-free Knowledge Distillation for Object Detection,
WACV21(3288-3297)
IEEE DOI 2106
Knowledge engineering, Training, Image synthesis, Neural networks, Object detection BibRef

Kothandaraman, D.[Divya], Nambiar, A.[Athira], Mittal, A.[Anurag],
Domain Adaptive Knowledge Distillation for Driving Scene Semantic Segmentation,
WACVW21(134-143) Autonomous Vehicle Vision
IEEE DOI 2105
Knowledge engineering, Adaptation models, Image segmentation, Semantics, Memory management BibRef

Kushawaha, R.K.[Ravi Kumar], Kumar, S.[Saurabh], Banerjee, B.[Biplab], Velmurugan, R.[Rajbabu],
Distilling Spikes: Knowledge Distillation in Spiking Neural Networks,
ICPR21(4536-4543)
IEEE DOI 2105
Knowledge engineering, Training, Image coding, Computational modeling, Artificial neural networks, Hardware BibRef

Sarfraz, F.[Fahad], Arani, E.[Elahe], Zonooz, B.[Bahram],
Knowledge Distillation Beyond Model Compression,
ICPR21(6136-6143)
IEEE DOI 2105
Training, Knowledge engineering, Neural networks, Network architecture, Collaborative work, Robustness BibRef

Ahmed, W.[Waqar], Zunino, A.[Andrea], Morerio, P.[Pietro], Murino, V.[Vittorio],
Compact CNN Structure Learning by Knowledge Distillation,
ICPR21(6554-6561)
IEEE DOI 2105
Training, Learning systems, Knowledge engineering, Network architecture, Predictive models BibRef

Ma, J.X.[Jia-Xin], Yonetani, R.[Ryo], Iqbal, Z.[Zahid],
Adaptive Distillation for Decentralized Learning from Heterogeneous Clients,
ICPR21(7486-7492)
IEEE DOI 2105
Learning systems, Adaptation models, Visualization, Biomedical equipment, Medical services, Collaborative work, Data models BibRef

Xu, Y.[Yi], Pu, J.[Jian], Zhao, H.[Hui],
Knowledge Distillation with a Precise Teacher and Prediction with Abstention,
ICPR21(9000-9006)
IEEE DOI 2105
Knowledge engineering, Supervised learning, Benchmark testing, Predictive models BibRef

Tsunashima, H.[Hideki], Kataoka, H.[Hirokatsu], Yamato, J.J.[Jun-Ji], Chen, Q.[Qiu], Morishima, S.[Shigeo],
Adversarial Knowledge Distillation for a Compact Generator,
ICPR21(10636-10643)
IEEE DOI 2105
Training, Image resolution, MIMICs, Generators BibRef

Zhang, Z.X.[Zhe-Xi], Zhu, W.[Wei], Yan, J.C.[Jun-Chi], Gao, P.[Peng], Xie, G.T.[Guo-Tong],
Automatic Student Network Search for Knowledge Distillation,
ICPR21(2446-2453)
IEEE DOI 2105
Knowledge engineering, Performance evaluation, Computational modeling, Bit error rate, Neural networks, Natural language processing BibRef

Kim, J.H.[Jang-Ho], Hyun, M.S.[Min-Sung], Chung, I.[Inseop], Kwak, N.[Nojun],
Feature Fusion for Online Mutual Knowledge Distillation,
ICPR21(4619-4625)
IEEE DOI 2105
Neural networks, Education, Performance gain, Pattern recognition BibRef

Mitsuno, K.[Kakeru], Nomura, Y.[Yuichiro], Kurita, T.[Takio],
Channel Planting for Deep Neural Networks using Knowledge Distillation,
ICPR21(7573-7579)
IEEE DOI 2105
Training, Knowledge engineering, Heuristic algorithms, Neural networks, Computer architecture, Network architecture BibRef

Finogeev, E., Gorbatsevich, V., Moiseenko, A., Vizilter, Y., Vygolov, O.,
Knowledge Distillation Using GANs for Fast Object Detection,
ISPRS20(B2:583-588).
DOI Link 2012
BibRef

Cui, W., Li, X., Huang, J., Wang, W., Wang, S., Chen, J.,
Substitute Model Generation for Black-Box Adversarial Attack Based on Knowledge Distillation,
ICIP20(648-652)
IEEE DOI 2011
Perturbation methods, Task analysis, Training, Computational modeling, Approximation algorithms, black-box models BibRef

Xu, K.R.[Kun-Ran], Rui, L.[Lai], Li, Y.S.[Yi-Shi], Gu, L.[Lin],
Feature Normalized Knowledge Distillation for Image Classification,
ECCV20(XXV:664-680).
Springer DOI 2011
BibRef

Yang, Y., Qiu, J., Song, M., Tao, D., Wang, X.,
Distilling Knowledge From Graph Convolutional Networks,
CVPR20(7072-7081)
IEEE DOI 2008
Knowledge engineering, Task analysis, Computational modeling, Computer science, Training, Neural networks BibRef

Yun, J.S.[Ju-Seung], Kim, B.[Byungjoo], Kim, J.[Junmo],
Weight Decay Scheduling and Knowledge Distillation for Active Learning,
ECCV20(XXVI:431-447).
Springer DOI 2011
BibRef

Li, C.L.[Chang-Lin], Tang, T.[Tao], Wang, G.[Guangrun], Peng, J.F.[Jie-Feng], Wang, B.[Bing], Liang, X.D.[Xiao-Dan], Chang, X.J.[Xiao-Jun],
BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search,
ICCV21(12261-12271)
IEEE DOI 2203
Training, Visualization, Correlation, Architecture, Computational modeling, Sociology, Computer architecture, Representation learning BibRef

Li, C.L.[Chang-Lin], Peng, J.F.[Jie-Feng], Yuan, L.C.[Liu-Chun], Wang, G.R.[Guang-Run], Liang, X.D.[Xiao-Dan], Lin, L.[Liang], Chang, X.J.[Xiao-Jun],
Block-Wisely Supervised Neural Architecture Search With Knowledge Distillation,
CVPR20(1986-1995)
IEEE DOI 2008
Computer architecture, Network architecture, Knowledge engineering, Training, DNA, Convergence, Feature extraction BibRef

Wei, L.H.[Long-Hui], Xiao, A.[An], Xie, L.X.[Ling-Xi], Zhang, X.P.[Xiao-Peng], Chen, X.[Xin], Tian, Q.[Qi],
Circumventing Outliers of Autoaugment with Knowledge Distillation,
ECCV20(III:608-625).
Springer DOI 2012
BibRef

Walawalkar, D.[Devesh], Shen, Z.Q.[Zhi-Qiang], Savvides, M.[Marios],
Online Ensemble Model Compression Using Knowledge Distillation,
ECCV20(XIX:18-35).
Springer DOI 2011
BibRef

Xiang, L.Y.[Liu-Yu], Ding, G.G.[Gui-Guang], Han, J.G.[Jun-Gong],
Learning From Multiple Experts: Self-paced Knowledge Distillation for Long-tailed Classification,
ECCV20(V:247-263).
Springer DOI 2011
BibRef

Zhou, B.[Brady], Kalra, N.[Nimit], Krähenbühl, P.[Philipp],
Domain Adaptation Through Task Distillation,
ECCV20(XXVI:664-680).
Springer DOI 2011
BibRef

Li, Z.[Zheng], Huang, Y.[Ying], Chen, D.F.[De-Fang], Luo, T.R.[Tian-Ren], Cai, N.[Ning], Pan, Z.G.[Zhi-Geng],
Online Knowledge Distillation via Multi-branch Diversity Enhancement,
ACCV20(IV:318-333).
Springer DOI 2103
BibRef

Ye, H.J.[Han-Jia], Lu, S.[Su], Zhan, D.C.[De-Chuan],
Distilling Cross-Task Knowledge via Relationship Matching,
CVPR20(12393-12402)
IEEE DOI 2008
Task analysis, Neural networks, Training, Knowledge engineering, Predictive models, Stochastic processes, Temperature measurement BibRef

Yao, A.B.[An-Bang], Sun, D.W.[Da-Wei],
Knowledge Transfer via Dense Cross-layer Mutual-distillation,
ECCV20(XV:294-311).
Springer DOI 2011
BibRef

Yue, K.Y.[Kai-Yu], Deng, J.F.[Jiang-Fan], Zhou, F.[Feng],
Matching Guided Distillation,
ECCV20(XV:312-328).
Springer DOI 2011
BibRef

Zhang, Y.C.[You-Cai], Lan, Z.H.[Zhong-Hao], Dai, Y.C.[Yu-Chen], Zeng, F.G.[Fan-Gao], Bai, Y.[Yan], Chang, J.[Jie], Wei, Y.C.[Yi-Chen],
Prime-aware Adaptive Distillation,
ECCV20(XIX:658-674).
Springer DOI 2011
Student-Teacher learning. BibRef

Xu, G.D.[Guo-Dong], Liu, Z.W.[Zi-Wei], Li, X.X.[Xiao-Xiao], Loy, C.C.[Chen Change],
Knowledge Distillation Meets Self-Supervision,
ECCV20(IX:588-604).
Springer DOI 2011
Extracting the dark knowledge from a teacher network to guide the learning of a student network, for transfer learning. BibRef

Li, X.J.[Xiao-Jie], Wu, J.L.[Jian-Long], Fang, H.Y.[Hong-Yu], Liao, Y.[Yue], Wang, F.[Fei], Qian, C.[Chen],
Local Correlation Consistency for Knowledge Distillation,
ECCV20(XII: 18-33).
Springer DOI 2010
Knowledge extraction from the teacher network plays a critical role in the knowledge distillation task to improve the performance of the student network. BibRef

Passalis, N.[Nikolaos], Tzelepi, M.[Maria], Tefas, A.[Anastasios],
Heterogeneous Knowledge Distillation Using Information Flow Modeling,
CVPR20(2336-2345)
IEEE DOI 2008
From complex teacher to smaller student. Training, Neural networks, Knowledge engineering, Data models, Convergence, Data mining, Transforms BibRef

Chen, Z.L.[Zai-Liang], Zheng, X.X.[Xian-Xian], Shen, H.L.[Hai-Lan], Zeng, Z.Y.[Zi-Yang], Zhou, Y.K.[Yu-Kun], Zhao, R.C.[Rong-Chang],
Improving Knowledge Distillation via Category Structure,
ECCV20(XXVIII:205-219).
Springer DOI 2011
Training student to mimic the teacher, but not capture the structure. BibRef

Wang, D.Y.[De-Yu], Wen, D.C.[Dong-Chao], Liu, J.J.[Jun-Jie], Tao, W.[Wei], Chen, T.W.[Tse-Wei], Osa, K.[Kinya], Kato, M.[Masami],
Fully Supervised and Guided Distillation for One-stage Detectors,
ACCV20(III:171-188).
Springer DOI 2103
BibRef

Itsumi, H., Beye, F., Shinohara, Y., Iwai, T.,
Training With Cache: Specializing Object Detectors From Live Streams Without Overfitting,
ICIP20(1976-1980)
IEEE DOI 2011
Training, Data models, Solid modeling, Adaptation models, Training data, Streaming media, Legged locomotion, Online training, Knowledge distillation BibRef

Liu, B.L.[Ben-Lin], Rao, Y.M.[Yong-Ming], Lu, J.W.[Ji-Wen], Zhou, J.[Jie], Hsieh, C.J.[Cho-Jui],
Metadistiller: Network Self-boosting via Meta-learned Top-down Distillation,
ECCV20(XIV:694-709).
Springer DOI 2011
BibRef

Choi, Y., Choi, J., El-Khamy, M., Lee, J.,
Data-Free Network Quantization With Adversarial Knowledge Distillation,
EDLCV20(3047-3057)
IEEE DOI 2008
Generators, Quantization (signal), Training, Computational modeling, Data models, Machine learning, Data privacy BibRef

de Vieilleville, F., Lagrange, A., Ruiloba, R., May, S.,
Towards Distillation of Deep Neural Networks for Satellite On-board Image Segmentation,
ISPRS20(B2:1553-1559).
DOI Link 2012
BibRef

Wang, X.B.[Xiao-Bo], Fu, T.Y.[Tian-Yu], Liao, S.C.[Sheng-Cai], Wang, S.[Shuo], Lei, Z.[Zhen], Mei, T.[Tao],
Exclusivity-Consistency Regularized Knowledge Distillation for Face Recognition,
ECCV20(XXIV:325-342).
Springer DOI 2012
BibRef

Guan, Y.S.[Yu-Shuo], Zhao, P.Y.[Peng-Yu], Wang, B.X.[Bing-Xuan], Zhang, Y.X.[Yuan-Xing], Yao, C.[Cong], Bian, K.G.[Kai-Gui], Tang, J.[Jian],
Differentiable Feature Aggregation Search for Knowledge Distillation,
ECCV20(XVII:469-484).
Springer DOI 2011
BibRef

Gu, J.D.[Jin-Dong], Wu, Z.L.[Zhi-Liang], Tresp, V.[Volker],
Introspective Learning by Distilling Knowledge from Online Self-explanation,
ACCV20(IV:36-52).
Springer DOI 2103
BibRef

Guo, Q.S.[Qiu-Shan], Wang, X.J.[Xin-Jiang], Wu, Y.C.[Yi-Chao], Yu, Z.P.[Zhi-Peng], Liang, D.[Ding], Hu, X.L.[Xiao-Lin], Luo, P.[Ping],
Online Knowledge Distillation via Collaborative Learning,
CVPR20(11017-11026)
IEEE DOI 2008
Knowledge engineering, Training, Collaborative work, Perturbation methods, Collaboration, Neural networks, Logic gates BibRef

Li, T., Li, J., Liu, Z., Zhang, C.,
Few Sample Knowledge Distillation for Efficient Network Compression,
CVPR20(14627-14635)
IEEE DOI 2008
Training, Tensile stress, Knowledge engineering, Convolution, Neural networks, Computational modeling, Standards BibRef

Wang, D., Li, Y., Wang, L., Gong, B.,
Neural Networks Are More Productive Teachers Than Human Raters: Active Mixup for Data-Efficient Knowledge Distillation From a Blackbox Model,
CVPR20(1495-1504)
IEEE DOI 2008
Neural networks, Computational modeling, Data models, Training, Knowledge engineering, Visualization, Manifolds BibRef

Farhadi, M.[Mohammad], Yang, Y.Z.[Ye-Zhou],
TKD: Temporal Knowledge Distillation for Active Perception,
WACV20(942-951)
IEEE DOI 2006
Code, Object Detection.
WWW Link. Temporal knowledge over NN applied over multiple frames. Adaptation models, Object detection, Visualization, Computational modeling, Task analysis, Training, Feature extraction BibRef

Seddik, M.E.A., Essafi, H., Benzine, A., Tamaazousti, M.,
Lightweight Neural Networks From PCA LDA Based Distilled Dense Neural Networks,
ICIP20(3060-3064)
IEEE DOI 2011
Neural networks, Principal component analysis, Computational modeling, Training, Machine learning, Lightweight Networks BibRef

Tung, F.[Fred], Mori, G.[Greg],
Similarity-Preserving Knowledge Distillation,
ICCV19(1365-1374)
IEEE DOI 2004
learning (artificial intelligence), neural nets, semantic networks, Task analysis BibRef

Zhang, M.Y.[Man-Yuan], Song, G.L.[Guang-Lu], Zhou, H.[Hang], Liu, Y.[Yu],
Discriminability Distillation in Group Representation Learning,
ECCV20(X:1-19).
Springer DOI 2011
BibRef

Jin, X.[Xiao], Peng, B.Y.[Bao-Yun], Wu, Y.C.[Yi-Chao], Liu, Y.[Yu], Liu, J.H.[Jia-Heng], Liang, D.[Ding], Yan, J.J.[Jun-Jie], Hu, X.L.[Xiao-Lin],
Knowledge Distillation via Route Constrained Optimization,
ICCV19(1345-1354)
IEEE DOI 2004
face recognition, image classification, learning (artificial intelligence), neural nets, optimisation, Neural networks BibRef

Mullapudi, R.T., Chen, S., Zhang, K., Ramanan, D., Fatahalian, K.,
Online Model Distillation for Efficient Video Inference,
ICCV19(3572-3581)
IEEE DOI 2004
convolutional neural nets, image segmentation, inference mechanisms, learning (artificial intelligence), Cameras BibRef

Zhang, L., Song, J., Gao, A., Chen, J., Bao, C., Ma, K.,
Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation,
ICCV19(3712-3721)
IEEE DOI 2004
convolutional neural nets, learning (artificial intelligence), knowledge distillation, student neural networks, Computational modeling BibRef

Cho, J.H., Hariharan, B.,
On the Efficacy of Knowledge Distillation,
ICCV19(4793-4801)
IEEE DOI 2004
learning (artificial intelligence), neural nets, Probability distribution, teacher architectures, knowledge distillation performance. BibRef

Peng, B., Jin, X., Li, D., Zhou, S., Wu, Y., Liu, J., Zhang, Z., Liu, Y.,
Correlation Congruence for Knowledge Distillation,
ICCV19(5006-5015)
IEEE DOI 2004
correlation methods, face recognition, image classification, learning (artificial intelligence), instance-level information, Knowledge transfer BibRef

Vongkulbhisal, J.[Jayakorn], Vinayavekhin, P.[Phongtharin], Visentini-Scarzanella, M.[Marco],
Unifying Heterogeneous Classifiers With Distillation,
CVPR19(3170-3179).
IEEE DOI 2002
BibRef

Yan, M., Zhao, M., Xu, Z., Zhang, Q., Wang, G., Su, Z.,
VarGFaceNet: An Efficient Variable Group Convolutional Neural Network for Lightweight Face Recognition,
LFR19(2647-2654)
IEEE DOI 2004
Code, Face Recognition.
WWW Link. convolutional neural nets, face recognition, learning (artificial intelligence), student model, teacher model, knowledge distillation BibRef

Yoshioka, K., Lee, E., Wong, S., Horowitz, M.,
Dataset Culling: Towards Efficient Training of Distillation-Based Domain Specific Models,
ICIP19(3237-3241)
IEEE DOI 1910
Object Detection, Training Efficiency, Distillation, Dataset Culling, Deep Learning BibRef

Yang, C.L.[Cheng-Lin], Xie, L.X.[Ling-Xi], Su, C.[Chi], Yuille, A.L.[Alan L.],
Snapshot Distillation: Teacher-Student Optimization in One Generation,
CVPR19(2854-2863).
IEEE DOI 2002
BibRef

Kundu, J.N., Lakkakula, N., Radhakrishnan, V.B.,
UM-Adapt: Unsupervised Multi-Task Adaptation Using Adversarial Cross-Task Distillation,
ICCV19(1436-1445)
IEEE DOI 2004
generalisation (artificial intelligence), image classification, object detection, unsupervised learning, task-transferability, Adaptation models BibRef

Park, W.[Wonpyo], Kim, D.J.[Dong-Ju], Lu, Y.[Yan], Cho, M.[Minsu],
Relational Knowledge Distillation,
CVPR19(3962-3971).
IEEE DOI 2002
BibRef

Liu, Y.F.[Yu-Fan], Cao, J.J.[Jia-Jiong], Li, B.[Bing], Yuan, C.F.[Chun-Feng], Hu, W.M.[Wei-Ming], Li, Y.X.[Yang-Xi], Duan, Y.Q.[Yun-Qiang],
Knowledge Distillation via Instance Relationship Graph,
CVPR19(7089-7097).
IEEE DOI 2002
BibRef

Ahn, S.S.[Sung-Soo], Hu, S.X.[Shell Xu], Damianou, A.[Andreas], Lawrence, N.D.[Neil D.], Dai, Z.W.[Zhen-Wen],
Variational Information Distillation for Knowledge Transfer,
CVPR19(9155-9163).
IEEE DOI 2002
BibRef

Minami, S.[Soma], Yamashita, T.[Takayoshi], Fujiyoshi, H.[Hironobu],
Gradual Sampling Gate for Bidirectional Knowledge Distillation,
MVA19(1-6)
DOI Link 1911
Transfer knowledge from large pre-trained network to smaller one. data compression, learning (artificial intelligence), neural nets, gradual sampling gate, Power markets BibRef

Chen, W.C.[Wei-Chun], Chang, C.C.[Chia-Che], Lee, C.R.[Che-Rung],
Knowledge Distillation with Feature Maps for Image Classification,
ACCV18(III:200-215).
Springer DOI 1906
BibRef

Hou, S.H.[Sai-Hui], Pan, X.Y.[Xin-Yu], Loy, C.C.[Chen Change], Wang, Z.L.[Zi-Lei], Lin, D.H.[Da-Hua],
Lifelong Learning via Progressive Distillation and Retrospection,
ECCV18(III: 452-467).
Springer DOI 1810
BibRef

Pintea, S.L.[Silvia L.], Liu, Y.[Yue], van Gemert, J.C.[Jan C.],
Recurrent Knowledge Distillation,
ICIP18(3393-3397)
IEEE DOI 1809
small network learns from larger network. Computational modeling, Memory management, Training, Color, Convolution, Road transportation, Knowledge distillation, recurrent layers BibRef

Lee, S.H.[Seung Hyun], Kim, D.H.[Dae Ha], Song, B.C.[Byung Cheol],
Self-supervised Knowledge Distillation Using Singular Value Decomposition,
ECCV18(VI: 339-354).
Springer DOI 1810
BibRef

Yim, J., Joo, D., Bae, J., Kim, J.,
A Gift from Knowledge Distillation: Fast Optimization, Network Minimization and Transfer Learning,
CVPR17(7130-7138)
IEEE DOI 1711
Feature extraction, Knowledge engineering, Knowledge transfer, Optimization, Training BibRef

Gupta, S.[Saurabh], Hoffman, J.[Judy], Malik, J.[Jitendra],
Cross Modal Distillation for Supervision Transfer,
CVPR16(2827-2836)
IEEE DOI 1612
BibRef

Chapter on Matching and Recognition Using Volumes, High Level Vision Techniques, Invariants continues in
Explainable Aritficial Intelligence .


Last update:May 23, 2024 at 14:31:23