LAPSE:2024.0237
Published Article
LAPSE:2024.0237
A Lightweight Identification Method for Complex Power Industry Tasks Based on Knowledge Distillation and Network Pruning
February 10, 2024
Lightweight service identification models are very important for resource-constrained distribution grid systems. To address the increasingly larger deep learning models, we provide a method for the lightweight identification of complex power services based on knowledge distillation and network pruning. Specifically, a pruning method based on Taylor expansion is first used to rank the importance of the parameters of the small-scale network and delete some of the parameters, compressing the model parameters and reducing the amount of operation and complexity. Then, knowledge distillation is used to migrate the knowledge from the large-scale network ResNet50 to the small-scale network so that the small-scale network can fit the soft-label information output from the large-scale neural network through the loss function to complete the knowledge migration of the large-scale neural network. Experimental results show that this method can compress the model size of the small network and improve the recognition accuracy. Compared with the original small network, the model accuracy is improved by 2.24 percentage points to 97.24%. The number of model parameters is compressed by 81.9% and the number of floating-point operations is compressed by 92.1%, making it more suitable for deployment in resource-constrained devices.
Record ID
Keywords
knowledge distillation, network pruning, power industry, service identification
Subject
Suggested Citation
Wang W, Zhou X, Jiang C, Zhu H, Yu H, Wang S. A Lightweight Identification Method for Complex Power Industry Tasks Based on Knowledge Distillation and Network Pruning. (2024). LAPSE:2024.0237
Author Affiliations
Wang W: Nanjing Power Supply Branch, State Grid Jiangsu Electric Power Co., Ltd., Nanjing 210019, China
Zhou X: State Grid Hubei Electric Power Co., Ltd., Wuhan 430077, China
Jiang C: State Grid Jiangsu Electric Power Co., Ltd., Nanjing 210024, China
Zhu H: Nanjing Power Supply Branch, State Grid Jiangsu Electric Power Co., Ltd., Nanjing 210019, China
Yu H: Nanjing Power Supply Branch, State Grid Jiangsu Electric Power Co., Ltd., Nanjing 210019, China
Wang S: Nanjing Power Supply Branch, State Grid Jiangsu Electric Power Co., Ltd., Nanjing 210019, China
Zhou X: State Grid Hubei Electric Power Co., Ltd., Wuhan 430077, China
Jiang C: State Grid Jiangsu Electric Power Co., Ltd., Nanjing 210024, China
Zhu H: Nanjing Power Supply Branch, State Grid Jiangsu Electric Power Co., Ltd., Nanjing 210019, China
Yu H: Nanjing Power Supply Branch, State Grid Jiangsu Electric Power Co., Ltd., Nanjing 210019, China
Wang S: Nanjing Power Supply Branch, State Grid Jiangsu Electric Power Co., Ltd., Nanjing 210019, China
Journal Name
Processes
Volume
11
Issue
9
First Page
2780
Year
2023
Publication Date
2023-09-18
Published Version
ISSN
2227-9717
Version Comments
Original Submission
Other Meta
PII: pr11092780, Publication Type: Journal Article
Record Map
Published Article
LAPSE:2024.0237
This Record
External Link
doi:10.3390/pr11092780
Publisher Version
Download
Meta
Record Statistics
Record Views
52
Version History
[v1] (Original Submission)
Feb 10, 2024
Verified by curator on
Feb 10, 2024
This Version Number
v1
Citations
Most Recent
This Version
URL Here
https://psecommunity.org/LAPSE:2024.0237
Original Submitter
Calvin Tsay
Links to Related Works