Large lifting equipment is used regularly in the maintenance operations of chemical plant installations, where safety controls must be carried out to ensure the safety of lifting operations. This paper presents a convolutional neural network (CNN) methodology, based on the PyTorch framework, to identify unsafe behavior among lifting operation drivers, specifically, by collecting 22,352 images of equipment lifting operations over a certain time period in a chemical plant. The lifting drivers’ behavior was divided into eight categories, and a ResNet50 network model was selected to identify the drivers’ behavior in the pictures. The results show that the proposed ResNet50 network model based on transfer learning achieves a 99.6% accuracy rate, a 99% recall rate and a 99% F1 value for the expected behaviors of eight lifting operation drivers. This knowledge regarding unsafe behavior in the chemical industry provides a new perspective for preventing safety accidents caused by the dangerous behaviors of lifting operation drivers.