Implementing intelligent identification of faults in hydroelectric units helps in the timely detection of faults and taking measures to minimize economic losses. Therefore, improving the accuracy of fault signal recognition has always been a research focus. This study is based on the improved empirical mode decomposition (EMD) theory to study the denoising and feature extraction of vibration signals of hydroelectric units and uses the backpropagation neural network (BPNN) to establish corresponding connections between signal features and vibration fault states. The improved EMD in this study can improve the performance of noise reduction processing and contribute to the accurate identification of vibration faults. The vibration fault identification criteria can adopt three dimensionless feature parameters: peak skewness coefficient, valley skewness coefficient, and kurtosis coefficient of the second- and third-order components of the signal, with recognition rates and accuracy reaching 90.6% and 96.2%, respectively. This paper’s area under the curve (AUC) values were 0.7365, 0.7335, 0.9232, and 0.9141 for abnormal sound detection of the fan, water pump, slide, and valve, respectively, with an average AUC value of 0.8268. This paper’s accuracy is 90.1%, and the loss function value is 0.27. The validation results demonstrate that this paper’s method has high intelligent fault analysis capabilities. The experimental results confirm that this method can effectively detect vibration signals in hydroelectric units and perform effective noise reduction processing, thereby improving the diagnostic accuracy of fault signals. Therefore, this method can be effectively applied to the detection of vibration faults in hydroelectric units.