Developing accurate models is necessary to optimize the operation of heating systems. A large number of field measurements from monitored heat pumps have made it possible to evaluate different heat pump models and improve their accuracy. This study used measured data from a heating system consisting of three heat pumps to compare five regression and two artificial neural network (ANN) models. The models’ performance was compared to determine which model was suitable during the design and operation stage by calibrating them using data provided by the manufacturer and the measured data. A method to refine the ANN model was also presented. The results indicate that simple regression models are more suitable when only manufacturers’ data are available, while ANN models are more suited to utilize a large amount of measured data. The method to refine the ANN model is effective at increasing the accuracy of the model. The refined models have a relative root mean square error (RMSE) of less than 5%.