In China, a surging urbanization highlights the significance of building energy conservation. However, most building energy-saving schemes are designed solely in compliance with prescriptive codes and lack consideration of the local situations, resulting in an unsatisfactory effect and a waste of funds. Moreover, the actual effect of the design has yet to be thoroughly verified through field tests. In this study, a method of modifying conventional building energy-saving design based on research into the local climate and residents’ living habits was proposed, and residential buildings in Panzhihua, China were selected for trial. Further, the modification scheme was implemented in an actual project with its effect verified by field tests. Research grasps the precise climate features of Panzhihua, which was previously not provided, and concludes that Panzhihua is a hot summer and warm winter zone. Accordingly, the original internal insulation was canceled, and the shading performance of the windows was strengthened instead. Test results suggest that the consequent change of SET* does not exceed 0.5 °C, whereas variations in the energy consumption depend on the room orientation. For rooms receiving less solar radiation, the average energy consumption increased by approximately 20%, whereas for rooms with a severe western exposure, the average energy consumption decreased by approximately 11%. On the other hand, the cost savings of removing the insulation layer are estimated at 177 million RMB (1 USD ≈ 6.5 RMB) per year. In conclusion, the research-based modification method proposed in this study can be an effective tool for improving building energy efficiency adapted to local conditions.