In the paper, we report our research on the improvement of thermal efficiency of refrigerated trailers by modification of their wall structure by placing a layer of phase change material inside them. The research was carried out in the field of transport, meeting the requirements of all classes provided for in the ATP agreement for refrigerated trailers. As part of the research, we formulated a numerical model of the proposed design of the refrigerator walls, which was subsequently validated by comparing the modeling results with the results of experimental tests carried out on a test bench designed specifically for this purpose. Based on the validated simulation conditions, we formulated the numerical model of a full-scale refrigerated semi-trailer, which was numerically tested under the conditions specified in the ATP Agreement. The results proved that adding a 6 mm layer of the SP-24 phase change material in each of the walls of the cold store allows the temperature inside the trailer to be kept below −20 °C for a period of 24 h without the need to supply cold from the outside during operation. The passive refrigerated semi-trailer system implemented in this manner with 6 mm PCM layer allows for a reduction in primary energy consumption by up to 86% in a period of 22 h. The mentioned percentage did not take into account the efficiency of the cooling system of the phase change material.