PHEVs (plug-in hybrid electric vehicles) equipped with diesel engines have multiple model transitions in the driving cycle for their particular structure. The high frequency of start−stop of a diesel engine will increase fuel consumption and reduce the catalytic efficiency of SCR (Selective Catalyst Reduction) catalysts, which will increase cold start emissions. For comprehensive optimization of fuel consumption and emissions, an optimal control strategy of PHEVs that originated from the PER-TD3 algorithm based on DRL (deep reinforcement learning) is proposed in this paper. The priority of samples is assigned with greater sampling weight for high learning efficiency. Experimental results are compared with those of the DP (dynamic programming)-based strategy in HIL (hardware in loop) equipment. The engine fuel consumption and NOX emissions were 2.477 L/100 km and 0.2008 g/km, nearly 94.1% and 90.1% of those of the DP-based control strategy. By contrast, the fuel consumption and NOx of DDPG (Deep Deterministic Policy Gradient)-based and TD3(Twin Delayed Deep Deterministic Policy Gradient) -based control strategy were 2.557, 0.2078, 2.509, and 0.2023, respectively. By comparative results, we can see that the comprehensive control strategy of PHEVs based on the PER-TD3 algorithm we proposed can achieve better performance with comparison to TD3-based and DDPG-based, which is the state-of-the-art strategy in DRL. The HIL-based experimental results prove the effectiveness and real-time potential of the proposed control strategy.