The SO2 adsorption efficiency of activated carbons (ACs) is clearly dependent on its physicochemical structure. Related to this, the effect of physical and mechanical activation on the physicochemical structure of coal-based ACs has been investigated in this work. In the stage of CO2 activation, the rapid decrease of the defective structure and the growth of aromatic layers accompanied by the dehydrogenation of aromatic rings result in the ordered conversion of the microstructure and severe carbon losses on the surfaces of Char-PA, while the oxygen content of Char-PA, including C=O (39.6%), C−O (27.3%), O−C=O (18.4%) and chemisorbed O (or H2O) (14.7%), is increased to 4.03%. Char-PA presents a relatively low SBET value (414.78 m2/g) owing to the high value of Non-Vmic (58.33%). In the subsequent mechanical activation from 12 to 48 h under N2 and dry ice, the strong mechanical collision caused by ball-milling can destroy the closely arranged crystalline layers and the collapse of mesopores and macropores, resulting in the disordered conversion of the microstructure and the formation of a defective structure, and a sustained increase in the SBET value from 715.89 to 1259.74 m2/g can be found with the prolonging of the ball-milling time. There is a gradual increase in the oxygen content from 6.79 to 9.48% for Char-PA-CO2-12/48 obtained by ball-milling under CO2. Remarkably, the varieties of physicochemical parameters of Char-PA-CO2-12/48 are more obvious than those of Char-PA-N2-12/48 under the same ball-milling time, which is related to the stronger solid-gas reactions caused by the mechanical collision under dry ice. Finally, the results of the SO2 adsorption test of typical samples indicate that Char-PA-CO2-48 with a desirable physicochemical structure can maintain 100% efficiency within 30 min and that its SO2 adsorption capacity can reach 138.5 mg/g at the end of the experiment. After the 10th cycle of thermal regeneration, Char-PA-CO2-48 still has a strong adsorptive capacity (81.2 mg/g).