In the preparation of high specific surface area activated carbon (AC) by KOH activation, the swelling of the reactant mixture and the particles’ agglomeration deteriorates the process and the property of product. In this study, a novel method using a rotary kiln loaded with steel balls has been developed for the preparation of AC from petroleum coke (PC) by KOH activation. It has been found that the molten KOH caused the swelling of the reaction mixture at a lower activation temperature, while the molten K2O led to the particles’ agglomeration at a higher temperature. The steel balls could relieve the swelling and agglomeration and enhance the pore structure development of the AC by boosting the heat and mass transfer in the reactor. At an activation temperature of 800 °C and a KOH/PC mass ratio of 3:1, the specific surface area of the AC obtained without the addition of steel balls in the kiln is 1492 m2/g, while that with the steel balls is 1996 m2/g. The introduction of CO2 during the activation could further decrease the particles’ agglomeration by converting the K2O into thermoset K2CO3 and develop more mesopores of AC. Specifically, the average pore size of the AC increased from 2.20 to 2.72 nm.