This paper presents an optimal scheduling solution for building thermal loads that simultaneously participate in the wholesale energy and frequency regulation markets. The solution combines (1) a lower-level regulation capacity reset strategy that identifies the available regulation capacity for each hour, and (2) an upper-level zone temperature scheduling algorithm to find the optimal load trajectory with a minimum net electricity cost. In the supervisory scheduling strategy, piece-wise linear approximations of representative air-conditioning equipment behaviors, derived from an offline analysis of the capacity reset mechanism, are used to predict the cooling power and regulation capacity; and a mixed-integer convex program is formulated and solved to determine the optimal control actions. In order to evaluate the performance of the developed control solution, two baseline strategies are considered, one with a conventional night setup/back control and the other utilizing an optimization procedure for minimizing the energy cost only. Five-day simulation tests were carried out for the various control strategies. Compared to the baseline night setup/back strategy, the energy-priority controller led to a 26% lower regulation credit and consequentially caused a net cost increase of 2%; the proposed bi-market control solution was able to increase the regulation credit by 118% and reduce the net electricity cost by 14%.