The specific surface area is an important parameter to characterize pore structure and adsorption properties, however, it is difficult to calculate accurately in shale rock due to its multiscale pore structure. In this paper, the representative 3D gray images of a microfracture sample, micropore subsample and nanopore subsample in shale rock were obtained with computed tomography (CT) scanning and focused ion beam-scanning electron microscopy (FIB-SEM) scanning. The multi-threshold segmentation algorithm with improved maximum inter-class variance method was introduced to construct the platform of multi-scale digital rock. Then, based on the fracture, micropore and nanopore digital rocks, the corresponding network models were extracted to obtain different-scale pore structures, respectively. Finally, based on the digital rock at different scales, the corresponding pore percentage, matrix percentage and specific surface area were calculated respectively. It was found that the specific surface areas of both microfractures and micropores are small, and their specific surface areas are 2~3 orders of magnitude smaller than that of nanopores, and the specific surface area of the shale formation is mainly contributed by nanopores. This paper provides an effective method to calculate the multi-scale specific surface area accurately in shale rock and has an important influence on the adsorption characteristics and swelling properties of the shale matrix.