Accurate prediction of the remaining useful life (RUL) of rolling bearings can effectively ensure the safety of complicated machinery and equipment in service. However, the diversity of rolling bearing degradation processes makes it difficult for deep learning-based RUL prediction methods to improve prediction accuracy further and provide generalizability for engineering applications. This study proposed a novelty RUL prediction model for rolling bearings based on a bi-channel hierarchical vision transformer to reduce the impact of the above problems on prediction accuracy improvement. Firstly, hierarchical vision transformer network structures based on different-sized patches were employed to extract depth features containing more degradation processes information from input samples. Second, the dual channel fusion method is implemented into classic RUL prediction networks based on a multi-layer fully connected network to improve prediction accuracy. With two distinct validation experimental arrangements utilizing the datasets from PHM 2012, the prediction accuracy of the proposed approach can be increased by up to 9.43% and 43.10%, respectively, compared with the current standard method. The results demonstrate that the proposed method is more suitable for rolling bearing RUL prediction.