As an important combustion aid for aerospace vehicles, subcooled liquid oxygen of high density can be used to increase loading capacity of a spacecraft. Providing a large amount of cryogenic propellant in a short time with a strict energy consumption limitation is a challenge in the design of the fuel filling system. The authors proposed a vacuumed subcooling system combined with an ejector and liquid ring pump to vacuum a liquid oxygen tank and obtain subcooled liquid oxygen. After the liquid oxygen tank is vacuumed to an intermediate pressure by the ejector, it is further vacuumed to 10 kPa using the liquid ring pump. The infinitesimal method was used to simulate the thermodynamic processes involved. Taking the ejector working fluid mass flow rate, jet pressure, intermediate pressure, initial tank liquid level, and liquid ring pump speed as optimizing variables, optimization was conducted to determine the optimal vacuuming time, remaining liquid level in the tank, pumping speed difference, and nitrogen consumption. The sample set was generated by the optimal Latin sampling algorithm. The surrogate assisted Non-dominated Sorting Genetic Algorithm (NSGA-III) multi-objective algorithm was used to construct a system optimization framework. The non-dominated solutions were added to the sample set to improve the generalization ability of the Gaussian Process Regression (GPR) in the Pareto front. A convergent Pareto solution set was obtained after multiple iterations. The influence of different optimization variables on each optimization objective was analyzed using the Pearson correlation coefficient method. The optimization results show that the trade-off scheme can obtain the subcooled liquid oxygen at 10 kPa and 73 K with a remaining liquid level of 74.84% in a total vacuum time of 19.93 h. The efficiency of the liquid oxygen vacuum subcooling system can be improved significantly.