A mathematical modeling of glucose−water separation through a reverse osmosis (RO) membrane was developed to research the membrane’s performance during the mass transfer process. The model was developed by coupling the concentration−polarization (CP) model, which uses one-dimensional flow assumption, with the irreversible thermodynamic Spiegler−Kedem model. A nonlinear parameter estimation technique was used to determine the model parameters Lp (hydraulic permeability constant), σ (reflection coefficient), and Bs (solute transport coefficient). Experimental data were obtained from the treatment of a pre-treated glucose solution using a laboratory-scale RO system, and studies on the validation of the model using experimental results are presented. The calculated results are consistent with the experimental data. The proposed model describes the RO membrane concentration process and deduces the expression of k (mass transfer coefficient in the CP layer). The verification shows that the expression of k well-describes the reverse osmosis mass transfer of a glucose solution.