Voltage sag detection is utilized to capture the sag occurrence moment and calculate the sag depth of power grid voltage in real time, so as to generate reference voltage for controlling voltage interactive equipment such as dynamic voltage restorers (DVRs). However, the traditional voltage sag detection methods based on synchronously rotating frames (SRFs) are unable to acquire high-precision sag information under nonideal grid conditions such as unbalance or harmonic interference. In order to enhance the immunity of the sag detection, a method based on a selective harmonic extraction algorithm (SHEA) is proposed in this paper. Firstly, the state-space model of SHEA is established using discrete orthogonal basis to decouple and separate the signal of target frequency and the signal of interference frequency. The controllability, stability and convergence of SHEA are analyzed theoretically and serve as the criteria for parameter tuning. Moreover, a gain compensator (GC) is used to improve the low and middle frequency gains of the voltage sag detection method based on SHEA so that the dynamic response speed for sag judgment can be optimized quantitatively. The simulation results indicate that the proposed voltage sag detection method has good dynamic and steady-state performance under nonideal power grid conditions such as unbalanced sag, frequency drift, phase variation and harmonic interference.