With an increase in renewable energy generation in the United States, there is a growing need for more frequency regulation to ensure the stability of the electric grid. Fast ramping natural gas plants are often used for frequency regulation, but this creates emissions associated with the burning of fossil fuels. Energy storage systems (ESSs), such as batteries and flywheels, provide an alternative frequency regulation service. However, the efficiency losses of charging and discharging a storage system cause additional electrical generation requirements and associated emissions. There is not a good understanding of these indirect emissions from charging and discharging ESSs in the literature, with most sources stating that ESSs for frequency regulation have lower emissions, without quantification of these emissions. We created a model to estimate three types of emissions (CO2, NOX, and SO2) from ESSs providing frequency regulation, and compare them to emissions from a natural gas plant providing the same service. When the natural gas plant is credited for the generated electricity, storage systems have 33% to 68% lower CO2 emissions than the gas turbine, depending on the US eGRID subregion, but higher NOX and SO2 emissions. However, different plausible assumptions about the framing of the analysis can make ESSs a worse choice so the true difference depends on the nature of the substitution between storage and natural gas generation.