The adoption of Electric Vehicles (EVs) will revolutionize the storage capacity in the power system and, therefore, will contribute to mitigate the uncertainty of renewable generation. In addition, EVs have fast response capabilities and are suitable for frequency regulation, which is essential for the proliferation of intermittent renewable sources. To this end, EV aggregators will arise as a market representative party on behalf of EVs. Thus, this player will be responsible for supplying the power needed to charge EVs, as well as offering their flexibility to support the system. The main goal of EV aggregators is to manage the potential participation of EVs in the reserve market, accounting for their charging and travel needs. This work follows this trend by conceiving a chance-constrained model able to optimize EVs participation in the reserve market, taking into account the uncertain behavior of EVs and their charging needs. The proposed model, includes penalties in the event of a failure in the provision of upward or downward reserve. Therefore, stochastic and chance-constrained programming are used to handle the uncertainty of a small fleet of EVs and the risk profile of the EV aggregator. Two different relaxation approaches, i.e., Big-M and McCormick, of the chance-constrained model are tested and validated for different number of scenarios and risk levels, based on an actual test case in Denmark with actual driving patterns. As a final remark, the McCormick relaxation presents better performance when the uncertainty budget increases, which is appropriated for large-scale problems.