This article presents a remote management architecture of an unmanned aerial vehicles (UAVs) fleet to aid in the management of solar power plants and object tracking. The proposed system is a competitive advantage for sola r energy production plants, due to the reduction in costs for maintenance, surveillance, and security tasks, especially in large solar farms. This new approach consists of creating a hardware and software architecture that allows for performing different tasks automatically, as well as remotely using fleets of UAVs. The entire system, composed of the aircraft, the servers, communication networks, and the processing center, as well as the interfaces for accessing the services via the web, has been designed for this specific purpose. Image processing and automated remote control of the UAV allow generating autonomous missions for the inspection of defects in solar panels, saving costs compared to traditional manual inspection. Another application of this architecture related to security is the detection and tracking of pedestrians and vehicles, both for road safety and for surveillance and security issues of solar plants. The novelty of this system with respect to current systems is summarized in that all the software and hardware elements that allow the inspection of solar panels, surveillance, and people counting, as well as traffic management tasks, have been defined and detailed. The modular system presented allows the exchange of different specific vision modules for each task to be carried out. Finally, unlike other systems, calibrated fixed cameras are used in addition to the cameras embedded in the drones of the fleet, which complement the system with vision algorithms based on deep learning for identification, surveillance, and inspection.