This paper presents the design and the realization of an ARM-based fly task control system in the UAV, guided by small TV and with the ability of attacking and scouting. As a central part of the UAV, this system utilizes the PHILIPS ARM7-LPC2138 and has embedded the real-time operating system USOS II to accomplish the task of both the real-time data exchange and data treatment with other four subsystems. The data exchange between the ground station and the system is achieved through the interface with the data link radio; the target tracking and locking is achieved through the interface with the TV control system; the flying path control under the condition of the locked target is achieved through the interface with the flight control system; the control of the TV control system is achieved through the interface with the Pan/tilt and the camera. The data exchange is stable and reliable with the error rate lower than 0.1%. The system controls of the Pan/tilt can meet the requirements of speed, veracity and stability. The control angle of the path deviation is accurate and it is transmitted to the flight control system. At present, the development of the prototype is completed, with test flights and simulated attacking experiments to provide technical supports for the large-scale UAV with the abilities of attacking and scouting.