In this work, a framework for high-energy-efficient UAV surveillance networks is proposed. A group of UAVs is deployed to cover an area that needs to be observed. A flocking algorithm is used to drive a group of UAVs moving to sensing areas. The algorithm guarantees that the UAV team can safely travel to required locations and forms an appropriate shape to cover the sensing areas. Then, an AI-based method is proposed with the aim of reducing redundant data for the UAVs while performing surveillance tasks of collecting data. The data processing algorithm can be divided into three main steps: (i) background modeling which removes all moving objects in scenes and background stitching that combines the background modeled from each UAV, (ii) noticed object extraction of each frame captured by UAVs, and (iii) data reconstruction of combined background modeling in step (i) and noticed objects from step (ii). The methodology can be referred as a kind of compress sensing technique which is aimed at saving power consumption by removing such redundant data of sensors [28, 29].

The rest of this paper is organized as follows: Section 2 provides briefly the system models that describe either the UAV network deploying in the sensing field or the AI-based methods to process the surveillance data collected from the UAVs. In Section 3, the whole problems are addressed. The flocking control algorithm and the AI-based data processing method are provided in detail. Section 4 presents both simulation and experimental results following all the steps modeled in Section 3. Finally, conclusions and future research directions are provided in Section 5.

2. System model 

In this section, the system models are presented. First is a model of an UAV network with the ability to travel, to avoid obstacles, and to collect video streaming data. Each distributed UAV can also be able to exchange the data with neighbors to construct the completed information of sensing regions. The AI-based data processing framework for enhancing an energy-efficient approach of a UAV network is analyzed.

2.1. Multiple UAV Systems

Considering a team of UAVs, the team is deployed in a ground center. After receiving a mission request task, the UAV team will move to a target location. The target location is defined as a virtual leader to be able to lead the UAVs in a flocking control algorithm. The collaborative algorithm in [30] is chosen to drive the UAV team. The UAV formation can safely reconfigure formation shapes to avoid collisions with obstacles while migrating. When UAV team arrives at the target place, the team gradually forms quasilattice formation to fully cover sensing areas.

Assuming each UAV can obtain its global position by sensors such as GPS. A downward camera is mounted on an UAV, which provides each UAV a constant sensing range of RS. UAVs are equipped with short-range wireless communication devices that allow them to wirelessly communicate with the others if the Euclidean distance between them is smaller than a constant , noted as the communication range. Different from [30], the sensing range  is not required to be smaller than the communication range  for ensuring nonoverlapped regions. In this work, overlapped regions are acceptable to guarantee coverage performance. While processing, the overlapped data is handled by an AI-based data processing algorithm, which is proposed as follows.

2.2. AI-Based Data Processing Method

The structure of the UAV system is given in Figure 1. An UAV monitors a distinct area, and the areas handled by different UAVs might be overlapped with each other. UAVs form a distributed network and share their local sensing information with others to reconstruct global sensing data.

In the first step, background modeling is performed on the UAVs. Captured videos are processed to create backgrounds that consist of only nonmoving objects. Then, the backgrounds are sent to the neighbors at the begging and only updated as there are any changes in backgrounds. The individual backgrounds are then stitched together to form a complete background of the sensing area. In the case of overlapped images, an overlapping detection algorithm is presented. In detecting keypoints and local invariant descriptors, then matching descriptors of overlapping images, a random sample consensus (RANSAC) algorithm is utilized to obtain homography. The obtained homography matrix is then used to warp and stitch overlapped pictures.

Secondly, UAVs perform object extraction functions where moving objects are detected by comparing differences in the continuous sequence of frames. If there are motions which is detected, details of moving objects are determined by a convolution neural network (CNN). These useful data are also shared among UAV networks.

Finally, the reconstructed images are built based on extracted data sent by other UAVs. Reconstructed processes can be performed on an UAV. As sensing data are reduced by the proposed method, a burden on transmission bandwidth and computational resources is greatly diminished.

3. Problem Formulation

This section presents an overview of approaches to the problems in multiple UAV-based surveillance systems. First, the flocking algorithm to drive the UAV formations to navigate sensing areas is presented. Next, the AI-based data processing method is given. Three steps as shown in Figure 2 are presented in details

In the background modeling process, there are two sub-steps: background modeling and background stitching captured by different UAVs.

The median filter technique is used to perform background modeling. The main idea of the median filter is to run through the signal entry by entry, replacing each entry with the median of neighboring entries. In this work, thanks to the idea of median filter technique, a number of frames are chosen randomly from the video captured by UAVs and background modeling tasks are performed. The number of frames chosen may vary through experiments to choose an appropriate number of frames to achieve better results of performance.

After UAVs perform their background modeling processes, the significant data will be sent to neighbors for reconstruction. However, in case of overlapped areas monitored by UAVs, it is not necessary to stitch those areas. In order to do that, the algorithm is called overlapped area detection in which the key points and its corresponding between backgrounds perform the background stitching. Algorithm 1 represents the overlapped area detection steps as follows.

Object detection task in aerial images is a challenging and interesting problem. With the cost of drones or UAVs decreasing, more aerial devices could be deployed. Hence, there is a surge in the amount of aerial data being generated. It will be very useful to have models that can extract valuable information from aerial data. However, since most objects are only a few pixels wide, some objects are occluded and objects in shade are even harder to detect. Thus, a hybrid noticed object extraction system that is a combination of existing method and custom object classification model to extract a valuable information from aerial data is proposed in this work.

Firstly, frame difference and thresholding technique are applied in each frame to estimate the moving areas that can be referred as the noticed area. For surveillance tasks with UAVs, the objects are often very small, so that directly applying object detection algorithms can result in missing or incorrectly detected objects. Therefore, in this paper, the first step is to determine the motion area by comparing frame  with frame , for  to find the difference and thus is motion area as shown in Algorithm 2.

4. Conclusions and Future Developments

This paper proposes new methods either to control multiple UAVs or to process video surveillance data based on AI techniques with CNN. The flocking control algorithms are applied into distributed UAVs to lead the UAVs travelling on the working fields and avoiding collision and obstacles. The AI-based data processing method that reduces significant redundant data streaming among UAVs is proposed. The method also reduces the training time and classification time compared to existing methods, such as YOLO detection. The overall proposed methods help reducing the storage capacity, transmission bandwidth, and performance in surveillance application of UAVs. Indeed, the proportion of objects in each frame is extremely small, and the transmission of redundancy in each frame is not necessary. The application of the method helps to reduce approximately 90% of the excess data capacity but still ensures the quality of the image. This significantly reduces the energy consumption for UAVs in their tasks.

Future research can be done to enhance the proposed solution. In order to improve the system performance, the crucial process is an object classification task that will classify the wrong area detected from the previous step, thereby improving the efficiency of the method. Moreover, when applied to more complex applications such as traffic surveillance and agriculture, more types of object should be considered.

                                                                                                               Tác giả: TS. Trần Thuận Hoàng