I. Introduction

Object tracking is the process of tracking the movement of an object over time through the camera [1]. Object tracking has many practical applications such as in security, surveillance systems, driverless vehicle control, automatic traffic control, biological image analysis, intelligent robot control. Like most computer vision systems, object tracking focuses on computing to extract the desired object from the continuous image captured by the camera. The higher image processing computational speed provides the better object tracking ability. In fact, when deploying object tracking applications, they are usually built on the OpenCV library [2],[3],[4] and run on windows or linux operating systems. Accordingly, the image processing process through the graphics libraries makes the execution speed of the program very dependent on the computer configuration, thus increasing the design cost. Therefore, today's demand requires tracking systems to both satisfy the constraints of processing speed well handling, accuracy, response time, and affordability at the lowest possible cost. Therefore, FPGA (Field Programmable Gate Array) is the right choice to develop relativistic tracking algorithms because of the short development time and low cost, yet still meet the requirements of response time and accuracy.

In our research, the system uses median filter, morphological filter to remove noise to improve the quality of images taken from the camera and relies on color (red) to separate the object to be chased. The position of the object is determined, thereby giving a control signal to the robot motor. The system is implemented on an FPGA using a combination of MicroBlaze embedded microcontrollers and hardware cores to increase the speed of the system.

 This paper consists of 4 parts. In which, part 1 is an introduction to some research related to object pursuit; part II presents an overview of the tracking system and describes  the algorithms used in the system; part III describes the structure diagram of the embedded system on the KIT FPGA and details the IP cores used in the system; part IV is the results of the implementation of the tracking system on FPGA and the experimental evaluation results; the final is conclusion of the article.

II. Proposed Object Tracking System

A.    System Overview

Firstly, the system will receive one image frame from camera, then this image goes through object separation image filtering to improve image quality, color separation (red) and remove non-object noise regions. After separating the object, it will determine the contour of the object, then find out the coordinates of the object's position relative to the center of the image to output a motor control signal for the robot to move in the direction of the object.

Figure 1.   A process of object tracking system

B. Algorithm Steps

1. Collect object images

Object images were collected continuously from the OV7670 camera. The system uses the standard SCCB (Serial Camera Control Bus) [5] to configure the camera Data is transmitted by the camera in parallel 8-bit frames. The received image is an RGB565 color image with a size of 320x240 pixels [5].

2. Image filtering, object separation

The system uses a 3x3 median filter on RGB images obtained from the camera and homomorphie filtering on binary images obtained after object color separation to improve image quality, find the most accurate way around the object.

a. Filter the median

The median filter, proposed by Boyle, is often used for small noise reduction (impulse noise). Pulse noise often appears distinct and its gray level value is also very different from the gray level of its neighbors, so the median filter reduces noise by replacing the pixel's value with the median of the gray levels of neighboring pixels.

The main idea of the median filter algorithm is to use a mask that scans each pixel of the input image sequence, the mask usually used here is a mask of size 3x3, 5x5, 7x7 …

At each pixel position, take the value of the corresponding pixels in the mask area to arrange in ascending or descending order. Finally, assign the pixel to the center (median) of the range pixels have been sorted for the pixel value under consideration of the output image [6].

Figure 2.   An example of Median filter. Current pixel value is replaced by the median value of it’s neighboring pixels (3x3)

b. Color separation of objects

The RGB image is obtained after filtering the noise through the color separation block, which is a block that identifies the object based on the color of the object compared to the image background.

To separate the color of the object, it is necessary to check the RGB values at different positions of the image. For this article, the object is red ball, the object's RGB color space has a value of (255,0,0) but the image obtained through the camera has a low resolution and is affected by light. light, so the red color obtained on the image with the obtained R,G,B image area fluctuates around the values 255, 0, 0.

The thresholds for color channels R,G,B are experimentally selected to separate the color, the object’s color value equal 1, and the background color returns the value 0. The resulting output image is a binary image at figure 3.

Figure 3.   Image after filtering and color separation

c. Morphological filtering

In practice, the background pattern is always more or less affected by factors such as light intensity or because certain areas of the subject coincide with the background color. The above factors make the object extracted from the background not only the object of the survey, but also include noise or some areas inside the object that are not filled because of confusion with the background. Thus, to ensure the best information is provided for the next blocks of the system, the object should be refined by removing the noise regions that are not the object and filling the empty areas inside the object. One of the methods used to filter objects after extracting from the background is using Mathematical Morphology (MM).

Mathematical morphology is the technique and treatment of geometrical structures based on set theory. Technical This algorithm is based on structure and shape, using basic morphological algorithms to simplify the image while retaining the main features. The main idea of MM is to detect an image with a given simple block, or structuring element, to determine whether this block fits or misses the shape in the image. 

III. Ffpga Embedded System Structure

A.    Embedded System Structure Diagram

Built-in detailed diagram of an embedded system on Spartan-6 FPGA SP605 Evaluation Kit consists of:

-         One 32-bit MicroBlaze processor core [7] operating at 100Mhz with instruction memory and 32K data, using the AXI interface to allows MicroBlaze to communicate and control highspeed compute and peripheral cores.

-         One core UART Controller: Transfer the image to be processed from the computer to the board and receive the processed image to display on the computer, with a baud rate of 128000.

-         One core SDRAM Controller communicates with external RAM with up to 128MB of memory to store the image to be filtered.

-         Median filter IP core, including 2 memory FIFO and 1 median filter.

-         The IP core performs the Dilation operation (performs morphological filtering), including 2 FIFO memory and 1 math filter Dilation.

-         The IP core performs Erosion math (performs morphological filtering), includes 2 FIFO memories and 1 math filter core Erosion.

-         DMA Controller core for data transfer between external Ram and hard core IP cores, improving data processing speed, powered by xillinx [8].

-         The PWM core performs pulse regulation to control the Robot motor.

-         Two clock sources powered by xilinx, one using oscillator 200Mhz (both positive and negative side) and a source using a 27Mhz oscillator (single rib) [9].

-         The system uses AXI interface including AXI4, AXI4-Lite, Axi stream [10] to communicate between peripheral devices and MicroBlaze.

Figure 4.   Structure diagram of embedded system on Spartan-6 FPGA SP605 Evaluation Kit from Xilinx

B. Structure of Hardware Cores

1. Filter median and color separation of objects

-         Median filter and color separation include 2 FIFOs responsible for synchronizing data between 2 clock domains, 1 data domain is taken from AXI stream with 100Mhz frequency and 30Mhz median filter hardcore clock domain.

-         A rigid core filters the median according to the mechanism pipeline is responsible for filtering the median

-         A control_bi block to control write data Median filter output data to FIFO OUT so that it is synchronized

-         A color separation block is essentially the task of binaryizing each R, G, and B channel with appropriate thresholds.

-         Find the median of a 3x3 mask: Consider a 3x3 mask, the pixels in each row are sorted in a descending order, then sorted in descending order for each row, and finally sorted diagonally, the median of the diagonal is the median of the 3x3 mask [11].

Basic node is a block used to compare two 8-bit A and B inputs and output the larger number H and the smaller number L. Sort block and find the median of a mask 3x3 is built from the above algorithm based on Basic nodes.

2. Morphological filtering

Based on the theory of homomorphie filtering, we build 2 IP cores include Dilation and Erosion. These 2 IP cores have the same I/O data flow and hardware architecture. We use a 9x9 structuring element to scan the entire image. The structuring element will move along the entire image in turn from left to right from top to bottom, 1 pixel each time. For each 9x9 block of the image corresponding to the detected 9x9 structural element, a new pixel of the image will be calculated. So we can see that with an image size of 320x240 pixels, if we translate the structure element by pixel from left to right from top to bottom, it will take a very long time. Therefore, in order to speed up the execution time, the article performs a pipeline calculation with the input data stream in each 9x9 image block.

IV. Results Of Implementation And Assessment

The system implemented on the Spartan-6 KIT used about 80% of the memory elements including RAM blocks, LUTs and about 20% of other logic of the Spartan-6 KIT.  To evaluate the system's tracking performance, the team evaluated the system based on the good light environment and the robot's ability to follow in the right direction.  This paper presented a low-cost and low-power real-time object tracking robot control system on Spartan®-6 FPGA SP605 Evaluation Kit. The system has been experimentally run on the kit and controls the robot to chase red objects in different lighting conditions with high accuracy. In terms of design, the system is built by high-speed pipelined hardware cores, and data between external DDR3 memory and hardware IP cores, is transported in bursts back and forth via DMA, so the system operates at high speed, the response time of the system to the object's movement is acceptable.

                                                                                                        Th.S Trần Lê Thăng Đồng