Accelerating Face Detection on Zynq-7020 Using High Level Synthesis

Do you have a question? Post it now! No Registration Necessary

Translate This Thread From English to

Threaded View
Hello, here is my question:
Purpose: realize face detection on zynq-7020 SoC
Platform: Zedboard with OV5640 camera
Completed work: capturing video from camera, writing into DDR for storage and reading from DDR for display
Question: how to realize a face detection IP and its throughput can reach 30fps(pixel 320*240)  

Here are my jobs:
Base on the Viola Jones algorithm, using HLS(high level synthesis) tool to realize hardware IP from a C++ design
And this is my reference: https://github.com/cornell-zhang/facedetect-fpga

I have simulate and synthesize it into hardware IP, but its throughput does not reach the goal because the interval and latency are very large. (latency is 338(min) to 576593236(max), interval is 336(min) to 142310514002(max))
Looking into the code, I find the latency is mainly caused by the following for loops, but I don't know how to optimize the latency compromising between area.

So you may help me a lot with these:
1.Another way to realize face detection on zynq-7020?
2.How to test the throughput of my system and the relation between real throughput and the synthesis result?
3.Any way to optimize the following for loops?

Looking forward to your reply. Please feel free to contact me at anytime.
Thanks.

----loop1:
imageScalerL1: for ( i = 0 ; i < IMAGE_HEIGHT ; i++ ){  
    imageScalerL1_1: for (j=0;j < IMAGE_WIDTH ;j++) {
      #pragma HLS pipeline
      if ( j < w2 && i < h2 )  
        IMG1_data[i][j] =  Data[(i*y_ratio)>>16][(j*x_ratio)>>16];
    }
  }
----loop2:
Pixely: for( y = 0; y < sum_row; y++ ){
    Pixelx: for ( x = 0; x < sum_col; x++ ){
      /* Updates for Integral Image Window Buffer (I) */
      SetIIu: for ( u = 0; u < WINDOW_SIZE; u++){
      #pragma HLS unroll
        SetIIj: for ( v = 0; v < WINDOW_SIZE; v++ ){
        #pragma HLS unroll
          II[u][v] = II[u][v] + ( I[u][v+1] - I[u][0] );
        }
      }
      
      /* Updates for Square Image Window Buffer (SI) */
      SII[0][0] = SII[0][0] + ( SI[0][1] - SI[0][0] );
      SII[0][1] = SII[0][1] + ( SI[0][WINDOW_SIZE] - SI[0][0] );
      SII[1][0] = SII[1][0] + ( SI[WINDOW_SIZE-1][1] - SI[WINDOW_SIZE-1][0] );
      SII[1][1] = SII[1][1] + ( SI[WINDOW_SIZE-1][WINDOW_SIZE] - SI[WINDOW_SIZE-1][0] );
      
      /* Updates for Image Window Buffer (I) and Square Image Window Bufer (SI) */
      SetIj: for( j = 0; j < 2*WINDOW_SIZE-1; j++){
      #pragma HLS unroll
        SetIi: for( i = 0; i < WINDOW_SIZE; i++ ){
        #pragma HLS unroll
          if( i+j != 2*WINDOW_SIZE-1 ){
            I[i][j] = I[i][j+1];
            SI[i][j] = SI[i][j+1];
          }
          else if ( i > 0 ){
            I[i][j] = I[i][j+1] + I[i-1][j+1];
            SI[i][j] = SI[i][j+1] + SI[i-1][j+1];
          }
        }
      }
      // Last column of the I[][] and SI[][] matrix  
      Ilast: for( i = 0; i < WINDOW_SIZE-1; i++ ){
      #pragma HLS unroll
        I[i][2*WINDOW_SIZE-1] = L[i][x];
        SI[i][2*WINDOW_SIZE-1] = L[i][x]*L[i][x];
      }
      I[WINDOW_SIZE-1][2*WINDOW_SIZE-1] = IMG1_data[y][x];
      SI[WINDOW_SIZE-1][2*WINDOW_SIZE-1] = IMG1_data[y][x]*IMG1_data[y][x];

      /** Updates for Image Line Buffer (L) **/
      LineBuf: for( k = 0; k < WINDOW_SIZE-2; k++ ){
      #pragma HLS unroll
        L[k][x] = L[k+1][x];
      }
      L[WINDOW_SIZE-2][x] = IMG1_data[y][x];

      /* Pass the Integral Image Window buffer through Cascaded Classifier. Only pass
       * when the integral image window buffer has flushed out the initial garbage data */              
      if ( element_counter >= ( ( (WINDOW_SIZE-1)*sum_col + WINDOW_SIZE ) + WINDOW_SIZE -1 ) ) {

         /* Sliding Window should not go beyond the boundary */
         if ( x_index < ( sum_col - (WINDOW_SIZE-1) ) && y_index < ( sum_row - (WINDOW_SIZE-1) ) ){
            p.x = x_index;
            p.y = y_index;
            
            result = cascadeClassifier ( p, II, SII );

           if ( result > 0 )
           {
             MyRect r = {myRound(p.x*factor), myRound(p.y*factor), winSize.width, winSize.height};
             AllCandidates_x[*AllCandidates_size]=r.x;
             AllCandidates_y[*AllCandidates_size]=r.y;
             AllCandidates_w[*AllCandidates_size]=r.width;
             AllCandidates_h[*AllCandidates_size]=r.height;
            *AllCandidates_size=*AllCandidates_size+1;
           }
         }// inner if
         if ( x_index < sum_col-1 )
             x_index = x_index + 1;
         else{  
             x_index = 0;
             y_index = y_index + 1;
         }
      } // outer if
      element_counter +=1;
    }  
  }


Re: Accelerating Face Detection on Zynq-7020 Using High Level Synthesis
yuning he wrote:
Quoted text here. Click to load it

Verilog is not my forte, but I think arrays are arrays.  In the initial  
loop you are retrieving the data from  
Data[(i*y_ratio)>>16][(j*x_ratio)>>16].  The range of one index is 0 to  
IMAGE_HEIGHT-1 and the other is 0 to IMAGE_WIDTH-1.  I can never recall  
which is the inner index and which is the outer, but the math involved  
in calculating the address of the data is simpler if the inner index  
range is a binary power.  Is that the case?  If not, you can achieve the  
simplification by declaring the inner index to have a range which is a  
binary power, but only use a subrange that you need.  The cost is wasted  
memory, but it will improve performance and size because the address  
calculation will not require multiplication, but rather shifts which are  
done by mapping the index to the right address lines.

--  

Rick C

Re: Accelerating Face Detection on Zynq-7020 Using High Level Synthesis
? 2017?5?23???? UTC
+8??3:11:40?rickman???
Quoted text here. Click to load it
ge and reading from DDR for display
Quoted text here. Click to load it
ch 30fps(pixel 320*240)
Quoted text here. Click to load it
 to realize hardware IP from a C++ design
Quoted text here. Click to load it
pga
does not reach the goal because the interval and latency are very large. (l
atency is 338(min) to 576593236(max), interval is 336(min) to 142310514002(
max))
Quoted text here. Click to load it
wing for loops, but I don't know how to optimize the latency compromising b
etween area.
Quoted text here. Click to load it
 throughput and the synthesis result?
Quoted text here. Click to load it
e.
-1][0] );
Quoted text here. Click to load it
INDOW_SIZE-1][0] );
Quoted text here. Click to load it
fer (SI) */
Quoted text here. Click to load it
y][x];
Quoted text here. Click to load it
ier. Only pass
Quoted text here. Click to load it
ial garbage data */
Quoted text here. Click to load it
ZE ) + WINDOW_SIZE -1 ) ) {
Quoted text here. Click to load it
m_row - (WINDOW_SIZE-1) ) ){
Quoted text here. Click to load it
nSize.width, winSize.height};
Quoted text here. Click to load it
  
Quoted text here. Click to load it
  
Quoted text here. Click to load it
  
Quoted text here. Click to load it
  
Quoted text here. Click to load it
  
Quoted text here. Click to load it
  
Quoted text here. Click to load it

Thank you for your reply.
Here IMAGE_HEIGHT equals to 240, and IMAGE_WIDTH equals to 320.According to
 your advice, I can change the inner index of the array to be a binary powe
r to accelerate the address access. Is this right?

Re: Accelerating Face Detection on Zynq-7020 Using High Level Synthesis
yuning he wrote:
Quoted text here. Click to load it

I believe so.  To minimize the waste of memory, I would make the 240 the  
inner index with a range of 256.  Then the multiplication becomes a  
matter of shifting the outer index by 8 bits and adding to the inner  
index.  I don't know for sure, but the tools should figure this out  
automatically.

Keep your loop range as 0 to 239 and everything will still work as you  
expect skipping over 16 array values at each increment of the outer  
index.  You will need to be consistent in all accesses to the memory.

--  

Rick C

Re: Accelerating Face Detection on Zynq-7020 Using High Level Synthesis
Quoted text here. Click to load it
ng for loops, but I don't know how to optimize the latency compromising bet
ween area.
Quoted text here. Click to load it
You're not meeting timing, which means you probably need to go look at the  
schematic of the critical paths.  How many levels are they?  How are the mu
ltipliers being synthesized?  Are they using DSP48s or fabric?  As a rule,  
the more abstract a synthesis tool is, the worse the synthesis results will
 be.

Also, where is "Data[]"?  Is that a blockRAM?  Or is it DRAM?  If you're ac
cessing DRAM directly without a cache, you might have problems.

Re: Accelerating Face Detection on Zynq-7020 Using High Level Synthesis
On 05/23/2017 11:21 AM, Kevin Neilson wrote:
Quoted text here. Click to load it

Sorry, I just walked back from lunch through a small horde of web  
developers, and was trying to envision the looks on their faces as C was  
referred to as being much too high level.

--  
Rob Gaddi, Highland Technology -- www.highlandtechnology.com
Email address domain is currently out of order.  See above to fix.

Re: Accelerating Face Detection on Zynq-7020 Using High Level Synthesis
Thank you for your reply.

The timing can meet when I slow down the clk to be 100MHz.  
The mutipliers are synthesized by DSP48s automatically.  
And "Data[]" is saved by BlockRAM dircetly.


Site Timeline