Seeing failures when clocking system-synchronous inter-FPGA interfaces at lower clock rate...

Do you have a question? Post it now! No Registration Necessary

Translate This Thread From English to

Threaded View
My setup has two Xilinx FPGA's which communicate with one another
through two unidirectional system-synchronous interfaces.  FPGA 1 is
an XC2V3000-FF1152-4 and FPGA 2 is an XC4VLX160-FF1148-11.  Both
devices get their clocks from the same input source on the board, and
the clock traces are well-matched.  FPGA 1 brings the input clock
through an IBUFG and then into a CLKDLL, in which case the CLK0 output
is used as both the DLL feedback clock and the system clock.  FPGA 2
brings the input clock through an IBUFG and then into a DCM_ADV, in
which case the CLK0 output is used as both the DCM feedback clock and
the system clock.  These unidirectional system-synchronous interfaces
are basically data and data valid in both directions, and both FPGA's
utilize the LVCMOS25 I/O standard and 12mA slow slew on their
outputs.  I have verified that all pads are utilizing the IOB
registers.  I have also verified that timing is met (with margin) on
both devices and there aren't any unconstrained timing paths.

The test sends packetized data from FPGA 1 to FPGA 2 in one direction,
and FPGA 2 sends the data back to FPGA 1 in the other direction, with
some data processing.  When I clock these interfaces at 80MHz,
everything works fine.  However, when I lower the clock frequency to
40MHz (via the on-board oscillator), the interfaces start failing.  I
also tried 20MHz and 60MHz and see failures as well.  What's really
interesting is that when I keep FPGA 1 the same, but build a new FPGA
2 which does not use a DCM, the interfaces work at 40MHz as well as at
20MHz, 60MHz, and 80MHz.  I have tried with my own DCM wrapper module
as well as with a Xilinx-generated COREgen module and see the same
behavior.

If anyone can perhaps help me understand what may be causing this
behavior I would certainly appreciate it.  Below I've listed Xilinx
Timing Analyzer data sheet report information for both FPGA 1 and FPGA
2 (with and without the DCM).  I'd recommend copy-pasting and viewing
with a fixed-point font.  =)

** FPGA 1 outputs to FPGA 2 **
<pin>                      <clk to pad>
FPGA_1_tx_data_out<0>         5.055
FPGA_1_tx_data_out<1>         5.055
FPGA_1_tx_data_out<2>         5.022
FPGA_1_tx_data_out<3>         5.028
FPGA_1_tx_data_out<4>         5.039
FPGA_1_tx_data_out<5>         5.050
FPGA_1_tx_data_out<6>         5.019
FPGA_1_tx_data_out<7>         5.019
FPGA_1_tx_valid_out           5.023

** FPGA 1 inputs from FPGA 2 (No DCM) **
<pin>                      <setup>  <hold>
FPGA_2_tx_data_in<0>        2.929    0.512
FPGA_2_tx_data_in<1>        2.895    0.562
FPGA_2_tx_data_in<2>        2.905    0.553
FPGA_2_tx_data_in<3>        2.842    0.625
FPGA_2_tx_data_in<4>        2.852    0.616
FPGA_2_tx_data_in<5>        2.899    0.554
FPGA_2_tx_data_in<6>        2.890    0.562
FPGA_2_tx_data_in<7>        2.840    0.625
FPGA_2_tx_valid_in          2.835    0.630

** FPGA 1 inputs from FPGA 2 (DCM) **
<pin>                      <setup>  <hold>
FPGA_2_tx_data_in<0>        1.415    -0.021
FPGA_2_tx_data_in<1>        1.381    0.029
FPGA_2_tx_data_in<2>        1.391    0.02
FPGA_2_tx_data_in<3>        1.328    0.092
FPGA_2_tx_data_in<4>        1.338    0.083
FPGA_2_tx_data_in<5>        1.385    0.021
FPGA_2_tx_data_in<6>        1.376    0.029
FPGA_2_tx_data_in<7>        1.326    0.092
FPGA_2_tx_valid_in          1.321    0.097

** FPGA 2 (No DCM) outputs to FPGA 1 **
<pin>                      <clk to pad>
FPGA_2_rx_data_out<0>         9.484
FPGA_2_rx_data_out<1>         9.496
FPGA_2_rx_data_out<2>         9.493
FPGA_2_rx_data_out<3>         9.492
FPGA_2_rx_data_out<4>         9.494
FPGA_2_rx_data_out<5>         9.516
FPGA_2_rx_data_out<6>         9.511
FPGA_2_rx_data_out<7>         9.529
FPGA_2_rx_valid_out           9.534

** FPGA 2 (DCM) outputs to FPGA 1 **
<pin>                      <clk to pad>
FPGA_2_rx_data_out<0>         4.383
FPGA_2_rx_data_out<1>         4.395
FPGA_2_rx_data_out<2>         4.392
FPGA_2_rx_data_out<3>         4.391
FPGA_2_rx_data_out<4>         4.393
FPGA_2_rx_data_out<5>         4.415
FPGA_2_rx_data_out<6>         4.41
FPGA_2_rx_data_out<7>         4.428
FPGA_2_rx_valid_out           4.433

** FPGA 1 inputs from FPGA 2 **
<pin>                      <setup>  <hold>
FPGA_1_rx_data_in<0>        1.999   -1.022
FPGA_1_rx_data_in<1>        2.001   -1.024
FPGA_1_rx_data_in<2>        1.975   -0.998
FPGA_1_rx_data_in<3>        1.963   -0.986
FPGA_1_rx_data_in<4>        2.007   -1.030
FPGA_1_rx_data_in<5>        1.946   -0.969
FPGA_1_rx_data_in<6>        1.986   -1.009
FPGA_1_rx_data_in<7>        1.944   -0.967
FPGA_1_rx_valid_in          1.990   -1.013

Thanks,
Brian


Re: Seeing failures when clocking system-synchronous inter-FPGA interfaces at lower clock rates
Quoted text here. Click to load it



One obvious question is do you change the clock frequency on the
fly, i.e. while the FPGA's are up and running, or do you re-load
the FPGA's aftere the change?  DCM's, at least in the Virtex 2
family, don't automatically re-synch after a frequency change.
You need to apply the reset signal to the DCM after the new
frequency is stable.  Also don't rely on the "LOCK" output of
the DCM for lock status.  Once locked, this tends to stay on
even if lock is lost.  I use bit 1 of the status bus to check
for lock.

HTH,
Gabor


Re: Seeing failures when clocking system-synchronous inter-FPGA interfaces at lower clock rates
Quoted text here. Click to load it

The system is completely restarted when selecting a new clock
frequency.


Re: Seeing failures when clocking system-synchronous inter-FPGA interfaces at lower clock rates
Quoted text here. Click to load it

<snip FPGA timing>

Quoted text here. Click to load it

Do your DCMs have any phase offsets?  The negative hold time for your last
bunch of registers is a bit curious.  If offsets are used to move the
sampling points around, changing the frequency will increase or decreas the
time shift attributed to the fixed phase shift, perhaps pushing your specs
out of spec.

Running the timing analyzer with different frequencies (one may be able to
do this with one compile and multiple frequency specification lines,
disabling the unneeded frequencies in the timing constraint list through the
Timein Analyzer GUI) will show to what effect the phase offsets will affect
your design.



Site Timeline