Do you have a question? Post it now! No Registration Necessary
Subject
- Posted on
- bwilson79
July 13, 2007, 6:38 pm

My setup has two Xilinx FPGA's which communicate with one another
through two unidirectional system-synchronous interfaces. FPGA 1 is
an XC2V3000-FF1152-4 and FPGA 2 is an XC4VLX160-FF1148-11. Both
devices get their clocks from the same input source on the board, and
the clock traces are well-matched. FPGA 1 brings the input clock
through an IBUFG and then into a CLKDLL, in which case the CLK0 output
is used as both the DLL feedback clock and the system clock. FPGA 2
brings the input clock through an IBUFG and then into a DCM_ADV, in
which case the CLK0 output is used as both the DCM feedback clock and
the system clock. These unidirectional system-synchronous interfaces
are basically data and data valid in both directions, and both FPGA's
utilize the LVCMOS25 I/O standard and 12mA slow slew on their
outputs. I have verified that all pads are utilizing the IOB
registers. I have also verified that timing is met (with margin) on
both devices and there aren't any unconstrained timing paths.
The test sends packetized data from FPGA 1 to FPGA 2 in one direction,
and FPGA 2 sends the data back to FPGA 1 in the other direction, with
some data processing. When I clock these interfaces at 80MHz,
everything works fine. However, when I lower the clock frequency to
40MHz (via the on-board oscillator), the interfaces start failing. I
also tried 20MHz and 60MHz and see failures as well. What's really
interesting is that when I keep FPGA 1 the same, but build a new FPGA
2 which does not use a DCM, the interfaces work at 40MHz as well as at
20MHz, 60MHz, and 80MHz. I have tried with my own DCM wrapper module
as well as with a Xilinx-generated COREgen module and see the same
behavior.
If anyone can perhaps help me understand what may be causing this
behavior I would certainly appreciate it. Below I've listed Xilinx
Timing Analyzer data sheet report information for both FPGA 1 and FPGA
2 (with and without the DCM). I'd recommend copy-pasting and viewing
with a fixed-point font. =)
** FPGA 1 outputs to FPGA 2 **
<pin> <clk to pad>
FPGA_1_tx_data_out<0> 5.055
FPGA_1_tx_data_out<1> 5.055
FPGA_1_tx_data_out<2> 5.022
FPGA_1_tx_data_out<3> 5.028
FPGA_1_tx_data_out<4> 5.039
FPGA_1_tx_data_out<5> 5.050
FPGA_1_tx_data_out<6> 5.019
FPGA_1_tx_data_out<7> 5.019
FPGA_1_tx_valid_out 5.023
** FPGA 1 inputs from FPGA 2 (No DCM) **
<pin> <setup> <hold>
FPGA_2_tx_data_in<0> 2.929 0.512
FPGA_2_tx_data_in<1> 2.895 0.562
FPGA_2_tx_data_in<2> 2.905 0.553
FPGA_2_tx_data_in<3> 2.842 0.625
FPGA_2_tx_data_in<4> 2.852 0.616
FPGA_2_tx_data_in<5> 2.899 0.554
FPGA_2_tx_data_in<6> 2.890 0.562
FPGA_2_tx_data_in<7> 2.840 0.625
FPGA_2_tx_valid_in 2.835 0.630
** FPGA 1 inputs from FPGA 2 (DCM) **
<pin> <setup> <hold>
FPGA_2_tx_data_in<0> 1.415 -0.021
FPGA_2_tx_data_in<1> 1.381 0.029
FPGA_2_tx_data_in<2> 1.391 0.02
FPGA_2_tx_data_in<3> 1.328 0.092
FPGA_2_tx_data_in<4> 1.338 0.083
FPGA_2_tx_data_in<5> 1.385 0.021
FPGA_2_tx_data_in<6> 1.376 0.029
FPGA_2_tx_data_in<7> 1.326 0.092
FPGA_2_tx_valid_in 1.321 0.097
** FPGA 2 (No DCM) outputs to FPGA 1 **
<pin> <clk to pad>
FPGA_2_rx_data_out<0> 9.484
FPGA_2_rx_data_out<1> 9.496
FPGA_2_rx_data_out<2> 9.493
FPGA_2_rx_data_out<3> 9.492
FPGA_2_rx_data_out<4> 9.494
FPGA_2_rx_data_out<5> 9.516
FPGA_2_rx_data_out<6> 9.511
FPGA_2_rx_data_out<7> 9.529
FPGA_2_rx_valid_out 9.534
** FPGA 2 (DCM) outputs to FPGA 1 **
<pin> <clk to pad>
FPGA_2_rx_data_out<0> 4.383
FPGA_2_rx_data_out<1> 4.395
FPGA_2_rx_data_out<2> 4.392
FPGA_2_rx_data_out<3> 4.391
FPGA_2_rx_data_out<4> 4.393
FPGA_2_rx_data_out<5> 4.415
FPGA_2_rx_data_out<6> 4.41
FPGA_2_rx_data_out<7> 4.428
FPGA_2_rx_valid_out 4.433
** FPGA 1 inputs from FPGA 2 **
<pin> <setup> <hold>
FPGA_1_rx_data_in<0> 1.999 -1.022
FPGA_1_rx_data_in<1> 2.001 -1.024
FPGA_1_rx_data_in<2> 1.975 -0.998
FPGA_1_rx_data_in<3> 1.963 -0.986
FPGA_1_rx_data_in<4> 2.007 -1.030
FPGA_1_rx_data_in<5> 1.946 -0.969
FPGA_1_rx_data_in<6> 1.986 -1.009
FPGA_1_rx_data_in<7> 1.944 -0.967
FPGA_1_rx_valid_in 1.990 -1.013
Thanks,
Brian
through two unidirectional system-synchronous interfaces. FPGA 1 is
an XC2V3000-FF1152-4 and FPGA 2 is an XC4VLX160-FF1148-11. Both
devices get their clocks from the same input source on the board, and
the clock traces are well-matched. FPGA 1 brings the input clock
through an IBUFG and then into a CLKDLL, in which case the CLK0 output
is used as both the DLL feedback clock and the system clock. FPGA 2
brings the input clock through an IBUFG and then into a DCM_ADV, in
which case the CLK0 output is used as both the DCM feedback clock and
the system clock. These unidirectional system-synchronous interfaces
are basically data and data valid in both directions, and both FPGA's
utilize the LVCMOS25 I/O standard and 12mA slow slew on their
outputs. I have verified that all pads are utilizing the IOB
registers. I have also verified that timing is met (with margin) on
both devices and there aren't any unconstrained timing paths.
The test sends packetized data from FPGA 1 to FPGA 2 in one direction,
and FPGA 2 sends the data back to FPGA 1 in the other direction, with
some data processing. When I clock these interfaces at 80MHz,
everything works fine. However, when I lower the clock frequency to
40MHz (via the on-board oscillator), the interfaces start failing. I
also tried 20MHz and 60MHz and see failures as well. What's really
interesting is that when I keep FPGA 1 the same, but build a new FPGA
2 which does not use a DCM, the interfaces work at 40MHz as well as at
20MHz, 60MHz, and 80MHz. I have tried with my own DCM wrapper module
as well as with a Xilinx-generated COREgen module and see the same
behavior.
If anyone can perhaps help me understand what may be causing this
behavior I would certainly appreciate it. Below I've listed Xilinx
Timing Analyzer data sheet report information for both FPGA 1 and FPGA
2 (with and without the DCM). I'd recommend copy-pasting and viewing
with a fixed-point font. =)
** FPGA 1 outputs to FPGA 2 **
<pin> <clk to pad>
FPGA_1_tx_data_out<0> 5.055
FPGA_1_tx_data_out<1> 5.055
FPGA_1_tx_data_out<2> 5.022
FPGA_1_tx_data_out<3> 5.028
FPGA_1_tx_data_out<4> 5.039
FPGA_1_tx_data_out<5> 5.050
FPGA_1_tx_data_out<6> 5.019
FPGA_1_tx_data_out<7> 5.019
FPGA_1_tx_valid_out 5.023
** FPGA 1 inputs from FPGA 2 (No DCM) **
<pin> <setup> <hold>
FPGA_2_tx_data_in<0> 2.929 0.512
FPGA_2_tx_data_in<1> 2.895 0.562
FPGA_2_tx_data_in<2> 2.905 0.553
FPGA_2_tx_data_in<3> 2.842 0.625
FPGA_2_tx_data_in<4> 2.852 0.616
FPGA_2_tx_data_in<5> 2.899 0.554
FPGA_2_tx_data_in<6> 2.890 0.562
FPGA_2_tx_data_in<7> 2.840 0.625
FPGA_2_tx_valid_in 2.835 0.630
** FPGA 1 inputs from FPGA 2 (DCM) **
<pin> <setup> <hold>
FPGA_2_tx_data_in<0> 1.415 -0.021
FPGA_2_tx_data_in<1> 1.381 0.029
FPGA_2_tx_data_in<2> 1.391 0.02
FPGA_2_tx_data_in<3> 1.328 0.092
FPGA_2_tx_data_in<4> 1.338 0.083
FPGA_2_tx_data_in<5> 1.385 0.021
FPGA_2_tx_data_in<6> 1.376 0.029
FPGA_2_tx_data_in<7> 1.326 0.092
FPGA_2_tx_valid_in 1.321 0.097
** FPGA 2 (No DCM) outputs to FPGA 1 **
<pin> <clk to pad>
FPGA_2_rx_data_out<0> 9.484
FPGA_2_rx_data_out<1> 9.496
FPGA_2_rx_data_out<2> 9.493
FPGA_2_rx_data_out<3> 9.492
FPGA_2_rx_data_out<4> 9.494
FPGA_2_rx_data_out<5> 9.516
FPGA_2_rx_data_out<6> 9.511
FPGA_2_rx_data_out<7> 9.529
FPGA_2_rx_valid_out 9.534
** FPGA 2 (DCM) outputs to FPGA 1 **
<pin> <clk to pad>
FPGA_2_rx_data_out<0> 4.383
FPGA_2_rx_data_out<1> 4.395
FPGA_2_rx_data_out<2> 4.392
FPGA_2_rx_data_out<3> 4.391
FPGA_2_rx_data_out<4> 4.393
FPGA_2_rx_data_out<5> 4.415
FPGA_2_rx_data_out<6> 4.41
FPGA_2_rx_data_out<7> 4.428
FPGA_2_rx_valid_out 4.433
** FPGA 1 inputs from FPGA 2 **
<pin> <setup> <hold>
FPGA_1_rx_data_in<0> 1.999 -1.022
FPGA_1_rx_data_in<1> 2.001 -1.024
FPGA_1_rx_data_in<2> 1.975 -0.998
FPGA_1_rx_data_in<3> 1.963 -0.986
FPGA_1_rx_data_in<4> 2.007 -1.030
FPGA_1_rx_data_in<5> 1.946 -0.969
FPGA_1_rx_data_in<6> 1.986 -1.009
FPGA_1_rx_data_in<7> 1.944 -0.967
FPGA_1_rx_valid_in 1.990 -1.013
Thanks,
Brian

Re: Seeing failures when clocking system-synchronous inter-FPGA interfaces at lower clock rates

One obvious question is do you change the clock frequency on the
fly, i.e. while the FPGA's are up and running, or do you re-load
the FPGA's aftere the change? DCM's, at least in the Virtex 2
family, don't automatically re-synch after a frequency change.
You need to apply the reset signal to the DCM after the new
frequency is stable. Also don't rely on the "LOCK" output of
the DCM for lock status. Once locked, this tends to stay on
even if lock is lost. I use bit 1 of the status bus to check
for lock.
HTH,
Gabor

Re: Seeing failures when clocking system-synchronous inter-FPGA interfaces at lower clock rates

<snip FPGA timing>

Do your DCMs have any phase offsets? The negative hold time for your last
bunch of registers is a bit curious. If offsets are used to move the
sampling points around, changing the frequency will increase or decreas the
time shift attributed to the fixed phase shift, perhaps pushing your specs
out of spec.
Running the timing analyzer with different frequencies (one may be able to
do this with one compile and multiple frequency specification lines,
disabling the unneeded frequencies in the timing constraint list through the
Timein Analyzer GUI) will show to what effect the phase offsets will affect
your design.
Site Timeline
- » What is the resistance of a big FPGA for VCCINT (unpowered)
- — Next thread in » Field-Programmable Gate Arrays
-
- » Newbie's first FPGA board !
- — Previous thread in » Field-Programmable Gate Arrays
-
- » MachXO2 internal clock tolerance / accuracy
- — Newest thread in » Field-Programmable Gate Arrays
-
- » Altera Cyclone replacement
- — Last Updated thread in » Field-Programmable Gate Arrays
-
- » 18650 battery holder
- — The site's Newest Thread. Posted in » Hobby Electronics Basics
-