Virtex 2 configuration problem

I'm looking for some suggestions as to what else to look at to fix this problem:

Using a Virtex II xc2v40 and xcf02s prom connected in master serial mode. JTAG is also implemented. We've built around 50 of these boards without this problem so I believe it's just this particular board. It intermittently doesn't configure on power up.

A power chip holds INIT_B low until 3.3v is 3v to delay configuration, not really needed since power ramps up in around 5ms. When it fails, I see on a scope that INIT_B never goes high after power is up and PROG is high. PROG is pulled up to 3.3v thru 4.75k. When it works, INIT_B goes high within 5ms of PROG going high.

As a test, I connected PROG to GND with a wire and then powered up the board. After say 5 secs, I disconnected GND, and sometimes INIT_B would go high as expected, But sometimes it would remain low. If it went high, I could pulse PROG low and INIT_B would work as expected everytime.

When the fpga is in its nonworking state, it doesn't respond on the JTAG port either.

I lifted the prom pin and the power chip connected to INIT_B to verify that when it fails, it's the fpga that is holding INIT_B low and not the other chips.

Any ideas on what else to look at?