complexity, perversity, general evil

All of the boards of two different products do the same thing.

the fact that it boots and reads part of the flash ok, leads me to think it is a bug in your code

something like a variable that isn't initalized right

things like reading a word where you really meant to read a (initialized) short could do that,

on a little endian machine you won't see a problem until the bytes you didn't initialize are not zero

code that runs after it has gone to lala land could write the same location explaining why it works the second time

-Lasse

Is any of this stuff checksummed? If not, include it, and use try again method until it gets it right.

We have made a small advance, which confirms the general concept of evil demons playing tricks on us.

It turns out that, at least on the product we're working on now, the FPGA creates the 50 MHz clock that the ARM Ethernet MAC uses. And if there's no Ethernet clock, the LPC3250 ARM code crashes when it tries to initialize the Ethernet interface. So the prime problem seems to be that the ARM can't configure the FPGA the first time a unit is powered up, so that makes the ARM code crash later.

Still no explanation for that. Causality sucks.

I don't like to sync switchers, but certain parties here do.

I didn't write the code or design the chips.

And I bet you didn't ever get that startup power supply to work. I know a great way to do it.

Something like that. What's weird is the roughly 3 months time constant, sitting on a shelf unpowered, before it breaks again.

I checked the power supplies. If I kill power, they all dive down to millivolts in a couple of seconds.

until it gets it right.

The FPGA config bitstream has its own CRCs. One thing we could do is keep checking the FPGA (read/write a test register) and repeating the n_config/configuration sequence until it works. I guess we'll do that if we can't find a way to actually understand it.

Where's the damned three month time constant?

method until it gets it right.

It would be better to use that AND a software checksum or some kind of equi valent of data integrity check. If you can include some kind of error reado ut then you'll be on your way to identifying the bad actor. It may not be a real fault but bad chips not tested for this particular kind of manufactur ing defect.

Code bug, I'd agree. perhaps you have a variable running over another. Like a Int placed as a char next to a char, once you write the Int the char gets overwritten. Sometimes its hard to find other times not so. Some RAM/NVRAM/Low power RAM can hold memory to very low voltages. The Mfg's don't characterize them that way because the entire array may not be affected the same way.

Cheers

To get anywhere, I think you're going to have to instrument it to find out where exactly it fails. Fly wires to get signals, and if that doesn't yield enough information, add some outputs (via code changes) to see what's going on inside (of course that might immediately mask the problem). When you get much, much closer, maybe look carefully at all the differences between the ones that always work and the ones that mostly have the fault in the areas that seem to relate to the problem. A lot of really complex things have to happen correctly and in the right order for those products to work at all.. what happens if there's a checksum error in loading the FPGA code, for example? Is there a self-test sequence that can fail silently at some point?

Working with other people's designs has taught me that there are almost an infinite number of ways to make things barely work or barely not work. 8-( Unfortunately, it often takes far more ingenuity and time to figure out what's wrong than it would have to do it methodically and correctly the first time. Of course, there may well be bugs in the parts that are poorly documented or completely undocumented. If it's intrinsic to, say, the FPGA, I don't know how you'd fix it other than automating the power cycling (sometimes spacecraft have that kind of functionality), but before that kind of neanderthal hack was added I'd want proof that it's inside that chip.

Today.. I found a design with an open-drain output feeding many SMPS chips of a couple types (that may each sink or source a few uA of current depending on various things including the shutdown state).. and pullup to 3V is a resistor in series with an LED. It sort-of works most of the time. 8-(

Best regards, Spehro Pefhany

That's idealistic point of view. Materialists allude to singular vortexes in

You bet. This kind of problems make people superstitious.

VLV

You could get a very professional "cleansing" from some famous Indonesian Shaman for a mere 100K$...very popular business these days.

bill

I have a feeling that a LOT of latent defects live in Ethernet stacks. I've seen what I think is memory management issues with crashes associated with heavy traffic under certain conditions that only occur during start-up. Long time constants due to ARP time-outs and such like.

Best regards, Spehro Pefhany

and

It might make sense to disable the Ethernet chip and / or the firmware driver, just to see if the rest comes up and blinks the run light.

If that fixes (or breaks) it, John's in the right neighborhood.

So how does it crash ? Does it just stop doing anything at all? Does the LPC3250 get stuck in a data abort loop ? If so, you could write some code to possibly find out its last area of code before the crash. We do that with our LPC ARM and Cortex M3 parts but we don't use that particular part.

boB

No way. John Larkin's thoughts about anthropogenic global warming are remarkably simple-minded - it isn't happening and it wouldn't matter if it did - which may be perverse and evil as well as ill-informed, but he doesn't see it that way.

Internal charge storage at/near overstressed (most likely positive WRT supply) gate input?

Apparently, just figured out late yesterday, the boot loader works. It reads the FPGA header/descriptor from serial flash; it reads the FPGA config stuff and tries to configure the FPGA, but that doesn't work; then it successfully reads the code header, then successfully loads and starts the application program. The FPGA doesn't generate the 50 MHz clock to the Ethernet mac in the ARM, and when the ARM tries to start up the ethernet driver, it crashes.

Not so. We are liberating trapped CO2, and it's good for the planet. If it is causing warming (which is by no means clear), that might prevent the next ice age. The computer climate models are mostly worthless, and climate experts are about as good at predicting the future as economists, who turn out to be wrong about 95% of the time.