Troubleshooting embedded controllers boards

I like the idea. $1 in the slot, and it saves all that bible bashing, hymn singing, and shalt not coveting thy neighbour's ass.

That's roughly the way it is. Without a circuit diagram, you're basically stuffed, and even with one you probably are. Simple circuitry, standard chips, someone might be able to do it if they charge their time cheaply enough. But most stuff is only worth hacking if the payoff is high, like the Romanian mafia with ATM machines.

Paul Burke

:-)

I would agree with Paul here - the cost of getting someone good enough to hack the boards is probably more than the cost of replacing them. You might be able to enthuse a student nearing the end of their degree to get involved at a reasonable cost and they might have the skills to do it but, in reality, fault-finding is all to do with experience.

However, there are some things you can do to which don't require knowledge of the circuit but it depends on what the fault is. You can look at all the solder joints and check their quality or you can see if there are obvious breaks in any PCB tracks. You can check electrolytic caps for leakage and look for any foreign bodies causing shorts on device pins. If the PCB doesn't have tracks internal to it then you can check them all for continuity and check connectivity to power planes and shorts between them and other tracks (some of which might be intentional).

If the circuit is a proven design which passed its initial tests and entered service then your fault is likely to be either a faulty device or one of the things I've mentioned above. Whilst this won't fix all your boards you might be able to fix some of them and the checks above don't require a hugely skilled person.

Compare with a known good board (system).

If you are in the business then you would have access to similar (identical) machines. It is not that difficult once you have something with which to compare the signals with.

Good luck.

As others have said the above points plus

1/ Compare against working board. 2/ Look for connectors wrong 3/ Look for BURN marks seriously with solenoid and other motor drives in some machines broken directly or indirectly (foreign material) can cause these sorts of faults.

Without building up your own knowledge database from knowing

A What the reported faults and fixes you find were

B What the circuit diagram of standard parts is

C Having the ability to rework newer boards with fine pitch surface mount components.

All you are likely to be able to do is change the periphery or power supply components at best.

That is if all the components can be easily got in small quantities, no point repairing a few boards with one component if you can only buy them in minimum quantities of 5000!

To achieve this may well be a lot more expensive than the value of the board.

If the boards are worth more than 5 x the managers effective hourly rate then it might just be worthwhile spending the time building up the knowledge.

--
Paul Carpenter          | paul@pcserviceselectronics.co.uk
    PC Services
              GNU H8 & mailing list info
             For those web sites you hate

It's simply a component of the Electric Monk.

(The rationale behind the Electric Monk, for those who don't know, is that an electric toothbrush cleans your teeth for you and saves time, an electric can-opener opens cans for you and saves time, while an electric monk believes in things for you and saves time).

Troubleshooting and fixing even a moderately complex microprocessor-based board can take days (or even weeks) and requires a pretty intimate knowledge of the board's design. It also probably requires access to parts that are hard or impossible to get.

For the difficult cases, you're going to need a pretty good logic analyzer and/or an in-circuit emulator for the processor that's used.

--
Grant Edwards                   grante             Yow!  Is this BOISE??
                                  at               
                               visi.com

So, then it is true that in a sense it can't be done? Even comparing signals doesn't always lead to the problem. You may get a signal that doesn't match up but the problem is caused further up the road. I end up removing good chips and causing more damage. Just to clear. If I sent someone a random board something like

could you fix it? Could anyone fix it? With no docs or experience on this particular board?

Someone was right about the experience thing. I talked with a guy who worked for one of these companies doing board level repair work and He knows what to fix only because he has seen it before and has been doing it for 16 years. But He can only work on one manufactures set of boards.

I guess I just have a hard time excepting this. I keep believing that somewhere out there there is someone who can do this. And maybe some machine that can help. Not the $100,000. ones. hopefully something much cheaper. Some of the boards we can''t get anymore. Which sometimes causes us to junk a huge piece of equipment because the board can't be repaired. I should just give up. But I want to make sure I check under every stone. I have tried most of the ideas suggested above but I still have a rack of boards that I can only get so far with.

Thank you everyone

an in-circuit emulator for the processor that's used.

Has anyone seen a universal microprocessors emulator? Something that works for most any microprocessor? I see ones made for one or two or a few different ones but hasn't someone come up with a computer that can emulate most of the common processors? I hate to buy the fluke

9010 and spend time and money hunting down pod after pod and still not have all I need.

It would need a detailed description of the fault, knowing what connected to all the connectors, especially if it was a quirk that was caused by RAM bit failure so that part way through the game it went wrong.

A lot of knowledge about the system is required to track a fault in boards like that.

Bear in mind it uses a ROM and a PLD (Altera EPF6016) both of these contain programs, so if faulty may well not be copyable without the necessary files to reprogram them. They may have special security bits blown.

These are complicated chipsets basically an embedded computer with sound, hard drive controller, graphics controller, no doubt game input channels. The smaller of these chips are around 144 pin and the larger around 208 pins. There are THOUSANDS of connections on the board, and lots of functions in the chips.

Then there is how the software works and without knowledge of what these are doing when working normally or how it reports faults (if at all) then it is difficult to judge where the fault is.

You would probably need a complete setup to test it in with a known working complete setup to test against, and reproduce the fault if not a simple one.

Seriously the circuit diagrams are probably many pages long.

--
Paul Carpenter          | paul@pcserviceselectronics.co.uk
    PC Services
              GNU H8 & mailing list info
             For those web sites you hate

No

IMO, no.

Certainly no.

--
Grant Edwards                   grante             Yow!  "DARK SHADOWS"
                                  at               is on!! Hey, I think
                               visi.com            the VAMPIRE forgot his
                                                   UMBRELLA!!

Generally speaking, the only person who can fix these boards is you. The boards need to be in the entire environment that they operate in to test them.

Typically the same thing tends to die on a given design and one a few boards are brought back from the brink it will be easier to fix the rest. You want to test them thinking of the order of badness- most likely to least likely:

1. visible stuff. covered above by another person. you will need a microscope. I have found many issues with the microscope
2. circuits near the edge of the design (connector interface parts)
3. chemical-based parts. capacitors. definitely tantalums. although they tend to explode if on the power supply. use a good board to measure the power supplies and write them down. start each dog (bad board) with the power supply tests.
4. you have to assume that memories, cplds, gate arrays, logic arrays, processors, jungle chips are all good at this stage.
5. make sure clocks are there, scope the crystals
6. if the board is still resisting, go to the next one until all of one type are touched.
7. at this stage you need to replace chips. awful. maybe cannabalize one or two. hot air remove and solder/ wick/ microscope on a replacement.

you can see how difficult this all is. I have a few dogs (of my own design! I have schematics, logic diagrams of Xilinx, code, etc.) that are waiting for my time to be worth little enough to go through all of this. I doubt it will happen anytime soon...

remember when things like cars could actually be fixed by almost anyone?

we have filled the world with impossible doorstop technology at the core of big valuable stuff. like, y'know, machine tools

and video games

Chris

Uriah,

So tell us -

Do you employ people to fix your boards or fix them yourself? Do you collect a bunch of the same board that have problems so you can get them fixed as a batch? What preliminary testing do you perform to isolate the nature of the problem - eg. . tapping the board to see if there may be a suspect solder joint, bad circuit trace, physically stressed component (IC, etc)? . applying spray freeze to see if the problem is temperature sensitive? . check voltages? . check capacitors? . switches? . dirt? What test equipment do you have (CRO, Logic Analyzer, etc.)? Do you have a variac? (Don't laugh, these prove useful for marginal voltage situations). Do you spend any time looking at known good units and noting keypoints and waveforms? eg. the video signals to monitors, the driver IC's feeding them, switches and their conditioning circuits, reset to CPU, xtal/clock waveforms. Do you check that digital signals transition correctly - ie. move between their design specs - I have seen some digital IC's go analogue from time to time?

There are some things that become second nature in troubleshooting any electronic device and only become apparent after some time has passed.