kablooey

One of my better customers just called. Seems six of my VME arbitrary waveform generators failed, in the same VME crate, simultaneously. We're talking over $30K of damage here.

I had them put a scope on the +5 supply and switch power on. It ramped up in maybe 100 ms, peaked at 5.6 for a few ms, then settled down to

5.1. That's above the 5.5 abs max for my CPU and FPGA chips, but not really that bad.

So I asked them to switch power off for various times, then back on. A

2 second delay gave 7.5 volts peak. An estimated 0.5 sec delay, as fast as they could work the switch, ramped up to 8.8. I'm guessing an optimum brownout might well hit 10 volts.

Dumb switching power supply design... stupid loop dynamics and no crowbar! I mean, one could easily load a VME crate with $100K worth of boards. They're ragging the crate vendor next. Don't know who is going to pay to fix the modules.

I could add a transzorb to my boards, I guess, but the overshoot might just blow a hole in the board. Transzorb+fuse would work, but that would be an ugly kluge. 8.8 volts is sort of past my responsibility, I'm thinking.

John

Reply to
John Larkin
Loading thread data ...

You didn't categorically say it but did you use an off-the-shelf OEM psu module in your equipment? If so, did you bother to carry out evaluation of the various PSU modules available during bench or field testing of your waveform gen to see which performed best and was the most suitable for your application?

If this is the casee I would say that you will have buckley's chance of shifting the repair cost to the PSU manf. It's the old caveat emptor (let the buyer beware) when it comes to choosing any component from an outside supplier I'm afraid. In saying this it would seem highly likely that since the designer of the equipment is responsible for choosing the components then it is he who will bear the cost of any repair attributable to a design flaw anywhere in the equipment.

Ross

Reply to
Ross Herbert

My read is that John supplied the VME modules, and the customer sourced the VME frame elsewhere...

I imagine John has spec'd a max input voltage somewhere in the docs, which would put the monkey on the customer's back, leaving them no recourse except to pursue the frame vendor. And rightly so.

Ironically, if John had put a fuse on his boards, the customer would probably be irate at having to repair fuses, rather than being grateful they'd saved anything. Some scenarios you can't win.

Richard

Reply to
Richard H.

Hi Richard, yes you are correct in what you say.

John didn't specifically say whether he supplied the PSU as part of the waveform gen and this info is necessary before a determination can be made. There may be other points to consider - such as;

Since a PSU is mandatory for the waveform gen to function it would normally be up to the designer of the equipment (waveform gen) to provide information as to the performance requirements of a suitable PSU. If John supplied the equipment without providing this information then John may be partly responsible. If the PSU specifications were not provided by John and the client selected a PSU without first asking John for recommendations as to a suitable unit then the client may be partly responsible.

Without further info from John it is difficult to say who is to be held responsible but I would guess that both parties might end up splitting the cost.

Reply to
Ross Herbert

It just occurred to me that John might have provided details on the VME board/s on his website. (I hadn't thought to visit before now).

formatting link

(nice board)

The nominal DC supply specs, which BTW, you see on spec sheets for equipment of this type, are provided. It would seem that the client has decided on a PSU, which as John intimates, has poor dynamic regulation characteristics so I think John may be in the clear on this one. In this case the client seems to have chosen a rack with a low or poor quality PSU so they are responsible for the stuff up I would say.

As to the final settlement, (John says the client is "one of his better customers"), there may have to be a compromise.

Reply to
Ross Herbert

My sincere apologies John. I only thought of looking for a website for your equipment after Richard H pricked my conscience.

If I had taken more notice of the "VME" part I would have guessed that the waveform gen was not a complete ready-to-go unit (as in a bench instrument) and would need to be plugged into a rack with its own integral PSU. Since this item is normally selected by the customer and may already be insitu then your equipment is in no way responsible for the failure. Short of trying to make the DC supply rails on your card "idiot proof" then there is a good chance this may happen to other customers who buy your cards. As Richard H said, sometimes you can't win...

Reply to
Ross Herbert

Crowbars for linear 5V supplies are typically set at 6.2V, but the spec on the crowbar would allow it to not trip until 6.6V. For better or worse, on switching supplies the specs are usually even looser.

VME crates are from the days of LS TTL, and those chips would generally take 7V on Vcc and still be within official max limits, and they usually wouldn't expire based on just getting a little spike of 10V.

If you get the boards back and determine what did blow, I'd be very interested in the details. Are they failing self-test, or have the bus interfaces blown, or is everything toast?

Tim.

Reply to
Tim Shoppa

Who would consider 8.8 volts to be a reasonable +5 logic supply?

The official IEEE 1014 VME specification states that the backplane +5 supply should never exceed 5.25. I sell the VME modules, and *** (a British company) sold my customer the VME crates.

If I build something that has, say, a small switcher on-board to make the +5 or whatever, I do usually consider the consequences of a failure and often include a transzorb or similar.

Most manufacturers - us included - deny responsibility for consequential damages, and the crate manufacturer may well do so. So we may fix the in-warranty modules for free, just to keep our customer happy.

John

Reply to
John Larkin

We got one back yesterday. Two chips are running hot, the MC68332 CPU (directly on +5) and a big Xilinx SpartanXL FPGA. The FPGA is powered from +3.3, linear-regulated from the +5 supply, but it connects to the CPU bus, and it's supposed to be 5-volt tolerant on its I/Os. So it looks like the +5 blew the CPU and it, in turn, pulled up a bunch of the FPGAs i/o pins and fried it too.

What's cool is that the CPU is running very hot but is still executing the firmware! And the FPGA is hot and really dead.

We're going to replace both chips (418 pins total!) and see how things look. There are 5 more FPGAs on the board, but we're optimistic they're OK.

Here's a pic of the board.

formatting link

John

Reply to
John Larkin

Sounds to me like a rather ignorant PSU manufacturer.

Nearly 30 years ago I was demonstrating to the digital guru at GenRad (Phoenix) my "indestructible" PSU.

He proceeded to grab the power switch and rock it back and forth until "kablooey" and I mean KABLOOEY, KABOOM, molten solder, etc. ;-)

Taught me a lesson... all my future GenRad PSU designs contained a timer/POR to prevent such switch AND brownout problems from screwing up the works.

...Jim Thompson

--
|  James E.Thompson, P.E.                           |    mens     |
|  Analog Innovations, Inc.                         |     et      |
|  Analog/Mixed-Signal ASIC\'s and Discrete Systems  |    manus    |
|  Phoenix, Arizona            Voice:(480)460-2350  |             |
|  E-mail Address at Website     Fax:(480)460-2142  |  Brass Rat  |
|       http://www.analog-innovations.com           |    1962     |
             
     It\'s what you learn, after you know it all, that counts.
Reply to
Jim Thompson

*** supplied goods 'not fit for the intended purpose'. Under UK law **** would get a good ******* and a bill. john
Reply to
John Jardine.

Several of my VME modules are used to test jet engines, and the tach module I'm doing right now has an overspeed trip output to shut down runaways. And all electronics has finite failure rates. If I had consequential liability for blowing up a $30e6 jet engine and maybe as much again in test stand, I wouldn't sell this stuff. Nobody would.

John

Reply to
John Larkin

Agree with all your points. I think you have taken the right steps to keep the customer happy. I think your customer is very lucky and should be appropriately chastened by this episode and hopefully they will take greater care in future when selecting peripheral equipment to house and power 3d party modules.

Reply to
Ross Herbert

$30,000 repair ----- / Rock \\ | _ | \\/ - -

_o/ | J. Larkin / |

**==========** **Hard Place** piss off the customer **==========**

My questions: what are you going to do on the repaired board to prevent a repeat? Can you offer to market that engineering change at x$ per for the other boards that haven't yet been blown? What happens with the

5 other boards that are blown?

There must be a way to turn this into a public relations coup ...

Ed

Reply to
ehsjr

Can i see to eproms toward the top?

Reply to
The Real Andy

SNIP

Hi John, what is the customer going to do to prevent a repeat?

Perhaps you can sell him a VME cage/backplane and PSU which you know is ok. That way he can definitely guarantee warranty repairs if a similar failure occurs within the warranty period, plus, you get an extra sale....

That's how Motorola used to do it back in the 70's - 90's. Customers were prepared to use their overpriced racks and PSU's just in case the boards failed due to an occurrence such as your customer encountered with the rack/PSU they sourced themselves. Possibly they were trying to reduce costs and chose cheapness over quality.

Reply to
Ross Herbert

Speaking generically, this is generally a symptom of something somewhere not tristating a bus. Depending on "who's strongest" sometimes things even mostly work.

Could be that guy, giving what you said about its voltage pickiness and that it hooks to the bus.

Back in the days of DIP's you'd just unplug chips until the bus contention went away.

Tim.

Reply to
Tim Shoppa

Just two nights ago a friend who works in the RAID industry, in Failure Analysis, described a rash of component failures (capacitor punch-through) that he diagnosed as caused by SMPS spikes at turn-on. 10vdc caps on a 5vdc line were being killed by 12vdc. Switchers save iron and weight, yes, but poorly designed ones can be a basket of headache.

Reply to
Michael

Dunno. He's harassing the crate vendor now. They're British, and what with time zones and the classic British time constants, it may take a while to resolve.

I don't want to be in the iron business; the margins are pitiful. And I don't want to sell anything that would break your toe if you dropped it.

I might sell him a high-power clipper he could hang on the 5-volt buss in the crate, to gobble up the overshoots.

Yeah, I'll have to try to get the history of the crate buy.

Anyway, I thought people ought to be reminded of how some power supplies can be "teased" to produce outrageous outputs.

John

Reply to
John Larkin

On my first job as an engineer, I was assigned to work with an older engineer who was responsible for switching power supply design. He gave me a brief course on his latest creation, emphasizing its simplicity and low cost. But the start up circuit had a small region of poorly defined operation (a race) if the line voltage were very low and varied in a certain way.

His explanation was that there was little likelihood of that ever happening. His prototype was connected to a Variac to test operation at both 120 and 240 volt input ranges, so I just fiddled a bit with the line voltage and blew his prototype to smitherines. I got the task of improving the start up circuit so any possible sequence of events (related to line voltage) were accounted for. I think he was looking forward to blowing up my design.

He couldn't. :-)

Reply to
John Popelish

ElectronDepot website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.