I am flying a R/C seaplane powered by two brushless DC motors. One of the motor controllers went up in smoke, apparently from getting wet.
I have done some reading on how these controllers work, and as far as I can tell, everything is really low-impedance stuff, so some water should not be able to cause a catastrophic failure like this.
I had been flying the plane several time before, so it is entirely possible the controller got wet on one or more of those occasions too. Could it be that the first dip caused some corrosion that later caused a low-impedance short?
I fly only off fresh water. There was some white powdery oxides on the solder joints, so some amount of corrosion is definitely going on.
Can anyone explain to me the specific process how some wetness can cause a full meltdown? Are the MOSFETs not water tight?
MOSFETs have a very high input impedance to the (control) gate. That means they can indeed be turned fully on by a voltage and just the conductivity of a little bit of water. Should the wrong MOSFET accidentally turn on at the wrong time it can be disastrous for the electronics.
Brushless ESC's have to deduce the motor's position with respect to the magnetic field by monitoring the back EMF of the open motor terminal. That involves small-signal op-amp stuff that could well have high impedances. They also sense current using op-amp stuff that could well have high impedances. If the current limit died you wouldn't notice it; if that event was followed by the motor phasing circuitry dying, the whole thing could have gotten wedged and applied full battery voltage across one motor winding.
For that matter, if the corrosion had been ongoing you could have a short bad enough to create cross-talk between processor PWM pins, or to open a connection.
Another strong possibility is that there was corrosion around the crystal oscillator for the processor (assuming that it used one), and a bit of moisture got in there and made things conductive enough to stop the clock. Depending on the microprocessor used, that could have made it turn one pair of MOSFETS on hard.
Did the motor melt, or just the ESC? Either way, I'd strongly consider retiring the battery you were using -- LiPos don't like high current, and that's just what it's gotten.
Did you note any funny behavior right before the smoke poured out? Different motor sounds, down on power, etc.?
--
Tim Wescott
Wescott Design Services
http://www.wescottdesign.com
Do you need to implement control loops in software?
"Applied Control Theory for Embedded Systems" was written for you.
See details at http://www.wescottdesign.com/actfes/actfes.html
** I used to build and run RC model boats - so water ingress and its ( horrible) effects are well known to me.
** Soon as water gets on a PCB that is energised with DC voltage, current starts to flow through the water from one solder pad to another. The current creates electrolysis and eats away at the solder creating a conductive white goo ( tin or lead oxide) in mater of minutes. The goo allows more current flow .......
Salt water is the worst by far, but most fresh water lakes and rivers have enough minerals in the water to conduct sufficiently to start the process off. Once started, it is self generating.
Also, normal solder flux on a PCB becomes conductive when wet - so even pure water can be very destructive.
The PCB for your speed controller/s should be placed in a watertight bag and the leads brought out through a water proof seal. A balloon is one possibility.
Model boat enthusiasts all use a " radio box " which is typically a painted plywood case with clear plastic lid made completely water proof by use of cork gaskets and flexible seals for the servo linkages. Any batteries go in the same box.
BTW:
If radio or other PCBs get wet, it is essential to switch off DC power, remove the board immediately and wash it in clean water and / or de natured alcohol ( ie metho) and dry it in the sun. Do not allow the white goo time to form.
For this exect reason, doesn't the designer have to make absolutely sure the gates are never left floating. And in order to make the MOSFETs switch quickly, the gates have to be driven by quite low-impedance circuitry.
The BEMF is provided by a motor winding which has less than one ohm resistance. I'll see if I can find out how they create the reference voltage that the BEMF is compared to. In the open designs I have found, they use either plain GND or three resistors in a star connection to each phase.
It seems this particular controller does not have any current sense. At least, I cannot find any part that could be a shunt resistor. Maybe they are simply using a piece of PCB copper as a shunt?
The motor phasing is normally all done in software. It is, of course, possible the software has freaked out and done something unplanned.
I can't see any crystal or oscillator, so they are probably using the internal oscillator. But the oscillator pins are connected to a row of pads along the side of the board, along with the programming pins. Would water on these be enough to stop the oscillator?
The motor got really hot. Not sure yet if it has survived. This would indicate that the fault was not because both MOSFETs in the same phase were on at the same time (shoot-through), but that the problem was due to a commutation hangup, I'd guess.
I'll keep an eye on it. There was no detectable temperature rise, though, and the ordeal lasted only a few seconds.
Yes. The plane crashed into the water a while ago, and has now been repaired. While testing the motors (there are two), one seemed to start at a much higher throttle than the other, and it would sound "unsmooth". After starting and stopping a couple of times, the motor would not start anymore, only wobble back and forth. After attempting to start a couple more times, I saw the smoke.
I did with the receiver, but I was worried about cooling for the ESCs, so I left them uncovered. Now that I have taken the ESC apart, I can see that the heatsink is mostly cosmetic: It only touched three or four of the fifteen MOSFETs.
I also have a boat. I used one of those freezer containers for food. Completely waterproof, but the lid is so much easier to remove and replace, since there are no screws.
For flying models, however, weight is critical, so a lighter solution must be found. I have simply used a plastic bag with a zip tie around the wires. Not completely watertight, of course, but probably good enough.
That BEMF signal has to be conditioned, then applied to an ADC (or perhaps a comparator). That signal conditioning circuit would be _hugely_ inefficient if it were all done with way-low impedances. You can expect resistances in the k ohms for a circuit like that, which would be more than enough to experience problems with accelerated corrosion in a current-carrying circuit.
Some switching supplies use the MOSFET on voltage as a rough current sense. PCB copper would certainly develop some voltage across it with tons-o-current, and there's certainly that in an ESC.
The motor phasing is done in software based on information acquired from analog electronics, and that's exactly where I would expect both the high impedances and the misbehavior.
Not all software is smart enough to realize that it's getting nonsensical information.
Dunno -- it would depend on the processor, and it would be Very Bad Style to make a processor whose internal oscillator croaks because of conditions on external pins. But that doesn't mean that there aren't such processors out there.
It sounds like it to me. Obviously I think the commutation hangup is coming from either a stopped clock or the signal conditioning circuitry going on the blink, with the software getting confused as a consequence.
Maybe you're lucky -- particularly if it was one battery pack sized for two motors, and only one was misbehaving.
Next time it gets dunked, if you haven't taken Phil's good advise about radio boxes, peel the heat-shrink wrapping off the ESC's _immediately_, rinse everything off in fresh water, then dry them out. Take 'em home and bake 'em at 150F in an oven for a few hours to _really_ drive the water out.
--
Tim Wescott
Wescott Design Services
http://www.wescottdesign.com
Do you need to implement control loops in software?
"Applied Control Theory for Embedded Systems" was written for you.
See details at http://www.wescottdesign.com/actfes/actfes.html
it doesn't have to switch quickly to screw you. A high impedance signal on the gate will turn it on for DC and if it is still switching it only has to come on to cause a short - fast switching speeds are essential for high efficiency. Disaster doesn't care about that.
My point is that the gate is connected to a low-impedance driver. Although the gate itself is high-impadance, it is always connected to its low-impadance driver circuitry.
Is the whole thing potted or can you see the circuitry?
My guess is there is a microprocessor in their or discrete IC's. A drop of water left there, with power applied, will eat the component leads even if doesn't cause an immediate malfunction via a high impedance signal path.
Water and electronics don't mix well. A coat of varnish can do wonders. If they used a hard potting compound, there's still a good possibility that water was able to migrate inside due to normal thermal expansion and contraction.
I like to pot using wax when working with a prototype and silicone in final designs. Epoxy is good, as a rule, as long as the parts are squeaky clean before potting, and the stuff stays somewhat plastic. The stuff that sets up like glass is to be avoided.
My point is that the gate is connected to a low-impedance driver. Although the gate itself is high-impadance, it is always connected to its low-impadance driver circuitry.
-- RoRo
------
In general the mosfet drivers have very low impedance outputs but not always(depends on the application and design goals).
If the mosfets themselves burned up then it was most likely due to cross conduction where the high and low side mosfets were both turned on at the same time and no safety protection prevented it(you can current sense on the mosfets to make sure no cross conduction occurs).
If the mosfets are hot and you throw some water on them they probably won't like it.
It's difficult to say what the problem is when there is no real information given. DC motor controllers can come in a wide variety of designs with some more prone to catastrophes than others.
Can you determine if it is the mosfets themselves or the controller that is the problem? Is it obvious what is broke or does it just not work? You can take a volt meter and place it on the gates and attempt to drive the motor(may not be a great idea without disconnecting the mosfets) and check the voltage on the gates while attempting engaging the controller.
You can also check the mosfets to see if they are working or not by applying a gate voltage(possibly not a good idea in circuit unless you are careful) and checking their conductivity.
For "water tight" boards you can apply an epoxy. If the components are getting very hot and not designed for rapid cooling there is not much you can do as it is a mechanical problem with the stresses of rapid cooling on the cases. If it is just the mosfets then possibly a heat sink will help if one doesn't exist.
Near the upper limits of a MOSFET's operating temperature, insufficient gate drive can cause the device to fall out of saturation and begin dissipating too much power, causing it to fail.
Moisture *can* be a real enemy to electronics and is to be avoided for sure. In this case however, it appears that there is a design issue in the Electronic Speed Controller which can kill it's pass element when operating under heavy load, particularly at high device temperatures as the batteries become depleted.
Advise you replace your MOSFET driver chip with one that is capable of higher gate drive at lower battery 'state- of-charge' to assure the pass element does not 'go linear' on you again.
It is completely open, only covered with heatshrink tube. The PCB has more than two layers, though, so reverse engineering it is a bit tricky.
There is an ATMEGA8, several resistors, a few capacitors and transistors, and on the other side, there are 15 MOSFETs; 3 P-channels for each phase on the high side, and two N-channels for each phase on the low side.
Although there is visible corrosion, it is not severe. After washing off the oxides, there is no visible corrosion damage. It seems the solder was the part that corroded the most.
That's what I am thinking. A couple of layers of plastic varnish, and then stick the controllers in a plastic bag while flying, and then remove the plastic bag afterwards, so that condensation and minor leaks can dry out. I'm planning on leaving the heatshrink off, so that water does not get trapped under it.
Potting is not an option. It would add too much unnecessary weight. Dipping them in wax might work, though. What type of wax is suitable for this?
Since the motor got very hot, it is more likely the problem was caused by the commutation somehow getting stuck. The windings in the motor are very low resistance, so it would have a very similar effect.
Certainly. But I do not think that is what happened here. The MOSFETs normally are just barely warm to the touch after a flight, and they were not submerged in water. The water must have seeped slowly in under the heatshrink.
I understand that, and I was not really seeking help with the specific troubleshooting. I was curious to see if anyone could explain why these controllers seem to fail so reliably in contact with water. This is the third one I have seen in a short time.
Indeed. But for R/C hobbyists, cost is important, so the simplest designs seem to be the most popular. In general, they all employ a microcontroller (almost exclusively an Atmel) and a bank of MOSFETs and a few passives to glue it together.
I have now removed all MOSFETs that are bad, and shuffled the rest around so that I have at least one on each rail on each phase. The controller now runs properly on a low-power motor (hard disk spindle motor), so only MOSFETs were broken. There is also a bipolar transistor (gate drive for the high-side P-channels) that is seriously scorched, but, amazingly, it seems to work.
The voltage from the multimeter is actually enough to turn them on, but since there are many of them in parallel, it is tricky to find the bad ones by measuring. I first removed all that had a short from gate to either D or S. After that, there were only a few left, so it got much easier.
There is a small heat sink, but I think it's mostly cosmetic. It is placed under the shrink hose, it does not contact all the MOSFETs, and it does not get particularly hot. If overloaded (usually due to a too large propeller), though, they get really hot.
If you're interested, no problem. I have left the pictures at their original resolution, so they're about 2MB each.
Here's the power side, with the seven surviving MOSFETs:
formatting link
Here's the logic side:
formatting link
Notice the 6-pin dual transistor in the upper right corner. It looks completely mangled, but it still seems to work.
The two 7806s are not really part of the controller. They are used to supply the radio gear from the motor battery, so that no dedicated receiver battery is needed. It is called a "BEC", a Battery Eliminator Circuit.
The SO-8 at the bottom, covered in glue, is the 5V regulator for the CPU.
It is possible, but the statistics suggest very strongly it is a water issue. I and my buddies have been flying electrics off land for years, and have never seen a controller fail in such a spectacular fashion. In fact, they are remarkably reliable. Now, after less than ten flying sessions off water, we have seen three controllers let the smoke out. All of them, however, have been quite recently, so it looks like repeated exposure might be a clue.
It may still be doing some good, by spreading the heat. Normally modern surface mount parts dump heat into the board copper -- are you sure the heat sink wasn't contacting that?
Or it could just be cheap.
Brand & model?
If you can figure out what it was you should consider replacing it. It's characteristics will have changed, and you can't really trust it not to die a sudden death, now.
--
Tim Wescott
Wescott Design Services
http://www.wescottdesign.com
Do you need to implement control loops in software?
"Applied Control Theory for Embedded Systems" was written for you.
See details at http://www.wescottdesign.com/actfes/actfes.html
Since the motor got very hot, it is more likely the problem was caused by the commutation somehow getting stuck. The windings in the motor are very low resistance, so it would have a very similar effect.
Certainly. But I do not think that is what happened here. The MOSFETs normally are just barely warm to the touch after a flight, and they were not submerged in water. The water must have seeped slowly in under the heatshrink.
I understand that, and I was not really seeking help with the specific troubleshooting. I was curious to see if anyone could explain why these controllers seem to fail so reliably in contact with water. This is the third one I have seen in a short time.
Indeed. But for R/C hobbyists, cost is important, so the simplest designs seem to be the most popular. In general, they all employ a microcontroller (almost exclusively an Atmel) and a bank of MOSFETs and a few passives to glue it together.
I have now removed all MOSFETs that are bad, and shuffled the rest around so that I have at least one on each rail on each phase. The controller now runs properly on a low-power motor (hard disk spindle motor), so only MOSFETs were broken. There is also a bipolar transistor (gate drive for the high-side P-channels) that is seriously scorched, but, amazingly, it seems to work.
The voltage from the multimeter is actually enough to turn them on, but since there are many of them in parallel, it is tricky to find the bad ones by measuring. I first removed all that had a short from gate to either D or S. After that, there were only a few left, so it got much easier.
There is a small heat sink, but I think it's mostly cosmetic. It is placed under the shrink hose, it does not contact all the MOSFETs, and it does not get particularly hot. If overloaded (usually due to a too large propeller), though, they get really hot.
If you're interested, no problem. I have left the pictures at their original resolution, so they're about 2MB each.
Here's the power side, with the seven surviving MOSFETs:
formatting link
Here's the logic side:
formatting link
Notice the 6-pin dual transistor in the upper right corner. It looks completely mangled, but it still seems to work.
The two 7806s are not really part of the controller. They are used to supply the radio gear from the motor battery, so that no dedicated receiver battery is needed. It is called a "BEC", a Battery Eliminator Circuit.
The SO-8 at the bottom, covered in glue, is the 5V regulator for the CPU.
-------------------------------------------
What maybe happening is there is not enough gate drive to drive all the mosfets. This can cause cross conduction due to the mosfets not turning off and on fast enough(you get times when they are both on on the same side).
This would be a controller issue and not the problem with the mosfets. Since you say it has happened several times that is most likely the problem if the drive circuitry is sub-par.
I can't tell what exactly the circuit is doing but it looks like the uC is directly driving the mosfets(considering all those resistors). Mosfets are not as ideal as one wishes and the biggest problem with paralleling them is the increased gate capacitance. Basically you can think of a gate of a mosfet as having a capacitor across it and it takes time to charge the capacitor. As it charges the mosfet transitions from open to closed or vice versa. The transition acts a resistor varying from R_ds(on)(for your mosfets it is 7.5mOhms) to some very large value. As current is flowing through the mosfet the resistance causes significant thermal dissipation. The goal is to have very quick transition so there is less time for the mosfet to heat up. Fast transitions require low gate capacitance. When you parallel mosfets you increase the effective gate capacitance(as seen by the driver) which slows all the transitions down for each mosfet.
The design parallels 5 mosfets per leg. Each mosfet has 7.5mOhms so the total resistance is 1.5mOhms but the gate charge is 105*5(*2) = 525nC. These are best case. If the drive voltage is lower than 10V that 7.5mOhms will increase which increases heat dissipation(but you said it wasn't getting too hot to the touch so the drive voltage maybe ok all the incidences happen when you are losing power).
You could possibly use 1 mosfet per leg instead of 5 by finding a better mofset.
formatting link
While the wrong package it has about the same ratings or better(from what I've checked).
formatting link
For pch it's not as good but may work(if you use better nch's(lower R_ds(on) then you can use worse pch if necessary).
The main thing you need to work out is if the controller really is failing because it gets wet or if it's coincidence. There is only 1 way a mosfet is going to fail in the way you are using it and it's overheating(I'm sure you know there are other ways to ruin them such as ESD but these effects shouldn't occur in SOC). For an h-bridge this will happen only due to cross conduction issues(assuming it was properly designed for the rated load and for a motor it generally is not meant run stalled).
Cross-conduction either occurs because the controller itself is malfunctioning and not synchronizing the switching properly or because it is not properly able to drive the mosfets.
Possible reasons: Stalled motor(draws more current), low power(increases cross conduction and R_ds(on)), malfunctioning controller(sync issues => more cross conduction). Water(not sure but this is not part of the SOC).
Note that significant cross conduction should only occur when changing polarity/reversing the motor. But if the controller or power is failing that it will more likely occur. If you are drastically changing the polarity of the motor(or the controller thinks you are) then it may be trying to reverse the polarity very often which will increase power. So in that case it could be an issue with the input to the controller(the controller should be able to prevent this quite easily).
Hopefully I've given enough information that you might be able to figure out what happened.
PS. That dual transistor you mentioned may be a clue to the cause. I see 2 others that look identical(I guess that?s a diode above each one). I don't know what they are doing but possibly the transistor is going back causing the mosfets to subsequently fail. You did say it's working but maybe not as well as it should be.
BTW, normally these motor controllers are designed by using a driver with high-side capabilities. p-ch mosfets generally have a much larger R_ds(on) than similar n-ch and so it is better to use n-ch's instead of p-ch's for the high side. This requires special driving circuitry though but generally is better. (basically you have to generate a voltage larger than your supply voltage to drive the high-side n-ch mosfets. There are dedicated IC's to do this that cost a few dollars and some have significant driving capabilities for paralleling(much more so than that uC).
The following test will help you separate a 'water' caused fault from a MOSFET 'circuit design fault'.
Can you set up your scope to read the gate voltage into each bank of P-channel MOSFETs and bag up the controller as per normal?
RoRo: For flying models, however, weight is critical, so a lighter RoRo: solution must be found. I have simply used a plastic bag with RoRo: a zip tie around the wires.
Run your seaplane model until the battery is quite low and the P-Channel MOSFETS are quite warm. Don't measure the heatsink because as you say, it doesn't cool the high side pass elements. Suggest tacking thermocouples to the Vcc leg of each P-MOSFET.
RoRo: There is a small heat sink, but I think it's mostly cosmetic. RoRo: It is placed under the shrink hose, it does not contact all RoRo: the MOSFETs, and it does not get particularly hot.
Keep an eye on the gate voltage into the paralleled P-MOSFETs and the battery voltage. The gate voltage should snap quickly almost to ground and stay there during the duration of phase commutation. Then it should snap back up very quickly and stay nearly at battery voltage while that phase is turned off. Subtract gate voltage from battery voltage to find out if your P-MOSFETSs are likely saturated when they need to be and likely cut- off when they need to be.
I think you will see that, at low battery voltage, your driver chip stops saturating your hot P-MOSFETS and they 'go linear'.
While the P-MOSFETs roast in their cooking bag, you will probably reach a point where one or two of them will shoot through into their associated N-MOSFET partner, bringing the test to a 'smoking close'.
ElectronDepot website is not affiliated with any of the manufacturers or service providers discussed here.
All logos and trade names are the property of their respective owners.