PLC reliabilty

[JimRowell 1]

I'm in the final stages of implementing a critical control system for a laboratory. It doesn't do a lot but there is no room for error in the things it does do. I've had to give special consideration to potential failure points and it got me to thinking about plc reliability. I've been asked many times by clients just how often do plcs fail and my reply is always "Failures are rare". Obviously, they do fail sometimes but how often and in what manner? I've seen no published results of studies on this. I'm sure every manufacturer has done their own internal studies. I'd like to be able to offer something better than saying "not very often". Personally, I've never had a plc failure. I've had I/O failures caused by external abuse but I consider that to be allowable damage. I've never had an internal problem. Has anyone ever seen any kind of test data on this? Or how about personal experiences? I know many people have failures once in a while on rack system modules. Everyone makes systems that will accept redundent processors, power supplies, etc so obviously it's an issue. Guess I've been lucky. Also most of my installs have been "bricks" which may be more reliable (maybe?). Any thoughts or info out there?

[Jay Anthony 118]

Most PLC manufacturers calculate MBTF - Mean Time Between Failure. This is a prediction of the weakest component in the PLC or module and it's number of hours before failure. Some publish this data - some do not. Here is a link to the MBTF for Omron FA products: MBTF-Omron FA Produsts

[JimRowell 1]

Thanks Jay, That's helpful. I hadn't seen that Omron doc before. Nonetheless, a simple lumped MTBF figure is only so useful. I'd be more interested in a breakdown of the faults with result analysis. For instance, in my current application, I can handle an output that fails to activate when it should and a power supply failure would be also be ok. On the other hand, it would be bad if an output activated when it shouldn't. Also bad would be misreading of inputs or a general failure of the program to execute correctly. I'd say that was typical of many situations in automation so I'd expect particular attention to be paid to those things in the plc construction. ie. Code should be checked for a correct crc while running in case of corruption, watchdog timer should be fail-safe, etc. We don't get too much real information from the makers on these points. That's why I'm interested in hearing about real-life stories about hardware failures. What type were they? What was the effect? These points should be one of the benefits of using a plc over a pc for control. I'm going to do some emailing to the manufacturers to see if I can get more info (doubt I'll get very far) but I'd like to hear from some people here too. I'll admit that redundent plcs (and i/o) cross-checking each other is the only great way to go but how often does the budget allow for that? The extra cost of the equipment and labour, cabinet size, etc is not minor.

[Jay Anthony 118]

This is the kind of forum where a discussion of this type would be useful. Perhaps chakorules would set up a category in each manufacturer's download section for individuals as yourself could post their findings and bug reports. For example, I am loath to use a triac ouput for any application for the shear fact that this device fails in the on condition.

[BobB 58]

They are generally very reliable. I use mainly 24VDC power supplies on my PLCs as they are, in the main, for power down systems. I have not had a power supply problem with Omron unless a DC/DC converter failed. This is the weak point but necessary as battery chargers these days tend to "pulse" and I am very suss about that. Also, Omron power 24VDC power supplies do not have a great range and batteries are generally charged to higher than their range. The lesser range appears to make the power supply more reliable but I am not sure. Having said that, I used GE-Fanuc 90-30 PLCs on one job. Their DC power supply is greater than Omron so I did not use a DC/DC converter. The battery charger was a constant charge type and so "pulse" did not come into the equation. After 2 1/2 years, the power supply failed, taking out the processor. I see on another site that there appear to be some over heating problems with Control Logix PLCs. It also appears from the comments there that AB are replacing components but keeping things very close to the chest. I hope that is not a trend.

[JimRowell 1]

Interesting. Bob, What do you mean by a "power down" system? Sounds like you are using battery backup? I've had one or 2 power supply failures (high quality units too) but with separate supplies rather than built-in ones so I can still say I've never had a plc failure. ;) To me, the scariest failure would be arbitrary or incorrect output activations. I've never heard of this happening but if it did, how many of us would just assume a software mistake on our part or operator error? Jay, I agree with you about triacs. I see very little point in triacs for general industrial control use. Transistors where you need the speed. Relays most everwhere else. Triacs are good for constant switching of heaters and cycling loads like that. I'll admit they are the best choice there. Generally though you don't need to switch high loads very often and an easily replacable relay or contactor brings piece of mind.

[Gerry 13]

In my experience, most failures result from misapplication or accident. However, if an application cannot tolerate even 'rare' failures, then system design has to cater to that requirement. In the system in question, it seems that output failure is the main concern. In the past, I have used an intermediate "diode-capacitor pump" circuit between a PLC output and the load-bearing relay. This requires a pulsing output from the PLC to keep the relay energised. I also used check-back inputs from the output relays for confirmation and a master relay (also pulsed) for added security. Critical inputs to the system were form C (changeover) contacts with NO and NC contacts wired to different input modules. This system was a burner management system for a dual fuel boiler and so shutdown was always the safe response to a fault. There are other fault-intolerant systems where shutdown is not a good option (e.g. pot line in an aluminum smelter). I consider this irresponsible and slanderous rumour-mongering. It earns you no respect to impugn a product with which you are unfamiliar on the basis of a single allegation from a single third party, with whom you are also most probably unfamiliar. Personally, I have commissioned more than a few CLX systems and never seen or heard of anything of the sort. This includes one in Fiji where I was close to expiring from the heat & humidity but the CLX (in a tightly-packed cabinet) has exhibited no problems. Edited 3 May 2004 by Gerry

[JimRowell 1]

Now that's a different approach. Any chance you could draw that? I presume if the pulses are either too fast or too slow then the output deactivates. I like it. It's an alternative to total redundancy. Not as good but could be quite cost effective. In a way it's similar to how I use pgr's but that's with constant human input. I'd still like to hear about real failure stories from somebody. Anybody?

[Sleepy Wombat 27]

Gerry, i like your approach, and i to am interested in the pulse config as Jim has eluded to. To Jim, I have seen power supplies fail, generally due to poor supply or conditions. I have also seen a PLC where the PLC outputs failed in their current state when there was a Internal error in the PLC...the conveyor it was controlling contiuned to run with disasterous consequences.... this was a few years back and hopefully the manufacturer has removed this bug. I will not name the brand but needless to say it was not an AB, Omron, Siemens, or Schneider OR Mitsubishi PLC that failed. The other failing i found with another block PLC from a different manufacturer was that the memory was only capacitor backed !!!! The equipemnt was shipped out programmed however they took two months befroe they could commision this plant and thus it didn;t work. Took a day to fly to site load the program and fly home, what an expensive exercise and time waster that was. That was the first and last i ever used that sorry excuse for a PLC. I will not name the brand but needless to say it was not an AB, Omron, Siemens, or Schneider Or Mitsubushi PLC that failed.

[Peter Nachtwey 15]

I (we) have found that DC-DC power supplies, opto isolators and oscillators are the main causes of hardware failure. Failures from wiring errors and abuse greatly exceed hardware failures. I feel that the MTBF number is meaningless. In some case adding protection circuitry reduces the MTBF yet you wouldn't want to design the card without the protection circuitry. My company designs motion control cards that fit into PLC racks so I can tell you what I know from a manufacturers view point. In some cases, all the parts we use must be approved by the PLC manufacturer because they have already ceritified the parts. However, there are still cases where we get a bad batch of parts. The end user doesn't normally see this because find these problem in burn in. In some cases that requires a lot of rework which isn't good and our supplier ends up on our s-list. In some cases the bad parts survive the breakin test. This is rare but it happens. What can be done? Remember, all the manufacturers have access to the same parts as long as they aren't proprietory. We are all at the mercy of those that build the components for us.

[JimRowell 1]

Hi Peter, Converters and ocsillators I can understand because of the power output quality but what have you got against optos?

[JimRowell 1]

LOL! Thanks, Sleepy! Now I worried since I'm using a "not an..." So you have seen a true freeze-up? That could be soooo... bad in many situations.

[Gerry 13]

The system was an AB PLC2/15 and I was never concerned about the pulse frequency being too high. I had to work with something under 10 Hz. Output failure, either on or off, would result in de-energising the output relay. :*-(

[BobB 58]

Gerry My apologies. It was not meant that way. I only relate the discussion elsewhere. I can point you there if you wish. By the way, there were several third parties involved. I might add I am looking at the possibility of using Compact Logix at the moment. I refuse to use Micrologix, all I would really require for small jobs, due to the fact that AB have opted to not support online program changes. Very annoying in this day and age. Jim, by power down systems I am referring to generator stand by systems for buildings and power stations. The whole PLC and control system is backed up by battery supply to get the generators going and supply power to the building/site. I have also done a system where there was a "black start" set to supply power to get the other generators going. It never went on line but supplied power to the auxiliaries (fuel pumps etc) of the working sets. When working sets come on line, the "black start" set goes off line. The other brand with which I "personally" have had a problem or two is the Hitachi EC series. Had several 240VAC power supply failures on those. Have also had a PLC power supply failure with Omron RT units but they were also on a battery supply. The company which installed them had not placed a regulator between the batteries and the PLC rack. Voltage got high when the charger crept up and "BANG". Not the fault of the PLC but the installer. Where several/many PLCs are networked I normally place pulse bits on the network (heart beat) from each PLC. If the heart beat stops from any PLC to any other PLC, a timer times out and registers an alarm. Edited 3 May 2004 by BobB

[KinK 0]

If the failure is of outputs is such a concern just make sure that the items the plc is turning on or off have feed back that way if the plc tells output 1 to turn on and you dont get input one changing then you know there is something wrong..... and in this case you should have some safety stuff inplace in order to prevent harm to man and machine.... ps I currently service 13 omron plcs in the worst posible conditions (Mining) lots of large motors pumps noise and other issues. I have never had (Knock on wood) a PLC failure. other than a dead input from someone hooking something up wrong or shorting it to ground. some of my plc's control High pressure Autoclaves filled with nasty stuff. I always sit down with the Eng's and hash out the worst possible things that you can think of happening and then try and make it so that there are safetys in place to prevent it or keep man and machine as safe as possible... hope this helps be safe KinK

[panic mode 246]

Hi everyone, Well, this "rumor" could be one of my posts. Unfortunatelly I can only confirm that this was true. I absolutelly agree that PLCs failures are quite rare and that most of the PLC problems come from accidents or mishandling. Please note, this is my post and it's only me who stands behind it. So before you read it keep in mind you can use the info or ignore it, I don't care... I choose to stay anonimous in public forums but if someone does want to discuss matter further, use personal message and I might provide more info. I wasn't anonimous in my feedback to manufacturers when problems were reported. Use your own judgement when evaluating free (any) information. I am not sure if anyone else had similar experience or if the problem was due early hardware revisions or something else (it happend during year 2000/2001). The fact is the rack was getting considerably hot (not just warm) even when PLC was sitting idle. Although I was advised not to worry, I decided to ensure plenty of cooling just in case - which still resulted in death of several cards (according to customer, it was one CPU and couple of M02Es). Since the topic started as reliability question, here are some other failures I witnessed over the past ca two years (I don't want to make it too long): Allan Bradley - one bad SLC504 (it died after ca 20 hours while runing demo program in dry cycle). - four bad 1746-OW16 (out of box / output wouldn't turn on) - three bad 1746-OB16 (all had one of the outputs slowly 'fading away') - two bad photoeyes (once they were triggered, output would stay high) - ten bad proximity sensors 3mm, smooth barrel (LEDs worked but no output) - two bad proximity sensors 18mm, shielded (LEDs worked but no output) etc. We talked to Allan Bradley reps but I don't know if this is still the case or if quality control improved - we don't buy AB sensors anymore even though they dropped price by some 80% (I used to pay for AB proxy ca. $160-$180 and now they are in the $29 range). The only exceptins are the OB16 which are recent failures. Omron - Only one CQM1A PLC (it worked fine but upon power down complained about too many writes so even though program was retained, you had to go online and manually clear the error flag, problem occured ca one year after product was purchased) Keyence - One CP751P vision system Reason was simply too flimsy housing of the unit with poor support for upper PCB and no insulation between PCBs (clearance is only 1mm or ca 1/16" at the power terminals because of slighly longer solder ice sickle on one of the power terminals). Just a touch on the power terminals while unit was running made upper PCB (where the terminals are soldered) bend and touch the camera connector (mounted on lower PCB) resulting in repair cost of $1500 (Canadian). Simple plastic sheet between PCBs could have prevented the whole problem (ca $0.02). If you want to check the voltage on running unit, think twice or try to be gentle. Don't even think of using screwdriver on power terminals while 24V power is on. Sick - three LCUP controllers (they all worked but on powerup drew huge current. two of them are still running - with bigger power supplies. One of the units which was provided by customer blew up in my face 20 minutes after commisioning, it was not too much of a bang but lots of thick smoke and some sparks). Note that we have installed many LCUPs that work just fine and stay within 12W limits as specified. We talked to Sick as well and they say the LCUPs are about to retire. Banner - three TwoHand safety modules - one optotouch push button - one light curtain sender Mitsubishi - one J2S amplifier (one of the CNs was broken by accidental pull on cable) Hammond - two transformers died within few months of operation (unrelated cases). Both were replaced with identical units which have been running since. Note: All components failed while runing (they were sized, wired and configured correctly). The only listed components that died due "operator handling" are the vision system and the servo amplifier. Edited 3 May 2004 by panic mode

[Gerry 13]

Jim, Sleepy, I wracked my brains over the circuit and came up with something that I think is close. I tried to add it to my previous post as an edit but it doesn't seem to have worked. Try #2:

[Gerry 13]

It worked! Note that the PLC output in my case was solid state, not a contact. Bob, You, me, panicmode, and many others all habituate both forums and I was aware of the post in question. I didn't comment there, just read with interest. I have no problem with people relating personal experiences. I felt that the way you related the discussion implied an endemic problem being covered up - most likely unintentional. You can't go for very long in this business without getting out-of-box failures and "infant mortality", regardless of brand-name. Generally, if a system survives intact for its first 2-3 months, then it will require some external stimulus to provoke a component failure.

[JimRowell 1]

Here's another (similar) approach, Gerry. I havn't tried this. Just a doodle from this afternoon's coffee break. R1, C1 form a basic RC timing circuit. C1 will gradually charge up to V+ (minus the R1 voltage drop) when the plc contacts are closed. If the plc contacts are opened, R2 will gradually drain C1 down to 0V. If the voltage on C1 is allowed to rise high enough, it will conduct across the Z1 zener & turn on Q1 which will turn on the output relay CR1. If the C1 voltage is allowed to rise too high, it will conduct across Z2 which will turn on Q2 which will shut off Q1 & kill CR1. CR1 output contacts are in series with the plc contacts in case the circuit fails such that Q1 is held closed. In order to maintain just the right voltage range on C1, the plc contacts would have to be constantly pulsed on and off within a certain range of frequencies. Otherwise the CR1 contacts will open. C2, R5 are to prevent a brief output closure that would occur if the plc contacts were "open too long" (no output) & then were latched closed (not pulsed). The voltage on C1 would rise up to an "ok" level & then continue into the "too high" area. C2 provides an output delay that ensures an "ok" level is not about to rise further. It gives Q2 an opportunity to prevent the output closure. Plc transistor ouptuts would be better able to survive the constant pulsing rather than the relay output as shown. One could take 2 of these circuits each fed from plc transistor outputs & use a PGR relay on each for the CR1's. Tie the CR1's together in series & you'd have a rather nice 2 channel safety device. Edited 3 May 2004 by JimRowell

[Gerry 13]

Though my recollection is sketchy, the circuit I used only had passive components (apart from diodes - 1N4001, robust for this app), reducing possible failure modes. By introducing more active components, are you increasing or decreasing reliability?

[JimRowell 1]

I'll take door #2 and say decreasing the reliability. That's why I would either place the output in series with the plc or turn it into a 2 channel device. Either way it needs to be monitored by the plc as well for failure. Both versions are pretty cool. Your's also needs a couple of output relays so that's also a source of potential failure (nowhere near as bad as mine though). The main thing with both versions is that the plc and the cct will not likely fail at the same time so as long as each is on the job, one will catch the other. Your's is simpler to build and more reliable. Mine has a high output capacity without the reed relay. Reed relays are bad in high vibration or high magnetic environments. Mine also rejects too high an input frequency and does not immediately close on startup until the plc proves a frequency type of output. Both are worth looking at depending on the circumstances.

[JimRowell 1]

Hi Kink, I am monitoring all of my outputs but sometimes letting the error occur at all is not acceptable. To give a better picture: In one part of the system I have 4 relays (controlled by 4 plc outputs) that must not be shut down without warning. Shutting them down for more than about 10 milliseconds can be absolute disaster causing 1 or more days of work lost and sometimes massive damage on 4 very expensive instruments. The last such event took one instrument out of service for 2 weeks. These things normally run 24/7 often without humans present. My biggest worry when designing the new system was sudden plc failure caused by loss of plc power or a major plc fault. To protect against this, there is a 5th relay that is self-latching (via simple holding contacts) and it is capable of holding in the critical 4 others even with the associated plc output contacts open. I have 2 separate power supplies fed from different ac circuits (both fed from a large ups) that redundently feed all of these relays. One of the supplies feeds other loads too but the other is dedicated only to this. The plc can only control the 4 relays by first unlatching the 5th relay. To do this it must pulse a 6th relay briefly. When it is done operating the 4 relays, it then relatches the 5th relay. Thus the plc has control but if it dies suddenly (or if we want to change the software), the 5th relay will maintain the system. The plc also works in conjunction with a pc and the two demand constant comms from each other. If one doesn't respond properly then an alarm is generated by the other. The plc also maintains a contact to the building security system. If it dies, these contacts should open and alert the monitoring station. The point is, I've gone to a bit of trouble to secure the system against normal, complete plc failure. That to me means that all outputs go open. I can even handle a freeze-up where nothing changes or everthing closes. But what about screwy operation? What if the plc just starts pulsing outputs on and off all over the place? Then I'm in trouble. I've no idea how likely that is other than to say I've never heard of it happening. So far no one here has come forward with any failures like that. Gerry's idea of a pulse filtering circuit is a great suggestion though. It won't make it into this project but I'm definitely going to do some testing with it.

[KinK 0]

Well I can say that I have never ever a plc fail by pulsing the outputs.... not in my 10 years and PLC's have got more and more reliable over the last few years. Can you not make the said 5th relay only close with a set amount of pulses ie 6 pulses only You would probably need 2 counters and a timer one up counter and a down counter then use the timer to ensure that there is no other pulse before turning the relay off there by ensuring that the signal is actualy from the plc to shut down.... actualy you could probably do it with just one counter and a timer (external from plc) Peace

[JimRowell 1]

Hi Kink, Yeah, that's a great idea. But frankly, I mainly wanted to get feedback about the likelyhood of that type of failure and since everyone including you seem to agree that they've never heard of it, I may as well relax about it. I've never seen it. The budget is not overly high on this project and I think I've gone beyond the call already. Of course, saying things like this means that the day after tomorrow, my plc will start to vibrate, speak Latin and toggle all of its outputs to the tune of She'll be Comin Round the Mountain... I have gotten some great ideas for future projects though.

[KinK 0]

LOL ya you had better Knock on wood quickly