PLC Troubleshooting - Debugging Systems you didn't Originally Write

[mmp 0]

Working on a PLC system that you did not originally write certainly has its own unique set of challenges, regardless of if you are troubleshooting a system that is down or if you are adding a new system or functionality. I am guessing this is something that many others here have had to do from time to time, either for a client or your employer. I wrote an article/blog on the subject, partially from the point of view of things that an employer or client should keep in mind, so it doesn't go into great detail, but there might even be a few things that a green PLC programmer could gain from it. Keep in mind the target audience, but I would be interested in receiving constructive feedback (please be nice;-) ). Thanks in advance, and here is the article: PLC Troubleshooting

[Armadillo852 25]

First off, great article. I do have some comments. 1) This comment about comments: For a basic PLC program this is not necessarily a problem. If this is Logix5000, no not a big issue. But if it is any other platform, comments must be present. For example, what if you had a rung with B3 bits and timers. I know because I have run into this before when doing contract work. No program, so you have to upload and poof no descriptions. 2) This is the best statement ever made, yet as you know, so hard to convince management of this. If a system that has been running for months or years just recently started having issues, it most likely will not require a change to the PLC code to fix the issue. Other than that that is a very good general rule of thumb article.

[ssommers 20]

I like this: "A well maintained structure is much easier to debug and much easier to modify than a large, kludgy 'rats nest' program." Having seen a lot of rat's nests & spaghetti programs, I appreciate the fact that memory is much cheaper today than it was 25 yrs ago and structured programming has become more the norm because of cheap memory. I spent 4 hrs working through an old Gould PLC program last week for a machine that would work in manual, but not auto. I hand wrote the ladder on paper using a handheld programmer. The program was only 15 rungs so I was able to struggle through and put names to all the internal bits & timers using the clues from the I/O diagram. It wasn't exactly spaghetti code, but it really took me back to my college days! The outcome... someone had miswired a switch many years ago and left the roller slightly loose, someone else came along last week and tightened all the screws when they fixed something else. It took another 2 hours to rewire that switch because it was in an awful spot and the broken sealtite had to be replaced at the same time. So rat's nests come in all flavors... programming, mechanical & electrical. I just hate when I get all 3 rolled up in one neat little package. Good basic article...

[Sleepy Wombat 27]

I am going to tell it as I see it, you can take my comments however you like. As an article I found it to be stating the obvious. If it was to be of any use to a newbee where are the examples - In a way it illustrated that you have only been exposed to what I would term relatively new technology, but I commend you on your effort even though it seems to be the ethos of your company is to encourage employees to write articles in down time. Commendable, I suppose. One of the most important sayings I want you to remember is (It is also a signature of a forum member as well who I have a lot of respect for) "Tell me, and I will forget. Show me, and I will remember. Involve me, and I will understand." I come accross spaghetie code a lot, as well as bad electrical practice in wiring etc. I feel lucky when I can find an old dusty electrical schematic in the panel, and of course you do not have access to a commented program. But the key to fault finding any machine that you are unfamilar with is - ask the operators how it works - a lot will be able to give you the finer details of the machine operation, the good ones even know what sensor does what - they can save you hours of trying to understand what is going on in the code, what to look for - thats even if you need to get on to the code at all. What about the basics of checking the IO before even looking at the code, from the description of the fault you can generally get a fair idea. I suppose that my point is that you need to and want to understand the process / machine operation before even attempting to simply debug a program. Heres another little gem that was printed on a t-shirt from Mr PLC ( I love this one also ) "The program never fails" I dont recall in your article anything about before even starting to write a program to just sit down, gather your thoughts and write down on paper what you need to do, any issues that you might need to think of etc.. My point is make a plan of attack. Armadillo852 Susan, I respect a lot of your comments and experience but I would disagree with you on this one. I remember cutting my teeth on hand held programming PLC's and also sequence controllers. A bad programmer is a bad programmer, and to be honest, now there is an abundance of memory I more often then not see wasted inefficient and messy code. Why? because some programmers become lazy - look at how fat MircoSux programs are now. Even when I was involved with writing programs that could not be broken into tasks or subroutines, as they freely are today, there was always a structure not necessarily in this order but as you scrolled down the program there would be code for - Outputs, Alarms, Sequence, Control, Plant Item etc In your case the system does not appear to be commissioned correctly in the first place otherwise when checking the IO they would of recognised a loose switch. It is unfortunate that you had to spend hours trying to find this out and what a pain it was. I really understand your frustration regards this. But then I also ask my self, the machine was running for years, why did it just stop working, who last worked on the machine, I am not rying to blame the last guy but I need to find out what they worked on, oh tightened that switch mmmm I'll investigate that.. I am not trying to tell you how to suck eggs (aussie slang) cause I am sure that that is one of the ways you found this out and you have learnt this from experience. I was just trying to illustrate my point above to the other newbees, you need to understand the process/machine operation first. So there you go. my 2 cents. I think that I get more bitter the older I get, and won;t apologise for it either.

[Ken Moore 23]

Have to agree, the article does not say much, we all struggle from time to time. The secret is to do the best you can and never let them see you sweat, lol.

[ssommers 20]

Thanks for the comments Sleepy. I don't think the situation would have been so frustrating for me if I had been in my own plant. I was "invited" to our Buffalo plant which has no permanent maintenance people, looking at the machine for the first time and both setup men were off that day. The senior setup man had been on vacation for a week. The junior setup man had tightened the screws, but didn't associate that with why the machine was stopping midcycle. The maintenance manager (who's newer than either setup man) had no idea what had or hadn't been done. So back to the PLC and the machine... I was fortunate to have a readable copy of the electrical prints. The spaghetti code in the PLC was not too bad, but I wish it had come with meatballs (aka comments). The thing that made me really crazy was trying to change the wiring inside the switch from NC to NO simply because of the cramped physical location. Based on other sensor locations on the same machine, I think that the original design engineer from 20+ years ago had never changed a sensor in his life. Re-thinking this situation... The people communications were messier than the PLC code. I yelled at both setup men by phone the next day - the senior man for allowing/knowing about and leaving the loose switch & miswire for years, the junior man for not telling me what he had done when I talked to him on the phone the day before I arrived. I privately chided at the maintenance manager for letting both setup men be off on the same day. If either setup man had been there, I would have given them instructions and walked away from it. Instead, the maintenance manager rolled up his sleeves and worked with me. I showed him everything all the way through debug and the sensor change and he got a good lesson in what his people do and don't do. What was really crazy though was that the company president visited while I was there and asked me if the PLC should be updated... and I actually said no. (Yep, I said leave the 20 yr old PLC alone because it's actually doing a good job, the handheld programmer works and they still have lots of spare I/O cards for it. The only thing I asked for is another time to visit and document the other 3 Gould PLC programs.) I think I may have gone way off topic, but I think these are things that newbies need to know. Manufacturing systems are more than just PLCs. They include the people - operators, setup, engineers & managers - as well as the machines. Communication is the real key. Well constructed comments in any program are just communication from the original programmer to future debuggers.

[Sleepy Wombat 27]

No Susan, you aint off topic at all. and mmp should take a leaf out of your book. You could not have said it better Funny, mmp aint got back, on the surface it might appear that he just wants people to visit his company site in a round about way. Prove me wrong mmp. If you want to help out noobs why not be more active in helping out with posts on this site. The BEST SITE for info and help !