Loss in communication PLC-5

[salman 0]

We are still facing a problem regarding communication loss in PLC-5/40L with 1771-ASB Adaptors (8 chassis and with single slot addressing we have racks 2 to 17 in REmote IOs) as described below NATURE OF PROBLEM 1- The systems goes to faulted state frequently 2- The major error code we receive is S:12=83 “Watchdog timer error”. And the bits S11 gets status 0000-0001-0000-0000 (this inidicates only watchdog timer error) 3- All the Remote IO communication module loss communication as soon as this fault occurs. 4- This fault is removed as soon as we recycle the power. TROUBLESHOOTING ACTIVITES DONE: 1- The scan time of PLC-5 was increased from 500ms to maximum (due to this frequency of fault has decreased but it’s still there). 2- The fault routine was developed in which as soon as this fault occurs the system goes to this fault routine and find the status of error . If error code matches the fault code 83 The system removes the fault . Due to this now as soon as this fault comes the PLC don’t go in fault state but the Remote IOs goes in non-responding state again. This is recovered at once only by recycling power. 3- All the Remote IO cables were checked and if there was some error in cables then the system must not get OK at once after power up. 4- The configuration of Remote IO adaptors is changed from manual startup to Auto Startup after fault. But still the problem is same. Is there any remedy for it? PLC-5 programming file is attached with iit.. Kindly help me in this regard

[paulengr 44]

You are going after communication problems but you are looking in the wrong place. All communication goes down because the processor is faulted, not the other way around. The watchdog timer runs independently of your program and the scan. It is set to a very long timeout (500 ms is the default I believe) just so that if for instance your program contains an infinite loop or some other sort of faulty logic, it will fault the processor and shut everything down. If it didn't do this, you couldn't even troubleshoot the program in online mode to debug the problem. All the other steps you've taken are simply masking the problem. Something in your system is taking an obscene amount of time in a scan and causing the processor to fault. It could be a slow IO card but I'm much more suspicious of the program because I have a processor that has been running for a couple years now with a total of 31 RIO modules (all quarter rack 1794 cards) on the same single RIO channel without any timeout faults. This is why in spite of the fact that you truly do have a massive amount of IO, I'm leaning more towards the program itself and not the IO. The program was NOT attached to the posting by the way. Normally, the place to look when you get watchdog timeouts is in the code. Look in loops and large file operations (especially an FAL or FOR/NXT instruction). Most PLC programs rarely exceed 100-200 ms for scan times in a very compute-intensive process. You really need to look at the scan times of your program and figure out where any loops, or huge file operations (FAL, especially) are at. It definitely could be IO such as someone programming an HMI to poll the PLC excessively, but I'm still leaning towards programming problems. If you are using FAL or similar operations a lot or FOR/NEXT loops, especially with variable length data, consider using the "incremental" modes that those kind of instructions provide. Redo your code so that for instance it only executes 1 cycle or perhaps 10-20 cycles per scan maximum. In addition, watch out for buffer overflows with BTR/BTW instructions. You can only have about a half dozen of these running simultaneously. I usually start setting up timers and manually scheduling them if I have more than 6-8. Otherwise they will cause some serious problems in your RIO system (poor response times). In addition, if this doesn't work and some of your code is low priority, you may want to schedule it periodically and/or take steps to manually run code in separate subroutines. For instance, a big totalizer routine that only runs once per second/minute/hour/day should be loaded in a separate subroutine that you JSR to only as-needed. In most discrete manufacturing plants, typical scan times are under 25 ms. In a process oriented plant with massive remote IO (full local IO, 2 racks of extended local IO, over a dozen 1794 standard remote IO blocks), and over a dozen PID loops, I've had scan times of 120 ms and peaks over 200 ms. Moving the totalizers into separate subroutines called on demand, slowing PID loop updates and putting them on an STI, and switching several FFL/FFU instructions to ring buffer code dropped typical scan times to 45 ms with peaks around 80 ms. Finally, watch out for multiple MCP's. If you use multiple MCP's, the processor will do housekeeping (3-10 ms) between EACH MCP. In addition if you don't turn it off, it will also do an IO scan. AND, I think it will always do IO on the communication (serial, Ethernet, DH+) ports even if it doesn't scan IO (you would have to check with AB on this). I once found a program which used MCP's instead of JSR's and had 13 MCP's. When I turned them all off and used simple JSR's, the scan time went from 120 ms down to 20 ms! As to the IO...the most I've ever done is about 500-1000 IO points. You are well beyond that; 8 chassis X 15 cards per chassis X 16 IO points per chassis = 1920 I/O points. This is quite possibly getting very excessive and your scan times (and response times) probably act like it. It doesn't sound like you are really exceeding any limits but I've never seen any system this loaded up without adding additional PLC's. First, you could put another PLC-5 in place of one of your ASB modules. Split your RIO run into two runs. Connect the two PLC-5's together with DH+. Move half the code into the second PLC and do any inter-PLC communication with MSG commands. Now you've cut the IO load in half. Second, you could migrate all your code to a CompactLogix or ControlLogix PLC and install one or two DHRIO modules. You will get a massive increase in processor horsepower. Your program scan times will probably be 3-5 milliseconds compared to what they are now. Program conversion isn't too hard. You can always split your RIO network while you are at it, too, since you get 2 channels per DHRIO card. In terms of hardware, cost will be similar to buying a new PLC-5, and you will have to buy new software and the learning curve going from PLC-5 to ControlLogix is pretty steep mostly because all your old IO concepts go out the window.

[salman 0]

Thanks for your thorough response.Thats the best response i got on all forums. Now let me come to the situation point wise. The situation is as follows 1- I have increased scan time + i have done "processor can restart chassis after fault" option. Now when the problem occurs the PLC DONT GO INTO FAULT STATE Now when the first time problem occured yesterday , Last 3 chassis (last 3 interms of Remote IO network) were showing NO LED on (manual says its communication entirely loss), other chassis before this were showing acting LED blinking. When we recycle power on any one of the chassis with all LEDs off(chassis power of that particular chassis) then all remote IO adaptors start communicating again. Remem in this entire process the PLC has not faulted so as soon as as the comunication establish again all IOs take the same position as before (all motors getting ON at once is tripping main breaker). When second time prob occured then last 6 chassis went in LED OFF state while first three chassis remaind with ACTIVE LED blinking. Other things remained same as above 2- I have done the testing while plant is running ...if i put power OFF on any one of the chassis only that chassis goes off , all other adaptors in other chassis remain same similarly if i take out Remote IO adaptor from any one of the chassis then only that chassis gets off but others dnt get disturb. 3- but yes one thing i have observed the normal scan time of PLC is Max scan is exceeding 515ms but last scan time remains 14 to 15 ms It seems to be data traffick porblem. Yes ther are more than 6 block transfers in the logic but how to see the buffer of it?? there is no FAL or FOR /NExt instruction aand i have checked all jump instructions (they dont produce infiined loop) u will find one in logic which i have disabled already The question is why the plant trip after such a long while and the PLC starts controlling Remote IOs as soon as REmote IO of only one chassis is power cycled. PLC is not recycled. LEt me post the program here again i m unable to attach here kindly give me ur emain wher i can sen the code its not my code.. the OEM of this chip board manufacturing company is Washington IRON works who has devleoped this extensive code

[BobLfoot 301]

With PLC 5 you can scan times of routines as well as main program. If you are insistent that it is I/O and not Program the split your i/o up. IF you are not using the DH+ then change channel 1A to RIO and use another BLue hose and pair of resistors to place racks 2 thru 7 on 1A and leave 10 thru 17 on 1B. This should allow you to eventually thru divide and conquer find your problem. BTW - I had a PLC 5/80 with Racks 2 thru 37 spread across channels 1B, 2A and 2B and Rack 37 was a PLC 5/40 in Adapter mode with racks 2 thru 17 on RIO of it's own. So this load can work, but the layout and programming is critical.

[newpageboba 7]

I don't agree with: "3- All the Remote IO cables were checked and if there was some error in cables then the system must not get OK at once after power up." I have seen incorrect installation run for a while and then fault for no reason. Can you post a wiring diagram of your Blue Hose (RIO)?

[paulengr 44]

If all your scan times are 10-15 ms then I'll buy off on the idea that it's most likely a communication problem. So I'd go back to inspecting connections very closely using the above suggestions. One of the most common problems with RIO wiring is that a small ground or even signal strand gets loose an intermittently touches another one. Vibration and temperature makes this fault hard to find since it is intermittent. The second most common problem is noise induced via the shielding (ground). The third problem is when people try to get away without the resistors since it often works on short runs or a small number of IO cards.