Sign in to follow this  
Followers 0
JRoss

Automation Riddle

39 posts in this topic

I was reading the "PLC Law" thread and got inspired to start a new thread. I've only been doing this stuff for about two years, so I still have some learning to do! I just had an experience that showed me the importance of including code that can easily eliminate the PLC as a problem source. That part didn't end up being a big deal, but the problem had us pulling out our hair for a while. The answer ended up being kind of interesting though. I'm going to pose this as a riddle, because I want to see how fast you experienced automation engineers can figure it out. I know you'll be at a disadvantage, not being able to see the machine, but I'll give as much information as I can. Here's a sketch of the problem: I just had a customer's machine start acting strangely. Using pneumatic pinch rollers with a VFD on the drive wheel and an encoder on the pinch wheel, the machine feeds straight rebar into position for another operation. There are two of these machines in existence, and they have both been working perfectly for months. Two weeks ago, one machine started feeding bar erratically. Where before we had about a half inch of variation, we started getting 6-12 inches of variation, sometimes stopping too long and sometimes way short. The end-user blamed the program, which is why I was called in. The system is using the Mitsubishi QPLC platform with a QD62D counter card for the input from the SICK|Stegmann differential encoder. The VFD is an A700 with a CC-Link card for communications back to the QPLC. Based on setup information from the GOT1000 HMI, the PLC calculates how far (in encoder counts) the bar must be fed. The bar is fed to the pinch rollers by a second independent feeder system (also driven by an A700), the counter card is enabled, and the two feeders begin feeding the bar together at about 40Hz. When the bar gets to a preset distance from the end of its feed length (about 2 feet), the feed speed drops to 5Hz. Once the encoder count is at or above the calculated number, the PLC shuts off the feeders, and moves on to the next operation. I'm simplifying things somewhat by not explaining the full machine system, but this is the problem portion. Let the "20 questions" begin!

Share this post


Link to post
Share on other sites
Encoder feedback missing pulses (wheel slipping)?

Share this post


Link to post
Share on other sites
Do you mean mechanical slip? As far as we could tell, there was none. The end-user physically marked bars and moved the bars back and forth to check for that. Also, missing pulses due to slip would only account for the bar being fed too far, but it occasionally didn't feed far enough.

Share this post


Link to post
Share on other sites
that happens when cable is not good or you have ground loops etc. my questions? - do they want it fixed or just need someone to blame so they can get freebees (like something new, another alarm, indication, button or fan...)? - how badly they want it fixed? - do you have right tools (something to sit on, put laptop, power it, right cable etc.)? - do I really really really have to use protection (glasses, hard hat,...)? - where are important places (caffeteria, washroom, telephone, vending machines...)? - who is going to have final word to accept any work as complete and how do i find him (cell phone, pager...)? - how close to machine you can find free parking spot? don't want to carry laptop and drawings for 3 miles. - did someone mess with it or abuse it? (changed hardware, altered program, adjusted anything, do short visual inspection of machine and panels for changes and or damage) - is the product still ok or they are trying to run something new? - what do operators say or person who reported problem? - are there any alarms? - is power ok? - is feedback working correctly? (cable, terminations, shielding, loose connection, loose power connections etc. is encoder reading really spot on when moved manually back and fourth several times to the same mark?) - is edge detection working correctly (dirty photoeye or changed position or missing reflector etc.) - how is drive interfaced with plc? digital I/O? (are there any loose connections or delays caused by worn or sticky relay maybe...?) - does it start and stop as commanded and is speed control ok?

Share this post


Link to post
Share on other sites
I'll deal with theses one by one: Drives are interfaced using CC-Link, a Mitsubishi protocol. No problems there. The drive starts and stops as commanded, changes speeds, and general operates as any well-behaved drive should.

Share this post


Link to post
Share on other sites
My first question when faced with a situation like this is "What's different between now and when it was working properly"? If we're looking at the feedback, stopping too long means missing pulses from the encoder. Stopping too short means extra pulses from the encoder. Is it a two-channel encoder with the counter card set up for quadrature counting so you can detect motion in both directions? If not, how do handle the possibility of reverse rotation? Are the encoder counts where you command slowdown and stop exactly related to the desired distance or is there a "fudge factor" included in the calculations? In other words, if you have a one foot circumference wheel attached to a 1000 PPR encoder, then to feed 20 feet, your count would be 20000 without any "fudge factor". Is the speed the same now as when it was working properly? Are you running the same grade/style/size rebar now as previously? Are you loading the pinch rollers at the same air pressure now as previously?

Share this post


Link to post
Share on other sites
Jeremy, you don't seam to be 100% sure that reading from encoder is 100% correct. that's bad. you do need to verify that. take the encoder of the machine, put masking tape on the shaft and make a mark, turn it X number of revolutions forward and then back to same position. reading must be same (within something like 1 or 2 degrees). then secure encoder shaft with piece of tape, (should have stable reading now), write that reading down and try moving terminations (how is this powered? 5V power supply? is this solid?). if all of this wiggling didn't produce change in reading, your feedback is probably ok. if this is not ok, you found your problem. scaling etc is something that happens later. your raw value reading must be repeatable or you have no reliable feedback. it's simple as that. if these things ware ok when encoder is taken out but you get reading errors when encoder is installed on machine (and you repeat moving back and forth to the same mark), something is slipping and it's mechanical issue. if the problem is feedback related, program didn't change (code and settings), cable is replaced, you still have to check if things other than program and cable are ok. get the scope (buy, rent, borrow) and check that power supply (not at the power supply, check at the encoder cable terminals, there could be loose wire from power supply to encoder cable or routed near noise source such as power supplies, drives etc.), then check the signals and see if you get neat square waves without spikes etc. how sound is shielding of the cable? 6 inches of shield stripped to be able to terminate? that's not good. if the issue is with the program and or parameters, compare what you have found with the old program to see if there are changes. some customers like to have their guys add things but that can interfere with initialization fo the high speed counter, or reset it when it shouldn't etc. if the raw value was found to be fine, move on and see if something happens to the scaling part (another HMI button or numeric entry writing to wrong plc address, scale value was modified etc.).

Share this post


Link to post
Share on other sites
Let's see if I can get the operation down to a sequence and correct. 1. Operator uses HMI to specify length of billet wanted. 2. PLC calculates this length in terms of encoder pulses. {For argument lets say 10,000 counts target length and 7,500 counts slow down target} 3. The PLC then turns on an internal running flag and sends the speed command to the VFD. {40 hz} 4. The VFD after some network {CCLINK} propogation delay ramps up to speed. 5. The VFD runs to drive wheel to speed. 6. Drive wheel is held against billet by some pressure agent which you didn't specify and which might be varying. 7. The billet is also held against the encoder wheel by some pressure agent whcih you didn't specify and which might be varying. 8. When the counter reads 7,500 counts the plc sees this count and issues the slow speed command {5hz} {Reaching count and PLC scan are async so slow command can vary by 1 scan time of PLC} 9. The VFD after some network {CCLINK} propogation delay ramps down to slow speed. 10. When the counter reads 10,000 counts the plc sees this count and issues the stop command {0hz} {Reaching count and PLC scan are async so slow command can vary by 1 scan time of PLC} 11. The VFD after some network {CCLINK} propogation delay ramps down to stop. 12. IF you have debug logic when the VFD responds stopped you log the count value. IS the logged count value consistent or widely varying? What mechanism applies pressure between billets and wheels? Any network or program changes which might affect response times?

Share this post


Link to post
Share on other sites
mechanical slip between the driving motor and the part?

Share this post


Link to post
Share on other sites
We had a problem similer to this once. machine would intermitantly screw up as if out of time. turned out to be coupling on the encoder shaft was cracked but looked intact on quick visual inspection. by using indexed addressing to .10 sec. snapshots of the encoder count. and logged into a data table. when the machine screwed up again we checked the log. the encoder actually counted backwords for about a1/2 sec. before continuing forward count. took a closer look at the encoder and found the coupling was broken. the crack was barely visible. on occasion during operation it would slip and then catch again at the point it was broken.

Share this post


Link to post
Share on other sites
I have seen something similar to this before also. I have seen that the motor (with built in encoder) was not coupled tightly to the leadscrew causing slippage in the system which caused both long and short measurements. Long, because, when the motor stopped, the momentum in the ball nut which was mechanically clamped to the part (in this case 6"x6"x1/2" x 40' long angle iron) caused the part to move forward. Short, because, the weight of the angle iron caused the lead screw to "drag" inside the loose coupling. I have also seen that if you use an encoder pulse as a PLC input, you need to ake sure that the input is a high speed input, or the counts will be erratic. Also, have you checked the calculation that is being done for the number of pulses for a required length? If the number of required pulses is greater than the buffer can handle (GE PLC INT = 32767 counts) you might need to change to a different type of register. ( GE PLC = Double INT). Edited by cmoore73

Share this post


Link to post
Share on other sites
I know you stated that all the cables were properly shielded and grounded, but I have seen instances where encoder signals STILL had noise on them, and miscounted horribly. Have you looked at the waveforms from the encoder to verify the signals look normal (i.e. 50% duty cycle @ rated voltage, 90 deg phasing between a & b pulses, valid a/ and b/, zero pulse ok (if used), little to no capacitance on signal, little or no noise or spikes, clean square wave, etc.)

Share this post


Link to post
Share on other sites
Wow, I've been out of the loop too long! Here's answers to each of you: Steve: Nothing was different in the machine or setup, we were running the same size rebar, and we checked and adjusted the air pressure on the pinch rollers. Intially we were running the same speed, though we slowed things down to see if that made a difference. It didn't. It is a two-channel differential encoder, and the counter card is set up for 2-phase quadrature counting, able to handle both directions. There is no fudge factor in the calculations, and they were working perfectly. panic mode: Removing and testing the encoder yields perfect results. We tried also replaced the encoder twice, and even tried a different brand. Switching from Stegmann to Encoder Products made the problem ten times worse. The encoder cable was checked backwards and forwards, and appears perfect. I'll address the results of the scope in my reply to mjisaacs. We replaced the current program and parameters (after saving, of course!) with a known working version. The counter number is not scaled. I convert the bar length to counts and do a simple comparison. All that works. BobLfoot: Your sequence is correct. I did have the presence of mind to log the count value after the VFD was stopped, and the count was extremely consistent. A variance of about 4 counts (0.11mm, linear). There were now network or program changes. Feed system is as follows: A fixed metal wheel with a ridged gripping surface is driven by the VFD/motor. A second identical wheel is on a free-spinning shaft that is attached to a pneumatic piston, which pushes the second wheel almost against the first. The rebar is pinched between the wheels. The encoder is coupled to the free-spinning shaft on the second wheel. TERdON: There is no mechanical slip, neither between the wheels and the rebar, or in the shaft/encoder coupling. cmoore73: We confirmed that the coupling was good. We also checked that the counter-card was setup properly. Remember, there is an almost identical machine that is working fine, and is actually a month older. The counter buffer is a double, and I made sure that I wasn't overwriting any registers. We actually did all our testing on bar that was half the maximum length. mjisaacs: We finally got hold of an oscilloscope and checked the channels. There was no discernable noise on the channels, no spikes or whatever. Turning the encoder by hand returned beautiful square waves in their proper places. However, when we ran bar through the machine, the square wave jumped around a lot. Not noise, and the squares were still fairly clear. But they weren't very even. Sort of skittering around.

Share this post


Link to post
Share on other sites
Stupid question, was the encoder measuring the mill scale on the rebar, or the ridges???

Share this post


Link to post
Share on other sites
Nope.

Share this post


Link to post
Share on other sites
Is there any pattern to the occurrence of long/short feed or does it appear random? What is the difference between the Stegmann encoder and the Encoder Products model? When you say the square wave pattern was "skittering around" when viewed on the scope, do you mean there was a variation in amplitude, frequency or phase?

Share this post


Link to post
Share on other sites
There was no pattern to the error, is was completely random and completely frustrating. By "skittering", I mean there was a variation mostly in frequency. The "skittering" didn't always stay in phase either. And yes, this variation is what caused the error. But what caused the variation? The Encoder Products was a direct cross from the Stegmann, so unless the distributor made a mistake the only difference was brand. I want to be honest here. As you will see if you click the link in my signature, I work for a distributor who sells Stegmann. We provided the original encoders for the machine, as part of the controls package. The machine builder wanted to try a different brand of encoder, and so they called another distributor. I wasn't even there when they switched encoders, so I didn't see the operation difference. I'm basing my information on their report. Edited by JRoss

Share this post


Link to post
Share on other sites
Well, a variation in pulse frequency indicates a variation in speed, but the encoder and high speed counter card should be able to handle that. A phase shift between channels A and B will result in count inaccuracies. Are you looking at something induced by vibration? Something that began abruptly rather than getting gradually worse could have been caused by a mechanical jam bending something in the drivetrain. Has the rebar worn a groove in the pinch rollers?

Share this post


Link to post
Share on other sites
What about magnetism. Are the rebar magnetec in any way? chas

Share this post


Link to post
Share on other sites
More questions before answers. 1. What is the distance between Encoder and Hi Speed Counter Card and more importantly what distance is specified as maximum for this encoder? 2. What is the cable routing and has anything along the route changed {new lighting fixture, new panel board, etc}? 3. Is this shielded cable and how is it grounded?

Share this post


Link to post
Share on other sites
Is the sheild hooked to a ground on both ends? If it is it will act as an antenna to any electrical noise around it. The noise will happen whenever the devfice generating the noise runs. Also if the groung is hook to the same ground as the drive you will induce noise that way. Drive grounds should always be tieed back to the motor they are driving and then to the ground bus so you create a ground loop between the drive and the motor.

Share this post


Link to post
Share on other sites
Steve: See if the attached image helps at all. It's a wave diagram that's pretty similar to what the o-scope gave us. There were no jams that caused the issue. It started fairly abruptly, but did get a little worse over time. The feed wheels had no worn grooves in them. chas183: No, the rebar is not magnetic in any way. BobLfoot: I didn't measure the distance exactl, but it was around 20 feet. It was with the max distance of the encoder. The cable was originally routed inside a cable trough that carried signal, communications, and power for the control system. Nothing along that route has changed. During our testing, we ran the cable outside of the cable trough, and as far away from any noise source as possible. The cable is shielded, and the shield is grounded only at the counter card end. Clay B.: The entire cabinet is grounded properly. Everybody keeps asking about the shielding/grounding issues. And yes, that is the most natural response. It was what I suspected up until we found the solution, and what all the tech support people I talked to said. However, since we have found and corrected the real culprit, I can say for a fact that EMI noise had nothing to do with the problem.

Share this post


Link to post
Share on other sites
Could it be a loose connection somewhere perhaps?

Share this post


Link to post
Share on other sites
There are no loose electrical connections.

Share this post


Link to post
Share on other sites
How do you ensure that the two drives are feeding the bar at the same speed? If one drive is faster than the other, they're going to fight each other. Either the upstream drive is going to try to push the downstream motor faster than it wants to go, the downstream drive is going to pull the upstream motor faster, or one of the sets of pinch rollers is going to slip.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0