Ethernet Communication Intermittent

[Infrin 0]

Hello. We have a machine with a Mitsubishi Q26UDHCPU PLC and a QJ71E71-100 Ethernet card. We use GXWorks 2. We use Kepware for data acquisition to/from the PLC. The PLC will randomly just stop communicating with Kepware. This issue has been ongoing for the last few years but has gone from once every two months to five times a shift. It'll come and go. One week it will do it frequently and then the next week it will work perfectly.

We get no errors when the communication stops. No errors on the Ethernet card and no errors on the PLC. The only thing that I have (very recently) noticed is that we get a generic warning on the processor in System Manager. I believe the warning is 10006.

What we have done to troubleshoot this so far: 1) changed the IP address of both HMI and PLC, 2) change the switch, 3) change to a completely different switch in a different part of the plant, 4) ran new Ethernet cables, 5) change the QJ71E71-100 with a brand new one, 6) installed a second QJ71E71-100 to create a private network for the HMI, 7) added channels to the Ethernet card in GXWorks2 for the HMI, Kepware and Melsoft Connection, all Unpassive, 8) changed the "Execute the process as the scan time proceeds" under PLC System tab of "PLC Parameter" from 10% to 15->30%.

The PLC stops pinging to my desktop when it stops communicating with Kepware. What I have noticed during this is that we have an HMI that is on the same switch with an IP address that is one number above the PLC. It maintains connection to the PLC always. Building off this, I found that plugging in my laptop to the same switch with an IP address on the same VLAN, it will ping to the laptop.

Does anyone have an idea why it keeps losing communication? Or what I can do to fix it. Our next step is to buy a new processor, the Q26UDEHCPU.

Edited 5 May 2021 by Infrin

[Theuns 14]

Any VSD's Located close to or in the same panel as the PLC ?

[Infrin 0]

Yes, there are.

[Theuns 14]

We had a similar situation and actually found that one of the VSD's was the culprit .

Every time that the specific drive was starting we lost communication until it stopped .

Swopped out the specific drive and installed extra filters to suppress noise even more and the problem was solved.

   

[panic mode 246]

yes, check if comms cables are routed away from sources of noise (VFDs, power cables etc). check grounding. if needed add divider plate between controller and VFDs (sheet metal, grounded of course). if this interferes with cooling, add some holes in it. check if drives are installed per manufacturer specification (clearances, shielding, grounding, reactors / chokes, ballast resistor etc.)

maybe it is not the EMI interference, perhaps the mains drop is overloaded and starting/stopping VFDs causes brownouts or transients. perhaps also check or replace 24VDC PSU (not all of them are created equal).

another possibility could be that something is wrong with the network as well. could be duplicate IP addresses, or that one or more of the nodes have incorrect subnet mask. are the addresses static or dynamic? if static, are they still on a same network as DHCP server? could it be that assigned static addresses are in range that DHCP server uses? that could create conflicts. it could also be the traffic is heavy or perhaps ringing. this can be result of redundant paths to same node. better ethernet switches have ability to handle this and use other path only as a fallback... this is easy to test by making sure all nodes are connected in star topology without redundant connections. i would make sure to draw topology as a reference. 

then it could be bad connectors/cables... i have seen my share of broken retention clips or cables with incorrect pinout or just broken wires.

 

 

[Crossbow 295]

Shielded Ethernet cables help too!

[Infrin 0]

Thank you guys for your replies!

We are now planning to add a filter to the processor and change and reroute the Ethernet cables to shielded types.

We came across something last week. We found that we can plug a laptop into a different switch, give it an IP address of the same vlan as the machine, and ping the machine. This causes the machine to start communicating again when it stops.

[panic mode 246]

that is odd... ping has no effect on other connections. but if one of the switches had a glitch or lost the port table, this could have triggered rediscovery of what is connected to it. maybe try to test another switch in its place?

[Infrin 0]

Originally we thought it was the switch so we replaced it and eventually moved the PLC to an entirely different switch.

However, it could be upstream from those switches.

But, I have to prove out "my side" of it first. Something is causing my PLC to not get past its gateway.