Ethernet/IP communication with Allen Bradley PLC in network

[Deeksha 0]

I am a newbie when it comes to Ethernet/IP communication and Allen Bradley PLC setup. I have written a program in PLC to communicate with my device over Ethernet/IP communication. Communication is work fine for 1-2 hours, but later class 3 communication stops. 

I captured wireshark log to debug the error. But on wireshark, I see the communication between IP address of PLC and PC port only. There is no device IP captured in wireshark. 

My first question is -

Is there any setting in PLC and wiresahrk to capture class 3 packets?

Also, in wireshark, I see error packets "Embedded service error" when I connect two devices in line topology. When checked on Studio5000, Module shows ring fault status. 

I am not sure why it took as ring topology, while my device is connected in line topology. I am not able to change it from ring to any other topology. 

Can someone guide me here, how can I remove this ring fault? and what does this "Embedded service error" means?

Thanks in advance!!!

 

[pturmel 160]

Share more details.  What is the network topology?  Are both PLCs Allen-Bradley?  Are you using message instructions in one to talk to the other? If so, show the three tabs of a message configuration.

Embedded service error indicates that a request used message router service 0x0A to embed multiple requests in a single packet, and one or more of those inner requests failed in the target device.  You would have to inspect the nested responses to determine the true error or errors.  (Wireshark will help you do that.)

[Deeksha 0]

22 hours ago, pturmel said:

Share more details.  What is the network topology?  Are both PLCs Allen-Bradley?  Are you using message instructions in one to talk to the other? If so, show the three tabs of a message configuration.

Embedded service error indicates that a request used message router service 0x0A to embed multiple requests in a single packet, and one or more of those inner requests failed in the target device.  You would have to inspect the nested responses to determine the true error or errors.  (Wireshark will help you do that.)

Thanks for the reply. My two devices are connected in line topology, Device 1's LAN  port 1 is connected to router and LAN port 2 is connected to device 2. Then PLC is connected to router. router is connected to my PC.

only one PLC and one EIP module, both are Allen Bradley. PLC - 1756-L81E and IO module - 1756-EN2TR. 

Yes, using msg instruction to talk to my both devices. snaps for three tabs are attached.

On wireshark, I am able to see packets only for port between PC and router. How to inspect nested responses in wireshark? I tried to look into the packet, but couldn't decode. 

[Deeksha 0]

Also, for my device 2, where port 0 is connected to device 1 and port 1 is unused. But on studio 5000, I see both ports are checked with port 1 marked as inactive. 

I tried to uncheck the port 1, but it says that can't perform operation due to module mode. I don't understand how to fix this. snaps are attached. 

[pturmel 160]

Wireshark can only capture packets that arrive at the PC's physical network interface.  That often requires using a special mode on your switch to "mirror" everything to the PC's port, and ensuring that all devices are connected to the switch.  You will not be able to capture traffic from daisy-chained devices.  (The dual ports on AB devices should only be used for real rings.)

The mode problem suggests that the device in question is "owned" by something.  What are these devices?  Please also show your Studio 5000 I/O device tree where Device1 and Device2 are present.

[Joe E. 259]

2 hours ago, pturmel said:

...  (The dual ports on AB devices should only be used for real rings.)...

I've seen many applications that used the dual ports in a daisy-chain configuration. Is the main drawback that all downstream devices go away if one in the middle is powered off? Or is there another reason too?

[pturmel 160]

10 hours ago, Joe E. said:

downstream devices go away

Primarily.

10 hours ago, Joe E. said:

another reason too?

Troubleshooting problems in cases like this.  Most modern managed switches have features to help technical staff connect wireshark to specific ports, so that sources of bad behavior can be examined unfiltered, and without relying on an intermediate two-port switch.

Actual DLR, though, is awesome.

[VFD Guy 13]

Port mirroring is complicated. Either find an old hub or use a sniffer is the easiest. I use shark tap. 

[pturmel 160]

11 hours ago, VFD Guy said:

find an old hub or use a sniffer is the easiest

No question about that at all. But I am often not on-site, so a port mirror that can be configured remotely has great value.  And, sometimes, simply moving connectors masks a problem.

[pturmel 160]

I just discovered that my Mikrotik router can do packet captures for me on selected interfaces and stream the results to a Wireshark instance on my laptop.  No need for the port mirror at all (but HW offload has to be disabled on the affected ports).

[Deeksha 0]

On 24/4/2024 at 5:36 PM, pturmel said:

Wireshark can only capture packets that arrive at the PC's physical network interface.  That often requires using a special mode on your switch to "mirror" everything to the PC's port, and ensuring that all devices are connected to the switch.  You will not be able to capture traffic from daisy-chained devices.  (The dual ports on AB devices should only be used for real rings.)

The mode problem suggests that the device in question is "owned" by something.  What are these devices?  Please also show your Studio 5000 I/O device tree where Device1 and Device2 are present.

Currently router I am using here doesn't support port mirror. I have requested team to get the router supporting port mirroring. I should get that sometime in next week. I guess till then I won't be able to debug "Embedded service error" in detail, I am stuck here, I guess. 

"Owned" by something - didn't understand this? 

Studio 5000 I/O device tree snap is attached. Device 1 and Device 2 are the I/O devices in which we have implemented Ethernet/IP communication. 

[pturmel 160]

6 hours ago, Deeksha said:

Device 1 and Device 2 are the I/O devices

Right, that means your PLC "owns" those devices.  Most I/O devices do not permit many, or any, configuration changes while a scanner owns them.  You will need to inhibit those modules in the I/O tree to be able to write configuration attributes.

[VFD Guy 13]

By owned he means implicit comms or cyclic. When a device is in cyclic ie implicit, class 1, that device only responds to the owner so to speak. I’m simplifying here. You can still do explicit but only 1 owner for cyclic

[Joe E. 259]

I know some devices at least used to allow a "listen only" connection where additional "masters" can read data out but can't send commands. For example, more than one PLC could read the input/output status but only one PLC can control the configuration and outputs. Not sure if these support listen-only and it's not relevant to OP's original problem, but it could be useful to know at some point.

[pturmel 160]

Every EDS file lists the kind of class one connections that are possible for the given device.  Listen-only connections only work when a master is already connected, and typically requires multi-cast mode for the T->O connection direction.

[Deeksha 0]

On 26/4/2024 at 5:34 PM, pturmel said:

Right, that means your PLC "owns" those devices.  Most I/O devices do not permit many, or any, configuration changes while a scanner owns them.  You will need to inhibit those modules in the I/O tree to be able to write configuration attributes.

Understood, Thanks. I will try to inhibit and check. Will update the outcome.  

[Deeksha 0]

On 1/5/2024 at 8:38 AM, VFD Guy said:

By owned he means implicit comms or cyclic. When a device is in cyclic ie implicit, class 1, that device only responds to the owner so to speak. I’m simplifying here. You can still do explicit but only 1 owner for cyclic

For my setup, cyclic communication starts as soon as I power on.

You said that 1 owner for cyclic, I got confused here. I have connected two I/O devices with one PLC. When I connect only one I/O device with 1 PLC, it works fine. But when I connect multiple I/O devices with 1 PLC, I start getting "Embedded service error". Is some additional configuration is missing while creating topology? 

[pturmel 160]

Without sharing packet captures for the traffic that includes the embedded service errors, I don't know any way to help further.

[Deeksha 0]

On 2/5/2024 at 5:27 PM, pturmel said:

Without sharing packet captures for the traffic that includes the embedded service errors, I don't know any way to help further.

I have got switch which allows me to mirror ports. Here are the wireshark trace. Setup is same, two I/O devices are connected in line topology and one PLC. 

IP address Device 1: 192.168.1.16

IP address Device 2: 192.168.1.17

PLC(1756-L81E) : 192.168.1.25

Ethernet/IP module (1756-EN2TR): 192.168.1.32

Laptop: 192.168.1.14

Trace captured on mirror port shows that "Embedded service error" is between laptop and PLC, didn't observe error packet on device IP address. 

 

Not sure what's the issue for this embedded service error. Please suggest if anything to look into. 

PLCtoLaptopTrace.pcapng

TraceOnMirrorPort.pcapng

[pturmel 160]

For some reason, those messages didn't auto-decode in Wireshark like usual.  I'll have a chance to manually decode them later today.

[VFD Guy 13]

Can you tell us what these devices are?  Static ip? Network topology?

Edited 8 May by VFD Guy

[Deeksha 0]

1 hour ago, VFD Guy said:

Can you tell us what these devices are?  Static ip? Network topology?

Two I/O devices that we are designing. 

Static IP for devices are: 192.168.1.16 and 192.168.1.17.

PLC IP address: 192.168.1.25

EIP module: 192.168.1.32

Laptop: 192.168.1.14

Both devices are connected in line topology, device 1 to device 2, device 2 to network switch. PLC to network switch and network to laptop. 

[pturmel 160]

On 5/7/2024 at 7:30 AM, pturmel said:

manually decode

Just did the first one.  The "errors" aren't errors at all, but "partial data" markers (CIP status code 6).  Logix returns these when a response won't fit in the return packet and the originator is required to make a follow-up request for continuation.

There were three of these in the one packet, and are perfectly normal.

[pturmel 160]

Spot checked some more.  Same pattern.  I think the originator's program is too dumb to forecast the expected return sizes to avoid large responses in return packets.  If a commercial package, can you name them so I can shame them?

[Deeksha 0]

13 minutes ago, pturmel said:

Just did the first one.  The "errors" aren't errors at all, but "partial data" markers (CIP status code 6).  Logix returns these when a response won't fit in the return packet and the originator is required to make a follow-up request for continuation.

There were three of these in the one packet, and are perfectly normal.

So I get these "Embedded service error" packets only when I connect more than one I/O device in topology. Could it be a chance that some configuration in PLC is not correct for topology? 

Also, I am little confused here for originator. So in trace, these error packets are returned by PLC (192.168.1.25). As per my understanding, originator is the controller (PLC), but just to confirm, what would be the originator here, RSlogix5000/PLC who sends command or I/O device who receive the command and sends feedback? 

I mean what I should debug, PLC/RSlogix5000 or my I/O devices?

My communication flow is : I send command from logix5000 to my I/O devices and receive command feedback on logix5000 from I/O device.