Hi,

Is it good practice to mix industrial ethernet with your office network. I have a situation where I have a allen bradley slc 505 plc and I want to access it from a remote location. All I want is an ordinary phone line connection so I can dial into the plc or computer.

The IT man has given me a broadband connection which means I have to access the plc through the office network. He gave me addresses for me to give my equipment and he connected the plc to the network and the network flooded with messages. I tried to tell him that industrial networks were not exactly the same as office networks but he wouldn't listen.

He is now getting a managed switch to connect the plc to the network.

Personally I would prefer to keep them separated from each other. I'm relatively new to networking so I would not really know much about managed switches etc and my " IT man" knows less than me but will not admit it. My level consists of connecting panelviews to plc's and setting up addresses.

Can someone give me advice or their opinion.

Thanks

That's really a great question. It depends more on your requirements than a cookie-cutter answer can provide. The first question is, "why?" If you're into separation for absolute security the answer is simple - physically divide your networks. For most applications the gain comes from the front office gaining process visibility and the plant floor gets the benefit of IT resources (VPN access, data logging to SQL databases, etc, etc).

My recommendation is this - if you don't have a clue what you're doing, keep the networks separate - it never gets worse than the obvious. A poor combined setup, like hubs daisy chaining busy networks together, can be dangerous. But controls guys tend to exaggerate the problems. They have a bad taste in their mouths from misapplied older technologies. Also, the convergence of IT and controls will give you less and less liberty to make that decision.

However, there are pretty simple things that you can do. For example, separating production from office with VLANs (virtual LANs, a common managed layer 2 feature) works well. Traffic is not passed between them, unless you want it to. This sort of scheme makes it pretty hard for traffic/packets on one side to affect the other. You get the benefit of being able to dynamically place machines on one side or both.

VPN access is much better than a dial up. It's considerably: faster, more secure, and more flexible.

Ultimately, it depends on the quality of the IT guy. If he's squared away he should be able to provide you with a transport network that is more robust and secure, and provide you with remote access, backups, and other conveniences. In that case I'd gladly follow his IP scheme. Your migration considerations might come up if you have a lot of legacy hardware that causes problems. Otherwise, what difference does it make to you? You should probably be using non-routable IP addresses following some pre-defined scheme anyway. If you jump on the IT guy's system and he does stupid things, you'll find yourself down more. Simple as that.

here is my two cents - OK maybe three - LOL

1. Keep Office and Plant Floor seperate except for a limited number of managed switches which will connect them.

2. Develop a numbering scheme with you IT poeple which makes sense and clearly identifies equipment quickly. for example My Hardwire Office IP is a 153.Bb.cc.dd , my wireless IP is 172.30.CC.DD, my PLC's and HMI's on the south of the plant are 172.30.11.DD and on the north of the plant 172.30.36.DD. Were adding 172.30.44.DD for the middle section of the plant later this year.

3. Read the tutorials and other TCP/Ip material here on mrplc.

I guess my question is specific to this situation. Why the flood of messages when you connected the 5/05? It should only speak when spoken to unless it has a slew of Ethernet MSGs in the program. Or do you have an HMI connected to it via Ethernet? Even then they usually don't request that much data. Of any of the AB products you can put on a plantwide network I would think the 5/05 would be one of the safest.

Keith

Bob - great advice. Do you consider VLANs to be "separate"?

I can be swayed by arguments of physically separating equipment and locking it up because knuckleheads do stupid things. It is important to convey to IT that they communicate with you prior to doing things like firmware upgrades if you're riding their infrastructure. I still hold that they can provide you with better services and support (at the transport level) than you may be capable of doing yourself (unless you live on site and happen to be a network guru).

Thanks for the compliment Nathan. I am not enough of a network guru to understand VLAN and render an opinion. I know that one place I worked we used CISCO switches which had 24 ports with 12 mapped to fiber A and 12 mapped to fiber B. Fiber A was office, Fiber B was PLant Floor. The only place they met and crossed was at a router in the Data Center.

We have placed most of our PLCs on the network. Our PLCs, HMIs, and RSView/Citect stations always sit behind a managed switch (there are several of these for different plant areas), effectively dividng the physical network into an office network and several controller networks to segregates the traffic. The PLC and RSView station can be on the plant network. IO networks are always seperate - that means two e-net cards in the chassis, but thats how we do it. Since we use the CLX platform, I can still browse through the chassis an onto the the IO network form my office to access drives, etc.

If you're just watching the traffic, you should expect to see a stream of messages all the time even when you are not connected to the PLC. These are broadcast traffic messages. For instance, all PC's will broadcast "beacon" type messages if they are running Microsoft's old networking protocols. Ethernet/IP's "discovery" protocol also broadcasts incessantly, especially if you don't have layer 3 switches doing IGMP snooping/negotiation. Anything that uses DHCP will broadcast a message when it first powers up to get an IP address. When you first power up a switch and during the first packet exchange from a new device, it will broadcast until it learns the MAC address mapping. When I check the managed switches in my network, there are usually about 30-40 broadcast MAC's in use in the buffer....and we do use VLAN's to a degree. All of this background "stuff" is perfectly normal on pretty much any network. Without heavy use of VLAN's and managed switches doing traffic cop functions, it is essentially impossible to eradicate this traffic.

All of this traffic will make your packet counters go up but as long as you don't get to the point where you are seeing hundreds of packets per second of broadcast traffic, it's harmless. Most PLC's, including the ControlLogix platform, tend to be limited to dealing with about 1000 packets per second or less. And with just broadcast traffic, it should well below that, because that 1000 packets per second includes your own expected communication.

IF you really want to look at what it is, you can download "Wireshark" or "Ethereal". The learning curve is a bit steep but what these programs do is that you can capture the entire packet data stream via a PC and examine it to see exactly what really is passing across your network. You can do a certain amount this way. If you really get heavy into packet analysis though you need a switch that supports port mirroring because normally switches intentionally try to minimize packet flow and don't leak unicast traffic. If you want to do this but don't want to pay for a managed switch, you can set these software programs into a mode where you use two Ethernet ports on your PC and treat it as a router, or you can buy a hub and use it temporarily (although normally I can't say enough bad things about hubs as a general rule).

In terms of your intended use, if you are just using the Ethernet port for remote diagnostics and programming, have at it. I have about 25 PLC's on my network and only one has a managed switch in front of it. The reason for that managed switch is that I'm doing long term testing. I have some limited (Modbus/TCP, not Ethernet/IP) I/O traversing the network as well. Currently there are no inter-PLC messages traversing the Ethernet network...

If you start adding in I/O, if you are just dealing with simple MSG'ing between PLC's and you put some "guard logic" between PLC's to deal with network failures (if the MSG commands start signalling errors, take appropriate fail-safe action) , then I suggest you install simple dumb switches in the network to isolate those PLC's from the rest of the network. This isn't for isolation in the case of VLAN's. It's just there to prevent someone from rebooting or reprogramming one of the switches and taking down your I/O. This has happened to me on more than one occasion.

Even with Modbus/TCP if you are doing "informational" type I/O, this is probably sufficient. This is exactly what I did with the Ethernet-based I/O that I already have. It works fine and the only hiccup I've had is that the switches are all either UPS protected (along with the PLC) or PoE and this particular I/O continued to operate in spite of a power failure (the I/O cards were also PoE which I didn't expect or plan for).

This is about as large as you want to get. In fact, we should have started migrating to more network isolation a long time ago, but I inherited a lot of network sins and I can only spend so much money eliminating them each year. Outside of the specific I/O cases I highlighted, if you go any further, you will be implementing at least some measure of traffic control or you are setting yourself up for a nasty issue down the road.

If you want to do traffic isolation/management, then there are a whole host of other things that you need to consider. By themselves, "isolated networks" and "VLAN's" are not a one-size-fits all answer to all issues. These are simply tools that are needed to get the job done. I'll also throw in RSTP, ring networks, managed switches, all the various fiber modes and connectors, and wireless into the mix that you need to understand before going any further. This is an area where you need to read up and fully understand networking, administration, network security, and traffic management. You need an IT guy with about the same amount of knowledge or you will automatically be boxed into having a separate network bridged only at the router level. Then you need to lay out your specific concerns. The network design plan will naturally fall out of those concerns.

Taking a step back for anyone needing it.

Node here is a device on the network - PC, PLC, router, etc. Usually I'm referring to Layer 3, so it has an IP address.

Collision - Bad juju. The "red light on a hub" 2 or more nodes sending a signal on the same wire. A Collision Domain is a network segment where this can happen.

A Broadcast Domain is a network segment where nodes can communicate directly (without going through a router, or equivalent). The good is that it's easy to talk. The bad is that it's easy to talk. Each node will inspect each broadcast/multicast packet (DHCP requests, streaming video, chatty traffic, etc).

What's an Ethernet hub? It's a dumb device that gets a signal on one port and retransmits on all ports - like everyone yelling in a room. One Broadcast Domain, one Collision Domain. Great for running packet sniffers (programs to inspect all traffic). Poor for performance.

What's a bridge? A bridge effectively combines nodes, or sets of nodes in separate Collision Domains, but they maintain one Broadcast Domain. So you don't get as many wire issues (network performance problems), but everyone's still listening to everyone.

So what's a switch? A switch is what you get if you bridge each port together. One Broadcast Domain, but a Collision Domain for each port (or set of ports??). In any event, you shouldn't get collisions, but everyone is still listening to everyone else.

Everything described so far falls under Layers 1 and (mostly) 2. So assume you want 2 "networks", office and plant. We might set up 2 disconnected switches. Traffic doesn't pass between them - they're in 2 separate broadcast domains.

Suppose we put the two on separate IP subnets (a layer 3 concept). We could bridge our 2 switches with a crossover cable. It will seem like devices on one switch can't talk (directly) to one another. That's because they're on separate IP networks (layer 3). But, nodes on both switches will listen to broadcasts from each other. Remember, our bridge maintains a broadcast domain.

The fix to that is putting a router between the 2 switches. A router is a device that has 2 network cards, one on each network (logically and physically) that can pass layer 3 traffic. Now chatty traffic won't pass between switches unless it's directed to an address and goes through our router. Note that your router doesn't know about ports/traffic type, that would be layer 4+ and we'd be talking about a firewall.

This is a good place to ask yourself - does this all make sense? If so...

A managed switch allows you mess with port level settings, VLANS, and things like that.

A VLAN, "virtual local area network", is a logical layer 2 network. Think of it exactly like our "separate switches" example. If we set up the top half of our managed switch ports on VLAN1 and the bottom half on VLAN2, it would appear to users as if they were plugged into separate switches. In fact, we could plug a physical router into both VLANS and it would route traffic as we described in our example. The cool thing is that you can segment VLANs by things other than physical ports (like hardware addresses) and that you can change settings without rewiring anything. This allows you to "spread" your network as you grow.

As you get more into networking you realize that a lot of the concepts are more flexible to deal with logically, rather than physically. Things also get crazy as you introduce: Inter-VLAN routing, DHCP proxies, and policies that selectively pass higher/lower layer traffic. There are also concepts like QoS (Quality of Service) that can substantially improve network performance with the same amount of throughput. There's also a lot of things that can be tuned (packet fragmenting works with frame size, latency issues, security stuff, "packet shaping", etc) that come into play if you have bigger networks especially with encryption devices, VPN tunnels, satellite links, etc.

Lots of cool stuff to networking, but if you understand the basics that I described here you should be on your way. (Paul's advice is solid - hopefully this post helps clarify some of the assumptions).

Almost. There is one more concept: "full" and "half" duplex. PLC-5 Ethernet modules that I've seen so far don't support full duplex. With half duplex, the switch port and the node attached to it form a collision domain. It is possible to get a collision if they both attempt to transmit a packet at the same time but the probability of this happening is very low. In full duplex mode, they use a totally separate pair of wires and the collision domain disappears entirely since there is only one device on each network segment.

The next issue you will not have to deal with until you bring your first ControlLogix processor into the plant. More definitions:

Unicast -- a packet with a specific source node and destination node
Broadcast -- a packet with a specific source node but the destination is "everyone"
Multicast -- a packet with multiple destinations (actually, there can also be multiple sources)

Ethernet is only capable of routing the first two types of packets (unicast and broadcast). Internet Protocol (IP) is capable of doing just about anything. Multicast packets are routed at the layer 3 level. To make the logic work correctly, they are marked as broadcast packets for Ethernet.

Multicast was something of an esoteric idea until recently. ControlLogix processors pretty much use multicasting by default for I/O and inter-PLC messages. This means that you need to use routers (layer 3 switches) or "IGMP snooping" layer 2 switches or you will flood everything with broadcast traffic. "IGMP snooping" is a way for a standard Ethernet switch to passively read just enough of the layer 3 traffic to correctly route multicast traffic.