UGA Network Design and Implementation Guidelines

Table of Contents

1. Introduction
2. Wiring Guidelines
2.1 Design Issues
2.2 Installation Issues
3. Ethernet Hub Guidelines
3.1 LAN Models
3.2 Common Hub Specifications
3.3 Shared Ethernet Hub Specifications
3.4 Switched Ethernet Hubs Specifications
3.5 Maintenance Issues
4. Bibliography

1. Introduction

This document provides guidelines that departments at the University of Georgia (UGA) can use in the development of robust networking infrastructures within buildings. The purpose of these guidelines is to maximize the University s investment by maximizing the efficiency of the network and minimizing outages while minimizing the staff necessary to provide the facilities and services.

A building s networking infrastructure involves a number of hardware components including wiring, connectors, racks, network interface cards, client and server workstations, and communications devices such as repeaters, bridges, shared and switched hubs, and routers. It also includes software such as network card drivers, communications protocols, network operating systems and network application tools. This document currently focuses on wiring and hub guidelines, but will be expanded to include other networking components in the future.

Although both Ethernet and Token Ring networks are deployed on the UGA campus, the predominant networking technology is Ethernet (approximately 95%). The predominance of Ethernet at UGA is mirrored in the deployment of networking infrastructures worldwide, and networking vendors continue to develop faster and better Ethernet products. Therefore, this document will focus exclusively on guidelines for Ethernet networking, and it is strongly recommended that all new infrastructure on the UGA campus be based on Ethernet, not Token Ring, for the aforementioned reasons.

2. Wiring Guidelines

Although building networks have been implemented on campus using thick and thin coaxial Ethernet cabling technologies (10Base5 and 10Base2, respectively), the desirable implementation strategy is to deploy structured wiring using fiber optic cabling for building backbones and Category 5 unshielded twisted pair (or Cat 5 UTP) cabling to connect end devices. Wiring for new and renovated buildings must comply with official Campus Wiring Specifications, as published in the Administrative Policies and Procedures Manual of the University. In particular, these specifications require that for all new and renovated buildings, any proposed building wiring designs be approved prior to installation. The deployment of data wiring in existing buildings should also adhere to these specifications whenever possible.

The Electronics Industry Association (EIA) and Telecommunications Industry Association (TIA) have jointly developed specifications for structured wiring. The EIA/TIA 568 specification defines a generic wiring system for a multi-product, multi-vendor environment. EIA/TIA 569 is the building specification for telecommunications pathways and spaces. It defines the minimum requirements for ducts, closets and others spaces needed for data and telecommunications wiring. The EIA/TIA 607 is the specification for grounding and bonding of telecommunications. All network components and wiring racks must be properly grounded. EIA/TIA 606 is the administration standard for telecommunications infrastructure. This specification covers cable labeling, telecommunications records, required drawings, and a method of knowing who to contact for each part of the infrastructure. Each of these standards has been condensed into a small pamphlet, which most networking vendors can provide.

Structured wiring, which uses a star topology, has a number of advantages over thick and thin coaxial Ethernet infrastructures which utilize bus topologies. First, it is easier to add, move, or change connections by moving patch cables in a wiring closet. Second, troubleshooting is easier and less time consuming since one can quickly disconnect network devices in the wiring closet rather than having to go to each workstation (or get into the ceiling) to disconnect malfunctioning devices; some vendor s management software can isolate and eliminate network problems as they occur and notify the network administrator of the problem. Third, workstations do not have to be powered down before making a cable switch, as they do in a thick coaxial infrastructure. In addition, the cable failure of one networked device does not generally affect others. Finally, new connection capacity can be added by installing additional hub devices in the wiring closet.

Cat 5 UTP cabling is the industry standard for connecting a networked device to a wiring closet. It can support data rates up to 500 Mbps using sophisticated encoding methods and will likely support local area network (LAN) traffic for the next 10 to 15 years, even though the active network components will have to be replaced in a much shorter time span to provide increased bandwidth and services.

2.1 Design Issues

In order to minimize costs over the long term, it is highly desirable to develop a comprehensive building network design that takes into account the needs of all of the building occupants. Once that design has been developed, it can be implemented in phases as funding permits. Departments are *strongly encouraged* to seek the assistance of professional network designers, rather than design the network themselves, before implementing a new networking infrastructure. The UGA Network Operations Center (NOC) is one source for professional design assistance. If a department insists on developing its own design, there are a number of concepts and issues that need to be understood before proceeding.

Building networks that adhere to structured wiring specifications typically have one central wiring closet called a main distribution frame (MDF) and one or more distributed wiring closets called intermediate distribution frames (IDFs). It is highly desirable to secure the MDF and IDFs behind locked doors, and they should be large enough to support all of the equipment with sufficient room to reach all devices in them. These facilities should also adhere to environmental specifications given in Section II.B. of the official Campus Wiring Specifications document.

Each IDF should be star wired back to the MDF via fiber optic cabling. Fiber cabling supports longer distances (2,000 meters for multi-mode fiber) than twisted pair copper wiring, and it is immune to electrical interference and grounding problems. It also has the potential for supporting high data transmission capacities (gigabits per second). Twelve strands of fiber cabling should be run from MDF to each IDF to provide for future growth and redundancy.

Networked devices, such as microcomputer workstations, connect to IDFs through star wired Cat 5 UTP. One Cat 5 cable should be installed for each networked device in a room location with at least one additional Cat 5 cable installed for growth and redundancy. It is best to locate an IDF in a central location on a floor, when possible, to limit the number of IDFs per floor. In addition, it would be ideal for the IDFs on each floor to be stacked on top of one another to minimize backbone cable paths. The relationship between MDF, IDFs and networked devices is depicted in the MDF/IDF Diagram.

The LAN components of an IDF minimally consist of one or more rack mounted hubs (either modular chassis or stackable, shared and/or switched) with each hub port connecting to a port on a rack mounted patch panel via a stranded wire Cat 5 UTP patch cable. Each port on the patch panel is connected to an RJ-45 wall plate in an office through a solid conductor horizontal Cat 5 UTP cable running through the building infrastructure. The networked device is connected to the wall plate via a stranded wire Cat 5 UTP station cable. These components are depicted schematically in the IDF Diagram.

The total cable length for Cat 5 UTP wiring is 100 meters (90 meters for horizontal cabling and 10 meters for both station and patch cables combined). As indicated above, fixed horizontal cables must use solid copper CAT 5 wire, whereas, patch cables must be stranded copper CAT 5 wire. When designing and installing Cat 5 wiring, it is important to stay away from sources of electrical interference, e.g., 12 inches from light ballasts and four feet from electrical devices such as high-voltage transformers, electric motors, microwave ovens and Xerox machines. Cable trays, which look like metal ladders, can be installed above ceilings to provide clearly defined paths for horizontal Cat 5 wiring, and can keep cables from sources of electrical interference. They also protect cables from damage by other personnel working in ceilings.

Rack mounted patch panels are ideal, direct termination points for Cat 5 wiring in the IDF. Although Cat 5 cabling can be terminated in 110-type punch down blocks, it is not recommended unless that type of termination block will be installed for both telephone and data services. Under no circumstances should 66-type punch down blocks be utilized since they can adversely affect data signals. One should also not plan to allow two signals (either LAN-LAN or LAN-voice) within the same four pair of a Cat 5 cable, since the signals may interfere with one another.

2.2 Installation Issues

Departments are *strongly encouraged* to hire qualified professionals to install and terminate cable. The UGA Electronics Design and Maintenance Shop (E-Shop) and Key Services through the Department of Administrative Services (DOAS) can install both Cat 5 and fiber optic cabling. The UGA NOC can oversee the installation process. If a department insists on installing their own Cat 5 cable (fiber optic cable requires special equipment and considerable experience to install), they are encouraged to attend professional cable installation training classes. They should also bear in mind the following installation issues.

When horizontal Cat 5 cabling is pulled, the maximum pulling tension is 25 lbs. Ivory soap and water can be used to pull cable through conduit, when utilized. Don t allow cables to kink and insure that the minimum bending radius is one inch throughout (if one wrapped a Cat 5 cable around a cylindrical object, the radius of the cylinder should mininimally be one inch). Cable ties should be loosely attached to avoid pinching the wires. Remember to stay clear of electrical interference, and use cable trays whenever feasible. One should also follow the installation guidelines given in Section II.C.1 of the official Campus Wiring Specifications document.

When terminating Cat 5 cables, make sure that the cable jacket stays on the cable until the end, and allow a maximum untwist of only one-half inch. Correct RJ-45 connectors should be utilized (stranded connectors for stranded wire, solid ones for solid wires), and the same pin configuration (or wire map) should be used throughout with correct color codes. At UGA the wire map standard is EIA/TIA 568B.

Patch cables should be installed in a neat and orderly fashion. Use cable management guides (brackets and D rings), and cut patch cables to length to avoid dangling, messy loops. If the patch panel serves different sections of the building, one can optionally use different colored patch cables for each section.

It is *vitally important* to accurately document the installation, whether professional cable installers or departmental staff are utilized. Before any cabling is installed, one should obtain accurate copies of building blueprints and document the end points (room and IDF) and path of all horizontal Cat 5 and fiber optic cable runs. Each port on the patch panel should have the same unique label as the wall plate port in a room. Both ends of the patch cable should also have an identical, unique label. In addition, one should maintain a database that minimally maps the patch panel/wall plate port label to a room location and includes the corresponding label for the patch cable as well as a unique hub port number (usually specified through management software).

Cable installations must comply with appropriate building codes. All penetrations through fire walls, ceilings and floors must be fire sealed. Many of the older buildings on campus contain asbestos, and installers should obtain training from Public Safety regarding asbestos precautions before drilling holes to potentially avoid installation delays. Plenum rated cable should always be utilized, but it is required when installed in air plenums. Riser rated cable should also be used where required. Furthermore, cables and hub components should be appropriately grounded. If there are any questions regarding building codes, one can contact Campus Planning or the UGA Fire Marshall in Public Safety. Questions regarding proper grounding techniques can be referred to the UGA NOC.

Before attaching networking equipment to the cable infrastructure, it is important that each terminated wire is checked with Cat 5 certification equipment (level II scanner/tester). The equipment should test and document:

  1. the wiring map or pinout to insure that the EIA/TIA 568B map is followed
  2. the signal attenuation through the cable (less than 24.0 dB)
  3. the cable length measurements
  4. near end crosstalk (NEXT) in both directions (greater than 27.1 dB)

3. Ethernet Hub Guidelines

Ethernet started as a shared networking media, i.e., all devices attached to the same physical network (backbone cable) and shared the 10 Mbps bandwidth among each other. Devices shared the bandwidth by detecting "collisions" (two or more devices trying to communicate at the same time), backing off, and attempting communications again at a later time. Repeaters, which are devices used to extend cable segments by "repeating" the electrical signals seen on its connected segments, also propagate collision information. The set of devices (network interface cards, cables, and repeaters) connected in this manner is sometimes referred to as a "collision domain". Shared Ethernet hubs are multi-port repeaters that connect Ethernet devices in a star-wired fashion to the same collision domain. Obviously, as the number of Ethernet devices within a collision domain increases, the amount of available bandwidth per device decreases.

Ethernet bridges are used to connect (and isolate) two or more collision domains. Switched Ethernet hubs are essentially per port bridges contained in a single box. A major problem with these types of communications devices is that they propagate broadcast and multicast frames onto all connected segments, which can be detrimental to the performance of networked devices on those segments. (Broadcast and multicast frames must be processed by a network device's CPU to determine whether the frames should be discarded or processed further.) The collection of network interface cards, cables, repeaters, bridges, and shared and switched Ethernet hubs is referred to as a "broadcast domain", i.e., all devices connected in this manner see the same broadcast and multicast frames.

Since broadcast and multicast frames can harm network performance, routers can be used to isolate these frames and their associated broadcast domains. Unlike bridges which make forwarding decisions based upon media access control (MAC) addresses, routers make forwarding decisions based upon higher-level network protocol (e.g., IP, IPX, AppleTalk) addresses. Working in conjunction with routers, virtual LANs (VLANs) involve relatively new methods of creating artificial broadcast domains by employing software on Ethernet switches to group devices connected to a set of ports in some logical fashion.

The goal of this section is to help decide when and how to connect networked devices to the same collision domain (shared hubs) or to the same broadcast domain (switched hubs and VLANs). When designing a LAN infrastructure one attempts to optimize performance, especially user perception of performance, while minimizing costs. If money is not a constraint then the highest performance technology, e.g. switched fast Ethernet through modular chassis hubs, is the answer. Sadly money usually is the major constraint with regard to what can and cannot be done. In this event a combination of shared and switched media (and possibly both modular chassis and stackable hubs) may give the best bang for the buck. (Shared and switched media are defined below.)

It should not be forgotten that the newest, fastest, most expensive Ethernet LAN equipment will not necessarily improve services located on other networks. When you design your LAN you should bear in mind that you can really only optimize performance for clients local to the LAN using services local to the same LAN. At the University of Georgia this typically means that clients and services are in the same building.

Finally, don't forget that you are not going to want to replace your LAN infrastructure in just a couple of years. Consider possible growth in the number of clients and local services on your LAN as well as the level of bandwidth needed by your local services a few years into the future.

3.1 LAN Models

This is an attempt to establish a "rule of thumb" guide to the LAN infrastructure needed to provide optimal connectivity based on four different models or classes of LAN likely to be found at UGA (or elsewhere for that matter). The intent being that knowing which model one's LAN most closely resembles will provide a first step toward what type of and how much LAN equipment is needed. The focus will be on Ethernet LAN equipment, the most commonly used medium at UGA and in most other networks.

Please note that the list of models is not believed to be exhaustive, nor are they completely discrete examples. Your LAN may resemble more than one model. In this event opting for the higher performance infrastructure is recommended.

Criteria:

Definitions: Models:

1) Non-local service(s)

This is the scenario in which all the client workstations access a service(s) non-local to the LAN. Such clients may well be able to be optimized using shared 10Mbps media depending upon both the bandwidth required by the services and the number of networked devices. In either case the bottleneck is likely to be non-local to the LAN and as such there may be little that can be done to optimize performance locally.

If your LAN currently most resembles this model it may be advisable to think hard on which model your LAN is most likely to resemble three years from now.

2) One or few local services used by all LAN clients

In this scenario all clients of the LAN access one or more (but not many) local services. In this case LAN performance can be optimized by providing high bandwidth connections, e.g. 100Mbps switched (and possibly duplexed) connections to the services and improved connections, e.g. switched 10Mbps connections, to the clients depending on the service provided and number of networked devices on the LAN.

3) Multiple local services used by respective subsets of clients

This situation can be referred to as a workgroup based LAN. Few or none of the clients need access to all local services. Rather they tend to be discrete groups of clients each using a different service.

For low bandwidth services this could be optimized by shared media workgroups and switching between the respective workgroups. Care will be needed in grouping on a physical/logical basis the clients with their appropriate services. Low bandwidth services may allow shared media within a workgroup to be filtered from other workgroups so that contention for bandwidth occurs mainly within each workgroup.

Medium bandwidth services may require switched connections to the services, but only the same shared media clusters for the clients. (The groups of clients should still be filtered from each other.)

High bandwidth services will require a LAN configuration consistent with those described in model 2.

4) Lattice-like clusters of high bandwidth services

This is a scenario in which all networked devices provide high bandwidth services used by all other devices. Optimization to each device in this scenario is imperative. High bandwidth connections, e.g. switched duplexed 100Mbps connections, should be provided to each device.

Note, that for low numbers of clients and local services the "stackable" class of network hub or switch is acceptable. For medium and high numbers "modular chassis" devices are a must.

Number of Shared Network Devices:

The number of devices connected to the same collision domain (shared Ethernet hubs) determine roughly how much bandwidth is available per device. As the number of devices within the collision domain increases, the bandwidth decreases and the response time increases. In order to provide estimates of the response times for typical network applications as a function of the number of shared devices, the Ethernet Response Time calculator can be utilized.

A more scientific method of characterizing bandwidth utilization in existing shared networks is to utilize network monitoring tools such as Network General Sniffers. These tools can assess peak and average bandwidth utilization and determine which devices are generating the most traffic on the LAN. This information can then be used to determine how to migrate from existing shared hubs to switched ones. The UGA NOC can assist departments in characterizing their existing LAN traffic.

3.2 Common Hub Specifications

When considering the purchase of a hub from a particular vendor, that vendor should have a successful corporate history as well as acceptance in the marketplace. The industry leaders with respect to hubs are (in alphabetical order) Bay Networks, Cabletron, Cisco, and 3COM. At UGA, Cabletron and Bay Networks are the two primary hub suppliers. Any selected hub should be non-proprietary, and the vendor should provide an upgrade path for their equipment.

Multi-port twisted pair hubs allows several point-to-point segments to be joined into one network in a star-like configuration. Each workstation can potentially communicate with any other workstation connected to the same hub. One end of the point-to-point link is attached to a port on the hub and the other is attached to a network interface card in a workstation. If the hub is also attached to a backbone (larger network), then all workstations at the end of the twisted pair segments can communicate with any device connected to the same backbone (e.g., on another hub).

Troubleshooting connectivity problems is enhanced by devices organized in star configurations in two ways:

The following is a description of the common specifications for both shared and switched Ethernet hubs.

3.3 Shared Ethernet Hub Specifications

Shared Ethernet hubs are devices which connect multiple network devices to the same physical network media (cable). Shared hubs function as "repeaters" because they take any incoming signal and repeat it out all ports. It is important to keep in mind that a repeater will only "clean up" and reshape signals crossing it; it cannot bridge or route network traffic because it operates solely at the physical layer (first logical layer of the OSI model).

The concept of shared access is related to the fact that all devices attached to the hub are contending for transmission of data onto a single network (i.e., a collision domain). This means that individual devices on a shared network will each only get a percentage of the available network bandwidth.

Shared hubs come in two varieties -- modular chassis and standalone (or stackable) workgroup hubs. See the section titled Modular Chassis vs. Stackable Hubs for a discussion of the issues surrounding these two implementation strategies.

Hub management is accomplished through software running remotely on a PC or workstation which can communicate with the hub module or stack using SNMP (Simple Network Management Protocol). Support for the RMON MIB (Remote Network Monitoring) should also be a feature of the hub's SNMP capabilities.

For a summary of shared Ethernet hub specs, see Summary of Ethernet Hub Specifications.

3.4 Switched Ethernet Hubs Specifications

Unlike a shared media hub in which devices connected to its ports must contend for available bandwidth, a switched hub provides the full bandwidth (typically 10 or 100 Mbps) to each of its ports. The following is a presentation of the issues associated with switched Ethernet implementations:

For a summary of switched Ethernet hub specs, see Summary of Ethernet Hub Specifications.

3.5 Maintenance Issues

Since the failure of any hub or hub component means that all devices attached to that hub or component will be unable to communicate, it is important to have a viable disaster recovery and maintenance plan in place, especially for server connections. One plan is to execute a hardware maintenance contract with the vendor who can supply a functioning hub component for the failed one. The contract should include software/firmware upgrades. The major downside to this type of plan is that there may be a unacceptable delay in delivering the replacement component in the event of a emergency.

An alternative plan is to maintain or have access to spare hub components on campus. The UGA NOC can provide hub components for most of the modular chassis Cabletron hubs (MMAC8, MMACPlus and SmartSwitch 6000) and the MicroMMAC stackable hub.

4. Bibliography

Trulove, James, LAN Wiring: An Illustrated Guide to Network Cabling, McGraw-Hill, 1997 (ISBN: 0-07-065302-X).

Charles Spurgeon's Ethernet Web Site (http://www.ethermanage.com/ethernet/ethernet.html). This site provides extensive information about Ethernet (IEEE 802.3) local area network (LAN) technology, including the original 10 Megabit per second (Mbps) system, the 100 Mbps Fast Ethernet system (802.3u), and the Gigabit Ethernet system (802.3z).

Tolly Group (http://www.tolly.com). This site contains testing and feature information regarding Ethernet switches. (Note: Requires user ID and password which can be created when initially viewing the Web pages.)