This document provides guidelines that departments at the University of Georgia (UGA) can use in the development of robust networking infrastructures within buildings. The purpose of these guidelines is to maximize the University s investment by maximizing the efficiency of the network and minimizing outages while minimizing the staff necessary to provide the facilities and services.
A building s networking infrastructure involves a number of hardware components including wiring, connectors, racks, network interface cards, client and server workstations, and communications devices such as repeaters, bridges, shared and switched hubs, and routers. It also includes software such as network card drivers, communications protocols, network operating systems and network application tools. This document currently focuses on wiring and hub guidelines, but will be expanded to include other networking components in the future.
Although both Ethernet and Token Ring networks are deployed on the UGA campus, the predominant networking technology is Ethernet (approximately 95%). The predominance of Ethernet at UGA is mirrored in the deployment of networking infrastructures worldwide, and networking vendors continue to develop faster and better Ethernet products. Therefore, this document will focus exclusively on guidelines for Ethernet networking, and it is strongly recommended that all new infrastructure on the UGA campus be based on Ethernet, not Token Ring, for the aforementioned reasons.
Although building networks have been implemented on campus using thick and thin coaxial Ethernet cabling technologies (10Base5 and 10Base2, respectively), the desirable implementation strategy is to deploy structured wiring using fiber optic cabling for building backbones and Category 5 unshielded twisted pair (or Cat 5 UTP) cabling to connect end devices. Wiring for new and renovated buildings must comply with official Campus Wiring Specifications, as published in the Administrative Policies and Procedures Manual of the University. In particular, these specifications require that for all new and renovated buildings, any proposed building wiring designs be approved prior to installation. The deployment of data wiring in existing buildings should also adhere to these specifications whenever possible.
The Electronics Industry Association (EIA) and Telecommunications Industry Association (TIA) have jointly developed specifications for structured wiring. The EIA/TIA 568 specification defines a generic wiring system for a multi-product, multi-vendor environment. EIA/TIA 569 is the building specification for telecommunications pathways and spaces. It defines the minimum requirements for ducts, closets and others spaces needed for data and telecommunications wiring. The EIA/TIA 607 is the specification for grounding and bonding of telecommunications. All network components and wiring racks must be properly grounded. EIA/TIA 606 is the administration standard for telecommunications infrastructure. This specification covers cable labeling, telecommunications records, required drawings, and a method of knowing who to contact for each part of the infrastructure. Each of these standards has been condensed into a small pamphlet, which most networking vendors can provide.
Structured wiring, which uses a star topology, has a number of advantages over thick and thin coaxial Ethernet infrastructures which utilize bus topologies. First, it is easier to add, move, or change connections by moving patch cables in a wiring closet. Second, troubleshooting is easier and less time consuming since one can quickly disconnect network devices in the wiring closet rather than having to go to each workstation (or get into the ceiling) to disconnect malfunctioning devices; some vendor s management software can isolate and eliminate network problems as they occur and notify the network administrator of the problem. Third, workstations do not have to be powered down before making a cable switch, as they do in a thick coaxial infrastructure. In addition, the cable failure of one networked device does not generally affect others. Finally, new connection capacity can be added by installing additional hub devices in the wiring closet.
Cat 5 UTP cabling is the industry standard for connecting a networked device to a wiring closet. It can support data rates up to 500 Mbps using sophisticated encoding methods and will likely support local area network (LAN) traffic for the next 10 to 15 years, even though the active network components will have to be replaced in a much shorter time span to provide increased bandwidth and services.
In order to minimize costs over the long term, it is highly desirable to develop a comprehensive building network design that takes into account the needs of all of the building occupants. Once that design has been developed, it can be implemented in phases as funding permits. Departments are *strongly encouraged* to seek the assistance of professional network designers, rather than design the network themselves, before implementing a new networking infrastructure. The UGA Network Operations Center (NOC) is one source for professional design assistance. If a department insists on developing its own design, there are a number of concepts and issues that need to be understood before proceeding.
Building networks that adhere to structured wiring specifications typically have one central wiring closet called a main distribution frame (MDF) and one or more distributed wiring closets called intermediate distribution frames (IDFs). It is highly desirable to secure the MDF and IDFs behind locked doors, and they should be large enough to support all of the equipment with sufficient room to reach all devices in them. These facilities should also adhere to environmental specifications given in Section II.B. of the official Campus Wiring Specifications document.
Each IDF should be star wired back to the MDF via fiber optic cabling. Fiber cabling supports longer distances (2,000 meters for multi-mode fiber) than twisted pair copper wiring, and it is immune to electrical interference and grounding problems. It also has the potential for supporting high data transmission capacities (gigabits per second). Twelve strands of fiber cabling should be run from MDF to each IDF to provide for future growth and redundancy.
Networked devices, such as microcomputer workstations, connect to IDFs through star wired Cat 5 UTP. One Cat 5 cable should be installed for each networked device in a room location with at least one additional Cat 5 cable installed for growth and redundancy. It is best to locate an IDF in a central location on a floor, when possible, to limit the number of IDFs per floor. In addition, it would be ideal for the IDFs on each floor to be stacked on top of one another to minimize backbone cable paths. The relationship between MDF, IDFs and networked devices is depicted in the MDF/IDF Diagram.
The LAN components of an IDF minimally consist of one or more rack mounted hubs (either modular chassis or stackable, shared and/or switched) with each hub port connecting to a port on a rack mounted patch panel via a stranded wire Cat 5 UTP patch cable. Each port on the patch panel is connected to an RJ-45 wall plate in an office through a solid conductor horizontal Cat 5 UTP cable running through the building infrastructure. The networked device is connected to the wall plate via a stranded wire Cat 5 UTP station cable. These components are depicted schematically in the IDF Diagram.
The total cable length for Cat 5 UTP wiring is 100 meters (90 meters for horizontal cabling and 10 meters for both station and patch cables combined). As indicated above, fixed horizontal cables must use solid copper CAT 5 wire, whereas, patch cables must be stranded copper CAT 5 wire. When designing and installing Cat 5 wiring, it is important to stay away from sources of electrical interference, e.g., 12 inches from light ballasts and four feet from electrical devices such as high-voltage transformers, electric motors, microwave ovens and Xerox machines. Cable trays, which look like metal ladders, can be installed above ceilings to provide clearly defined paths for horizontal Cat 5 wiring, and can keep cables from sources of electrical interference. They also protect cables from damage by other personnel working in ceilings.
Rack mounted patch panels are ideal, direct termination points for Cat 5 wiring in the IDF. Although Cat 5 cabling can be terminated in 110-type punch down blocks, it is not recommended unless that type of termination block will be installed for both telephone and data services. Under no circumstances should 66-type punch down blocks be utilized since they can adversely affect data signals. One should also not plan to allow two signals (either LAN-LAN or LAN-voice) within the same four pair of a Cat 5 cable, since the signals may interfere with one another.
Departments are *strongly encouraged* to hire qualified professionals to install and terminate cable. The UGA Electronics Design and Maintenance Shop (E-Shop) and Key Services through the Department of Administrative Services (DOAS) can install both Cat 5 and fiber optic cabling. The UGA NOC can oversee the installation process. If a department insists on installing their own Cat 5 cable (fiber optic cable requires special equipment and considerable experience to install), they are encouraged to attend professional cable installation training classes. They should also bear in mind the following installation issues.
When horizontal Cat 5 cabling is pulled, the maximum pulling tension is 25 lbs. Ivory soap and water can be used to pull cable through conduit, when utilized. Don t allow cables to kink and insure that the minimum bending radius is one inch throughout (if one wrapped a Cat 5 cable around a cylindrical object, the radius of the cylinder should mininimally be one inch). Cable ties should be loosely attached to avoid pinching the wires. Remember to stay clear of electrical interference, and use cable trays whenever feasible. One should also follow the installation guidelines given in Section II.C.1 of the official Campus Wiring Specifications document.
When terminating Cat 5 cables, make sure that the cable jacket stays on the cable until the end, and allow a maximum untwist of only one-half inch. Correct RJ-45 connectors should be utilized (stranded connectors for stranded wire, solid ones for solid wires), and the same pin configuration (or wire map) should be used throughout with correct color codes. At UGA the wire map standard is EIA/TIA 568B.
Patch cables should be installed in a neat and orderly fashion. Use cable management guides (brackets and D rings), and cut patch cables to length to avoid dangling, messy loops. If the patch panel serves different sections of the building, one can optionally use different colored patch cables for each section.
It is *vitally important* to accurately document the installation, whether professional cable installers or departmental staff are utilized. Before any cabling is installed, one should obtain accurate copies of building blueprints and document the end points (room and IDF) and path of all horizontal Cat 5 and fiber optic cable runs. Each port on the patch panel should have the same unique label as the wall plate port in a room. Both ends of the patch cable should also have an identical, unique label. In addition, one should maintain a database that minimally maps the patch panel/wall plate port label to a room location and includes the corresponding label for the patch cable as well as a unique hub port number (usually specified through management software).
Cable installations must comply with appropriate building codes. All penetrations through fire walls, ceilings and floors must be fire sealed. Many of the older buildings on campus contain asbestos, and installers should obtain training from Public Safety regarding asbestos precautions before drilling holes to potentially avoid installation delays. Plenum rated cable should always be utilized, but it is required when installed in air plenums. Riser rated cable should also be used where required. Furthermore, cables and hub components should be appropriately grounded. If there are any questions regarding building codes, one can contact Campus Planning or the UGA Fire Marshall in Public Safety. Questions regarding proper grounding techniques can be referred to the UGA NOC.
Before attaching networking equipment to the cable infrastructure, it is important that each terminated wire is checked with Cat 5 certification equipment (level II scanner/tester). The equipment should test and document:
Ethernet started as a shared networking media, i.e., all devices attached to the same physical network (backbone cable) and shared the 10 Mbps bandwidth among each other. Devices shared the bandwidth by detecting "collisions" (two or more devices trying to communicate at the same time), backing off, and attempting communications again at a later time. Repeaters, which are devices used to extend cable segments by "repeating" the electrical signals seen on its connected segments, also propagate collision information. The set of devices (network interface cards, cables, and repeaters) connected in this manner is sometimes referred to as a "collision domain". Shared Ethernet hubs are multi-port repeaters that connect Ethernet devices in a star-wired fashion to the same collision domain. Obviously, as the number of Ethernet devices within a collision domain increases, the amount of available bandwidth per device decreases.
Ethernet bridges are used to connect (and isolate) two or more collision domains. Switched Ethernet hubs are essentially per port bridges contained in a single box. A major problem with these types of communications devices is that they propagate broadcast and multicast frames onto all connected segments, which can be detrimental to the performance of networked devices on those segments. (Broadcast and multicast frames must be processed by a network device's CPU to determine whether the frames should be discarded or processed further.) The collection of network interface cards, cables, repeaters, bridges, and shared and switched Ethernet hubs is referred to as a "broadcast domain", i.e., all devices connected in this manner see the same broadcast and multicast frames.
Since broadcast and multicast frames can harm network performance, routers can be used to isolate these frames and their associated broadcast domains. Unlike bridges which make forwarding decisions based upon media access control (MAC) addresses, routers make forwarding decisions based upon higher-level network protocol (e.g., IP, IPX, AppleTalk) addresses. Working in conjunction with routers, virtual LANs (VLANs) involve relatively new methods of creating artificial broadcast domains by employing software on Ethernet switches to group devices connected to a set of ports in some logical fashion.
The goal of this section is to help decide when and how to connect networked devices to the same collision domain (shared hubs) or to the same broadcast domain (switched hubs and VLANs). When designing a LAN infrastructure one attempts to optimize performance, especially user perception of performance, while minimizing costs. If money is not a constraint then the highest performance technology, e.g. switched fast Ethernet through modular chassis hubs, is the answer. Sadly money usually is the major constraint with regard to what can and cannot be done. In this event a combination of shared and switched media (and possibly both modular chassis and stackable hubs) may give the best bang for the buck. (Shared and switched media are defined below.)
It should not be forgotten that the newest, fastest, most expensive Ethernet LAN equipment will not necessarily improve services located on other networks. When you design your LAN you should bear in mind that you can really only optimize performance for clients local to the LAN using services local to the same LAN. At the University of Georgia this typically means that clients and services are in the same building.
Finally, don't forget that you are not going to want to replace your
LAN infrastructure in just a couple of years. Consider possible
growth in the number of clients and local services on your LAN as
well as the level of bandwidth needed by your local services a few
years into the future.
This is an attempt to establish a "rule of thumb" guide to the LAN
infrastructure needed to provide optimal connectivity based on four
different models or classes of LAN likely to be found at UGA (or
elsewhere for that matter). The intent being that knowing which
model one's LAN most closely resembles will provide a first step
toward what type of and how much LAN equipment is needed. The focus
will be on Ethernet LAN equipment, the most commonly used medium at
UGA and in most other networks.
Please note that the list of models is not believed to be
exhaustive, nor are they completely discrete examples. Your LAN may
resemble more than one model. In this event opting for the higher
performance infrastructure is recommended.
Criteria:
1) Non-local service(s)
This is the scenario in which all the client workstations access a
service(s) non-local to the LAN. Such clients may well be able to
be optimized using shared 10Mbps media depending upon both the
bandwidth required by the services and the
number of networked devices. In either case
the bottleneck is likely to be non-local to the LAN and as such
there may be little that can be done to optimize performance
locally.
If your LAN currently most resembles this model it may be advisable
to think hard on which model your LAN is most likely to resemble
three years from now.
2) One or few local services used by all LAN
clients
In this scenario all clients of the LAN access one or more (but not
many) local services. In this case LAN performance can be optimized
by providing high bandwidth connections, e.g. 100Mbps switched (and
possibly duplexed) connections to the services and improved
connections, e.g. switched 10Mbps connections, to the clients
depending on the service provided and number of
networked devices on the LAN.
3) Multiple local services used by respective subsets of
clients
This situation can be referred to as a workgroup based LAN. Few or
none of the clients need access to all local services. Rather they
tend to be discrete groups of clients each using a different
service.
For low bandwidth services this could be optimized by shared media
workgroups and switching between the respective workgroups. Care
will be needed in grouping on a physical/logical basis the clients
with their appropriate services. Low bandwidth services may allow
shared media within a workgroup to be filtered from other workgroups
so that contention for bandwidth occurs mainly within each
workgroup.
Medium bandwidth services may require switched connections to the
services, but only the same shared media clusters for the clients.
(The groups of clients should still be filtered from each other.)
High bandwidth services will require a LAN configuration consistent
with those described in model 2.
4) Lattice-like clusters of high bandwidth services
This is a scenario in which all networked devices provide high
bandwidth services used by all other devices. Optimization to each
device in this scenario is imperative. High bandwidth connections,
e.g. switched duplexed 100Mbps connections, should be provided to
each device.
Note, that for low numbers of clients and local services the
"stackable" class of network hub or switch is acceptable. For
medium and high numbers "modular chassis" devices are a must.
Number of Shared Network
Devices:
The number of devices connected to the same collision domain (shared
Ethernet hubs) determine roughly how much bandwidth is available per
device. As the number of devices within the collision domain
increases, the bandwidth decreases and the response time increases.
In order to provide estimates of the response times for typical
network applications as a function of the number of shared devices,
the Ethernet Response Time calculator
can be utilized.
A more scientific method of characterizing bandwidth utilization in
existing shared networks is to utilize network monitoring tools such
as Network General Sniffers. These tools can assess peak and
average bandwidth utilization and determine which devices are
generating the most traffic on the LAN. This information can then
be used to determine how to migrate from existing shared hubs to
switched ones. The UGA NOC can assist departments in characterizing
their existing LAN traffic.
When considering the purchase of a hub from a particular vendor,
that vendor should have a successful corporate history as well as
acceptance in the marketplace. The industry leaders with respect to
hubs are (in alphabetical order) Bay Networks, Cabletron, Cisco, and
3COM. At UGA, Cabletron and Bay Networks are the two primary hub
suppliers. Any selected hub should be non-proprietary, and the
vendor should provide an upgrade path for their equipment.
Multi-port twisted pair hubs allows several point-to-point segments
to be joined into one network in a star-like configuration. Each
workstation can potentially communicate with any other workstation
connected to the same hub. One end of the point-to-point link is
attached to a port on the hub and the other is attached to a network
interface card in a workstation. If the hub is also attached to a
backbone (larger network), then all workstations at the end of the
twisted pair segments can communicate with any device connected to
the same backbone (e.g., on another hub).
Troubleshooting connectivity problems is enhanced by devices
organized in star configurations in two ways:
Standalone hubs are typically multiport 10BaseT
devices with anywhere from 4 to 24 ports. They should have
interfaces (cross-connect or proprietary SCSI) for adding
or stacking additional hubs in a cascading fashion. These
hubs are typically less expensive than modular chassis
hubs, historically have little intelligent circuitry (i.e.,
are not manageable) and have less fault-tolerance (e.g., no
redundant power supplies and no hot swappable components)
than modular chassis hubs. However, some vendors now sell
stackable hubs that contain intelligent circuitry in the
first hub in a stack which allows the entire stack of hubs
to be managed.
Stackable or chassis hubs should also have enough
management intelligence built-in so that a single node
malfunction (e.g., excessive collisions, malformed frames,
etc.) will result in auto-partitioning (turning off) of
that node's hub port. This will effectively limit the
impact of any single malfunctioning node disrupting the
entire network. Setting up port thresholds to activate
auto-partitioning requires proprietary MIBs (also called
enterprise or private MIBs) which can best be manipulated
by the vendor's SNMP management packages.
Some form of out-of-band management (typically modems
connected through console ports) is mandatory. Out-of-band
mangement that supports dialup TCP/IP (PPP) connections are
desirable but not mandatory.
The following constitute the nine groups (or classes) of
monitoring information defined in the RMON V1
specification.
The RMON V1 groups listed above only deal with information
at the physical and data link network levels. A new,
complementary network monitoring specification, RMON V2,
specifies information at network and application levels and
includes the following monitoring groups. In time,
switched and shared hubs should provide support for RMON V2
groups.
Since the new campus backbone will be based upon
asynchronous transfer mode (ATM) technology, it is strongly
recommended that one uplink option be ATM (minimally a 155
Mbps OC-3 connection). Fast Ethernet or ATM uplinks should
be considered for connecting hubs to building backbones.
(Note: Because shared Ethernet hubs have a
fixed bandwidth, high-speed uplinks will do little to
improve performance through building or campus backbones.)
3.1 LAN Models
Definitions:
Models:
3.2 Common Hub Specifications
The following is a description of the common specifications for both
shared and switched Ethernet hubs.
10BaseT, 100BaseTX (RJ-45)
10BaseFL, 100BaseFX, (Fiber Optic)
FDDI, ATM
10Base5 (AUI)
10Base2 (BNC)
3.3 Shared Ethernet Hub Specifications
Shared Ethernet hubs are devices which connect multiple network devices to the same physical network media (cable). Shared hubs function as "repeaters" because they take any incoming signal and repeat it out all ports. It is important to keep in mind that a repeater will only "clean up" and reshape signals crossing it; it cannot bridge or route network traffic because it operates solely at the physical layer (first logical layer of the OSI model).
The concept of shared access is related to the fact that all devices attached to the hub are contending for transmission of data onto a single network (i.e., a collision domain). This means that individual devices on a shared network will each only get a percentage of the available network bandwidth.
Shared hubs come in two varieties -- modular chassis and standalone (or stackable) workgroup hubs. See the section titled Modular Chassis vs. Stackable Hubs for a discussion of the issues surrounding these two implementation strategies.
Hub management is accomplished through software running remotely on a PC or workstation which can communicate with the hub module or stack using SNMP (Simple Network Management Protocol). Support for the RMON MIB (Remote Network Monitoring) should also be a feature of the hub's SNMP capabilities.
For a summary of shared Ethernet hub specs, see
Summary of Ethernet Hub
Specifications.
3.4 Switched Ethernet Hubs Specifications
Unlike a shared media hub in which devices connected to its ports must contend for available bandwidth, a switched hub provides the full bandwidth (typically 10 or 100 Mbps) to each of its ports. The following is a presentation of the issues associated with switched Ethernet implementations:
Some vendors support VLANs on a per port basis. Others
allow VLANs to be created based upon the media access
control (MAC) addresses of the network cards. More
sophisticated VLAN methodologies include grouping by
communications protocol type (e.g., IP, IPX, AppleTalk) and
possibly subtype (e.g., IP subnet addresses).
Communications between VLANs is usually accomplished via
routers, although some vendors have proprietary methods of
passing frames between VLANs on their switches without the
use of routers. It is important to note
that there is currently no standard for
VLANs and therefore no VLAN interoperability between switch
vendors can be guaranteed. Although it is
advisable to purchase a switch that minimally supports per
port VLANs, no VLAN guidelines are available at this
time.
For a summary of switched Ethernet hub specs, see
Summary of Ethernet Hub
Specifications.
3.5 Maintenance Issues
Since the failure of any hub or hub component means that all devices attached to that hub or component will be unable to communicate, it is important to have a viable disaster recovery and maintenance plan in place, especially for server connections. One plan is to execute a hardware maintenance contract with the vendor who can supply a functioning hub component for the failed one. The contract should include software/firmware upgrades. The major downside to this type of plan is that there may be a unacceptable delay in delivering the replacement component in the event of a emergency.
An alternative plan is to maintain or have access to spare hub components on campus. The UGA NOC can provide hub components for most of the modular chassis Cabletron hubs (MMAC8, MMACPlus and SmartSwitch 6000) and the MicroMMAC stackable hub.
Trulove, James, LAN Wiring: An Illustrated Guide to Network Cabling, McGraw-Hill, 1997 (ISBN: 0-07-065302-X).
Charles Spurgeon's Ethernet Web Site (http://www.ethermanage.com/ethernet/ethernet.html). This site provides extensive information about Ethernet (IEEE 802.3) local area network (LAN) technology, including the original 10 Megabit per second (Mbps) system, the 100 Mbps Fast Ethernet system (802.3u), and the Gigabit Ethernet system (802.3z).
Tolly Group (http://www.tolly.com). This site contains testing and feature information regarding Ethernet switches. (Note: Requires user ID and password which can be created when initially viewing the Web pages.)
