KRONE: 800-775-KRONE www.kroneamericas.com www.truenet-system.com No part of this document may be reproduced without permission ©2001 KRONE, Inc. Catching Up With TrueNet TM of the KRONE ® TrueNet TM structured cabling system, much of the cabling industry has been involved in a game of follow-the-leader. Numerous papers have been published, alternative theories of measuring throughput performance have been advanced, and generally, the rest of the industry has supported the notion that measuring active performance of cabling systems is the wave of the future. However, KRONE must repectfully admonish our industry peers for only getting the story half-correct. We understand that this can happen when knowledge is stretched into unfamiliar territories, and this paper represents KRONEs contribution toward setting the record straight. We will discuss the various errors and omissions made in industry research attempts, and demonstrate the correct methods of evaluating throughput performance. The goal is to help the reader understand the real value of throughput performance to the overall operation of the LAN. In this paper, we will reference two companion papers, Network Troubleshooting Using TrueNet, and The Effect of Errors on TCP Application Performance. These documents serve as additional background material for the theoretical impact of errors, and field experiences with poorly performing LANs. The first incorrect assumption that has been made is that there is no need for post-installation active testing on LANs. KRONE provides this service with every 200-plus node installation of TrueNet. This subject is dealt with extensively in the Network Troubleshooting paper, so let us merely summarize here. The two key benefits of post-installation active testing are: One: Prove the KRONE-certified cable installer did a quality job, and ensure all the cabling components work properly when installed as a complete system. Two: Check the integrity of the active devices on the end-users network. SINCE THE 99 LAUNCH You will notice that only one of these benefits has anything to do with the cabling itself. The assertion that quality checking components in the factory means that the network will run well ignores the fact that a network is made up of thousands of discrete compo- nents that all need to be installed properly and work together seamlessly. This leads us to our second incorrect assumptionthe one that really sinks most of the other papers weve seen out therethe assumption that all active ports are created equal. Laboratory and field research (shown in Network Troubleshooting) by KRONE shows without question that the individual ports on active devices perform differently, both as senders and receivers. Think all the ports on that 24 port switch perform exactly the same? Think again. If you have the exact same make and model of switches in your network, do they all have identical transmission characteristics? Again, no. Perhaps the most telling of all: is there any relationship between what you paid per port and how well it performs? Unfortunately, not really. Throughput experimentation that does not take into account the performance characteristics of individual transmit/receive ports can easily lead to an incorrect conclusion. Various experimental models weve seen have failed to include key pieces of relevant data: for example, were each of the sample channels tested using the exact same ports on the active devices? Were the experiments repeated using different ports? What were the characteristics of the senders and receivers? We can only assume that the omission of these critical points in the papers that we have read means that these facts arent understood, or arent considered relevant. Port performance is very relevant, as we will show. Other troubling examples seem to indicate that the experimenters dont understand how the active systems that they are testing even work. One demonstration in particular supposedly shows degradation of streaming video over poor cabling. The results claim that one lost packet caused one frame to drop, resulting in a jerky picture. Nice try, except the 40 second, 56 MB file used KRONE: 800-775-KRONE www.kroneamericas.com www.truenet-system.com No part of this document may be reproduced without permission ©2001 KRONE, Inc. simply connect- ing a 90 meter basic link or channel directly to a Smartbits at both ends to certify zero bit errors is extremely mis- leading. in the experiment at 15fps would mean that if one packet = one frame, then each packet would have to be 93,000 bytes long. Not likely, considering the Ethernet 1518 byte packet length limit. Other experiments test technologies that are irrelevant to the majority of LAN administrators, such as serial digital video, which was intended for broadcast television. This would be relevant if typical MIS departments also had a television studio in the office, but obviously this is not the case. The mathematical approach has also been used, trying to equate the probability of bit errors to such electrical parameters as near end crosstalk (NEXT) noise, insertion loss deviation, and signal to noise ratio. While this is theoretically interesting, it does not address the dynamics of a real world network. In an operating network, variables such as transmitters, receivers, external noise, cabling components and installation practices combine to produce a complex and dynamic system. Attempting to predict the performance of a complex system like a LAN by conducting mathematical analysis of just one element ignores the fact that network performance is more than just the sum of its parts. Finally, recent experimentation with a device called SmartBits TM manufactured by Spirent Communications has been widely reported in the trade literature. In fact, KRONE was the first to identify this product as a possible tool to evaluate different cabling systems. Originally, the product was designed to test switch and hub performance by bombarding ports with simulated network traffic, but only over a short distancein other words, connected by a patch cord only. It is this ability to generate traffic and count the packets as they arrive at the other end that seemed ideal to test whether or not cabling systems by themselves contrib- uted to packet loss. However, upon experimentation, we discovered that the signal characteristics in a SmartBits NIC is abnormally good, and as such is not representative of a real-word device such as an off-the shelf NIC or a switch port. Papers now circulating in the industry claim third party throughput verification of cabling systems using SmartBits. These experiments were conducted by simply connecting a 90 meter basic link or channel (with two, three or four connections) directly to a SmartBits at both ends to certify zero errors (10 -10 ). This conclusion is extremely misleading, because the methodology fails to take into account the abnormally good SmartBits NICs as mentioned earlier. What the methodology didnt take into account is that you can still get zero errors with a 140 meter channel40% longer than the standards allow! In fact, KRONE repeated the experiment shown in the third party verification with a 140 meter, four connector channel that had the cable torn apart in the middle, kinked, mangled,crushed, twisted, and with some of the insulation removed. We still got zero errors with the SmartBits. Clearly, this is due to unusually high signal strength and unusually capable receivers on the SmartBits NICs. A SmartBits NIC costs thousands of dollarsno wonder theyre so good. However, how many network managers ever paid that much for a NIC in a LAN? So, its no wonder that the third party verification of a reasonably well made 100 meter cabling channel showed no errors. Directly connecting a SmartBits will yield error-free results over almost any channel. How its Really Done Is the use of SmartBits incorrect? No, but the direct- connection methodology is flawed. First, one must normalize the strength of the sending card, and then collect the signal at the opposite end through something approximating a normal receiver, instead of using the super-receiver in the SmartBits. First, to normalize the SmartBits signal, the simulated traffic should be sent through a device that actually regenerates the signal, and sends it out again using one of the transmitters (ports) on the device. This takes the abnormally strong SmartBits signal and re- creates it along more normal parameters. You can do this either with a hub or a switch, by connecting it to the SmartBits NIC with a short patch cord. Using a store-and-forward device such as a switch also ensures that if SmartBits or the short patch cord corrupt any packets, they are discarded before being sent out over the cabling system under test. An A channel with this mess in it, and still ZERO ERRORS with SmartBits! KRONE: 800-775-KRONE www.kroneamericas.com www.truenet-system.com No part of this document may be reproduced without permission ©2001 KRONE, Inc. Is the use of SmartBits incorrect? No, but the direct- connection methdology is flawed. incoming packet from SmartBits that doesnt pass the switchs CRC check would be discarded before being sent out over the cabling system; therefore, only packets corrupted by the switchs transmitter or the cabling system itself would be counted at the far end of the channel. If you used a hub for this purpose, since a hub just repeats a signal and doesnt read the CRC, it would pass along errors between the SmartBits and the hub, so it really doesnt isolate the channel under test. Therefore, a switch is a better choice. Second, on the receiving end, you need to use some- thing approximating a normal NIC. There are various NICs out there that have special drivers, making them capable of counting the packets received, and reporting errors. We used a Digitech ® LAN900 TM card and software to do this, although there are other products on the market also capable of this same thing. One also has to decide whether to use a PCI (equivalent to a desktop PC NIC) or PCMCIA (equivalent to a laptop NIC) version of the receiving card. Upon evaluation of various options, we chose the PCMCIA version, because these proved to be more likely to count an error when receiving a packet with marginal signal characteristics. A better NIC might be able to equalize partially corrupted signals and make sense out of them; so it is best to use a NIC with marginal receiving properties to represent real-world worst case. In addition, notebook computers that use PCMCIA cards have become commonplace in the LAN. Throughput Test Methodology: Throughput testing of cabling channels is best accom- plished with the three devices mentioned previously: 1. A SmartBits SB200 chassis with MIL-7710 10/100 Base-TX cards as the packet generator. 2. A Cisco Catalyst 2900 10/100 Base-TX switch to regenerate and normalize the signal. 3. A typical 10/100 NIC with a special driver to report errors (we used a Digitech LAN900). For these experiments, we tested Fast Ethernet 100 Base-TX performance, as this is the most widely deployed technology in todays LANs. We tested two different cabling channels using this set-up, a Cat 5e KRONE TrueNet C5eT channel, and a channel of off-the shelf Cat 5e components from recognized manufactur- ers that approximates a mix and match cabling solution. The channels will be described in more detail later. It is important to first establish if there is any error rate inherent to the devices themselves, prior to testing channel throughput. To do this, we connected the SmartBits card directly to the Cisco switch with a short patch cord, and then out again directly to the LAN900 card with another short patch cord. This will show us if any of the port combinations are grossly better or worse than others. Upon testing numerous port pairings, it was found that the error rate was zero, in all cases. A screen shot of the LAN900 control window is shown below. The detail indicates that 6,482,321 frames were sent with no (frame) errors. In all the experiments, the maximum Ethernet frame size of 1518 bytes was used. There did not appear to be any inherent problems with the ports used on the switch, at least when signal strength is optimum over a short distance. Now, it is important to discern the subtle differences between the ports, so that we can see a real-world best and worst case scenario from this particular switch. Using the two cabling channels evaluated in the experiments, we tested throughput performance on the switch using various port combinations to see which combinations tend to perform better and which ones are worse. We finally settled on ports 10 and 16 as the best case and 1 and 2 as the worst case for this switch. Cabling Channel Test Bed Finally, we get to channel comparisons. For both the KRONE channel, and the mix and match channel, we used a 100 meter four-connection Category 5e channel, representing the greatest number of connections and the greatest physical distance allowed by the TIA/EIA standards. Each channel consisted of a patch panel, a block representing a cross connect equipment field, a block representing a consolidation point, a jack repre- senting the workstation outlet, Category 5e UTP cable for connection, and two seven-foot patch cords. The first channel that we tested was a KRONE TrueNet C5eT certified channel. LAN900 Control Screen KRONE: 800-775-KRONE www.kroneamericas.com www.truenet-system.com No part of this document may be reproduced without permission ©2001 KRONE, Inc. Testing the KRONE TrueNet Channel As the detail from the LAN900 screen shot on the left shows, for the best case port combination, out of 4,089,553 frames, there was only one error detected. This corresponds to a bit error rate of 2*10 -11 , by the following math (this will serve as the basis for all subsequent examples): We first need to assume that the one frame error was the result of just one bit error (this carries forward to all examples). In reality, its possible that there was more than one bit error, but the probability of this is extremely small and the net effect of one bit error or two bit errors (or ten, for that matter), in a frame is exactly the samethat one frame is thrown out. Therefore: 4,089,553 frames * 1518 bytes per frame * 8 bits/byte = 49,663,531,632 bits. 1 bit error divided by 49,663,531,632 bits = 2*10 -11 or .000000002% KRONE Channel with Best Case Ports on the Switch In the second test, we tested the worst case port combination and found that the error rate increased to six errors in 3,122,967 frames, or an error rate of 1.6*10 -10 (.000000016%). KRONE Channel with Worst Case Ports on the Switch Since this is the exact same channel we tested in the best case port combination, we can attribute the rise in errors to the subtly different performance of these ports (perhaps also in combination with the receiving characteristics of the LAN900 card). Testing the Mix and Match Channel The second channel we used was laid out exactly the same as the first, and is comprised of off the shelf cabling components, all Category 5e compliant from recognized manufacturers. This would approximate a mix and match solution, often chosen by combining various manufacturers productssince few manufacturers besides KRONE actually make all the pieces needed to create a channel. Again, the best and worst case ports on the switch are used in the measure- ment. As the screen shots show, the error rate increases dramati- cally. For the best case channel, the error rate is 1873 bit errors out of 1,928,463 frames, or 8*10 -8 (.000006%) . The worst case channel has a staggering 5291 errors after just 183,405 frames, a bit error rate of 2.4* 10 -6 (.00024%). (Note the difference between the error rate in the screen shots, and the BER rate calculated. Remem- ber the screen shots display the frame error rate, not the Bit Error Rate.) Effect of Alien Crosstalk Finally, to demonstrate the dramatic effect that outside noise can have on a cabling system, we devised a test to simulate very strong alien crosstalk. Alien Crosstalk is the noise from adjacent transmitting cables in a bundle that can be heard through the sheath on the cable being tested. The test bed was modified to include an identical channel directly next to the channel being tested. Mix and Match Channel with Worst Case Ports on the Switch Mix and Match Channel with Best Case Ports on the Switch KRONE: 800-775-KRONE www.kroneamericas.com www.truenet-system.com No part of this document may be reproduced without permission ©2001 KRONE, Inc. The second channel is energized by the SmartBits in exactly the same manner as the first; in order to create noise that replicates real traffic. This represents an extreme worst-case alien crosstalk scenario, since in reality its unlikely that a neighboring cable would be constantly running at 100% utilization, and directly adjacent for the entire run. However, this is a more realistic simulation of outside noise than other attempts we have seen, notably an experiment where the two unused pairs under the same sheath were energized to simulate crosstalk. In reality, that situation would not occur. In the extreme crosstalk scenario, it was observed that even the best case channels are affected. The TrueNet best case channel with the external noise records 35 bit errors in 2,793,835 packets, or a BER of 1*10 -9 (.0000001%), while the off the shelf best case channel clocks 5640 errors in 1,331,291 packets, or a BER of 3.5*10 -7 (.00003%). We would expect to see an increase in errors as noise in the system rises, as the cables used are unshielded (UTP), and hence not completely immune to outside noise. The Final Straw Just to prove that these experiments were not shaded to favor the TrueNet solution, we offer one of the most troubling pieces of evidence of all. Most end-users insist on hand-held test results from their installation contrac- tors for new installations. We demonstrated that the real-world performance (bit errors), of these channels are dramatically different. Now look at the actual hand-held test result screens. Which is the TrueNet and which is the one with the errors? Actually, the TrueNet channel is the one with the lower reading. So how is an end-user to know if the Category 5e compliant system they purchased has errors? The only way to tell the difference between meeting the standards and delivering optimum performance is to perform active testing on the operating network. KRONE has recognized this situation, and developed the TrueNet cabling system and active testing methodology to ensure that IT managers get a true picture of what they paid for. KRONE remains the only structured cabling provider in the world that performs on-site active testing of the installed network. The Effect of Errors on your Applications Clearly, there is a significant difference between the error rates of the two channels tested, but at first glance, even an error rate of .000024% seems tiny. Therefore, lets look at the previous examples in another way. The LAN900 card indicates an average frame per second rate (FPS) for each of the experi- ments. Lets look at Table 1, which shows the error performance of the best and worst case channels as a function of time. Now the effect of all those errors seems much more tangiblebut just what would happen to your applications if you had nodes racking up 148 errors per second? Tested Channel FPS* # Frames Test Duration Errors Result (seconds) KRONE w/ best case ports 5736 4,089,553 712 1 One error per 11.9 minutes KRONE w/ worst case ports 5642 3,122,967 554 6 One error per 1.5 minutes Mix&Match w/ best case ports 4061 1,929,463 475 1873 4 errors per second Mix&Match w/ worst case ports 5147 183,405 36 5291 148 errors per second Table 1: Error Performance Over Time *Average FPS reported; variation in FPS is a result of the switch. KRONE: 800-775-KRONE www.kroneamericas.com www.truenet-system.com No part of this document may be reproduced without permission ©2001 KRONE, Inc. KRONE, Inc. North American Headquarters 6950 South Tucson Way Englewood, CO 80112-3922 Telephone: (303) 790.2619 Toll-Free: (800) 775.KRONE Facsimile: (303) 790.2117 info.usa@kroneamericas.com www.kroneamericas.com www.truenet-system.com The time impact of errors on TCP is espe- cially important, because IS managers can buy more band- width to deal with efficiency loss, but they can never buy more time to speed up transaction processing. The effect of errors on application performance is the ultimate question at hand. The KRONE white paper, Effects of Errors on TCP Performance provides some answers. To summarize the main conclusions of this very in-depth paper, we learn the following: · The majority of current applications rely on TCP (embedded in the Ethernet frame) as the main information delivery protocol. This includes FTP, virtually all web-related applications, and virtually all transaction-oriented systems (such as databases). · Errors at the Ethernet level effect the proper functioning of TCP. · In a perfectly functioning node, Ethernet efficiency is mathematically limited to 92.9%, due to TCP and Ethernet overheadmeaning that only 92.9% of the transmitted information is the actual data packetthe rest is setup, addressing and other information. · The paper illustrates various examples of efficiency degradation from bit errors, including a mathemati- cal example where just two bit errors over a short period of time reduces Ethernet efficiency to 63.5%. · Even more dramatic than the mathematical calculation of efficiency degradation is the time impact of a few errors. In the same mathematical examples, the projected time to complete a transaction actually doubles, due to the logical construction of TCP, and how it deals with errors. · The time impact of errors on TCP is especially important, because while IS managers could buy more bandwidth to deal with efficiency loss, they cant buy more time to speed up the transactions that are bogged down trying to resolve errors. Now consider the impact that a 2.4*10 -6 BER (148 errors per second, from the previous example) can have on a database transaction, if just two bit errors could double the time necessary to complete an operation. Conclusion It can be very difficult to make sense of all the claims in the marketplace regarding throughput performance of cabling systems. This paper was designed to demon- strate that this very complex issue has been embraced by the industry, but often poorly executed in subse- quent counter-claims to the KRONE TrueNet message. Well meaning experimentation without a proper understanding of the underlying principles of networks and TCP have often led others in the industry to declare that throughput is important, without understanding why. Many of the claims have been made based on experiments that involve forms of video that are very unlikely to appear in a data center. The underlying implication is that if these systems carry video, they should easily carry data efficiently. However, this conclusion is not necessarily valid. To gain a real sense of how well a system will carry data, the experiments need to focus on the technologies that are widely deployed in the field. Hence, our focus was on Fast Ethernet and transaction oriented processing. This paper demonstrated that the SmartBits testing methodology that has been employed in the industry requires further steps to make it a valid tool for throughput measurement. We also demonstrated that not all Category 5e is what it seems to be, especially as reported by hand-held field testers. The conclusion reached is that to ensure proper performance in a complex installed system like a LAN, on-site active testing must be done to tell the difference between merely meeting the standards, and delivering error-free performance. Finally, by reference to another KRONE paper on TCP application performance, we concluded that errors in networks are a very real problem for applications, particularly those that are transaction oriented. Such errors manifest themselves in slower application performance and additional bandwidth requirements. However, since errors cause a demand for more transaction time, these issues cannot automatically be fixed merely by throwing bandwidth at the problem. Root cause elimination of errors is a much more effective strategy for optimal LAN performance. About The Authors: Tim Takala is the Technical and Laboratory Manager for KRONE, Inc., having recently joined the US team from KRONE Australia. He helped develop and implement the technical methodologies behind the TrueNet warranty program. He also designed and supervised the testing methodolgies employed in this paper. Mr. Takala holds an Industrial Engineering degree from the Sydney Institute of Technology. Bill Fetter is the TrueNet Global Coordinator for KRONE, Inc. He has been instrumental in the form and formation of the TrueNet program since its inception. Mr. Fetter holds an MBA from Indiana University. KRONE, TrueNet and C5eT are trademarks of KRONE All other trademarks are property of their respective owners. . KRONE: 80 0-7 75 -KRONE www.kroneamericas.com www .truenet- system.com No part of this document may be reproduced without permission 2001 KRONE, Inc. Catching. switch. KRONE: 80 0-7 75 -KRONE www.kroneamericas.com www .truenet- system.com No part of this document may be reproduced without permission 2001 KRONE, Inc. KRONE,