Internet Delay Measurements using Test Traffic Installing and hosting a Test Box Henk Uijterwaal (henk@ripe.net) RIPE NCC March 27, 1998 Document: RIPE-168.ps Version 2.0 Abstract: In RIPE-158, we presented a proposal to do Internet performance measurements using test traffic generated by test-boxes. This document is intended for ISP's interested in hosting a test-box. It describes the requirements for hosting a test-box at an ISP. It also describes the installation and day-to-day management of such a test-box. After reading this document, one should be able to judge if one can successfully host a test-box at a site. This document is not a formal request asking which sites are interested in hosting a test-box. 1 Introduction In a previous document [1], we described a proposal to do Internet performance measurements using test traffic. The test traffic used in this project is generated by test-boxes installed at the ISP's participating in this project. This document lists the requirements that have to be met before one can host a test-box. It also gives details about installation and day-to-day operation of the test-box. This document will be updated as we gain more experience with the installation and operation of the test-boxes. The latest version will always be available from our website (http://www.ripe.net/test-traffic/ . This document is intended to solicit input from interested parties. It should provide all information that one might have to know before installing a test-box. If you think that our requirements are unreasonable or impractical, do not hesitate to contact us. A formal request asking for sites interested in hosting a test-box will be sent around in the spring of 1998. 2 Requirements for hosting a Test Box This section discusses the requirements for hosting a test-box. Please read this section and make sure that you can meet all requirements, before applying to host a test-box. An application sheet can be found online (http://www.ripe.net/test-traffic/host_testbox.html) . If you cannot meet all requirements but still want to host a test-box, or want to discuss your specific situation before applying to host a test-box, do not hesitate to contact us. 2.1 Rack space The RIPE-NCC test-boxes consists of a 19" wide crate. The crate is 4 u (approximately 20 cm) high. It is recommended to mount the crate in a standard 19" rack in a computer room or the like, though the crate can be located anywhere where one would consider installing a networked PC. However, one has to keep in mind that the test-boxes have to be connected to an external antenna, see section 2.2. The operating system of the machine is FreeBSD. The security issues related to the installation of a test-box will be discussed in a separate document [2]. The test-box has to be connected to a 110 or 220 V power supply and the local network using 10base-T Ethernet. We will supply a power cable with a plug for a standard European, grounded, 220 V socket. In some places (the UK, for example) you may have to replace this plug with one that fits into your sockets. 2.2 Antenna Figure 1: Placing the antenna on a building. The best choice is somewhere in the dark grey area, the second choice is somewhere in the light grey area. The white area is a third choice. Figure 2: Mounting options for the GPS antenna. We will support the magnetic mount and direct-mount options (first and third option from the left). The test-boxes have to be connected to a GPS antenna. The antenna has to be mounted outside, with a clear view of of about half the sky (see figure 1, 1600, elevation 500). If there are local obstructions, try to find a location such that the antenna has a clear view to the south. Our experience with placing antennas so-far shows that an antenna sitting on the inside of a window will often work but that putting the antenna just outside the window is generally a major improvement on the reception conditions. The antenna will not work inside buildings made of reinforced concrete or near window-frames made of metal. High buildings in the neighborhood of the antenna may have a negative impact on the reception conditions. Finally, our experience shows that the reception conditions often improve by moving the antenna around by a couple of meters. Therefore, if you have to pull cables in advance, please make them a bit longer than absolutely necessary. We will support two mounting options (see figure 2): * A direct mount antenna. (default choice). * A magnetic mount antenna. A post-mounting bracket can be ordered. Please indicate on the application sheet if you need such a bracket. Figure 2 shows two other mounting options. However, support from the manufacturer (Motorola) for these option is or will be discontinued soon. We will therefore not offer these options for our test-boxes. Motorola will replace its line of antennas with new models soon. We have already bought the antennas for the first series of test-boxes so their antennas will be identical to the ones on the drawing, for next series of test-boxes, the antennas will be slightly smaller, but the mounting options will be the same. The antenna has to be mounted (roughly) horizontally. The maximum cable length to the antenna is about 50 m. For calibration purposes, the length of the antenna cable has to be known. Therefore, before installing any antenna cables, measure their length. 2.3 Network topology As discussed in a previous document [1], the test-boxes should connected hops away from a border router, or, if that is not feasible, as close to the border router as possible. On the information sheet, please provide a short description or drawing of your local network situation. In the same document, we discussed also discussed the case where an ISP has more border routers. If this applies to you, please provide a list of possible locations. We will then discuss the preferred location of the test-box with you, taking into account the location and connectivity to test-boxes at other ISP's. 2.4 Local Contact Although we plan to control the test-boxes from a central point with no operators or service required at the local sites, we do expect that each site that hosts a test-box appoints a local contact. This contact should take care of things that cannot be done remotely, such as rebooting the machine or copy information from the console in case of network or hardware problems. The local contact, obviously, has to be reachable by phone or email. In an ideal situation, the contact would be reachable 24 hours a day, 7 days a week (``24/7'') and could take care of our email's and phone-calls immediately. This could be the case at, for example, a place with operators around on a 24/7 basis. However, we do not think that there are many places where this ideal situation will exist. The minimum requirement for the local contact would be that he is reachable during office hours (9am-5pm, Monday to Friday, except for public holidays), is able to respond to our requests within a day and can appoint a replacement during holidays and the like. In the latter case, if a test-box causes trouble outside the hours where the local contact is reachable, it will removed from the measurement network until the problems can be solved. There several possible intermediate steps between the minimum requirements and the ideal situation possible. In general, the shorter the time between a request from our side and an action from the local contact, will increase fraction of time that the measurement network is active (``duty-cycle''). The higher the duty-cycle, the more that can be derived from the data. 2.5 Console The test-boxes do not have and, for normal operation, do not need a console. They will be delivered with the correct IP and Gateway numbers pre-configured and, when powered up, automatically connect to the network assuming that real configuration is the same as the one described on the information sheet. However, for trouble-shooting it is useful if you can provide a standard PC monitor and keyboard that can be connected to the test-box for trouble shooting. That way, it might be possible to solve problems while the test-box is at your site. If no console is available, then the only solution will be to send the test-box back for repair. 2.6 DNS Issues The hosting organization should provide an IP number and name for this box, following your normal numbering scheme. The name will be aliased to a name of the form: ttXYZU.ripe.net where XYZU is a number ranging from 0001 to 9999. The mapping of names to IP numbers will be set up in the the RIPE name server. The host should set up the in-addr mapping shortly after the domain name is known. In order to configure the test-boxes, the host should also provide the IP address of at least one topologically nearby resolving name-server. 2.7 Routing Issues The test-boxes are set up to listen to the RIP and IRDP protocols. If those protocols are not available on the local net, then the address of a static gateway should be provided. 2.8 Disclosure of data In section 4.1 of [1], the disclosure of data was discussed. We realize that this is a delicate matter and that all sites hosting a test-box should agree on the policy in order to make the project a success. We therefor plan to organize a BoF at the RIPE-30 meeting (May 1998) where all issues regarding data-disclosure can be discussed. We will then write a data-disclosure policy based on the outcome of the discussions at this BoF and ask for comments from the sites that have installed or applied for a test-box before the RIPE-30 meeting. It is our goal to write a data-disclosure policy that is acceptable to all participants in the project. Until this data-disclosure policy is in place, sites hosting a test-box will get access to the data from the test-box at their site but they agree not to publish any results outside their own organization or the relevant RIPE working groups. The how one can obtain a copy of the data as well as format of the data will be published on our web-site (http://www.ripe.net/test-traffic/) . After the data-disclosure policy is in place, new requests for hosting test-boxes will only be granted if the new site agrees with the data-disclosure policy. It is clear that, as the project moves along, the data-disclosure policy might have to be changed. Any changes in the policy will be discussed with the participating ISP's before they are being implemented. The new data-disclosure policy will only apply to data taken after the change in the policy. 2.9 Form If you meet all the requirements, please let us know by filling out the application sheet. This sheet can be found online at: http://www.ripe.net/test-traffic/host_testbox.html. Again, if you cannot meet all requirements or want to discuss your specific situation first, please do not hesitate to contact us. 3 First installation Figure 3: A test-box. This section describes the first installation of a test-box. Figure 3 shows a drawing of a test-box that should help you to locate the various connectors on the test-box. The connectors will be labelled. This section will, as we get more experience with the installation of our test-boxes, be updated. Before you begin with the installation, please visit our web-site (http://www.ripe.net/test-traffic/) and download the latest version of this document. 3.1 Installation 3.1.1 Installing the hardware When the test-box arrives check it for obvious shipment damage before mounting it into a rack. If the test-box has been damaged, notify the delivery company as well as the RIPE-NCC. Depending on the nature of the problem, we will then decide how to proceed. Do not switch on a damaged test-box. On the back of the test-box, you will find a grey flat-cable with a DB9-connector. This cable has to be connected to the left DB9-connector on the back of the machine. If the cable has been disconnected during shipping, please put it back in. The antenna has to be mounted outside. Details depend on the mounting option selected. If necessary, one can open the antenna and unplug the cable by removing the black, plastic, nut on the bottom of the antenna. The antenna cable can be extended using standard 50 Ohm coaxial cable and BNC plugs up to 50 m. However, for calibration purposes, we have to know the length of the antenna cable with an accuracy of about 1 m, so please measure the length of the cables before installing them. It might be necessary to move the antenna a bit in order to optimize reception, so do not cut any cables or fix the antenna in a permanent position yet. The antenna cable has to be connected to the BNC plug on the back of the crate (figure 3). Then connect the test-box to Ethernet using the RJ-45 connector on the back. If your site uses BNC connectors for the local Ethernet, get an adapter. Do not use the BNC connector on the back to connect the box to the local Ethernet, this may seriously damage your routers as well as the test-box. Finally, connect the test-box to 110 or 220 V The power-supply will recognize the voltage automatically. A power cord is included, you may, however, have to replace the plug. 3.1.2 Booting the test-box Now switch on the test-box. A console monitor can optionally be attached to the test-box to monitor the boot process but this is not necessary. The test-box will boot automatically. There is no need for the local contact to configure the test-box, as the test-box will be pre-configured at the RIPE-NCC before it is sent to you. Also, system maintenance and further configuration will be done from the RIPE-NCC. When the test-box is running (approximately 3 minutes after it has been switched on, or when a login prompt appears on the screen), check if it is visible from your local network by pinging it at the pre-configured address. If it doesn't respond, first check the green status LED on the Ethernet card. If it is off, check your cables, reboot the machine and ping it again. If the LED is on, attach a console monitor, reboot the machine and write down any errors that appear. After that, please contact us and we will try to solve the problem. Now wait for at least 30 minutes for the GPS clock to scan the sky for satellites and to get an accurate signal. If you have the NTP-software [3] installed on your local machines, you can check if the antenna signals are sufficient using the sequence of commands shown in figure 4. Figure 4: Testing the reception conditions for the GPS receiver. If 0100 bit of the status word is NOT on, please contact us. The most likely explanation is that the antenna does not have a clear view of the sky or that the signal is too weak. We can check this remotely. If the problem is indeed caused by the positioning of the antenna, we will ask you to find another position while we are monitoring the antenna signals. You can also try to move the antenna and see if the conditions improve before contacting us. Note, however, that it can take up to 30 minutes before a chance in antenna position is reflected in the status word. If there are no further problems, simply confirm the installation by sending an email to tt-ops@ripe.net. Please put the length of the antenna cable in this email. 3.1.3 The final steps When the optimal antenna position has been found, we suggest that you fix the antenna at this position. The easiest way to do this is with cellotape. Alternatively, you can unscrew the base of the antenna, drill a hole and then put the antenna back together. If you have to unplug the antenna cables for this, please inform us in advance. The front panel of the test-box can be locked. After you have powered up the test-box, we suggest that you lock the front panel, remove the key from the lock and put the key in a safe place away from the test-box. This will make it harder, though by no means impossible, to tamper with the test-box. 3.2 Troubleshooting We will, as we get more experience with the installation of our test-boxes, put a list with questions and problems, as well as their solution, related to the installation of test-boxes on our web page. 4 Day to day management 4.1 Security There are several security related issues involved in day to day management. These are discussed elsewhere [2]. We assume that the local contact has read and will follow (the spirit of) this document. 4.2 Reboots First of all, do not reboot the test-box unless you are asked to do so by the RIPE-NCC. The test-box can be rebooted by pressing the reset button. The reboot process will take a couple of minutes and can optionally be monitored by attaching a console monitor. The test-box is running again when the login prompt appears or when the box can be ping-ed from another machine. 4.3 Moving test-boxes, power outages If a test-box has to have its IP or gateway numbers changed, please provide us with the new numbers and the day that they become effective as soon as possible but at least a week in advance. We will then reconfigure the test-box remotely, shut it down and let you know when it can be moved. After the test-box has been moved, follow the procedures for first installation. If a power outage is planned at your site, please notify us a couple of days in advance. We will then do an orderly shutdown of the test-box shortly before the power outage. After the power is switched back on, the test-box can be booted as usual. 4.4 Disk problems Our test-boxes are equipped with two swappable hard-disks. The data from the top (master) disk is copied to the second (slave) disk at regular intervals. If the disks are swapped In case of a disk related problem, we will ask the local contact to swap the disks. First switch off the machine. Use the second key to unlock the disks and carefully pull them out of their slots using the black handle. Insert the disks in the other slots and lock them again. Then switch on the machine again. We will first try to reformat the second disk and start using it as a slave. If this does not work, we will ask the local contact to switch off the machine, unplug the second disk and send the entire unit back to the RIPE-NCC. 4.5 Operating system upgrades If we decide to upgrade the operating system, we will download and install the new kernel on the second hard-disk. After that, we will ask the local contact for a suitable time to swap the disks. At this time, the local contact has to switch off the machine, swap the disks as described in section 4.4, reboot the machine and optionally monitor the boot process. As soon as the machine is up again, we will check if the new kernel works as it should. If not, then the local contact will be asked to swap the disks back and reboot the machine again. 4.6 Other hardware problems In the first instance, test-boxes with hardware problems not related to the disks have to be dismounted and sent back to the RIPE-NCC for repair. Unless the problem might be related to the GPS receiver, the antenna does not have to be dismounted. As we get more experience with our test-boxes, we might provide instructions to the local contacts for doing small repairs. 5 Using the test-box as a Stratum-1 NTP server Our test-boxes run the NTP [3] software and can be used as a stratum-1 NTP server for your local machines. If you want to do this, add the following line to your NTP configuration file (usually /etc/ntp.conf): server hostname prefer and restart the NTP deamon. References 1 H. Uijterwaal, O. Kolkman, ``Internet Delay Measurements using Test-Traffic, Design Note'', RIPE-158 . 2 O. Kolkman, ``Internet Delay Measurements using Test-Traffic, Security Document'', RIPE-179 . 3 NTP version 3-5.92 or later, the software can be downloaded from http://www.eecis.udel.edu/ ntp/