Trapeze/IP: TCP/IP at Near-Gigabit Speeds

This paper presents experiences with high-speed TCP/IP networking on a gigabit-per-second Myrinet network, using a Myrinet messaging system called Trapeze. We explore the effects of common optimizations above and below the TCP/IP protocol stack, including zero-copy sockets, large packets with scatter/gather I/O, checksum offloading, message pipelining, and interrupt suppres- sion. Our experiments use extended FreeBSD 4.0 ker- nels on a range of Intel and Compaq Alpha platforms. These experiments give a snapshot of the FreeBSD TCP/IP implementation running at bandwidths as high as 956 Mb/s. We also report some results using Gigabit Ethernet products from Alteon Networks, which yielded a TCP bandwidth of 988 Mb/s using zero-copy sockets on a 500 MHz Compaq Alpha 21264 workstation.