Ricci Street < Port 80 < Customhouse || search | sitemap | help gazette | theater | bistro

|

The Internet other pages New Media aka Distributed Networks Tin-Can Networks | Packet-Switched Networks | Internet Firsts this page Metrics | Architecture | Plumbing

Metrics

How do we measure the Internet? What everyone wants to know:

how many people are online?
what are they doing?

As with any measurement of people and behavior, we aren't even sure how to ask the question. For example, "58% of Americans are online."

Does "are" mean now, as in how many people this very moment are sitting at a computer? Does it mean how many people use the Internet regularly? Define "regularly". Does it mean how many people have ever used the Internet personally more than once? Does the internal network at my job count?

Let's start with what we can count directly. Brad Myers at Carnegie Mellon maintains the Computer Almanac, a collection of interesting and useful numbers about the Internet. Hobbes's Timeline has graphs at the bottom of the page for hosts (assigned Internet Protocol number) and registered domain names. In the popular press, this is known as an adoption or diffusion curve.

How many web sites?

Another tough question: how many web sites are there? Well, what's a web site? Do you count domain names or IP numbers?

Medaille.edu is a registered domain name. Each PC/laptop in our classroom has an IP number, assigned dynamically when you turn the machine on. As you can see by running a public ARIN Whois query (download the software from PCHelp's Network Tracer), Medaille gets its upstream service from Drexel University in Philadelphia. Drexel has the block of addresses from 144.118.0.0 through 144.118.255.255. Medaille's is 144.118.31.65.

RicciStreet.net is a registered domain name that shares an IP# of 216.117.150.129 with several other domains, such as toLearn.net and ClearLightStudio.com.

If you count IP#'s, then Medaille seems to get over-counted and 216.117.150.129 gets undercounted.

AOL and most other ISPs have fewer IP# than subscribers, so they hand them out as users log on.

The Internet Traffic Report monitors the flow of data around the world.

Your Internet surfing safari may be smooth today, but perhaps you can't reach Yahoo or a few web sites in Europe. This web site will tell you if those regions of the Internet are currently slowed down. By checking the Internet Traffic Report, you can determine if your problems are global or local.

The Online Computer Library Center's Web Characterization Project paints an interesting ongoing portrait. It uses frames, so you'll have to click Statistics at the top left.

According to Tim Berners-Lee's The Myth of Names and Addresses, info.cern.ch was the "name" of the first World Wide Web server. In that article, Berners-Lee makes it very clear why it will always be hard to count the Web.

How many hits?

An individual server's logs record traffic in terms of numbers of requests ("hits") and bandwidth usage. See Ricci Street's Traffic Report for more details.

What else can we count?

Indirectly, we have recently been able to count:

PC sales
ISP revenues
mentions in broadcast and dead-tree media
advertising both online and off

Finally, there are a raft of surveys. How many hours a week do you spend at work goofing off online? Clearly, the surveys are the least reliable. Perhaps all we can say for sure is that Internet usage is growing. To which I would add that we haven't begun to see the big backlash yet; the recent flap over Napster is a harbinger.

How wide is the Web?

Across the Web in 19 Clicks
by John Allen Paulos
ABCNEWS.com, December 1999

How many clicks on average does it take to get from one of two randomly selected documents to another?

The answer: An average 19 clicks can take you from this page to most any other page on the Web.

Architecture

The Internet is a distributed network with an open architecture.

Where traditional media are one-to-many, such as newspapers, radio, and television, the Internet is many-to-many.

Where traditional organizations are top-down power structures, the Internet isn't even bottom-up because there is no up.

Where traditional mainframe computer networks are built on the model of the telephone system with all the intelligence at the center and either "dumb terminals" or a telephone handset at the periphery, the Internet is dumb or stupid inside. All the intelligence is in your PC and in your head.

Start with the Atlas of Cyberspaces. The maps that will probably make most sense are the ones based on geographic metaphors. For maps of ISPs and regional backbones try Martin Dodge's explorations of the geographies of the Internet, the Web, and other emerging cyberspaces at University College, London. To get a sense of the difficulties of mapping the whole Internet, learn more from the Cooperative Association for Internet Data Analysis (CAIDA).

Check out World Com's UUNet North American map.

Tier 1 carrier

A Tier 1 ISP is a telco or Internet service provider IP network which connects to the rest of the Internet only via a practice known as peering.

The name Tier 1 refers to their position at the top level of the "food chain" of network providers. ...

Tier 1 providers own the physical medium over which information is carried, as well as the network equipment which manages that information, and are either telcos who pre-dated the Internet or early movers in the Internet market who managed to build up critical mass in the days prior to the introduction of paid transit agreements.

To supplement these macroscopic visualization efforts, you can get microscopic details from Mappa.Mundi Magazine's Mapping How The Data Flows. For traceroute and other mapping tools, such as ping, all on one page, try Network Tools.

For example, if you want to see the route a packet takes, how long each leg of the journey takes, and who operates every computer it routes through, I use Sam Spade. Comparisons of routes during busy and no-so busy hours will tell you a lot about the overall structure of the Internet.

April 17, 2003, Traceroute Results from Genuity's Mountain View CA Data Center to www.BudapestSun.com:

gentrcrte to www.budapestsun.com (213.134.7.164), 20 hops max, 40 byte packets

1 vlan1.mtvwca1-dc1-dfa1-rc1.bbnplanet.net (128.11.177.2)
2 gigabitethernet4-0.mtvwca1-dc-dbe1.bbnplanet.net (128.11.193.65)3 p3-1.mtvwca1-br1.bbnplanet.net (4.24.9.165)
4 p4-0.paix-bi2.bbnplanet.net (4.24.7.22)

5 82.ATM1-0.BR5.SAC1.ALTER.NET (204.255.168.169)
6 0.so-2-1-0.XL1.SAC1.ALTER.NET (152.63.52.226)
7 0.so-3-0-0.TL1.SAC1.ALTER.NET (152.63.53.250)
8 0.so-1-2-0.TL1.NYC9.ALTER.NET (152.63.10.77)
9 0.so-3-0-0.XL1.NYC1.ALTER.NET (152.63.27.29)
10 0.so-0-0-0.XR1.NYC1.ALTER.NET (152.63.19.85)
11 507.ATM6-0.GW10.NYC1.ALTER.NET (152.63.26.145)
12 bt-gw.customer.ALTER.NET (157.130.17.2)

13 t2c1-ge6-1.us-nyb.concert.net (166.49.208.205)
14 t2c2-p4-0.uk-lon2.concert.net (166.49.208.245)
15 t2c1-ge6-1.uk-lon2.concert.net (166.49.164.213)
16 t3a2-p4-0-0.hu-bud.concert.net (166.49.208.254)
17 t3a1-ge5-0-0.hu-bud.concert.net (166.49.160.17)
18 166-49-161-14.concert.net (166.49.161.14)

19 62.80.64.194 (62.80.64.194)
20 nextra.bt.hu (62.80.64.86)

Try some yourself at TraceRT's Multiple Traceroute Gateway.

For a 3D-version of the packets' route, try Holger Lembke's 3D Traceroute.

Learn more about Geek Tools.

How Internet Infrastructure Works
by Jeff Tyson

How the Internet infrastructure might scale ...

The Corner Internet Network vs. the Cellular Giants
by John Markoff
NY Times, March 4, 2002

Many Silicon Valley engineers now believe that it will be possible to take the tens of thousands of inexpensive wireless network connections that are popping up in homes and coffee shops all over the country and lash them together into a single anarchic wireless network. Connections could theoretically be passed from one Wi-Fi node to another, similar to the way wireless phone signals pass from cell to cell, thereby significantly extending the wired Internet.

Modeled closely on the original nature of the Internet, which grew by chaining together separate computer networks, the technology — known as wireless mesh routing — is being rapidly embraced in the United States as well as in the developing world, where it is viewed as a low-cost method for quickly building network infrastructure.

If the engineers are right, the popular and inexpensive Wi-Fi wireless standard, also known as 802.11, could serve as the wedge for the next-generation Internet, enabling a new wave of wireless portable gadgets that ultimately blanket homes, schools and shopping malls with Internet access.

Plumbing

A protocol is a set of rules that governs how data (the long string of ones and zeros that represents an email or a graphics file or your voice):

travels across digital networks from one computer to another
is organized and displayed on either end

TCP/IP

Transmission Control Protocol / Internet Protocol

The network of networks. The trainers at Network Solutions use a house-moving analogy.

You pack your house in boxes and put your new address on them.

The moving company picks them up, makes a list of the boxes, and ships them across country along the most efficient route -- this may mean putting your dishes and your bedroom furniture on different trucks.

Your belongings arrive at your new address. You consult your list to make sure that everything you shipped has arrived (in good shape), then you unpack your boxes and 'reassemble' your house.

As with all other communications protocols, TCP/IP has layers:

The Transmission Control Protocol (TCP) part breaks data into packets that the network can handle efficiently. It verifies that they all arrive and asks for those that didn't to be re-sent. It reassembles the data where it can be read and displayed.

The Internet Protocol (IP) part sizes, envelops and addresses each packet so that the computers at each stop along the network can move the packet along to its destination. IP forwards each packet based on a four-byte destination address, the IP number. Ricci Street's IP # is 208.234.16.68. Which is easier to remember?

PC Lube and Tune's Introduction to TCP/IP
February 25, 1995

A Uncertain Path

Every time a message arrives at an IP router, the router makes an individual decision about where to send it next. There is concept of a session with a preselected path for all traffic. Consider a company with facilities in New York, Los Angeles, Chicago and Atlanta. It could build a network from four phone lines forming a loop (NY to Chicago to LA to Atlanta to NY). A message arriving at the NY router could go to LA via either Chicago or Atlanta. The reply could come back the other way.

How does the router make a decision between routes? There is no correct answer. Traffic could be routed by the "clockwise" algorithm (go NY to Atlanta, LA to Chicago). The routers could alternate, sending one message to Atlanta and the next to Chicago. More sophisticated routing measures traffic patterns and sends data through the least busy link.

If one phone line in this network breaks down, traffic can still reach its destination through a roundabout path. After losing the NY to Chicago line, data can be sent NY to Atlanta to LA to Chicago. This provides continued service though with degraded performance. This kind of recovery is the primary design feature of IP.

The loss of the line is immediately detected by the routers in NY and Chicago, but somehow this information must be sent to the other nodes. Otherwise, LA could continue to send NY messages through Chicago, where they arrive at a "dead end." Each network adopts some Router Protocol which periodically updates the routing tables throughout the network with information about changes in route status.

In my experience, "periodically" can mean hours, not days or minutes.

A dozen protocols run on top of TCP/IP to provide the Internet's basic services. Think of them as a cross between a traffic cop and a summer camp counselor. After TCP / IP gets the packets where they're going, these application protocols arrange the files so that you can get to them through a graphical user interface (GUI). I'll focus here on the five that you're most likely to use:

email - SMTP | the Web - HTTP
file transfer - FTP | chat - IRC | newsgroups - NNTP

SMTP - Email

Simple Mail Transfer Protocol

You use this protocol over port 25 to send mail and receive it at your service provider's mailbox. Then, you use one of two other protocols, POP3 or IMAP, to save the email and download it periodically from the server. Ricci Street uses sendmail, the most widely-used SMTP server for e-mail, which includes a POP3 server. SMTP, used since the early 1970s, is fast and accurate partly because it sends only text that uses the American Standard Code for Information Interchange (ASCII). That is, it works with only the alphanumeric characters in the middle of your keyboard (and a few special characters for a total of 128). That's why email looks so plain.

Learn more about ASCII Characters.

If you want to send a Word file, an Excel file, a sound file, or an image file via email, you're clearly out of ASCII territory. Since the early 1990s, you've been able to attach the non-ASCII file to the email and use the Multi-Purpose Internet Mail Extensions (MIME) to send it as a binary (ones and zeros) stream.

HTTP - The Web

Hypertext Transfer Protocol

The PC running my Web server has HTML files, graphics files, and some text files, mostly style sheets and JavaScripts. It also has an HTTP daemon, which waits for HTTP requests on port 80. Your Web browser, an HTTP client, makes those requests when you type in a Uniform Resource Locator (URL) or click on a hypertext link. The HTTP daemon receives the request and, after any necessary processing, returns the requested file. If it's an .htm file that calls other files such as image or scripts, those files will start additional transfer requests. You've noticed how the page can display and then the images can come in one at a time and often slowly.

This innocent-looking protocol was introduced by Tim Berners-Lee in 1991. Two years later, a bunch of grad students at the University of Illinois built Mosaic, the first freely available browser, to display the HTML code that the HTTP protocol delivered. In no time, there were a couple hundred browsers available for the downloading and they all parsed and displayed the HTML a little differently. By 1999, two browsers dominated and HTTP was delivering trillions of packets every month to them all over the world.

The MIME header, by the way, begins every Web transmission. Your brower uses it to select an appropriate "player" application for the type of data the header indicates. Players for .htm, .gif, and .jpg are built into the browser. Other players, such as those for .doc or .xcl, are probably already on most PCs. Still other players, such as those for .mpg or .pdf, need to be downloaded; they are called plug-ins.

In the news ...

STANDARDS BODY APPROVES HTTP 1.1
C|Net, July 5, 1999

An international standards group declared HTTP 1.1 a draft standard this week. The new version resolves some of the design flaws of HTTP 1.0 and improves the speed of the transfer of information, according to its designers at the World Wide Web Consortium and the Internet Engineering Task Force, which approved its draft standard status.

HTTP is layered on top of two other basic protocols to govern the Internet. Internet protocol (IP) is located at the bottom and defines the Internet, while transport control protocol (TCP) sits in the middle and establishes a stream for transferring data, and HTTP, at the top, wraps data into packets and determines how they are sent. While HTTP 1.0 slowed the transfer of information by requiring a new stream for each packet of data sent, the new version can send multiple packets along the same stream.

Furthermore, HTTP 1.1 will allow content providers to choose more specifically which content can be cached, allowing Internet service providers to keep copies of frequently accessed content closer to the client to lessen the distance required for information to travel over the Internet.

FTP - File Transfer

File Transfer Protocol

This protocol is the simple brute-force way to move files as binary or ASCII bitstreams between computers on the Internet. You will use FTP to load HTML files from your desktop or laptop onto the Ricci Street server in North Carolina that will serve them to any browser that requests them. You will also use FTP to download programs and other files from servers on the Internet onto your desktop or laptop.

FTP has been used for almost thirty years and is traditionally received on port 21. Even though the downloading FTP is built into your browser and the uploading FTP is built into some of them, I recommend that you to upload with a standalone FTP client such as WS_FTP from Ipswitch. It's still point-and-click, and it's as geeky as I believe a self-respecting MBA needs to get. Learn more.

IRC - Chat

Internet Relay Chat

Chatting is talking to other people who are using the Internet at the same time you are. Usually, one site is the repository for the messages (or "chat site") and the chatters can take part from anywhere on the Internet.

The trick with IRC is making your message appear, as soon as possible, on the screen of everyone you're chatting with. That's why it's also called Instant Messaging. It's still a bunch of packets zooming around the Internet. But it takes a different set of rules and conventions, a different protocol. The software for both server and client is also different.

If you want to get geeky about it, sites such as ICQ and Talk City or IRC networks such as the Undernet provide the servers and help you download an IRC client to your PC. Talk City also offers an IRC client applet that it downloads for you as part of their home page so that you can start chatting right away.

The IRC protocol was introduced in 1988 and by convention uses port 6667. It now has many variations or forks.

History of IRC (Internet Relay Chat)
Daniel Stenberg
Version: 0.8 - September 24, 2002

In addition to the four main IRC networks -- ; all the other protocols that the Internet runs on are open-source protocols. In addition, the competing IM systems from AOL, Microsoft, and Yahoo all use different and incompatible proprietary protocols.

NNTP - Newsgroups

Network News Transfer Protocol

Usenet newsgroups are the more popular form of organized email. (Mailing lists, the less popular form, use SMTP.) Since 1986, NNTP servers, including the one at your ISP, have managed the network of collected Usenet newsgroups. The full public Usenet feed includes 80,000 + newsgroups making up terabytes of data. Your ISP no doubt gets a partial feed and keeps it only a week or two. The NNTP client is included as part of your Netscape or Microsoft browser or you may use a separate client program called a newsreader, such as AOL's. Learn more about the software.

The newsgroups back to 1995 are archived and searchable at Deja.com, now part of Google. The newsgroups themselves are public, which means they have a high noise-to-signal ratio. That means there's a lot of ka-ka, repetition, and unreliable invalid information compared to the real stuff. However, next to mailing lists, newsgroups are the best source of up-to-the-minute information on a huge array of topics and to the experts who can help.

Learn more at Google's Basics of Usenet.

other protocols

Sandvine identifies 25 application protocols in Internet traffic, mid-2003.

Sources: Sandvine, Point Topic
 

CacheLogic breaks it down this way and claims that over two-thirds of Internet traffic is Peer-to-Peer, especially video.

PROTOCOL RECOGNITION

CacheLogic's P2P Traffic Analysis - 2004

File-Sharing Winners and Losers of 2005
Slyck, December 24, 2005

BitTorrent - There is little doubt BitTorrent has emerged as the quintessential file-sharing protocol in 2005. Estimates on its size are staggering – anywhere from 60%-90% of a ISPs bandwidth is consumed by this protocol. In addition, it’s suggested that up to 10 million individuals are transferring files via the BitTorrent protocol at any given moment.