Peer-to-Peer Frameworks: The Future at Port 80: Charthouse

By frameworks, I mean the hardware / software rules. The networks and the platforms. The plumbing and the wiring; the software code. Code is law, folks, and this is a very fateful decision.

Ten years ago, Tim Berners-Lee designed the Web to be a peer-to-peer network. Every computer connected to the Web would be both giver and receiver of information. For murky (to me) commercial reasons and because most people don't have always-on, high-bandwidth connections to the Internet, the client-server model came to dominate. The servers stayed on all the time and the clients (the browsers) were separate software applications.

By the late 90's, enough students were using always-on, high-bandwidth connections in college dorms to make Napster not only a reality but a threat to the music distribution industry, which got a lot of press attention. Every client is a server?

The Napster peer-to-peer network was simply a use of the Internet that was always possible. Now that that particular cat is out of the bag, what's next?

Definitions |

Distributed computing is a peer-to-peer network that also takes advantage of the fact that PC's connected to the network are idle most of the time.

Peer to Peer (P2P) is simply one PC talking to another without using a server on the Internet. P2P is nothing new, but with Napster's notoriety it's seeing something of a resurgence. P2P puts control back into the hands of users.

Typically, servers control the flow of data in and out of computers, but with peer-to-peer (P2P) the server is removed from the equation, allowing computers and their users to share files and other data directly.

The big-name online publishers are weighing in, too. Byte says there more to all this than just piracy and Wired has a guide to over 200 global file-sharing lists.

With the rise of the Internet, many-to-many communication became possible. Everyone can reach everyone, theoretically at least. In a "many-to-many" environment, "intellectual property" laws can no longer be enforced without controlling any information exchanged between any two persons or groups. This cannot be automated since automats cannot be taught the difference between "good information" and "bad information", there are too many false positives and false negatives, as porn filters clearly show (although porn should be easier to control since there are so many people who claim to recognize it when they see it -- have you ever met someone who recognizes copyrighted content when he sees it?) . Additionally, in a restrictive environment, users will find ways to disguise "improper" information as "proper" information.

So you would need nearly one human "watcher" for each user, since you would have to control them all of the time.

This is, obviously, impossible.

The only way to enforce "intellectual property" in such an environment is therefore by changing the environment back to what it was before, i.e. by turning back time. This is what the industry is currently trying to do. By outlawing hyperlinks and peer-to-peer sharing software, they are effectively trying to pull us back from many-to-many to few-to-many. They would like the Internet to be just TV with some extra features.

This is, obviously, in violation of the First Amendment, other national free speech laws, and the internationally guaranteed right to freedom of speech. It is also, obviously, extremely stupid and dangerous.

a set of open protocols that allow any connected device on the network ranging from cell phones and wireless PDAs to PCs and servers to communicate and collaborate in a P2P manner.

JXTA peers create a virtual network where any peer can interact with other peers and resources directly even when some of the peers and resources are behind firewalls and NATs or are on different network transports.

>Interoperability - across different peer-to-peer systems and communities
>Platform independence - multiple/diverse languages, systems, and networks
>Ubiquity - every device with a digital heartbeat

The IRIS project is developing a novel decentralized infrastructure, based on distributed hash tables (DHTs), that will enable a new generation of large-scale distributed applications. DHTs are robust in the face of failures, attacks and unexpectedly high loads. They are scalable, achieving large system sizes without incurring undue overhead. They are self-configuring, automatically incorporating new nodes without manual intervention or oversight. They provide a simple and flexible interface and are simultaneously usable by many applications.

This work is funded under NSF Cooperative Agreement No ANI-0225660.

A team of government-funded US scientists is building a Peer-2-Peer (P2P) network that they say will solve technical problems with existing P2P networks, such as Gnutella and Kazaa, and might even one day supersede the web.

The network, dubbed the Infrastructure for Resilient Internet Systems (IRIS), will speed up searches and information transfer over the internet, and aims to foil "Denial of Service" attacks by hackers - in which a web server is swamped with requests for a page until it crashes. ...

Its three design criteria are to guarantee:

• that as long as there is no physical break in the network the target file will always be found

• that adding more information to the network will not affect its performance

• that machines can be added and removed from the network without any noticeable adverse affects.

Five institutions, including MIT and the University of California at Berkley, who have jointly received a $12 million grant from the National Science Foundation to develop IRIS.

While media companies step up their legal crackdown on Internet song-swappers, separate teams of software developers -- from the Middle East to Madrid -- toil away on a foiling technology: an anonymous file-sharing network.

The technology behind Napster has found a legitimate role in business. ...

P-to-p will also make collecting data a lot easier. The U.S. government gathers statistical information in 5,000 categories from 3,000 counties across the nation, and the process is cumbersome and wide open to error. Local workers in the field record information on paper forms, which are sent to an agency's state office. The information is then typed into a computer and forwarded to Washington.

With the technology ... from a company called NextPage, each agency's county representative would enter data on a handheld computer and copy the file to a local server. The data would be automatically available to everyone else on the system and to the public; every time the file is modified, everyone would see the update.

"The potential time and cost savings are enormous," notes Niemann. "And you wouldn't have an annual statistical abstract; it would be a real-time statistical abstract."

Peer-to-peer applications have wide-ranging business applications beyond the MP3-sharing utility that made it a household name. In fact, peer-to-peer is at the center of the next computing revolution and will fundamentally change the face of the Internet.

Combat life-threatening illnesses by linking millions of PCs like yours .... This "virtual supercomputer" uses peer-to-peer technology to make unprecedented amounts of processing power available to medical researchers to accelerate the development of improved treatments and drugs that could potentially cure diseases.

... on the need for regulators to understand that the end-to-end architecture of the Internet needs to be protected against the central-planning model of telephony and cable television.

An Example of P2P's True Business Value
by Nate Zelnick
Internet World email newsletter

"Peer-to-peer" may be perhaps the last great buzzword forced into general parlance by Internet bubbliness, but it's also probably the most important. In essence, of course, peer-to-peer is the most basic kind of network, so it's less a breakthrough than a rediscovery. But after years of trying to make the Internet look like client-server or a shopping mall, back to basics is more than it at first might seem.

But it's hard to explain to people why and in what ways P2P changes things until there's something to show. Luckily Tim Berners-Lee, who invented the Web and went through similar contortions trying to get people to see why that was important until he had a Web to show, has been screaming this from every mountaintop he can find. His Semantic Web idea envisions layers of meaning that turn data into information.

Although Gnutella, Freenet, and Napster have shown that information wants to be free even if media companies object, these file-sharing schemes are pretty blunt examples of P2P. For new approaches to sink into general understanding you need an exemplar. For Semantic Web P2P systems, you need an information-rich subculture that struggles with the problems of gathering massive amounts of data but eventually turns chaos into widely used content. Say, something like the U.S. Federal Government. A few weeks ago, a 70-agency working group announced a new initiative that will use the Internet as a widely distributed system for sharing information --exactly what it was designed to do. The Feds have been doing this forever. What's new in the FedStats working group's efforts is that this data will be "alive."

Using NextPage's Context Server, which makes heavy use of metadata to describe the inner workings of data so that it can be efficiently found and used, agencies like the EPA and the Department of Agriculture hope to transform the arduous process of collecting and collating data from thousands of local offices and sites into a constant stream of contextually useful data. Not only does this make efforts like publishing the biannual Statistical Abstract (a catalog of everything) easier, it creates a different data environment in which statistics reflect events in process instead of providing a few discrete snapshots. It's the analytical equivalent of motion pictures.

Getting the databases and document repositories integrated was trivial, according to EPA computer scientist Brand Niemann. The real trick is figuring out how to tell people what they can do now. Niemann sees the context-driven FedStats as eventually being able to produce documents and reporting systems that consist of references to an ever-changing data set. Combined with the ability to do cross-tabular analysis and data mining on the spot and across an ever-expanding number of local nodes, the possibilities become seriously interesting.

Learn more: Spinning Your Web
by Tony Rizzo
Internet World Magazine, April 15, 2001

P2P in organizations

Don't even think about it. And for goodness sake, don't even mention the "N" word to the new generation of business-oriented peer-to-peer distribution and storage technology. Companies like NextPage, Kontiki, InView, and Bandwiz have moved way past the file-swapping infamy of the early MP3 craze. These days, the preferred nomenclature is "content networks," or "grid distribution," even though the core principles and some of the very technologies are similar. But by any name, P2P is about decentralizing content delivery. It allows PCs or local servers to swap and update content more efficiently by circumventing a centralized repository and instead creating a virtual space that looks like a single source of information, but is in fact made up of these multiple, sometimes thousands, of peer machines.

When EContent looked at P2P last year, much of P2P's promise for B2B was still only theoretical, but this year we were surprised to see how quickly a number of companies—not to mention the U.S. government—are adopting or exploring various types of peer-to-peer content delivery for internal and business-to-business purposes. Many early adopters and their vendors argue that this approach not only saves money, but it expands the range and size of content that the enterprise can push across its network.

Kontiki - Kontiki Grid Delivery Technology creates a network of shared resources where any PC or server can deliver content on demand. This flexible and scalable architecture provides IT managers with complete control.

An Example of P2P's True Business Value
by Nate Zelnick
Internet World email newsletter

Using NextPage's Context Server, which makes heavy use of metadata to describe the inner workings of data so that it can be efficiently found and used, agencies like the EPA and the Department of Agriculture hope to transform the arduous process of collecting and collating data from thousands of local offices and sites into a constant stream of contextually useful data. Not only does this make efforts like publishing the biannual Statistical Abstract (a catalog of everything) easier, it creates a different data environment in which statistics reflect events in process instead of providing a few discrete snapshots. It's the analytical equivalent of motion pictures.

Many organizations fail to notice that intranets support and encourage a definite management and cultural style, one that may not be compatible with their incumbent managers.

Many countries fail to notice that the Internet supports and encourages a definite politics, economy, and culture, and that they may not be compatible with the current political, economic, and cultural structures.

Who owns the Internet? Until recently, nobody. That's because, although the Internet was "Made in the U.S.A.," its unique design transformed it into a resource for innovation that anyone in the world could use. Today, however, courts and corporations are attempting to wall off portions of cyberspace. In so doing, they are destroying the Internet's potential to foster democracy and economic growth worldwide.

The Internet revolution has ended just as surprisingly as it began. None expected the explosion of creativity that the network produced; few expected that explosion to collapse as quickly and profoundly as it has. The phenomenon has the feel of a shooting star, flaring unannounced across the night sky, then disappearing just as unexpectedly. Under the guise of protecting private property, a series of new laws and regulations are dismantling the very architecture that made the Internet a framework for global innovation.

In other words, we ain't seen nothin' yet. We are not prepared for what's about to happen. Many won't like it. Then will come the backlash, and it won't be pretty.

If I speak with assurance here, it's because of an assumption. What will happen next is analogous to what happened in England when engine-powered machines took over for muscles more than two hundred years ago. Now a computer-powered network is going to take over for brains? Can we stop it before it's too late?

The networking approach that threatens to make the recording industry obsolete could also bring about a more reliable Internet.

While legislators in Washington work to outlaw peer-to-peer networks, one website is turning the peer-to-peer technology back on Washington to expose its inner, secretive workings.

But outragedmoderates.org isn't offering copyright music and videos for download. The site, launched two weeks ago, has aggregated more than 600 government and court documents to make them available for download through the Kazaa, LimeWire and Soulseek P2P networks in the interest of making government more transparent and accountable.

The documents include such items as recent torture memos related to the Abu Ghraib prison scandal, a Senate Intelligence Committee report on what the government knew before it invaded Iraq and a document showing how the Bush administration suppressed information about the full cost of its Medicare plan until after Congress passed the plan. There is also a copy of a no-bid contract obtained by a Halliburton subsidiary for work in Iraq and congressional testimony from former employees of the subsidiary showing how their company engaged in wasteful and costly conduct in Iraq (such as abandoning an $85,000 Mercedes truck after its tires went flat).

Thad Anderson, a second-year student at St. John's School of Law in Queens, New York, said he was driven to launch the site by what he says is the current administration's disregard for fundamental democratic structures and its increasing practice of withholding information from the public. He wanted to give people access to crucial data about what elected officials were doing.

Peer-to-peer technology is too scary for the average enterprise, but what if it is blended with client/server?

Five peer-to-peer models

Atomistic: Considered the truest p-to-p architecture: direct client-to-client connectivity with no server present; no method of creating communications links based on data availability or user identity.

User-centered: Directories let users make connections with other users on a network.

Data-centered: Users search and access data held on other users' systems.

Web Mk 2: This is a convergence of the above three models with Web architectures and infrastructure. In this model, browsers evolve into user-configurable workspace managers that integrate these three types of p-to-p models. Multiple directory services can link users together on an ad hoc basis. Multiple indexes allow access to different forms of data whether it is on servers or clients.

Compute-centered: In this approach, instead of using a single large processor, an application's processing is divided among multiple clients and a server is used to coordinate the split processing. The distinction with this approach and parallel processing is that nodes are spread over the Internet and can be accessed on an as-needed basis.

Freenet

If you're concerned about tracking and monitoring your online activity, Ian Clarke's Freenet Project deserves your consideration.

Freenet is free software which lets you publish and obtain information on the Internet without fear of censorship. To achieve this freedom, the network is entirely decentralized and publishers and consumers of information are anonymous. Without anonymity there can never be true freedom of speech, and without decentralization the network will be vulnerable to attack.

Communications by Freenet nodes are encrypted and are "routed-through" other nodes to make it extremely difficult to determine who is requesting the information and what its content is.

Users contribute to the network by giving bandwidth and a portion of their hard drive (called the "data store") for storing files. Unlike other peer-to-peer file sharing networks, Freenet does not let the user control what is stored in the data store. Instead, files are kept or deleted depending on how popular they are, with the least popular being discarded to make way for newer or more popular content. Files in the data store are encrypted to reduce the likelihood of prosecution by persons wishing to censor Freenet content.

The network can be used in a number of different ways and isn't restricted to just sharing files like other peer-to-peer networks. It acts more like an Internet within an Internet. ...

Freenet is not just theoretical, it has been downloaded by over 1.2 million users since the project started, and it is used for the distribution of censored information all over the world, including countries such as China and the Middle East. Ideas and concepts pioneered in Freenet have inspired hundreds of academic papers in the fields of computer communication, security, and law. Freenet has also received significant coverage in the mainstream press.

Freenet uses a decentralized P2P architecture to create an
uncensorable and secure global information storage system.

Shawn Fanning, on the right, with Ian Clarke. These two, along with KaZaA, can bring down the music industry? Was it that fragile? Did it add so little value?

A robust key-indexed information storage and retrieval system with no element of central control or administration. It allows information to be made available to a large group of people in a similar manner to the World Wide Web.

Improvements over this existing system include:

- No central control or administration required
- Anonymous information publication and retrieval
- Dynamic duplication of popular information
- Transfer of information location depending upon demand

The technology of P2P systems is stretching the applicability of modern copyright law to its limits. Content owners will possibly see the day that a technology like Freenet will render the current law ineffective against digital copyright infringement on the Internet. However, it is more likely that both P2P technology and copyright law will ultimately survive the current conflict intact, as they both adapt to legal and technological changes driven by worried content owners, digital libertarian programmers, and the file-hungry public.

That was the essential insight behind BitTorrent. Under older file-sharing systems like Napster and Kazaa, only a small subset of users actually share files with the world. Most users simply download, or leech, in cyberspace parlance.

BitTorrent, however, uses what could be called a Golden Rule principle: the faster you upload, the faster you are allowed to download. BitTorrent cuts up files into many little pieces, and as soon as a user has a piece, they immediately start uploading that piece to other users. So almost all of the people who are sharing a given file are simultaneously uploading and downloading pieces of the same file (unless their downloading is complete).

The practical implication is that the BitTorrent system makes it easy to distribute very large files to large numbers of people while placing minimal bandwidth requirements on the original "seeder." That is because everyone who wants the file is sharing with one another, rather than downloading from a central source. A separate file-sharing network known as eDonkey uses a similar system.

Groove

Other systems

A free personal peer-to-peer web, file and application server. It allows you and your associates to access areas of your computer from any location using a standard web browser, just like any other internet web site. MyWebServer is the only tool you'll need to create your own secure virtual community. With this software running you have the capability to share information and files on your computer with your friends and business collaborators.

As computers become ubiquitous, ideas for implementation and use of peer-to-peer computing are developing rapidly and gaining prominence. But issues of interoperability, security, performance, management, privacy and many more issues may stifle the growth of innovation. Many of these issues relate to infrastructure, where no value will be gained from proprietary solutions.

grids

The Globus Alliance is developing fundamental technologies needed to build computational grids. Grids are persistent environments that enable software applications to integrate instruments, displays, computational and information resources that are managed by diverse organizations in widespread locations.

Grids are persistent environments that enable software applications to integrate instruments, displays, computational and information resources that are managed by diverse organizations in widespread locations.

An open source community effort to facilitate the adoption of distributed computing solutions. Sponsored by Sun Microsystems and hosted by CollabNet, the Grid Engine project provides enabling distributed resource management software for wide ranging requirements from compute farms to grid computing.

"I've often been asked, 'What's the next big thing for the Internet?'" said John Patrick, IBM's vice president for Internet strategies. "Until now, I didn't have the answer. I'm very confident now that the next big thing will be grid computing."

the technology infrastructure for Massively Multiplayer Games (MMGs) that connect players on PCs, consoles and mobile devices

A team of physics researchers and computer scientists has carried out a successful simulation on a grid of computers at five universities and research sites, representing the next step in advancing distributed computing.

The project is part of the development of the Compact Muon Solenoid Collaboration, one of many experiments that will be run on the Large Hadron Collider, a massive particle accelerator being built in Switzerland.

When you think of the Internet as being just another way to
print, or another way to broadcast, or another way to run a
store and market to people, you're really limiting your
view of the thing. ...

The Internet is just starting to transform the world we live in, as more and more people think of it in more and more exciting, new, and different ways. So forget all your cares and financial woes. These are the early days, mate. You're in on the ground floor.


trends and currents for exploring webs


Customhouse concepts and buzzwords	Charthouse trends and currents	Boardwalk people and communities
Lighthouse information and research	Shoreline issues and policies	Docks systems and processes


search \| sitemap \| help
Ricci Green \| Digital Wares \| Gizmos, Inc. CyberSea Inn \| Port 80

modified: June 8, 2002 by Douglas Anderson http://RicciStreet.net/port80/charthouse/future/frameworksp2p.htm

Frameworks for the Future:

Peer-to-Peer and Grids

P2P in organizations

Freenet

Groove

Other systems

grids