Usenet (short for "User's Network") is a worldwide, distributed messaging system, used largely for discussion
and exchange of binary data. Users download and post messages to-and-from a server,
which communicates with other servers to propagate those messages
around the world.
Messages are organized, by subject, into 'newsgroups'. Each Usenet server
administrator has complete control over that server: what groups it makes available
to it's users, what groups it propagates, how long messages stay on the system before
'expiring', who can be a user etc. etc. The only clear-cut case of ownership or
control of Usenet are those administrators; they can pretty much do as they please.
Usenet operates much like the Internet-at-large; there is no single governing
body or owner with absolute control, but there are recognized systems and groups who
keep the whole thing almost organized. Much of that organization is done via Usenet in
the 'news' hierarchy; that is where lists of newsgroups are published, new groups created
and announced and site administrators and other volunteers generally agree how things
are going to run. Periodic postings to these groups are a large part of what keeps
Usenet a cohesive system. Documents describing the systems for newsgroup creation,
active groups, what is discussed in certain groups, who is responsible for certain
actions and how to create groups are updated and posted on a regular basis. These
documents form the 'official' view of Usenet only because they are widely propagated
and followed.
A Brief History
In 1979, Jim Ellis and Tom Truscott, graduate students at Duke University, wanted to give
universities the same kind of functionality that the ARPANET had with its mailing lists;
they came up with a simple system to share files between UNIX systems using modem connections.
Steve Bellovin, a University of North Carolina grad student, wrote the first version of the
software as a few Bourne shell scripts, using the UUCP protocol.
The software periodically checked the timestamp of files in a specified directory, and
sent files updated since the last check to the other system. The first two systems on
Usenet were the UNC and Duke Computer Science departments. In 1980, the Duke Medical
Center Department of Physiology was added.
Usenet was featured at the 1980 Usenix conference and the scripts were re-written as freely
distributed C programs. Other Universities were invited to join the network, and as more nodes
joined, the more useful and popular Usenet became.
Maintenance and development of the main software changed hands a few times and was constantly
improved to keep up with increased demands. Since the whole system was open source and freely
distributed, many other people began writing compatible server software and today server and
reader software is available on many platforms including Linux and Windows.
News Propagation Algorithm
This process is described in RFC 850: Standard for Interchange of USENET Messages.
Usenet is generally described by a directed graph. Each node is a host and each link between
hosts describes information flow to or from the designated host. Generally these links are
bi-directional, but not necessarily so.
- A user posts an article to a server on a list of newsgroups.
- The server accepts it locally and then sends that article to it's neighbors. The neighboring
servers will have notified the server as to which groups they are interested in receiving, and
the server will honour those notifications.
- Those neighbors will then receive the article locally and again send
it out to their neighbors, and so on, resulting in the propagation of the message to the entire
network.
Obviously it's important that there's a mechanism here to prevent loops, or else these
articles would be propagated endlessly. Each host keeps track of what message it has seen
according to a unique message ID. If an incoming message bears a previously seen message
ID, the host will immediately drop that message.
RFC 850 also outlines a few additional optimizations (Section 5, page 17). Both optimizations
involve checking the message header to see what hosts have left their mark in the 'path-line'
or 'posting-version' (origination). If a neighbor is discovered to have already seen the
message it will not be forwarded to it.
Usenet's Future
Usenet's discussion forum is a unique thing. It's still in wide use even though there are
many distributed 'chat' and messaging systems that are very popular like IRC or ICQ. One
possibility is that the Usenet discussion community is made up solely of old-timers and
once they die off, so will Usenet. It's also possible that the Usenet forum provides
something that other messaging systems don't. For instance, perceived weaknesses such
as Usenet's lack of 'real-time' discussion add a different dimension to the conversation;
since it's not real-time, statements are usually more thought out. The discussions are not
constrained by time: it allows seamless communication between multiple people in multiple
time zones in the same discussion. That's an application with longevity.
It's a system that has been evolving for 20 years, and has a flexibility limited only by the
imagination of its users. Users have adapted the system for file sharing, and who says that file sharing is the last revolution to be introduced
to the Usenet system?
Timeline
- 1979
-
North Carolina: Duke grad students Tom Truscott and Jim Ellis thought of an idea to hook
computers together to share information with the Unix community.
Steve Bellovin, UNC grad student wrote the first version of Usenet. The shell scripts were
installed on the first two sites: "unc" and "duke".
- 1980
-
Another machine at Duke, "phs" is added to the network.
Usenet was described at the January Usenix conference.
Steve Bellovin rewrote the scripts into C programs; Steve Daniel wrote another implementation
in C for distribution.
Tom Truscott made further modifications, resulting in the "A" news release.
- 1981
-
U.C. Berkeley grad student Mark Horton and high school student Matt Glickman rewrote the
software to add functionality and performance (the original version was only intended for a
few articles per group per day) creating the "B" version.
- 1982
-
B News is publicly released. As the net grew the news software was expanded and modified.
RFC 822: Standard for the Format of ARPA Internet Text Messages, is published.
- 1983
-
RFC 850: Standard for Interchange of Usenet Messages is published
- 1984
-
Rick Adams from the Center for Seismic Studies took over development of the B News software.
Increasing volume of news was becoming a concern, and the mechanism for
moderated groups was added to the software.
Brain Kantor (U.C. San Diego) and Phil Lapsley (Berkeley) develop the Network News
Transfer Protocol (NNTP) between 1984 and 1985.
- 1986
-
A package was released implementing news transmission, posting, and reading using NNTP.
RFC 977: Network News Transfer Protocol, is published.
- 1987
-
C News is released by Geoff Collyer and Henry Spencer from the University of Toronto.
This is a big-time rewrite improving many aspects such as speed and reliability.
RFC 1036: Standard for Interchange of Usenet Messages published.
- 1989
-
RFC 1123: Requirements for Internet Hosts -- Application and Support published.
- 1990
-
RFC 1153: Digest Message Format published.
- 1992
-
InterNetNews (INN), news software written for large UNIX hosts, is released by
Rick Salz (UUNET).
Geoff Collyer (U of T) implemented News Overview (NOV)