\begindata{text,539034464}
\textdsversion{12}
\template{help}
\define{global
}
 

\chapter{AMS:  The Andrew Message System}


This help document provides an overview of some technical aspects of the 
Andrew Message System.  It is not necessary to read this document if you only 
want to use the AMS programs ( \italic{\helptopic{Messages}}  and 
 \italic{\helptopic{SendMessage} }, \italic{\helptopic{VUI} }, and 
\italic{\helptopic{CUI} }).  However, if you are interested in understanding 
more about the underlying components of the Andrew Message System and how they 
work together to provide mail and bulletin board services to Andrew users, you 
should read this document.  It includes sections on


\leftindent{What the Andrew Message System is

Major components of the Andrew Message System

Mail delivery

Message authentication and message captions

Name validation: the White Pages

Message-tree folder structure and maintenance

One and two process versions

Headmagic

Related tools}


Terms that are printed in \bold{bold} are defined or explained in this help 
document.  Terms printed in little grey boxes have additional help information 
available.  


\section{What the Andrew Message System is

}\leftindent{
The Andrew Message System is the system by which mail and bulletin board 
messages are transmitted on Andrew.  The Andrew Message System includes user 
programs (such as Messages and SendMessage) as well as underlying programs and 
subroutines (the message server, the CUI library, the SNAP remote procedure 
call package, and the guardian).  The Andrew Message System also provides a 
White Pages facility that can be used by mail-sending programs for matching 
user names to user ids so that mail can be sent to Andrew users by their names 
as well as by their user ids. 

}
\section{Major components of the Andrew Message System

}\leftindent{
The design of the Andrew Message System is strongly influenced by a desire to 
provide "universal message service" to as wide a range of users and machines 
as possible.  In other words, the goal is for users to be able to read mail 
and/or bboards one day on an Andrew workstation, the next day on an IBM PC, 
the next day on a Macintosh, and the next day on some yet-unknown machine, and 
have the message system behave consistently and properly recognize which 
messages you have and have not seen.


To achieve this goal, the functions involved in message reading and sending 
are divided into two parts:  a user program and a database access program. 
 The database access program is called the Message Server, or \bold{ms}.  The 
message server must run on an Andrew workstation and it manipulates messages 
in a database stored in the Andrew File System.  User programs, such as 
Messages or VUI, communicate with the message server via the 
\italic{\helptopic{CUI}} library of subroutines and a remote procedure call 
mechanism called \bold{SNAP}.


Each user program is free to use whatever methods its designers deem best for 
interacting with the users; there are three different officially-supported 
user programs (\helptopic{Messages}, \helptopic{VUI}, and \helptopic{CUI}) and 
a number of user-supported programs (like Batmail), all of which which 
represent different ideas about how users want to manipulate messages. 
  However, user agent programs all use the same library to perform their 
underlying manipulations of the message database: the \bold{CUI library}. 
  (CUI stands for Common User Interface.)  In other words, regardless of which 
interface you are using, the underlying actions are the same, which means that 
you can switch from one program to another without losing any status 
information (that is, without having a message presented to you as "new" 
twice).


The CUI library translates requests that it receives from the user agent 
program into remote procedure calls to the message server (using SNAP) and 
returns to the user agent program the response it receives from the message 
server.  When an interface program starts, it initializes the CUI library to 
establish a connection to the message server.  The connection to the message 
server is obtained via a process called the \bold{guardian}.  The CUI library 
contacts the guardian on the machine where it hopes to find a message server. 
 (On Andrew workstations, this is the machine to which the user is logged on, 
but the message server can be run on a different workstation than the one you 
are using.)


The guardian performs Andrew File System authentication functions and keeps 
track of existing services it knows about.  (For example, if there is already 
a message server for the user on the machine, the guardian will connect 
clients to that one instead of creating a new one; a message server can handle 
multiple clients, as long as they are all being used by the same user.)  The 
guardian can be configured to refuse requests from other machines, so that you 
can guarantee that no one on another machine can run a message server on 
yours.  (This is in fact the default; server machines require special 
initialization, explained below)  


The only purpose of the guardian in the message system is to establish the 
connection between the CUI library and the message server.  Once that 
connection is made, the CUI library communicates directly with the message 
server (using SNAP calls) to perform the operations requested by the user 
agent program.  (The guardian is also used in a different context to connect a 
PC to the PC server.)


At the very top of the system sit the user interface programs, using the CUI 
library, SNAP, the guardian, and the message server to provide message service 
to users.  

}
\section{Mail delivery

}\leftindent{
User programs deliver mail by two different methods on Andrew: the 
\bold{\helptopic{dropoff}} library routine or the \italic{\helptopic{queuemail}} 
program.  This section describes the actions of the dropoff routine, which is 
easier for programmers to use and has better performance, and is consequently 
used by many of the AMS user programs.  For detailed information on the use of 
queuemail, consult the help file on \italic{queuemail}.


\bold{Outgoing mail}.  Each of the current AMS mail sending programs uses the 
message server to call the mail delivery routine, dropoff.


Dropoff looks for a directory called .Outgoing in the user's home directory. 
 If this directory is non-existent, dropoff will create it (with appropriate 
protection).  Dropoff attempts to write three files into this directory.  The 
names of each of these files are identical except for the first character; 
they begin with SF, QF, or GF.  If the cache manager finishes storing the GF 
file, the last one written, your mail has been safely put into a mail delivery 
queue and will eventually be delivered.  Mail is not considered for delivery 
until the GF file is written.  (Note that this is the same protocol that 
\italic{\helptopic{queuemail}} uses.)


If the files are successfully written to the .Outgoing directory the dropoff 
routine then sends a signal to the local queuemail daemon running on the host 
machine (see the help entry on \italic{\helptopic{queuemail}} for more 
information).  The signal contains the user id of the user and his or her 
current tokens, and tells queuemail to wake up and deliver all mail in 
.Outgoing for the user.  Once this signal is sent successfully, dropoff 
returns indicating success to the caller.  The signal is sent as an insecure 
datagram, which means that there is a possibility that the signal will be lost 
before it can be received by the queuemail daemon.  In many cases loss of the 
datagram would mean that the piece of mail would not be delivered.  To catch 
any mail stranded by loss of its datagram, a daemon runs on one of the 
postoffice machine and looks through all .Outgoing directories for just such 
undelivered mail.  When the daemon finds such a piece of mail, it delivers the 
mail.


If the attempt to write to .Outgoing fails, dropoff uses the same file name 
protocol to attempt to write the message to one of a number of global File 
System queues.


If this second delivery attempt fails (as can only happen because of File 
System problems), dropoff will try to queue your mail on the local machine. 
  It attempts to write the SF, QF, and GF files into a directory on your local 
disk.  Later a daemon on the local machine will try to transfer these files to 
the File System, which should succeed sometime after the File System comes 
back up.  However, this mechanism is not completely secure; anyone with 
control over the workstation can delete items from the local queue. 
 Eventually, we intend to provide mechanisms which let users choose whether to 
allow mail to take this "insecure" path through the delivery system.


\bold{Incoming mail}.  All incoming network mail for Andrew users is delivered 
first to one of the postoffice machines (there are currently four) and then to 
individual mailboxes in the file system by a daemon.

}
\section{Message authentication and message captions

}\leftindent{
The Andrew Message System tries to establish who sent which messages in order 
to protect Andrew users from forged mail.  This process is called message 
authentication.  There are three possible levels of authentication, each of 
which is reflected in the message caption that accompanies the message.


\description{1.  Complete authentication (messages from Andrew users can 
generally be definitively authenticated)


The caption of a fully-authenticated message looks like this:


	24-Sep-87   Egg on my face  Jello Biafra


2.  Partial authentication (messages that arrive from users on other networks 
can be authenticated to the extent that the name of the host network or 
address can be attached to the user name; however we do not guarantee that the 
name of the host is correct, only that the mail originated somewhere other 
than on Andrew)


	25-Sep-87   Breakfast    Kyle Face@g.pg.cs.cmu.edu


3.  Failed authentication (an attempt is made to demonstrate when a piece of 
mail could not be authenticated in case forgery is involved.  Question marks 
are added to the end of the user id if authentication is not accomplished.)


	24-Sep-86   Re: Breakfast  Judith Biafra@???}


When network names are too long to fit, both the name and the host may get 
shortened arbitrarily, but there will always be at least an "@" and a few 
characters of the host name, to indicate that the mail is not of local origin. 




}\section{Name validation: the White Pages

}\leftindent{
The Andrew Message System provides a name-lookup service that can be used by 
user programs to match names supplied by the user (for example, as recipients 
for pieces of mail) with a database of user ids in order to validate that the 
name supplied corresponds to an Andrew user to whom mail can be delivered. 
 The database of user ids and the subroutines for using it are called the 
\bold{White Pages}. 


The presence of a name-lookup service means that mail can be sent to Andrew 
users by their names as well as by their user ids.  If you are sending mail to 
an Andrew user, you can use any name that will identify the person uniquely. 
 Suppose, for example, that there were an Andrew user called Jello Biafra 
whose user id was "jbRo".  The name "Jello Biafra" could be used as a mail 
address because it would identify the Andrew user "jbRo" uniquely and be 
accepted by the White Pages as a valid name.  However, the name "Biafra" might 
not be sufficient to identify "jbRo" because there might be other Andrew users 
whose names are also "Biafra."  


Name validation is used two ways in the Andrew Message System.  First, name 
validation can be done interactively if the user program chooses to offer 
interactive validation.  For example, when you send a message with the 
SendMessage program, SendMessage tries to validate the names given before it 
submits the mail to be delivered.  If one or more of the names typed by the 
user as recipients (in either the To: or CC: field) is ambiguous, it informs 
you that an invalid name exists and waits for you to correct the problem and 
attempt to send the message again.  If SendMessage finds more than one user 
who could match a name that you supplied, it will often present a dialogue box 
from which you can choose the user you intended.  In other words, SendMessage 
attempts to catch and help you correct any names that would cause the mail to 
be undeliverable and result in an "error notice" or "rejection notice" to you 
from the underlying delivery mechanisms.  (You can also check the names 
separately by using the optional \bold{Check Recipients} menu option in 
SendMessage.  You cause this option to appear on your SendMessage menus by 
using the \bold{Set Options} menus in Messages.  The CUI "whois" command can 
also be used to check user names.)


Name validation is also used on mail that comes to Andrew users from remote 
sites, such as on BITNET or the ARPA Internet.  If the validation procedure 
cannot match the name given with an Andrew user id, it will send an error 
notice to the sender of the mail explaining that the name he or she supplied 
is ambiguous or not found.  If there are only a few possible matches for the 
name, the error notice will contain a list of possible recipients and their 
user ids so that the sender can choose which recipient was intended.  


When name validation is done on a user name, the user id that corresponds to 
the user name is supplied with a "+" attached to the end.  In SendMessage, for 
example, the name "Jello Biafra" would be replaced by the address


jbRo+@andrew.cmu.edu


The "+" is used in Andrew mail addresses to signify that the characters 
preceding the "+" have a special meaning.  Usually the characters before the 
"+" are an Andrew user id and the "+" tells the validation procedure that the 
name is a legitimate mail address.  For more information about the uses of the 
"+" in Andrew mail addresses, see the  \italic{\helptopic{ms-plus}}  help 
file.

}
\section{Message-tree folder structure and maintenance

}
\leftindent{Messages, having been delivered to a Mailbox and processed, are 
stored in folders beneath a message-tree root.  Folders are simply directories 
that contain messages, a few files describing those messages, and other 
folders.  A message-tree root is a directory whose name starts with .MESSAGES, 
and contains files describing the folders beneath it.  Treeroots themselves 
contain no messages.


\bold{Adding a message to a folder:}

When a message is filed into a folder, several pieces of information must be 
found and (possibly) updated.  First, the message server decides which folder 
the user is interested in by comparing the folder-name specified (such as 
"mail.work") with names in the .SubscriptionMap files in the treeroots 
specified in the *.mspath preference.  The treeroots' subscription maps are 
searched in the following default order, unless the *.mspath preference is 
changed:  

\description{
$MAIL (the user's own .MESSAGES root), 

$OFFICIAL (the root holding official bboards, ~bb/off/.MESSAGES), 

$OTHER (other Andrew bboards under ~bb/.MESSAGES, network bboards under 
~netbb/.MESSAGES).}


(See the  \italic{\helptopic{preferences}}  help document for more information 
about *.mspath.) Each line of the .SubscriptionMap is a short folder-name 
followed by the full pathname to that folder's directory.  Once the 
folder-name has been resolved, its associated directory is opened for writing 
and the directory's .MS_MsgDir file is read in.  The user must, of course, 
have appropriate rights on the directory to do this.


The .MS_MsgDir file contains a bunch of information about the messages in that 
folder.  It is a binary file, and should not, in general, be edited by hand. 
 Each message has a constant-length entry in the .MS_MsgDir holding (in hashed 
form): the caption, date of insertion into the folder, date sent, message-ID, 
chain link (for "related message" marking in Messages), In-Reply-To 
message-ID, and various other things.  When a new message is added to the 
folder, a new entry is built for the .MS_MsgDir.  The new entry is added 
either to the end of the list of entries ("append" from Messages), or inserted 
into the list chronologically by sending-date ("classify" from Messages).


\bold{Reading a message from a folder:}

Reading the messages in a folder is essentially the same process, except the 
user only needs read and lookup access to the directory.  In order to show 
only those messages appropriate to the kind of subscription the user has to 
the folder, a list of all the folders the user has ever read and the insertion 
date of the last message read in each folder is kept in ~/.AMS.prof .  This 
file also contains a folder-name to directory mapping and the kind of 
subscription the user has to each folder (none, ShowAll, Print, or normal).


\bold{Orphans and Reconstruction:}

A message with no associated .MS_MsgDir entry is called an orphan.  There is 
no way for the message server to read the orphaned message since the message 
has no handle in the .MS_MsgDir.  When this happens, messages will seem to 
disappear out their folder.  Conversely, if adding a message to a folder fails 
midway through, there may be an entry in the .MS_MsgDir for a message that 
never got written into the folder.  This, too, will cause problems.


The usual course of action is "reconstruction" of the .MS_MsgDir file.  The 
wizard-level CUI command \bold{reconstruct} will replace the old .MS_MsgDir 
file with a completely rebuilt one generated by looking at all of the message 
files in the directory.  For more information about reconstruction, see the 
 \italic{\helptopic{cui-wizard}}  help document.


\bold{Epoch-ing:

}From time to time, the bboard maintainers remove old message from a bboard. 
 The CUI wizard-level \bold{epoch} command will remove all messages in a 
folder and all its subfolders that were inserted before a specified date.  For 
more information about \bold{epoch}, see the  \italic{\helptopic{cui-wizard}} 
 help document.

}
\section{One and two-process versions}


\leftindent{The Andrew Message System programs are available in two versions: 
one in which the message server is a separate process (the "s" versions, for 
"server") and one in which the message server is linked into the user 
interface program (the "n" versions, for "no server").  The latter have 
slightly better performance because they use less virtual memory in total.


Currently, the default version (the one you get when you run "messages," 
"sendmessage," or "cui") is the "n" version, without the message server as a 
separate process.


There are times when you might want to run the "s" version; the two process 
version.  To run the two process version, several things need to be done.  


\bold{The machine on which the message server is to be run} needs to have the 
following:


-- a file called /RemoteGuardianRequests.  The file does not need to contain 
anything.


-- a file called /etc/guardian.svc with the following contents:


MS.mmm.nnn AUTH_ONLY TRUE 200 /usr/snap/services/MS.mmm.nnn


(all on one line) for every AMS major (mmm) and minor (nnn) version 
(AMS_MAJOR_VERSION and AMS_MINOR_VERSION in DESTDIR/include/ams/amsvers.h) you 
expect to run.


-- after these files are in place, the machine should be rebooted so guardian 
will start accepting remote requests.


\bold{The person running the interface} needs to do the following:


-- specify the machine on which the message server will run by putting the 
following line in their preferences:


AMS_RemoteServer: machine.name


-- explicitly run the message server as a separate process, by typint the name 
of the program plus an "s":


\bold{\leftindent{messagess}}


If there is any ambiguity about which version you are running as the default, 
you can explicitly use the one-process version by typing the name of the 
program plus an "n":


\leftindent{\bold{messagesn}}


\bold{Security considerations.}  Currently both versions of the programs are 
equally secure. However, the -s versions may become slightly less secure 
(about as secure as butler) in the future. }


\section{Headmagic}

\leftindent{
The Andrew Message System has a feature that allows you to specify that any 
message containing a certain header field will be filtered through a certain 
filter program.  


It works for all AMS interfaces which work on machines that support filter 
processes; in particular, on any machine where the libraries have 
ENABLEFILTERING defined. At present, therefore, this works on all applications 
running on UNIX machines.


For more information, see the  \italic{\helptopic{ams-headmagic} } help file.}


\section{Related tools

}
Select (highlight) one of the italicized names and choose "Show Help on 
Selected Word" from the pop-up menu to see the help document for:


\leftindent{\italic{\helptopic{Andrew}		}(the Tour)

\italic{\helptopic{bboards}

\helptopic{CUI}

\helptopic{flames}}		(filtering language for the AMS)\italic{	          

\helptopic{forward}		}(how to have your Andrew mail forwarded)\italic{

\helptopic{mail}

\helptopic{Messages}}

\italic{\helptopic{ms-aliases}		}(how to do mail aliases on Andrew)

\italic{\helptopic{ms-mailinglists}		}(how to do mailing lists on Andrew)

\italic{\helptopic{ms-plus}		(}the + in local addresses)\italic{

\helptopic{SendMessage}

\helptopic{VUI}

\helptopic{white-pages}

\helptopic{headmagic}}}


\begindata{bp,537558784}
\enddata{bp,537558784}
\view{bpv,537558784,1447,0,0}
Copyright 1992 Carnegie Mellon University and IBM.  All rights reserved.

\smaller{\smaller{$Disclaimer: 

Permission to use, copy, modify, and distribute this software and its 

documentation for any purpose is hereby granted without fee, 

provided that the above copyright notice appear in all copies and that 

both that copyright notice, this permission notice, and the following 

disclaimer appear in supporting documentation, and that the names of 

IBM, Carnegie Mellon University, and other copyright holders, not be 

used in advertising or publicity pertaining to distribution of the software 

without specific, written prior permission.



IBM, CARNEGIE MELLON UNIVERSITY, AND THE OTHER COPYRIGHT HOLDERS 

DISCLAIM ALL WARRANTIES WITH REGARD TO THIS SOFTWARE, INCLUDING 

ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS.  IN NO EVENT 

SHALL IBM, CARNEGIE MELLON UNIVERSITY, OR ANY OTHER COPYRIGHT HOLDER 

BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY 

DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, 

WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS 

ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE 

OF THIS SOFTWARE.

 $

}}\enddata{text,539034464}
