Google
 

Trailing-Edge - PDP-10 Archives - SRI_NIC_PERM_SRC_3_19910112 - stanford/5-swskit/archive.mem
There are 5 other files named archive.mem in the archive. Click here to see a list.
	***** REVISION HISTORY *****

THE FOLLOWING CHANGES HAVE BEEN MADE TO THE FUNCTIONALITY
OF THE ARCHIVING SOFTWARE. NOTE THAT THESE CHANGES TAKE
PRECEDENCE OVER THE CONTENTS OF THIS SPEC.  THAT IS, IF
THEY DISAGREE, THE REVISION HISTORY SHOULD BE USED.

1. THE "ARCHIVE" SUBCOMMAND TO THE DIRECTORY COMMAND WILL
   PRINT OUT THE NAMES OF ALL ARCHIVED FILES, REGARDLESS
   OF WHETHER THEY ARE VISIBLE OR INVISIBLE.

2. THE ^ESET [NO] RETRIEVAL-WAITS COMMAND IS NOT CURRENTLY
   IMPLEMENTED.

	***** END OF REVISION HISTORY *****

DEC archive system spec                                     Draft
Implementation details
DUMPER

          This  section  will  describe  the  tasks  DUMPER  must
perform   to  support  the  Archive/Virtual  Disk  system.   This
includes copying files to tape, removing the contents from  disk,
etc.

          DUMPER  will not contain any of the policy for deciding
when files may be copied to tape and have their contents  removed
from  the  disk.   This  task  is performed by REAPER, the policy
module for the Archive/Virtual Disk system.  Instead, DUMPER will
only dump those files marked  for  voluntary  archival  or  those
files  past  their  expiration  date  or  marked  for involuntary
migration, depending on which function  the  operator  wishes  to
perform.   The process of dumping files to tape will be called an
archival run if voluntary archivals are  being  processed,  or  a
collection  run  if  involuntary  migrations  and expirations are
being processed.  In either case, the operation is  very  similar
to  performing  an  incremental  dump.   The  command  syntax for
performing an archival run is "DUMPER>SAVE/ARCHIVE  PS:<*>*.*.*";
the   syntax   for  an  collection  run  is  "DUMPER>SAVE/COLLECT
PS:<*>*.*.*".  Alternatively, the switch "/MIGRATE" can  be  used
in  place of "/COLLECT" to process files marked for migration and
ignore files which have expired.  Naturally, any filespec can  be
used  in these commands.  If no filespec is given, the default is
to process all files on the connected structure.

          Each archival or collection run consists of two logical
passes.  During the first pass, selected files  are  copied  onto
tape.   When  this  pass is complete, a second "clean up" pass is
made to mark data as valid and delete the disk contents of  those
files  which  have been reliably copied to two tapes.  (If a file
is being archived, and AR%NDL is set, then the disk contents  are
not deleted.) ___

          Now in detail, here is what DUMPER will do during  each
pass of the  archival or  collection run.  Pass  1: Examine  each
file in the specified path.  If  this is an archival run and  the
file is marked for voluntary archival, or if this is a collection
run and the  file is past  its expiration date  (/COLLECT) or  is
marked for involuntary migration (/COLLECT or /MIGRATE), then set
AR%1ST in the  file's FDB  to indicate that  a DUMPER  run is  in
progress.  Then, set  the tape  information in the  FDB via  ARCF
(.ARSST), and  dump the  file to  tape.  This  process  continues
until all files in  the path have been  examined.  At this  point
DUMPER ends the saveset on the tape and rewinds it.  Pass 2,  the
clean up pass, now begins.  Once again the file path is  scanned,
and any file  with AR%1ST  set is considered.   First, AR%1ST  is
cleared, thereby  marking  the  file contents  valid.   The  tape
information is then examined, and if it is complete (the file has
been dumped successfully to two tapes), ARCF (.ARSST, AR%OFL)  is
invoked to delete the disk contents.  If this is an archival run,
FB%INV is set to make the file invisible.  Finally, ARCF (.ARSST,
AR%CRQ) is

                              - 1 -

DEC archive system spec                                     Draft
Implementation details
DUMPER

invoked to clear the request bits.  At the end of each  directory
(GNJFN  notes this by GN%DIR being returned) a message is sent to
the owner listing all files that have been successfully copied to
two tapes.  This process continues  until  all  files  have  been
processed.

          Should  a  file  with  AR%1ST be encountered during the
first pass of DUMPER's  archival  or  collection  run,  the  most
recent  set  of  tape numbers must be cleared.  This is necessary
since AR%1ST indicates that the contents of  the  file  were  not
reliably copied to tape.

          The   retrieval  operation  is  handled  by  DUMPER  as
follows.  Retrieval requests are stored in a queue maintained  by
GALAXY.  When the command "DUMPER>RETRIEVE STR:<*>*.*.*" is used,
DUMPER  first  sends  an  IPCF  message  to GALAXY indicating its
desire to process the retrieval request queue.  It then  receives
retrieval  requests  from  GALAXY  one  by  one.  If a particular
retrieval request falls within the file  specification  given  in
the  command,  DUMPER  attempts  to retrieve that file; otherwise
DUMPER requests that GALAXY requeue the file for retrieval later.
Since the requests are ordered by the tape information, more than
one file can be retrieved from a single tape without rewinding or
dismounting the tape.  Should  the  retrieval  of  a  file  fail,
either because the proper tape could not be mounted or because of
data   errors,  DUMPER  will  request  that  GALAXY  requeue  the
retrieval request, considering  the  first  tape  information  in
place  of  the  more  recent copy.  If DUMPER fails to retrieve a
file from  the  both  tapes,  and  the  retrieval  was  requested
explicitly  by  a  user,  that  user  is notified by mail.  If an
implicitly generated retrieval request  cannot  be  honored,  the
process awaiting the retrieval receives an OPNXxx error.

          For each file that is to be retrieved, a temporary file
is  created.   The  file  contents are restored from tape to this
temporary file, and then the file's FDB is changed to reflect the
file information on tape, except  for  the  archive/virtual  disk
information.   Then  ARCF  (.ARRST)  is  used  to adjust the disk
contents  pointer  of  the  old   FDB,   bring   over   the   non
archive/virtual  disk  information to it from the temporary file,
and delete and expunge the temporary file.

          The  DUMPER  RETRIEVE  command  works  only  from   the
retrieval  requests  in  the GALAXY retrieval queue.  The RESTORE
command, as previously, requires  only  the  file  specification,
along with the appropriate tape already mounted.  When RESTORE is
used,  the  tape  header  record is checked for membership in the
Archive/Virtual Disk system.  If the membership test succeeds,  a
message  is  output  indicating  that RETRIEVE should normally be
used for Archive/Virtual Disk system tapes for reasons of  system
security.    Only   on   confirmation   and   wheel  or  operator


                              - 2 -

DEC archive system spec                                     Draft
Implementation details
DUMPER

capabilities enabled will  the  RESTORE  continue.   The  RESTORE
command  can  thus  retrieve a previously migrated file which has
since been deleted  and  expunged.   If  the  file  as  specified
already  exists  on disk with archive status, the RESTORE command
skips that file.

          Consider   now   the   use   of   tapes   other    than
Archive/Virtual  Disk  system tapes.  When an archived or expired
file is SAVEd, the archive/virtual disk information follows it on
to tape.  On RESTORE,  no  archive/virtual  disk  information  is
brought  back  to  disk  in  the  absence  of  wheel  or operator
capabilities.  In the case  of  wheel  or  operator  capabilities
enabled,   a   simple   RESTORE   command  does  bring  back  the
archive/virtual disk information.  This command  should  be  used
for (local) system tapes only.  An operator handling a non-system
tape  (typically a user tape brought in from another site) should
use RESTORE with  the  NOARCHIVE  switch,  which  suppresses  the
transfer  of archive/virtual disk information and thus avoids the
importation of bogus information into the  backup  words  of  the
FDB,  and  more  importantly,  guards  against attempts to create
counterfeit tape pointers.
































                              - 3 -

DEC Archive/virtual disk system                             Draft
Introduction

          System managers are recognizing the  growing  need  for
reliable   and   convenient   offline   storage   of  data.   Its
availability for general use allows valuable disk space now  tied
up  in  inactive  storage  to be freed for active use.  Users are
provided an equally-handy alternative to file  deletion  to  meet
directory  allocation requirements.  At the same time, users need
a permanent repository for  important  files  and  files  not  in
current  use.   A  reasonable model to use for this purpose is an
archive. Items entered into an archive never  change,  and  never
leave; only copies of items can be removed.












































                              - 1 -

DEC Archive/virtual disk system                             Draft
User overview

          The  offline  storage  system  is presently implemented
using a collection of magnetic tapes each holding many individual
files.  This collection of tapes is divided  into  two  sections:
archive tapes in permanent storage (say 10 years) and non-archive
tapes in storage for a shorter period, say 6 months.

Archived files

          The  archives  can be viewed as unchanging data that at
one time existed as disk files on some system. Each user  has  in
their directory pointers into the archives for any data they have
placed  in  the  archives.   These pointers can be manipulated as
files, which for practical purposes, they are. When a file  gains
archive status(1) it can no longer be modified. This is necessary
to assure one and only one version of the data exists. Data of an
archived file exists in the archives and may simultaneously exist
on  disk.  These  copies  will  always  be  identical.  A file is
archived only by the explicit action (the ARCHIVE command) on the
part of the user.

Ordinary files in Offline Storage

          The non-archive section  of  offline  storage  provides
shorter- term storage of files with contents no longer desired on
disk,  but  not  considered  so  valuable as to require permanent
storage.  This decision may be made explicitly by  an  individual
user   managing   his   directory   within  the  disk  allocation
requirement, or implicitly by the user as files pass their online
expiration dates (which are user- settable, up to a point), or by
a system policy program used in the case of  insufficient  system
disk  resources.   Such  files have no special status and will be
treated as closely as possible to the way disk  files  have  been
treated  in  the past.  To differentiate them from archive files,
and point up their lack of special properties, we will  refer  to
non-archived  files  as  ordinary files.  Using this terminology,
offline storage is divided into storage  of  ordinary  files  and
archived files.

Tape Recycling

          The  crucial  difference  between  archived  files  and
ordinary files in offline storage is their allowed  lifetimes  on
tape.   The  maximum such lifetime is limited by the tape recycle
time.  The archive and non-archive recycle times are set  by  the
system  manager  at each installation with an eye to the tape and
tape storage resources.  Since files are never marked to indicate
a lifetime on tape beyond the tape recycle time in  duration,  an

_________________________________________________________________
(1) A file is considered to have archive status at  the  time  of
the ARCHIVE command. For a file to be completely "archived" there
must be two copies of the file in the archives.


                              - 2 -

DEC Archive/virtual disk system                             Draft
User overview

individual  tape  may  be  recycled  as soon as its (most recent)
write date lies in the past by the amount  of  the  tape  recycle
time.


Concept of a file with contents online and/or offline

          A  file is considered to be a structure consisting of a
single FDB,  its  associated  name  string,  and  its  associated
contents.  Renditions of the file contents may reside on disk, in
offline storage, or both.  To maintain the essential identity  of
the  contents  of  each file,the disk contents of online archived
files are unalterable;  such disk contents are  reference  copies
of  archived  files.   If a writable copy of the same contents is
desired, it is easily obtained through use of the  COPY  command.
In  the case of ordinary files with offline contents, the offline
contents are considered subsidiary  to  the  disk  contents.  The
essential  identity  of contents is maintained by simply clearing
the tape information from the FDB  when  the  disk  contents  are
altered.   Thus just after a successful alteration of contents, a
file is certain to be a disk-only file.

Offline storage

          When entered into offline  storage,  files  (FDB,  file
specification,   and  contents)  are  copied  onto  two different
magnetic tapes, which  are  stored  in  the  installation's  tape
vault.   If  the  installation consists of more than one computer
system, the tape storage may still be made common to all  of  its
systems,  and it is possible to put files in offline storage from
one system, have the directory moved to another system, and  then
retrieve the files in the ordinary manner onto the second system.

Pointers to offline storage

          A pointer into offline storage consists of the tape ID,
saveset  number,  and  tape  file number for each of the two tape
copies  of  the  file  contents.   The  pointer  holds  all   the
information  necessary  to retrieve a single file.  It ordinarily
resides in the file's FDB, where it  is  protected  in  the  same
manner   as  other  essential  file  information,  including  the
analogous disk contents pointer.

Delete protection of archived files

          Archived  files  are  protected   against   casual   or
accidental  deletion by the requirement, for successful deletion,
of the use  of  an  archive-specific  subcommand  to  the  DELETE
command.  Deletion may be undone by an ordinary UNDELETE command.
Further, it will be impossible to overwrite an archived file with
a  RENAME  or  COPY  command.   The  archive  pointer  itself  is
preserved past file expunge in a message sent the user.  The user


                              - 3 -

DEC Archive/virtual disk system                             Draft
User overview

may maintain a file (itself archivable) of such messages for dead
storage of archived files.

          Ordinary  files  have  no  such delete protection.  Any
case in which  disk-only  files  lose  their  contents  (rewrite,
destructive rename, simple delete followed by expunge) extends to
all ordinary files.  If an ordinary file is accidentally deleted,
a  backup  operation should be initiated just as in the case of a
disk-only file.

Messages sent to Users

          Messages  are  sent  to  users  when  files  are  taken
offline, archived or not, when retrieve operations are completed,
and when archived files are expunged.  An individual site may use
DEC mail, or no mail at all.  Assume here that mail
is used.  Retrieval messages go to the user who made the request.
For the other messages, the directory in which the file exists is
examined  as  follows.   If  the  file DIRECTORY.OWNER exists, it
contains the name of the user for such mail.  If  DIRECTORY.OWNER
does not exist, the messages go to MAIL.TXT in the directory.

Visible and Invisible Files

          A  well-established  user typically has files, archived
or not, online or offline, which are not in current use and  tend
to  clutter  the  directory.   To  solve  the  problem of clutter
without complicating the system for the casual  or  inexperienced
user,  files  have been given a visible/invisible option which is
changed only in conjunction with voluntary offline  archival  and
retrieval or explicitly by the user.

          A  visible  file  is one which shows up on a simple (no
subcommands) directory listing and is accessible to programs  and
EXEC  commands  dealing  with  files.  In some cases, such access
immediately brings about an error  return,  for  example,  in  an
attempted copy of an visible but offline file.  An invisible file
does  not  show  up  on  a  simple  directory  listing and is not
accessible to  programs  and  EXEC  commands,  except  for  those
programs  and commands which specifically provide for them (as is
the case with deleted  files.)   Like  deleted  files,  invisible
files are not totally transparent:  they affect the determination
of  default  generation numbers by virtue of their ownership of a
certain file specification.

          A visible file may be made invisible as follows:

@SET FILE INVISIBLE (FILES) PROG.MAC
 PROG.MAC.1 [OK]



                              - 4 -

DEC Archive/virtual disk system                             Draft
User overview

Similarly, an invisible file may be made visible:

@SET FILE VISIBLE (FILES) TST.FOR
 TST.FOR.1 [OK]

          Any file, archived  or  not,  may  be  set  visible  or
invisible.   In this sense, the visibility feature is independent
of the archive system.  The default setting, however, depends  on
the archive history of the file, as follows.  A file is (default)
visible  unless  it  is archived and offline, in which case it is
(default) invisible.  Ordinary files maintain their visibility or
invisibility through online-offline transitions.










































                              - 5 -

DEC Archive/virtual disk system                             Draft
EXEC Modifications

EXEC Commands Controlling File Content Location

          Three  commands  handle  the  explicit   requests   for
transfer  of  file  contents  to  and  from offline storage.  The
ARCHIVE command requests the contents be entered in  the  archive
section  of  offline  storage.  The SET FILE EXPIRED command sets
the online-expiration date of the file to the  present  date  and
thus  in  effect  requests  the contents be placed in non-archive
section of offline storage.  The RETRIEVE  command  requests  the
contents  be  restored  to disk from whichever section of offline
storage they reside in.

          For each of the following  examples,  assume  that  the
connected    directory    <CALVIN>    has    just   five   files,
ARCHIVED.OFFLINE.1,    ARCHIVED.ONLINE.1,     ORDINARY.DISK-ONLY,
ORDINARY.OFFLINE, and ORDINARY.BOTH.  ARCHIVED.OFFLINE is offline
archived,  that is, its contents are on tape but not on disk.  It
is  invisible,  the   default   for   offline   archived   files.
Consequently  it  will  not  be accessible to EXEC commands other
than  SET  FILE  VISIBLE,  the   invisible-file   subcommand   to
DIRECTORY,  and  RETRIEVE.   ARCHIVED.ONLINE  is  archived,  with
contents on disk as well as tape.  Again  following  the  default
conditions,  this  file  is  visible.   ORDINARY.DISK-ONLY  is an
ordinary disk file with no pointer to offline  storage,  i.e.  an
old-fashioned  disk  file.   ORDINARY.OFFLINE is an ordinary file
with contents in offline storage but not on disk.  It is visible,
the default case here since visibility  status  does  not  change
with  transfer  of contents to and from the ordinary file section
of offline storage.  Similarly, ORDINARY.BOTH, an  ordinary  file
with contents on both disk and in offline storage, is visible for
these examples.

The ARCHIVE Command

          The  ARCHIVE  command  marks  the  specified  files for
archival by turning on the archive-request bit in the file's FDB.

@ARCHIVE (FILES) ORDINARY.*,ARCHIVED.*
 ORDINARY.BOTH.1 [Requested]
 ORDINARY.OFFLINE.1 %File is not online
 ORDINARY.DISK-ONLY.1 [Requested]
 ARCHIVED.ONLINE %File already archived

          Note the invisibility of  ARCHIVED.OFFLINE.   Here  the
disk  contents  of ORDINARY.DISK-ONLY.1 become unalterable at the
time of the ARCHIVE command, and are expunged from  disk  at  the
end of the archival process.  The resulting offline archived file
is   then   set   invisible.    The   same   treatment  is  given
ORDINARY.BOTH, with the pointer  into  the  ordinary  section  of
offline  storage  being  replaced  by  a pointer into the archive
section.



                              - 6 -

DEC Archive/virtual disk system                             Draft
EXEC Modifications

@ARCHIVE (FILES) ORDINARY.DISK-ONLY,
@@RETAIN (DISK CONTENTS)
@@
 ORDINARY.DISK-ONLY.1 [Requested]

          In this case, the disk contents become  unalterable  at
the  time  of  the archive request, and remain so past the end of
the archival process.  The resulting online archived file retains
its previous visibility or invisibility.

          The archival request may be changed  from  the  default
disk-expunge  to  retain,  or  vice  versa, by simply redoing the
command in the preferred fashion.  The request for  archival  may
be cancelled:

@CANCEL ARCHIVE-REQUESTS (FOR FILES) ORDINARY.DISK-ONLY

The DISCARD Command

          The  DISCARD  command  is  used  to throw away the tape
information associated with a file. When this  action  is  taken,
the  tape  information is deleted from the FDB of the file. Files
must have the contents of the file online for DISCARD to succeed.

@DISCARD (TAPE INFORMATION OF FILES) ARCHIVED.*,ORDINARY.*
 ARCHIVED.ONLINE.1 [OK]
 ORDINARY.BOTH.1 [OK]
 ORDINARY.DISK-ONLY.1 %File has no tape pointer
 ORDINARY.OFFLINE.1 %File not online

The SET FILE ONLINE-EXPIRATION Command

          The SET  FILE  ONLINE-EXPIRATION  command  is  used  to
specify  when a file's disk contents expire. When this expiration
date/interval is reached, the file's contents are  moved  offline
and  remain  there  until the offline expiration date/interval is
reached.

@SET FILE ONLINE-EXPIRATION (OF FILES) ORDINARY.DISK-ONLY (TO) 60
 ORDINARY.DISK-ONLY.1 [OK]

@SET FILE ONLINE-EXPIRATION (OF FILES) ORDINARY.* (TO) 18-JUL-78
 ORDINARY.BOTH.1 [OK]
 ORDINARY.DISK-ONLY.1 [OK]
 ORDINARY.OFFLINE.1 %File is offline

          In the first example, ORDINARY.DISK-ONLY will have  its
disk  contents expire 60 days after any reference to the file has
occurred. In the second example, the two files that  were  "[OK]"
will   have   the   disk  contents  moved  offline  on  or  after
18-Jul-1978, depending when the system  personnel  run  the  tape
program.


                              - 7 -

DEC Archive/virtual disk system                             Draft
EXEC Modifications

The SET FILE OFFLINE-EXPIRATION Command

          The  SET FILE OFFLINE-EXPIRATION command specifies when
the contents of an offline file  can  be  expunged  from  offline
storage.  This  parameter will either specify an explicit date or
an interval of days after the file was moved to offline  storage.
In  either  case,  the  specified value may not exceed the system
tape recycle time.

@SET FILE OFFLINE-EXPIRATION (OF FILES) ORDINARY.OFFLINE (TO) 60

@SET FILE OFFLINE-EXPIRATION (OF FILES) ORDINARY.BOTH (TO) 13-SEP-78

          The first example will  have  the  file  expunged  from
offline  storage  60  days after it was moved to offline storage.
The second example will have the file expunged on 13-SEP-78.

          To avoid losing a  file  and  its  contents,  the  file
contents  may be retrieved from offline storage and then the tape
information discarded by way of the DISCARD command.

The SET FILE EXPIRED Command

          The SET FILE EXPIRED command causes the specified files
to become online-expired. This is  accomplished  by  setting  the
ONLINE-EXPIRATION date to the present date. These files will then
(at a somewhat later time) be moved to offline storage.

@SET FILE EXPIRED *.*
 ARCHIVED.ONLINE.1 %File is archived
 ORDINARY.BOTH.1 [OK]
 ORDINARY.DISK-ONLY [OK]
 ORDINARY.OFFLINE.1 %File is offline

          Here ARCHIVED.OFFLINE is completely passed over because
it  is  invisible.   If  it  were  made  visible  and the command
reissued, it would result  in  the  message  "File  is  offline".
ORDINARY.DISK-ONLY  is  marked  for transfer of contents to tape.
Unlike the  archive-requested  file,  ORDINARY.DISK-ONLY  can  be
modified at any time while it continues to have contents on disk.
At   any   such  modification,  it  immediately  loses  any  tape
information (then obsolete, since tape  contents  are  considered
subsidiary  to  disk contents).  The request for transfer offline
still remains in effect unless cancelled.  When the file contents
have been copied onto two tapes, the disk contents  are  deleted.
The resulting offline ordinary file remains visible (since it was
visible  before)  and  gains  the  ;OFFLINE  attribute to mark it
offline.  Once it is offline, attempts to read or  append  to  it
fail,  unless SET RETRIEVAL WAIT is set (see below), but attempts
to rewrite it succeed, again with the concurrent loss of  offline
contents.



                              - 8 -

DEC Archive/virtual disk system                             Draft
EXEC Modifications

The SET DIRECTORY ONLINE-EXPIRATION-DEFAULT Command

          The  SET DIRECTORY ONLINE-EXPIRATION-DEFAULT command is
used to specify the online expiration date/interval  to  use  for
new files created in the directory. This default may be specified
as an explicit date or as an interval in days.

@SET DIRECTORY ONLINE-EXPIRATION-DEFAULT (OF DIRECTORY) PS:<EONEIL> (TO) 90

Or

@SET DIRECTORY ONLINE-EXPIRATION-DEFAULT (OF DIRECTORY) PS:<CRDAVIS> (TO) 22-SEP-78

The SET DIRECTORY OFFLINE-EXPIRATION-DEFAULT Command

          The SET DIRECTORY OFFLINE-EXPIRATION-DEFAULT command is
used  to  specify  the  default  date/interval  at which files in
offline storage are expunged from offline storage.  This  default
may  be  specified as an explicit date or as an interval in days.
The interval is the number of days since since the file was moved
to offline storage.  In either case its setting is required to be
consistent with the system-wide non-archive tape recycle time.

@SET DIRECTORY OFFLINE-EXPIRATION-DEFAULT (OF DIRECTORY) PS:<EONEIL> (TO) 60

  Or

@SET DIRECTORY OFFLINE-EXPIRATION-DEFAULT (OF DIRECTORY) PS:<CRDAVIS> (TO) 28-SEP-78

          The offline expiration date of a file is set when it is
moved to offline storage.  The default  expiration  date/interval
is  used  when  a  file  is  moved  to offline storage to set its
offline-expiration date.

RESIST and PROHIBIT Settings for a File

          A user may mark certain online files for  consideration
last by REAPER, for involuntary migration (including the deletion
of  disk-contents  of  files  with  contents  on tape as well) by
setting the RESIST bit.

@SET FILE RESIST-MIGRATION ARCHIVED.ONLINE, ORDINARY.DISK-ONLY
 ARCHIVED.ONLINE.1 [OK]
 ORDINARY.DISK-ONLY.1 [OK]

          The RESIST setting can be undone as follows.

@SET FILE NO RESIST-MIGRATION ORDINARY.ONLINE
 ORDINARY.ONLINE.1 [OK]

          A file may be absolutely  protected  against  migration
from  disk  by  marking  it PROHIBIT.  This may be done only by a


                              - 9 -

DEC Archive/virtual disk system                             Draft
EXEC Modifications

wheel or operator with the command SET  FILE  PROHIBIT-MIGRATION.
It may be undone with the command SET FILE NO PROHIBIT-MIGRATION.

The RETRIEVE Command

          The  RETRIEVE  command sends a message to the system to
request  the  restoration  of  the  file  contents  from  offline
storage, archive or ordinary, to disk.

@RETRIEVE (ARCHIVED FILES) ORDINARY.*,ARCHIVED.*
 ORDINARY.BOTH.1 %File not offline
 ORDINARY.DISK-ONLY.1 %File not in offline storage
 ORDINARY.OFFLINE.1 [Requested]
 ARCHIVED.OFFLINE.1 [Requested]
 ARCHIVED.ONLINE.1 %File not offline

          Note  that  RETRIEVE  is  one  of the very few commands
which acts on invisible files such  as  ARCHIVED.OFFLINE  in  the
above  example.   This  property allows users to retrieve offline
invisible files, archived or not, with a single  command.   After
retrieval, the disk contents of an archived file are unalterable.
The  disk  contents  of  an ordinary file may be changed at will,
with the concurrent loss of the pointer to the obsoleted contents
in offline storage.

          The retrieval requests  not  yet  carried  out  can  be
cancelled by the CANCEL RETRIEVAL-REQUESTS command.

New Aspects of EXEC Informational Commands

          The  DIRECTORY and INFORMATION commands provide natural
vehicles for offline storage information.   Since  the  directory
listing  sets  the  stage  for  the user's concept of his working
area, it is important to  implement  DIRECTORY  in  a  clear  cut
manner  which  reflects  the  realities  of the situation.  Since
retrieval of a file  does  involve  delay,  an  offline  file  is
necessarily  considered  one  significant  step  away from online
capabilities.  Correspondingly, offline files, archived  or  not,
are  conspicuously  marked  by  the  attribute  ;OFF-LINE  on all
DIRECTORY listings.  Users can make  files  invisible  to  remove
them  from  a  default  DIRECTORY  listing  as well as from their
routine working environment, since very few programs  can  access
them.   Offline  files  are  particularly  suited to invisibility
since they are a retrieval operation away from routine  use.   By
listing all visible files, with the offline visible files clearly
marked,  with a simple DIRECTORY command, a user sees his routine
working environment at a glance.

          Consider  the  directory  <CALVIN>  again.   A   simple
DIRECTORY lists all the visible files:




                             - 10 -

DEC Archive/virtual disk system                             Draft
EXEC Modifications

@DIRECTORY

   <CALVIN>
 ARCHIVED.ONLINE.1
 ORDINARY.BOTH.1
 ORDINARY.DISK-ONLY.1
 ORDINARY.OFFLINE.1;OFF-LINE

          The  DIRECTORY subcommand to examine just the invisible
files corresponds operationally to the subcommand to examine just
the deleted files.  That is,  with  the  INVISIBLE  (FILES  ONLY)
subcommand, only the invisible files, as otherwise specified, are
listed.   If  this subcommand is omitted, only the visible files,
as otherwise specified, are listed.

@DIRECTORY,
@@INVISIBLE (FILES ONLY)
@@

   <CALVIN>
 ARCHIVED.OFFLINE.1;OFF-LINE

          The DIRECTORY subcommand particular to  archived  files
is  merely  selective:   without  it, files are displayed without
regard to their archival  status,  and  with  it,  only  archived
files, as otherwise specified, are displayed.  A complete archive
directory is obtained by listing the (visible) archived files and
then the invisible archived files.

@DIRECTORY,
@@ARCHIVED (FILES ONLY)
@@

   <CALVIN>
 ARCHIVED.ONLINE.1


@DIRECTORY,
@@ARCHIVED (FILES ONLY)
@@INVISIBLE (FILES ONLY)
@@

   <CALVIN>
 ARCHIVED.OFFLINE.1;OFF-LINE

          Offline  storage  information is accessible through use
of DIRECTORY subcommands:

@DIR,
@@TIMES (AND DATES OF) TAPE-WRITE
@@



                             - 11 -

DEC Archive/virtual disk system                             Draft
EXEC Modifications

   <CALVIN>
                    Tape Write
ARCHIVED.ONLINE.1    7-Nov-77 15:36
ORDINARY.BOTH.1     15-Jul-78 12:05
ORDINARY.DISK-ONLY.1    Never
ORDINARY.OFFLINE.1;OFF-LINE 10-Nov-77 18:22


@DIR,
@@DATES (OF) OFFLINE-EXPIRATION
@@DATES (OF) ONLINE-EXPIRATION
@@

        <CALVIN>
                         Online expiration   Offline expiration
 ARCHIVED.ONLINE.1       13-Aug-78           5-Jul-88
 ORDINARY.BOTH.1         4-Sep-78            14-Oct-78
 ORDINARY.DISK-ONLY.1    30-Aug-78           N/A
 ORDINARY.OFFLINE.1;OFF-LINE  N/A            28-Sep-78

          The current archival status of files may be checked  by
the  INFORMATION command.  Suppose for the following example that
an   archive    request    has    recently    been    made    for
ORDINARY.DISK-ONLY.1.

@INFORMATION (ON) ARCHIVE-STATUS (OF FILES) *.*
ARCHIVED.ONLINE.1     Archived
ORDINARY.BOTH.1       None
ORDINARY.DISK-ONLY.1  Archive requested
ORDINARY.OFFLINE.1;OFF-LINE None

ARCHIVE Subcommand to the DELETE Command

          As   noted   before,   deletion   of   archived   files
intentionally involves some extra effort.  The following  example
shows  the failure of a simple DELETE command applied to archived
files, and its success on ordinary files with or without contents
in offline storage.


@DELETE ARCHIVED.*,ORDINARY.*
 ARCHIVED.ONLINE.1 %File has archive status
 ORDINARY.BOTH.1 [OK]
 ORDINARY.DISK-ONLY.1 [OK]
 ORDINARY.OFFLINE.1 [OK]

          On EXPUNGE, the three files ORDINARY.* will  completely
leave  the  system, in the sense that no FDB will be left on disk
to point to the copy of the contents in offline storage.

          To succeed in deleting archived files,  the  user  must
supply the archive-specific subcommand.


                             - 12 -

DEC Archive/virtual disk system                             Draft
EXEC Modifications

@DELETE ORDINARY.*,ARCHIVED.*,
@@ARCHIVED (FILES INCLUDED)
@@
 ORDINARY.BOTH.1 [OK]
 ORDINARY.DISK-ONLY.1 [OK]
 ORDINARY.OFFLINE.1 [OK]
 ARCHIVED.ONLINE.1 [OK]


@EXPUNGE
 [21 pages freed]

          To  delete  invisible  files,  first make them visible,
then delete them, using the ARCHIVED (FILES INCLUDED)  subcommand
if appropriate.

@SET FILE VISIBLE ARCHIVED.OFFLINE
 ARCHIVED.OFFLINE.1 [OK]

@DELETE ARCHIVED.*,
@@ARCHIVED (FILES INCLUDED)
@@
 ARCHIVED.OFFLINE.1 [OK]
 ARCHIVED.ONLINE.1 [OK]

@EXPUNGE
 [21 pages freed]

          Naturally,  UNDELETE  may be used before the expunge if
desired.  When archived files are expunged, a message is sent  to
the user giving the archive pointer and related file information.
CONTENTS (ONLY) Subcommand to DELETE

          The  DELETE  command  is  provided  a new subcommand to
selectively expunge the disk contents of a file with a valid tape
pointer.  UNDELETE will not work to recover the disk contents but
RETRIEVE is available with some delay.

@DELETE *.*,
@@CONTENTS (ONLY)
@@
 ARCHIVED.ONLINE.1 [OK]
 ORDINARY.BOTH.1 [OK]
 ORDINARY.DISK-ONLY.1 %File has no tape pointer
 ORDINARY.OFFLINE.1 %File is offline

The SET RETRIEVAL-WAIT Command

          Normally, an attempt to read  an  offline  file  simply
fails,  with  an  appropriate message.  Under some circumstances,
such as batch files, it is preferable for the program to wait for
the retrieval rather than to fail because of the lack of a  file.


                             - 13 -

DEC Archive/virtual disk system                             Draft
EXEC Modifications

This  mode  of operation can be set for a job using the following
command:

@SET RETRIEVAL-WAIT (For off-line files)

          Note that invisible files are still inaccessible.   The
mode of operation for the job returns to normal with the command:

@SET NO RETRIEVAL-WAIT (For off-line files)

          Also  note  that  retrieval-waits may be suppressed for
the whole system under various conditions as  determined  by  the
system manager.

Copying and Renaming Files

          There  are  several  aspects  of  the  COPY  and RENAME
commands which involve offline  ordinary  files  and  online  and
offline  archived  files.  A file must have online contents to be
the source file for a successful COPY command. If,  in  addition,
it  has a tape pointer, the tape pointer will not be carried over
to the target file, but will remain solely with the source  file.
This  treatment avoids needless duplication of tape pointers.  By
the same token, the COPY command may be used to make  a  writable
copy of an archived file whenever it is needed.

          Consider  the following unsuccessful attempt to copy an
offline archived file.

@COPY ARCHIVED.OFFLINE.1
 ?File not found

          Here the invisibility of the file prevented  access  to
it.  The file may be made visible, but the resulting access leads
to an error message, since the file is offline:

@SET FILE VISIBLE ARCHIVED.OFFLINE
 ARCHIVED.OFFLINE.1 [OK]

@COPY ARCHIVED.OFFLINE.1 (TO FILE) MUMBLE..1
 ARCHIVED.OFFLINE.1 => MUMBLE..1 ?Source file is offline

          An online archived file is successfully copied:

@COPY ARCHIVED.ONLINE (TO FILE) MUMBLE..1
 ARCHIVED.ONLINE.1 => MUMBLE..1 [OK]

          In  this  case, MUMBLE..1 is created with the same disk
contents as ARCHIVED.ONLINE but without archive status,  so  that
write  and append are allowed according to the protection status.




                             - 14 -

DEC Archive/virtual disk system                             Draft
EXEC Modifications

          Any visible file, archived or not, online  or  offline,
may  be  the source file for a RENAME command.  As with disk-only
files, the RENAME command changes only the name string associated
with the source file, and never  involves  the  contents  of  the
file.

          The  delete protection provided only for archived files
affects COPY  and  RENAME  as  follows.   Attempts  to  overwrite
archived  files,  visible  or invisible, with COPY or RENAME must
fail, as the following examples show:

@COPY tty: (TO FILE) ARCHIVED.OFFLINE.1
 TTY: => ARCHIVED.OFFLINE.1 %Target file has archive status

@RENAME DISKFILE..1 (TO BE) ARCHIVED.ONLINE.1
 DISKFILE..1 => ARCHIVED.ONLINE.1 %Target file has archive status

          Similarly, an archived file may not be changed  through
use  of  the  APPEND command, but, if online, its contents may be
appended to another file.

Other command changes

          The BUILD command has  been  modified  to  include  the
OFFLINE-EXPIRATION-DEFAULT      and     ONLINE-EXPIRATION-DEFAULT
parameters. Both of these take either  an  explicit  date  or  an
interval of days.

          ^ECREATE      has      the      addition     of     the
OFFLINE-EXPIRATION-DEFAULT,  ONLINE-EXPIRATION-DEFAULT,  and  the
ARCHIVE-ONLINE-EXPIRED-FILES  commands.   The  first two commands
take either an explicit date or an interval of  days,  the  third
merely sets/clears a mode bit.

          Finally  the ^ESET [NO] RETRIEVAL-WAITS command is used
by  the  operator  to  specify  that  retrievals  are  not  being
processed.  This  is  used  to  cause OPENF's to fail rather than
waiting for the file to be restored to disk.  Explicit  retrieval
requests are still queued in the duration of this command.















                             - 15 -

DEC archive system spec                                     Draft
Implementation details
GALAXY modifications

          Two major pieces of  the  Archive/Virtual  Disk  system
will  be implemented within GALAXY.  The first and most important
of these is the addition of a queue to  hold  retrieval  requests
for  archived  and expired files.  The second modification is the
addition of an internal queue and supporting  code  to  implement
user  notification  by  mail  of  archived  files  that have been
expunged from the disk.

          The  retrieval  request  queue  maintained  by   GALAXY
contains all outstanding retrieval requests for the system.  When
the  retrieval of an offline file is requested via the ARCF JSYS,
the retrieval information is sent to GALAXY in an  IPCF  message,
and the request is entered into the queue.  The retrieval request
queue  is  ordered  by  tape information (volume ID, tape saveset
number, and tape file number), and each new retrieval request  is
inserted   in  the  proper  sorted  position  within  the  queue.
Ordering the retrieval requests in this way  potentially  reduces
the  amount  of  tape  mounting  and  dismounting  that  must  be
performed  by  the  operator  when  the  retrieval  requests  are
eventually  processed.   Other operations on the retrieval queue,
such as listing and cancelling  retrieval  requests,  are  easily
implemented within the framework provided by GALAXY.

          When  a  retrieval  request  is  initially  queued as a
result of the ARCF JSYS, it is inserted into the queue  according
to  the  second  set  of tape numbers associated with the offline
file.  When the retrieval queue is processed by DUMPER's RETRIEVE
command, requests can be requeued for two  reasons:   either  the
request  does  not fall within the path specified by the operator
in the RETRIEVE command, or the retrieval request  could  not  be
honored  for  some  reason.   In  the  first case, the request is
requeued according to the same set of tape numbers as before,  so
that the request can be processed by some later invocation of the
DUMPER  RETRIEVE  command.   In  the  latter case, the request is
requeued according to the first  set  of  tape  numbers,  on  the
assumption that the more recent tape had data errors or could not
be  mounted.   Requests  that  are  requeued  in  this manner are
processed immediately; that is, by the current invocation of  the
DUMPER RETRIEVE command.

          To  implement  this  protocol,  GALAXY  makes  repeated
passes over the retrieval queue, passing to DUMPER  each  request
that  needs  "consideration."   Those  requests  that DUMPER will
consider later are requeued with a time  stamp  which  identifies
the   current   invocation  of  DUMPER's  RETRIEVE  command.   On
subsequent passes through the queue, GALAXY skips  over  requests
that  were  time  stamped during the current retrieval operation.
Requests that are requeued because of  failure  have  their  time
stamp  field  zeroed,  so that GALAXY will reconsider the request
during the  same  retrieval  operation.   When  a  complete  pass
through  the  queue  results in no new requests to be considered,
GALAXY informs DUMPER that the queue has been exhausted.

                              - 1 -

DEC archive system spec                                     Draft
Implementation details
GALAXY modifications

          The  deletion  protection  special  to  archived  files
includes  notifying users by mail whenever valid tape information
is discarded  from  the  archive  system.   In  this  case,  mail
notification  provides  a  "paper  copy"  of the tape information
which can later be used to restore previously archived  files  to
disk.   Whenever  an  archived  file  is expunged, or its archive
status is discarded, an IPCF message is sent to GALAXY,  and  the
tape pointers are stored in the notification queue.  Periodically
(perhaps once per day) GALAXY processes the notification queue by
grouping  the  entries  according  to  the  owner of the expunged
files, and sending one message to each distinct owner giving  the
name  and  tape  information  for  each expunged file.  Since the
notification queue is internal to GALAXY,  users  may  not  list,
modify, or cancel requests in this queue.

          During  the initialization that GALAXY performs after a
system  reload,  those  retrieval  requests   which   were   made
"implicitly"  by  opening  an  offline  file are cancelled.  This
prevents potentially unnecessary transfers from offline to online
storage.

































                              - 2 -

DEC archive/virtual disk system                             DRAFT
FDB modifications

Note that  all  mentioned  bits  and  offsets  in  the  following
descriptions  are those used in our 3A development system and may
be modified during the integration into release 4 at DEC.

                            FBCTL (1)

FB%ARC (1B11) - File has archive status. Appropriate words in the
FDB (see below) specify where the file is archived. This  bit  is
modified by the ARCF JSYS, CHFDB cannot change it.

FB%INV  (1B12)  -  File  is  invisible.  Puts  the  file  in  the
"invisible" section of the directory. These  files  can  be  seen
only  by using the G1%IIN option to GTJFN. FB%INV can be modified
with CHFDB by anyone with owner access.

FB%OFF (1B13) - File is offline. This is changed by ARCF when  it
removes the contents from disk, and when it restores the contents
to disk. FB%OFF cannot be changed with CHFDB.

                   FBBBT (22) (Formerly FBBK3)

The  left  half  of  FBBBT  is used for bits while the right half
contains the number of pages in the file when the  contents  were
deleted  from  disk. The following describes the bits in the left
half of FBBBT.

AR%RAR (1B1) - User request for a file to be archived.  This  can
only be modified by ARCF.

AR%RIV  (1B2)  - System request for an involuntary migration of a
file. The ARCF jsys is used to modify this bit,  and  the  caller
must have enabled WHEEL or OPERATOR capabilities.

AR%NDL  (1B3)  - Do not delete the contents of the file from disk
when the archival is complete. Set by ARCF.

AR%NAR (1B4) - Resist involuntary migration. This bit is  a  note
from  the user to the system policy module asking the file not be
moved offline if possible.  Set and cleared by ARCF.

AR%EXM  (1B5)  -  File  is  exempt  from  involuntary  migration.
Modified  only by ARCF and enabled WHEEL or OPERATOR capabilities
are required.

AR%1ST (1B6) - First pass of an  archival/collection  run  is  in
progress.  Modified  by  CHFDB and used only by DUMPER.  Requires
enabled WHEEL or OPERATOR capabilities.

AR%RFL (1B7) - Restore failed. Set by ARCF to indicate to another
process that the restore it is waiting on has failed.




                              - 1 -

DEC archive/virtual disk system                             DRAFT
FDB modifications

AR%PSZ (1B18-1B35) - The right half of FBBBT is used to store the
number of pages in a file when the  contents  were  removed  from
disk.  This value is set by the DELF JSYS (DF%CNO option) and can
be read with CHFDB or ARCF.

                   FBNET (23) (Formerly FBBK4)

          On-line expiration date and time. Specifies a date  and
time  at  which  a  file  is  considered expired, or specifies an
interval (in days) after which the file  is  considered  expired.
Modified by SFTAD.

                        FBTDT (31) (New)

          Archive or collection tape-write date and time. This is
the  date  and  time (in internal format) of the most recent tape
write to the archives or  virtual  disk.  Modified  by  ARCF  and
requires enabled WHEEL or OPERATOR capabilities.

                        FBFET (32) (New)

          Offline  expiration  date  and time. Specifies the date
and time (or interval) after which a file in the archives  or  on
virtual  disk  is  considered  expired.  Used for tape recycling.
Modified by SFTAD.

                        FBTP1 (33) (New)

          FBTP1  contains  tape  ID  for  the  first  archive  or
collection  run.   Modified  only  by  ARCF  and enabled WHEEL or
OPERATOR capabilities are required.

                        FBSS1 (34) (New)

          FBSS1 contains the saveset and tape  file  numbers  for
the  first  tape. The left half is the saveset number the file is
recorded in, the right half is the tape file number  within  that
saveset.   Modified  only  by  ARCF and enabled WHEEL or OPERATOR
capabilities are required.

                        FBTP2 (35) (New)

          Tape ID for second archive or collection run. (as  with
FBTP1)

                        FBSS2 (36) (New)

          Saveset and tape file numbers for the second archive or
collection run.  (as with FBSS1)





                              - 2 -

DEC archive/virtual disk system                             DRAFT
JSYS descriptions

                Modifications to existing JSYSes

                              CHFDB

          Some  restrictions have been placed on CHFDB. CHFDB can
no longer modify FBBK3 and FBBK4 with the  exception  of  several
bits  in  FBBK3  (now  FBBBT).   These  modifiable  bits are: B0,
AR%1ST, and the right half of FBBBT.  Further, FB%DEL may not  be
set if FB%ARC (in FBCTL) is on.

                              CRDIR

          Default  online  expiration  date  and time and default
offline  expunge  date  and  time  have  been  added.  These  are
specified  by way of the .CDDNE (20) and .CDDFE (21) words in the
CRDIR argument block. These date and times may be  explicit  date
and  times (internal format) or an interval (in days).  In either
case, the specified  date/interval  may  not  exceed  the  system
maximum.   These  parameters are taken from the argument block if
CD%NED (1B2) or  CD%FED  in  .CDLEN  (1B3)  are  set.  If  a  new
directory   is   being  created  and  these  parameters  are  not
specified, the system default is used.

          An unprivileged user may modify his defaults, but  only
to a value less than or equal those which are currently specified
or the system maximum, whichever is greater.

          Additionally,  a  new  mode  bit (CD%DAR 1B3), has been
defined. This  bit  specifies  that  when  online  expiration  is
reached,  the  files  concerned  should  be  archived rather than
migrated to virtual disk.

                              DELDF

          When  a  file  with  archive  status  is  deleted   and
expunged,  DELDF  sends a message to galaxy. The message contains
all archive status information which includes tape information as
well as the present file name, user who expunged  the  file,  and
the  time  at which it was expunged.  These messages (of the IPCF
flavor) will be processed by GALAXY at a later time.

                         DELF and DELNF

          DELF and DELNF fail (DELXnn)  for  files  with  archive
status.  For  the  delete  to succeed, DF%ARC (1B4) must be on in
AC1. This bit specifies permission to delete files  with  archive
status.   A  second  bit has been defined in DELF and DELNF. This
bit, DF%CNO (1B5), causes only the contents of  the  file  to  be
deleted  and  immediately  expunged. The file's FDB and name will
remain as it was (with the exception of the page count  and  page
table  address).   A  DELF  or DELNF with DF%CNO set will fail if
AR%NDL is set in FBBBT or  if  a  complete  set  of  tape  backup
information is not in the FDB.

                              - 3 -

DEC archive/virtual disk system                             DRAFT
JSYS descriptions

                         GTJFN and GNJFN

          GTJFN  has  only  one area of modification. It must now
ignore files that have the invisible bit (FB%INV) set. GTJFN  has
a  new  bit (G1%IIN 1B5) which specifies that GTJFN should ignore
the invisible status of a file. G1%IIN is given to GTJFN in .GJF2
of the extended argument block. GTJFN will return a  bit  (GJ%GIV
1B17) (like GJ%GND for deleted files) for use by GNJFN.

                              JFNS

          JFNS  can  now  output  a new attribute ";OFFLINE". The
output of this can be requested by the JS%OFL (1B29) bit in  AC3.

                              OPENF

          OPENF  has  been  modified to deal with archived and/or
offline files.  The following table describes OPENF's action  for
each possible state of a file.

                   Archived                     Non-archived
OPENF Access|  Online  |  Offline    ||  Online    |  Offline
------------|----------|-------------||------------|-----------
   Read     |    Ok    |  Fail/Hang  ||    Ok      |  Fail/Hang
   Write    |   Fail   |    Fail     ||    Ok (discard implied)
   Append   |   Fail   |    Fail     ||Ok (discard |  Fail/Hang (discard
                                         implied)          implied)

          The  fail  cases  will  all return an appropriate error
message (OPNXnn).  The Fail/Hang cases will return an  error  for
fail  or wait until the OPENF can be successfully completed. Fail
or hang will be determined by the setting of OF%NWT  (never  wait
for file), OF%RAR (retrieve file if necessary), or the setting of
the job default (retrieve file if necessary). If OF%NWT is set on
the  OPENF call, OPENF will always fail (in the Fail/Hang cases).
If OF%RAR or the job default is set, the OPENF will wait for  the
file  to  be retrieved and then complete successfully.  In the Ok
(discard implied) cases, tape pointers, if any, for the file  are
discarded.

                              RFTAD

          Three  new  words  can  be  returned by RFTAD. They are
.RSTDT  (4)  (tape-write  date  and  time),  .RSNET  (5)  (online
expiration  date  and  time),  and .RSFET (6) (offline expiration
date and time). Of these, .RSNET and .RSFET may  be  a  date  and
time  (in  internal  format)  or an interval (specified in days).
Intervals are limited to 2**18-1 days (a half word).

                              RNAMF




                              - 4 -

DEC archive/virtual disk system                             DRAFT
JSYS descriptions

          Renaming a file with tape information carries the  tape
information to the new file name. Renames which would effectively
destroy a file with archive status fail.

                              SETJB

          A  new  function  code  (.SJDFR  (6)  set  job  default
retrieval) has been added to enable a user to override the system
default for OPENF. .SJRFA (0) specifies that OPENF should  always
fail  if the file's contents are not online. .SJRWA (1) specifies
that OPENF should wait for ARCF to restore the  contents  of  the
file to disk. .SJRFA is the system default.

                              SFTAD

          Three  additions  have  been  made  to  SFTAD. They are
.RSTDT  (4)  (tape-write  date  and  time),  .RSNET  (5)  (online
expiration  date  and  time),  and .RSFET (6) (offline expiration
date and time). Modification of .RSTDT requires enabled WHEEL  or
OPERATOR capabilities. .RSNET and .RSFET may be either a date and
time  (internal  format)  or  an  interval  (specified  in days).
Intervals are limited  to  2**18-1  days.   Date  and  times  and
intervals may not exceed system/directory maximum.

                              SMON

          New function codes .SFMCY (54), .SFACY (55), and .SFRTW
(56)  have  been  added.  .SFMCY  specifies  the  maximum offline
expiration  (tape  recycle  time)  for  ordinary  files.   .SFACY
specifies  the  expiration period for archive files (tape recycle
period). .SFRTW is used to set/clear the NO RETRIEVAL-WAITS  flag
in  the  monitor.  When  set this specifies that those retrievals
waiting for the retrieval should fail rather than hang.

                              TMON

          .SFMCY and .SFACY have been added to read  the  maximum
offline  expiration  time  for ordinary and archived files. These
values are the recycle times for tapes containing either ordinary
or archived files.


                           New Jsyses

                            ARCF Jsys

          The ARCF jsys will perform all operations pertaining to
the archive and virtual disk systems.  This  includes  requesting
archival and migration, requesting retrieval, and setting archive
status and tape information for a file.




                              - 5 -

DEC archive/virtual disk system                             DRAFT
JSYS descriptions

                Accepts:

                    1: JFN
                    2: Function code
                    3: Function dependent, normally 0

                              ARCF

                Returns:

                    +1 Always

Functions:

          .ARRAR  (0)  -  This  function  sets/clears  AR%RAR  in
.FBBBT, a user request for archival. The value .ARSET in AC3  (1)
will  request an archive while .ARCLR (0) will clear the request.
Setting AR%NDL with .ARSET in AC3 requests the  contents  of  the
file not be flushed from disk upon archival.

          .ARRIV  (1)  -  This  function  sets/clears  AR%RIV  in
.FBBBT, a system request to migrate a file off  disk.  The  value
.ARSET  in AC3 will request migration while .ARCLR will clear the
request. The caller must be an enabled WHEEL or OPERATOR.

          .AREXM (2) - Set/Clear AR%EXM (exempt from  involuntary
migration)  in  .FBBBT. A 0 in AC3 clears AR%EXM while a 1 in AC3
sets it.  Requires enabled WHEEL or OPERATOR capabilities.

          .ARRFR (3) - Request a file's contents be  restored  to
disk.   Normally  .ARRFR returns without waiting for the contents
of the file to be restored to disk.
Options in AC3 are:
AR%NMS (1B0) - Do not  send  a  message  to  the  requestor  upon
restoration.
AR%WAT  (1B1)  -  Wait  (hang in the ARCF jsys) until the file is
restored.

          .ARDIS (4) - Discard tape  information.  Clears  FB%ARC
(if  set),  .FBTP1, .FBTP2, .FBTSN, .FBTFN, and .FBTDT.  The file
must be online for the function to  succeed.   AC3  option  bits:
AR%CR1  (1B0)  clears  only  first  run information. AR%CR2 (1B1)
clears only pass two  information.  Requires  WHEEL  or  OPERATOR
capabilities to use AR%CR1 or AR%CR2 individually.

          .ARSST  (5)  -  Set  tape  information  for  a file. It
requires WHEEL  or  OPERATOR  capabilities.   AC3  points  to  an
argument block as follows:

               0: (.AROFL) Option bits
                   AR%O1 (1B0) - Set run 1 information
                   AR%O2 (1B1) - Set run 2 information


                              - 6 -

DEC archive/virtual disk system                             DRAFT
JSYS descriptions

                   AR%OFL (1B2) - Delete disk contents of file
                            when done. Requires bothsets of
                            tape info be set.
                            (.ARGST returns 0 for word .AROFL)
                   AR%ARC (1B3) - Set FB%ARC (archive the file)
                   AR%CRQ  (1B4) - Clear archive and/or migration
                            requests (AR%RAR and AR%RIV)
               1: (.ARTP1) Tape 1 ID
               2: (.ARSF1) XWD TSN 1, TFN 1  see footnote(1)
               3: (.ARTP2) Tape 2 ID
               4: (.ARSF2) XWD TSN 2, TFN 2
               5: (.ARODT) GTAD of tape-write; 0 implies
                           present time
               6: (.ARPSZ) Number of pages in the file.
                           This word is only returned
                           by .ARGST (.ARSST cannot set it)
This  function (.ARSST) is used to set information for the first,
second, or both tape runs. AR%O1 and AR%O2 are used together when
restoring files to disk via DUMPER's RESTORE command.

          .ARRST (6) - Restore the contents of a  file  to  disk.
AC3  is  a  JFN  for  a temporary file that contains the data for
currently offline archived file. After copying .FBADR, .FBBSY and
.FBSIZ the temporary file is deleted.  Both files must be on  the
same   device   or   structure  and  enabled  WHEEL  or  OPERATOR
capabilities are required.

          .ARGST (7) - Get tape information for file. AC3  points
to an arguement block to be filled. The format is the same as for
.ARSST.

          .ARRFL (10) - The restore for this file has failed. Set
AR%RFL  in  FBBBT  to notify a waiting process that the retrieval
request cannot be completed.  Requires enabled WHEEL or  OPERATOR
capabilities.

          .ARNAR  (11)  -  Resist  involuntary migration. Sets or
clears AR%NAR in .FBBBT. .ARSET in AC3 will cause  resist  to  be
set, while .ARCLR will clear it.  Error returns:

CAPX1 - WHEEL or OPERATOR capabilities required.

ARGX02 - Invalid function code

ARCFX2 - File already has archive status


_________________________________________________________________
(1) TFN is for Tape File Number. It is a number recorded  on  the
tape  with  the file which is its index in a given save set.  TSN
is for Tape Saveset Number. The saveset number on a tape is  also
recorded in the save set name.


                              - 7 -

DEC archive/virtual disk system                             DRAFT
JSYS descriptions

ARCFX3  - Cannot perform ARCF functions on non-multiple directory
devices

ARCFX4 - File is not online

ARCFX5 - Files are not on the same device or structure.

ARCFX6 - File does not have archive status

ARCFX7 - Invalid parameter for .ARSST

ARCFX8 - Archive not complete

ARCFX9 - File not offline

ARCX10 - Archive prohibited

ARCX11 - Archive requested, modification prohibited

ARCX12 - Archive requested, delete prohibited

ARCX13 - Archive system request not completed

ARCX14 - Restore failed

ARCX15 - Migration prohibited

ARCX16 - Cannot exempt  offline,  archived,  or  archive  pending
files

ARCX17 - FDB improper format for ARCF

ARCX18 - Retrieval wait cannot be fulfilled for waiting process

ARCX19 - Migration already pending



















                              - 8 -


DEC archive system spec                                     Draft
Implementation details
Policy module

          The policy module for the Archive/Virtual Disk  system,
REAPER,  is intended for use by system administrators to increase
free disk space.  It can free up disk space in several  ways:  by
marking  for  involuntary  migration  files  which  have not been
referenced within a specified period of time;   by  deleting  the
disk  contents  of  archived or migrated files that have not been
referenced within a specified period; and by trimming directories
that are over their permanent disk allocations.

          A typical  use  of  REAPER  is  to  give  a  number  of
commands,  one  for  each  operation  to  be performed.  When all
desired operations have been  specified,  the  BEGIN  command  is
given,  specifying the path of files to be considered.  Following
is a list of REAPER's commands, with a description of each.

          BEGIN (Processing Files) -  Specifies  which  files  to
process  in a scan over the disk system.  Normally, one specifies
PS:<*>*.*.*,  but  one  could  specify  other  paths,   such   as
PS:<*>*.REL.*,   or  FOO:<TST*>*.*.*,  etc.   This  command  also
initiates the scan over the disk system, so  it  should  only  be
given  after  all desired options have been specified.  After the
specified files have been  processed,  REAPER  prints  the  total
number of pages reclaimed and then exits.

          DELETE  (Disk contents) - Specifies that files existing
both on disk and on an Archive/Virtual  Disk  tape  are  to  have
their  disk  contents deleted if the file has not been referenced
within a specified time period.  The time period is specified  by
the PERIOD command.  If the DELETE command is not given, the disk
contents of these files remains online.

          EXIT - Exits from REAPER, without processing any files.
Normally,  the  BEGIN  command  is used to exit from REAPER after
processing some files.

          LIST (To file) -  Specifies  an  output  file  for  the
listing  produced  by  REAPER.  If this command is not given, the
listing is written to the file DSK:ARCHIVE.LIST.

          MIGRATE  (Old  files)  -  Specifies  that   files   not
referenced  within  a  specified time period are to be marked for
involuntary migration.  The  time  period  is  specified  by  the
PERIOD command.  If the MIGRATE command is not given, these files
are not marked for migration and remain online.

          ORDER  (For trimming) - Specifies the order in which to
mark files for migration when TRIMming a directory that  is  over
its   permanent  allocation.   This  order  is  a  list  of  file
specifications, such as:  *.TMP.*, *.LST.*, *.REL.*.  This  order
list is only used if the TRIM command is given.



                              - 1 -

DEC archive system spec                                     Draft
Implementation details
Policy module

          PERIOD  -  Specifies  an age limit in days within which
files will not be MIGRATEd or have their disk  contents  DELETEd.
The usual period is 60 days.  This means that files older than 60
days  will  be  marked  for involuntary migration (if the MIGRATE
command is given) or will have their disk  contents  deleted  (if
the  DELETE  command  is  given.)   If  the PERIOD command is not
given, a default period of 60 days is assumed.

          SCAN (Only) - Specifies that no files  should  actually
be  DELETEd, MIGRATEd, or TRIMmed, but that only a listing should
be produced.  If the user is not an enabled  wheel  or  operator,
this is the default mode of operation (see below).

          SKIP  (Directories)  -  Specifies a list of directories
that are  not  to  be  considered  when  performing  the  DELETE,
MIGRATE,  and  TRIM operations.  This should be a select group of
directories; for instance, system directories and  other  special
directories should be included in this list.  If the SKIP command
is  not  given, all directories encompassed by the filespec given
in the BEGIN command will be processed.

          TAKE (Command file) - Reads a file of REAPER  commands.
This  file  can  contain  a list of directories to be SKIPped, an
ORDER list, a standard PERIOD, etc.  Although the  BEGIN  command
can  be  included in the command file, it probably should not be.
The default TAKE file is SYSTEM:REAPER.CMD.   This  command  file
can  contain the REAPER commands that implement a site's policies
toward involuntary migration.

          TRIM (Directories) - Specifies  that  directories  over
their  permanent disk allocations should be trimmed down to size.
The order  in  which  files  are  marked  for  migration  can  be
specified  in  two  ways.   If  the directory under consideration
contains a file named MIGRATION.ORDER, then the contents of  that
file  constitutes  the order list for that directory.  If no such
file exists, then the order list is taken as given in the  REAPER
ORDER command.  Thus, a per-directory order list takes precedence
over  a system wide order list.  REAPER makes several passes over
a  directory  attempting  to  trim  it  back  to  its   permanent
allocation.   First,  all  files in the order list are considered
(subject to the PERIOD in effect) except those having the "resist
migration" or "exempt from migration" bits set.  If there  is  no
order  list,  then this pass is skipped.  If the first pass fails
to bring the directory under its allocation,  all  files  in  the
directory (*.*.*) are considered, except those marked "resist" or
"exempt".  If necessary, these two passes are then repeated, this
time ignoring the "resist" bit.

          When  the  BEGIN command is given, REAPER processes the
specified file path on a directory by  directory  basis.   Within
each  directory,  the  MIGRATE,  DELETE,  and TRIM operations are


                              - 2 -

DEC archive system spec                                     Draft
Implementation details
Policy module

performed, in that order.   (TRIM  is  performed  last  to  avoid
unnecessary  file migration.)  Two other operations are performed
as well:  temporary (;T) files which are older  than  the  PERIOD
are  deleted  and  expunged;  and the directory is purged of FDBs
which point to offline files which  have  reached  their  offline
expiration  date.   If Control-A is typed while the BEGIN command
is underway, REAPER prints  the  name  of  the  directory  it  is
working on at the moment.

          If the user is not an enabled wheel or operator, REAPER
does  not  prompt  for commands.  Instead, it simply asks for the
name of the listing file, and  the  file  path  which  is  to  be
scanned.   REAPER  then  processes  this  path as if the commands
"TAKE SYSTEM:REAPER.CMD"  and  "SCAN"  were  given,  producing  a
listing  of  those  files  that  would be taken given the present
system policies toward file migration.





































                              - 3 -