Trailing-Edge
-
PDP-10 Archives
-
BB-H311C-RM
-
swskit-documentation/archive.mem
There are 5 other files named archive.mem in the archive. Click here to see a list.
***** REVISION HISTORY *****
THE FOLLOWING CHANGES HAVE BEEN MADE TO THE FUNCTIONALITY
OF THE ARCHIVING SOFTWARE. NOTE THAT THESE CHANGES TAKE
PRECEDENCE OVER THE CONTENTS OF THIS SPEC. THAT IS, IF
THEY DISAGREE, THE REVISION HISTORY SHOULD BE USED.
1. THE "ARCHIVE" SUBCOMMAND TO THE DIRECTORY COMMAND WILL
PRINT OUT THE NAMES OF ALL ARCHIVED FILES, REGARDLESS
OF WHETHER THEY ARE VISIBLE OR INVISIBLE.
2. THE ^ESET [NO] RETRIEVAL-WAITS COMMAND IS NOT CURRENTLY
IMPLEMENTED.
***** END OF REVISION HISTORY *****
DEC archive system spec Draft
Implementation details
DUMPER
This section will describe the tasks DUMPER must
perform to support the Archive/Virtual Disk system. This
includes copying files to tape, removing the contents from disk,
etc.
DUMPER will not contain any of the policy for deciding
when files may be copied to tape and have their contents removed
from the disk. This task is performed by REAPER, the policy
module for the Archive/Virtual Disk system. Instead, DUMPER will
only dump those files marked for voluntary archival or those
files past their expiration date or marked for involuntary
migration, depending on which function the operator wishes to
perform. The process of dumping files to tape will be called an
archival run if voluntary archivals are being processed, or a
collection run if involuntary migrations and expirations are
being processed. In either case, the operation is very similar
to performing an incremental dump. The command syntax for
performing an archival run is "DUMPER>SAVE/ARCHIVE PS:<*>*.*.*";
the syntax for an collection run is "DUMPER>SAVE/COLLECT
PS:<*>*.*.*". Alternatively, the switch "/MIGRATE" can be used
in place of "/COLLECT" to process files marked for migration and
ignore files which have expired. Naturally, any filespec can be
used in these commands. If no filespec is given, the default is
to process all files on the connected structure.
Each archival or collection run consists of two logical
passes. During the first pass, selected files are copied onto
tape. When this pass is complete, a second "clean up" pass is
made to mark data as valid and delete the disk contents of those
files which have been reliably copied to two tapes. (If a file
is being archived, and AR%NDL is set, then the disk contents are
not deleted.) ___
Now in detail, here is what DUMPER will do during each
pass of the archival or collection run. Pass 1: Examine each
file in the specified path. If this is an archival run and the
file is marked for voluntary archival, or if this is a collection
run and the file is past its expiration date (/COLLECT) or is
marked for involuntary migration (/COLLECT or /MIGRATE), then set
AR%1ST in the file's FDB to indicate that a DUMPER run is in
progress. Then, set the tape information in the FDB via ARCF
(.ARSST), and dump the file to tape. This process continues
until all files in the path have been examined. At this point
DUMPER ends the saveset on the tape and rewinds it. Pass 2, the
clean up pass, now begins. Once again the file path is scanned,
and any file with AR%1ST set is considered. First, AR%1ST is
cleared, thereby marking the file contents valid. The tape
information is then examined, and if it is complete (the file has
been dumped successfully to two tapes), ARCF (.ARSST, AR%OFL) is
invoked to delete the disk contents. If this is an archival run,
FB%INV is set to make the file invisible. Finally, ARCF (.ARSST,
AR%CRQ) is
- 1 -
DEC archive system spec Draft
Implementation details
DUMPER
invoked to clear the request bits. At the end of each directory
(GNJFN notes this by GN%DIR being returned) a message is sent to
the owner listing all files that have been successfully copied to
two tapes. This process continues until all files have been
processed.
Should a file with AR%1ST be encountered during the
first pass of DUMPER's archival or collection run, the most
recent set of tape numbers must be cleared. This is necessary
since AR%1ST indicates that the contents of the file were not
reliably copied to tape.
The retrieval operation is handled by DUMPER as
follows. Retrieval requests are stored in a queue maintained by
GALAXY. When the command "DUMPER>RETRIEVE STR:<*>*.*.*" is used,
DUMPER first sends an IPCF message to GALAXY indicating its
desire to process the retrieval request queue. It then receives
retrieval requests from GALAXY one by one. If a particular
retrieval request falls within the file specification given in
the command, DUMPER attempts to retrieve that file; otherwise
DUMPER requests that GALAXY requeue the file for retrieval later.
Since the requests are ordered by the tape information, more than
one file can be retrieved from a single tape without rewinding or
dismounting the tape. Should the retrieval of a file fail,
either because the proper tape could not be mounted or because of
data errors, DUMPER will request that GALAXY requeue the
retrieval request, considering the first tape information in
place of the more recent copy. If DUMPER fails to retrieve a
file from the both tapes, and the retrieval was requested
explicitly by a user, that user is notified by mail. If an
implicitly generated retrieval request cannot be honored, the
process awaiting the retrieval receives an OPNXxx error.
For each file that is to be retrieved, a temporary file
is created. The file contents are restored from tape to this
temporary file, and then the file's FDB is changed to reflect the
file information on tape, except for the archive/virtual disk
information. Then ARCF (.ARRST) is used to adjust the disk
contents pointer of the old FDB, bring over the non
archive/virtual disk information to it from the temporary file,
and delete and expunge the temporary file.
The DUMPER RETRIEVE command works only from the
retrieval requests in the GALAXY retrieval queue. The RESTORE
command, as previously, requires only the file specification,
along with the appropriate tape already mounted. When RESTORE is
used, the tape header record is checked for membership in the
Archive/Virtual Disk system. If the membership test succeeds, a
message is output indicating that RETRIEVE should normally be
used for Archive/Virtual Disk system tapes for reasons of system
security. Only on confirmation and wheel or operator
- 2 -
DEC archive system spec Draft
Implementation details
DUMPER
capabilities enabled will the RESTORE continue. The RESTORE
command can thus retrieve a previously migrated file which has
since been deleted and expunged. If the file as specified
already exists on disk with archive status, the RESTORE command
skips that file.
Consider now the use of tapes other than
Archive/Virtual Disk system tapes. When an archived or expired
file is SAVEd, the archive/virtual disk information follows it on
to tape. On RESTORE, no archive/virtual disk information is
brought back to disk in the absence of wheel or operator
capabilities. In the case of wheel or operator capabilities
enabled, a simple RESTORE command does bring back the
archive/virtual disk information. This command should be used
for (local) system tapes only. An operator handling a non-system
tape (typically a user tape brought in from another site) should
use RESTORE with the NOARCHIVE switch, which suppresses the
transfer of archive/virtual disk information and thus avoids the
importation of bogus information into the backup words of the
FDB, and more importantly, guards against attempts to create
counterfeit tape pointers.
- 3 -
DEC Archive/virtual disk system Draft
Introduction
System managers are recognizing the growing need for
reliable and convenient offline storage of data. Its
availability for general use allows valuable disk space now tied
up in inactive storage to be freed for active use. Users are
provided an equally-handy alternative to file deletion to meet
directory allocation requirements. At the same time, users need
a permanent repository for important files and files not in
current use. A reasonable model to use for this purpose is an
archive. Items entered into an archive never change, and never
leave; only copies of items can be removed.
- 1 -
DEC Archive/virtual disk system Draft
User overview
The offline storage system is presently implemented
using a collection of magnetic tapes each holding many individual
files. This collection of tapes is divided into two sections:
archive tapes in permanent storage (say 10 years) and non-archive
tapes in storage for a shorter period, say 6 months.
Archived files
The archives can be viewed as unchanging data that at
one time existed as disk files on some system. Each user has in
their directory pointers into the archives for any data they have
placed in the archives. These pointers can be manipulated as
files, which for practical purposes, they are. When a file gains
archive status(1) it can no longer be modified. This is necessary
to assure one and only one version of the data exists. Data of an
archived file exists in the archives and may simultaneously exist
on disk. These copies will always be identical. A file is
archived only by the explicit action (the ARCHIVE command) on the
part of the user.
Ordinary files in Offline Storage
The non-archive section of offline storage provides
shorter- term storage of files with contents no longer desired on
disk, but not considered so valuable as to require permanent
storage. This decision may be made explicitly by an individual
user managing his directory within the disk allocation
requirement, or implicitly by the user as files pass their online
expiration dates (which are user- settable, up to a point), or by
a system policy program used in the case of insufficient system
disk resources. Such files have no special status and will be
treated as closely as possible to the way disk files have been
treated in the past. To differentiate them from archive files,
and point up their lack of special properties, we will refer to
non-archived files as ordinary files. Using this terminology,
offline storage is divided into storage of ordinary files and
archived files.
Tape Recycling
The crucial difference between archived files and
ordinary files in offline storage is their allowed lifetimes on
tape. The maximum such lifetime is limited by the tape recycle
time. The archive and non-archive recycle times are set by the
system manager at each installation with an eye to the tape and
tape storage resources. Since files are never marked to indicate
a lifetime on tape beyond the tape recycle time in duration, an
_________________________________________________________________
(1) A file is considered to have archive status at the time of
the ARCHIVE command. For a file to be completely "archived" there
must be two copies of the file in the archives.
- 2 -
DEC Archive/virtual disk system Draft
User overview
individual tape may be recycled as soon as its (most recent)
write date lies in the past by the amount of the tape recycle
time.
Concept of a file with contents online and/or offline
A file is considered to be a structure consisting of a
single FDB, its associated name string, and its associated
contents. Renditions of the file contents may reside on disk, in
offline storage, or both. To maintain the essential identity of
the contents of each file,the disk contents of online archived
files are unalterable; such disk contents are reference copies
of archived files. If a writable copy of the same contents is
desired, it is easily obtained through use of the COPY command.
In the case of ordinary files with offline contents, the offline
contents are considered subsidiary to the disk contents. The
essential identity of contents is maintained by simply clearing
the tape information from the FDB when the disk contents are
altered. Thus just after a successful alteration of contents, a
file is certain to be a disk-only file.
Offline storage
When entered into offline storage, files (FDB, file
specification, and contents) are copied onto two different
magnetic tapes, which are stored in the installation's tape
vault. If the installation consists of more than one computer
system, the tape storage may still be made common to all of its
systems, and it is possible to put files in offline storage from
one system, have the directory moved to another system, and then
retrieve the files in the ordinary manner onto the second system.
Pointers to offline storage
A pointer into offline storage consists of the tape ID,
saveset number, and tape file number for each of the two tape
copies of the file contents. The pointer holds all the
information necessary to retrieve a single file. It ordinarily
resides in the file's FDB, where it is protected in the same
manner as other essential file information, including the
analogous disk contents pointer.
Delete protection of archived files
Archived files are protected against casual or
accidental deletion by the requirement, for successful deletion,
of the use of an archive-specific subcommand to the DELETE
command. Deletion may be undone by an ordinary UNDELETE command.
Further, it will be impossible to overwrite an archived file with
a RENAME or COPY command. The archive pointer itself is
preserved past file expunge in a message sent the user. The user
- 3 -
DEC Archive/virtual disk system Draft
User overview
may maintain a file (itself archivable) of such messages for dead
storage of archived files.
Ordinary files have no such delete protection. Any
case in which disk-only files lose their contents (rewrite,
destructive rename, simple delete followed by expunge) extends to
all ordinary files. If an ordinary file is accidentally deleted,
a backup operation should be initiated just as in the case of a
disk-only file.
Messages sent to Users
Messages are sent to users when files are taken
offline, archived or not, when retrieve operations are completed,
and when archived files are expunged. An individual site may use
DEC mail, or no mail at all. Assume here that mail
is used. Retrieval messages go to the user who made the request.
For the other messages, the directory in which the file exists is
examined as follows. If the file DIRECTORY.OWNER exists, it
contains the name of the user for such mail. If DIRECTORY.OWNER
does not exist, the messages go to MAIL.TXT in the directory.
Visible and Invisible Files
A well-established user typically has files, archived
or not, online or offline, which are not in current use and tend
to clutter the directory. To solve the problem of clutter
without complicating the system for the casual or inexperienced
user, files have been given a visible/invisible option which is
changed only in conjunction with voluntary offline archival and
retrieval or explicitly by the user.
A visible file is one which shows up on a simple (no
subcommands) directory listing and is accessible to programs and
EXEC commands dealing with files. In some cases, such access
immediately brings about an error return, for example, in an
attempted copy of an visible but offline file. An invisible file
does not show up on a simple directory listing and is not
accessible to programs and EXEC commands, except for those
programs and commands which specifically provide for them (as is
the case with deleted files.) Like deleted files, invisible
files are not totally transparent: they affect the determination
of default generation numbers by virtue of their ownership of a
certain file specification.
A visible file may be made invisible as follows:
@SET FILE INVISIBLE (FILES) PROG.MAC
PROG.MAC.1 [OK]
- 4 -
DEC Archive/virtual disk system Draft
User overview
Similarly, an invisible file may be made visible:
@SET FILE VISIBLE (FILES) TST.FOR
TST.FOR.1 [OK]
Any file, archived or not, may be set visible or
invisible. In this sense, the visibility feature is independent
of the archive system. The default setting, however, depends on
the archive history of the file, as follows. A file is (default)
visible unless it is archived and offline, in which case it is
(default) invisible. Ordinary files maintain their visibility or
invisibility through online-offline transitions.
- 5 -
DEC Archive/virtual disk system Draft
EXEC Modifications
EXEC Commands Controlling File Content Location
Three commands handle the explicit requests for
transfer of file contents to and from offline storage. The
ARCHIVE command requests the contents be entered in the archive
section of offline storage. The SET FILE EXPIRED command sets
the online-expiration date of the file to the present date and
thus in effect requests the contents be placed in non-archive
section of offline storage. The RETRIEVE command requests the
contents be restored to disk from whichever section of offline
storage they reside in.
For each of the following examples, assume that the
connected directory <CALVIN> has just five files,
ARCHIVED.OFFLINE.1, ARCHIVED.ONLINE.1, ORDINARY.DISK-ONLY,
ORDINARY.OFFLINE, and ORDINARY.BOTH. ARCHIVED.OFFLINE is offline
archived, that is, its contents are on tape but not on disk. It
is invisible, the default for offline archived files.
Consequently it will not be accessible to EXEC commands other
than SET FILE VISIBLE, the invisible-file subcommand to
DIRECTORY, and RETRIEVE. ARCHIVED.ONLINE is archived, with
contents on disk as well as tape. Again following the default
conditions, this file is visible. ORDINARY.DISK-ONLY is an
ordinary disk file with no pointer to offline storage, i.e. an
old-fashioned disk file. ORDINARY.OFFLINE is an ordinary file
with contents in offline storage but not on disk. It is visible,
the default case here since visibility status does not change
with transfer of contents to and from the ordinary file section
of offline storage. Similarly, ORDINARY.BOTH, an ordinary file
with contents on both disk and in offline storage, is visible for
these examples.
The ARCHIVE Command
The ARCHIVE command marks the specified files for
archival by turning on the archive-request bit in the file's FDB.
@ARCHIVE (FILES) ORDINARY.*,ARCHIVED.*
ORDINARY.BOTH.1 [Requested]
ORDINARY.OFFLINE.1 %File is not online
ORDINARY.DISK-ONLY.1 [Requested]
ARCHIVED.ONLINE %File already archived
Note the invisibility of ARCHIVED.OFFLINE. Here the
disk contents of ORDINARY.DISK-ONLY.1 become unalterable at the
time of the ARCHIVE command, and are expunged from disk at the
end of the archival process. The resulting offline archived file
is then set invisible. The same treatment is given
ORDINARY.BOTH, with the pointer into the ordinary section of
offline storage being replaced by a pointer into the archive
section.
- 6 -
DEC Archive/virtual disk system Draft
EXEC Modifications
@ARCHIVE (FILES) ORDINARY.DISK-ONLY,
@@RETAIN (DISK CONTENTS)
@@
ORDINARY.DISK-ONLY.1 [Requested]
In this case, the disk contents become unalterable at
the time of the archive request, and remain so past the end of
the archival process. The resulting online archived file retains
its previous visibility or invisibility.
The archival request may be changed from the default
disk-expunge to retain, or vice versa, by simply redoing the
command in the preferred fashion. The request for archival may
be cancelled:
@CANCEL ARCHIVE-REQUESTS (FOR FILES) ORDINARY.DISK-ONLY
The DISCARD Command
The DISCARD command is used to throw away the tape
information associated with a file. When this action is taken,
the tape information is deleted from the FDB of the file. Files
must have the contents of the file online for DISCARD to succeed.
@DISCARD (TAPE INFORMATION OF FILES) ARCHIVED.*,ORDINARY.*
ARCHIVED.ONLINE.1 [OK]
ORDINARY.BOTH.1 [OK]
ORDINARY.DISK-ONLY.1 %File has no tape pointer
ORDINARY.OFFLINE.1 %File not online
The SET FILE ONLINE-EXPIRATION Command
The SET FILE ONLINE-EXPIRATION command is used to
specify when a file's disk contents expire. When this expiration
date/interval is reached, the file's contents are moved offline
and remain there until the offline expiration date/interval is
reached.
@SET FILE ONLINE-EXPIRATION (OF FILES) ORDINARY.DISK-ONLY (TO) 60
ORDINARY.DISK-ONLY.1 [OK]
@SET FILE ONLINE-EXPIRATION (OF FILES) ORDINARY.* (TO) 18-JUL-78
ORDINARY.BOTH.1 [OK]
ORDINARY.DISK-ONLY.1 [OK]
ORDINARY.OFFLINE.1 %File is offline
In the first example, ORDINARY.DISK-ONLY will have its
disk contents expire 60 days after any reference to the file has
occurred. In the second example, the two files that were "[OK]"
will have the disk contents moved offline on or after
18-Jul-1978, depending when the system personnel run the tape
program.
- 7 -
DEC Archive/virtual disk system Draft
EXEC Modifications
The SET FILE OFFLINE-EXPIRATION Command
The SET FILE OFFLINE-EXPIRATION command specifies when
the contents of an offline file can be expunged from offline
storage. This parameter will either specify an explicit date or
an interval of days after the file was moved to offline storage.
In either case, the specified value may not exceed the system
tape recycle time.
@SET FILE OFFLINE-EXPIRATION (OF FILES) ORDINARY.OFFLINE (TO) 60
@SET FILE OFFLINE-EXPIRATION (OF FILES) ORDINARY.BOTH (TO) 13-SEP-78
The first example will have the file expunged from
offline storage 60 days after it was moved to offline storage.
The second example will have the file expunged on 13-SEP-78.
To avoid losing a file and its contents, the file
contents may be retrieved from offline storage and then the tape
information discarded by way of the DISCARD command.
The SET FILE EXPIRED Command
The SET FILE EXPIRED command causes the specified files
to become online-expired. This is accomplished by setting the
ONLINE-EXPIRATION date to the present date. These files will then
(at a somewhat later time) be moved to offline storage.
@SET FILE EXPIRED *.*
ARCHIVED.ONLINE.1 %File is archived
ORDINARY.BOTH.1 [OK]
ORDINARY.DISK-ONLY [OK]
ORDINARY.OFFLINE.1 %File is offline
Here ARCHIVED.OFFLINE is completely passed over because
it is invisible. If it were made visible and the command
reissued, it would result in the message "File is offline".
ORDINARY.DISK-ONLY is marked for transfer of contents to tape.
Unlike the archive-requested file, ORDINARY.DISK-ONLY can be
modified at any time while it continues to have contents on disk.
At any such modification, it immediately loses any tape
information (then obsolete, since tape contents are considered
subsidiary to disk contents). The request for transfer offline
still remains in effect unless cancelled. When the file contents
have been copied onto two tapes, the disk contents are deleted.
The resulting offline ordinary file remains visible (since it was
visible before) and gains the ;OFFLINE attribute to mark it
offline. Once it is offline, attempts to read or append to it
fail, unless SET RETRIEVAL WAIT is set (see below), but attempts
to rewrite it succeed, again with the concurrent loss of offline
contents.
- 8 -
DEC Archive/virtual disk system Draft
EXEC Modifications
The SET DIRECTORY ONLINE-EXPIRATION-DEFAULT Command
The SET DIRECTORY ONLINE-EXPIRATION-DEFAULT command is
used to specify the online expiration date/interval to use for
new files created in the directory. This default may be specified
as an explicit date or as an interval in days.
@SET DIRECTORY ONLINE-EXPIRATION-DEFAULT (OF DIRECTORY) PS:<EONEIL> (TO) 90
Or
@SET DIRECTORY ONLINE-EXPIRATION-DEFAULT (OF DIRECTORY) PS:<CRDAVIS> (TO) 22-SEP-78
The SET DIRECTORY OFFLINE-EXPIRATION-DEFAULT Command
The SET DIRECTORY OFFLINE-EXPIRATION-DEFAULT command is
used to specify the default date/interval at which files in
offline storage are expunged from offline storage. This default
may be specified as an explicit date or as an interval in days.
The interval is the number of days since since the file was moved
to offline storage. In either case its setting is required to be
consistent with the system-wide non-archive tape recycle time.
@SET DIRECTORY OFFLINE-EXPIRATION-DEFAULT (OF DIRECTORY) PS:<EONEIL> (TO) 60
Or
@SET DIRECTORY OFFLINE-EXPIRATION-DEFAULT (OF DIRECTORY) PS:<CRDAVIS> (TO) 28-SEP-78
The offline expiration date of a file is set when it is
moved to offline storage. The default expiration date/interval
is used when a file is moved to offline storage to set its
offline-expiration date.
RESIST and PROHIBIT Settings for a File
A user may mark certain online files for consideration
last by REAPER, for involuntary migration (including the deletion
of disk-contents of files with contents on tape as well) by
setting the RESIST bit.
@SET FILE RESIST-MIGRATION ARCHIVED.ONLINE, ORDINARY.DISK-ONLY
ARCHIVED.ONLINE.1 [OK]
ORDINARY.DISK-ONLY.1 [OK]
The RESIST setting can be undone as follows.
@SET FILE NO RESIST-MIGRATION ORDINARY.ONLINE
ORDINARY.ONLINE.1 [OK]
A file may be absolutely protected against migration
from disk by marking it PROHIBIT. This may be done only by a
- 9 -
DEC Archive/virtual disk system Draft
EXEC Modifications
wheel or operator with the command SET FILE PROHIBIT-MIGRATION.
It may be undone with the command SET FILE NO PROHIBIT-MIGRATION.
The RETRIEVE Command
The RETRIEVE command sends a message to the system to
request the restoration of the file contents from offline
storage, archive or ordinary, to disk.
@RETRIEVE (ARCHIVED FILES) ORDINARY.*,ARCHIVED.*
ORDINARY.BOTH.1 %File not offline
ORDINARY.DISK-ONLY.1 %File not in offline storage
ORDINARY.OFFLINE.1 [Requested]
ARCHIVED.OFFLINE.1 [Requested]
ARCHIVED.ONLINE.1 %File not offline
Note that RETRIEVE is one of the very few commands
which acts on invisible files such as ARCHIVED.OFFLINE in the
above example. This property allows users to retrieve offline
invisible files, archived or not, with a single command. After
retrieval, the disk contents of an archived file are unalterable.
The disk contents of an ordinary file may be changed at will,
with the concurrent loss of the pointer to the obsoleted contents
in offline storage.
The retrieval requests not yet carried out can be
cancelled by the CANCEL RETRIEVAL-REQUESTS command.
New Aspects of EXEC Informational Commands
The DIRECTORY and INFORMATION commands provide natural
vehicles for offline storage information. Since the directory
listing sets the stage for the user's concept of his working
area, it is important to implement DIRECTORY in a clear cut
manner which reflects the realities of the situation. Since
retrieval of a file does involve delay, an offline file is
necessarily considered one significant step away from online
capabilities. Correspondingly, offline files, archived or not,
are conspicuously marked by the attribute ;OFF-LINE on all
DIRECTORY listings. Users can make files invisible to remove
them from a default DIRECTORY listing as well as from their
routine working environment, since very few programs can access
them. Offline files are particularly suited to invisibility
since they are a retrieval operation away from routine use. By
listing all visible files, with the offline visible files clearly
marked, with a simple DIRECTORY command, a user sees his routine
working environment at a glance.
Consider the directory <CALVIN> again. A simple
DIRECTORY lists all the visible files:
- 10 -
DEC Archive/virtual disk system Draft
EXEC Modifications
@DIRECTORY
<CALVIN>
ARCHIVED.ONLINE.1
ORDINARY.BOTH.1
ORDINARY.DISK-ONLY.1
ORDINARY.OFFLINE.1;OFF-LINE
The DIRECTORY subcommand to examine just the invisible
files corresponds operationally to the subcommand to examine just
the deleted files. That is, with the INVISIBLE (FILES ONLY)
subcommand, only the invisible files, as otherwise specified, are
listed. If this subcommand is omitted, only the visible files,
as otherwise specified, are listed.
@DIRECTORY,
@@INVISIBLE (FILES ONLY)
@@
<CALVIN>
ARCHIVED.OFFLINE.1;OFF-LINE
The DIRECTORY subcommand particular to archived files
is merely selective: without it, files are displayed without
regard to their archival status, and with it, only archived
files, as otherwise specified, are displayed. A complete archive
directory is obtained by listing the (visible) archived files and
then the invisible archived files.
@DIRECTORY,
@@ARCHIVED (FILES ONLY)
@@
<CALVIN>
ARCHIVED.ONLINE.1
@DIRECTORY,
@@ARCHIVED (FILES ONLY)
@@INVISIBLE (FILES ONLY)
@@
<CALVIN>
ARCHIVED.OFFLINE.1;OFF-LINE
Offline storage information is accessible through use
of DIRECTORY subcommands:
@DIR,
@@TIMES (AND DATES OF) TAPE-WRITE
@@
- 11 -
DEC Archive/virtual disk system Draft
EXEC Modifications
<CALVIN>
Tape Write
ARCHIVED.ONLINE.1 7-Nov-77 15:36
ORDINARY.BOTH.1 15-Jul-78 12:05
ORDINARY.DISK-ONLY.1 Never
ORDINARY.OFFLINE.1;OFF-LINE 10-Nov-77 18:22
@DIR,
@@DATES (OF) OFFLINE-EXPIRATION
@@DATES (OF) ONLINE-EXPIRATION
@@
<CALVIN>
Online expiration Offline expiration
ARCHIVED.ONLINE.1 13-Aug-78 5-Jul-88
ORDINARY.BOTH.1 4-Sep-78 14-Oct-78
ORDINARY.DISK-ONLY.1 30-Aug-78 N/A
ORDINARY.OFFLINE.1;OFF-LINE N/A 28-Sep-78
The current archival status of files may be checked by
the INFORMATION command. Suppose for the following example that
an archive request has recently been made for
ORDINARY.DISK-ONLY.1.
@INFORMATION (ON) ARCHIVE-STATUS (OF FILES) *.*
ARCHIVED.ONLINE.1 Archived
ORDINARY.BOTH.1 None
ORDINARY.DISK-ONLY.1 Archive requested
ORDINARY.OFFLINE.1;OFF-LINE None
ARCHIVE Subcommand to the DELETE Command
As noted before, deletion of archived files
intentionally involves some extra effort. The following example
shows the failure of a simple DELETE command applied to archived
files, and its success on ordinary files with or without contents
in offline storage.
@DELETE ARCHIVED.*,ORDINARY.*
ARCHIVED.ONLINE.1 %File has archive status
ORDINARY.BOTH.1 [OK]
ORDINARY.DISK-ONLY.1 [OK]
ORDINARY.OFFLINE.1 [OK]
On EXPUNGE, the three files ORDINARY.* will completely
leave the system, in the sense that no FDB will be left on disk
to point to the copy of the contents in offline storage.
To succeed in deleting archived files, the user must
supply the archive-specific subcommand.
- 12 -
DEC Archive/virtual disk system Draft
EXEC Modifications
@DELETE ORDINARY.*,ARCHIVED.*,
@@ARCHIVED (FILES INCLUDED)
@@
ORDINARY.BOTH.1 [OK]
ORDINARY.DISK-ONLY.1 [OK]
ORDINARY.OFFLINE.1 [OK]
ARCHIVED.ONLINE.1 [OK]
@EXPUNGE
[21 pages freed]
To delete invisible files, first make them visible,
then delete them, using the ARCHIVED (FILES INCLUDED) subcommand
if appropriate.
@SET FILE VISIBLE ARCHIVED.OFFLINE
ARCHIVED.OFFLINE.1 [OK]
@DELETE ARCHIVED.*,
@@ARCHIVED (FILES INCLUDED)
@@
ARCHIVED.OFFLINE.1 [OK]
ARCHIVED.ONLINE.1 [OK]
@EXPUNGE
[21 pages freed]
Naturally, UNDELETE may be used before the expunge if
desired. When archived files are expunged, a message is sent to
the user giving the archive pointer and related file information.
CONTENTS (ONLY) Subcommand to DELETE
The DELETE command is provided a new subcommand to
selectively expunge the disk contents of a file with a valid tape
pointer. UNDELETE will not work to recover the disk contents but
RETRIEVE is available with some delay.
@DELETE *.*,
@@CONTENTS (ONLY)
@@
ARCHIVED.ONLINE.1 [OK]
ORDINARY.BOTH.1 [OK]
ORDINARY.DISK-ONLY.1 %File has no tape pointer
ORDINARY.OFFLINE.1 %File is offline
The SET RETRIEVAL-WAIT Command
Normally, an attempt to read an offline file simply
fails, with an appropriate message. Under some circumstances,
such as batch files, it is preferable for the program to wait for
the retrieval rather than to fail because of the lack of a file.
- 13 -
DEC Archive/virtual disk system Draft
EXEC Modifications
This mode of operation can be set for a job using the following
command:
@SET RETRIEVAL-WAIT (For off-line files)
Note that invisible files are still inaccessible. The
mode of operation for the job returns to normal with the command:
@SET NO RETRIEVAL-WAIT (For off-line files)
Also note that retrieval-waits may be suppressed for
the whole system under various conditions as determined by the
system manager.
Copying and Renaming Files
There are several aspects of the COPY and RENAME
commands which involve offline ordinary files and online and
offline archived files. A file must have online contents to be
the source file for a successful COPY command. If, in addition,
it has a tape pointer, the tape pointer will not be carried over
to the target file, but will remain solely with the source file.
This treatment avoids needless duplication of tape pointers. By
the same token, the COPY command may be used to make a writable
copy of an archived file whenever it is needed.
Consider the following unsuccessful attempt to copy an
offline archived file.
@COPY ARCHIVED.OFFLINE.1
?File not found
Here the invisibility of the file prevented access to
it. The file may be made visible, but the resulting access leads
to an error message, since the file is offline:
@SET FILE VISIBLE ARCHIVED.OFFLINE
ARCHIVED.OFFLINE.1 [OK]
@COPY ARCHIVED.OFFLINE.1 (TO FILE) MUMBLE..1
ARCHIVED.OFFLINE.1 => MUMBLE..1 ?Source file is offline
An online archived file is successfully copied:
@COPY ARCHIVED.ONLINE (TO FILE) MUMBLE..1
ARCHIVED.ONLINE.1 => MUMBLE..1 [OK]
In this case, MUMBLE..1 is created with the same disk
contents as ARCHIVED.ONLINE but without archive status, so that
write and append are allowed according to the protection status.
- 14 -
DEC Archive/virtual disk system Draft
EXEC Modifications
Any visible file, archived or not, online or offline,
may be the source file for a RENAME command. As with disk-only
files, the RENAME command changes only the name string associated
with the source file, and never involves the contents of the
file.
The delete protection provided only for archived files
affects COPY and RENAME as follows. Attempts to overwrite
archived files, visible or invisible, with COPY or RENAME must
fail, as the following examples show:
@COPY tty: (TO FILE) ARCHIVED.OFFLINE.1
TTY: => ARCHIVED.OFFLINE.1 %Target file has archive status
@RENAME DISKFILE..1 (TO BE) ARCHIVED.ONLINE.1
DISKFILE..1 => ARCHIVED.ONLINE.1 %Target file has archive status
Similarly, an archived file may not be changed through
use of the APPEND command, but, if online, its contents may be
appended to another file.
Other command changes
The BUILD command has been modified to include the
OFFLINE-EXPIRATION-DEFAULT and ONLINE-EXPIRATION-DEFAULT
parameters. Both of these take either an explicit date or an
interval of days.
^ECREATE has the addition of the
OFFLINE-EXPIRATION-DEFAULT, ONLINE-EXPIRATION-DEFAULT, and the
ARCHIVE-ONLINE-EXPIRED-FILES commands. The first two commands
take either an explicit date or an interval of days, the third
merely sets/clears a mode bit.
Finally the ^ESET [NO] RETRIEVAL-WAITS command is used
by the operator to specify that retrievals are not being
processed. This is used to cause OPENF's to fail rather than
waiting for the file to be restored to disk. Explicit retrieval
requests are still queued in the duration of this command.
- 15 -
DEC archive system spec Draft
Implementation details
GALAXY modifications
Two major pieces of the Archive/Virtual Disk system
will be implemented within GALAXY. The first and most important
of these is the addition of a queue to hold retrieval requests
for archived and expired files. The second modification is the
addition of an internal queue and supporting code to implement
user notification by mail of archived files that have been
expunged from the disk.
The retrieval request queue maintained by GALAXY
contains all outstanding retrieval requests for the system. When
the retrieval of an offline file is requested via the ARCF JSYS,
the retrieval information is sent to GALAXY in an IPCF message,
and the request is entered into the queue. The retrieval request
queue is ordered by tape information (volume ID, tape saveset
number, and tape file number), and each new retrieval request is
inserted in the proper sorted position within the queue.
Ordering the retrieval requests in this way potentially reduces
the amount of tape mounting and dismounting that must be
performed by the operator when the retrieval requests are
eventually processed. Other operations on the retrieval queue,
such as listing and cancelling retrieval requests, are easily
implemented within the framework provided by GALAXY.
When a retrieval request is initially queued as a
result of the ARCF JSYS, it is inserted into the queue according
to the second set of tape numbers associated with the offline
file. When the retrieval queue is processed by DUMPER's RETRIEVE
command, requests can be requeued for two reasons: either the
request does not fall within the path specified by the operator
in the RETRIEVE command, or the retrieval request could not be
honored for some reason. In the first case, the request is
requeued according to the same set of tape numbers as before, so
that the request can be processed by some later invocation of the
DUMPER RETRIEVE command. In the latter case, the request is
requeued according to the first set of tape numbers, on the
assumption that the more recent tape had data errors or could not
be mounted. Requests that are requeued in this manner are
processed immediately; that is, by the current invocation of the
DUMPER RETRIEVE command.
To implement this protocol, GALAXY makes repeated
passes over the retrieval queue, passing to DUMPER each request
that needs "consideration." Those requests that DUMPER will
consider later are requeued with a time stamp which identifies
the current invocation of DUMPER's RETRIEVE command. On
subsequent passes through the queue, GALAXY skips over requests
that were time stamped during the current retrieval operation.
Requests that are requeued because of failure have their time
stamp field zeroed, so that GALAXY will reconsider the request
during the same retrieval operation. When a complete pass
through the queue results in no new requests to be considered,
GALAXY informs DUMPER that the queue has been exhausted.
- 1 -
DEC archive system spec Draft
Implementation details
GALAXY modifications
The deletion protection special to archived files
includes notifying users by mail whenever valid tape information
is discarded from the archive system. In this case, mail
notification provides a "paper copy" of the tape information
which can later be used to restore previously archived files to
disk. Whenever an archived file is expunged, or its archive
status is discarded, an IPCF message is sent to GALAXY, and the
tape pointers are stored in the notification queue. Periodically
(perhaps once per day) GALAXY processes the notification queue by
grouping the entries according to the owner of the expunged
files, and sending one message to each distinct owner giving the
name and tape information for each expunged file. Since the
notification queue is internal to GALAXY, users may not list,
modify, or cancel requests in this queue.
During the initialization that GALAXY performs after a
system reload, those retrieval requests which were made
"implicitly" by opening an offline file are cancelled. This
prevents potentially unnecessary transfers from offline to online
storage.
- 2 -
DEC archive/virtual disk system DRAFT
FDB modifications
Note that all mentioned bits and offsets in the following
descriptions are those used in our 3A development system and may
be modified during the integration into release 4 at DEC.
FBCTL (1)
FB%ARC (1B11) - File has archive status. Appropriate words in the
FDB (see below) specify where the file is archived. This bit is
modified by the ARCF JSYS, CHFDB cannot change it.
FB%INV (1B12) - File is invisible. Puts the file in the
"invisible" section of the directory. These files can be seen
only by using the G1%IIN option to GTJFN. FB%INV can be modified
with CHFDB by anyone with owner access.
FB%OFF (1B13) - File is offline. This is changed by ARCF when it
removes the contents from disk, and when it restores the contents
to disk. FB%OFF cannot be changed with CHFDB.
FBBBT (22) (Formerly FBBK3)
The left half of FBBBT is used for bits while the right half
contains the number of pages in the file when the contents were
deleted from disk. The following describes the bits in the left
half of FBBBT.
AR%RAR (1B1) - User request for a file to be archived. This can
only be modified by ARCF.
AR%RIV (1B2) - System request for an involuntary migration of a
file. The ARCF jsys is used to modify this bit, and the caller
must have enabled WHEEL or OPERATOR capabilities.
AR%NDL (1B3) - Do not delete the contents of the file from disk
when the archival is complete. Set by ARCF.
AR%NAR (1B4) - Resist involuntary migration. This bit is a note
from the user to the system policy module asking the file not be
moved offline if possible. Set and cleared by ARCF.
AR%EXM (1B5) - File is exempt from involuntary migration.
Modified only by ARCF and enabled WHEEL or OPERATOR capabilities
are required.
AR%1ST (1B6) - First pass of an archival/collection run is in
progress. Modified by CHFDB and used only by DUMPER. Requires
enabled WHEEL or OPERATOR capabilities.
AR%RFL (1B7) - Restore failed. Set by ARCF to indicate to another
process that the restore it is waiting on has failed.
- 1 -
DEC archive/virtual disk system DRAFT
FDB modifications
AR%PSZ (1B18-1B35) - The right half of FBBBT is used to store the
number of pages in a file when the contents were removed from
disk. This value is set by the DELF JSYS (DF%CNO option) and can
be read with CHFDB or ARCF.
FBNET (23) (Formerly FBBK4)
On-line expiration date and time. Specifies a date and
time at which a file is considered expired, or specifies an
interval (in days) after which the file is considered expired.
Modified by SFTAD.
FBTDT (31) (New)
Archive or collection tape-write date and time. This is
the date and time (in internal format) of the most recent tape
write to the archives or virtual disk. Modified by ARCF and
requires enabled WHEEL or OPERATOR capabilities.
FBFET (32) (New)
Offline expiration date and time. Specifies the date
and time (or interval) after which a file in the archives or on
virtual disk is considered expired. Used for tape recycling.
Modified by SFTAD.
FBTP1 (33) (New)
FBTP1 contains tape ID for the first archive or
collection run. Modified only by ARCF and enabled WHEEL or
OPERATOR capabilities are required.
FBSS1 (34) (New)
FBSS1 contains the saveset and tape file numbers for
the first tape. The left half is the saveset number the file is
recorded in, the right half is the tape file number within that
saveset. Modified only by ARCF and enabled WHEEL or OPERATOR
capabilities are required.
FBTP2 (35) (New)
Tape ID for second archive or collection run. (as with
FBTP1)
FBSS2 (36) (New)
Saveset and tape file numbers for the second archive or
collection run. (as with FBSS1)
- 2 -
DEC archive/virtual disk system DRAFT
JSYS descriptions
Modifications to existing JSYSes
CHFDB
Some restrictions have been placed on CHFDB. CHFDB can
no longer modify FBBK3 and FBBK4 with the exception of several
bits in FBBK3 (now FBBBT). These modifiable bits are: B0,
AR%1ST, and the right half of FBBBT. Further, FB%DEL may not be
set if FB%ARC (in FBCTL) is on.
CRDIR
Default online expiration date and time and default
offline expunge date and time have been added. These are
specified by way of the .CDDNE (20) and .CDDFE (21) words in the
CRDIR argument block. These date and times may be explicit date
and times (internal format) or an interval (in days). In either
case, the specified date/interval may not exceed the system
maximum. These parameters are taken from the argument block if
CD%NED (1B2) or CD%FED in .CDLEN (1B3) are set. If a new
directory is being created and these parameters are not
specified, the system default is used.
An unprivileged user may modify his defaults, but only
to a value less than or equal those which are currently specified
or the system maximum, whichever is greater.
Additionally, a new mode bit (CD%DAR 1B3), has been
defined. This bit specifies that when online expiration is
reached, the files concerned should be archived rather than
migrated to virtual disk.
DELDF
When a file with archive status is deleted and
expunged, DELDF sends a message to galaxy. The message contains
all archive status information which includes tape information as
well as the present file name, user who expunged the file, and
the time at which it was expunged. These messages (of the IPCF
flavor) will be processed by GALAXY at a later time.
DELF and DELNF
DELF and DELNF fail (DELXnn) for files with archive
status. For the delete to succeed, DF%ARC (1B4) must be on in
AC1. This bit specifies permission to delete files with archive
status. A second bit has been defined in DELF and DELNF. This
bit, DF%CNO (1B5), causes only the contents of the file to be
deleted and immediately expunged. The file's FDB and name will
remain as it was (with the exception of the page count and page
table address). A DELF or DELNF with DF%CNO set will fail if
AR%NDL is set in FBBBT or if a complete set of tape backup
information is not in the FDB.
- 3 -
DEC archive/virtual disk system DRAFT
JSYS descriptions
GTJFN and GNJFN
GTJFN has only one area of modification. It must now
ignore files that have the invisible bit (FB%INV) set. GTJFN has
a new bit (G1%IIN 1B5) which specifies that GTJFN should ignore
the invisible status of a file. G1%IIN is given to GTJFN in .GJF2
of the extended argument block. GTJFN will return a bit (GJ%GIV
1B17) (like GJ%GND for deleted files) for use by GNJFN.
JFNS
JFNS can now output a new attribute ";OFFLINE". The
output of this can be requested by the JS%OFL (1B29) bit in AC3.
OPENF
OPENF has been modified to deal with archived and/or
offline files. The following table describes OPENF's action for
each possible state of a file.
Archived Non-archived
OPENF Access| Online | Offline || Online | Offline
------------|----------|-------------||------------|-----------
Read | Ok | Fail/Hang || Ok | Fail/Hang
Write | Fail | Fail || Ok (discard implied)
Append | Fail | Fail ||Ok (discard | Fail/Hang (discard
implied) implied)
The fail cases will all return an appropriate error
message (OPNXnn). The Fail/Hang cases will return an error for
fail or wait until the OPENF can be successfully completed. Fail
or hang will be determined by the setting of OF%NWT (never wait
for file), OF%RAR (retrieve file if necessary), or the setting of
the job default (retrieve file if necessary). If OF%NWT is set on
the OPENF call, OPENF will always fail (in the Fail/Hang cases).
If OF%RAR or the job default is set, the OPENF will wait for the
file to be retrieved and then complete successfully. In the Ok
(discard implied) cases, tape pointers, if any, for the file are
discarded.
RFTAD
Three new words can be returned by RFTAD. They are
.RSTDT (4) (tape-write date and time), .RSNET (5) (online
expiration date and time), and .RSFET (6) (offline expiration
date and time). Of these, .RSNET and .RSFET may be a date and
time (in internal format) or an interval (specified in days).
Intervals are limited to 2**18-1 days (a half word).
RNAMF
- 4 -
DEC archive/virtual disk system DRAFT
JSYS descriptions
Renaming a file with tape information carries the tape
information to the new file name. Renames which would effectively
destroy a file with archive status fail.
SETJB
A new function code (.SJDFR (6) set job default
retrieval) has been added to enable a user to override the system
default for OPENF. .SJRFA (0) specifies that OPENF should always
fail if the file's contents are not online. .SJRWA (1) specifies
that OPENF should wait for ARCF to restore the contents of the
file to disk. .SJRFA is the system default.
SFTAD
Three additions have been made to SFTAD. They are
.RSTDT (4) (tape-write date and time), .RSNET (5) (online
expiration date and time), and .RSFET (6) (offline expiration
date and time). Modification of .RSTDT requires enabled WHEEL or
OPERATOR capabilities. .RSNET and .RSFET may be either a date and
time (internal format) or an interval (specified in days).
Intervals are limited to 2**18-1 days. Date and times and
intervals may not exceed system/directory maximum.
SMON
New function codes .SFMCY (54), .SFACY (55), and .SFRTW
(56) have been added. .SFMCY specifies the maximum offline
expiration (tape recycle time) for ordinary files. .SFACY
specifies the expiration period for archive files (tape recycle
period). .SFRTW is used to set/clear the NO RETRIEVAL-WAITS flag
in the monitor. When set this specifies that those retrievals
waiting for the retrieval should fail rather than hang.
TMON
.SFMCY and .SFACY have been added to read the maximum
offline expiration time for ordinary and archived files. These
values are the recycle times for tapes containing either ordinary
or archived files.
New Jsyses
ARCF Jsys
The ARCF jsys will perform all operations pertaining to
the archive and virtual disk systems. This includes requesting
archival and migration, requesting retrieval, and setting archive
status and tape information for a file.
- 5 -
DEC archive/virtual disk system DRAFT
JSYS descriptions
Accepts:
1: JFN
2: Function code
3: Function dependent, normally 0
ARCF
Returns:
+1 Always
Functions:
.ARRAR (0) - This function sets/clears AR%RAR in
.FBBBT, a user request for archival. The value .ARSET in AC3 (1)
will request an archive while .ARCLR (0) will clear the request.
Setting AR%NDL with .ARSET in AC3 requests the contents of the
file not be flushed from disk upon archival.
.ARRIV (1) - This function sets/clears AR%RIV in
.FBBBT, a system request to migrate a file off disk. The value
.ARSET in AC3 will request migration while .ARCLR will clear the
request. The caller must be an enabled WHEEL or OPERATOR.
.AREXM (2) - Set/Clear AR%EXM (exempt from involuntary
migration) in .FBBBT. A 0 in AC3 clears AR%EXM while a 1 in AC3
sets it. Requires enabled WHEEL or OPERATOR capabilities.
.ARRFR (3) - Request a file's contents be restored to
disk. Normally .ARRFR returns without waiting for the contents
of the file to be restored to disk.
Options in AC3 are:
AR%NMS (1B0) - Do not send a message to the requestor upon
restoration.
AR%WAT (1B1) - Wait (hang in the ARCF jsys) until the file is
restored.
.ARDIS (4) - Discard tape information. Clears FB%ARC
(if set), .FBTP1, .FBTP2, .FBTSN, .FBTFN, and .FBTDT. The file
must be online for the function to succeed. AC3 option bits:
AR%CR1 (1B0) clears only first run information. AR%CR2 (1B1)
clears only pass two information. Requires WHEEL or OPERATOR
capabilities to use AR%CR1 or AR%CR2 individually.
.ARSST (5) - Set tape information for a file. It
requires WHEEL or OPERATOR capabilities. AC3 points to an
argument block as follows:
0: (.AROFL) Option bits
AR%O1 (1B0) - Set run 1 information
AR%O2 (1B1) - Set run 2 information
- 6 -
DEC archive/virtual disk system DRAFT
JSYS descriptions
AR%OFL (1B2) - Delete disk contents of file
when done. Requires bothsets of
tape info be set.
(.ARGST returns 0 for word .AROFL)
AR%ARC (1B3) - Set FB%ARC (archive the file)
AR%CRQ (1B4) - Clear archive and/or migration
requests (AR%RAR and AR%RIV)
1: (.ARTP1) Tape 1 ID
2: (.ARSF1) XWD TSN 1, TFN 1 see footnote(1)
3: (.ARTP2) Tape 2 ID
4: (.ARSF2) XWD TSN 2, TFN 2
5: (.ARODT) GTAD of tape-write; 0 implies
present time
6: (.ARPSZ) Number of pages in the file.
This word is only returned
by .ARGST (.ARSST cannot set it)
This function (.ARSST) is used to set information for the first,
second, or both tape runs. AR%O1 and AR%O2 are used together when
restoring files to disk via DUMPER's RESTORE command.
.ARRST (6) - Restore the contents of a file to disk.
AC3 is a JFN for a temporary file that contains the data for
currently offline archived file. After copying .FBADR, .FBBSY and
.FBSIZ the temporary file is deleted. Both files must be on the
same device or structure and enabled WHEEL or OPERATOR
capabilities are required.
.ARGST (7) - Get tape information for file. AC3 points
to an arguement block to be filled. The format is the same as for
.ARSST.
.ARRFL (10) - The restore for this file has failed. Set
AR%RFL in FBBBT to notify a waiting process that the retrieval
request cannot be completed. Requires enabled WHEEL or OPERATOR
capabilities.
.ARNAR (11) - Resist involuntary migration. Sets or
clears AR%NAR in .FBBBT. .ARSET in AC3 will cause resist to be
set, while .ARCLR will clear it. Error returns:
CAPX1 - WHEEL or OPERATOR capabilities required.
ARGX02 - Invalid function code
ARCFX2 - File already has archive status
_________________________________________________________________
(1) TFN is for Tape File Number. It is a number recorded on the
tape with the file which is its index in a given save set. TSN
is for Tape Saveset Number. The saveset number on a tape is also
recorded in the save set name.
- 7 -
DEC archive/virtual disk system DRAFT
JSYS descriptions
ARCFX3 - Cannot perform ARCF functions on non-multiple directory
devices
ARCFX4 - File is not online
ARCFX5 - Files are not on the same device or structure.
ARCFX6 - File does not have archive status
ARCFX7 - Invalid parameter for .ARSST
ARCFX8 - Archive not complete
ARCFX9 - File not offline
ARCX10 - Archive prohibited
ARCX11 - Archive requested, modification prohibited
ARCX12 - Archive requested, delete prohibited
ARCX13 - Archive system request not completed
ARCX14 - Restore failed
ARCX15 - Migration prohibited
ARCX16 - Cannot exempt offline, archived, or archive pending
files
ARCX17 - FDB improper format for ARCF
ARCX18 - Retrieval wait cannot be fulfilled for waiting process
ARCX19 - Migration already pending
- 8 -
DEC archive system spec Draft
Implementation details
Policy module
The policy module for the Archive/Virtual Disk system,
REAPER, is intended for use by system administrators to increase
free disk space. It can free up disk space in several ways: by
marking for involuntary migration files which have not been
referenced within a specified period of time; by deleting the
disk contents of archived or migrated files that have not been
referenced within a specified period; and by trimming directories
that are over their permanent disk allocations.
A typical use of REAPER is to give a number of
commands, one for each operation to be performed. When all
desired operations have been specified, the BEGIN command is
given, specifying the path of files to be considered. Following
is a list of REAPER's commands, with a description of each.
BEGIN (Processing Files) - Specifies which files to
process in a scan over the disk system. Normally, one specifies
PS:<*>*.*.*, but one could specify other paths, such as
PS:<*>*.REL.*, or FOO:<TST*>*.*.*, etc. This command also
initiates the scan over the disk system, so it should only be
given after all desired options have been specified. After the
specified files have been processed, REAPER prints the total
number of pages reclaimed and then exits.
DELETE (Disk contents) - Specifies that files existing
both on disk and on an Archive/Virtual Disk tape are to have
their disk contents deleted if the file has not been referenced
within a specified time period. The time period is specified by
the PERIOD command. If the DELETE command is not given, the disk
contents of these files remains online.
EXIT - Exits from REAPER, without processing any files.
Normally, the BEGIN command is used to exit from REAPER after
processing some files.
LIST (To file) - Specifies an output file for the
listing produced by REAPER. If this command is not given, the
listing is written to the file DSK:ARCHIVE.LIST.
MIGRATE (Old files) - Specifies that files not
referenced within a specified time period are to be marked for
involuntary migration. The time period is specified by the
PERIOD command. If the MIGRATE command is not given, these files
are not marked for migration and remain online.
ORDER (For trimming) - Specifies the order in which to
mark files for migration when TRIMming a directory that is over
its permanent allocation. This order is a list of file
specifications, such as: *.TMP.*, *.LST.*, *.REL.*. This order
list is only used if the TRIM command is given.
- 1 -
DEC archive system spec Draft
Implementation details
Policy module
PERIOD - Specifies an age limit in days within which
files will not be MIGRATEd or have their disk contents DELETEd.
The usual period is 60 days. This means that files older than 60
days will be marked for involuntary migration (if the MIGRATE
command is given) or will have their disk contents deleted (if
the DELETE command is given.) If the PERIOD command is not
given, a default period of 60 days is assumed.
SCAN (Only) - Specifies that no files should actually
be DELETEd, MIGRATEd, or TRIMmed, but that only a listing should
be produced. If the user is not an enabled wheel or operator,
this is the default mode of operation (see below).
SKIP (Directories) - Specifies a list of directories
that are not to be considered when performing the DELETE,
MIGRATE, and TRIM operations. This should be a select group of
directories; for instance, system directories and other special
directories should be included in this list. If the SKIP command
is not given, all directories encompassed by the filespec given
in the BEGIN command will be processed.
TAKE (Command file) - Reads a file of REAPER commands.
This file can contain a list of directories to be SKIPped, an
ORDER list, a standard PERIOD, etc. Although the BEGIN command
can be included in the command file, it probably should not be.
The default TAKE file is SYSTEM:REAPER.CMD. This command file
can contain the REAPER commands that implement a site's policies
toward involuntary migration.
TRIM (Directories) - Specifies that directories over
their permanent disk allocations should be trimmed down to size.
The order in which files are marked for migration can be
specified in two ways. If the directory under consideration
contains a file named MIGRATION.ORDER, then the contents of that
file constitutes the order list for that directory. If no such
file exists, then the order list is taken as given in the REAPER
ORDER command. Thus, a per-directory order list takes precedence
over a system wide order list. REAPER makes several passes over
a directory attempting to trim it back to its permanent
allocation. First, all files in the order list are considered
(subject to the PERIOD in effect) except those having the "resist
migration" or "exempt from migration" bits set. If there is no
order list, then this pass is skipped. If the first pass fails
to bring the directory under its allocation, all files in the
directory (*.*.*) are considered, except those marked "resist" or
"exempt". If necessary, these two passes are then repeated, this
time ignoring the "resist" bit.
When the BEGIN command is given, REAPER processes the
specified file path on a directory by directory basis. Within
each directory, the MIGRATE, DELETE, and TRIM operations are
- 2 -
DEC archive system spec Draft
Implementation details
Policy module
performed, in that order. (TRIM is performed last to avoid
unnecessary file migration.) Two other operations are performed
as well: temporary (;T) files which are older than the PERIOD
are deleted and expunged; and the directory is purged of FDBs
which point to offline files which have reached their offline
expiration date. If Control-A is typed while the BEGIN command
is underway, REAPER prints the name of the directory it is
working on at the moment.
If the user is not an enabled wheel or operator, REAPER
does not prompt for commands. Instead, it simply asks for the
name of the listing file, and the file path which is to be
scanned. REAPER then processes this path as if the commands
"TAKE SYSTEM:REAPER.CMD" and "SCAN" were given, producing a
listing of those files that would be taken given the present
system policies toward file migration.
- 3 -