Trailing-Edge
-
PDP-10 Archives
-
BB-H311D-RM
-
documentation/ci-info.memos
There is 1 other file named ci-info.memos in the archive. Click here to see a list.
THIS FILE CONTAINS:
------------------
o Description of CI20 support (CI-SUPPORT.TXT)
o KLIPA (CI20) driver description (KLPDES.MEM)
o KLIPA (CI20) driver functional specification (KLPFUN.MEM)
o CI20 architechtural specification (LCG-CI-PORT-ARCHITECTURE.SPC)
CI-SUPPORT.TXT
This is an attempt to describe the specific CI products in release 6.
Included in this description are the appropriate configuration
guidelines and the rationale for any restrictions.
Release 6 of TOPS-20 is the introductory release for the CI
and certain CI products. The significant new features afforded
by this new technology are CFS-20 and HSC-50.
HSC-50 support
The HSC-50 is the corporate CI-based disk controller. Each HSC-50
will support up to twenty-four (24) disks. The supported disk
types are RA81 and RA60 (if available in time for testing and release).
Each disk may be connected to one or two HSC-50 controllers. If
a disk is dual-ported, it will be "owned" by one of the HSC-50s
at any time. That is, even though the disk is connected to two
controllers, it may be accessed by only one of them and therefore
the dual-porting does not enhance throughput but simply
increases availability.
TOPS-20 release 6 will support up to three HSC-50 controllers
and up to sixty-four HSC-based disks. This limitation is
imposed because of the unknown nature of the CI under
extremely heavy demand and because of the limited hardware
available to us during load-test and field-test. If we are
able to conduct measurements that show we could support more
controllers and/or disks, we will consider relaxing this
constraint.
Furthermore, we recommend that as many of the disks as possible
be dual-ported, and therefore that each installation have at
least two HSC-50 controllers. This recommendation reflects
the novelty of the HSC-50 and of the complexity of the hardware,
software and protocols required to support it. We anticipate
that HSC software crashes will be one of the leading causes
of failures (TOPS-20 crashes being the other dominant factor) in
a CI environment, and consequently dual-porting of disks
will ameliorate the lost time from these crashes. Although
an HSC software crash should be infrequent, perhaps two or
three a week, the long reload time for the software (more than
five minutes) and the importance of the disk data dictate
measures to guarantee the availability of the disks. As with
any product, it will become more reliable with time and use. However,
the dual-porting should be an attractive item for our customers
throughout the lifetime of the product.
BOOT will not support CI devices. For this and other reasons we
do not support using HSC-based disks as the public structure.
In particular, specific support in TOPS-20 required to
fully support HSC disks as the public structure is not being done.
Among these tasks are:
1. Making TOPS-20 less dependent on accessing the public
structure so that an HSC crash will be less likely
to crash TOPS-20 (this is particularly important if the
structure is not dual-ported).
2. Insuring that the resident monitor, perhaps with the
aid of BOOT, can reload the KLIPA microcode.
Even if we were to do these tasks, we could still not boot the
system from an HSC disk as BOOT would not support SCA and MSCP.
In general, we feel that the quailty of the product with HSC-based
disks as the public structure is less than desirable and therefore
not to be supported.
CFS-20
CFS-20 is the first stage in TOPS-20 loosely coupled multi-processing.
With this product, one is able to connect two KL-10s to a single
CI so that the KL-10s share a single file system. The file sharing
works in three ways:
1. Disks connected to HSC-50s are directly accessible by
each KL-10 and therefore sharable.
2. Massbuss disks that are ported to each processor are
also sharable.
3. Massbus disks connected to only one of the KL-10s is
sharable by virtue of each of the system's MSCP server.
Each of these will be explored in turn.
Originally, CFS-20 was to support up to four KL-10 processors
in a single configuration. This ambitous goal has been
modified in order to reduce the risks of producing release 6,
both in terms of development time and testing time, and
in hardware resources required to develop the product. Engineering
remains committed to four-processor CFS, however, and every
effort will be made to ensure that the FRS release 6 is capable
of supporting three and four node CFS field-test.
Sharing of HSC disks is straight-forward as the basic product
was designed for just such an application. However, we know very
little about how an HSC behaves when it is serving two (or more)
processors. In particular, TOPS-20's use of "single page requests"
may create unacceptable overhead in the HSC whereas batching
requests into "multi-page" requests may lessen the overhead. It
is unlikely that we will make this determination early enough
to be able to make the appropriate changes to TOPS-20.
A Massbus disk, RP06 or RP07, is shared by connecting it to
both of the CFS systems. Release 6 of TOPS-20 has many improvements
in the handling of dual-ported disks, so that the operation of
this configuration should be relatively smooth. However,
there are some important considerations:
1. TOPS-20 release 6 does not have code to recover
the latency optimization decisions made for dual-ported
disks. Therefore, if a system has decided on a specific request
because it exhibits the least latency, and the other system
intervenes before the transfer is started, the selected transfer
will still be the next one issued to the drive. Therefore,
when the drive is again available to the first system, there
may be considerable latency in completing the request. This
is important for if the committed request is a data transfer
request, and if the drive must perform an "implied seek"
before starting the transfer, the entire Massbuss will be committed
to this transfer during the seek latency time. So, not only
may the throughput of the drive be affected, but all devices
on that Massbus will be implicitly affected. And, of course, the
time lost due to port contention may be significant.
2. Preliminary mesurements for RP07s show that two systems
accessing the same drive achieve a 50% increase in disk
throughput over a single system accessing the drive. However,
the same experiment performed for an RP06 shows a 50%
decrease in disk throughput. If this preliminary data holds
true, dual-porting RP06 drives may reduce the overall IO
efficiency of the systems.
3. We've yet to conduct measurements of channel throughput in
the case of multiple devices, some ported to two system, on the
same Massbus. This is an important datum in the evaluation
of shared Massbus disks.
A Massbus disk may also be shared by using the MSCP server provided
in TOPS-20. The MSCP server implements the passive role of
the MSCP protocol and therefore looks very much like an HSC-50
disk controller to other systems. The MSCP server allows another
KL-10 to access the local disks of a KL-10 system.
In general, we recommend that disks that are to be shared
by the CFS systems be directly accessible to each system. However,
in some cases this is impractical, such as with the disk that
is connected to the RH11. For these devices, the MSCP
server is the only way the drive can be shared.
Also, if one of the ports of a disk drive is broken, the MSCP server
may be used to achieve sharing, at the cost of increased processor
overhead to support the activity.
Using the MSCP server will increase processor load. This self-evident
statement is nonetheless important to note. We've yet to measure
the overhead incurred in using the server, and this must be
done.
The potential benefit of using the server for sharing disks is that
the disk latency algorithms will not be compromised. However,
without comparative measurements of these two costs, we cannot
evaluate the proper configurations.
CFS-20 implements a distributed control scheme. This has a number
of important implications, but the most significant is that
each processor is responsible for managing the data stuctures
on the disk structures it uses. This means that writing files has
implicitly more overhead than reading files, as writing
involves managing the structure bit table (It turns out that
any scheme that allows sharing of structures among processors
would exhibit the same phenomenon, viz. writing incurs
more overhead than reading).
The implication of this aspect of CFS depends a great deal on
the applications that are sharing the structure. Data base
kinds of applications, that is programs that read and write
a common file over a long period of time, may well incur
considerable overhead and a concomitant reduction in disk
throughput. In fact, preliminary measurements of such an application
(PAGES) shows a 30% to 60% reduction in disk bandwidth when writing
to the shared structure.
Applications that are closer to time-sharing use, where files
are written and closed in short interactions, should exhibit
a much lower level of overhead due to write sharing. For example,
our intended use of CFS to share the monitor sources between two
processors should not show a significant reduction in disk
throughput.
The reduction in disk throughput seen in shared write environments
is a combination of port contention, latency increase, and
additional reads and writes of the structure bit table. In fact,
the total IO to the disk is increased, but a significant amount
of the IO is for coordinating control rather than writing data.
Of course, it can rightly be argued that no use of a shared structure
is purely one kind of application or the other. Therefore, it is
unlikely anyone will see the drastic penalties observed with PAGES.
PS: structures are sharable. However, the following caveats
apply:
1. No drive that is part of PS: should be ported
to another system.
2. No files on PS: should be critical to the operation
of any other system. Therefore, the files that
are shared should be ones infrequently needed by
other systems.
The first item derives from the difficulty in performing maintenance
(CHECKD) on PS:. Also, the CFS error recovery schemes assume
that PS: is always accessible, and therefore the techniques
used to prevent uncontrolled sharing of dual-ported Massbus
disks do not apply to PS:. Although there are solutions to these
problems, we will not have time to implement them for release 6.
The second item derives from the importance of PS: to its
"home system". It is foolish to jeopardize the performance of
a system because another one is interfering with swapping.
Also, since PS: drives may not be ported to both systems, if
the home system for a PS: is down, the other system has no
way of accessing the files on that PS:.
CFS-20 will recover from CI failures by "splitting" the system
automatically. This splitting is accomplished by declaring
sharable resources to be unavailable to the system
no longer on the CI. We need to conduct extensive experiments
to insure this works properly.
CFS-20 includes a TOPS-20 to TOPS-20 CI/DECnet product. This
limited-use DECnet product is intended for use in CFS-specific
applications. In particular, we envision CI/DECnet:
1. As the initial, and for now only, IPC mechanism
between the CFS processors. As such, applications
being written for or modified for the CFS environment
will be able to provide inter-process signalling and
coordination via CI/DECnet much as IPCF is used now.
2. As the principal tool for adapting GALAXY to
the CFS environment. Although release 6 will not
provide a single, CFS-wide operator sysytem, CI/DECnet
will allow us to implement such a system.
We anticipate that subsequent releases of TOPS-20 will provide
CFS-wide IPCF and ENQ/DEQ, and therefore applications will
be able to use the conventional inter-process signalling and
coordination now used in TOPS-20.
NOT SUPPORTED or NOT BEING DONE
TOPS-20 release 6 will not support the following:
1. KL-10s and VAX processors on the same CI.
2. HSC-50 tapes (TA78s)
3. Using any of the HSC disks as the public structure.
4. Three and four node CFS
5. Distinguishing disks ported to the RSX20F front-end from
disks ported to other systems. This means that a CI
failure could result in TOPS-20 not being able to proceed
without manual intervention. However, the MBTF of the KLIPA
is expected to be so much greater than thr MBTF of the KL-10
processor, that such a failure seems unlikely.
6. Other LCS features, such as: inter-system IPCF, inter-
system ENQ/DEQ and DBMS support.
SUMMARY
1. HSC-50 disks should be dual-ported
2. RP07 dual-porting seems acceptable, RP06 may not be (RP20 untested).
3. MSCP server use should be limited, but measurements needed.
4. Measurements needed on channel throughput with dual-ported disks.
5. Shared writing of CFS structures needs careful planning.
6. Measurements needed of how an HSC behaves supporting multiple hosts.
7. PS: drives must not be ported to two KL-10s.
8. PS: file sharing should be limited.
9. Fault insertion testing needed.
KLPDES.MEM
1.0 OVERVIEW
The KLIPA driver is the lowest level of support for the
CI-20. The driver is responsible for coordinating all
access to the device and for implementing the queued
interface between the device and the operating system. It
also must provide services for diagnostic functions as well
as for "DMA" functions specified in the SCA layer.
The driver is part of the PHYSIO system. The decision to
place the KLIPA within the bounds of PHYSIO was made in
order to minimize the effort within TOPS-20 to interface to
the KLIPA. The KLIPA is the first device in the PHYSIO
system that is not an RH20 and therefore we will have to
"teach" PHYSIO how to manage such devices. It seemed
appropriate to make this alteration to PHYSIO so that any
other additions to the KL device repertoire will be easily
integrated into the monitor.
This document describes the structure of the KLIPA driver,
the specific services it performs and its relationship to
the other CI-related services and to the monitor in general.
1.1 References
1. PHYKLP functional specification (R60SPC:KLPFUN).
10-AUG-83 (Grant/Miller)
2. LCG CI Port Architecture Specification, 11-July-83
(Keenan)
3. IPA20-L Error Spec, 30-Sept-82 (Holewa)
4. Systems Communication Architecture, 20-July-82
(Strecker)
1.2 Purpose Of This Document
This document describes how PHYKLP implements the services
described in the functional specification [1]. Ideally,
this document is a template for the implementor to follow
and therefore will serve as a "road map" to the code.
Portions of this document will appear as commentary in the
source module so that maintainers will benefit from this
design plan.
Page 2
1.3 Driver Functions
The KLIPA driver is a TOPS-20 device driver. It is
generally a passive service that responds to requests from
higher level components and simply performs the
device-specific operations required by the requests.
Because the KLIPA is a smart and somewhat independent
device, the driver must perform some actions on its own
without being directed by higher-level services. However,
these operations are transparent to the rest of the monitor.
The driver is responsible for maintaining the port-to-port
virtual circuits to other CI drops. That is, the driver
must locate the other CI drops, and negotiate a "three-way
handshake" initialization in order to establish a "reliable
communication path" between the nodes. In support of this
operation, the driver polls the CI drops from time-to-time
to detect any changes to the CI configuration. It turns out
that the other nodes are also polling the CI, and therefore
nodes may recognize one another during either of the polling
actions. Whereas some nodes will only allow the
port-to-port circuit when its poller detects another node (a
la the HSC-50), the TOPS-20 KLIPA driver will cooperate with
any node that wishes to establish a port-to-port circuit.
The only active part of the driver is the polling operation
and the concomitant "virtual circuit initialization"
operation. All other data transfers, including datagrams,
messages and DMA transfers are requested explicitly by SCA.
The interfaces to PHYKLP and SCA are diagrammed below:
+----------+ +--------------+
! SCA ! ! PHYSIO !
+----------! +--------------+
\ ^ config change / poll
\ \ data received / interrupt
DMA \ \ /
messages \ \ /
datagrams \ \--+----------------+</
---> ! PHYKLP !
+-------+--------+
! device interface
+-------+--------+
! IPA20 !
+----------------+
As can be seen, the PHYSIO interface is used only to drive
the interrupts and as a source of polling calls. Unlike
other PHYSIO drivers, PHYKLP is not called by PHYSIO to
perform register reads, register writes and other
"housekeeping" functions. PHYKLP appears to PHYSIO as a
device other than an RH20 and one that is relatively
Page 3
self-contained.
1.4 Driver Components And Data Structures
The driver, PHYKLP, maintains the following structures
. The port control block (PCB). This is a
"communication" area between the KLIPA and the
driver. The PCB contains interlock words, queue
headers and control information necessary for the
KLIPA to monitor protocol. It is described fully in
[2].
. System blocks. A system block is a data structure
describing a node on the CI. Each system block
contains information about the node as well as the
state of the connection between this node and the
remote node. The system block is the fundamental
data structure for maintaining the virtual circuit
between two nodes. There is a unique system block
for each of the nodes on the CI.
The fundamental pieces of the driver are:
. Poller. This is the code that attempts to
recognize new nodes appearing on the CI and to
detect and report path failures in the CI. It is
called as part of the scheduler.
. Interrupt service. This code is called by PHYSIO
when the KLIPA interrupts the monitor. Most of the
real work in the driver happens in the interrupt
service routine, including the crux of the
port-to-port VC initialization. The interrupt
service routine is responsible for detecting device
errors, protocol violations and system errors that
affect the KLIPA.
. Message and datagram transmission. This service
is used by SCA to send messages and datagrams.
. DMA services. These routines are used to create
and validate CI buffers and to provide the software
required portions of the DMA message service.
. diagnostic services. These routines support the
DIAG% and SCS% functions specified for the KLIPA.
The remaining sections of this document will explore each of
the pieces described in the overview as well as provide an
operational description of PHYKLP.
Page 4
2.0 DATA DESCRIPTIONS
2.1 PCB
The PCB, port control block, is the interface communications
region between PHYKLP and the KLIPA. PHYKLP initializes the
PCB and then identifies its physical address to the KLIPA.
The PCB is as follows:
+=======================================================+
.PBBDT | Buffer Descriptor Table Starting Address |
+-------------------------------------------------------+
.PBQEL | Queue Entry Length |
+-------------------------------------------------------+
.PBQ3I | Command Queue 3 Interlock |
+-------------------------------------------------------+
.PBQ3F | Command Queue 3 FLINK |
+-------------------------------------------------------+
.PBQ3B | Command Queue 3 BLINK |
+-------------------------------------------------------+
.PBQ2I | Command Queue 2 Interlock |
+-------------------------------------------------------+
.PBQ2F | Command Queue 2 FLINK |
+-------------------------------------------------------+
.PBQ2B | Command Queue 2 BLINK |
+-------------------------------------------------------+
.PBQ1I | Command Queue 1 Interlock |
+-------------------------------------------------------+
.PBQ1F | Command Queue 1 FLINK |
+-------------------------------------------------------+
.PBQ1B | Command Queue 1 BLINK |
+-------------------------------------------------------+
.PBQ0I | Command Queue 0 Interlock |
+-------------------------------------------------------+
.PBQ0F | Command Queue 0 FLINK |
+-------------------------------------------------------+
.PBQ0B | Command Queue 0 BLINK |
+-------------------------------------------------------+
.PBRQI | Response Queue Interlock |
+-------------------------------------------------------+
.PBRQF | Response Queue FLINK |
+-------------------------------------------------------+
.PBRQB | Response Queue BLINK |
+-------------------------------------------------------+
.PBMFI | Message Free Queue Interlock |
+-------------------------------------------------------+
.PBMFB | Message Free Queue FLINK |
+-------------------------------------------------------+
.PBMFB | Message Free Queue BLINK |
+-------------------------------------------------------+
.PBDFI | Datagram Free Queue Interlock |
+-------------------------------------------------------+
Page 5
.PBDFF | Datagram Free Queue FLINK |
+-------------------------------------------------------+
.PBDFB | Datagram Free Queue BLINK |
+-------------------------------------------------------+
.PBRSV | Reserved to Port |
+-------------------------------------------------------+
.PBER1 | Error Word 1 |
+-------------------------------------------------------+
.PBER2 | Error Word 2 |
+-------------------------------------------------------+
.PBPBA | PCB Base Address |
+-------------------------------------------------------+
.PBPIA | PI Level |
+-------------------------------------------------------+
.PBIVA | Interrupt Vector Assignment |
+-------------------------------------------------------+
.PBCCW | Channel Command Word |
+-------------------------------------------------------+
.PBRSP | Reserved to Port |
+=======================================================+
As can be seen, each of the queues requires three words: an
interlock word, a forward pointer (FLINK) and a back pointer
(BLINK).
All addresses in the PCB, and all addresses given to the
KLIPA are physical memory addresses. This is so because the
KLIPA accesses memory directly over either the E-bus or the
C-bus. If it uses the C-bus, it has no way of performing a
virtual to physical address translation. Consequently, the
monitor must perform this translation in advance of
specifying the address, and it must insure that the
association remains valid until the KLIPA is finished with
the data.
Queue interlock words are conventional "spin locks". That
is, to lock a queue, one simply performs an
AOSE ADD
{lock previously locked, test for timeout}
{lock successfully locked}
and one unlocks a queue by
SETOM ADD
This is easy for the processor, but not so easy for the
KLIPA. In order to assist the KLIPA, a new microcode IOP
function has been added that performs the equivalent of:
AOS EBUS,ADD
That is, the function directs the microcode to increment the
value at ADD and to send the resulting value over the E-bus
to the requesting device. It is then up to the KLIPA to
Page 6
determine if it owns the interlock by examining the result.
PHYKLP's lock procedure includes a "time out" if the lock is
unavailable for a long time. If the spin lock does not
succeed within one second, PHYKLP will consider the KLIPA to
be malfunctioning and reload its microcode.
The PCB is essential to all interactions with the KLIPA.
Therefore, anytime PHYKLP wishes to send a message or
process an interrupt, it must determine the PCB address so
that it can access the appropriate queue or status
information.
In general, the PCB address is passed as an argument to the
inner routines of PHYKLP. This is to allow additional
KLIPAs someday without changing the bulk of PHYKLP. As can
be seen in the next section, the system block associated
with other node points to the appropriate PCB.
2.2 System Blocks
A system block describes the status of another node on the
CI. PHYKLP builds a system block whenever it first
encounters a node. In general, this will be when the poller
receives a valid reply to a "request ID message", but it may
also occur when the other node attempts to begin the
initialization protocol and PHYKLP has not yet noticed the
node. The system block is as follows:
!=======================================================!
.SBANB ! Address of next system block !
!-------------------------------------------------------!
.SBAPB ! Address of associated port control block !
!-------------------------------------------------------!
.SBACD ! Address of associated channel data block !
!-------------------------------------------------------!
.SBVCS ! Path validity info ! Dest vir cir state !
!-------------------------------------------------------!
.SBDSP ! Destination port !
!-------------------------------------------------------!
.SBDRQ ! Datagram return queue header !
!-------------------------------------------------------!
.SBLMB ! Local message buffer header !
!-------------------------------------------------------!
.SBSBI ! Reserved ! SBI of this SB !
!-------------------------------------------------------!
.SBFCB ! Pointer to first connection block !
!-------------------------------------------------------!
.SBLCB ! Pointer to last connection block !
!-------------------------------------------------------!
Page 7
.SBTWQ ! FLINK for SCA work queue !
!-------------------------------------------------------!
.SQBWQ ! BLINK for SCA work queue !
!-------------------------------------------------------!
.SBQOR ! Pointer to queue of outstanding requests !
!-------------------------------------------------------!
.SBDSS \ \
\ Destination system \
\ \
!-------------------------------------------------------!
.SBMMS ! Max mess size (bytes) ! Max DG size (Bytes) !
!-------------------------------------------------------!
.SBDST ! Destination software type !
!-------------------------------------------------------!
.SBDSV ! Destination software version !
!-------------------------------------------------------!
.SBDSE ! Destination software edit level !
!-------------------------------------------------------!
.SBDHT ! Destination hardware type !
!-------------------------------------------------------!
.SBDHV ! Destination hardware version !
!-------------------------------------------------------!
.SBDPC \ \
\ Destination port characteristics \
\ \
!-------------------------------------------------------!
.SBTIM ! TODCLK at last message from this remote !
!-------------------------------------------------------!
.SBFLG ! Flags !
!=======================================================!
As can be seen, the system block contains some fields and
information maintained by SCA alone. That is, this data
structure, although created by PHYKLP, is shared by SCA and
PHYKLP. In effect, this is much like the other PHYSIO data
structures, UDB and CDB, that contain information for
several of the layers.
The fields describing the destination system are acquired
from the "ID request" message exchanged by the ports.
A system block contains all of the information needed for
PHYKLP to correctly process a request from a higher level
service (viz. SCA). In particular, the system block
contains the PCB to be used and the state of the
port-to-port virtual circuit between that system and the
local port. Therefore, each request made by SCA will use
the system block to validate that the operation is legal for
the current virtual circuit state (see section 3.1.2) and
for locating the PCB and its data structures.
Interrupt requests begin with the PCB of the interrupting
KLIPA and then determine the appropriate system block from
the source node address in the received message (see section
3.3). Once the system block is determined, then PHYKLP can
Page 8
validate tha the operation is valid for the current virtual
circuit state.
3.0 PHYKLP DESCRIPTION
3.1 State Representation Of The Driver
3.1.1 Initialization -
The port-to-port VC is opened by means of a "three way
handshake" protocol. Diagrammatically, the protocol looks
like:
(Figure 1)
Notes to figure 1:
The messages, START, STACK and ACK are always to be
sent from the lowest priority command queue (see
6.2.2).
INIT is a holding state. In the case of a
previously known system, INIT will "latch" until SCA
releases it (see KLPOPN call ). In the case of a
newly located node, INIT releases immediately.
Although error detection is not mandatory, it is
recommended in order to avoid unnecessary
interactions with higher-level services.
This protocol insures that the two systems reach the "open"
state at the same time and that they maintain this state
together.
The initialization sequence is begun once PHYKLP recognizes
another node.
3.1.2 Other States -
PHYKLP operates as a state machine. Anytime the local
system wishes to communicate with any system, or the local
system receives data from another system, the state machine
is used to validate the operation and to perform state
transitions and notifications.
In particular, the system block describes the state of the
connection between the local system and the remote system.
Page 9
Any operation has a unique definition for the state in the
system block. Data handling, error handling and calls to
other services are all defined by the current state.
The initialization sequence states and transitions are
defined in figure 1. Table 1 shows the additional
transitions and operations.
Table 1
\
\ EVENT msg datagram send err send err VC closed VC open
S \ rec rec datagram message
T \
A \
T \
E \
ISTRT KLIPA | drop | ignore| NA | NA | tell SCA
error | it | | | |
| | | | |
ASTRT "" | "" | "" | "" | "" | tell SCA
| | | | |
RUN tell SCA| tell SCA| "" | Close VC | tell SCA | NA
| | | goto INIT | goto INIT|
The entries containing "tell SCA" indicate that PHYKLP will
call the appropriate routine in SCA; each of the events
makes use of a unique entry point in SCA.
3.2 Message Handling
The CI supports various types of port-to-port packets.
There is a class of packets directed to SCA defined as
messagess and datagrams. In addition, there are other types
of packets, used for CI housekeeping functions and for
maintenance, that are exchanged by the ports. In fact, the
three initialization messages, START, STACK and ACK are of
this latter class.
All of the housekeeping and maintenance packets are
datagrams. That is, they are handled on a "best effort
delivery" basis. The protocol that uses these packets must
either be tolerant of lost packets or must provide some
other means of guaranteeing no data is lost. The timers
used in the initialization protocol are one example of how a
datagram-based protocol tolerates and recovers from lost
data.
Packets intended for SCA are sent only when a port-to-port
virtual circuit has been established. Until the
Page 10
port-to-port circuit is RUNning, the only data exchanges are
port housekeeping datagrams.
SCA message delivery is guaranteed by the KLIPA. That is,
any SCA message is either correctly delivered, or the
port-to-port circuit is closed. This low-level service
off-loads considerable responsibility from the software
thereby freeing it to provide other, more sophisticated
services.
SCA datagram delivery is not guaranteed (as one might
imagine).
3.2.1 Buffers -
All buffers, datagram and message, are provided by SCA.
PHYKLP has no local buffer pool and takes no responsibility
for acquiring or maintaining buffers.
PHYKLP will enqueue and dequeue buffers, datagram or message
buffers, on request. Only SCA should use these routines as
only SCA should be concerned with CI buffer management.
3.3 Message Formats
The CI portion of a packet looks like:
+-----------------------------------------------+
.PKFLI ! FLINK ! 0
+-----------------------------------------------+
.PKBLI ! BLINK ! 1
+-----------------------------------------------+
.PKRSV ! Reserved for software ! 2
+---------+---------+---------+---------+-------+
.PKSTS !00<--->07!08<--->15!16<--->23!24<--->31!32<->35!
! Status ! Flags ! Opcode ! Port ! MBZ ! 3
+---------+---------+---------+---------+-------+
.PKLEN !00<------------->15!16<--------------------->35!
! PPD byte ! Length of text data ! 4
!-------------------+---------------------------+
! ! 5
! !
! !
! Text !
! !
! ! Queue
! ! Length
Page 11
! ! - 1
+-----------------------------------------------+
The opcode field describes the type of CI message. Packets
intended for SCA are distinguishable from packets used by
the ports.
The information above is called the "PPD header" and is
meant for use by the port and its software driver. SCA has
its own header information that is part of the "text"
carried by the PPD packet.
3.4 Polling
The poller, which is part of the scheduler's context, is
responsible for testing the validity of the port-to-port VC
for each of the known nodes. Part of this testing function
is to implement the "timer" noted in figure 1.
In addition, the poller attempts to "locate" newly "on line"
nodes by periodically sending "request id" messages to any
node not yet identified. This is necessary because the
"request id" message is a datagram, as are all port-to-port
overhead messages, and therefore there is no guarantee when
and if one such message and its repsonse will be
successfully delivered. Therefore, each node on the CI must
continually poll the other nodes in the fervent hope that
one of them will succeed. Although this may sound
hopelessly frustrating, the chances of not succeeding are
actually quite small, and therefore the polling activity is
infrequent. Were it not for the likelihood of losing a
request id or its response, the polling of unknown nodes
could be replaced by the requirement that each newly
activated port simply send each other CI drop a "request
id". This would be sufficient to guarantee complete
connectivity of all CI ports. So much for the ideal world.
The poller's chief testing function is to implement the
timers for the ISTRT and the ASTRT states. In addition to
this, the poller could test the connectivity of the RUN
connections by periodically sending "request id" messages to
these ports as well. However, the SCA idle chatter is a
more inclusive test of port-to-port connectivity and
therefore will be used instead of this more primitive
testing facility.
Page 12
4.0 PHYKLP OPERATION
4.1 Major Interfaces
4.1.1 PHYSIO Interface -
PHYKLP is a device driver. As such, it is called by PHYSIO
to perform device polling and to process interrupts.
The KLIPA does not support "vectored interrupts". PHYSIO
expects its drivers to behave like RH20 channels, and
therefore to request "vectored interrupts". In order to
make all of this balance, the interrupt skip chain for
channel 5 calls directly into PHYKLP when the KLIPA requests
service. PHYKLP then fills in the CDB locations for its
channel (RH slot 7) and calls PHYSIO as if a normal
RH20-style vectored interrupt had occurred. PHYSIO is then
in control of the rest of the interrupt processing.
PHYKLP is defined as a new type of channel controller. As a
result, PHYSIO will acquire some new conventions for
managing controllers other than RH20-style controllers.
The poller is called from PHYSIO's once-a-second routine.
The action of the poller has already been described.
4.1.2 SCA Interface -
SCA relies on PHYKLP to send messages and datagrams. Also,
it requires that PHYKLP be able to establish "named buffers"
for DMA transfers and for PHYKLP to manage the DMA transfer
including notifying SCA of the completion status. Finally,
SCA relies on PHYKLP to inform it of nodes coming on-line or
going off-line.
SCA is responsible for providing message and datagram
buffers sufficient for all of the SCA traffic on the CI. In
particular, SCA cannot rely on the meager number of
datagrams that PHYKLP owns for all of the datagram traffic.
4.2 PHYKLP Structure
Page 13
4.2.1 Initialization -
PHYKLP is initialized by a call to PPDINX. On this call,
PHYKLP constructs the PCB, calls SCA so it may initialize
and create the buffers that PHYKLP needs.
Once SCA has been called, PHYKLP will enable the KLIPA.
4.2.2 CI Configuration -
The CI is "configured" by a combination of interrupt level
and polling activity. The poller continually sends request
ID messages to other CI drops in an effort to locate
previously unknown or not operating nodes.
The result of the polling activity, the ID message, is
received at interrupt level. Once a node is recognized, or
reintialized, the intialization sequence is carried out at
interrupt level.
SC.ONL is called whenever the interrupt level code manages
to create a port-to-port virtual circuit.
The configuration activity is ongoing as the CI
configuration may change at any time.
PHYKLP will establish a VC with another node that it "sees"
for the first time. However, if any existing port-to-port
VC is closed, SCA must call KLPOPN before PHYKLP will
attempt to open, or allow the other node to open, the VC
again. This is provided to allow SCA to clean up its own
data base and to prevent races between incarnations of the
port-to-port VC. The system block for the remote node
indicates whether a VC may be established.
4.2.3 Other Interrupt Activity -
Fig. 2 is a flow chart describing the general flow of the
KLIPA interrupt service.
The interrupt code processes the KLIPA's response queue as
well as detects changes in the device's state. The chief
source of response queue entries is received messages and
datagrams. Any received message must be given to SCA as
PHYKLP has no interest in CI messages.
Datagrams are divided into two classes: CI housekeeping
datagrams and SCA datagrams. PHYKLP distinguishes these
classes by the PPD opcode byte in the received datagram. CI
Page 14
housekeeping datagrams are either intialization datagrams or
maintenance datagrams.
All KLIPA interrupts are processed in the routine KLPINT.
The KLIPA is a standard device in that it has a CONI word
describing the events that promoted the interrupt.
A KLIPA interrupt occurs for one of the following reasons:
1. free queue error
2. response queue available
3. E-bus or M-bus error
4. CRAM parity error
The "normal" case is "response queue available" meaning that
some data has arrived that needs the attention of the
monitor. The other cases are all abnormal conditions that
require special handling.
In all cases, the interrupt entry routine determines the PCB
address for the interrupting device, and loads it into one
of the permanent ACs.
A response is one of the following:
1. A received message or datagram
2. A sent message or datagram that PHYKLP specified
the reponse bit for
3. A sent message or datagram that encountered an
error
4. DMA transfer done
Messsages or datagrams meant for SCA are given directly to
SCA from the KLPINT code. Of course, it is important that
PHYKLP first determine if the port-to-port circuit is
RUNning. If it isn't, then either the KLIPA or the monitor
has made a disasterous error. The port to port curcuit is
validated by finding the system block for the other node and
then checking the virtual circuit state in the system block.
A datagram meant for PHYKLP is one of the "port housekeeping
messages". In general, these will be initialization
messages, but they could be messages generated by PHYKLP
itself to read information from the KLIPA. If the datagram
is an initialization message, then the state description in
Figure 1 applies.
In all cases, table 1 defines the legal operations for data
transmission.
Page 15
All port messages must be sent using the lowest priority
message queue. This is to allow any messages still enqueued
to be processed before any of the VC establishing messages.
If this is not done, then it is possible for the
port-to-port VC to be reestablished before the old message
is processed and therefore the old message will be
erroneously sent. For example, consider the case where SCA
has requested that a message be sent to node A, but the
request is still on the command queue. If node A sends a
START, then the port-to-port VC will be closed. If the
STACK sent in reply is sent at a higher priority than the
queued message, and the ACK in reply is received promptly,
the port-to-port VC could be reestablished before the SCA
message is processed. If this happens, then the KLIPA will
send the message and node A will declare a protocol error
when it receives thereby closing the port-to-port VC.
Although this situation should eventually reach a quiescent
state, much time could pass before the nodes can once again
communicate.
Sending the START, STACK and ACK datagrams at the lowest
priority avoids this unfortunate condition.
4.2.4 Data Services -
The other major portion of PHYKLP is the interfaces for
sending messages and datagrams used by SCA. These are
simply a collection of interfaces that manages the PPD layer
of the packets.
In general, there are buffers owned by SYSAPs that the SYSAP
wants returned when the contents have been transmitted.
This is accomplished by setting the "response bit" in the
packet flag byte. When the data has been sent, the pakcet
will be placed on the response queue and a repsonse queue
available interrupt produced. KLPINT simply informs SCA
that the transmission is complete, and SCA must inform the
appropriate SYSAP. The following cases are known to set the
response bit:
. SCA control messages
. CFS messages
Page 16
4.2.5 Packet Conventions -
As was mentioned earlier, the driver must provide physical
addresses to the KLIPA. Consequently, the link words of
linked structures are physical addresses, not virtual
addresses.
In order to make accessing these structures as simple as
possible, many of the structures contain a word reserved for
use by software. PHYKLP stores the virtual address of the
datum in this word and uses the routines PMOVE and PMOVEM to
reference the physical location.
5.0 ERRORS
Errors are detected at interrupt level. They fall into two
broad categories: transmission errors and device errors.
Each time PHYKLP detects an error, it will either execute a
BUG. or it will explicitly make a SPEAR entry.
5.1 Transmission Errors
Transmission errors occur for following reasons:
1. The port-to-port curcuit is in the wrong state
2. The CI itself is defective
3. The destination port does not ACK the message
It turns out there are other conditions, namely bugs in the
ports, that can appear to be one of these conditions. For
example, a defective collisions avoidance algorithm can
appear to be a defective CI.
A transmission error on a message is a fatal error. That
is, the SCA virtual circuit carrying the message ceases to
be usable. PHYKLP must inform SCA of this condition.
A transmission error on a datagram is not a fatal error. If
the datagram is one of SCA's, then PHYKLP can simply log the
error and not bother notifying SCA. This is acceptable
because datagrams may be lost for many reasons,
transmissions errors being only one of the possible
failures.
A transmission error on an intialization message is of
Page 17
concern to PHYKLP. As noted in Figure 1, errors during
initialization may be ignored and caught as "time out"
conditions. However, to expedite establishing the
port-to-port VC, PHYKLP will detect transmission errors
during initialization and attempt to recover immediately.
In general, a transmission error happens either because the
destination port is not operational (no ACK), or the CI is
defective. The former case is an acceptable failure and
does not require logging or any other special action. The
latter case must be logged and, in addition, PHYKLP will
declare the port-to-port VC down and attempt to initialize
the circuit again. This implies that SCA must be notified
of a "node offline" condition.
5.2 Device Errors
These errors correspond to bits in the device's CONI status
word. Each is detected at interrupt level and handled
immediately.
5.2.1 Free Queue Error -
A free queue error implies that SCA has not provided
sufficient buffering for the KLIPA. Of course a flurry of
activity, such as many nodes coming online at once, can
provoke this error as well.
A free queue error is simply an "data overrun" condition.
It may not indicate a problem with the software or with the
CI, and therefore PHYKLP will simply log the event. If, in
fact, the error resulted in the loss of important data, the
sending system will execute the appropriate recovery (e.g.
close the VC).
5.2.2 CRAM Parity Error -
This means that the KLIPA has halted. Most likely the error
is a result of the UCODE executing a predefined location
that contains a parity error; in other words, it's a
BUGHLT!
PHYKLP will log the error, including the LAR and will
request that the KLIPA microcode be reloaded.
Page 18
5.2.3 Bus Errors -
These are errors that may or may not result in a protocol
error. That is, the data that is lost might be unimportant
(i.e. a datagram).
Since PHYKLP has no way of knowing if any informationw as
lost, it must assume that a bus error is a fatal error and
shut all of the port-to-port VCs.
6.0 OPEN ISSUES
PHYKLP is being written to be initialized during PHYINI as
well as later on the monitor start-up procedure. Once
PHYSIO is modified to allow interrupt handling during
initialization, this feature of PHYKLP will have to be
thoroughly tested.
KLPFUN.MEM
TOPS-20 KLIPA Driver Functional Specification
Arnold Miller
Clair Grant
Version 1.6
December 14, 1983
TOPS-20 KLIPA Driver Functional Specification Page 2
1.0 ARCHITECTURAL POSITION . . . . . . . . . . . . . . . 2
2.0 RELATED STANDARDS AND SPECIFICATIONS . . . . . . . . 3
3.0 TERMINOLOGY . . . . . . . . . . . . . . . . . . . . 3
4.0 KLIPA MICROCODE INTERFACE . . . . . . . . . . . . . 4
5.0 GLOBAL DATA STRUCTURES . . . . . . . . . . . . . . . 5
5.1 Channel Data Block . . . . . . . . . . . . . . . . 5
5.2 Port Control Block . . . . . . . . . . . . . . . . 6
5.3 System Block . . . . . . . . . . . . . . . . . . . 7
6.0 PHYSIO INTERFACE . . . . . . . . . . . . . . . . . . 8
6.1 Initialization . . . . . . . . . . . . . . . . . . 8
6.2 Once-a-second Device Polling . . . . . . . . . . . 9
6.3 Interrupt Service . . . . . . . . . . . . . . . . 9
6.4 Channel Dispatch Vector . . . . . . . . . . . . . 9
7.0 SCAMPI INTERFACE . . . . . . . . . . . . . . . . . 10
7.1 Services Provided For SCAMPI . . . . . . . . . . 11
7.1.1 Port-to-Port Virtual Circuit Maintenance . . . 11
7.1.2 Datagrams . . . . . . . . . . . . . . . . . . 12
7.1.3 Messages . . . . . . . . . . . . . . . . . . . 13
7.1.4 Named Buffers . . . . . . . . . . . . . . . . 14
7.1.5 Get The Local Port's CI Number . . . . . . . . 15
7.1.6 KLIPA Maintenance Support . . . . . . . . . . 15
7.1.7 Callbacks To SCAMPI . . . . . . . . . . . . . 18
7.2 Services Required Of SCAMPI . . . . . . . . . . 18
8.0 DIAGNOSTIC INTERFACE . . . . . . . . . . . . . . . 19
9.0 ERROR PROCESSING . . . . . . . . . . . . . . . . . 19
9.1 CSR Status . . . . . . . . . . . . . . . . . . . 19
9.2 PCB Error Words . . . . . . . . . . . . . . . . 20
9.3 Response Queue Entries . . . . . . . . . . . . . 20
9.4 TOPS-20 Error Reporting . . . . . . . . . . . . 20
9.5 Reloading The KLIPA Microcode . . . . . . . . . 20
10.0 KEY ALGORITHMS . . . . . . . . . . . . . . . . . 20
10.1 KLIPA Initialization . . . . . . . . . . . . . . 21
10.2 Once-a-Second Poller . . . . . . . . . . . . . . 21
10.3 KLIPA Interrupt Service . . . . . . . . . . . . 21
11.0 TESTING . . . . . . . . . . . . . . . . . . . . . 22
TOPS-20 KLIPA Driver Functional Specification Page 3
1.0 ARCHITECTURAL POSITION
PHYKLP.MAC is TOPS-20's port driver for the IPA-20 (KLIPA); thus, it is
the lowest level of software support for the CI-20. It is responsible for
coordinating all operating system access to the KLIPA; it does this by:
1. maintenance of the port-to-port virtual circuits between the local
system and remote systems on the CI
2. providing an interface to the CI services desired by higher level
protocols
The port driver provides a generally passive service that responds to
requests from the higher level protocol module SCAMPI.MAC and performs the
device-specific operations required by the requests. Because the KLIPA is
a smart and somewhat independent device, the driver must perform some
actions on its own without being directed by higher level services.
--------- --------
! DIAG% ! ! SCS% !
--------- --------
! ! User
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
! ! Monitor
! ! ----------
! ! ! PHYMVR !--------------------------
! ! ---------- !
! ! ! --------- !
! ! ! ! PAGEM ! ------------ ----------
! ! ! --------- -----------! PHYSIO !
! ! ! ! / ----------
! ! ! ! / / /
-------- ---------- ! ---------- ---------- /
! DIAG ! ! SCSJSY ! ! ! CFSSRV ! ! PHYMSC ! /
-------- ---------- ! ---------- ---------- /
\ \ \ ! / /
\ \ ! ! / --- -----
\ ---------- /
\ ! SCAMPI ! /
\ ---------- /
\ ! /
\ ---------- /
----- ! PHYKLP ! --------
----------
! Monitor
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
! Hardware
---------
! IPA20 !
---------
Figure 1 - CI-20 Software Components
TOPS-20 KLIPA Driver Functional Specification Page 4
2.0 RELATED STANDARDS AND SPECIFICATIONS
This document describes, at a functional level, those aspects of the KLIPA
driver not specified in other documents. In order for you to understand
this functional specification, you must be familiar with at least the
terminology, if not the details, of the following:
1. LCG CI Port Architecture Specification, 18-Aug-83 (Dossa/Keenan)
R60SPC:LCG-CI-PORT-ARCHITECTURE.SPC
2. IPA20-L Error Spec, 23-Nov-83 (Holewa)
R60SPC:IPA20-ERROR-SPEC.TXT
3. Systems Communications Architecture, 20-July-82 (Strecker)
R60SPC:SCA-CORP-SPEC.MEM
4. TOPS-20 SCA Functional Specification, 21-Nov-83 (Dunn)
R60SPC:SCAFUN.MEM
5. TOPS-20 System Communication Service Tester, 10-May-83 (Grant)
R60SPC:SCSTST.MEM
6. TOPS-20 Coding Standard, 23-Mar-83 (Murphy)
DOC:CSTAND.MEM
The implementation details of the KLIPA port driver are described in:
TOPS-20 KLIPA Driver Design Specification, 16-Aug-83 (Grant/Miller)
R60SPC:KLPDES.MEM
3.0 TERMINOLOGY
The terms in the following list are defined in an attempt to clarify some
potentially confusing topics discussed in this document.
1. port - an entity connected directly to the CI, capable of
creating/destroying virtual circuits and transmitting packets; it
may not have the ability to provide CI services to high-level
protocols
2. node - an entity on the CI, possibly directly connected, possibly
connected via a port; it provides higher-level protocols with an
interface to the CI
TOPS-20 KLIPA Driver Functional Specification Page 5
3. port-to-port virtual circuit - the logical communications path
between 2 ports
4. packet - a series of bytes (identified as a single entity)
transmitted from one port to another
5. datagram - a packet which is not error controlled, that is, its
delivery is not guaranteed
6. message - a packet whose delivery is guaranteed
7. SCA connection - a logical communications path between 2 SCAs
TOPS-20 KLIPA Driver Functional Specification Page 6
4.0 KLIPA MICROCODE INTERFACE
The complete software/microcode interface is described in [1]. The
following is a brief summary of the interface.
PHYKLP and the port communicate by using a queued protocol whose basic
structure relies on 7 different doubly-linked queues. There are 4 command
queues, a response queue, and 2 free queues (one for datagrams, one for
messages). When PHYKLP wants to give the port a command, it places an
entry on the tail of one of the 4 command queues. The port communicates
with PHYKLP by putting entries on the tail of the response queue.
Both PHYKLP and the port use the free queues as a source of available
buffers and as a repository of processed and discarded buffers.
This queue structure is defined and controlled by the Port Control Block
(PCB), a data structure in the KL's memory. Both PHYKLP and the port
read/write the PCB. The PCB is described in the GLOBAL DATA STRUCTURE
section.
The CI-20 requires special KL10 microcode to allow the port to perform a
read-pause-write increment memory reference. This is used by the port to
interlock the queues in cooperation with AOSNs done by PHYKLP.
The port requires PHYKLP to perform the following initialization functions:
1. create initialize the PCB
2. Put a channel jump word in the EPT
3. Create a CCW in the PCB
4. load the KLIPA microcode
5. Start the KLIPA
The port reports error information by:
1. setting bits in the CSR
2. placing packets on the response queue
3. making entries in the PCB's error words
There may be at most one IPA-20 attached to any given KL-10 and the device
must be located in RH slots 6 and 7. This restriction exists because there
is no way to distinguish a KLIPA from a KLNI and therefore the monitor must
have some a priori knowledge of where a KLIPA, if it exists, will be found.
Aside from this restriction, however, we will make no conscious effort in
the software to preclude multiple KLIPAs attached to a KL-10 in the future.
TOPS-20 KLIPA Driver Functional Specification Page 7
5.0 GLOBAL DATA STRUCTURES
5.1 Channel Data Block
PHYKLP creates and uses a channel data block (CDB), and uses the permanent
AC conventions (P1-P4) used by the rest of the I/O system. There is also
an entry in CHNTAB+7 for the KLIPA. However, the KLIPA has its own channel
type; it is not an RH20. The KLIPA redefines some of the RH20 CDB words;
they are:
CDBFLG==CDBCAD ;FLAGS
CB.DED==1B1 ;PORT IS DEAD
CB.PST==1B2 ;PORT IS STOPPED
CB.VOK==1B3 ;PORT VERSION IS OK (UCODE VERSION WORD)
CDBVER==CDBCAD+1 ;MICROCODE VERSION NUMBER
CDBLG0==CDBCC1 ;LOGOUT WORD 0
CDBLG1==CDBCC2 ;LOGOUT WORD 2
CDBLG2==CDBICR ;LOGOUT WORD 3
CDBQRQ==CDBRST ;NON-0 IF HAD TO REQUEUE A REQUEST
CDBCTR==CDBCL2 ;MONOTONIC NUMBER,,FORK WHICH OWNS COUNTERS
CDBFQE==CDBCL2+1 ;MESSAGE,,DATAGRAM FREE QUEUE ERROR COUNT
CDBECW==CDBCL2+2 ;CCW FROM PCB AT ERROR
The KLIPA defines the device-depenedent words in its CDB as follows:
CDBPCB==CDBDDP ;ADDR OF PCB FOR MSCP CHANS
CDBSBS==CDBDDP+1 ;ADDR OF 1ST SYSTEM BLOCK FOR MSCP CHANS
CDBNOD==CDBDDP+2 ;OUR NODE NUMBER ON THIS PORT (MSCP CHANS)
CDBDGB==CDBDDP+3 ;LOC OF BUFFER FOR DATAGRAMS FOR PORT USE
5.2 Port Control Block
The PCB, officially defined in [1], is created by PHYKLP and is used by the
port and PHYKLP as a communications region.
+-------------------------------------------------------+
0 | Buffer Descriptor Table Starting Address |
+-------------------------------------------------------+
1 | Message Free Queue Entry Length |
+-------------------------------------------------------+
2 | Datagram Free Queue Entry Length |
+-------------------------------------------------------+
3 | Reserved |
+-------------------------------------------------------+
4 | Command Queue 3 Interlock |
+-------------------------------------------------------+
5 | Command Queue 3 FLINK |
+-------------------------------------------------------+
6 | Command Queue 3 BLINK |
+-------------------------------------------------------+
7 | Command Queue 2 Interlock |
TOPS-20 KLIPA Driver Functional Specification Page 8
+-------------------------------------------------------+
8 | Command Queue 2 FLINK |
+-------------------------------------------------------+
9 | Command Queue 2 BLINK |
+-------------------------------------------------------+
10 | Command Queue 1 Interlock |
+-------------------------------------------------------+
11 | Command Queue 1 FLINK |
+-------------------------------------------------------+
12 | Command Queue 1 BLINK |
+-------------------------------------------------------+
13 | Command Queue 0 Interlock |
+-------------------------------------------------------+
14 | Command Queue 0 FLINK |
+-------------------------------------------------------+
15 | Command Queue 0 BLINK |
+-------------------------------------------------------+
16 | Response Queue Interlock |
+-------------------------------------------------------+
17 | Response Queue FLINK |
+-------------------------------------------------------+
18 | Response Queue BLINK |
+-------------------------------------------------------+
19 | Message Free Queue Interlock |
+-------------------------------------------------------+
20 | Message Free Queue FLINK |
+-------------------------------------------------------+
21 | Message Free Queue BLINK |
+-------------------------------------------------------+
22 | Datagram Free Queue Interlock |
+-------------------------------------------------------+
23 | Datagram Free Queue FLINK |
+-------------------------------------------------------+
24 | Datagram Free Queue BLINK |
+-------------------------------------------------------+
25 | Reserved |
+-------------------------------------------------------+
26 | Reserved |
+-------------------------------------------------------+
27 | Reserved |
+-------------------------------------------------------+
28 | Reserved |
+-------------------------------------------------------+
29 | Port Error Word 0 |
+-------------------------------------------------------+
30 | Port Error Word 1 |
+-------------------------------------------------------+
31 | PCB Base Address |
+-------------------------------------------------------+
32 | PI Level |
+-------------------------------------------------------+
33 | Reserved |
+-------------------------------------------------------+
34 | Channel Command Word |
+-------------------------------------------------------+
TOPS-20 KLIPA Driver Functional Specification Page 9
35 | Reserved to Port |
+-------------------------------------------------------+
5.3 System Block
PHYKLP creates a system block (SB) for each node it finds on the CI. The
SB cells are defined as offsets in the device-dependent section of the
controller data block (KDB). A system block is used by both PHYKLP and
SCAMPI and looks like:
!=======================================================!
.SBANB ! Address of next system block !
!-------------------------------------------------------!
.SBAPB ! Address of associated port control block !
!-------------------------------------------------------!
.SBACD ! Address of associated channel data block !
!-------------------------------------------------------!
.SBVCS ! ! Dest vir cir state !
!-------------------------------------------------------!
.SBDSP ! Destination port !
!-------------------------------------------------------!
.SBDRQ ! Datagram return queue header !
!-------------------------------------------------------!
.SBLMB ! Local message buffer header !
!-------------------------------------------------------!
.SBSBI ! Reserved ! SBI of this SB !
!-------------------------------------------------------!
.SBFCB ! Pointer to first connection block !
!-------------------------------------------------------!
.SBLCB ! Pointer to last connection block !
!-------------------------------------------------------!
.SBTWQ ! FLINK for SCA work queue !
!-------------------------------------------------------!
.SQBWQ ! BLINK for SCA work queue !
!-------------------------------------------------------!
.SBQOR ! Pointer to queue of outstanding requests !
!-------------------------------------------------------!
.SBDSS \ \
\ Destination system \
\ \
!-------------------------------------------------------!
.SBMMS ! Max mess size (bytes) ! Max DG size (Bytes) !
!-------------------------------------------------------!
.SBDST ! Destination software type !
!-------------------------------------------------------!
.SBDSV ! Destination software version !
!-------------------------------------------------------!
.SBDSE ! Destination software edit level !
!-------------------------------------------------------!
.SBDHT ! Destination hardware type !
!-------------------------------------------------------!
.SBDHV ! Destination hardware version !
TOPS-20 KLIPA Driver Functional Specification Page 10
!-------------------------------------------------------!
.SBDPC \ \
\ Destination port characteristics \
\ \
!-------------------------------------------------------!
.SBTIM ! TODCLK at last message from this remote !
!-------------------------------------------------------!
.SBFLG ! Flags !
!-------------------------------------------------------!
.SBSST ! Start sequence timer !
!=======================================================!
TOPS-20 KLIPA Driver Functional Specification Page 11
6.0 PHYSIO INTERFACE
PHYKLP is a TOPS-20 device driver. As such, it is called by PHYSIO to
perform channel polling and to process interrupts. PHYKLP does not provide
many of the housekeeping and register operations which PHYSIO uses for
RH20-type devices; such operations are simply not pertinent to KLIPA
devices. Thus, there are few expected calls from PHYSIO to PHYKLP.
6.1 Initialization
PHYKLP's KLPINI routine is called from PHYSIO's PHYINI for channel
initialization. KLPINI does the following:
1. resets the KLIPA
2. creates the channel data block (CDB)
3. fills in CHNTAB
4. sets up time out values for REQUEST-IDs and the START/STACK/ACK
sequence
6.2 Once-a-second Device Polling
The poller, routine KLPCHK, is called by PHYSIO's PHYCHK routine by
dispatching through KLPDSP.
The poller attempts to detect newly on-line nodes by periodically sending
request-id packets to all nodes. This is necessary because the "request
id" packet is a datagram, as are all port-to-port packets, and therefore
there is no guarantee that it will be successfully delivered. Therefore,
each node on the CI must continually poll the other nodes in the fervent
hope that one of them will succeed. Although this may sound hopelessly
frustrating, the chances of not succeeding are actually quite small, and
therefore the polling activity is infrequent.
Once another node responds to a request-id, a three-way-handshake
(START/STACK/ACK) initialization is used to open the port-to-port virtual
circuit. The initialization packets are always sent from the lowest
priority command queue (queue 3) and are timed by the poller. The protocol
insures the two systems reach the open state by mutual agreeement.
6.3 Interrupt Service
The KLIPA does not support vectored interrupts; PHYSIO expects its device
drivers to request vectored interrupts. In order to make all of this
balance, the non-vectored interrupt skip chain will verify that the
interrupt was generated by the KLIPA and then jump to PHYSIO as if a
TOPS-20 KLIPA Driver Functional Specification Page 12
vectored interrupt had occurred. A new channel is defined for the KLIPA so
PHYSIO will not process the KLIPA as a funny RH20 device, but as a unique
entity.
6.4 Channel Dispatch Vector
PHYKLP's channel dispatch vector is as follows:
KLPDSP::JRST KLPINI ;0 - INITIALIZATION
JRST DSPBUG ;1 - STACK SECOND CHANNEL COMMAND
JRST KLPSIO ;2 - START I/O
JRST DSPBUG ;3 - POSITION REQUEST
JRST DSPBUG ;4 - RETURN BEST XFER
JRST KLPINT ;5 - INTERRUPT PROCESSING
JRST KLPCCW ;6 - MAKE CHANNEL XFER WORD
JRST KLPHUN ;7 - TRANSFER HUNG
JRST KLPZAP ;10 - RESET CHANNEL
JRST KLPCHK ;11 - PERIODIC CHECK
JRST KLPEXT ;12 - CHECK UNIT EXISTENCE
JRST KLPCCA ;13 - EXTRACT ADDRESS FROM CCW WORD
PHYKLP's controller/unit dispatch vector is as follows:
KLDSP:: JRST DSPBUG ;0 - INITIALIZATION
JRST DSPBUG ;1 - START I/O
JRST DSPBUG ;2 - HANDLE INTERRUPT
JRST DSPBUG ;3 - ERROR RECOVERY
JRST DSPBUG ;4 - HUNG DEVICE
JRST DSPBUG ;5 - CONVERT BLK # TO CYLINDER/SURF-SEC
JRST DSPBUG ;6 - LATENCY COMPUTATION
JRST DSPBUG ;7 - START POSITIONING
JRST DSPBUG ;10 - ATTENTION INTERRUPT
JRST DSPBUG ;11 - SKIP IF POSITIONING REQUIREDD
JRST DSPBUG ;12 - STACK SECOND TRANSFER COMMAND
JRST KLEXT ;13- CHECK EXISTANCE OF UNIT
RET ;14- CHECK FOR HALTED CONTROLLER
DSPBUG is a BUGHLT. We do not expect to get called from PHYSIO for this
function if the channel is a KLIPA.
TOPS-20 KLIPA Driver Functional Specification Page 13
7.0 SCAMPI INTERFACE
The SCA protocol and its associated terminology are completely described in
[3]. The following is a brief description of the services PHYKLP provides
for SCAMPI to implement the protocol used by SYSAPs.
SCA defines 3 types of data transmissions:
1. messages
2. datagrams
3. named buffers
SCA data transmissions are sent only when a port-to-port virtual circuit
has been established by PHYKLP and an SCA connection has been opened. SCA
relies on PHYKLP to maintain the port-to-port virtual circuits. The states
of a virtual circuit are 1) closed, 2) start-sent, 3) start-received, and
4) open.
SCA message delivery is guaranteed by the KLIPA. That is, any SCA message
is either correctly delivered, or the port-to-port virtual circuit is
closed by PHYKLP and SCA is notified. This low-level, guaranteed service
off-loads considerable responsibility from the higher-level software
thereby freeing it to provide other, more sophisticated services. MSCP and
CFS use the message service.
SCA datagram delivery is not guaranteed so applications using SCA must be
prepared for lost datagrams. When DECnet is implemented on the CI, it will
use the datagram service.
SCA requires PHYKLP to establish named buffers and manage their transfers,
including notifying SCA of the completion status.
SCA relies on PHYKLP to inform it of nodes coming on-line or going
off-line.
Although SCA will test an existing port-to-port virtual circuit with idle
chatter over an SCA connection on that virtual circuit, PHYKLP will still
inform SCA when a virtual circuit must be closed, just in case the timing
is such that PHYKLP detects the problem first.
7.1 Services Provided For SCAMPI
This section lists the routines in PHYKLP which are provided for SCA to
manage use of the CI by SYSAPs.
The following is a list of error codes which may be returned by PHYKLP.
1. KLPX1 - no BHDs available
TOPS-20 KLIPA Driver Functional Specification Page 14
2. KLPX2 - no BSDs available
3. KLPX3 - no datagram buffers available
4. KLPX4 - no message buffers available
5. KLPX5 - KLIPA is not enabled
6. KLPX6 - KLIPA is in maintenance mode
7. KLPX7 - no KLIPA on system
8. KLPX8 - packet is bad
9. KLPX9 - no virtual circuit
10. KLPX10 - don't know our own CI node number
7.1.1 Port-to-Port Virtual Circuit Maintenance -
1. Open a port-to-port virtual circuit
SCA must call KLPOPN for any system block that has previously had
an open VC. Once a virtual circuit is closed, PHYKLP will not
reopen it without a direct request from SCA. This is provided to
allow SCA to clean up its own data base and to prevent races
between incarnations of an open port-to-port VC.
BLCAL. (OPENVC,<SBA>)
ACCEPTS: SBA/ System Block Address
RETURNS: +1 Failed. T1/ error code
+2 Success
Possible errors: KLPX3
2. Close a port-to-port virtual circuit
BLCAL. (CLOSVC,<SBA>)
ACCEPTS: SBA/ System Block Address
RETURNS: +1 Failed. T1/ error code
+2 Success
Possible errors: KLPX3
TOPS-20 KLIPA Driver Functional Specification Page 15
7.1.2 Datagrams -
1. send a datagram
BLCAL. (SNDDG,<SBA,DGA,LEN,FLG,PRI,PATH>)
ACCEPTS: SBA/ System Block Address
DGA/ Datagram Address
LEN/ Length of Datagram
word count if high density
byte count industry compatible
FLG/ Flags
F.RTB ;1 - return buffer to SCA
;0 - return buffer to free Q
F.SPM ;1 - Send in high density mode
;0 - Send in industry compatible mode
PRI/ Priority (command queue number)
PATH/ 0 - KLIPA selects
1 - Path A
2 - Path B
RETURNS: +1 Failed. T1/ error code.
+2 Success
Possible errors: KLPX9
2. link datagram buffers onto the datagram free queue
BLCAL. (LNKDFQ,<SBA,BFA>)
ACCEPTS: SBA/ System Block Address
BFA/ Datagram Buffer Address
RETURNS: +1
3. unlink a datagram buffer from the datagram free queue
BLCAL. (ULNKDG,<SBA>)
ACCEPTS: SBA/ System Block Address
RETURNS: +1 Failed. T1/ error code
+2 Success - T1/ Datagram address
Possible errors: KLPX8
TOPS-20 KLIPA Driver Functional Specification Page 16
7.1.3 Messages -
1. send a message
BLCAL. (SNDMSG,<SBA,MSGA,LEN,FLG,PRI,PATH>)
ACCEPTS: SBA/ System Block Address
MSGA/ Message Address
LEN/ Length of Message
word count if high density
byte count industry compatible
FLG/ Flags
F.RTB ;1 - return buffer to SCA
;0 - return buffer to free Q
F.SPM ;1 - Send in high density mode
;0 - Send in industry compatible mode
PRI/ Priority (command queue number)
PATH/ 0 - KLIPA selects
1 - Path A
2 - Path B
RETURNS: +1 Failed. T1/ error code
+2 Success
Possible errors: KLPX9
2. link message buffers onto the message free queue
BLCAL. (LNKMFQ,<SBA,BFA>)
ACCEPTS: SBA/ System Block Address
BFA/ Message Buffer Address
RETURNS: +1
3. unlink a message buffer from the message free queue
BLCAL. (ULNKMG,<SBA>)
ACCEPTS: SBA/ System Block Address
RETURNS: +1 Failed. T1/ error code
+2 Success - T1/ message address
Possible errors: KLPX8
TOPS-20 KLIPA Driver Functional Specification Page 17
7.1.4 Named Buffers -
1. map buffers
BLCAL. (MAPBUF,<BDA>)
ACCEPTS: BDA/ Buffer Descriptor Address
RETURNS: +1 Failed. T1/ error code
+2 Success T1/ buffer name
Possible errors: KLPX1, KLPX2
The buffer descriptor has the following format:
!=======================================================!
! Address of next buffer descriptor (zero if none) !
!-------------------------------------------------------!
! Length of segment #0 !
!-------------------------------------------------------!
! Address of the segment #0 !
!-------------------------------------------------------!
\ \
\ \
\ segment descriptor pairs \
\ \
\ \
!-------------------------------------------------------!
! Length of buffer segment #n !
!-------------------------------------------------------!
! Address of the segment #n !
!-------------------------------------------------------!
! 0 !
!=======================================================!
2. unmap a buffer
BLCAL. (UMAP,<BNAM>)
ACCEPTS: BNAM/ Buffer Name
RETURNS: +1 Failed. T1/ error code
+2 success
Possible errors:
3. send a buffer
BLCAL. (SNDDAT,<SNAM,RNAM,SBYT,RBYT,CID>)
ACCEPTS: SNAM/ name of send buffer
RNAM/ name of receive buffer
SBYT/ Byte offset into send buffer
RBYT/ Byte offset into receive buffer
CID/ Unique code given back when operation completed
TOPS-20 KLIPA Driver Functional Specification Page 18
RETURNS: +1 Failed. T1/ error code
+2 Success
Possible errors: KLPX9
4. request a buffer
BLCAL. (REQDAT,<SNAM,RNAM,SBYT,RBYT,CID>)
ACCEPTS: SNAM/ name of send buffer
RNAM/ name of receive buffer
SBYT/ Byte offset into send buffer
RBYT/ Byte offset into receive buffer
CID/ Unique code given back when operation completed
Returns: +1 Failed. T1/ error code
+2 Success
Possible errors: KLPX9
7.1.5 Get The Local Port's CI Number -
CALL LOCPRT
RETURNS: +1 Failed. T1/ error code
+2 Success. T1/ our CI port number
Possible errors: KLPX7, KLPX10
7.1.6 KLIPA Maintenance Support - The maintenance functions do not require
an SCA connection.
1. send a RESET or START maintenance packet
BLCAL. (PPDSRS,<SBA,PKA,FUNC>)
ACCEPTS: SBA/ System block address
PKA/ Packet Address
FUNC/ Function Bits ,, Start Address
F.SRS ;1 - start
;0 - reset
F.FRC ;1 - force
;0 - non-force
RETURNS: +1 Failed - T1/ error code
+2 Success
TOPS-20 KLIPA Driver Functional Specification Page 19
Possible errors: KLPX7
2. send maintenance data
BLCAL. (PPDSMD,<SBA,PKA,SNAM,RNAM,SBYT,RBYT,CBACK>)
ACCEPTS: SBA/ System block address
PKA/ Packet Address
SNAM/ name of send buffer
RNAM/ name of receive buffer
SBYT/ Byte offset into send buffer
RBYT/ Byte offset into receive buffer
CBACK/ Callback address
RETURNS: +1 Failed - T1/ error code
+2 Success
Possible errors: KLPX7
3. request maintenance data
BLCAL. (PPDRMD,<SBA,PKA,SNAM,RNAM,SBYT,RBYT,CBACK>)
ACCEPTS: SBA/ System block address
PKA/ Packet Address
SNAM/ name of send buffer
RNAM/ name of receive buffer
SBYT/ Byte offset into send buffer
RBYT/ Byte offset into receive buffer
CBACK/ Callback address
RETURNS: +1 Failed - T1/ error code
+2 Success
Possible errors: KLPX7
4. read port counters
Request the port to return the value of the performance counters.
This is an asynchronous operation; when the reply is received,
PHYKLP will call SCAMPI.
BLCAL. (PPDRPT,<PKA,CID>)
ACCEPTS: PKA/ Packet Address
CID/ Connect ID
RETURNS: +1 Failed. T1/ error code
+2 Success
The following is the format of the counter data:
TOPS-20 KLIPA Driver Functional Specification Page 20
!=======================================================!
\ \
\ Port header \
\ \
!-------------------------------------------------------!
.PCMCV! Microcode version !
!-------------------------------------------------------!
.PCAAC! Path A ACK count !
!-------------------------------------------------------!
.PCANC! Path A NAK count !
!-------------------------------------------------------!
.PCANR! Path A no response count !
!-------------------------------------------------------!
.PCBAC! Path B ACK count !
!-------------------------------------------------------!
.PCBNC! Path B NAK count !
!-------------------------------------------------------!
.PCBNR! Path B no response count !
!-------------------------------------------------------!
.PCDGD! Datagrams discarded !
!-------------------------------------------------------!
.PCPTX! Packets transmitted !
!-------------------------------------------------------!
.PCPRX! Packets received !
!-------------------------------------------------------!
.PCDPT! Designated port !
!-------------------------------------------------------!
\ \
\ Reserved for software \
\ \
!=======================================================!
Possible errors: KLPX7
5. point and/or clear counters
BLCAL. (PPDSPT,<PKA,NODE,CTRS>)
CALL PPDSPT
ACCEPTS: NODE/ CI Node Number (377 for all)
CTRS/ Counter Bit Mask
B0 If on, count ACKs received on Path A.
B1 If on, clear the counter.
B2 If on, count NAKs received on Path A.
B3 If on, clear the counter.
B4 If on, count NO_RSPs received on Path A.
B5 If on, clear the counter.
B6 If on, count ACKs received on Path B.
B7 If on, clear the counter.
B8 If on, count NAKs received on Path B.
B9 If on, clear the counter.
B10 If on, count NO_RSPs received on Path B.
B11 If on, clear the counter.
TOPS-20 KLIPA Driver Functional Specification Page 21
B12 The count of discarded datagrams because
of no DGFree Queue entries.
B13 If on, clear the counter.
B14 Count the packets transmitted to the
designated port.
B15 If on, clear the counter.
B16 Count the packets received from the
designated port.
B17 If on, clear the counter.
RETURNS: +1 Failed. T1/ error code
+2 success
Possible errors: KLPX7
7.1.7 Callbacks To SCAMPI - PHYKLP will call SCAMPI when the following
events occur:
1. port-to-port virtual cirucit opened (SC.ONL)
2. port-to-port virtual cirucit closed (SC.OFL)
3. SCA packet or application (SYSAP) packet arrived (SC.INT)
4. named buffer transfer completed (SC.DMA)
5. port counter read completed (SC.RPC)
6. the port has detected an error (SC.ERR)
7. maintenance data transfer complete (SC.MDC)
7.2 Services Required Of SCAMPI
SCA is responsible for providing PHYKLP with all buffers to be used for CI
message and datagram transmission.
Also, once a port-to-port virtual circuit is closed, SCA must specifically
request opening it again. PHYKLP, will not, on its own, reopen a circuit.
TOPS-20 KLIPA Driver Functional Specification Page 22
8.0 DIAGNOSTIC INTERFACE
The following routines are provided for use by diagnostic functions:
1. start the KLIPA
CALL STRKLP
RETURNS: +1 Failed. T1/ Error code
+2 Success.
Possible errors: KLPX7
2. stop the KLIPA
CALL STPKLP
RETURNS: +1
3. test the CI - send a loopback message to the star coupler
CALL CIONLT
RETURNS: +1 Failed. T1/ error code
+2 Success.
Possible error: KLPNO, KLPMAN, KLPBUF, KLPENA
TOPS-20 KLIPA Driver Functional Specification Page 23
9.0 ERROR PROCESSING
TOPS-20 finds out about errors detected by the port by seeing certain CSR
bits on, examining the error words in the PCB, and finding packets with
errors on the response queue.
9.1 CSR Status
The control and status register (CSR) is used by the port and the software
to communicate with one another. PHYKLP does a CONI KLP,AC to read the
CSR; the complete set of CSR bits is in [1]. There are 5 CSR bits which
indicate errors detected by the port:
CI.CPE ;CRAM PARITY ERROR
CI.MBE ;MBUS ERROR
CI.EPE ;EBUS PARITY ERROR
CI.FQE ;FREE QUEUE ERROR
CI.DPE ;DATA PATH ERROR
The port generates an interrupt whenever it sets one of these bits. Why
the port sets these bits and what is expected of TOPS-20 is described in
[2].
9.2 PCB Error Words
The port places information in the PCB's error words and generates an
interrupt whenever it has problems processing a command queue entry. The
contents of the error words are defined in [1].
9.3 Response Queue Entries
The port generates an interrupt whenever it places a packet on the response
queue and the response queue is empty. When PHYKLP processes a response
queue packet it will check the status field to see if the port has set any
error bits, indicating there was a problem in packet processing or
transmission. These status field bits are defined in [1].
9.4 TOPS-20 Error Reporting
PHYKLP will make ERROR.SYS entries, BUGINF/CHK/HLT, and produce CTY
messages as stated in [2].
TOPS-20 KLIPA Driver Functional Specification Page 24
9.5 Reloading The KLIPA Microcode
Whenever PHYKLP determines that the KLIPA microcode must be reloaded, it
arranges for the program SYSTEM:IPALOD.EXE to be run by Job 0. This
program is run by Job 0 at the earliest opportunity and a message is sent
to the CTY.
IPALOD contains the appropriate KLIPA microcode version and the necessary
code to load the microcode. If there is no file called SYSTEM:IPALOD.EXE,
the KLIPA microcode cannot be reloaded.
TOPS-20 KLIPA Driver Functional Specification Page 25
10.0 KEY ALGORITHMS
10.1 KLIPA Initialization
Immediately after the swappable monitor has been loaded, the routine PPDINX
is called from job 0; it does the following:
1. create and initialize the PCB
2. call SCA's initialization routine
3. stock the datagram and message free queues
4. enable the KLIPA
5. throw away any packets on the queues
6. give the KLIPA its PIA
7. determine the local port number
8. send REQUEST-IDs to the other 15 nodes
9. set SF%KLP in FACTSW indicating KLIPA initialization has completed
This is done in process context (rather than during PHYSIO start-up) so
that any memory management or interrupt processing can happen in a
conducive environment.
NOTE
If we decide to support PS: on CI disks,
initialization will have to take place during
the call to PHYINI and therefore we will have
to redesign some of the implementation.
The processing of response queue packets during initialization is done in
the asynchronous interrupt routine KLPINT and not synchronously within
PPDINX. The one exception to this is the short spin-wait for the response
to the READ-REGISTER commmand in which the port indicates our CI node
number.
10.2 Once-a-Second Poller
The poller, routine KLPCHK, runs in scheduler context as part of PHYSIO's
once-a-second routine. It is responsible for:
1. checking to see that the KLIPA is still running
TOPS-20 KLIPA Driver Functional Specification Page 26
2. verifying that the KLIPA still knows its PIA
3. locating newly on-line nodes
4. checking the paths of open virtual circuits
5. checking the REQUEST-ID timers
6. checking the start sequence timers
The poller is on when TOPS-20 is booted; SCA does not have to start it.
10.3 KLIPA Interrupt Service
All KLIPA interrupts are processed in the routine KLPINT which PHYSIO's
PHYINT routine dispatches to through KLPDSP. A KLIPA interrupt occurs for
one of the following reasons:
1. response queue packet available
2. free queue error
3. E-bus or M-bus error
4. CRAM parity error
5. data path error
6. command queue processing error
The normal case is response-queue-available, meaning some data has arrived
that needs the attention of the port driver. Some arriving packets are for
the port driver itself; thus, PHYKLP processes those and takes the
appropriate action. The other packets are for SCA, so PHYKLP simply calls
SCAMPI to say there is an incoming packet.
The error cases are descibed in the Error Processing section.
TOPS-20 KLIPA Driver Functional Specification Page 27
11.0 TESTING
PHYKLP will be tested by 3 major efforts: DVT (design validation testing),
the SCSTST program, and the PAGES program.
DVT is conducted by hardware engineering and includes a comprehensive fault
insertion effort aimed at validating the operation of the hardware,
microcode and software in the face of failures. This should provide
adequate assurance that PHYKLP's error recovery procedures are functioning
correctly.
SCSTST is a test system developed to test the SCS%, DIAG%, and SCAMPI.
However, as SCAMPI is the only user of PHYKLP, an adequate exerciser for
SCAMPI is also an adequate exerciser for PHYKLP. SCSTST is capable of
testing the major features of PHYKLP and using it in conjunction with fault
insertion procedures should provide a comprehensive test of PHYKLP. To
this end, there will be a standard set of SCSTST test scripts to serve as
regression tests for PHYKLP and the associated hardware and software.
PAGES is a program which does continuous I/O to a file. This will be used
as a major tool in verifying that disk I/O works.
LCG-CI-PORT-ARCHITECTURE.SPC
LCG CI Port Architecture Specification
Sean Keenan
Don Dossa
LSEG
MR1-2/E18
DTN 231-4463
August 1984
Revision 5.7
- 1 -
LCG CI Port Architecture Specification Page 2
1.0 SCOPE . . . . . . . . . . . . . . . . . . . . . . . 4
2.0 REFERENCES . . . . . . . . . . . . . . . . . . . . . 4
3.0 GOALS . . . . . . . . . . . . . . . . . . . . . . . 5
4.0 NOTATIONAL CONVENTIONS . . . . . . . . . . . . . . . 5
5.0 INTRODUCTION . . . . . . . . . . . . . . . . . . . . 6
6.0 ARCHITECTURE OVERVIEW . . . . . . . . . . . . . . . 7
6.1 Port Control Block (PCB) . . . . . . . . . . . . . 7
6.2 States And Programming . . . . . . . . . . . . . 15
6.2.1 Port States . . . . . . . . . . . . . . . . . 15
6.3 Queue Structures . . . . . . . . . . . . . . . . 15
6.4 Port Commands And Responses . . . . . . . . . . 17
6.4.1 Command Processing . . . . . . . . . . . . . . 17
6.4.2 Self Directed Commands . . . . . . . . . . . . 17
6.4.3 Responses . . . . . . . . . . . . . . . . . . 17
7.0 PACKET TRANSMISSION CHARACTERISTICS . . . . . . . 18
7.1 Basic Command Queue Format . . . . . . . . . . . 18
7.2 Datagram Service . . . . . . . . . . . . . . . . 21
7.3 Virtual Circuits . . . . . . . . . . . . . . . . 23
7.4 Path Selection . . . . . . . . . . . . . . . . . 24
8.0 MESSAGE TRANSMISSION . . . . . . . . . . . . . . . 28
9.0 DATA TRANSMISSION . . . . . . . . . . . . . . . . 30
9.1 Buffer Descriptors . . . . . . . . . . . . . . . 30
9.2 Reading Data . . . . . . . . . . . . . . . . . . 35
9.3 Writing Data . . . . . . . . . . . . . . . . . . 41
9.4 Data Formatting Modes . . . . . . . . . . . . . 44
9.4.1 Industry Compatible Mode . . . . . . . . . . . 44
9.4.2 Core Dump Mode . . . . . . . . . . . . . . . . 45
9.4.3 High Density Mode . . . . . . . . . . . . . . 45
10.0 STATUS FIELD . . . . . . . . . . . . . . . . . . . 46
11.0 PORT PERFORMANCE MONITORING . . . . . . . . . . . 50
12.0 MISCELLANEOUS MESSAGES . . . . . . . . . . . . . . 56
12.1 Configuration Information . . . . . . . . . . . 56
12.2 Maintenance Commands . . . . . . . . . . . . . . 58
12.2.1 Resetting Remote Systems . . . . . . . . . . . 59
12.2.2 Starting Remote Systems . . . . . . . . . . . 60
12.2.3 Reading Remote System Memories . . . . . . . . 62
12.2.4 Writing Remote System Memories . . . . . . . . 62
12.2.5 Maintenance Confirmation Packets . . . . . . . 64
12.3 Diagnostic Operation . . . . . . . . . . . . . . 65
12.3.1 Loopback . . . . . . . . . . . . . . . . . . . 65
12.4 Data Buffer Maintenance . . . . . . . . . . . . 67
12.5 Illegal Packets . . . . . . . . . . . . . . . . 68
13.0 PORT REGISTERS . . . . . . . . . . . . . . . . . . 69
13.1 Control And Status Register (CSR) . . . . . . . 69
13.2 Diagnostic Registers . . . . . . . . . . . . . . 70
14.0 PROGRAMMING NOTES . . . . . . . . . . . . . . . . 76
14.1 PORT/PORT DRIVER COMMUNICATION . . . . . . . . . 76
- 2 -
LCG CI Port Architecture Specification Page 3
14.2 LOADING AND DUMPING OF THE PORT . . . . . . . . 78
14.3 Reset . . . . . . . . . . . . . . . . . . . . . 80
14.4 Initialization . . . . . . . . . . . . . . . . . 80
15.0 ERROR CONDITIONS . . . . . . . . . . . . . . . . . 80
16.0 IMPLEMENTED FUNCTIONALITY . . . . . . . . . . . . 82
17.0 OPCODE SUMMARY . . . . . . . . . . . . . . . . . . 83
- 3 -
LCG CI Port Architecture Specification Page 4
1.0 SCOPE
This document describes the architecture of the Computer
Interconnect (CI) as it is implemented on LCG products. Included
in this document is the interface the port provides to the
operating system software.
2.0 REFERENCES
The following is a list of CI-related documents that the
reader is assumed to be familiar with:
1. Computer interconnect Specification. Authors: D. Thompson,
J. Buzinski, J. Hutchinson. This documents describes and
specifies the implementation independent functional
characteristics of the CI. Much of the material for the LCG-CI
specification comes from this document.
2. VAX-11 CI Port Architecture Specification. Authors: W.
Strecker, D. Thompson. This document describes the port/port
driver interface for the CI port interfaced to the VAX family
of processors.
3. CI20 Port Hardware Specification. Author: Elbert Bloom. This
spec contains a full description of the hardware in the KL10 CI
port option. Portions of that document have been reproduced
here.
4. PILA Hardware Specification. Author: Shu-Shia Chow. This
spec describes the register addresses and functions of the
registers on the Packet Buffer and Link modules that can be
accessed via the PLI.
5. KLCI ERROR SPEC. Author: Joe Holewa. This spec describes all
the possible errors detected and generated by the port. It
also defines algorithms for error retry and recovery.
- 4 -
LCG CI Port Architecture Specification Page 5
3.0 GOALS
The LCG ports must provide a mechanism for LCG computers to
interface to the CI bus. This requires that the LCG ports fully
comply with the CI wire architecture specification.
4.0 NOTATIONAL CONVENTIONS
All bit numbers and word offsets in this document are in
decimal. All field values within words are defined in octal. All
bytes are always 8 bits long in this document with the most
significant bit of any byte or field defined to be the left-most
bit. It is also a goal to use the same terminology and definitions
as the VAX CI port architecture spec to avoid confusion.
- 5 -
LCG CI Port Architecture Specification Page 6
5.0 INTRODUCTION
The LCG CI port architecture is the operating system interface
to a Computer Interconnect (CI). The CI is a multi-drop bus used
to closely link computer systems and intelligent mass storage
controllers. Characteristics of the CI include:
1. High bandwith of 70 Megabits/second,
2. Packet oriented transmissions,
3. Low error rates,
4. Immediate acknowledgement of successfully-received packets.
Up to 16 ports may be connected onto a single CI bus.
The following figure shows a number of computers and
intelligent mass storage controllers connected to a CI. A single
CI consists of 2 paths. A single system may be connected to more
than one CI.
CI path A
<-------+---------------+---------------+------------------------->
| | |
| | | CI path B
<---------------+---------------+---------------+----------------->
| | | | | |
| | | | | |
+-----------+ +-----------+ +-----------+
| Port 1 | | Port 2 | | Port 3 |
| | | | | |
+-----------+ +-----------+ +-----------+
| Computer | | Computer | |Controller |
| | | | | |
+-----------+ +-----------+ +-----------+
- 6 -
LCG CI Port Architecture Specification Page 7
6.0 ARCHITECTURE OVERVIEW
The port architecture implemented on LCG computers will comply
with the corporate CI architecture specification electrically at
the physical link level. It will also comply at the data link
level in terms of packet formats, acknowledgements, path selection,
sequentiality of data packets, error reporting and recovery
algorithms, datagram service, and Virtual Circuit service.
The mechanism for the host to communicate with the port is by
use of the queued protocol. The basic data structure of the queued
protocol relies on 7 different doubly-linked queues. The queues
are referred to as Command Queues 0-3, the Response Queue, and two
free queues, the Datagram Free Queue and the Message Free Queue.
When the host wishes to send the port a command, it places an entry
on the tail of one of the four command queues. The port
communicates with the host by linking entries onto the tail of the
response queue. The host processes the response queue entry by
delinking the queue entry from the head of the response queue.
Both the host and port use the free queues as a source of available
queue entries and as a repository of processed and discarded queue
entries. The data structure that ties the queue link words
together is the Port Control Block (PCB).
6.1 Port Control Block (PCB)
The mechanism where the host and the port share the queue
structures is controlled by the Port Control Block. The Port
Control Block is a data structure that exists in the physical
memory space of the host computer. Both the host and the port read
and write the data in the PCB. The PCB contains the link words for
the queues and other control information. There is exactly one
unique PCB for each CI port; the ports may not share PCB's. The
port driver has the responsibility of initializing all entries in
the PCB. All reserved entries must be zero and all of the queue
FLINK and BLINK words must be set to valid values. The format of
the PCB follows:
- 7 -
LCG CI Port Architecture Specification Page 8
PORT CONTROL BLOCK (PCB)
+-------------------------------------------------------+
0 | Buffer Descriptor Table Starting Address |
+-------------------------------------------------------+
1 | Message Free Queue Entry Length |
+-------------------------------------------------------+
2 | Datagram Free Queue Entry Length |
+-------------------------------------------------------+
3 | Reserved |
+-------------------------------------------------------+
4 | Command Queue 3 Interlock |
+-------------------------------------------------------+
5 | Command Queue 3 FLINK |
+-------------------------------------------------------+
6 | Command Queue 3 BLINK |
+-------------------------------------------------------+
7 | Command Queue 2 Interlock |
+-------------------------------------------------------+
8 | Command Queue 2 FLINK |
+-------------------------------------------------------+
9 | Command Queue 2 BLINK |
+-------------------------------------------------------+
10 | Command Queue 1 Interlock |
+-------------------------------------------------------+
11 | Command Queue 1 FLINK |
+-------------------------------------------------------+
12 | Command Queue 1 BLINK |
+-------------------------------------------------------+
13 | Command Queue 0 Interlock |
+-------------------------------------------------------+
14 | Command Queue 0 FLINK |
+-------------------------------------------------------+
15 | Command Queue 0 BLINK |
+-------------------------------------------------------+
16 | Response Queue Interlock |
+-------------------------------------------------------+
17 | Response Queue FLINK |
+-------------------------------------------------------+
18 | Response Queue BLINK |
+-------------------------------------------------------+
19 | Message Free Queue Interlock |
+-------------------------------------------------------+
20 | Message Free Queue FLINK |
+-------------------------------------------------------+
21 | Message Free Queue BLINK |
+-------------------------------------------------------+
- 8 -
LCG CI Port Architecture Specification Page 9
+-------------------------------------------------------+
22 | Datagram Free Queue Interlock |
+-------------------------------------------------------+
23 | Datagram Free Queue FLINK |
+-------------------------------------------------------+
24 | Datagram Free Queue BLINK |
+-------------------------------------------------------+
25 | Reserved |
+-------------------------------------------------------+
26 | Reserved |
+-------------------------------------------------------+
27 | Reserved |
+-------------------------------------------------------+
28 | Reserved |
+-------------------------------------------------------+
29 | Port Error Word 0 |
+-------------------------------------------------------+
30 | Port Error Word 1 |
+-------------------------------------------------------+
31 | Port Error Word 2 |
+-------------------------------------------------------+
32 | Port Error Word 3 |
+-------------------------------------------------------+
33 | Port Error Word 4 |
+-------------------------------------------------------+
34 | PCB Base Address |
+-------------------------------------------------------+
35 | PI Level |
+-------------------------------------------------------+
36 | Channel Logout Word 1 Address |
+-------------------------------------------------------+
37 | Channel Command Word |
+-------------------------------------------------------+
38 | Reserved to Port |
+-------------------------------------------------------+
The Buffer Descriptor Table (BDT) base address contains the
physical address of the first word of the buffer descriptors.
Buffer descriptors contain all of the information necessary to
specify to a port where in the host system's memory a data buffer
is located and what access methods are to be used. Buffer
descriptors are described in detail in a separate section of this
document.
- 9 -
LCG CI Port Architecture Specification Page 10
The CI20 requires special KL10 microcode support to allow the
CI20 to perform a memory increment operation using read-pause-write
memory references. This is needed to allow the port to interlock
the queues.
There is a separate interlock word for each queue. When a
queue is available, the corresponding interlock word has a value of
-1. When either the operating system or the port want to interlock
the queue, they must perform a non-interruptable
increment-store-test operation, such as an AOSE. If the
incremented location has a value of zero, then the queue has been
successfully interlocked and the process may now manipulate the
queues. If the incremented value is greater than zero, then the
queue is not available. The interlock word should not be set back
to zero. When the process is finished with the queues, the
interlock word must be set back to -1 (all ones). This marks the
queue as available. Both the port driver and the port microcode
are responsible for leaving the queues in a well defined state.
Error Words 0,1 (words 29,30) are written by the port when it
encounters fatal errors associated with Queue manipulation. This
error reporting strategy requires the port to write as much
information as possible directly into the host memory. This
approach requires the smallest subset of port hardware and
microcode to be working to report these errors.
The information in these words provides sufficient data for
the port driver to determine the type of error and where the error
occurred. When the error is detected, the port will write the
contents of the Error Words in the PCB, enter the Disabled State,
and generate a host interrupt.
The format of Error Word 0 is:
0 1 2 3 4 5 6 7 11 12 35
+-----+---+-----+-----+-----+-----+-------+---------------------+
| CMD | Q | RSP | DFQ | MFQ | D_L | MBZ | FLINK ADDRESS |
+-----+---+-----+-----+-----+-----+-------+---------------------+
BITS NAME DESCRIPTION
==== ==== ===========
0 CMD Error occurred while touching a command
queue entry. The queue with the error is
in QUEUE.
- 10 -
LCG CI Port Architecture Specification Page 11
1-2 QUEUE This is the command queue that had the
error. These bits are only valid if the
CMD bit is on.
00 = CMD QUEUE 0
01 = CMD QUEUE 1
10 = CMD QUEUE 2
11 = CMD QUEUE 3
3 RSP This bit is on if the error occurred
while the port was attempting to build a
response queue entry.
4 DFQ This bit is on if the error occurred
while the port was touching a command on
the datagram free queue.
5 MFQ This bit is on if the error occurred
while the port was touching a command on
the message free queue.
6 D_L This bit is on if the error occurred
while the port was linking a command to
a queue. This bit is off if the error
occured while the port was delinking a
command from a queue. This bit is valid
only with bits 0,4 and 5.
7-11 MBZ These bits will be zero.
12-35 FLINK ADR This is the address of the FLINK word of
the queue entry in question.
Error Word 1 (word 30) contains the API function word that the
port processor used to access memory when the memory error
occurred. This word is written here in the same format as it
appeared on the EBUS. The format of this word is:
0 2 3 5 6 7 12 13 35
+------+------+---+----+----------------------------+
| Addr | Func | Q | 0 | Physical Address |
| Code | | | | |
+------+------+---+----+----------------------------+
- 11 -
LCG CI Port Architecture Specification Page 12
Error Word 2 (word 31) contains the register data on Transmitter or
Receiver spurious attention. The format of Error Word 2 is:
+--------------------------------+------------------+
|00<-------------------------->27|28<------------>35|
| ZERO | DATA |
+--------------------------------+------------------+
Error Word 3 (word 32) contains the Channel Logout Word 1
written by port on any kind of channel error detected during or
immediatly after, a DMA transfer. The format of Error Word 3 is:
0 01 02 03 04 05-08 09 10 11 12 13 14 35
+-+---+----+---+---+-----+---+-+----+-----+----+----------------------+
|1|MEM|-ADR|-WC|NXM| 0 |LXE|0|LONG|SHORT|OVER| COMMAND LIST POINTER |
| |PE | PE |=0 | | | | | WC | WC |RUN |(ADR OF CURRENT CCW+1)|
+-+---+----+---+---+-----+---+-+----+-----+----+----------------------+
- 12 -
LCG CI Port Architecture Specification Page 13
BITS NAME DESCRIPTION
=== ==== ===========
01 MEM PE Memory Parity Error.
02 -ADR PE Not Address Parity Error.
03 -WC=0 Chan Word Count did not = 0 when chan did
a store to EPT.
04 NXM Chan ref non exist mem.
09 LXE Error detected after port term transfer,
Chan aborts next transfer.
11 LONG WC Port comp Xfer, But word count in CCW
not reached.
12 SHORT WC Chan Xferred data spec by CCW, But port
still has data.
13 OVER RUN If dev read, Port sent data but chan buff
were full.
If dev write, Port req data but chan buff
were empty.
Error Word 4 (word 33) contains Channel Logout Word 2 written by port
on any kind of channel error detected during or immediate after,
DMA transfer. The format of Error Word 4 is:
+--------+------------------+-----------------------------+
|00<-->02|03<------------>13|14<----------------------->35|
| OPCODE |CURRENT WORD COUNT| CURRENT DATA BUFFER ADDRESS|
+--------+------------------+-----------------------------+
Word 34 of the PCB is the address of the first word of the
PCB; the CI20 has no other way of finding the PCB.
Word 35 contains the PI Level that the CI20 is assigned.
Word 36 contains the address of the Channel Logout Word 1.
The Channel Logout Word is used in CBUS Error recovery processing.
- 13 -
LCG CI Port Architecture Specification Page 14
Word 37 is reserved for the Channel Command Word. The port
will write a CCW-style word here when it wishes to transfer data
over the KL10 CBUS. The port driver is responsible for writing a
Channel Jump Word in the appropriate EPT location corresponding to
the RH20 backplane slot where the CI20 is installed.
Word 38 is always reserved to the port microcode for its use;
the port driver should never write this location nor depend upon
its value.
When the CI20 is being initialized, the port driver must set
up the channel to transfer the contents of the PCB into the port.
This is done by setting up a CCW to transfer 3 words (words 34-36
of the PCB) from KL10 memory to the channel. The port will start
the channel and will read the contents of these locations. This
provides the port with the base of the PCB and the current PI
assignment.
It is important to realize that since the port will be using
the channel to transfer large blocks of data, the channel will be
writing logout information into the EPT. An error that the channel
discovers will be reported in the usual manner via the EPT.
- 14 -
LCG CI Port Architecture Specification Page 15
6.2 States And Programming
6.2.1 Port States -
There are 3 defined states in which the port can be. They
are:
1. Uninitialized - The port is not running. This is the power up
state. The port enters this state after a power-on or a master
reset. The port may exit this state only if valid microcode is
loaded into the port and the port clocks are started.
2. Disabled - The port is running, but it is not accepting CI
packets. The port will process entries from the command queues
with opcodes of 200 or greater. This state can only be entered
from the uninitialized state by starting the port microcode
running and setting the Disable bit (Bit 30) in the CSR. The
microcode will put the port into this state and signal it by
setting Disable Complete (Bit 12) in the CSR. The Disabled
state is entered from the Enabled state through a command from
the host to enter this state or when the port microcode detects
a non-recoverable internal port error.
3. Enabled - The port is fully functional; it is processing
commands and CI packets. This is the normal state for the
port. This state can only be entered from the Disabled state
via command from the host (Bit 31 in CSR).
6.3 Queue Structures
There are 7 queues used by the host and port. The four
command queues contain commands for the port that it has not
processed yet; the port will delink and execute queue entries from
the head of these queues. The priority ordering of the command
queues is from Command Queue 3 to Command Queue 0. The port will
examine Command Queue 3 and process all commands on this queue
before moving to a command queue of lower priority. When Command
Queue 3 is empty, the port will examine Command Queue 2 for any
valid commands and process the first command it finds. From this
point on, the port will process a command from a lower priority
queue only after it has determined that no command exists on a
higher priority queue. The port determines that a command has been
placed upon a previously empty command queue by examining a bit in
a control register that is set by the host when it has placed a
- 15 -
LCG CI Port Architecture Specification Page 16
command upon a previously empty command queue. This guarantees
that any command placed on a command queue of higher priority will
be processed before the next command on a lower priority queue.
Similarly, if a response queue entry is required, the port
will place responses onto the tail of the response queue when it
has finished processing either a locally generated or remotely
generated command. The host will process response queue entries by
delinking them from the head of the response queue, processing the
results, and placing the discarded entries on the tail of one of
the free queues. If a response does not have to be generated by
the port, it will place the discarded entry onto the tail of the
free queue. All datagram-class commands and responses use the
Datagram Free Queue as both a source and sink for queue entries.
All commands and responses that are executed under Virtual Circuit
control use the Message Free Queue. The description of Virtual
Circuit commands is covered in a later section.
The general format of a queue entry is:
+-----------------------+
| FLINK | 0
+-----------------------+
| BLINK | 1
+-----------------------+
| Reserved for Software | 2
+-----------------------+
| Command Data | 3
+-----------------------+
| . | 4
| . |
| . | Queue length
| . | -1
+-----------------------+
The length of a queue entry is specified by the host operating
system to the port via words in the Port Control Block. Because
the queue FLINK and BLINK words are interlocked, it is required
that all queue insertions and deletions follow the proper protocol
to avoid race conditions between the port driver and the port
microcode. The reserved for software word is never written by the
port.
- 16 -
LCG CI Port Architecture Specification Page 17
6.4 Port Commands And Responses
6.4.1 Command Processing -
When the port driver places a command upon one of the command
queues, the port microcode will delink the first command in a
particular command queue, process the command, and build a response
if either an error occurred or if the port driver specifically
requested a response packet by setting the Response bit (R bit) in
the FLAGS field.
The PORT field in a command refers to the destination port;
this field will be ignored if the command does not reference a
remote port.
6.4.2 Self Directed Commands -
Commands that have the destination port field equal to the
port's own number will cause the port to perform all of the normal
functions including Virtual Circuit checking, if required by the
packet type. The port will use internal loopback mode on self
directed commands. The port will receive its own message and
process it normally. On LOOPBACK commands, it is necessary for the
port driver to append the 32-bit CRC characters at the end of the
data transfer. In this case, the 4 bytes of CRC must directly
follow the last byte of text, but the 4 CRC characters are not
included in the text byte count specified in datagrams. It must be
recognized that while in internal loopback the port is unable to
receive packets from the CI wire.
6.4.3 Responses -
All locally-generated commands that require a response will
cause the generation of a response packet that will be linked onto
the tail of the response queue. Remotely generated packets will
also cause responses to be generated. This is true for all
datagrams and messages, configuration packets, maintenance packets,
unknown or illegal type packets, and data transmission confirmation
packets.
The port allows the port driver to determine which responses
are generated because of the execution of locally-generated
commands and which response queue entries are generated because of
- 17 -
LCG CI Port Architecture Specification Page 18
reception of packets over the CI wire. All commands which the port
driver builds and the port executes are called locally-generated
commands; the corresponding response queue entries have the
identical opcode and are referred to as locally-generated
responses. Response queue entries which are generated because the
port received a packet over the CI are referred to as
remotely-generated responses. The port will always add a decimal
32 to the value of the opcode field in remotely-generated commands
when building the response queue entry.
In locally-generated responses, the PORT field is not modified
by the port. In remotely-generated responses, the PORT field
contains the port number from which the packet was received.
7.0 PACKET TRANSMISSION CHARACTERISTICS
7.1 Basic Command Queue Format
The basic queue entry format is the same for all command queue
entries. The format of this queue entry follows:
+-------------------------------------------------------+
| FLINK | 0
+-------------------------------------------------------+
| BLINK | 1
+-------------------------------------------------------+
| RESERVED FOR SOFTWARE | 2
+-----------+-----------+-----------+-----------+-------+
|00<----->07|08<----->15|16<----->23|24<----->31|32<->35|
| STATUS | FLAGS | OPC | PORT | MBZ | 3
+-----------+-----------+-----------+-----------+-------+
| | 4
| |
| |
| |
| COMMAND |
| SPECIFIC | Queue
| |Length
| | - 1
+-------------------------------------------------------+
- 18 -
LCG CI Port Architecture Specification Page 19
WORD:BITS NAME DESCRIPTION
========= ==== ===========
3:0-7 STATUS These bits must be zero when the port
driver places this command upon the
port's command queue. The port microcode
will report the status of this command
via this field when the port moves this
command to the corresponding response
queue.
3:8-15 FLAGS This field is a collection of bits that
allow command modifiers to be given to
the port. These bits are defined below
for all commands.
3:8 PACKET SIZE This bit defines the base size of the
data packets. If the bit is off, the
base size is 512 bytes; if the bit is
on, the base size is 576 bytes. This bit
definition is only valid for data
packets. Refer to the definition below
if the packet is a datagram or message
packet.
3:8 FORMAT If this bit is off, LCG ports assume the
datagram and message data is packed in
industry compatible mode. If the bit is
on, the data is in high density mode.
This bit definition is only valid for
datagram and message packets; refer to
the above definition for data packets.
3:9-11 M These bits determine the actual size of
the data packets to be sent. The packet
size used = (PACKET SIZE) * (M + 1).
These bits are valid only for data
packets.
3:13-14 PATH SELECT These are the Path Select bits.
PS = 0 => Automatic path selection
PS = 1 => Select path A
PS = 2 => Select path B
PS = 3 => Invalid
- 19 -
LCG CI Port Architecture Specification Page 20
3:15 RESPONSE If this bit is on, the port will always
(R bit) generate a response packet for this
command. If the bit is off, the port
will generate a response only if there
was some error encountered during the
processing of the command.
3:16-23 OPCODE This is the opcode field for the
command. Refer to the individual
commands for details.
3:24-31 PORT This is the source or destination port
number.
3:32-35 MBZ The port expects these bits to always be
zero.
- 20 -
LCG CI Port Architecture Specification Page 21
7.2 Datagram Service
The datagram service supplied by the port microcode provides a
best-effort delivery service of messages from one port to another
port over the CI. The text portion is transmitted over the CI from
left to right within each 36 bit word. The first 2 bytes of text
are stored in the same word as the text length, which is not
transmitted over the CI. These first 2 text bytes are also
transmitted from left to right. All datagrams received over the CI
are also written to host memory from left to right in the order
they are received.
The format of the send datagram command is:
+-------------------------------------------------------+
| FLINK | 0
+-------------------------------------------------------+
| BLINK | 1
+-------------------------------------------------------+
| RESERVED FOR SOFTWARE | 2
+-----------+-----------+-----------+-----------+-------+
|00<----->07|08<----->15|16<----->23|24<----->31|32<->35|
| STATUS | FLAGS | OPC | PORT | MBZ | 3
+-----------+-----------+-----------+-----------+-------+
|00<----------------->15|16<------------------------->35|
| TEXT | LENGTH OF TEXT DATA | 4
+-----------------------+-------------------------------+
| | 5
| |
| |
| TEXT |
| |
| | Queue
| | Length
| | - 1
+-------------------------------------------------------+
WORD:BITS NAME DESCRIPTION
========= ==== ===========
3:16-23 OPCODE OPCODE = 1 octal (SNDDG)
41 octal (DGREC)
- 21 -
LCG CI Port Architecture Specification Page 22
4:00-15 TEXT The left-most byte in this word is the
first text byte to be sent/received over
the CI wire. The number of bytes is
determined from the length field and the
packing format is determined from the
FLAGS field.
4:16-35 LENGTH This 20-bit quantity is the number of
bytes in the message. This number must
be between 0 and the lesser of the
remaining number of bytes in the command
queue entry and 4089. Note that the port
hardware does not currently support
packets of greater than 1017. bytes.
This field is not sent over the CI.
5:0-?? TEXT The remaining text.
At the completion of the send datagram command, the port
microcode will determine if it should build a response queue entry
for this command.
The response is called a Datagram Sent Response (DGSNT). If
the Response bit (R bit) is set in the FLAGS words, the port will
always build a response entry. If the R bit is off for this
particular command, no response will be built unless an error
occurred during the processing or transmission of the packet. Any
error condition always causes the port to build a response packet.
The format of the response packet is similar to the send datagram
packet; the only difference is that the STATUS field is returned
with a non-zero value to indicate the type of failure. Refer to
the section on responses for a detailed description of all of the
allowable values for the STATUS field.
If no response is to be built, the SNDDG command will be
linked onto the tail of the datagram free queue.
A remotely-generated Datagram is a DGREC.
- 22 -
LCG CI Port Architecture Specification Page 23
7.3 Virtual Circuits
A Virtual Circuit is the mechanism that is used to provide a
higher quality delivery service for messages than the datagram
service. Delivery of packets under Virtual Circuit control is
guaranteed to be sequential, error-free, and non-duplicated. The
circuit is constructed of 3 state variables in the sending and
receiving nodes.
Virtual Circuits are maintained on a per-node basis. Each
circuit is independently controlled. Each pair of nodes has a
Virtual Circuit state with respect to each other. The state of the
circuit is determined by three bits internal to the port as
specified by the Set Circuit (SETCKT) Command. These bits are the
circuit state (CST), the sending sequence number (NS) and the
receiving sequence number (NR). The CST bit is set to a one for
each port when the Virtual Circuit is opened between them. NS and
NR are used to guarantee delivery, sequentiality and
non-duplication. The NS is the number of the next packet to be
sent (or the present packet being transmitted). The NR is the
number of the next packet to be received.
When a packet is to be sent under Virtual Circuit control, the
transmitting port first checks to make sure that the Virtual
Circuit is opened between the two ports. If it is not, the packet
will not be sent over the CI wire, but an appropriate error
response will be generated by the port. If the Virtual Circuit is
opened, the transmitting port loads the value of NS into the
defined bit in the FLAGS field. When successful transmission is
determined NS is complemented.
In the receiving node, the state of the Virtual Circuit is
determined by the CST bit, and if the Virtual Circuit is opened
between the two ports, then the value of NS carried in the packet
is compared with the state of the NR bit for the circuit. If they
are equal, the packet is accepted and NR is complemented. If the
values are not equal, the packet is discarded.
The Virtual Circuit will be closed if any transmission fails;
any condition that may result in the disruption of sequentiality
also causes the port to close the Virtual Circuit. In addition, a
port may close a virtual circuit at any time.
The port microcode also maintains the status of the CI paths.
This information is kept with each Virtual Circuit. The port
driver determines the path availability when it establishes a
Virtual Circuit.
- 23 -
LCG CI Port Architecture Specification Page 24
7.4 Path Selection
Because there are normally two paths available on the CI
ports, there is a mechanism to determine on which path a CI packet
should be sent. There are two modes - Select and Automatic.
Select mode is used when the port driver wants a particular path to
be used. Automatic mode is used when the port driver does not care
which path is used. The port microcode will implement a mechanism
in which the port will randomly select a path in the automatic
mode.
The Virtual Circuit state between any two ports is set via the
Set Virtual Circuit (SETCKT) command. The format of the command
is:
+-------------------------------------------------------+
| FLINK | 0
+-------------------------------------------------------+
| BLINK | 1
+-------------------------------------------------------+
| RESERVED FOR SOFTWARE | 2
+-----------+-----------+-----------+-----------+-------+
|00<----->07|08<----->15|16<----->23|24<----->31|32<->35|
| STATUS | FLAGS | OPC | PORT | MBZ | 3
+-----------+-----------+-----------+-----------+-------+
|00<----------------------------------------->31|32<->35|
| XCT_ID | MBZ | 4
+-----------------------------------------------+-------+
|00<----------------------------------------->31|32<->35|
| XCT_ID | MBZ | 5
+------+-----+----+----+------+-------+---------+-------+
| 00 | 01 | 02 | 03 | 04 |05<->06|07<----------->35|
| LDST | CST | NR | NS | LDPS | PS | MBZ | 6
+------+-----+----+----+------+-------+-----------------+
| RESERVED | 7
| FOR | Queue
| RESPONSE |Length
+-------------------------------------------------------+ -1
WORD:BITS NAME DESCRIPTION
========= ==== ===========
3:16-23 OPCODE OPCODE = 200 octal (SETCKT).
- 24 -
LCG CI Port Architecture Specification Page 25
4-5:0-31 XCT_ID The XCT_ID is a 64-bit, 2 word quantity
that is reserved for use by the port
driver. The port will never modify this
quantity but preserves it so that
response queue entries can be associated
with a particular command.
6:00 LDST If this bit is set, the CST, NR and NS
bits are set according to bits 1-3.
6:01 CST Virtual Circuit State
CST = 0 => Circuit closed.
CST = 1 => Circuit opened.
6:02 NR Receiving Sequence Number
6:03 NS Sending Sequence Number
6:04 LDPS If this bit is set, the Path Select bits
are set according to bits 5-6.
6:5-6 PS Path Select Bits
PS = 0 => Neither path allowed..
PS = 1 => Path A allowed.
PS = 2 => Path B allowed.
PS = 3 => Both paths allowed.
6:7-35 MBZ These bits must always be zero.
- 25 -
LCG CI Port Architecture Specification Page 26
If the R bit is set in the FLAGS field, then the port will
generate a response by moving the SETCKT command queue entry to the
response queue. The port microcode will also return the current
state of the specified Virtual Circuit in word 7 of the packet.
The actual format of the CKTSET command is:
+-------------------------------------------------------+
| FLINK | 0
+-------------------------------------------------------+
| BLINK | 1
+-------------------------------------------------------+
| RESERVED FOR SOFTWARE | 2
+----------+----------+-----------+----------+----------+
|00<---->07|08<---->15|16<----->23|24<---->31|32<---->35| 3
| STATUS | FLAGS | OPCODE | PORT | MBZ |
+----------+----------+-----------+----------+----------+
|00<-------------------------------------->31|32<---->35| 4
| XCT_ID | MBZ |
+--------------------------------------------+----------+
|00<-------------------------------------->31|32<---->35| 5
| XCT_ID | MBZ |
+------+-----+----+----+------+-------+------+----------+
| 00 | 01 | 02 | 03 | 04 |05<->06|07<----------->35| 6
| LDST | CST | NR | NS | LDPS | PS | MBZ |
+------+-----+----+----+------+-------+-----------------+
| 00 | 01 | 02 | 03 | 04 |05<->06|07<----------->35| 7
| ZERO | CST | NR | NS | ZERO | PS | MBZ |
+------+-----+----+----+------+-------+-----------------+
| | 8
| RESERVED |
| FOR | Queue
| SOFTWARE |Length
| | - 1
+-------------------------------------------------------+
WORD:BITS NAME DESCRIPTION
========= ==== ===========
7:00 ZERO This bit will be zero.
7:01 CST The current state of the Virtual Circuit
between this port and the port specified
in the PORT field is returned here.
7:02 NR This is the current state of the NR bit.
- 26 -
LCG CI Port Architecture Specification Page 27
7:03 NS This is the current state of the NS bit.
7:04 ZERO This bit will be zero.
7:5-6 PS This is the current state of the paths.
7:7-35 MBZ These bits will be zero.
- 27 -
LCG CI Port Architecture Specification Page 28
8.0 MESSAGE TRANSMISSION
The message mechanism provides highly reliable delivery of
single message packets under Virtual Circuit control. Messages are
of variable length, ranging from 0 to 4089 bytes in textual length.
The maximum size messages that may be sent from one port to another
is determined at a higher level protocol. As with datagrams, the
bytes will be transmitted over the CI from left to right within
each host memory word. The same left to right order will also
apply to all received messages.
To send a message, the port driver places a command upon one
of the command queues in the host memory. The format of the basic
message command (SNDMSG) is:
+-------------------------------------------------------+
| FLINK | 0
+-------------------------------------------------------+
| BLINK | 1
+-------------------------------------------------------+
| RESERVED FOR SOFTWARE | 2
+-----------+-----------+-----------+-----------+-------+
|00<----->07|08<----->15|16<----->23|24<----->31|32<->35|
| STATUS | FLAGS | OPC | PORT | MBZ | 3
+-----------+-----------+-----------+-----------+-------+
|00<----------------->15|16<------------------------->35|
| TEXT | LENGTH | 4
+-----------------------+-------------------------------+
| |
| |
| TEXT |
| |
| | Queue
| | Length
| | - 1
+-------------------------------------------------------+
WORD:BITS NAME DESCRIPTION
========= ==== ===========
3:16-23 OPCODE OPCODE = 2 octal (SNDMSG)(MSGSNT)
42 octal (MSGREC)
- 28 -
LCG CI Port Architecture Specification Page 29
4:00-15 TEXT The left-most byte in this word is the
first text byte to be sent over the CI
wire. The number of bytes is determined
from the length field and the packing
format of either industry compatible or
high density is determined from the
FLAGS field.
4:16-35 LENGTH This 20-bit quantity is the number of
bytes in the message. This number must
be between 0 and the lesser of the
remaining number of bytes in the command
queue entry and 4089. Note that the port
hardware does not currently support
packets of greater than 1017. bytes.
This field is not sent over the CI.
5:0-?? TEXT The remaining text.
The format of the Message Sent (MSGSNT) response is the same
as the SNDMSG command except that the STATUS byte will contain the
result of the command execution. The description of the STATUS
byte contains all of the allowed values for this field.
A remotely-generated response is a MSGREC.
- 29 -
LCG CI Port Architecture Specification Page 30
9.0 DATA TRANSMISSION
Data packets provide a mechanism to allow the CI ports to move
large amounts of data with a minimal amount of port driver
overhead. Data transfers use the Virtual Circuit mechanism to
provide a high quality transmission medium. All data transfers
reference named buffers in hosts' memories. All commands contain
the buffer names of the sending and receiving buffers and offsets
which the ports use to determine where data is read from and
written to. The buffer names and offsets must be agreed upon
between the hosts' port drivers at a higher level of protocol. LCG
hosts must also set up the buffer descriptor chains prior to
transferring data. The Virtual Circuit must also be opened before
the data transfer command is executed by the port.
9.1 Buffer Descriptors
Named memory buffers in the physical address space of the host
computers are described by buffer descriptors. Each buffer
descriptor consists of a header, which uniquely defines a named
memory buffer. Associated with each buffer header descriptor is a
linked chain of buffer segment descriptors, which uniquely define
contiguous segments of the buffer. The buffer header descriptor
and all associated linked buffer segment descriptors constitute a
buffer descriptor.
The buffer descriptors are assembled and inserted into a
Buffer Descriptor Table (BDT) by the port driver. The Buffer
Descriptor Table also resides in the host computer's physical
address space. Since the Buffer Descriptor Table is accessed via
the buffer names, it is not necessary that the entire table be
physically contiguous; the only restriction on the placement of
Buffer Header Descriptors and Buffer Segment Descriptors is that
they must be allocated on 4 word boundaries.
A buffer name is a 32-bit quantity which is passed between
ports via the name fields of the commands and their associated
control packets or data packets. The name fields are supplied and
inserted into the applicable commands by the port drivers. The
port uses the name fields to access the buffer descriptors for
commands which are associated with data transfers between named
buffers.
- 30 -
LCG CI Port Architecture Specification Page 31
The format of a buffer name, as it appears in the host
computer's memory, follows:
00 15 16 31 32 35
+-----------------------+-----------------------+------+
| KEY | INDEX | ZERO |
+-----------------------+-----------------------+------+
The INDEX field is used by the port as a word offset from the
start of the buffer descriptor table to find the buffer header
descriptor for this buffer name. The port will add the contents of
the index field to the contents of word 0 of the PCB. The port
microcode will then compare the KEY field fetched from the BHD with
the KEY field supplied with this buffer name. This is a check on
the value of the INDEX field. If the keys do not match, then the
buffer name received from either a command queue or a packet from
the CI bus is in error and appropriate error processing is
initiated. If the keys do match, the port microcode will continue
processing, knowing that this is the correct BHD to associate with
the supplied buffer name.
Buffer Header Descriptors contain all of the information that
the port microcode needs to find the buffers that contain the data
to transfer over the CI. The BHD contains the start of the buffer
in the host computer's physical address space, its length, some
access control bits, and pointers to buffer segment descriptors
(BSD) which contain further information. Each buffer name has no
more that 1 BHD associated with it and the BHD points to the first
BSD which in turn points to the first physically contiguous segment
of the data buffer. Each succeeding BSD points to another
physically contiguous segment of data. There are as many BSDs as
necessary to completely describe the data buffer. The VALID bit in
the BHD provides a mechanism for the port driver to inform the port
that this BHD and its BSDs are valid and opened. The port will
always assume that each BHD and all BSDs start on 4-word
boundaries; this allows the port to access the data in the most
efficient manner possible.
- 31 -
LCG CI Port Architecture Specification Page 32
The actual format of a BHD is:
BUFFER HEADER DESCRIPTOR
+----------------+-----------------------+--+--+--+--+
|00<---------->15|16<----------------->31|32|33|34|35|
| KEY | MBZ |N |V |E |W | 0
+-------------+--+-----------------------+--+--+--+--+
|00<------->11|12<-------------------------------->35|
| MBZ | FIRST BUFFER SEGMENT DESCRIPTOR ADDR |--+ 1
+-------------+--------------------------------------+ |
|00<---------------------------------------------->35| |
| LENGTH | | 2
+----------------------------------------------------+ |
| RESERVED | | 3
+----------------------------------------------------+ |
V |
V |
V |
V |
BUFFER SEGMENT DESCRIPTOR # 1 |
+-------+-------+-------+----------------------------+ |
|00<->05|06<->07|08<->11|12<---------------------->35|<-+
| MBZ | MODE | MBZ | BUFFER SEGMENT BASE ADDR | 0
|-------+-------+--+----+----------------------------+
|00<------------>11|12<--------------------------->35|
| MBZ | NEXT BUFFER SEGMENT DESCRIPTOR |--+ 1
|------------------+---------------------------------+ |
|00<---------------------------------------------->35| |
| SEGMENT LENGTH | | 2
|----------------------------------------------------+ |
| RESERVED | | 3
+----------------------------------------------------+ |
V |
V |
- 32 -
LCG CI Port Architecture Specification Page 33
V |
V |
BUFFER SEGMENT DESCRIPTOR # n |
+-------+-------+-------+----------------------------+ |
|00<->05|06<->07|08<->11|12<---------------------->35|<-+
| MBZ | MODE | MBZ | BUFFER SEGMENT BASE ADDR | 0
+-------+-------+--+----+----------------------------+
|00<------------>11|12<--------------------------->35|
| MBZ | NEXT BUFFER SEGMENT DESCRIPTOR | 1
|------------------+---------------------------------+
|00<---------------------------------------------->35|
| SEGMENT LENGTH | 2
|----------------------------------------------------+
| RESERVED | 3
+----------------------------------------------------+
BUFFER HEADER DESCRIPTOR FIELD DEFINITIONS
WORD:BITS NAME DESCRIPTION
========= ==== ===========
0:0-15 KEY This is the KEY that the port microcode
compares with the KEY supplied in the
buffer name. These keys must match for
the port to continue processing the
request for this buffer name.
0:16-31 MBZ These bits must be zero.
0:32 NO CLEAR VALID The port driver software must set this
bit if it does not want the port micro-
code to clear the VALID bit.
0:33 VALID The port driver software must set this
bit to declare that this BHD is opened.
Once this bit is set, the port microcode
will assume that no further changes to
this BHD or its associated BSDs will be
made by the port driver software. This
allows the port microcode to internally
cache this data. The VALID bit will be
cleared by the port microcode when the
DMA transfer completes without error.
- 33 -
LCG CI Port Architecture Specification Page 34
0:34 ERROR This bit must be initially cleared by
the port driver software when it marks
the buffer as open by setting the VALID
bit. The port microcode will set this
bit to indicate to the port driver that
an access violation or buffer length
overflow occurred using this BHD or one
of its BSDs.
0:35 WRITABLE This bit is set by the port driver when
the BHD is opened if the port microcode
is allowed to write data into this
buffer.
1:0-11 MBZ These bits must be zero.
1:12-35 BSD ADDRESS This is the address of the first word of
the first buffer segment descriptor
associated with this BHD. This address
is a physical address within the host
computer's memory space.
2:0-35 LENGTH This is the number of 4-bit nibbles that
are to be either read or written by the
port in the buffer.
3:0-35 RESERVED This word is reserved for future use.
- 34 -
LCG CI Port Architecture Specification Page 35
BUFFER SEGMENT DESCRIPTOR FIELD DEFINITIONS
WORD:BITS NAME DESCRIPTION
========= ==== ===========
0:0-5 MBZ This field absolutely MUST be zero; if
this field is not zero, correct
operation of the port cannot be
guaranteed.
0:6-7 MODE This is the data packing mode for this
BSD.
0 => Industry Compatible
1 => Core Dump
2 => High Density
3 => Illegal
0:8-11 MBZ These bits must be zero.
0:12-35 BSD BASE ADDR This field contains the address of the
start of a physically contiguous block
of host memory. This address is the
first word of the data buffer.
1:12-35 NEXT BSD This is the physical address of the
first word of the next BSD. If there are
no more BSDs, this entire word must be
zero.
2:0-35 LENGTH This is the number of 4-bit nibbles that
are to be either read or written by the
port in this segment of the buffer.
3:0-35 RESERVED This word is reserved for future use.
9.2 Reading Data
To read data from a remote host, a host places a special
request packet on the port's command queue. This command packet,
called a request data packet, (REQDAT), contains the buffer names
and offsets of the sending and receiving buffers.
- 35 -
LCG CI Port Architecture Specification Page 36
The format of the Request Data (REQDAT) packet is:
+-------------------------------------------------------+
| FLINK | 0
+-------------------------------------------------------+
| BLINK | 1
+-------------------------------------------------------+
| RESERVED FOR SOFTWARE | 2
+----------+----------+-----------+----------+----------+
|00<---->07|08<---->15|16<----->23|24<---->31|32<---->35|
| STATUS | FLAGS | OPCODE | PORT | MBZ | 3
+----------+----------+-----------+----------+----------+
|00<-------------------------------------->31|32<---->35| 4
| XCT_ID | MBZ |
+--------------------------------------------+----------+
|00<-------------------------------------->31|32<---->35| 5
| XCT_ID | MBZ |
+--------------------------------------------+----------+
|00<-------------------------------------->31|32<---->35| 6
| XCT_LENGTH | MBZ |
+--------------------------------------------+----------+
|00<-------------------------------------->31|32<---->35| 7
| SND_NAME | MBZ |
+--------------------------------------------+----------+
|00<-------------------------------------->31|32<---->35| 8
| SND_OFFSET | MBZ |
+--------------------------------------------+----------+
|00<-------------------------------------->31|32<---->35| 9
| REC_NAME | MBZ |
+--------------------------------------------+----------+
|00<-------------------------------------->31|32<---->35| 10
| REC_OFFSET | MBZ |
+--------------------------------------------+----------+
| | 11
| RESERVED |
| FOR | Queue
| SOFTWARE | Length
| | - 1
+-------------------------------------------------------+
WORD:BITS NAME DESCRIPTION
========= ==== ===========
3:16-23 OPCODE OPCODE = 10 octal for REQDAT0
11 octal for REQDAT1
12 octal for REQDAT2
- 36 -
LCG CI Port Architecture Specification Page 37
4-5:0-31 XCT_ID This 64 bit word is for the port
driver's use; the port will never modify
this word. The contents of this location
are sent on the CI wire to the other
port.
6:0-31 XCT_LENGTH This is the number of 8-bit bytes that
are to be transferred. The actual
transfer may be made up of several data
packets; this length, however, is the
total length of the transfer.
7:0-31 SND_NAME This is the buffer name of the
originating buffer; the data will be
read from this buffer.
8:0-31 SND_OFFSET This is the number of 8-bit bytes,
measured from the first byte of the
buffer pointed to by the BHD, that the
other port will skip over before sending
data bytes to this port. These bytes are
skipped according to the mode specified
by the MODE bits in the first BSD and
any succeeding BSD.
9:0-31 REC_NAME This is the buffer name that the port
will write the data into.
10:0-31 REC_OFFSET This is number of byte positions to skip
in the receiving buffer before the first
byte of data transmitted over the CI
wire is written.
This causes the port to send a Data Request (DATREQ) packet to the
other port. The format of the Data Request (DATREQ) is identical
to the Request Data (REQDAT). The data is returned in as many
packets as necessary by the sending port. After execution, the
REQDAT command is normally placed on the MFree Queue unless:
1. The command fails due to an error or the non-receipt of a CI
ackowledgment packet.
2. A CI path fails but the command succeeds.
3. The R bit is set in the command.
- 37 -
LCG CI Port Architecture Specification Page 38
Upon receipt of a DATREQ packet, the port will build a RETDAT
command from an entry taken from the MFree Queue and may place it
upon Command Queue 2, 1 or 0 depending upon the opcode of the
DATREQ command. The port will put the RETDAT on a command queue
only if there is no internal command queue cache available.
The format of the Return Data (RETDAT) packet is:
+-------------------------------------------------------+
| FLINK | 0
+-------------------------------------------------------+
| BLINK | 1
+-------------------------------------------------------+
| RESERVED FOR SOFTWARE | 2
+----------+----------+-----------+----------+----------+
|00<---->07|08<---->15|16<----->23|24<---->31|32<---->35|
| STATUS | FLAGS | OPCODE | PORT | MBZ | 3
+----------+----------+-----------+----------+----------+
|00<-------------------------------------->31|32<---->35| 4
| XCT_ID | MBZ |
+--------------------------------------------+----------+
|00<-------------------------------------->31|32<---->35| 5
| XCT_ID | MBZ |
+--------------------------------------------+----------+
|00<-------------------------------------->31|32<---->35| 6
| REC_NAME | MBZ |
+--------------------------------------------+----------+
|00<-------------------------------------->31|32<---->35| 7
| REC_OFFSET | MBZ |
+--------------------------------------------+----------+
| | 8
| |
| DATA | Packet
| | Length
| | - 1
+-------------------------------------------------------+
WORD:BITS NAME DESCRIPTION
========= ==== ===========
3:16-23 OPCODE OPCODE = 21 octal for RETDAT
- 38 -
LCG CI Port Architecture Specification Page 39
4-5:0-31 XCT_ID This 64 bit word is for the port
driver's use; the port will never modify
this word. The contents of this location
are sent on the CI wire to the other
port.
9:0-31 REC_NAME This is the buffer name that the port
will write the data into.
10:0-31 REC_OFFSET This is number of byte positions to skip
in the receiving buffer before the first
byte of data transmitted over the CI
wire is written.
The last data packet of the transfer has the Last Packet (LP) flag
set to inform the receiving port that all of the data has been
sent. When the RETDAT command has been executed, it will be
returned to the MFree Queue unless:
1. The command fails due to an error.
2. A CI path fails but the command succeeds.
In these cases a Data Returned (DATRET) response is generated
and placed on the Response Queue by the sending port. The format
of a Data Returned (DATRET) response is identical to a Return Data
(RETDAT) except that no data appears in the command. The port
driver should never build a RETDAT command, but must be able to
tolerate finding a RETDAT command that the port placed upon its
command queue after receiving a DATREQ packet. The port receiving
the data will generate a Data Received (DATREC) response when all
of the data has been received (LP flag set) or an error occurs.
The format of the Data Received Response (DATREC) is:
- 39 -
LCG CI Port Architecture Specification Page 40
+-------------------------------------------------------+
| FLINK | 0
+-------------------------------------------------------+
| BLINK | 1
+-------------------------------------------------------+
| RESERVED FOR SOFTWARE | 2
+----------+----------+-----------+----------+----------+
|00<---->07|08<---->15|16<----->23|24<---->31|32<---->35|
| STATUS | FLAGS | OPCODE | PORT | MBZ | 3
+----------+----------+-----------+----------+----------+
|00<-------------------------------------->31|32<---->35| 4
| XCT_ID | MBZ |
+-------------------------------------------------------+
|00<-------------------------------------->31|32<---->35| 5
| XCT_ID | MBZ |
+-------------------------------------------------------+
| | 6
| |
| |
| UNDEFINED | Queue
| |Length
| | - 1
+-------------------------------------------------------+
WORD:BITS NAME DESCRIPTION
========= ==== ===========
3:16-23 OPCODE OPCODE = 61 octal (DATREC).
- 40 -
LCG CI Port Architecture Specification Page 41
9.3 Writing Data
To write data to a remote port, the port driver places a send
data (SNDDAT) command upon the port's command queue. The port will
then access the BHD and associated BSDs, transmitting as many data
packets as necessary to move the data from the sending buffer to
the receiving buffer's port, as designated in the port field of the
command. BHD's and BSD's are discussed earlier. This operation
takes place under Virtual Circuit control; any error causes the
circuit to be closed. After execution, the SNDDAT queue entry is
normally inserted on the MFree Queue but it will be placed on the
Response Queue as a Data Sent (DATSNT) response if one of the
following conditions occurs:
1. The command fails due to an error or non-receipt of a packet
acknowledgement. This closes the Virtual Circuit.
2. A CI path fails but the command succeeds on a retry.
3. The R bit of the command is set.
The format of a Send Data (SNDDAT) and a Data Sent (DATSNT)
are identical to a Request Data (REQDAT) command except that the
opcode is 20 octal.
As data is received, the data is stored by the receiving port.
Upon receipt of a Sent Data packet (SNTDAT) with the last packet
flag set, the receiving port will:
1. Remove an entry from the MFree Queue.
2. Generate a Return Confirm (RETCNF) command.
3. Insert the RETCNF command on Command Queue 3.
When executed a RETCNF sends a confirmation packet to the
initiating port. After execution the RETCNF command is normally
placed on the MFree Queue but it will be placed upon the Response
Queue as a Confirm Returned (CNFRET) Response if:
1. The command fails due to an error or non-receipt of a CI
acknowledgement packet.
- 41 -
LCG CI Port Architecture Specification Page 42
2. A CI path fails but the command succeeds.
If any data packet can not be properly processed by the
receiving port, it will:
1. Close the Virtual Circuit.
2. Remove an entry from the MFree Queue.
3. Build a Confirm Received (CNFREC) response and insert it on the
Response Queue.
The format of the response queue entry for a CNFRET packet is:
+-------------------------------------------------------+
| FLINK | 0
+-------------------------------------------------------+
| BLINK | 1
+-------------------------------------------------------+
| RESERVED FOR SOFTWARE | 2
+----------+----------+-----------+----------+----------+
|00<---->07|08<---->15|16<----->23|24<---->31|32<---->35|
| STATUS | FLAGS | OPCODE | PORT | MBZ | 3
+----------+----------+-----------+----------+----------+
|00<-------------------------------------->31|32<---->35|
| XCT_ID | MBZ | 4
+--------------------------------------------+----------+
|00<-------------------------------------->31|32<---->35|
| XCT_ID | MBZ | 5
+--------------------------------------------+----------+
| | 6
| RESERVED FOR SOFTWARE |
| | Queue
| |Length
| | - 1
+-------------------------------------------------------+
WORD:BITS NAME DESCRIPTION
========= ==== ===========
3:16-23 OPCODE OPCODE = 3 octal (CNFRET).
43 octal (CNFREC).
- 42 -
LCG CI Port Architecture Specification Page 43
4-5:0-31 XCT_ID This is the transaction ID that
accompanied all of the data packets.
This information is for the use of the
port driver.
- 43 -
LCG CI Port Architecture Specification Page 44
9.4 Data Formatting Modes
The KL10 CI port supports 3 data formatting modes. These
modes allow the operating system to specify how the data is packed
in the host's memory on memory read operations and how the port
should write the data into the host's memory on write operations.
These modes are specified in each Buffer Segment Descriptor for
data transfers via named buffers.The three modes are Industry
Compatible, High Density, and Core Dump. The first two of these
modes may also be specified for datagram and message transfers
through the various Send Message and Send Datagram commands. In
all of these modes, the most significant byte is left-justified so
that normal PDP10-style byte pointers may be used to access the
data. Note that it is not trivial to use standard PDP10-style byte
pointers in high density mode. The port will always start reading
or writing data with the most significant byte within the word.
The number of bits within each byte follows the PDP-11 standard.
The detailed descriptions of these packing modes follow.
9.4.1 Industry Compatible Mode -
The following figure illustrates the industry compatible mode
for mapping 8 bit CI bytes into 36 bit KL10 words.
INDUSTRY COMPATIBLE
KL10 WORD
0 7 8 15 16 23 24 31 32 35
+------------+------------+------------+------------+------+
| | | | | |
| Byte 0 | Byte 1 | Byte 2 | Byte 3 | ZERO |
| | | | | |
+------------+------------+------------+------------+------+
7 0 7 0 7 0 7 0
PLI BYTE
- 44 -
LCG CI Port Architecture Specification Page 45
9.4.2 Core Dump Mode -
The following figure illustrates the core dump mode for
mapping 8 bit CI bytes into 36 bit KL10 words. It is important to
note that in core dump mode the high order 4 bits (PLI bits 7 to 4)
of the last byte of each 36 bit word are written to the CI as
zeroes and are read from the CI, but are discarded by the port.
Core Dump BHD'S and BSD's are expected to be in multiples of 10.
even though only 9. are moved for each 36 bit word.
CORE DUMP
KL10 WORD
0 7 8 15 16 23 24 31 32 35
+------------+------------+------------+------------+------+
| | | | | Byte |
| Byte 0 | Byte 1 | Byte 2 | Byte 3 | |
| | | | | 4 |
+------------+------------+------------+------------+------+
7 0 7 0 7 0 7 0 3 0
PLI BYTE
9.4.3 High Density Mode -
The following figure illustrates the high density mode for
mapping 8 bit CI bytes into 36 bit KL10 words. In this mode, it is
important to realize that 9 bytes are packed into 2 36-bit words.
Byte 4 is split across a word boundary; the most significant 4
bits are stored in bits 32-35 of the first word of the pair and the
least significant 4 bits are stored in bits 0-3 of the second word
of the pair. Because of the 4 bits stored in the most significant
position of the second word, the remaining four bytes are stored
right-justified in the second word. Because of this byte-packing
mode, standard PDP10 byte pointers cannot be easily used to access
all of the bytes in the buffer.
- 45 -
LCG CI Port Architecture Specification Page 46
HIGH DENSITY FORMAT
KL10 WORD PAIR
FIRST WORD OF PAIR
0 7 8 15 16 23 24 31 32 35
+------------+------------+------------+------------+------+
| | | | | Byte |
| Byte 0 | Byte 1 | Byte 2 | Byte 3 | |
| | | | | 4 |
+------------+------------+------------+------------+------+
7 0 7 0 7 0 7 0 7 4
PLI BYTE
SECOND WORD OF PAIR
0 3 4 11 12 19 20 27 28 35
+------+------------+------------+------------+------------+
| Byte | | | | |
| | Byte 5 | Byte 6 | Byte 7 | Byte 8 |
| 4 | | | | |
+------+------------+------------+------------+------------+
3 0 7 0 7 0 7 0 7 0
PLI BYTE
10.0 STATUS FIELD
The STATUS field is updated by the port when it builds a
response queue entry. The various valid values of the STATUS field
are defined below. Note that bit 0 of the STATUS field defines the
definition of the remaining bits.
0 1 2 3 4 5 6 7
+-----+------+-----+-----+------+-----+-----+------+
| | | PATH A | PATH B |
| 0 | CLOS | ACK | NAK | NRSP | ACK | NAK | NRSP |
+-----+------+-----+-----+------+-----+-----+------+
- 46 -
LCG CI Port Architecture Specification Page 47
BIT NAME DESCRIPTION
==== ==== ===========
1 CLOS A packet had a retry failure on a path
but was transmitted successfully on the
other path. The path that failed and
the type of failure is indicated in the
Path bits. The indicated path is also
marked as being bad in the VCDT (Virtual
Circuit Descriptor Table).
2 PATH A ACK The packet was ACKed on this path.
3 PATH A NAK The packet was NAKed at least once on
this path.
4 PATH A NRSP The packet received No ReSPonse at least
once on this path.
5 PATH B ACK The packet was ACKed on this path.
6 PATH B NAK The packet was NAKed at least once on
this path.
7 PATH B NRSP The packet received No ReSPonse at least
once on this path.
- 47 -
LCG CI Port Architecture Specification Page 48
0 1 2 3 4 5 6 7
+-----+-----+-----+----+----+----+----+-----+
| 1 | A | B | ERROR TYPE |
+-----+-----+-----+----+----+----+----+-----+
BITS NAME DESCRIPTION
==== ==== ===========
1 PTH_A The error is associated with path A.
2 PTH_B The error is associated with path B.
3-7 ERROR TYPE
NO PATH ERRORS
==============
(402) ERROR = 1 => Access Control violation.
(404) ERROR = 2 => Invalid Buffer Name.
(406) ERROR = 3 => Buffer Length violation.
(410) ERROR = 4 => Packet size violation.
(412) ERROR = 5 => Transmit Buffer Parity Error
(414) ERROR = 6 => Local unrecognized command.
(416) ERROR = 7 => Mover Parity Predictor Error.
(420) ERROR = 10 => Invalid Remote port.
(422) ERROR = 11 => Non-recoverable Long Word
Count error
(424) ERROR = 12 => No legal path.
(426) ERROR = 13 => Command not legal in disabled
state.
(430) ERROR = 14 => PLI data PE in SRC byte.
(432) ERROR = 15 => PLI data PE in OPC byte.
(434) ERROR = 16 => PLI data PE in body.
(436) ERROR = 17 => Port disabled during
processing.
(440) ERROR = 20 => Xmit Buffer Parity Error.
(442) ERROR = 21 => Channel Error.
(444) ERROR = 22 => Spurious Channel Error.
Path B Errors
=============
(502) ERROR = 41 => Remote unrecognized command
(504) ERROR = 42 => Virtual Cicuit closed
(506) ERROR = 43 => Retries Exhausted (NAK)
(510) ERROR = 44 => Retries Exhausted (NRSP)
(512) ERROR = 45 => Transmitter Timeout
- 48 -
LCG CI Port Architecture Specification Page 49
PATH A Errors
=============
(602) ERROR = 101 => Remote unrecognized command
(604) ERROR = 102 => Virtual Cicuit closed
(606) ERROR = 103 => Retries Exhausted (NAK)
(610) ERROR = 104 => Retries Exhausted (NRSP)
(612) ERROR = 105 => Transmitter Timeout
PATHS A,B Errors
================
(704) ERROR = 142 => Virtual Cicuit closed
(706) ERROR = 143 => Retries Exhausted (NAK)
(710) ERROR = 144 => Retries Exhausted (NRSP)
(712) ERROR = 145 => Transmitter Timeout
- 49 -
LCG CI Port Architecture Specification Page 50
11.0 PORT PERFORMANCE MONITORING
The port microcode implements several counters which are under
the control of the port driver. The command queue entry Set
Counters (SETCNT) allows the port driver to point and/or clear the
counters. It also allows the port driver to enable or disable the
event counting. There is a mask that is used to control the
loading and enabling of the various event counters. For each
counter, there are 2 bits in the mask; the first bit enables the
counting of the event, and the second bit controls the clearing of
the event counter. The port driver may instruct the port to count
events for a specified port or a cumulative count for all ports.
The format of the SETCNT command is:
+-------------------------------------------------------+
| FLINK | 0
+-------------------------------------------------------+
| BLINK | 1
+-------------------------------------------------------+
| RESERVED FOR SOFTWARE | 2
+-----------+-----------+-----------+-----------+-------+
|00<----->07|08<----->15|16<----->23|24<----->31|32<->35|
| STATUS | FLAGS | OPC | MBZ | MBZ | 3
+-----------+-----------+-----------+-----------+-------+
|00<----------------------------------------->31|32<->35|
| XCT_ID | MBZ | 4
+-------------------------------------------------------+
|00<----------------------------------------->31|32<->35|
| XCT_ID | MBZ | 5
+-------------------------------------------------------+
|00<-------------------->19|20<-------------->31|32<->35|
| MASK | MBZ |THRESH | 6
+-------------------------------------------------------+
|00<----------------------------->23|24<----->31|32<->35|
| MBZ | PORT | MBZ | 7
+-------------------------------------------------------+
| RESERVED |
| FOR |
| SOFTWARE | Queue
| | Length
| | - 1
+-------------------------------------------------------+
- 50 -
LCG CI Port Architecture Specification Page 51
WORD:BITS NAME DESCRIPTION
========= ==== ===========
3:16-23 OPCODE OPCODE = 201 octal (SETCNT).
6:0-19 MASK This is the 18 bit mask used to control
the enabling and loading of the
counters.
6:0 PTH_A ACK If on, count ACKs received on Path A.
6:1 PTH_A ACKC If on, clear the counter.
6:2 PTH_A NAK If on, count NAKs received on Path A.
6:3 PTH_A NAKC If on, clear the counter.
6:4 PTH_A NRSP If on, count NO_RSPs received on Path A.
6:5 PTH_A NRSPC If on, clear the counter.
6:6 PTH_B ACK If on, count ACKs received on Path B.
6:7 PTH_B ACKC If on, clear the counter.
6:8 PTH_B NAK If on, count NAKs received on Path B.
6:9 PTH_B NAKC If on, clear the counter.
6:10 PTH_B NRSP If on, count NO_RSPs received on Path B.
6:11 PTH_B NRSPC If on, clear the counter.
6:12 DG DISCARDED The count of discarded datagrams because
of no DGFree Queue entries.
6:13 DG DISC CLR If on, clear the counter.
6:14 XMT CNT Count the packets transmitted to the
designated port.
6:15 XMT CLR If on, clear the counter.
6:16 RCV CNT Count the packets received from the
designated port.
6:17 RCV CLR If on, clear the counter.
- 51 -
LCG CI Port Architecture Specification Page 52
6:18 ERR_CNTR_CLR If on, clear all error counters
(see CNTRD response).
6:19 SET_THRESH If on, load Port Recoverable Error
Threshold value.
6:32-35 THRESH_VAL Value to load for Port Recoverable Error
Threshold.
7:24-31 PORT This is the designated port for which
the above counters will be tracked. If
the port value is set to 255, then the
counting will be done for all ports.
If the R (response) bit is set in the Set Counters command
(SETCNT) it will be placed on the Response Queue instead of the
DGFree Queue as a counters Set (CNTSET) command. The format for a
Counters Set (CNTSET) command is identical to the Set Counters
(SETCNT) command.
Every time the port enters the Enabled state, it will clear
all of the counters and set the PORT field to the "all ports"
value. The port driver reads these counters, with a Read Counters
(RDCNT) command. This command will return the information in the
various counters.
- 52 -
LCG CI Port Architecture Specification Page 53
The format of a Read Counters (RDCNT) command is:
+-------------------------------------------------------+
| FLINK | 0
+-------------------------------------------------------+
| BLINK | 1
+-------------------------------------------------------+
| RESERVED FOR SOFTWARE | 2
+-----------+-----------+-----------+-----------+-------+
|00<----->07|08<----->15|16<----->23|24<----->31|32<->35|
| STATUS | FLAGS | OPC | MBZ | MBZ | 3
+-----------+-----------+-----------+-----------+-------+
|00<----------------------------------------->31|32<->35|
| XCT_ID | MBZ | 4
+-------------------------------------------------------+
|00<----------------------------------------->31|32<->35|
| XCT_ID | MBZ | 5
+-------------------------------------------------------+
| | 6
| RESERVED |
| FOR | Queue
| SOFTWARE |Length
| | - 1
+-------------------------------------------------------+
WORD:BITS NAME DESCRIPTION
========= ==== ===========
3:16-23 OPCODE OPCODE = 202 octal (RDCNT).
- 53 -
LCG CI Port Architecture Specification Page 54
The port will always generate a Counters Read (CNTRD) response
to the Read Counters (RDCNT) command.
The format of the Counters Read (CNTRD) response is :
+-------------------------------------------------------+
| FLINK | 0
+-------------------------------------------------------+
| BLINK | 1
+-------------------------------------------------------+
| RESERVED FOR SOFTWARE | 2
+-----------+-----------+-----------+-----------+-------+
|00<----->07|08<----->15|16<----->23|24<----->31|32<->35|
| STATUS | FLAGS | OPC | MBZ | MBZ | 3
+-----------+-----------+-----------+-----------+-------+
|00<----------------------------------------->31|32<->35|
| XCT_ID | MBZ | 4
+-----------------------------------------------+-------+
|00<----------------------------------------->31|32<->35|
| XCT_ID | MBZ | 5
+-----------------------------------------------+-------+
| MICROCODE VERSION | 6
+-------------------------------------------------------+
| PATH A ACK COUNT | 7
+-------------------------------------------------------+
| PATH A NAK COUNT | 8
+-------------------------------------------------------+
| PATH A NO RESPONSE COUNT | 9
+-------------------------------------------------------+
| PATH B ACK COUNT | 10
+-------------------------------------------------------+
| PATH B NAK COUNT | 11
+-------------------------------------------------------+
| PATH B NO RESPONSE COUNT | 12
+-------------------------------------------------------+
| DATAGRAMS DISCARDED | 13
+-------------------------------------------------------+
| PACKETS TRANSMITTED | 14
+-------------------------------------------------------+
| PACKETS RECEIVED | 15
+-------------------------------------------------------+
- 54 -
LCG CI Port Architecture Specification Page 55
+-------------------------------+---------------+-------+
|0<-------------------------->23|24<--------->31|32<->35|
| |DESIGNATED PORT| | 16
+--------------------------+----+---------------+-------+
| PACKETS RECEIVED WITH CRC ERRORS | 17
+-------------------------------------------------------+
|0<--------------------->17|18<---------------------->35|
| MOVER PAR PRE ERRORS | CBUS PAR ERRORS | 18
+--------------------------+----------------------------+
|0<--------------------->17|18<---------------------->35|
| REG PLIPE ERRORS | DATA PLIPE ERRORS | 19
+--------------------------+----------------------------+
|0<--------------------->17|18<---------------------->35|
| CHANNEL ERRORS | EBUS PAR ERRORS | 20
+--------------------------+----------------------------+
|0<--------------------->17|18<---------------------->35|
| SPURR CHANNEL ERRORS | CBUS AVAIL TIMEOUTS | 21
+--------------------------+----------------------------+
|0<--------------------->17|18<---------------------->35|
| SPURR RCV ATTENTIONS | SPURR XMIT ATTENTIONS | 22
+--------------------------+----------------------------+
|0<--------------------->17|18<---------------------->35|
| XMIT BUFF PAR ERRORS | TRANSMIT TIMEOUTS | 23
+--------------------------+----------------------------+
| | 24
| RESERVED |
| FOR |
| SOFTWARE | Queue
| | Length
| | - 1
+-------------------------------------------------------+
WORD:BITS NAME DESCRIPTION
========= ==== ===========
3:12 ERROR This bit is set if the CNTRD was
generated as a result of a Planned
CRAM Parity Error (see KLCI Error spec).
3:16-23 OPCODE OPCODE = 202 octal (CNTRD).
Words 18-23 are called the Port Recoverable Error Counters.
The errors have a threshold initially set to 5 by the port during
initialization. The threshold can be changed by the port driver
with the SETCNT command. The threshold has a value range of 0-17.
- 55 -
LCG CI Port Architecture Specification Page 56
12.0 MISCELLANEOUS MESSAGES
12.1 Configuration Information
There is also a class of commands called ID commands. The
Request ID (REQID) command enables a host to determine the
configuration of the CI bus. This command generates an ID Request
(IDREQ) packet that is sent to the appropriate CI port. The
receiving port responds with a Send ID (SNDID) packet. A command
is taken from the DGFree Queue and placed on the Response Queue as
a ID Received (IDREC) command. All ID packets are datagram-class
packets. It is illegal for the port driver to place a Send ID
(SNDID) command on the command queue; SNDID packets are generated
by the port microcode only. When the port is in the Enabled state,
it will automatically respond to REQID packets without informing
the port driver.
The format of the REQID command and the IDREQ response is:
+-------------------------------------------------------+
| FLINK | 0
+-------------------------------------------------------+
| BLINK | 1
+-------------------------------------------------------+
| RESERVED FOR SOFTWARE | 2
+-----------+-----------+-----------+-----------+-------+
|00<----->07|08<----->15|16<----->23|24<----->31|32<->35|
| STATUS | FLAGS | OPC | PORT | MBZ | 3
+-----------+-----------+-----------+-----------+-------+
|00<----------------------------------------->31|32<->35|
| XCT_ID | MBZ | 4
+-----------------------------------------------+-------+
|00<----------------------------------------->31|32<->35|
| XCT_ID | MBZ | 5
+-----------------------------------------------+-------+
| | 6
| RESERVED |
| FOR | Queue
| SOFTWARE |Length
| | - 1
+-------------------------------------------------------+
WORD:BITS NAME DESCRIPTION
========= ==== ===========
3:16-23 OPCODE OPCODE = 5 octal (REQID)(IDREQ).
- 56 -
LCG CI Port Architecture Specification Page 57
4-5:0-31 XCT_ID Transaction ID for software.
The format of the ID Received (IDREC) response is:
+-------------------------------------------------------+
| FLINK | 0
+-------------------------------------------------------+
| BLINK | 1
+-------------------------------------------------------+
| RESERVED FOR SOFTWARE | 2
+-----------+-----------+-----------+-----------+-------+
|00<----->07|08<----->15|16<----->23|24<----->31|32<->35|
| STATUS | FLAGS | OPC | PORT | MBZ | 3
+-----------+-----------+-----------+-----------+-------+
|00<----------------------------------------->31|32<->35|
| XCT_ID | MBZ | 4
+-----------------------------------------------+-------+
|00<----------------------------------------->31|32<->35|
| XCT_ID | MBZ | 5
+---+-------------------------------+-----------+-------+
| 0 |01<------------------------->25|26<----->31|32<->35|
| D | MUST BE ZERO | PORT TYPE | MBZ | 6
+---+-------------------------------+-----------+-------+
|00<----------------------------------------->31|32<->35|
| CODE REV LEVEL | MBZ | 7
+----------------------------------+------------+-------+
|00<---------------------------->23|24<-------------->35|
| PORT FUNCTIONALITY | MBZ | 8
+--------------------+-------+-----+------------+-------+
|00<-------------->20|21<->22| 23 |24<------>31|32<->35|
| MBZ | STATE |MAINT| RESET PORT | MBZ | 9
+--------------------+-------+-----+------------+-------+
| | 10
| MUST BE ZERO |
| | Queue
| |Length
| | - 1
+-------------------------------------------------------+
WORD:BITS NAME DESCRIPTION
========= ==== ===========
3:16-23 OPCODE OPCODE = 53 octal (IDREC).
4-5:0-31 XCT_ID Transaction ID for software.
- 57 -
LCG CI Port Architecture Specification Page 58
6:0 DUAL The responding port has two paths.
6:1-25 MBZ Must be zero.
6:26-31 PORT TYPE This is the port type. (6 = KL10)
6:32-35 MBZ Must be zero.
7:0-31 CODE REV This is the microcode edit level.
8:0-23 FUNCTIONALITY This is a bit mask describing the
functionality of the originating port.
Refer to the corporate CI spec for the
bit definitions.
9:21-22 PORT STATE 0 => Unitialized
1 => Disabled
2 => Enabled
9:23 MAINTENANCE The port is in the maintenance state.
0 => not in maintenance mode
1 => in maintenance mode
9:24-31 RESET PORT This is the port number of the last port
which sent the port a reset packet. This
will always be our port number.
12.2 Maintenance Commands
The LCG port supports the CI maintenance packets for the
purpose of loading, starting, reading, and writing of other hosts'
memories. All maintenance packets on the CI are given
datagram-class service. Since the LCG CI port is not down-line
loadable and since a remote port may not start or stop LCG host
computers, all those maintenance packets are passed to the port
driver via the Response Queue when they are received over the CI
wire. Maintenance class commands are Reset (SNDRST), Start
(SNDSTRT), Request Maintenance Data (REQMDAT), Send Maintenance
Data (SNDMDAT), and Maintenance Confirm Return (MCNFRET). Because
the LCG CI port does not have a Maintenence state all Maintenence
class commands and responses are always valid.
- 58 -
LCG CI Port Architecture Specification Page 59
12.2.1 Resetting Remote Systems -
The Send Reset (SNDRST) command is used to reset a nodes'
interface and host to a known state (Unitialized/Maintenance) and
to halt the host computers' execution. See the Computer
Interconnect Spec for an explaination of port states.
The format of the Send Reset (SNDRST) command and Reset Sent
(RSTSNT) and Restart Received (RSTREC) responses is:
+-------------------------------------------------------+
| FLINK | 0
+-------------------------------------------------------+
| BLINK | 1
+-------------------------------------------------------+
| RESERVED FOR SOFTWARE | 2
+-----------+-----------+-----------+-----------+-------+
|00<----->07|08<----->15|16<----->23|24<----->31|32<->35|
| STATUS | FLAGS | OPC | PORT | MBZ | 3
+-----------+-----------+-----------+-----------+-------+
|00<----------------------------------------->31|32<->35|
| XCT_ID | MBZ | 4
+-----------------------------------------------+-------+
|00<----------------------------------------->31|32<->35|
| XCT_ID | MBZ | 5
+-----------------------------------------------+-------+
| |
| RESERVED FOR SOFTWARE | Queue
| |Length
| | - 1
+-------------------------------------------------------+
WORD:BITS NAME DESCRIPTION
========= ==== ===========
3:8 FORCE If this bit is off, the port will
compare the source port number to the
port number stored in the last resetting
port number. If there is a match, the
reset occurs, otherwise the packet is
ignored. If the bit is on, the port
will perform an unconditional reset.
3:16-23 OPCODE OPCODE = 6 octal (SNDRST)(RSTSNT).
46 octal (RSTREC)
- 59 -
LCG CI Port Architecture Specification Page 60
12.2.2 Starting Remote Systems -
A Remote System Start (SNDSTRT) command is sent to ports that
have been stopped by a SNDRST command. The start function will be
performed only if the RSTPORT field of the receiveing port is equal
to the source port number. A node may be started only by the port
that stopped it.
The format of the Send Start (SNDSTRT) command and Start Sent
(STRTSNT) and Start Received (STRTREC) responses is:
+-------------------------------------------------------+
| FLINK | 0
+-------------------------------------------------------+
| BLINK | 1
+-------------------------------------------------------+
| RESERVED FOR SOFTWARE | 2
+-----------+-----------+-----------+-----------+-------+
|00<----->07|08<----->15|16<----->23|24<----->31|32<->35|
| STATUS | FLAGS | OPC | PORT | MBZ | 3
+-----------+-----------+-----------+-----------+-------+
|00<----------------------------------------->31|32<->35|
| XCT_ID | MBZ | 4
+-----------------------------------------------+-------+
|00<----------------------------------------->31|32<->35|
| XCT_ID | MBZ | 5
+-----------------------------------------------+-------+
|00<----------------------------------------->31|32<->35|
| START ADDRESS | MBZ | 6
+-----------------------------------------------+-------+
| | 7
| |
| RESERVED FOR SOFTWARE | Queue
| |Length
| | - 1
+-------------------------------------------------------+
WORD:BITS NAME DESCRIPTION
========= ==== ===========
3:8 USE DSA If this bit is set, the remote host
should use the default starting address.
3:16-23 OPCODE OPCODE = 7 octal (SNDSTRT)(STRTSNT)
47 octal (STRTREC)
- 60 -
LCG CI Port Architecture Specification Page 61
5-6:0-31 START ADDRESS This is the starting address if the
Default Starting Address bit is not on.
- 61 -
LCG CI Port Architecture Specification Page 62
12.2.3 Reading Remote System Memories -
Remote host system memory may be read via the Request
Maintenance Data (REQMDAT) packet. Data read in this manner is
returned via the Returned Maintenance Data (RETMDAT) packet. The
resulting data will appear in the specified buffer. The receiving
buffer must have a valid buffer name and the BHD and BSDs must be
properly set up as if it were a normal data transfer. Since the
data transfer takes place as a datagram, a Virtual Circuit is not
necessary. Whenever a maintenance data transfer is complete, the
port microcode will build a Maintenance Confirm response on the
response queue. The format of the REQMDAT command queue entry is
the same as a REQDAT command except that the opcode is 16.
12.2.4 Writing Remote System Memories -
The Send Maintenance Data (SNDMDAT) command packet is used to
write a remote host's memory. The length of the data to be sent is
no greater than 512 bytes or 576 bytes, according to the Packing
Size bit in the FLAGS field.
- 62 -
LCG CI Port Architecture Specification Page 63
The format of the SNDMDAT command queue entry is:
+-------------------------------------------------------+
| FLINK | 0
+-------------------------------------------------------+
| BLINK | 1
+-------------------------------------------------------+
| RESERVED FOR SOFTWARE | 2
+----------+----------+-----------+----------+----------+
|00<---->07|08<---->15|16<----->23|24<---->31|32<---->35|
| STATUS | FLAGS | OPCODE | PORT | MBZ | 3
+----------+----------+-----------+----------+----------+
|00<-------------------------------------->31|32<---->35| 4
| XCT_ID | MBZ |
+--------------------------------------------+----------+
|00<-------------------------------------->31|32<---->35| 5
| XCT_ID | MBZ |
+--------------------------------------------+----------+
|00<-------------------------------------->31|32<---->35| 6
| XCT_LENGTH | MBZ |
+--------------------------------------------+----------+
|00<-------------------------------------->31|32<---->35| 7
| SND_NAME | MBZ |
+--------------------------------------------+----------+
|00<-------------------------------------->31|32<---->35| 8
| SND_OFFSET | MBZ |
+--------------------------------------------+----------+
|00<-------------------------------------->31|32<---->35| 9
| REC_NAME | MBZ |
+--------------------------------------------+----------+
|00<-------------------------------------->31|32<---->35| 10
| REC_OFFSET | MBZ |
+--------------------------------------------+----------+
| | 11
| RESERVED |
| FOR | Queue
| SOFTWARE | Length
| | - 1
+-------------------------------------------------------+
WORD:BITS NAME DESCRIPTION
========= ==== ===========
3:16-23 OPCODE OPCODE = 22 octal (SNTMDAT).
- 63 -
LCG CI Port Architecture Specification Page 64
12.2.5 Maintenance Confirmation Packets -
Whenever a maintenance data read or write is completely
successfully at the remote port, a Maintenance Confirm packet will
be sent back to the originating port. The port microcode will
place this confirmation packet on the tail of the response queue.
The format of the MCNF queue entry is:
+-------------------------------------------------------+
| FLINK | 0
+-------------------------------------------------------+
| BLINK | 1
+-------------------------------------------------------+
| RESERVED FOR SOFTWARE | 2
+-----------+-----------+-----------+-----------+-------+
|00<----->07|08<----->15|16<----->23|24<----->31|32<->35|
| STATUS | FLAGS | OPC | PORT | MBZ | 3
+-----------+-----------+-----------+-----------+-------+
|00<----------------------------------------->31|32<->35|
| XCT_ID | MBZ | 4
+-----------------------------------------------+-------+
|00<----------------------------------------->31|32<->35|
| XCT_ID | MBZ | 5
+-----------------------------------------------+-------+
| | 6
| |
| RESERVED FOR SOFTWARE | Queue
| |Length
| | - 1
+-------------------------------------------------------+
WORD:BITS NAME DESCRIPTION
========= ==== ===========
3:16-23 OPCODE OPCODE = 44 octal (MCNF)
4-5:0-31 XCT_ID This is the transaction ID specified in
the original maintenance read or write
memory command.
- 64 -
LCG CI Port Architecture Specification Page 65
12.3 Diagnostic Operation
12.3.1 Loopback -
There is a special diagnostic command called Send Loopback
(SNDLB). This command is a Datagram class command. This command
along with the IDREQ command are used by ports to verify and
isolate faults in cluster path connections. When a Loopback
Received (LBREC) packet is received it is placed on the Response
Queue even if it is from a port other that itself.
The format of the Send Loopback (SNDLB) command and Loopback
Sent (LBSNT) and Loopback received (LBREC) responses is:
+-------------------------------------------------------+
| FLINK | 0
+-------------------------------------------------------+
| BLINK | 1
+-------------------------------------------------------+
| RESERVED FOR SOFTWARE | 2
+-----------+-----------+-----------+-----------+-------+
|00<----->07|08<----->15|16<----->23|24<----->31|32<->35|
| STATUS | FLAGS | OPC | PORT | MBZ | 3
+-----------+-----------+----------++-----------+-------+
|00<---------------------------->19|20<-------------->35|
| MBZ | LENGTH | 4
+----------------------------------+--------------------+
| | 5
| |
| RESERVED FOR SOFTWARE |
| |
| |
+-----------------------------------------------+-------| Queue
|00<----------------------------------------->31|32<->35|Length
| 32 BIT CRC | MBZ | - 1
+-----------------------------------------------+-------+
WORD:BITS NAME DESCRIPTION
========= ==== ===========
3:16-23 OPCODE OPCODE = 15 octal (SNDLB).
- 65 -
LCG CI Port Architecture Specification Page 66
4:20-35 LENGTH This is the number of bytes, not
including the 4 bytes of CRC, that are
in the text portion of the LOOPBACK
packet.
??:0-31 32-BIT CRC The software must place the correct
32-bit CRC polynomial as calculated via
the algorithm in the VAX11/780
Architecture Handbook, page 9-13.
- 66 -
LCG CI Port Architecture Specification Page 67
12.4 Data Buffer Maintenance
The operating system requires the ability of flushing any
information cached by the port concerning data buffers that require
closing. The Close Buffer (CLSBUF) command forces the port to
close the data buffer named by clearing the "V" bit in the BHD and
flush any commands that are using that buffer. The commands that
are using the buffer and any subsequent command or packet will be
placed on the response queue with the STATUS field indicating an
error.
The format of the Close Buffer (CLSBUF) command and the Buffer
Closed response (BUFCLS) is:
+-------------------------------------------------------+
| FLINK | 0
+-------------------------------------------------------+
| BLINK | 1
+-------------------------------------------------------+
| RESERVED FOR SOFTWARE | 2
+-----------+-----------+-----------+-----------+-------+
|00<----->07|08<----->15|16<----->23|24<----->31|32<->35|
| STATUS | FLAGS | OPC | PORT | MBZ | 3
+-----------+-----------+-----------+-----------+-------+
|00<----------------------------------------->31|32<->35|
| BUFFER NAME | MBZ | 4
+-------------------------------------------------------+
| |
| | Queue
| |length
| | -1
+-------------------------------------------------------+
WORDS:BITES NAME DESCRIPTION
=========== ==== ===========
3:16-23 OPCODE OPCODE = 205 octal (CLSBUF)(BUFCLS)
4 BUFNAM This is the name of the buffer to be
closed. The buffer name is the same format
as is used in named buffer transfers.
See section 9.1 for a description of buffer
names.
- 67 -
LCG CI Port Architecture Specification Page 68
12.5 Illegal Packets
Any packet which is processed while the port is in the enabled
state and is an illegal packet due to either the opcode or some
other field will be treated as a datagram and placed on the host's
response queue. If the packet was locally generated, then the port
microcode will place the response on the tail of the response queue
with the STATUS field indicating a local unrecognized command. If
the packet was received over the CI wire, the STATUS field will be
set to remote unrecognized command. The entire contents of the
packet, up to the specified queue entry length, will be placed on
the response queue. Since this packet is considered to be a
datagram, it is subject to being discarded if there is a lack of
buffer space in the port. Also note that the packet length field
will be filled in by the port just as if it were a regular datagram
packet. The opcode will be unchanged.
- 68 -
LCG CI Port Architecture Specification Page 69
13.0 PORT REGISTERS
13.1 Control And Status Register (CSR)
The Control and Status Register (CSR) is the mechanism used by
the LCG CI port and it's host to communicate with each other. The
CSR is described in a later section and in the CI20 Port
Architecture Spec.
+---+----------------+-----------+ +---+----------------+-----------+
|BIT| BIT DEFINITION | RD/WR | |BIT| BIT DEFINITION | RD/WR |
| | +-----+-----+ | | +-----+-----+
|NO.| |KL10 | PORT| |NO.| |KL10 | PORT|
+---+----------------+-----+-----+ +---+----------------+-----+-----+
|00 |PORT PRESENT | R | H | |18 |CLEAR PORT | W | * |
+---+----------------+-----+-----+ +---+----------------+-----+-----+
|01 |DIAG RQST CSR | R | H | |19 |DIAG TEST EBUF | R/W | * |
+---+----------------+-----+-----+ +---+----------------+-----+-----+
|02 |DIAG CSR CHNG | R/H | H | |20 |DIAG GEN EBUS PE| R/W | * |
+---+----------------+-----+-----+ +---+----------------+-----+-----+
|03 | | * | * | |21 |DIAG SEL LAR | R/W | * |
+---+----------------+-----+-----+ +---+----------------+-----+-----+
|04 |RQST EXAM OR DEP| R/H | R/S | |22 |DIAG SINGLE CYC | R/W | * |
+---+----------------+-----+-----+ +---+----------------+-----+-----+
|05 |RQST INTERRUPT | R/H | R/S | |23 |SPARE | R/W | * |
+---+----------------+-----+-----+ +---+----------------+-----+-----+
|06 |CRAM PARITY ERR | R/C | H | |24 |EBUS PARITY ERR |H/R/C| R |
+---+----------------+-----+-----+ +---+----------------+-----+-----+
|07 |MBUS ERROR | R | H | |25 |FREE QUEUE ERR | R/C | R/S |
+---+----------------+-----+-----+ +---+----------------+-----+-----+
|08 | | * | * | |26 |DATA PATH ERR | R/C | R/S |
+---+----------------+-----+-----+ +---+----------------+-----+-----+
|09 | | * | * | |27 |CMD QUEUE AVAIL | R/S | R/C |
+---+----------------+-----+-----+ +---+----------------+-----+-----+
|10 | | * | * | |28 |RSP QUEUE AVAIL | R/C | R/S |
+---+----------------+-----+-----+ +---+----------------+-----+-----+
|11 |IDLE LOOP | R | R/W | |29 | | * | * |
+---+----------------+-----+-----+ +---+----------------+-----+-----+
|12 |DISABLE COMPLETE| R | R/W | |30 |DISABLE | R/S | R/C |
+---+----------------+-----+-----+ +---+----------------+-----+-----+
|13 |ENABLE COMPLETE| R | R/W | |31 |ENABLE | R/S | R/C |
+---+----------------+-----+-----+ +---+----------------+-----+-----+
|14 | | * | * | |32 |MPROC RUN | R/W | R/H |
+---+----------------+-----+-----+ +---+----------------+-----+-----+
- 69 -
LCG CI Port Architecture Specification Page 70
+---+----------------+-----+-----+ +---+----------------+-----+-----+
|15 |PORT ID CODE 00 | R | H | |33 |PIA 00 | R/W | R |
+---+----------------+-----+-----+ +---+----------------+-----+-----+
|16 |PORT ID CODE 01 | R | H | |34 |PIA 01 | R/W | R |
+---+----------------+-----+-----+ +---+----------------+-----+-----+
|17 |PORT ID CODE 02 | R | H | |35 |PIA 02 | R/W | R |
+---+----------------+-----+-----+ +---+----------------+-----+-----+
"*" indicates that the bit is not defined
"R" indicates that the bit is readable
"W" indicates that the bit is writeable
"C" indicates that the bit may be cleared only as a single bit
"S" indicates that the bit may be set only
"H" indicates that the bit is hardware controlled
13.2 Diagnostic Registers
In order to support the diagnosis of the port, all of the
packet buffer board and link registers are accessable to a
diagnostic program. These registers are accessed via two local
commands, the Read Register (RDREG) and Write Register (WRREG)
commands.
The following are the packet buffer board and link board
register definitions for the CI20.
NUMBER PLI FUNCTION
====== ============
1 Read Receiver Status
2 Reset Xmit Status
3 Read Buffer
4 Enable Link
5 Read Switches
6 Select Buffer
7 Load Xmit Buffer
10 Read Xmit Status
11 Abort Transmitter
12 Set Mode
13 Transmit
14 Read Node Address
15 Disable Link
- 70 -
LCG CI Port Architecture Specification Page 71
16 Release Receiver Buffer
17 Sync
These registers are accessable in all port states except the
uninitialized state. For details as to the format of these
registers, refer to the PILA Hardware Specification by Shu-Shia
Chow Dated 10-Aug-82.
- 71 -
LCG CI Port Architecture Specification Page 72
The format of the RDREG command is:
+-------------------------------------------------------+
| FLINK | 0
+-------------------------------------------------------+
| BLINK | 1
+-------------------------------------------------------+
| RESERVED FOR SOFTWARE | 2
+-----------+-----------+-----------+-----------+-------+
|00<----->07|08<----->15|16<----->23|24<----->31|32<->35|
| STATUS | FLAGS | OPC | MBZ | MBZ | 3
+-----------+-----------+-----------+-----------+-------+
|00<----->07|08<----->15|16<----->23|24<------------->35|
| MBZ | MBZ | REGISTER | MUST BE ZERO | 4
+-----------+-----------+-----------+-------------------+
| | 5
| |
| RESERVED FOR SOFTWARE | Queue
| |Length
| | - 1
+-------------------------------------------------------+
WORD:BITS NAME DESCRIPTION
========= ==== ===========
3:16-23 OPCODE OPCODE = 203 octal (RDREG)
4:16-23 REGISTER This is the register to read.
- 72 -
LCG CI Port Architecture Specification Page 73
The format of the Register Read Response (REGRD) follows:
+-------------------------------------------------------+
| FLINK | 0
+-------------------------------------------------------+
| BLINK | 1
+-------------------------------------------------------+
| RESERVED FOR SOFTWARE | 2
+-----------+-----------+-----------+-----------+-------+
|00<----->07|08<----->15|16<----->23|24<----->31|32<->35|
| STATUS | FLAGS | OPC | MBZ | MBZ | 3
+-----------+-----------+-----------+-----------+-------+
|00<----->07|08<----->15|16<----->23|24<------------->35|
| DATA | MBZ | REGISTER | MUST BE ZERO | 4
+-----------+-----------+-----------+-------------------+
| | 5
| |
| RESERVED FOR SOFTWARE | Queue
| |Length
| | - 1
+-------------------------------------------------------+
WORD:BITS NAME DESCRIPTION
========= ==== ===========
3:16-23 OPCODE OPCODE = 203 octal (REGRD).
4:0-7 DATA The data from the specified register is
returned in this field.
4:16-23 REGISTER This is the register that was read.
- 73 -
LCG CI Port Architecture Specification Page 74
To write a packet buffer board or link register, the Write
Register Command (WRREG) is used. The format of this command is:
+-------------------------------------------------------+
| FLINK | 0
+-------------------------------------------------------+
| BLINK | 1
+-------------------------------------------------------+
| RESERVED FOR SOFTWARE | 2
+-----------+-----------+-----------+-----------+-------+
|00<----->07|08<----->15|16<----->23|24<----->31|32<->35|
| STATUS | FLAGS | OPC | MBZ | MBZ | 3
+-----------+-----------+-----------+-----------+-------+
|00<----------------->15|16<----->23|24<----->31|32<->35|
| MBZ | REGISTER | DATA | MBZ | 4
+-----------------------+-----------+-----------+-------+
| | 5
| |
| RESERVED FOR SOFTWARE | Queue
| |Length
| | - 1
+-------------------------------------------------------+
WORD:BITS NAME DESCRIPTION
========= ==== ===========
3:16-23 OPCODE OPCODE = 204 octal (WRREG)
4:16-23 REGISTER This is the register to write.
4:24-31 DATA The data from this field is written to
the register specified in the Register
field.
- 74 -
LCG CI Port Architecture Specification Page 75
The format of the Register Written Response (REGWR) is:
+-------------------------------------------------------+
| FLINK | 0
+-------------------------------------------------------+
| BLINK | 1
+-------------------------------------------------------+
| RESERVED FOR SOFTWARE | 2
+-----------+-----------+-----------+-----------+-------+
|00<----->07|08<----->15|16<----->23|24<----->31|32<->35|
| STATUS | FLAGS | OPC | MBZ | MBZ | 3
+-----------+-----------+-----------+-----------+-------+
|00<----->07|08<----->15|16<----->23|24<----->31|32<->35|
| MBZ | MBZ | REGISTER | DATA | MBZ | 4
+-----------+-----------+-----------+-----------+-------+
| | 5
| |
| RESERVED FOR SOFTWARE | Queue
| |Length
| | - 1
+-------------------------------------------------------+
WORD:BITS NAME DESCRIPTION
========= ==== ===========
3:16-23 OPCODE OPCODE = 204 octal (REGWR)
4:16-23 REGISTER This is the register that was written.
4:24-31 DATA The data from this field was written to
the register specified in the Register
field.
- 75 -
LCG CI Port Architecture Specification Page 76
14.0 PROGRAMMING NOTES
14.1 PORT/PORT DRIVER COMMUNICATION
The port driver uses KL10 I/O instructions to communicate with
and control the port. Starting, stopping and communicating with
the port is done by the CONI (CONditions In) and CONO (CONditions
Out) instructions. Loading and dumping of the microcode and
manipulation of the port while it is not running is done by the
DATAI (DATA In) and DATAO (DATA Out) instructions.
The CSR (Control and Status Register) is the mechanism used by
the port and the port driver to indicate happenings or events. The
format of the CSR is:
0 1 2 3 4 5 6 7 8 10 12 14 18 20 22 24 26 28 30 32 35
+-+-+-+-+-+-+-+-+----+-+-+-+-+---+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+
|P|D|C| |R|R|C|M| |I|D|E| |PRT|C|D|D|D|D| |E|F|D|C|R| |D|E|R|PIA|
|P|R|C| |E|I|P|B| |L|C|C| |ID |L|T|E|S|S| |P|Q|P|Q|Q| |I|N|U| |
| |C| | |D| |E|E| | | | | | |R|E|P|L|S| |E|E|E|A|A| |S|A|N| |
+-+-+-+-+-+-+-+-+----+-+-+-+-+---+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+
BITS NAME DESCRIPTION
==== ==== ===========
0 PORT_PRESENT This bit is a hardware wire which
indicates the presence of a CI20.
1 DIAG_RQST_CSR Diagnostic purposes only.
2 DIAG_CSR_CHNG Diagnostic purposes only.
4 RQST_EXAM_DEP Diagnostic purposes only.
5 RQST_INT Diagnostic purposes only.
6 CRAM_PARITY_ERROR Set by the port hardware when a word
with bad parity is read from the CRAM.
7 MBUS_ERROR Set by the port hardware when more
than 1 driver connected to the MBUS
is detected as being active.
11 IDLE_LOOP Indicates when the microcode is in
the IDLE LOOP.
- 76 -
LCG CI Port Architecture Specification Page 77
12 DISABLE_COMPLETE Indicates the microcode has placed
the port in the Disabled state.
13 ENABLE_COMPLETE Indicates the microcode has placed
the port in the Enabled state.
15-17 PORT_ID A code for the type of port present.
Only 1 code is implemented at this
time (7).
18 CLEAR_PORT When set by the port driver the port
is placed in a RESET state.
19 DIAG_TEST_EBUF Diagnostic purposes only.
20 DIAG_GEN_EBUS_PE Diagnostic purposes only.
21 DIAG_SELECT_LAR Diagnostic purposes. If this bit
is set by the port drivr when the
port is not running a DATAI will
get the contents of the Last Address
Register. The LAR contains the
address of the last microinstruction
executed.
22 DIAG_SINGLE_CYCLE Diagnostic purposes only.
24 EBUS_PARITY_ERROR Set by the hardware when an EBUS
Parity Error is detected by the
port during an examine operation.
25 FREE_QUEUE_ERROR Set by the port microcode when it
detects that the Datagrm or Message
Free Queue in the PCB is empty.
26 DATA_PATH_ERROR Set by the port microcode when it
detects the MVR/FMTR Error Threshold
has been reached.
27 CMD_QUEUE_AVAIL Set by the port driver to indicate
that a command has been placed on a
previously empty Command Queue in
the PCB.
- 77 -
LCG CI Port Architecture Specification Page 78
28 RSP_QUEUE_AVAIL Set by the port microcode when a
command has been placed on the
Response Queue in the PCB and it
was empty.
30 DISABLE Set by the port driver to put the
port into the Disabled state.
31 ENABLE Set by the port driver to put the
port into the Enabled state.
32 MPROC_RUN Set by the port driver to start
the port microcode running.
33-35 PIA These bits are set by the port
driver to indicate what priority
level an interrupt by the port
will be given.
14.2 LOADING AND DUMPING OF THE PORT
The port microcode generates an interrupt when it sets certain
bits in the CSR, A CONI instruction is used by the port driver to
read the contents of the CSR. The CONO instruction is used by the
port driver to write bits 18-35 of the CSR. When the port driver
writes the CSR, bit 2 of the CSR gets set by the port hardware and
is sensed by the port microcode. When the port microcode reads the
CSR bit 2 in the CSR is cleared. Both DATAI and DATAO instructions
have multiple functions with this port. A DATAI with bit 21 of the
CSR set and bit 19 of the CSR not set reads the LAR (Last Address
Register) in the port. The LAR contains the address of the last
micro instruction fetched by the port. The format of the LAR as it
is returned by the DATAI is:
0 1 13 14 15 35
+-+----------------+-+----------------------------------+
|1| LAR |0| -1 |
+-+----------------+-+----------------------------------+
If bit 19 of the CSR is set a DATAI reads the EBUF. The EBUF
is the register in the port that is used to communicate over the
EBUS. The format of the EBUF is:
- 78 -
LCG CI Port Architecture Specification Page 79
0 35
+-------------------------------------------------------+
| EBUF |
+-------------------------------------------------------+
If both bits 19 and 21 of the CSR are not set a DATAI reads
either the right or left half of a microword (see DATAO). The
format of the microword half returned by the DATAI is:
0 5 6 35
+------+------------------------------------------------+
| -1 | BITS 0-29 OR 30-59 OF UWORD |
+------+------------------------------------------------+
A DATAO with bit 19 of the CSR set will write data to the
EBUF. The format of the written data is:
0 35
+-------------------------------------------------------+
| EBUF DATA TO BE WRITTEN |
+-------------------------------------------------------+
A DATAO with bit 19 of the CSR not set will write data to the
right or left half of the microword addressed by the RAR (Ram
Address Register). The format of the data to be written is:
0 5 6 35
+------+------------------------------------------------+
| N/A | BITS 0-29 OR 30-59 OF UWORD |
+------+------------------------------------------------+
- 79 -
LCG CI Port Architecture Specification Page 80
A DATAO with bit 0 of the CSR set and bit 19 of the CSR not
set will be write the RAR. The RAR is used to address the CRAM for
purpose of writing and reading the CRAM. The format of the data to
be written is:
0 1 13 14 15 35
+---+----------------+-+----------------------------------+
|N/A| RAR |L| N/A |
+---+----------------+-+----------------------------------+
If bit 14 is set the operation is performed on the left half
of the microword. If bit 14 is not set the operation is performed
on the right half.
14.3 Reset
When a port reset is issued to a running port, the hardware
will be reset and the port microcode PC will be set to zero so that
the power-up, self-check initialization code can be executed. This
will guarantee that the port will enter a well-defined state after
the reset. It is important to realize that all of the port
registers will be set to the power-up state.
14.4 Initialization
When the port is powered-up, there is no valid microcode in
the CRAM; valid microcode must be loaded and started. During the
initialization, the port microcode will perform a self-check test
and generate a planned CRAM Parity Error if it detects any error
(see KLCI ERROR SPEC). The microcode will also clear all of the
registers.
15.0 ERROR CONDITIONS
The port microcode is capable of retrying many operations that
fail; this allows a large part of the error recovery to be built
into the port microcode and removed from the port driver (see KLCI
ERROR SPEC for definition of errors).
- 80 -
LCG CI Port Architecture Specification Page 81
While the port is waiting for port driver intervention, it
will not be emptying packets out of the receive buffer so packets
may be NAKed and Virtual Circuits closed by other ports during this
time. The class of non-recoverable errors that cause the port to
shut down in this manner are:
1. Inability to get the queue interlock,
2. Non-recoverable internal port error (other than CRAM parity
error),
3. Non-recoverable EBUS/CBUS error.
- 81 -
LCG CI Port Architecture Specification Page 82
16.0 IMPLEMENTED FUNCTIONALITY
The LCG CI ports will support the following packet formats:
1. DG - send/receive
2. MSG - send/receive
3. SNDDAT - send/receive
4. REQDAT - send/receive
5. RETDAT - send/receive
6. REQID/ID - send/receive
7. CNF - send/receive
8. LB - send/receive
9. REQMDAT - send
10. RETMDAT - receive
11. SNDMDAT - send
12. MCNF - send
13. RST - send
14. STRT - send
If the port is in the enabled state, the port will place any
maintenance packet it receives over the CI, except RETMDAT, on the
port driver's response queue.
- 82 -
LCG CI Port Architecture Specification Page 83
17.0 OPCODE SUMMARY
The following is a table of command and local-response opcode
assignments.
COMMAND/RESPONSE CI PACKET DEC OCTAL BINARY
SNDDG/DGSNT DG 1 1 0000 0001
SNDMSG/MSGSNT MSG 2 2 0000 0010
<none>/CNFRET CNF 3 3 0000 0011
REQID/IDREQ IDREQ 5 5 0000 0101
SNDRST/RSTSNT RST 6 6 0000 0110
SNDSTRT/STRTSNT STRT 7 7 0000 0111
REQDAT0/DATREQ0 DATREQ0 8 10 0000 1000
REQDAT1/DATREQ1 DATREQ1 9 11 0000 1001
REQDAT2/DATREQ2 DATREQ2 10 12 0000 1010
SNDLB/LBSNT LB 13 15 0000 1101
REQMDAT/MDATREQ MDATREQ 14 16 0000 1110
SNDDAT/DATSNT SNTDAT 16 20 0001 0000
<none>/DATRET RETDAT 17 21 0001 0001
SNDMDAT/MDATSNT SNTMDAT 18 22 0001 0010
SETCKT/CKTSET <none> 128 200 1000 0000
SETCNT/CNTSET <none> 129 201 1000 0001
RDCNT/CNTRD <none> 130 202 1000 0010
RDREG/REGRD <none> 131 203 1000 0011
WRTREG/REGWRT <none> 132 204 1000 0100
CLSBUF/BUFCLS <none> 133 205 1000 0101
- 83 -
LCG CI Port Architecture Specification Page 84
The following table contains the opcode assignments for
remotely-generated responses.
RESPONSE CI PACKET DEC OCTAL BINARY
DGREC DG 33 41 0010 0001
MSGREC MSG 34 42 0010 0010
CNFREC CNF 35 43 0010 0011
MCNFREC MCNF 36 44 0010 1001
IDREC ID 43 53 0010 1011
LBREC LB 45 55 0010 1101
DATREC RETDAT 49 61 0011 0001
MDATREC RETMDAT 51 63 0011 0011
- 84 -
LCG CI Port Architecture Specification Page Index-1
INDEX
BHD, 30-31, 41, 44, 62 Industry Compatible, 44
BSD, 30-31, 41, 62 Datagram, 17, 21
Buffer Descriptor, 30 Diagnostic Operation, 65
Buffer Descriptor Table
BDT, 9, 30 EPT, 14
Buffer Descriptors, 9 Error Words, 10
Buffer Header Descriptor,
30-31, 41, 44, 62 FLAGS, 19, 23
VALID, 31
Buffer Name, 30, 62 Illegal Packets, 68
Buffer Segment Descriptor, INDEX, 31
30-31, 41, 62 IPA20-L, 10
CCW, 14 KEY, 31
Channel Command Word, 14
CI, 6 Last Packet, 39
Command Loopback, 65
CLSBUF, 67
RDCNT, 52 Maintenance, 17, 58
RDREG, 70 Message, 17, 28
REQDAT, 35
REQID, 56 NR, 23
REQMDAT, 62 NS, 23
SETCKT, 23-24
SETCNT, 50 Opcode, 18
SNDDAT, 41 local, 18
SNDDG, 21 remote, 18
SNDLB, 65
SNDMDAT, 62 Path Selection, 24
SNDMSG, 28 PCB, 13
SNDRST, 59 Port Control Block, 7, 16
SNDSTRT, 60 Port registers
WRREG, 70, 74 CSR, 69
Configuration, 17, 56 Port State, 15, 58, 68
IDREC, 56-57 disabled, 15
REQID, 56 enabled, 15
Confirmation, 41 uninitialized, 15
CST, 23 Programming
CONI, 76
Data, 17, 30 CONO, 76
512, 62 CSR, 76
576, 62 DATAI, 76
Data Buffer Maintenance, 67 DATAO, 76
- 85 -
LCG CI Port Architecture Specification Page Index-2
Data Formats, 44 error conditions, 80
Core Dump, 44-45 initialization, 80
High Density, 44-45 reset, 80
- 86 -
LCG CI Port Architecture Specification Page Index-3
Queues, 10
BLINK, 16
Command, 7, 15, 17
FLINK, 16
Free, 7, 16
Response, 7, 16-17
Self-Directed Commands, 17
Remotely-generated response
CNFREC, 42
DATREC, 39
DGREC, 22
IDREC, 57
LBREC, 65
MSGREC, 29
RSTREC, 59
STRTREC, 60
Response
BUFCLS, 67
CKTSET, 26
CNFRET, 41
CNTRD, 54
CNTSET, 52
DATRET, 39
DATSNT, 41
DGSNT, 22
IDREQ, 56
LBSNT, 65
MCNF, 64
MSGSNT, 29
REGRD, 73
REGWR, 75
RETMDAT, 62
RSTSNT, 59
STRTSNT, 60
Response Bit, 17, 22
STATUS, 46, 68
Virtual Circuit, 16-17, 23
- 87 -