mirrors/emacs
1
Fork 0
mirror of git://git.sv.gnu.org/emacs.git synced 2026-04-27 16:51:06 -07:00
Code Issues Projects Releases Wiki Activity

Convert alloc-frame, diag, interface-c, and io design documents to restructuredtext. and that's the lot! (cbs and poolmvt are already converted on mps/branch/2013-05-17/emergency).

Browse source 
Copied from Perforce
 Change: 182275
 ServerID: perforce.ravenbrook.com
This commit is contained in:
Gareth Rees 2013-05-28 11:38:46 +01:00
parent 03a844556e
commit 4e26e67f3a
22 changed files with 2520 additions and 2538 deletions
Download patch file Download diff file Expand all files Collapse all files

513
mps/design/alloc-frame.txt Normal file
View file

@ -0,0 +1,513 @@
.. mode: -*- rst -*-
Allocation frame protocol
=========================
:Tag: design.mps.alloc-frame
:Author: Tony Mann
:Date: 1998-10-02
:Status: incomplete document
:Revision: $Id$
:Copyright: See `Copyright and License`_.
Introduction
------------
_`.intro`: This document explains the design of the support for
allocation frames in MPS.
_`.readership`: This document is intended for any MM developer.
_`.overview`: Allocation frames are used for implementing stack pools;
each stack frame corresponds to an allocation frame. Allocation frames
may also be suitable for implementing other sub-pool groupings, such
as generations and ramp allocation patterns.
_`.overview.ambition`: We now believe this to be a design that loses
too many advantages of stack allocation for questionable gains. The
requirements are almost entirely based on unanalysed anecdote, instead
of actual clients.
.. note::
We plan to supersede this with a stack pool design at some point
in the future. Pekka P. Pirinen, 2000-03-09.
Definitions
-----------
_`.def.alloc-frame`: An allocation frame is a generic name for a
device which groups objects together with other objects at allocation
time, and which may have a parent/child relationship with other
allocation frames.
Purpose
-------
_`.purpose.stack-allocation`: The allocation frame protocol is
intended to support efficient memory management for stack allocation,
that is, the allocation of objects which have dynamic extent.
_`.purpose.general`: The allocation frame protocol is intended to be
sufficiently general that it will be useful in supporting other types
of nested allocation patterns too. For example, it could be used to
for EPVM-style save and restore, ramp allocation patterns or
generations.
Requirements
------------
Known requirements
..................
_`.req.stack-alloc`: Provide a interface for clients to describe a
stack allocation pattern, as an alternative to using the control
stack.
_`.req.efficient`: Permit an implementation which is comparable in
efficiency to allocating on the control stack.
_`.req.ap`: Support allocation via allocation points (APs).
_`.req.format`: Support the allocation of formatted objects.
_`.req.scan`: Ensure that objects in allocation frames can participate
in garbage collection by being scanned.
_`.req.fix`: Ensure that objects in allocation frames can participate
in garbage collection by accepting Fix requests.
_`.req.condemn`: Ensure that objects in allocation frames can
participate in garbage collection by being condemned.
_`.attr.locking`: Minimize the synchronization cost for the creation
and destruction of frames.
Proto-requirements
..................
_`.proto-req`: The following are possible requirements that might be
important in the future. The design does not necessarily meet all
these requirements, but it does consider them all. Each requirement
either has direct support in the framework, or could be supported with
future additions to the framework.
_`.req.parallels`: The allocation frame protocol should provide a
framework for exploiting the parallels between stack extents,
generations and "ramps".
_`.req.pool-destroy`: It should be possible to use allocation frames
to free all objects in a pool without destroying the pool.
_`.req.epvm`: It should be possible to implement EPVM-style save and
restore operations by creating and destroying allocation frames.
_`.req.subst`: It should be possible to substitute a stack pool with a
GC-ed pool so that erroneous use of a stack pool can be detected.
_`.req.format-extensions`: It should be possible for stack pools to
utilize the same format as any other pool, including debugging formats
that include fenceposting, etc.
_`.req.mis-nest`: Should ensure "mis-nested" stacks are safe.
_`.req.non-top-level`: Should support allocation in the non-top stack
extent.
_`.req.copy-if-necessary`: Should ensure that stack pools can support
"copy-if-necessary" (so that low-level system code can heapify stack
objects.)
_`.req.preserve`: When an object is in an allocation frame which is
being destroyed, it should be possible to preserve that object in the
parent frame.
_`.req.contained`: Should allow clients to ask if an object is
"contained" in a frame. The object is contained in a frame if it is
affected when the frame is ended.
_`.req.alloc-with-other`: Should allow clients to allocate an object
in the same frame as another object.
Overview
--------
_`.frame-classes`: The protocol supports different types of allocation
frames, which are represented as "frame classes". It's up to pools to
determine which classes of allocation frames they support. Pools which
support more than one frame class rely on the client to indicate which
class is currently of interest. The client indicates this by means of
an operation which stores the class in the buffer to which the
allocation point is attached.
_`.frame-handles`: Allocation frames are described via abstract "frame
handles". Pools may choose what the representation of a frame handle
should be. Frame handles are static, and the client need not store
them in a GC root.
_`.lightweight-frames`: The design includes an extension to the
allocation point protocol, which permits the creation and destruction
of allocation frames without the necessity for claiming the arena
lock. Such frames are called "lightweight frames".
Operations
----------
_`.op.intro`: Each operation has both an external (client) interface
and an internal (MPS) interface. The external function takes an
allocation point as a parameter, determines which buffer and pool it
belongs to, and calls the internal function with the buffer and pool
as parameters.
_`.op.obligatory`: The following operations are supported on any
allocation point which supports allocation frames:-
_`.operation.push`: The ``PushFrame()`` operation creates a new
allocation frame of the currently chosen frame class, makes this new
frame the current frame, and returns a handle for the frame.
_`.operation.pop`: The ``PopFrame()`` operation takes a frame handle
as a parameter. Some pool classes might insist or assume that this is
the handle for the current frame. It finds the parent of that frame
and makes it the current frame. The operation indicates that all
children of the new current frame contain objects which are likely to
be dead. The reclaim policy is up to the pool; some classes might
insist or assume that the objects must be dead, and eagerly free them.
Note that this might introduce the possibility of leaving dangling
pointers elsewhere in the arena. If so, it's up to the pool to decide
what to do about this.
_`.op.optional`: The following operations are supported for some
allocation frames, but not all. Pools may choose to support some or
all of these operations for certain frame classes. An unsupported
operation will return a failure value:-
_`.operation.select`: The ``SelectFrame()`` operation takes a frame
handle as a parameter and makes that frame the current frame. It does
not indicate that any children of the current frame contain objects
which are likely to be dead.
_`.operation.select-addr`: The ``SelectFrameOfAddr()`` operation takes
an address as a parameter and makes the frame of that address the
current frame. It does not indicate that any children of the current
frame contain objects which are likely to be dead.
_`.operation.in-frame`: The ``AddrInFrame()`` operation determines
whether the supplied address is the address of an object allocated in
the supplied frame, or any child of that frame.
_`.operation.set`: The ``SetFrameClass()`` operation takes a frame
class and an allocation point as parameters, and makes that the
current frame class for the allocation point. The next ``PushFrame()``
operation will create a new frame of that class.
Interface
---------
External types
..............
_`.type.client.frame-handle`: Frame handles are defined as the abstract
type ``mps_frame_t``.
``typedef struct mps_frame_class_s *mps_frame_class_t``
_`.type.client.frame-class`: Frame classes are defined as an abstract
type.
_`.type.client.frame-class.access`: Clients access frame classes by
means of dedicated functions for each frame class.
External functions
..................
_`.fn.client.push`: ``mps_ap_frame_push()`` is used by clients to
invoke the ``PushFrame()`` operation. For lightweight frames, this
might not invoke the corresponding internal function.
_`.fn.client.pop`: ``mps_ap_frame_pop()`` is used by clients to invoke
the ``PopFrame()`` operation. For lightweight frames, this might not
invoke the corresponding internal function.
``mps_res_t mps_ap_frame_select(mps_ap_t buf, mps_frame_t frame)``
_`.fn.client.select`: This following function is used by clients to
invoke the ``SelectFrame()`` operation.
``mps_res_t mps_ap_frame_select_from_addr(mps_ap_t buf, mps_addr_t addr)``
_`.fn.client.select-addr`: This function is used by clients to invoke
the ``SelectFrameOfAddr()`` operation.
``mps_res_t mps_ap_addr_in_frame(mps_bool_t *inframe_o, mps_ap_t buf, mps_addr_t *addrref, mps_frame_t frame)``
_`.fn.client.in-frame`: This function is used by clients to invoke the
``AddrInFrame()`` operation.
``mps_res_t mps_ap_set_frame_class(mps_ap_t buf, mps_frame_class_t
class)``
_`.fn.client.set`: This function is used by clients to invoke the
``SetFrameClass()`` operation.
``mps_frame_class_t mps_alloc_frame_class_stack(void)``
_`.fn.client.stack-frame-class`: This function is used by clients to
access the frame class used for simple stack allocation.
Internal types
..............
``typedef struct AllocFrameStruct *AllocFrame``
_`.type.frame-handle`: Frame handles are defined as an abstract type.
``typedef struct AllocFrameClassStruct *AllocFrameClass``
_`.type.frame-class`: Frame classes are defined as an abstract type.
``typedef Res (*PoolFramePushMethod)(AllocFrame *frameReturn, Pool pool, Buffer buf)``
_`.fn.push`: A pool method of this type is called (if needed) to
invoke the ``PushFrame()`` operation.
``typedef Res (*PoolFramePopMethod)(Pool pool, Buffer buf, AllocFrame frame)``
_`.fn.pop`: A pool method of this type is called (if needed)
to invoke the PopFrame operation:
``typedef Res (*PoolFrameSelectMethod)(Pool pool, Buffer buf, AllocFrame frame)``
_`.fn.select`: A pool method of this type is called to invoke the
``SelectFrame()`` operation.
``typedef Res (*PoolFrameSelectFromAddrMethod)(Pool pool, Buffer buf, Addr addr)``
_`.fn.select-addr`: A pool method of this type is called to invoke the
``SelectFrameOfAddr()`` operation.
``typedef Res (*PoolAddrInFrameMethod)(Bool *inframeReturn, Pool pool, Seg seg, Addr *addrref, AllocFrame frame)``
_`.fn.in-frame`: A pool method of this type is called to invoke the
``AddrInFrame()`` operation.
``typedef Res (*PoolSetFrameClassMethod)(Pool pool, Buffer buf, AllocFrameClass class)``
_`.fn.set`: A pool method of this type is called to invoke the
``SetFrameClass()`` operation.
Lightweight frames
-------------------
Overview
........
_`.lw-frame.overview`: Allocation points provide direct support for
lightweight frames, and are designed to permit PushFrame and PopFrame
operations without the need for locking and delegation to the pool
method. Pools can disable this mechanism for any allocation point, so
that the pool method is always called. The pool method will be called
whenever synchronization is required for other reasons (e.g. the
buffer is tripped).
_`.lw-frame.model`: Lightweight frames offer direct support for a
particular model of allocation frame use, whereby the PushFrame
operation returns the current allocation pointer as a frame handle,
and the PopFrame operation causes the allocation pointer to be reset
to the address of the frame handle. This model should be suitable for
simple stack frames, where more advanced operations like SelectFrame
are not supported. It may also be suitable for more advanced
allocation frame models when they are being used simply. The use of a
complex operation always involves synchronization via locking, and the
pool may disable lightweight synchronization temporarily at this time.
State
.....
_`.lw-frame.states`: Allocation points supporting lightweight frames
will be in one of the following states:
============ ================================================================
Valid Indicates that ``PushFrame()`` can be a lightweight
operation and need not be synchronized.
PopPending Indicates that there has been a ``PopFrame()`` operation
that the pool must respond to.
Disabled Indicates that the pool has disabled support for lightweight
operations for this AP.
============ ================================================================
These states are in addition to the state normally held by an AP for
allocation purposes. An AP will be in the Disabled state at creation.
_`.lw-frame.transitions`: State transitions happen under the following
circumstances:
======================= ====================================================
Valid → PopPending As a result of a client ``PopFrame()``
operation.
Valid → Disabled At the choice of the pool (for example, when
responding to a ``SelectFrame()`` operation).
PopPending → Valid At the choice of the pool, when processing a
``PopFrame()``.
PopPending → Disabled At the choice of the pool, when processing a
``PopFrame()``.
Disabled → Valid At the choice of the pool.
Disabled → Popframe Illegal.
======================= ====================================================
_`.lw-frame.state-impl`: Each AP contains 3 additional fields to hold this state::
mps_addr_t frameptr;
mps_bool_t enabled;
mps_bool_t lwPopPending;
_`.lw-frame.enabled`: The ``enabled`` slot holds the following values for
each state:
========== ==========
Valid ``TRUE``
PopPending ``TRUE``
Disabled ``FALSE``
========== ==========
_`.lw-frame.frameptr`: The ``frameptr`` slot holds the following values
for each state:
========== ============================================
Valid ``NULL``
PopPending Frame handle for most recently popped frame.
Disabled ``NULL``
========== ============================================
_`.lw-frame.lwPopPending`: The ``lwPopPending`` slot holds the
following values for each state:
========== =========
Valid ``FALSE``
PopPending ``TRUE``
Disabled ``FALSE``
========== =========
_`.lw-frame.state-for-gc`: It is not necessary for the tracer, format
code, pool, or any other part of the GC support in MPS to read either
of the two additional AP fields in order to scan a segment which
supports a lightweight allocation frame.
Synchronization
...............
_`.lw-frame.sync`: The purpose of the design is that mutator may
access the state of an AP without locking with MPS (via the external
functions). The design assumes the normal MPS restriction that an
operation on an AP may only be performed by a single mutator thread at
a time. Each of the operations on allocation frames counts as an
operation on an AP.
_`.lw-frame.sync.pool`: Pools are permitted to read or modify the
lightweight frame state of an AP only in response to an operation on
that AP.
_`.lw-frame.sync.external`: The external functions
``mps_ap_frame_push()`` and ``mps_ap_frame_pop()`` are permitted to
read the values of the ``enabled`` and ``frameptr`` fields for the
supplied AP without claiming the arena lock. They are permitted to
modify the ``frameptr`` field if and only if ``enabled == FALSE``.
_`.lw-frame.sync.trip`: When a buffer trip happens, and the trap
wasn't set by MPS itself (that is, it wasn't because of a flip or for
logging), then the buffer code must check whether the AP has state
PopPending. If it does, the buffer code must call the Pool.
Implementation
..............
_`.lw-frame.push`: The external ``PushFrame()`` operation
(``mps_ap_frame_push()``) performs the following operations::
IF (!APIsTrapped(ap) && StateOfFrame(ap) == Valid && ap->init == ap->alloc)
*frame_o = ap->init;
ELSE
WITH_ARENA_LOCK
PerformInternalPushFrameOperation(...)
END
END
_`.lw-frame.pop`: The external ``PopFrame()`` operation
(``mps_ap_frame_pop()``) performs the following operations::
IF (StateOfFrame(ap) != Disabled)
TrapAP(ap); /* ensure next allocation or push involves the pool */
ap->frameptr = frame;
ap->lwpopPending = TRUE;
ELSE
WITH_ARENA_LOCK
PerformInternalPopFrameOperation(...)
END
END
Document History
----------------
- 1998-10-02 Tony Mann. Incomplete document.
- 2002-06-07 RB_ Converted from MMInfo database design document.
- 2013-05-23 GDR_ Converted to reStructuredText.
.. _RB: http://www.ravenbrook.com/consultants/rb/
.. _GDR: http://www.ravenbrook.com/consultants/gdr/
Copyright and License
---------------------
Copyright © 2013 Ravenbrook Limited. All rights reserved.
<http://www.ravenbrook.com/>. This is an open source license. Contact
Ravenbrook for commercial licensing options.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. Redistributions in any form must be accompanied by information on how
to obtain complete source code for this software and any
accompanying software that uses this software. The source code must
either be included in the distribution or be available for no more than
the cost of distribution plus a nominal fee, and must be freely
redistributable under reasonable conditions. For an executable file,
complete source code means the source code for all modules it contains.
It does not include source code for modules or files that typically
accompany the major components of the operating system on which the
executable file runs.
**This software is provided by the copyright holders and contributors
"as is" and any express or implied warranties, including, but not
limited to, the implied warranties of merchantability, fitness for a
particular purpose, or non-infringement, are disclaimed. In no event
shall the copyright holders and contributors be liable for any direct,
indirect, incidental, special, exemplary, or consequential damages
(including, but not limited to, procurement of substitute goods or
services; loss of use, data, or profits; or business interruption)
however caused and on any theory of liability, whether in contract,
strict liability, or tort (including negligence or otherwise) arising in
any way out of the use of this software, even if advised of the
possibility of such damage.**

517
mps/design/alloc-frame/index.html
View file

@ -1,517 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<title>Design of the MPS allocation frame protocol</title>
</head>
<body bgcolor="#FFFFFF" text="#000000" link="#000099" vlink="#660066" alink="#FF0000">
<div align="center">
<p>
<a href="/">Ravenbrook</a> /
<a href="/project/">Projects</a> /
<a href="/project/mps/">Memory Pool System</a> /
<a href="/project/mps/master/">Master Product Sources</a> /
<a href="/project/mps/master/design/">Design Documents</a>
</p>
<p><i><a href="/project/mps/">Memory Pool System Project</a></i></p>
<hr />
</div>
<pre>
DESIGN OF THE MPS ALLOCATION FRAME PROTOCOL
design.mps.alloc-frame
incomplete doc
tony 1998-10-02
INTRODUCTION
<a id="intro" name="intro">.intro</a>: This document explains the design of the support for allocation frames
in MPS.
<a id="readership" name="readership">.readership</a>: This document is intended for any MM developer.
<a id="overview" name="overview">.overview</a>: Allocation frames are used for implementing stack pools; each stack
frame corresponds to an allocation frame. Allocation frames may also be
suitable for implementing other sub-pool groupings, such as generations and
ramp allocation patterns. <a id="overview.ambition" name="overview.ambition">.overview.ambition</a>: We now believe this to be a
design that loses too many advantages of stack allocation for questionable
gains. The requirements are almost entirely based on unanalysed anecdote,
instead of actual clients. [We plan to supersede this with a stack pool design
at some point in the future. pekka 2000-03-09]
<a id="hist.0" name="hist.0">.hist.0</a>: Written by Tony 1998-10-02
DEFINITIONS
<a id="def.alloc-frame" name="def.alloc-frame">.def.alloc-frame</a>: An allocation frame is a generic name for a device which
groups objects together with other objects at allocation time, and which may
have a parent/child relationship with other allocation frames.
PURPOSE
<a id="purpose.stack-allocation" name="purpose.stack-allocation">.purpose.stack-allocation</a>: The allocation frame protocol is intended to support
efficient memory management for stack allocation, i.e., the allocation of
objects which have dynamic extent.
<a id="purpose.general" name="purpose.general">.purpose.general</a>: The allocation frame protocol is intended to be sufficiently
general that it will be useful in supporting other types of nested allocation
patterns too. For example, it could be used to for EPVM-style save and
restore, ramp allocation patterns or generations.
REQUIREMENTS
Known requirements
<a id="req.stack-alloc" name="req.stack-alloc">.req.stack-alloc</a>: Provide a interface for clients to describe a stack
allocation pattern, as an alternative to using the control stack.
<a id="req.efficient" name="req.efficient">.req.efficient</a>: Permit an implementation which is comparable in efficiency to
allocating on the control stack.
<a id="req.ap" name="req.ap">.req.ap</a>: Support allocation via allocation points (APs).
<a id="req.format" name="req.format">.req.format</a>: Support the allocation of formatted objects.
<a id="req.scan" name="req.scan">.req.scan</a>: Ensure that objects in allocation frames can participate in garbage
collection by being scanned.
<a id="req.fix" name="req.fix">.req.fix</a>: Ensure that objects in allocation frames can participate in garbage
collection by accepting Fix requests.
<a id="req.condemn" name="req.condemn">.req.condemn</a>: Ensure that objects in allocation frames can participate in
garbage collection by being condemned.
<a id="attr.locking" name="attr.locking">.attr.locking</a>: Minimize the synchronization cost for the creation and
destruction of frames.
Proto-requirements
<a id="proto-req" name="proto-req">.proto-req</a>: The following are possible requirements that might be important in
the future. The design does not necessarily meet all these requirements, but it
does consider them all. Each requirement either has direct support in the
framework, or could be supported with future additions to the framework.
<a id="req.parallels" name="req.parallels">.req.parallels</a>: The allocation frame protocol should provide a framework for
exploiting the parallels between stack extents, generations and "ramps".
<a id="req.pool-destroy" name="req.pool-destroy">.req.pool-destroy</a>: It should be possible to use allocation frames to free all
objects in a pool without destroying the pool.
<a id="req.epvm" name="req.epvm">.req.epvm</a>: It should be possible to implement EPVM-style save and restore
operations by creating and destroying allocation frames.
<a id="req.subst" name="req.subst">.req.subst</a>: It should be possible to substitute a stack pool with a GC-ed pool
so that erroneous use of a stack pool can be detected.
<a id="req.format-extensions" name="req.format-extensions">.req.format-extensions</a>: It should be possible for stack pools to utilize the
same format as any other pool, including debugging formats that include
fenceposting, etc.
<a id="req.mis-nest" name="req.mis-nest">.req.mis-nest</a>: Should ensure "mis-nested" stacks are safe.
<a id="req.non-top-level" name="req.non-top-level">.req.non-top-level</a>: Should support allocation in the non-top stack extent.
<a id="req.copy-if-necessary" name="req.copy-if-necessary">.req.copy-if-necessary</a>: Should ensure that stack pools can support
"copy-if-necessary" (so that low-level system code can heapify stack objects.)
<a id="req.preserve" name="req.preserve">.req.preserve</a>: When an object is in an allocation frame which is being
destroyed, it should be possible to preserve that object in the parent frame.
<a id="req.contained" name="req.contained">.req.contained</a>: Should allow clients to ask if an object is "contained" in a
frame. The object is contained in a frame if it is affected when the frame is
ended.
<a id="req.alloc-with-other" name="req.alloc-with-other">.req.alloc-with-other</a>: Should allow clients to allocate an object in the same
frame as another object.
OVERVIEW
<a id="frame-classes" name="frame-classes">.frame-classes</a>: The protocol supports different types of allocation frames,
which are represented as "frame classes". It's up to pools to determine which
classes of allocation frames they support. Pools which support more than one
frame class rely on the client to indicate which class is currently of
interest. The client indicates this by means of an operation which stores the
class in the buffer to which the allocation point is attached.
<a id="frame-handles" name="frame-handles">.frame-handles</a>: Allocation frames are described via abstract "frame handles".
Pools may choose what the representation of a frame handle should be. Frame
handles are static, and the client need not store them in a GC root.
<a id="lightweight-frames" name="lightweight-frames">.lightweight-frames</a>: The design includes an extension to the allocation point
protocol, which permits the creation and destruction of allocation frames
without the necessity for claiming the arena lock. Such frames are called
"lightweight frames".
OPERATIONS
<a id="op.intro" name="op.intro">.op.intro</a>: Each operation has both an external (client) interface and an
internal (MPS) interface. The external function takes an allocation point as a
parameter, determines which buffer and pool it belongs to, and calls the
internal function with the buffer and pool as parameters.
<a id="op.obligatory" name="op.obligatory">.op.obligatory</a>: The following operations are supported on any allocation point
which supports allocation frames:-
<a id="operation.push" name="operation.push">.operation.push</a>: The PushFrame operation creates a new allocation frame of the
currently chosen frame class, makes this new frame the current frame, and
returns a handle for the frame.
<a id="operation.pop" name="operation.pop">.operation.pop</a>: The PopFrame operation takes a frame handle as a parameter.
Some pool classes might insist or assume that this is the handle for the
current frame. It finds the parent of that frame and makes it the current
frame. The operation indicates that all children of the new current frame
contain objects which are likely to be dead. The reclaim policy is up to the
pool; some classes might insist or assume that the objects must be dead, and
eagerly free them. Note that this might introduce the possibility of leaving
dangling pointers elsewhere in the arena. If so, it's up to the pool to decide
what to do about this.
<a id="op.optional" name="op.optional">.op.optional</a>: The following operations are supported for some allocation
frames, but not all. Pools may choose to support some or all of these
operations for certain frame classes. An unsupported operation will return a
failure value:-
<a id="operation.select" name="operation.select">.operation.select</a>: The SelectFrame operation takes a frame handle as a
parameter and makes that frame the current frame. It does not indicate that
any children of the current frame contain objects which are likely to be dead.
<a id="operation.select-addr" name="operation.select-addr">.operation.select-addr</a>: The SelectFrameOfAddr operation takes an address as a
parameter and makes the frame of that address the current frame. It does not
indicate that any children of the current frame contain objects which are
likely to be dead.
<a id="operation.in-frame" name="operation.in-frame">.operation.in-frame</a>: The AddrInFrame operation determines whether the supplied
address is the address of an object allocated in the supplied frame, or any
child of that frame.
<a id="operation.set" name="operation.set">.operation.set</a>: The SetFrameClass operation takes a frame class and an
allocation point as parameters, and makes that the current frame class for the
allocation point. The next PushFrame operation will create a new frame of that
class.
INTERFACE
External types (used by all client of allocation frames)
<a id="type.client.frame-handle" name="type.client.frame-handle">.type.client.frame-handle</a>: Frame handles are defined as an abstract type:
typedef struct mps_frame_s *mps_frame_t;
External types (complete set)
<a id="type.client.frame-class" name="type.client.frame-class">.type.client.frame-class</a>: Frame classes are defined as an abstract type:
typedef struct mps_frame_class_s *mps_frame_class_t;
<a id="type.client.frame-class.access" name="type.client.frame-class.access">.type.client.frame-class.access</a>: Clients access frame classes by means of
dedicated functions for each frame class.
External functions (used by all client of allocation frames)
<a id="fn.client.push" name="fn.client.push">.fn.client.push</a>: The following function is used by clients to invoke the
PushFrame operation. For lightweight frames, this might not invoke the
corresponding internal function:
mps_res_t mps_ap_frame_push(mps_frame_t *frame_o, mps_ap_t buf);
<a id="fn.client.pop" name="fn.client.pop">.fn.client.pop</a>: The following function is used by clients to invoke the
PopFrame operation. For lightweight frames, this might not invoke the
corresponding internal function:
mps_res_t mps_ap_frame_pop(mps_ap_t buf, mps_frame_t frame);
External functions (complete set)
<a id="fn.client.select" name="fn.client.select">.fn.client.select</a>: The following function is used by clients to invoke the
SelectFrame operation:
mps_res_t mps_ap_frame_select(mps_ap_t buf, mps_frame_t frame);
<a id="fn.client.select-addr" name="fn.client.select-addr">.fn.client.select-addr</a>: The following function is used by clients to invoke
the SelectFrameOfAddr operation:
mps_res_t mps_ap_frame_select_from_addr(mps_ap_t buf, mps_addr_t addr);
<a id="fn.client.in-frame" name="fn.client.in-frame">.fn.client.in-frame</a>: The following function is used by clients to invoke the
AddrInFrame operation:
mps_res_t mps_ap_addr_in_frame(mps_bool_t *inframe_o, mps_ap_t buf,
mps_addr_t *addrref, mps_frame_t frame);
<a id="fn.client.set" name="fn.client.set">.fn.client.set</a>: The following function is used by clients to invoke the
SetFrameClass operation:
mps_res_t mps_ap_set_frame_class(mps_ap_t buf, mps_frame_class_t
class);
<a id="fn.client.stack-frame-class" name="fn.client.stack-frame-class">.fn.client.stack-frame-class</a>: The following function is used by clients to
access the frame class used for simple stack allocation:
mps_frame_class_t mps_alloc_frame_class_stack(void);
Internal types (used by all implementations of allocation frames)
<a id="type.frame-handle" name="type.frame-handle">.type.frame-handle</a>: Frame handles are defined as an abstract type:
typedef struct AllocFrameStruct *AllocFrame;
Internal types (complete set)
<a id="type.frame-class" name="type.frame-class">.type.frame-class</a>: Frame classes are defined as an abstract type:
typedef struct AllocFrameClassStruct *AllocFrameClass;
Internal functions (used by all implementations of allocation frames)
<a id="fn.push" name="fn.push">.fn.push</a>: A pool method of the following type is called (if needed) to invoke
the PushFrame operation:
typedef Res (*PoolFramePushMethod)(AllocFrame *frameReturn,
Pool pool, Buffer buf);
<a id="fn.pop" name="fn.pop">.fn.pop</a>: A pool method of the following type is called (if needed) to invoke
the PopFrame operation:
typedef Res (*PoolFramePopMethod)(Pool pool, Buffer buf,
AllocFrame frame);
Internal functions (complete set)
<a id="fn.select" name="fn.select">.fn.select</a>: A pool method of the following type is called to invoke the
SelectFrame operation:
typedef Res (*PoolFrameSelectMethod)(Pool pool, Buffer buf,
AllocFrame frame);
<a id="fn.select-addr" name="fn.select-addr">.fn.select-addr</a>: A pool method of the following type is called to invoke the
SelectFrameOfAddr operation:
typedef Res (*PoolFrameSelectFromAddrMethod)(Pool pool, Buffer buf,
Addr addr);
<a id="fn.in-frame" name="fn.in-frame">.fn.in-frame</a>: A pool method of the following type is called to invoke the
AddrInFrame operation:
typedef Res (*PoolAddrInFrameMethod)(Bool *inframeReturn,
Pool pool, Seg seg, Addr *addrref,
AllocFrame frame);
<a id="fn.set" name="fn.set">.fn.set</a>: A pool method of the following type is called to invoke the
SetFrameClass operation:
typedef Res (*PoolSetFrameClassMethod)(Pool pool, Buffer buf,
AllocFrameClass class);
LIGHTWEIGHT FRAMES
Overview
<a id="lw-frame.overview" name="lw-frame.overview">.lw-frame.overview</a>: Allocation points provide direct support for lightweight
frames, and are designed to permit PushFrame and PopFrame operations without
the need for locking and delegation to the pool method. Pools can disable this
mechanism for any allocation point, so that the pool method is always called.
The pool method will be called whenever synchronization is required for other
reasons (e.g. the buffer is tripped).
<a id="lw-frame.model" name="lw-frame.model">.lw-frame.model</a>: Lightweight frames offer direct support for a particular model
of allocation frame use, whereby the PushFrame operation returns the current
allocation pointer as a frame handle, and the PopFrame operation causes the
allocation pointer to be reset to the address of the frame handle. This model
should be suitable for simple stack frames, where more advanced operations like
SelectFrame are not supported. It may also be suitable for more advanced
allocation frame models when they are being used simply. The use of a complex
operation always involves synchronization via locking, and the pool may disable
lightweight synchronization temporarily at this time.
State
<a id="lw-frame.states" name="lw-frame.states">.lw-frame.states</a>: Allocation points supporting lightweight frames will be in
one of the following states:-
Valid: Indicates that PushFrame can be a lightweight operation and need
not be synchronized
PopPending: Indicates that there has been a PopFrame operation that the pool
must respond to
Disabled: Indicates that the pool has disabled support for lightweight
operations for this AP
These states are in addition to the state normally held by an AP for allocation
purposes. An AP will be in the Disabled state at creation.
<a id="lw-frame.transitions" name="lw-frame.transitions">.lw-frame.transitions</a>: State transitions happen under the following
circumstances:-
Valid -&gt; PopPending : As a result of a client PopFrame operation
Valid -&gt; Disabled : At the choice of the pool (e.g. when responding to a
SelectFrame operation)
PopPending -&gt; Valid : At the choice of the pool, when processing a PopFrame
PopPending -&gt; Disabled : At the choice of the pool, when processing a PopFrame
Disabled -&gt; Valid : At the choice of the pool
Disabled -&gt; Popframe : Illegal
<a id="lw-frame.state-impl" name="lw-frame.state-impl">.lw-frame.state-impl</a>: Each AP contains 3 additional fields to hold this state:
mps_addr_t frameptr;
mps_bool_t enabled;
mps_bool_t lwPopPending;
<a id="lw-frame.enabled" name="lw-frame.enabled">.lw-frame.enabled</a>: The enabled slot holds the following values for each state:
Valid: TRUE
PopPending: TRUE
Disabled: FALSE
<a id="lw-frame.frameptr" name="lw-frame.frameptr">.lw-frame.frameptr</a>: The frameptr slot holds the following values for each state:
Valid: NULL
PopPending: Frame handle for most recently popped frame.
Disabled: NULL
.lw-frame.lwPopPending: The lwPopPending slot holds the following values for
each state:
Valid: FALSE
PopPending: TRUE
Disabled: FALSE
<a id="lw-frame.state-for-gc" name="lw-frame.state-for-gc">.lw-frame.state-for-gc</a>: It is not necessary for the tracer, format code, pool,
or any other part of the GC support in MPS to read either of the 2 additional
AP fields in order to scan a segment which supports a lightweight allocation
frame.
Synchronization
<a id="lw-frame.sync" name="lw-frame.sync">.lw-frame.sync</a>: The purpose of the design is that mutator may access the state
of an AP without locking with MPS (via the external functions). The design
assumes the normal MPS restriction that an operation on an AP may only be
performed by a single mutator thread at a time. Each of the operations on
allocation frames counts as an operation on an AP.
<a id="lw-frame.sync.pool" name="lw-frame.sync.pool">.lw-frame.sync.pool</a>: Pools are permitted to read or modify the lightweight
frame state of an AP only in response to an operation on that AP.
<a id="lw-frame.sync.external" name="lw-frame.sync.external">.lw-frame.sync.external</a>: The external functions mps_ap_frame_push and
mps_ap_frame_pop are permitted to read the values of the enabled and frameptr
fields for the supplied AP without claiming the arena lock. They are permitted
to modify the frameptr field iff enabled == FALSE.
<a id="lw-frame.sync.trip" name="lw-frame.sync.trip">.lw-frame.sync.trip</a>: When a buffer trip happens, and the trap wasn't set by MPS
itself (i.e. it wasn't because of a flip or for logging), then the buffer code
must check whether the AP has state PopPending. If it does, the buffer code
must call the Pool.
Implementation
<a id="lw-frame.push" name="lw-frame.push">.lw-frame.push</a>: The external PushFrame operation (mps_ap_frame_push) performs
the following operations:
IF (!APIsTrapped(ap) &amp;&amp; StateOfFrame(ap) == Valid &amp;&amp; ap-&gt;init == ap-&gt;alloc)
*frame_o = ap-&gt;init;
ELSE
WITH_ARENA_LOCK
PerformInternalPushFrameOperation(...)
END
END
<a id="lw-frame.pop" name="lw-frame.pop">.lw-frame.pop</a>: The external PopFrame operation (mps_ap_frame_pop) performs the
following operations:
IF (StateOfFrame(ap) != Disabled)
TrapAP(ap); /* ensure next allocation or push involves the pool */
ap-&gt;frameptr = frame;
ap-&gt;lwpopPending = TRUE;
ELSE
WITH_ARENA_LOCK
PerformInternalPopFrameOperation(...)
END
END
</pre>
<h2><a id="section-A" name="section-A">A. References</a></h2>
<!-- Template Entry
<table>
<tr valign="top">
<td>[<a id="ref-#REF#" name="ref-#REF#" href="#REF_URL#">#REF_NAME#</a>]</td>
<td>
"#REF_TITLE#";
#REF_AUTHOR#;
&lt;URL: <a href="#REF_URL#">#REF_URL#</a>&gt;;
#REF_DATE#.
</td>
</tr>
</table>
-->
<h2><a id="section-B" name="section-B">B. Document History</a></h2>
<table>
<tr valign="top">
<td>2002-06-07</td>
<td><a href="mailto:rb@ravenbrook.com">RB</a></td>
<td>Converted from MMInfo database design document.</td>
</tr>
</table>
<h2><a id="section-C" name="section-C">C. Copyright and License</a></h2>
<p> This document is copyright &copy; 1997-2002 <a href="http://www.ravenbrook.com/">Ravenbrook Limited</a>. All rights reserved. This is an open source license. Contact Ravenbrook for commercial licensing options. </p>
<p> Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: </p>
<ol>
<li> Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. </li>
<li> Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. </li>
<li> Redistributions in any form must be accompanied by information on how to obtain complete source code for the this software and any accompanying software that uses this software. The source code must either be included in the distribution or be available for no more than the cost of distribution plus a nominal fee, and must be freely redistributable under reasonable conditions. For an executable file, complete source code means the source code for all modules it contains. It does not include source code for modules or files that typically accompany the major components of the operating system on which the executable file runs. </li>
</ol>
<p> <strong> This software is provided by the copyright holders and contributors "as is" and any express or implied warranties, including, but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement, are disclaimed. In no event shall the copyright holders and contributors be liable for any direct, indirect, incidental, special, exemplary, or consequential damages (including, but not limited to, procurement of substitute goods or services; loss of use, data, or profits; or business interruption) however caused and on any theory of liability, whether in contract, strict liability, or tort (including negligence or otherwise) arising in any way out of the use of this software, even if advised of the possibility of such damage. </strong> </p>
<hr />
<div align="center">
<p><code>$Id$</code></p>
<p>
<a href="/">Ravenbrook</a> /
<a href="/project/">Projects</a> /
<a href="/project/mps/">Memory Pool System</a> /
<a href="/project/mps/master/">Master Product Sources</a> /
<a href="/project/mps/master/design/">Design Documents</a>
</p>
</div>
</body>
</html>

355
mps/design/diag.txt Normal file
View file

@ -0,0 +1,355 @@
.. mode: -*- rst -*-
Diagnostic feedback
===================
:Tag: design.mps.diag
:Author: Richard Kistruck
:Date: 2007-06-28
:Status: incomplete design
:Revision: $Id$
:Copyright: See `Copyright and License`_.
Introduction
------------
_`.intro`: This document describes how to use the diagnostic feedback
mechanism in the Memory Pool System.
_`.sources`: Initially abased on `[RHSK 2007-04-13]`_ and `[RHSK
2007-04-18]`_.
Overview
--------
Diagnostic feedback is information created by the MPS diagnostic
system for the purpose of helping MPS programmers client-code
programmers.
Such a piece of information is called "a diagnostic". (See also
`.parts`_.)
A diagnostic is not intended to be end-user readable (or visible), or
machine-parseable.
A diagnostic is not intended to be stable from one release to the
next: it may be modified or removed at any time.
Requirements
------------
MPS diagnostic feedback code must do these things:
- calculate, store, and propagate data;
- collate, synthesise, and format it into a human-useful diagnostic;
- control (for example, filter) output of diagnostics;
- use a channel to get the diagnostic out.
Note: the knowledge/code/logic for constructing the human-useful
message is kept inside normal MPS source code. This means it is always
in-sync with changes to the MPS. This also means that any external
utilities used to display the messages do not need to understand, or
keep in sync with, the details of what's going inside the MPS.
Usage
-----
To run the MPS and get diagnostic output from it:
1. Use a variety with diagnostics compiled-in. Currently, that means
variety.di. See ``config.h``.
2. Check that the diagnostics you require are generated, by looking in
MPS source for invocations of the appropriate macro (for example,
``DIAG_SINGLEF()``).
3. Check that the diagnostics you require will be output, by looking
at the diagnostic filter rules in ``diag.c``.
4. Run the MPS and client in an environment that supports the channel
used (for example, at a command-line if using ``WriteF()``).
What is a diagnostic?
.....................
A diagnostic has three parts:
1. a trigger condition, that causes this diagnostic to be emitted;
2. a text tag (for example, "TraceStart") which is the name of this
diagnostic; and
3. a paragraph of human-useful text.
A diagnostic is emitted by the MPS at a certain point in time when a
certain event happens.
Diagnostics are not nested. Every diagnostic must have a tag. Each
diagnostic should have a unique tag (uniqueness is just to help the
humans; the diagnostic system does not care).
The paragraph of text can be many lines long. It usually explains what
event caused the diagnostic to be emitted, and commonly also includes
the output of some ``Describe()`` methods for various relevant
objects. (For example, the ``TraceStart`` diagnostic might call, and
include the output generated by, the ``TraceDescribe()`` method).
How do I control (filter) which diagnostics I see?
..................................................
All diagnostics are emitted and then filtered according to the
"diagnostic filter rules".
The first level of control is filtering by tag. (For example, only
show ``TraceStart`` diagnostics).
The second level of control is filtering by paragraph content. (For
example, only show ``TraceStart`` diagnostics where the trace is
started because a nursery generation is full).
The third level of control is filtering by line content. (For example,
only show lines containing the word ``whiteSet``).
See ``diag.c`` for details.
Note: the entire filtering mechanism can be turned off, so that
diagnostics go immediately to ``mps_lib_get_stdout(0``, with no
buffering or filtering See impl.c.diag.filter-disable.
How to write a diagnostic
-------------------------
Improve stateless Describe methods where possible
.................................................
Where possible, don't put clever code into an event-triggered
diagnostic: put it into a stateless ``Describe()`` method instead, and
then call that method when emitting your diagnostic.
For example::
FooDescribe(Foo foo, mps_lib_FILE *stream)
{
/* show value of new "quux" field */
WriteF(stream, "Foo: $P { quux: $U }\n", foo, foo->quux);
}
FooWibble(Foo foo)
{
...
DIAG_FIRSTF(( "FooWibble", "Wibbling foo $P", foo, NULL));
DIAG( FooDescribe(foo, DIAG_STREAM); );
DIAG_END("FooWibble");
...
}
This is much better, because other people can use your human-useful
output in their diagnostics, or 'live' in a debugger.
Use the output macros
.....................
For a simple diagnostic, use ``DIAG_SINGLEF()``. This begins the tag,
puts text into the paragraph, and ends the tag immediately.
For a more complex diagnostic, the first call must be
``DIAG_FIRSTF()``, which begins a diag tag.
While a tag is current, you can add text to the diagnostic's paragraph
using ``DIAG_MOREF()``, and ``WriteF( DIAG_STREAM, ... )``.
.. note::
``DIAG_STREAM`` is not a real standard C library stream. If you
want stream-level access, you may use ``Stream_fputc()`` and
``Stream_fputs()``.
End the tag by calling ``DIAG_END``.
Compile away in non-diag varieties; no side effects
...................................................
Wrap non-output code with the ``DIAG()`` and ``DIAG_DECL()`` macros,
to make sure that non-diag varieties do not execute
diagnostic-generating code.
For complex diagnostic-generating code, it may be cleaner to move it
into a separate local function. Put ``_diag`` on the end of the function
name (for example, ``TraceStart_diag()``).
Obviously, diagnostic-generating code must have no side effects.
Choosing tags
.............
Tags should be valid C identifiers. Unless you know of a good reason
why not. (Not currently checked).
There's no formal scheme for tag naming, but make it helpful and
informally hierarchical, for example, ``TraceBegin``, ``TraceStart``,
``TraceEnd``, and so on, not ``BeginTrace``, ``EndTrace``.
Writing good paragraph text
...........................
IMPORTANT: Make your diagnostics easy to understand! Other people will
read your diagnostics! Make them clear and helpful. Do not make them
terse and cryptic. If you use symbols, print a key in the diagnostic.
(If you don't want to see this the screen clutter, then you can always
add a filter rule to your personal rule set to filter it out).
Maintaining helpful filter rules
................................
If you add a noisy diagnostic, add a rule to the default ruleset to
turn it off.
How the MPS diagnostic system works
-----------------------------------
Channels
........
The recommended channel is ``WriteF()`` to standard output.
Other possible of future channels might be:
- ``printf()``;
- a new type (yet to be defined) of ``mps_message``;
- squirt them into the telemetry-log-events system;
- telnet.
Currently, only ``printf()`` and ``WriteF()`` are supported. See the
``DIAG_WITH_`` macros in ``mpm.h``.
You can also use a debugger to call ``Describe()`` methods directly,
from within the debugger.
Note: it is unfortunate that choice of channel may (for some channels)
also dictate the form of the code that synthesises the message. (For
example, ``WriteF()`` style parameter-expansion is not possible when
using the ``printf()`` channel, because there is no way to get
``WriteF()`` to produce its output into a string). This is just a
technical hitch; logically, the code that synthesises a diagnostic
message should not care which channel will be used to transmit it out
of the MPS.
Parts of the MPS diagnostic system
..................................
_`.parts`: The following facilities are considered part of the MPS
diagnostic system:
- the ``Describe()`` methods.
- the ``DIAG`` macros (``DIAG``, ``DIAG_DECL``, ``DIAG_*F``, and so on);
- the ``STATISTIC`` macros (see ``mpm.h``);
- the ``METER`` macros and meter subsystem.
Related systems
...............
The MPS diagnostic system is separate from the following other MPS
systems:
- The telemetry-log-events system. This emits much more data, in a
less human-readable form, requires MPS-aware external tools, and is
more stable from release to release). In non-diagnostic telemetry
varieties, the telemetry-log-events system emits events that log all
normal MPS actions. In diagnostic telemetry varieties, it may emit
additional events containing diagnostic information. Additionally,
the telemetry-log-events stream might in future be available as a
channel for emitting human-readable text diagnostics. See also
design.mps.telemetry.
- The MPS message system. This is present in all varieties, and
manages asynchronous communication from the MPS to the client
program). However, the MPS message system might in future also be
available as a channel for emitting diagnostics. See also
design.mps.message.
References
----------
.. _[RHSK 2007-04-13]:
Richard Kistruck. 2007-04-13. "`diagnostic feedback from the MPS <https://info.ravenbrook.com/mail/2007/04/13/13-07-45/0.txt>`_".
.. _[RHSK 2007-04-18]:
Richard Kistruck. 2007-04-18. "`Diverse types of diagnostic feedback <http://info.ravenbrook.com/mail/2007/04/18/10-58-49/0.txt>`_".
Document History
----------------
- 2007-06-28 Richard Kistruck Create. Telemetry-log-events system is a possible channel.
- 2007-06-29 Richard Kistruck Feedback (not output), each "a
diagnostic". Parts of the system; related systems. Link to initial
design email.
- 2007-08-14 Richard Kistruck (Diag Filtering). Expand section: How to
see some MPS diagnostic output, with what a diagnostic is, and how
to filter it. New section: How to write a diagnostic. Various minor
updates and corrections.
- 2013-05-23 GDR_ Converted to reStructuredText.
.. _GDR: http://www.ravenbrook.com/consultants/gdr/
Copyright and License
---------------------
Copyright © 2013 Ravenbrook Limited. All rights reserved.
<http://www.ravenbrook.com/>. This is an open source license. Contact
Ravenbrook for commercial licensing options.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. Redistributions in any form must be accompanied by information on how
to obtain complete source code for this software and any
accompanying software that uses this software. The source code must
either be included in the distribution or be available for no more than
the cost of distribution plus a nominal fee, and must be freely
redistributable under reasonable conditions. For an executable file,
complete source code means the source code for all modules it contains.
It does not include source code for modules or files that typically
accompany the major components of the operating system on which the
executable file runs.
**This software is provided by the copyright holders and contributors
"as is" and any express or implied warranties, including, but not
limited to, the implied warranties of merchantability, fitness for a
particular purpose, or non-infringement, are disclaimed. In no event
shall the copyright holders and contributors be liable for any direct,
indirect, incidental, special, exemplary, or consequential damages
(including, but not limited to, procurement of substitute goods or
services; loss of use, data, or profits; or business interruption)
however caused and on any theory of liability, whether in contract,
strict liability, or tort (including negligence or otherwise) arising in
any way out of the use of this software, even if advised of the
possibility of such damage.**

482
mps/design/diag/index.html
View file

@ -1,482 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<title>Diagnostic feedback</title>
</head>
<body bgcolor="#FFFFFF" text="#000000" link="#000099" vlink="#660066" alink="#FF0000">
<div align="center">
<p>
<a href="/">Ravenbrook</a> /
<a href="/project/">Projects</a> /
<a href="/project/mps/">Memory Pool System</a> /
<a href="/project/mps/master/">Master Product Sources</a> /
<a href="/project/mps/master/design/">Design Documents</a>
</p>
<p><i><a href="/project/mps/">Memory Pool System Project</a></i></p>
<hr />
<h1>Diagnostic feedback</h1>
</div>
<p>
This document contains a <a href="#guide">guide</a> to MPS
diagnostic feedback, followed by the historical
<a href="#initial-design">initial design</a>.
References, History, Copyright and License are
<a href="#section-A">at the end</a>.
</p>
<p> Readership: any MPS developer. Not confidential. </p>
<hr />
<h1> <a id="guide">Guide</a> </h1>
<h2> Introduction </h2>
<p>
Diagnostic feedback is information created by the MPS diagnostic
system for the purpose of helping <strong>MPS programmers</strong>
and <strong>client-code programmers</strong>.
</p>
<p>
Such a piece of information is called "a diagnostic".
(See also <a href="#parts-of-diagnostic-system">Parts of the
MPS diagnostic system</a>, below).
</p>
<p>
A diagnostic is not intended to be end-user readable (or visible),
or machine-parseable.
A diagnostic is not intended to be stable from one release to the
next: it may be modified or removed at any time.
</p>
<p>
MPS diagnostic feedback code must do these things:
</p>
<ol>
<li> calculate, store, and propagate data; </li>
<li> collate, <strong>synthesise</strong>, and format it into a
human-useful diagnostic; </li>
<li> <strong>control</strong> (eg. filter) output of diagnostics;
</li>
<li> use a <strong>channel</strong> to get the diagnostic out. </li>
</ol>
<p>
Note: the knowledge/code/logic for constructing the
human-useful message is kept <em>inside normal MPS
source code</em>. This means it is always in-sync with changes
to the MPS. This also means that any external utilities used
to display the messages do not need to understand, or keep in
sync with, the details of what's going inside the MPS.
</p>
<h2> How to see some MPS diagnostic output </h2>
<p>
To run the MPS and get diagnostic output from it:
</p>
<ul>
<li> Use a variety with diagnostics
compiled-in. Currently, that means
<strong>variety.di</strong>. See config.h. </li>
<li> Check that the diagnostics you require are generated,
by looking in MPS source for invocations of the appropriate
macro (eg. DIAG_SINGLEF()). </li>
<li> Check that the diagnostics you require will be output,
by looking at the diagnostic filter rules in diag.c. </li>
<li> Run the MPS and client in an environment that supports the
channel used (eg. at a command-line if using WriteF). </li>
</ul>
<h3> What is a diagnostic? </h3>
<p> A diagnostic has three parts:</p>
<ul>
<li> a trigger condition, that causes this diagnostic to be
emitted; </li>
<li> a text tag (eg "TraceStart") which is the name of this
diagnostic; and </li>
<li> a paragraph of human-useful text.</li>
</ul>
<p> A diagnostic is emitted by the MPS at a certain
point in time when a certain event happens. </p>
<p> Diagnostics are not nested. Every diagnostic must have a tag.
Each diagnostic should have a unique tag (uniqueness is just
to help the humans; the diagnostic system does not care). </p>
<p> The paragraph of text can be many lines long. It usually
explains what event caused the diagnostic to be emitted, and
commonly also includes the output of some &lt;object&gt;Describe()
methods for various relevant objects. (For example, the TraceStart
diagnostic might call, and include the output generated by, the
TraceDescribe() method). </p>
<h3> How do I control (filter) which diagnostics I see? </h3>
<p> All diagnostics are emitted and then filtered according
to the "diagnostic filter rules".</p>
<p> The first level of control is filtering by tag. (For example,
only show TraceStart diagnostics). </p>
<p> The second level of control is filtering by paragraph content.
(For example, only show TraceStart diagnostics where the trace is
started because a nursery generation is full).</p>
<p> The third level of control is filtering by line content.
(For example, only show lines containing the word "whiteSet").</p>
<p> See diag.c for details.</p>
<p>Note: the entire filtering mechanism can be turned
off, so that diagnostics go immediately to mps_lib_stdout,
with no buffering or filtering See diag.c#.filter-disable.</p>
<h2> How to write a diagnostic </h2>
<h3> Improve stateless Describe methods where possible </h3>
<p> Where possible, don't put clever code into an event-triggered
diagnostic: put it into a stateless &lt;object&gt;Describe()
method instead, and then call that method when emitting your diagnostic.</p>
<p> For example: </p>
<blockquote><pre><code>FooDescribe(Foo foo, mps_lib_FILE *stream)
{
/* show value of new "quux" field */
WriteF(stream, "Foo: $P { quux: $U }\n", foo, foo->quux);
}
FooWibble(Foo foo)
{
...
DIAG_FIRSTF(( "FooWibble", "Wibbling foo $P", foo, NULL));
DIAG( FooDescribe(foo, DIAG_STREAM); );
DIAG_END("FooWibble");
...
}
</code></pre></blockquote>
<p> This is much better, because other people can use your
human-useful output in their diagnostics, or 'live' in a
debugger. </p>
<h3> How to use DIAG_*F output macros </h3>
<p> For a simple diagnostic, use DIAG_SINGLEF. This begins the tag,
puts text into the paragraph, and ends the tag immediately.</p>
<p> For a more complex diagnostic, the first call must be DIAG_FIRSTF,
which begins a diag tag. </p>
<p> While a tag is current, you can add text to the diagnostic's
paragraph using DIAG_MOREF, and WriteF( DIAG_STREAM, ... ).</p>
<p> (Note: DIAG_STREAM is not a real standard C library
stream. If you want stream-level access, you may use Stream_fputc()
and Stream_fputs().) </p>
<p> End the tag by calling DIAG_END.</p>
<h3> Compile away in non-diag varieties; no side effects </h3>
<p> Wrap non-output code with the DIAG() and DIAG_DECL() macros, to
make sure that non-diag varieties do not execute
diagnostic-generating code.</p>
<p> For complex diagnostic-generating code, it may be cleaner to move
it into a separate local function. Put "_diag" on the end of the
function name (eg. "TraceStart_diag()").</p>
<p> Obviously, diagnostic-generating code must have no side effects.
</p>
<h3> Choosing tags </h3>
<p> Tags should be valid C identifiers. Unless you know of a good
reason why not. (Not currently checked).</p>
<p> There's no formal scheme for tag naming, but make it helpful and
&mdash; informally &mdash; hierarchical, eg. TraceBegin,
TraceStart, TraceEnd, etc. (not BeginTrace, EndTrace, ...).</p>
<h3> Writing good paragraph text </h3>
<p> IMPORTANT: Make your diagnostics easy to understand!
<strong>Other people</strong> will read your diagnostics!
Make them clear and helpful.
Do not make them terse and cryptic.
If you use symbols, print a key in the diagnostic. (If you don't
want to see this the screen clutter, then you can always add a
filter rule to your personal rule set to filter it out).</p>
<h3> Maintaining helpful filter rules </h3>
<p> If you add a 'noisy' diagnostic, add a rule to the default
ruleset to turn it off.</p>
<h2> How the MPS diagnostic system works </h2>
<h3> Channels </h3>
<p>The recommended channel is WriteF to stdout.</p>
<p>
Other possible of future channels might be:
</p>
<ul>
<li> printf; </li>
<li> a new type (yet to be defined) of mps_message; </li>
<li> squirt them into the telemetry-log-events system; </li>
<li> telnet. </li>
</ul>
<p>
Currently, only printf and WriteF are supported.
See the DIAG_WITH_ macros in mpm.h.
</p>
<p>
You can also use a debugger to call &lt;type&gt;Describe() methods
directly, from within the debugger.
</p>
<p>
Note: it is unfortunate that choice of channel may (for some
channels) also dictate the form of the code that synthesises the
message. (For example, WriteF-style parameter-expansion is not
possible when using the printf channel, because there is no way
to get WriteF to produce its output into a string). This is just
a technical hitch; logically, the code that synthesises a
diagnostic message should not care which channel will be used to
transmit it out of the MPS.
</p>
<h3 id="parts-of-diagnostic-system"> Parts of the MPS diagnostic
system </h3>
<p>
The following facilities are considered part of the MPS diagnostic
system:
</p>
<ul>
<li> the &lt;type&gt;Describe() methods. </li>
<li> the DIAG* macros (DIAG, DIAG_DECL, DIAG_*F, etc);
</li>
<li> the STATISTIC macros (see mpm.h); </li>
<li> the METER macros and meter subsystem; </li>
</ul>
<h3 id="related-systems"> Related Systems </h3>
<p>
The MPS diagnostic system is separate from the following other MPS
systems:
</p>
<ul>
<li> The <strong>telemetry-log-events system</strong>
(which emits much more data,
in a less human-readable form, requires MPS-aware external
tools, and is more stable from release to release).
In non-diagnostic telemetry varieties, the telemetry-log-events
system
emits events that log all normal MPS actions. In diagnostic
telemetry varieties, it may emit additional events containing
diagnostic information.
Additionally, the telemetry-log-events stream
might in future be available as a channel for emitting
human-readable text diagnostics.
See also <a href="../design/telemetry/">design/telemetry</a>.
</li>
<li> The <strong>MPS message system</strong>
(which is present in all varieties, and
manages asynchronous communication from the MPS to the client
program). However, the MPS message system might in future also
be available as a channel for emitting diagnostics.
See also <a href="../design/message/">design/message</a>.
</li>
</ul>
<hr />
<h1> <a id="initial-design">Initial Design</a> </h1>
<p>
See <a href="http://info.ravenbrook.com/mail/2007/04/13/13-07-45/0.txt">http://info.ravenbrook.com/mail/2007/04/13/13-07-45/0.txt</a>:
"diagnostic feedback from the MPS" by RHSK.
</p>
<p>
Diverse types of diagnostic feedback:
<a href="http://info.ravenbrook.com/mail/2007/04/18/10-58-49/0.txt">http://info.ravenbrook.com/mail/2007/04/18/10-58-49/0.txt</a>.
</p>
<p>
RHSK 2007-06-28
</p>
<p>
It must be possible to add, modify, or remove diagnostics to affect
diagnostic varieties, without a substantial secondary effect
on non-diagnostic varieties. It is not a requirement that there
be zero effect.
</p>
<p>
This means:
</p>
<ul>
<li> for actual output, you must use appropriate DIAG macros
(which compile away to nothing in non-diagnostic varieties);
</li>
<li> for propagating and synthesising data, use appropriate
DIAG macros where it is reasonably clean to do so, such as for
local variables and statements used only for diagnostic output;
</li>
<li> where it is clearer, and without substantial secondary effect,
to include code (such as additional function parameters) in
all varieties, that's okay. </li>
</ul>
<p>Note: the MPS diagnostic feedback system was initially developed in
mps/branch/2007-04-18/diag, mps/branch/2007-07-19/gcdiag, and
mps/branch/2007-08-07/diagtag.</p>
<hr />
<h2><a id="section-A" name="section-A">A. References</a></h2>
<!-- Template Entry
<table>
<tr valign="top">
<td>[<a id="ref-#REF#" name="ref-#REF#" href="#REF_URL#">#REF_NAME#</a>]</td>
<td>
"#REF_TITLE#";
#REF_AUTHOR#;
&lt;URL: <a href="#REF_URL#">#REF_URL#</a>&gt;;
#REF_DATE#.
</td>
</tr>
</table>
-->
<h2><a id="section-B" name="section-B">B. Document History</a></h2>
<table>
<tr valign="top">
<td>2007-06-28</td>
<td><a href="mailto:rhsk@ravenbrook.com">RHSK</a></td>
<td>Create.</td>
</tr>
<tr valign="top">
<td>2007-06-28</td>
<td><a href="mailto:rhsk@ravenbrook.com">RHSK</a></td>
<td>Telemetry-log-events system is a possible channel.</td>
</tr>
<tr valign="top">
<td>2007-06-29</td>
<td><a href="mailto:rhsk@ravenbrook.com">RHSK</a></td>
<td>Feedback (not output), each "a diagnostic".
Parts of the system; related systems. Link to initial design
email.
</td>
</tr>
<tr valign="top">
<td>2007-08-14</td>
<td><a href="mailto:rhsk@ravenbrook.com">RHSK</a></td>
<td>(Diag Filtering). Expand section: How to see some MPS
diagnostic output, with what a diagnostic is, and how to
filter it. New section: How to write a diagnostic.
Various minor updates and corrections.
</td>
</tr>
</table>
<h2><a id="section-C" name="section-C">C. Copyright and License</a></h2>
<p> This document is copyright &copy; 2007 <a href="http://www.ravenbrook.com/">Ravenbrook Limited</a>. All rights reserved. This is an open source license. Contact Ravenbrook for commercial licensing options. </p>
<p> Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: </p>
<ol>
<li> Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. </li>
<li> Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. </li>
<li> Redistributions in any form must be accompanied by information on how to obtain complete source code for the this software and any accompanying software that uses this software. The source code must either be included in the distribution or be available for no more than the cost of distribution plus a nominal fee, and must be freely redistributable under reasonable conditions. For an executable file, complete source code means the source code for all modules it contains. It does not include source code for modules or files that typically accompany the major components of the operating system on which the executable file runs. </li>
</ol>
<p> <strong> This software is provided by the copyright holders and contributors "as is" and any express or implied warranties, including, but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement, are disclaimed. In no event shall the copyright holders and contributors be liable for any direct, indirect, incidental, special, exemplary, or consequential damages (including, but not limited to, procurement of substitute goods or services; loss of use, data, or profits; or business interruption) however caused and on any theory of liability, whether in contract, strict liability, or tort (including negligence or otherwise) arising in any way out of the use of this software, even if advised of the possibility of such damage. </strong> </p>
<hr />
<div align="center">
<p><code>$Id$</code></p>
<p>
<a href="/">Ravenbrook</a> /
<a href="/project/">Projects</a> /
<a href="/project/mps/">Memory Pool System</a> /
<a href="/project/mps/master/">Master Product Sources</a> /
<a href="/project/mps/master/design/">Design Documents</a>
</p>
</div>
</body>
</html>

4
mps/design/guide.impl.c.format.txt
View file

@ -1,4 +1,6 @@
C Style -- Formatting
.. mode: -*- rst -*-
C Style -- formatting
=====================
:Tag: guide.impl.c.format

14
mps/design/index.html
View file

@ -57,7 +57,7 @@
<tr valign="top">
<td> <code><a href="alloc-frame/">alloc-frame/</a></code> </td>
<td> <code><a href="alloc-frame">alloc-frame</a></code> </td>
<td> Design of the MPS allocation frame protocol </td>
@ -145,7 +145,7 @@
<tr valign="top">
<td> <code><a href="diag/">diag/</a></code> </td>
<td> <code><a href="diag">diag</a></code> </td>
<td> The design of MPS diagnostic feedback </td>
@ -185,7 +185,7 @@
<tr valign="top">
<td> <code><a href="interface-c/">interface-c/</a></code> </td>
<td> <code><a href="interface-c">interface-c</a></code> </td>
<td> The design of the Memory Pool System interface to C </td>
@ -193,9 +193,9 @@
<tr valign="top">
<td> <code><a href="io/">io/</a></code> </td>
<td> <code><a href="io">io</a></code> </td>
<td> The design of the MPS i/o subsystem </td>
<td> The design of the MPS I/O subsystem </td>
</tr>
@ -225,7 +225,7 @@
<tr valign="top">
<td> <code><a href="locus/">locus/</a></code> </td>
<td> <code><a href="locus">locus</a></code> </td>
<td> The design for the locus manager </td>
@ -523,7 +523,7 @@
<tr valign="top">
<td> <code><a href="vman/">vman/</a></code> </td>
<td> <code><a href="vman">vman</a></code> </td>
<td> ANSI fake VM </td>

389
mps/design/interface-c.txt Normal file
View file

@ -0,0 +1,389 @@
.. mode: -*- rst -*-
C interface design
==================
:Tag: design.mps.interface.c
:Author: Richard Brooksby
:Date: 1996-07-29
:Status: incomplete document
:Revision: $Id$
:Copyright: See `Copyright and License`_.
Introduction
------------
_`.scope`: This document is the design for the Memory Pool System
(MPS) interface to the C Language, impl.h.mps.
_`.bg`: See mail.richard.1996-07-24.10-57.
Analysis
--------
Goals
.....
_`.goal.c`: The file impl.h.mps is the C external interface to the
MPS. It is the default interface between client code written in C and
the MPS. _`.goal.cpp`: impl.h.mps is not specifically designed to be
an interface to C++, but should be usable from C++.
Requirements
............
_`.req`: The interface must provide an interface from client code
written in C to the functionality of the MPS required by the product
(see req.product), Dylan (req.dylan), and the Core RIP (req.epcore).
``mps.h`` may not include internal MPS header files (such as
``pool.h``).
It is essential that the interface cope well with change, in order to
avoid restricting possible future MPS developments. This means that
the interface must be "open ended" in its definitions. This accounts
for some of the apparently tortuous methods of doing things
(``mps_fmt_A_t``, for example). The requirement is that the MPS should
be able to add new functionality, or alter the implementation of
existing functionality, without affecting existing client code. A
stronger requirement is that the MPS should be able to change without
*recompiling* client code. This is not always possible.
.. note::
`.naming.global` was presumably done in response to unwritten
requirements regarding the use of the name spaces in C, perhaps
these:
- _`.req.name.iso`: The interface shall not conflict in terms of
naming with any interfaces specified by ISO C and all reasonable
future versions.
- _`.req.name.general`: The interface shall use a documented and
reasonably small portion of the namespace so that clients can
interoperate easily.
David Jones, 1998-10-01.
Architecture
------------
_`.fig.arch`: The architecture of the MPS Interface
[missing figure]
Just behind ``mps.h`` is the file ``mpsi.c``, the "MPS interface
layer" which does the job of converting types and checking parameters
before calling through to the MPS proper, using internal MPS methods.
General conventions
-------------------
_`.naming`: The external interface names should adhere to the
documented interface conventions; these are found in
doc.mps.ref-man.if-conv(0).naming. They are paraphrased/recreated
here.
_`.naming.unixy`: The external interface does not follow the same
naming conventions as the internal code. The interface is designed to
resemble a more conventional C, Unix, or Posix naming convention.
_`.naming.case`: Identifiers are in lower case, except
non-function-like macros, which are in upper case.
_`.naming.global`: All publicised identifiers are
prefixed ``mps_`` or ``MPS_``.
_`.naming.all`: All identifiers defined by the MPS
should begin ``mps_`` or ``MPS_`` or ``_mps_``.
_`.naming.type`: Types are suffixed ``_t``.
_`.naming.struct`: Structure types and tags are suffixed ``_s``.
_`.naming.union`: Unions types and tags are suffixed ``_u``.
_`.naming.scope`: The naming conventions apply to all identifiers (see
ISO C §6.1.2); this includes names of functions, variables, types
(through typedef), structure and union tags, enumeration members,
structure and union members, macros, macro parameters, labels.
_`.naming.scope.labels`: labels (for ``goto`` statements) should be
rare, only in special block macros and probably not even then.
.. note::
This principle is not adhered to in the source code, which uses
``goto`` for handling error cases. Gareth Rees, 2013-05-27.
_`.naming.scope.other`: The naming convention would also extend to
enumeration types and parameters in functions prototypes but both of
those are prohibited from having names in an interface file.
_`.type.gen`: The interface defines memory addresses as ``void *`` and
sizes as ``size_t`` for compatibility with standard C (in particular,
with ``malloc()``). These types must be binary compatible with the
internal types ``Addr`` and ``Size`` respectively. Note that this
restricts the definitions of the internal types ``Addr`` and ``Size``
when the MPS is interfaced with C, but does not restrict the MPS in
general.
_`.type.opaque`: Opaque types are defined as pointers to structures
which are never defined. These types are cast to the corresponding
internal types in ``mpsi.c``.
_`.type.trans`: Some transparent structures are defined. The client is
expected to read these, or poke about in them, under restrictions
which should be documented. The most important is probably the
allocation point (``mps_ap_s``) which is part of allocation buffers.
The transparent structures must be binary compatible with
corresponding internal structures. For example, the fields of
``mps_ap_s`` must correspond with ``APStruct`` internally. This is
checked by ``mpsi.c`` in ``mps_check()``.
_`.type.pseudo`: Some pseudo-opaque structures are defined. These only
exist so that code can be inlined using macros. The client code
shouldn't mess with them. The most important case of this is the scan
state (``mps_ss_s``) which is accessed by the in-line scanning macros,
``MPS_SCAN_*`` and ``MPS_FIX*``.
_`.type.enum`: There should be no enumeration types in the interface.
Note that enum specifiers (to declare integer constants) are fine as
long as no type is declared. See guide.impl.c.misc.enum.type.
_`.type.fun`: Whenever function types or derived function types (such
as pointer to function) are declared a prototype should be used and
the parameters to the function should not be named. This includes the
case where you are declaring the prototype for an interface function.
_`.type.fun.example`: So use::
extern mps_res_t mps_alloc(mps_addr_t *, mps_pool_t, size_t, ...);
rather than::
extern mps_res_t mps_alloc(mps_addr_t *addr_return, mps_pool_t pool , size_t size, ...);
and::
typedef mps_addr_t (*mps_fmt_class_t)(mps_addr_t);
rather than::
typedef mps_addr_t (*mps_fmt_class_t)(mps_addr_t object);
See guide.impl.c.misc.prototype.parameters.
Checking
--------
_`.check.space`: When the arena needs to be recovered from a parameter
it is check using ``AVERT(Foo, foo)`` before any attempt to call
``FooArena(foo)``. The macro ``AVERT()`` in impl.h.assert performs
simple thread-safe checking of ``foo``, so it can be called outside of
``ArenaEnter()`` and ``ArenaLeave()``.
_`.check.types`: We use definitions of types in both our external
interface and our internal code, and we want to make sure that they
are compatible. (The external interface changes less often and hides
more information.) At first, we were just checking their sizes, which
wasn't very good, but I've come up with some macros which check the
assignment compatibility of the types too. This is a sufficiently
useful trick that I thought I'd send it round. It may be useful in
other places where types and structures need to be checked for
compatibility at compile time.
These macros don't generate warnings on the compilers I've tried.
``COMPATLVALUE(lvalue1, lvalue2)``
This macro checks the assignment compatibility of two lvalues. It uses
``sizeof()`` to ensure that the assignments have no effect. ::
#define COMPATLVALUE(lv1, lv2) \
((void)sizeof((lv1) = (lv2)), (void)sizeof((lv2) = (lv1)), TRUE)
``COMPATTYPE(type1, type2)``
This macro checks that two types are assignment-compatible and equal
in size. The hack here is that it generates an lvalue for each type by
casting zero to a pointer to the type. The use of ``sizeof()`` avoids
the undefined behaviour that would otherwise result from dereferencing
a null pointer. ::
#define COMPATTYPE(t1, t2) \
(sizeof(t1) == sizeof(t2) && \
COMPATLVALUE(*((t1 *)0), *((t2 *)0)))
``COMPATFIELDAPPROX(structure1, field1, structure2, field2)``
This macro checks that the offset and size of two fields in two
structure types are the same. ::
#define COMPATFIELDAPPROX(s1, f1, s2, f2) \
(sizeof(((s1 *)0)->f1) == sizeof(((s2 *)0)->f2) && \
offsetof(s1, f1) == offsetof(s2, f2))
``COMPATFIELD(structure1, field1, structure2, field2)``
This macro checks the offset, size, and assignment-compatibility of
two fields in two structure types. ::
#define COMPATFIELD(s1, f1, s2, f2) \
(COMPATFIELDAPPROX(s1, f1, s2, f2) && \
COMPATLVALUE(((s1 *)0)->f1, ((s2 *)0)->f2))
Binary compatibility issues
---------------------------
As in, "Enumeration types are not allowed" (see
mail.richard.1995-09-08.09-28).
There are two main aspects to run-time compatibility: binary interface
and protocol. The binary interface is all the information needed to
correctly use the library, and includes external symbol linkage,
calling conventions, type representation compatibility, structure
layouts, etc. The protocol is how the library is actually used by the
client code -- whether this is called before that -- and determines
the semantic correctness of the client with respect to the library.
The binary interface is determined completely by the header file and
the target. The header file specifies the external names and the
types, and the target platform specifies calling conventions and type
representation. There is therefore a many-to-one mapping between the
header file version and the binary interface.
The protocol is determined by the implementation of the library.
Constraints
-----------
_`.cons`: The MPS C Interface constrains the MPS in order to provide
useful memory management services to a C or C++ program.
_`.cons.addr`: The interface constrains the MPS address type, Addr
(design.mps.type.addr), to being the same as C's generic pointer type,
``void *``, so that the MPS can manage C objects in the natural way.
_`.pun.addr`: We pun the type of ``mps_addr_t`` (which is ``void *``)
into ``Addr`` (an incomplete type, see design.mps.type.addr). This
happens in the call to the scan state's fix function, for example.
_`.cons.size`: The interface constrains the MPS size type, ``Size``
(design.mps.type.size), to being the same as C's size type,
``size_t``, so that the MPS can manage C objects in the natural way.
_`.pun.size`: We pun the type of ``size_t`` in mps.h into ``Size`` in
the MPM, as an argument to the format methods. We assume this works.
_`.cons.word`: The MPS assumes that ``Word`` (design.mps.type.word)
and ``Addr`` (design.mps.type.addr) are the same size, and the
interface constrains ``Word`` to being the same size as C's generic
pointer type, ``void *``.
Notes
-----
The file ``mpstd.h`` is the MPS target detection header. It decodes
preprocessor symbols which are predefined by build environments in
order to determine the target platform, and then defines uniform
symbols, such as ``MPS_ARCH_I3``, for use internally by the MPS.
There is a design document for the mps interface,
design.mps.interface, but it was written before we had the idea of
having a C interface layer. It is quite relevant, though, and could be
updated. We should use it during the review.
All exported identifiers and file names should begin with ``mps_`` or
``MPS_`` so that they don't clash with other systems.
We should probably have a specialized set of rules and a special
checklist for this interface.
_`.fmt.extend`: This paragraph should be an explanation of why
``mps_fmt_A_t`` is so called. The underlying reason is future
extensibility.
_`.thread-safety`: Most calls through this interface lock the space
and therefore make the MPM single-threaded. In order to do this they
must recover the space from their parameters. Methods such as
``ThreadSpace()`` must therefore be callable when the space is *not*
locked. These methods are tagged with the tag of this note.
_`.lock-free`: Certain functions inside the MPM are thread-safe and do
not need to be serialized by using locks. They are marked with the tag
of this note.
_`.form`: Almost all functions in this implementation simply cast
their arguments to the equivalent internal types, and cast results
back to the external type, where necessary. Only exceptions are noted
in comments.
Document History
----------------
- 1996-07-29 RB_ Incomplete document. The first draft of this document
was generated in response to review.impl.h.mps.10 which revealed the
lack of a detailed design document and also the lack of conventions
for external interfaces. The aim of the draft was to record this
information, even if it isn't terribly well structured.
- 2002-06-07 RB_ Converted from MMInfo database design document.
- 2013-05-23 GDR_ Converted to reStructuredText.
.. _RB: http://www.ravenbrook.com/consultants/rb/
.. _GDR: http://www.ravenbrook.com/consultants/gdr/
Copyright and License
---------------------
Copyright © 2013 Ravenbrook Limited. All rights reserved.
<http://www.ravenbrook.com/>. This is an open source license. Contact
Ravenbrook for commercial licensing options.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. Redistributions in any form must be accompanied by information on how
to obtain complete source code for this software and any
accompanying software that uses this software. The source code must
either be included in the distribution or be available for no more than
the cost of distribution plus a nominal fee, and must be freely
redistributable under reasonable conditions. For an executable file,
complete source code means the source code for all modules it contains.
It does not include source code for modules or files that typically
accompany the major components of the operating system on which the
executable file runs.
**This software is provided by the copyright holders and contributors
"as is" and any express or implied warranties, including, but not
limited to, the implied warranties of merchantability, fitness for a
particular purpose, or non-infringement, are disclaimed. In no event
shall the copyright holders and contributors be liable for any direct,
indirect, incidental, special, exemplary, or consequential damages
(including, but not limited to, procurement of substitute goods or
services; loss of use, data, or profits; or business interruption)
however caused and on any theory of liability, whether in contract,
strict liability, or tort (including negligence or otherwise) arising in
any way out of the use of this software, even if advised of the
possibility of such damage.**

376
mps/design/interface-c/index.html
View file

@ -1,376 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<title>The design of the Memory Pool System interface to C</title>
</head>
<body bgcolor="#FFFFFF" text="#000000" link="#000099" vlink="#660066" alink="#FF0000">
<div align="center">
<p>
<a href="/">Ravenbrook</a> /
<a href="/project/">Projects</a> /
<a href="/project/mps/">Memory Pool System</a> /
<a href="/project/mps/master/">Master Product Sources</a> /
<a href="/project/mps/master/design/">Design Documents</a>
</p>
<p><i><a href="/project/mps/">Memory Pool System Project</a></i></p>
<hr />
</div>
<pre>
THE DESIGN OF THE MEMORY POOL SYSTEM INTERFACE TO C
design.mps.interface.c
incomplete doc
richard 1996-07-29
INTRODUCTION
Scope
<a id="scope" name="scope">.scope</a>: This document is the design for the Memory Pool System (MPS) interface
to the C Language, impl.h.mps.
Background
<a id="bg" name="bg">.bg</a>: See mail.richard.1996-07-24.10-57.
Document History
<a id="hist.0" name="hist.0">.hist.0</a>: The first draft of this document was generated in response to
review.impl.h.mps.10 which revealed the lack of a detailed design document and
also the lack of conventions for external interfaces. The aim of the draft was
to record this information, even if it isn't terribly well structured.
ANALYSIS
Goals
<a id="goal.c" name="goal.c">.goal.c</a>: The file impl.h.mps is the C external interface to the MPS. It is the
default interface between client code written in C and the MPS. <a id="goal.cpp" name="goal.cpp">.goal.cpp</a>:
impl.h.mps is not specifically designed to be an interface to C++, but should
be usable from C++.
Requirements
<a id="req" name="req">.req</a>: The interface must provide an interface from client code written in C to
the functionality of the MPS required by the product (see req.product), Dylan
(req.dylan), and the Core RIP (req.epcore).
mps.h may not include internal MPS header files such as "pool.h" etc.
It is essential that the interface cope well with change, in order to avoid
restricting possible future MPS developments. This means that the interface
must be "open ended" in its definitions. This accounts for some of the
apparently tortuous methods of doing things (fmt_A, for example). The
requirement is that the MPS should be able to add new functionality, or alter
the implementation of existing functionality, without affecting existing client
code. A stronger requirement is that the MPS should be able to change without
_recompiling_ client code. This is not always possible.
[.naming.global wqas presumable done in response to an unwritten requirement
regarding the use of the name spaces in C, perhaps something like
"<a id="req.name.iso" name="req.name.iso">.req.name.iso</a>: The interface shall not conflict in terms of naming with any
interfaces specified by ISO C and all reasonable future versions" and
"<a id="req.name.general" name="req.name.general">.req.name.general</a>: The interface shall use a documented and reasonably small
portion of the namespace so that clients can interoperate easily" drj
1998-10-01]
ARCHITECTURE
<a id="fig.arch" name="fig.arch">.fig.arch</a>: The architecture of the MPS Interface
Just behind mps.h is the file mpsi.c, the "MPS interface layer" which does the
job of converting types and checking parameters before calling through to the
MPS proper, using internal MPS methods.
GENERAL CONVENTIONS
<a id="naming" name="naming">.naming</a>: The external interface names should adhere to the documented interface
conventions; these are found in doc.mps.ref-man.if-conv(0).naming.
(paraphrased/recreated here) <a id="naming.unixy" name="naming.unixy">.naming.unixy</a>: The external interface does not
follow the same naming conventions as the internal code. The interface is
designed to resemble a more conventional C, Unix, or Posix naming convention.
<a id="naming.case" name="naming.case">.naming.case</a>: Identifiers are in lower case, except non-function-like macros,
which are in upper case. <a id="naming.global" name="naming.global">.naming.global</a>: All publicised identifiers are
prefixed "mps_" or "MPS_". <a id="naming.all" name="naming.all">.naming.all</a>: All identifiers defined by the MPS
should begin "mps_" or "MPS_" or "_mps_". <a id="naming.type" name="naming.type">.naming.type</a>: Types are suffixed
"_t". <a id="naming.struct" name="naming.struct">.naming.struct</a>: Structure types and tags are suffixed "_s".
<a id="naming.union" name="naming.union">.naming.union</a>: Unions types and tags are suffixed "_u".
<a id="naming.scope" name="naming.scope">.naming.scope</a>: The naming conventions apply to all identifiers (see ISO C
clause 6.1.2); this includes names of functions, variables, types (through
typedef), structure and union tags, enumeration members, structure and union
members, macros, macro parameters, labels. <a id="naming.scope.labels" name="naming.scope.labels">.naming.scope.labels</a>: labels (for
goto statements) should be rare, only in special block macros and probably not
even then. <a id="naming.scope.other" name="naming.scope.other">.naming.scope.other</a>: The naming convention would also extend to
enumeration types and parameters in functions prototypes but both of those are
prohibited from having names in an interface file.
<a id="type.gen" name="type.gen">.type.gen</a>: The interface defines memory addresses as "void *" and sizes as
"size_t" for compatibility with standard C (in particular, with malloc etc.).
These types must be binary compatible with the internal types "Addr" and "Size"
respectively. Note that this restricts the definitions of the internal types
"Addr" and "Size" when the MPS is interfaced with C, but does not restrict the
MPS in general.
<a id="type.opaque" name="type.opaque">.type.opaque</a>: Opaque types are defined as pointers to structures which are
never defined. These types are cast to the corresponding internal types in
mpsi.c.
<a id="type.trans" name="type.trans">.type.trans</a>: Some transparent structures are defined. The client is expected
to read these, or poke about in them, under restrictions which should be
documented. The most important is probably the allocation point (mps_ap_s)
which is part of allocation buffers. The transparent structures must be binary
compatible with corresponding internal structures. For example, the fields of
mps_ap_s must correspond with APStruct internally. This is checked by mpsi.c
in mps_check().
<a id="type.pseudo" name="type.pseudo">.type.pseudo</a>: Some pseudo-opaque structures are defined. These only exist so
that code can be inlined using macros. The client code shouldn't mess with
them. The most important case of this is the scan state (mps_ss_s) which is
accessed by the in-line scanning macros, MPS_SCAN_* and MPS_FIX*.
<a id="type.enum" name="type.enum">.type.enum</a>: There should be no enumeration types in the interface. Note that
enum specifiers (to declare integer constants) are fine as long as no type is
declared. See guide.impl.c.misc.enum.type.
<a id="type.fun" name="type.fun">.type.fun</a>: Whenever function types or derived function types (such as pointer
to function) are declared a prototype should be used and the parameters to the
function should not be named. This includes the case where you are declaring
the prototype for an interface function. <a id="type.fun.example" name="type.fun.example">.type.fun.example</a>: So use "extern
mps_res_t mps_alloc(mps_addr_t *, mps_pool_t, size_t, ...);" rather than
"extern mps_res_t mps_alloc(mps_addr_t *addr_return, mps_pool_t pool , size_t
size, ...);" and "typedef mps_addr_t (*mps_fmt_class_t)(mps_addr_t);" rather
then "typedef mps_addr_t (*mps_fmt_class_t)(mps_addr_t object);". See
guide.impl.c.misc.prototype.parameters.
Checking
<a id="check.space" name="check.space">.check.space</a>: When the space needs to be recovered from a parameter it is check
using AVER(CHECKT(Foo, foo)) before any attempt to access FooSpace(foo).
CHECKT (impl.h.assert) performs simple thread-safe checking of foo, so it can
be called outside of SpaceEnter/SpaceLeave. [perhaps this should be a special
macro. "AVER(CHECKT(" can look like the programmer made a mistake. drj
1998-11-05]
<a id="check.types" name="check.types">.check.types</a>: We use definitions of types in both our external interface and
our internal code, and we want to make sure that they are compatible. (The
external interface changes less often and hides more information.) At first,
we were just checking their sizes, which wasn't very good, but I've come up
with some macros which check the assignment compatibility of the types too.
This is a sufficiently useful trick that I thought I'd send it round. It may
be useful in other places where types and structures need to be checked for
compatibility at compile time.
These macros don't generate warnings on the compilers I've tried.
This macro checks the assignment compatibility of two lvalues. The hack is
that it uses sizeof to guarantee that the assignments have no effect.
#define check_lvalues(_e1, _e2) \
(sizeof((_e1) = (_e2)), sizeof((_e2) = (_e1)), sizeof(_e1) == sizeof(_e2))
This macro checks that two types are compatible and equal in size. The hack
here is that it generates an lvalue for each type by casting zero to a pointer
to the type.
#define check_types(_t1, _t2) check_lvalues(*((_t1 *)0), *((_t2 *)0))
This macro just checks that the offset and size of two fields are the same.
#define check_fields_approx(_s1, _f1, _s2, _f2) \
(offsetof(_s1, _f1) == offsetof(_s2, _f2) &amp;&amp; \
sizeof(((_s1 *)0)-&gt;_f1) == sizeof(((_s2 *)0)-&gt;_f2))
This macro checks the offset, size, and compatibility of fields.
#define check_fields(_s1, _f1, _s2, _f2) \
(check_lvalues(((_s1 *)0)-&gt;_f1, ((_s2 *)0)-&gt;_f2) &amp;&amp; \
check_fields_approx(_s1, _f1, _s2, _f2))
Binary Compatibility Issues
As in Enumeration types are not allowed (see mail.richard.1995-09-08.09-28).
There are two main aspects to run-time compatibility: binary interface and
protocol. The binary interface is all the information needed to correctly use
the library, and includes external symbol linkage, calling conventions, type
representation compatibility, structure layouts, etc. The protocol is how the
library is actually used by the client code -- whether this is called before
that -- and determines the semantic correctness of the client with respect to
the library.
The binary interface is determined completely by the header file and the
target. The header file specifies the external names and the types, and the
target platform specifies calling conventions and type representation. There
is therefore a many-to-one mapping between the header file version and the
binary interface.
The protocol is determined by the implementation of the library.
Constraints
<a id="cons" name="cons">.cons</a>: The MPS C Interface constrains the MPS in order to provide useful memory
management services to a C or C++ program.
<a id="cons.addr" name="cons.addr">.cons.addr</a>: The interface constrains the MPS address type, Addr
(design.mps.type.addr), to being the same as C's generic pointer type, void *,
so that the MPS can manage C objects in the natural way.
<a id="pun.addr" name="pun.addr">.pun.addr</a>: We pun the type of mps_addr_t (which is void *) into Addr (an
incomplete type, see design.mps.type.addr). This happens in the call to the
scan state's fix function, for example.
<a id="cons.size" name="cons.size">.cons.size</a>: The interface constrains the MPS size type, Size
(design.mps.type.size), to being the same as C's size type, size_t, so that the
MPS can manage C objects in the natural way.
<a id="pun.size" name="pun.size">.pun.size</a>: We pun the type of size_t in mps.h into Size in the MPM, as an
argument to the format methods. We assume this works.
<a id="cons.word" name="cons.word">.cons.word</a>: The MPS assumes that Word (design.mps.type.word) and Addr
(design.mps.type.addr) are the same size, and the interface constrains Word to
being the same size as C's generic pointer type, void *.
NOTES
The file mpstd.h is the MPS target detection header. It decodes
preprocessor symbols which are predefined by build environments in order to
determine the target platform, and then defines uniform symbols, such as
MPS_ARCH_I3, for use internally by the MPS.
There is a design document for the mps interface, design.mps.interface, but
it was written before we had the idea of having a C interface layer. It is
quite relevant, though, and could be updated. We should use it during the
review.
All exported identifiers and file names should begin with mps_ or mps so
that they don't clash with other systems.
We should probably have a specialized set of rules and a special checklist
for this interface.
<a id="fmt.extend" name="fmt.extend">.fmt.extend</a>: This paragraph should be an explanation of why mps_fmt_A is so
called. The underlying reason is future extensibility.
<a id="thread-safety" name="thread-safety">.thread-safety</a>: Most calls through this interface lock the space and therefore
make the MPM single-threaded. In order to do this they must recover the space
from their parameters. Methods such as ThreadSpace() must therefore be
callable when the space is _not_ locked. These methods are tagged with the tag
of this note.
<a id="lock-free" name="lock-free">.lock-free</a>: Certain functions inside the MPM are thread-safe and do not need to
be serialized by using locks. They are marked with the tag of this note.
<a id="form" name="form">.form</a>: Almost all functions in this implementation simply cast their arguments
to the equivalent internal types, and cast results back to the external type,
where necessary. Only exceptions are noted in comments.
</pre>
<h2><a id="section-A" name="section-A">A. References</a></h2>
<!-- Template Entry
<table>
<tr valign="top">
<td>[<a id="ref-#REF#" name="ref-#REF#" href="#REF_URL#">#REF_NAME#</a>]</td>
<td>
"#REF_TITLE#";
#REF_AUTHOR#;
&lt;URL: <a href="#REF_URL#">#REF_URL#</a>&gt;;
#REF_DATE#.
</td>
</tr>
</table>
-->
<h2><a id="section-B" name="section-B">B. Document History</a></h2>
<table>
<tr valign="top">
<td>2002-06-07</td>
<td><a href="mailto:rb@ravenbrook.com">RB</a></td>
<td>Converted from MMInfo database design document.</td>
</tr>
</table>
<h2><a id="section-C" name="section-C">C. Copyright and License</a></h2>
<p> This document is copyright &copy; 1995-2002 <a href="http://www.ravenbrook.com/">Ravenbrook Limited</a>. All rights reserved. This is an open source license. Contact Ravenbrook for commercial licensing options. </p>
<p> Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: </p>
<ol>
<li> Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. </li>
<li> Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. </li>
<li> Redistributions in any form must be accompanied by information on how to obtain complete source code for the this software and any accompanying software that uses this software. The source code must either be included in the distribution or be available for no more than the cost of distribution plus a nominal fee, and must be freely redistributable under reasonable conditions. For an executable file, complete source code means the source code for all modules it contains. It does not include source code for modules or files that typically accompany the major components of the operating system on which the executable file runs. </li>
</ol>
<p> <strong> This software is provided by the copyright holders and contributors "as is" and any express or implied warranties, including, but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement, are disclaimed. In no event shall the copyright holders and contributors be liable for any direct, indirect, incidental, special, exemplary, or consequential damages (including, but not limited to, procurement of substitute goods or services; loss of use, data, or profits; or business interruption) however caused and on any theory of liability, whether in contract, strict liability, or tort (including negligence or otherwise) arising in any way out of the use of this software, even if advised of the possibility of such damage. </strong> </p>
<hr />
<div align="center">
<p><code>$Id$</code></p>
<p>
<a href="/">Ravenbrook</a> /
<a href="/project/">Projects</a> /
<a href="/project/mps/">Memory Pool System</a> /
<a href="/project/mps/master/">Master Product Sources</a> /
<a href="/project/mps/master/design/">Design Documents</a>
</p>
</div>
</body>
</html>

465
mps/design/io.txt Normal file
View file

@ -0,0 +1,465 @@
.. mode: -*- rst -*-
I/O subsystem
=============
:Tag: design.mps.io
:Author: Richard Brooksby
:Date: 1996-08-30
:Status: incomplete design
:Revision: $Id$
:Copyright: See `Copyright and License`_.
Introduction
------------
_`.intro`: This document is the design of the MPS I/O Subsystem, a
part of the plinth.
_`.readership`: This document is intended for MPS developers.
Background
----------
_`.bg`: This design is partly based on the design of the Internet User
Datagram Protocol (UDP). Mainly I used this to make sure I hadn't left
out anything which we might need.
Purpose
-------
_`.purpose`: The purpose of the MPS I/O Subsystem is to provide a
means to measure, debug, control, and test a memory manager build
using the MPS.
_`.purpose.measure`: Measurement consists of emitting data which can
be collected and analysed in order to improve the attributes of
application program, quite possibly by adjusting parameters of the
memory manager (see overview.mps.usage).
_`.purpose.control`: Control means adjusting the behaviour of the MM
dynamically. For example, one might want to adjust a parameter in
order to observe the effect, then transfer that adjustment to the
client application later.
_`.purpose.test`: Test output can be used to ensure that the memory
manager is behaving as expected in response to certain inputs.
Requirements
------------
General
.......
_`.req.fun.non-hosted`: The MPM must be a host-independent system.
_`.req.attr.host`: It should be easy for the client to set up the MPM
for a particular host (such as a washing machine).
Functional
..........
_`.req.fun.measure`: The subsystem must allow the MPS to transmit
quantitative measurement data to an external tool so that the system
can be tuned.
_`.req.fun.debug`: The subsystem must allow the MPS to transmit
qualitative information about its operation to an external tool so
that the system can be debugged.
_`.req.fun.control`: The subsystem must allow the MPS to receive
control information from an external tool so that the system can be
adjusted while it is running.
_`.req.dc.env.no-net`: The subsystem should operate in environments
where there is no networking available.
_`.req.dc.env.no-fs`: The subsystem should operate in environments
where there is no filesystem available.
Architecture
------------
_`.arch.diagram`: I/O Architecture Diagram
[missing diagram]
_`.arch.int`: The I/O Interface is a C function call interface by
which the MPM sends and receives "messages" to and from the hosted I/O
module.
_`.arch.module`: The modules are part of the MPS but not part of the
freestanding core system (see design.mps.exec-env). The I/O module is
responsible for transmitting those messages to the external tools, and
for receiving messages from external tools and passing them to the
MPM.
_`.arch.module.example`: For example, the "file implementation" might
just send/write telemetry messages into a file so that they can be
received/read later by an off-line measurement tool.
_`.arch.external`: The I/O Interface is part of interface to the
freestanding core system (see design.mps.exec-env). This is so that
the MPS can be deployed in a freestanding environment, with a special
I/O module. For example, if the MPS is used in a washing machine the
I/O module could communicate by writing output to the seven-segment
display.
Example configurations
......................
_`.example.telnet`: This shows the I/O subsystem communicating with a
telnet client over a TCP/IP connection. In this case, the I/O
subsystem is translating the I/O Interface into an interactive text
protocol so that the user of the telnet client can talk to the MM.
[missing diagram]
_`.example.file`: This shows the I/O subsystem dumping measurement
data into a file which is later read and analysed. In this case the
I/O subsystem is simply writing out binary in a format which can be
decoded.
[missing diagram]
_`.example.serial`: This shows the I/O subsystem communicating with a
graphical analysis tool over a serial link. This could be useful for a
developer who has two machines in close proximity and no networking
support.
_`.example.local`: In this example the application is talking directly
to the I/O subsystem. This is useful when the application is a
reflective development environment (such as MLWorks) which wants to
observe its own behaviour.
[missing diagram]
Interface
---------
_`.if.msg`: The I/O interface is oriented around opaque binary
"messages" which the I/O module must pass between the MPM and external
tools. The I/O module need not understand or interpret the contents of
those messages.
_`.if.msg.opaque`: The messages are opaque in order to minimize the
dependency of the I/O module on the message internals. It should be
possible for clients to implement their own I/O modules for unusual
environments. We do not want to reveal the internal structure of our
data to the clients. Nor do we want to burden them with the details of
our protocols. We'd also like their code to be independent of ours, so
that we can expand or change the protocols without requiring them to
modify their modules.
_`.if.msg.dgram`: Neither the MPM nor the external tools should assume
that the messages will be delivered in finite time, exactly once, or
in order. This will allow the I/O modules to be implemented using
unreliable transport layers such as the Internet User Datagram Protocl
(UDP). It will also give the I/O module the freedom to drop
information rather than block on a congested network, or stop the
memory manager when the disk is full, or similar events which really
shouldn't cause the memory manager to stop working. The protocols we
need to implement at the high level can be design to be robust against
lossage without much difficulty.
I/O module state
................
_`.if.state`: The I/O module may have some internal state to preserve.
The I/O Interface defines a type for this state, ``mps_io_t``, a
pointer to an incomplete structure ``mps_io_s``. The I/O module is at
liberty to define this structure.
Message types
.............
_`.if.type`: The I/O module must be able to deliver messages of
several different types. It will probably choose to send them to
different destinations based on their type: telemetry to the
measurement tool, debugging output to the debugger, etc. ::
typedef int mps_io_type_t;
enum {
MPS_IO_TYPE_TELEMETRY,
MPS_IO_TYPE_DEBUG
};
Limits
......
_`.if.message-max`: The interface will define an unsigned integral
constant ``MPS_IO_MESSAGE_MAX`` which will be the maximum size of
messages that the MPM will pass to ``mps_io_send()`` (`.if.send`_) and
the maximum size it will expect to receive from ``mps_io_receive()``.
Interface set-up and tear-down
..............................
_`.if.create`: The MPM will call ``mps_io_create()`` to set up the I/O
module. On success, this function should return ``MPS_RES_OK``. It may
also initialize a "state" value which will be passed to subsequent
calls through the interface.
_`.if.destroy`: The MPM will call ``mps_io_destroy()`` to tear down
the I/O module, after which it guarantees that the state value will
not be used again. The ``state`` parameter is the state previously
returned by ``mps_io_create()`` (`.if.create`_).
Message send and receive
........................
``extern mps_res_t mps_io_send(mps_io_t state, mps_io_type_t type, void *message, size_t size)``
_`.if.send`: The MPM will call ``mps_io_send()`` when it wishes to
send a message to a destination. The ``state`` parameter is the state
previously returned by ``mps_io_create()`` (`.if.create`_). The
``type`` parameter is the type (`.if.type`_) of the message. The
``message`` parameter is a pointer to a buffer containing the message,
and ``size`` is the length of that message, in bytes. The I/O module
must make an effort to deliver the message to the destination, but is
not expected to guarantee delivery. The function should return
``MPS_RES_IO`` only if a serious error occurs that should cause the
MPM to return with an error to the client application. Failure to
deliver the message does not count.
.. note::
Should there be a timeout parameter? What are the timing
constraints? ``mps_io_send()`` shouldn't block.
``extern mps_res_t mps_io_receive(mps_io_t state, void **buffer_o, size_t *size_o)``
_`.if.receive`: The MPM will call ``mps_io_receive()`` when it wants
to see if a message has been sent to it. The ``state`` parameter is
the state previously returned by ``mps_io_create()`` (`.if.create`_).
The ``buffer_o`` parameter is a pointer to a value which should be
updated with a pointer to a buffer containing the message received.
The ``size_o`` parameter is a pointer to a value which should be
updated with the length of the message received. If there is no
message ready for receipt, the length returned should be zero.
.. note::
Should we be able to receive truncated messages? How can this be
done neatly?
I/O module implementations
--------------------------
Routeing
........
The I/O module must decide where to send the various messages. A
file-based implementation could put them in different files based on
their types. A network-based implementation must decide how to address
the messages. In either case, any configuration must either be
statically compiled into the module, or else read from some external
source such as a configuration file.
Notes
-----
The external tools should be able to reconstruct stuff from partial
info. For example, you come across a fragment of an old log containing
just a few old messages. What can you do with it?
Here's some completely untested code which might do the job for UDP.
::
#include "mpsio.h"
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <fcntl.h>
#include <errno.h>
typedef struct mps_io_s {
int sock;
struct sockaddr_in mine;
struct sockaddr_in telemetry;
struct sockaddr_in debugging;
} mps_io_s;
static mps_bool_t inited = 0;
static mps_io_s state;
mps_res_t mps_io_create(mps_io_t *mps_io_o)
{
int sock, r;
if(inited)
return MPS_RES_LIMIT;
state.mine = /* setup somehow from config */;
state.telemetry = /* setup something from config */;
state.debugging = /* setup something from config */;
/* Make a socket through which to communicate. */
sock = socket(AF_INET, SOCK_DGRAM, 0);
if(sock == -1) return MPS_RES_IO;
/* Set socket to non-blocking mode. */
r = fcntl(sock, F_SETFL, O_NDELAY);
if(r == -1) return MPS_RES_IO;
/* Bind the socket to some UDP port so that we can receive messages. */
r = bind(sock, (struct sockaddr *)&state.mine, sizeof(state.mine));
if(r == -1) return MPS_RES_IO;
state.sock = sock;
inited = 1;
*mps_io_o = &state;
return MPS_RES_OK;
}
void mps_io_destroy(mps_io_t mps_io)
{
assert(mps_io == &state);
assert(inited);
(void)close(state.sock);
inited = 0;
}
mps_res_t mps_io_send(mps_io_t mps_io, mps_type_t type,
void *message, size_t size)
{
struct sockaddr *toaddr;
assert(mps_io == &state);
assert(inited);
switch(type) {
MPS_IO_TYPE_TELEMETRY:
toaddr = (struct sockaddr *)&state.telemetry;
break;
MPS_IO_TYPE_DEBUGGING:
toaddr = (struct sockaddr *)&state.debugging;
break;
default:
assert(0);
return MPS_RES_UNIMPL;
}
(void)sendto(state.sock, message, size, 0, toaddr, sizeof(*toaddr));
return MPS_RES_OK;
}
mps_res_t mps_io_receive(mps_io_t mps_io,
void **message_o, size_t **size_o)
{
int r;
static char buffer[MPS_IO_MESSAGE_MAX];
assert(mps_io == &state);
assert(inited);
r = recvfrom(state.sock, buffer, sizeof(buffer), 0, NULL, NULL);
if(r == -1)
switch(errno) {
/* Ignore interrupted system calls, and failures due to lack */
/* of resources (they might go away.) */
case EINTR: case ENOMEM: case ENOSR:
r = 0;
break;
default:
return MPS_RES_IO;
}
*message_o = buffer;
*size_o = r;
return MPS_RES_OK;
}
Attachments
-----------
"O Architecture Diagram"
"O Configuration Diagrams"
Document History
----------------
- 1996-08-30 RB_ Document created from paper notes.
- 1997-06-10 RB_ Updated with mail.richard.1997-05-30.16-13 and
subsequent discussion in the Pool Hall at Longstanton. (See also
mail.drj.1997-06-05.15-20.)
- 2002-06-07 RB_ Converted from MMInfo database design document.
- 2013-05-23 GDR_ Converted to reStructuredText.
.. _RB: http://www.ravenbrook.com/consultants/rb/
.. _GDR: http://www.ravenbrook.com/consultants/gdr/
Copyright and License
---------------------
Copyright © 2013 Ravenbrook Limited. All rights reserved.
<http://www.ravenbrook.com/>. This is an open source license. Contact
Ravenbrook for commercial licensing options.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. Redistributions in any form must be accompanied by information on how
to obtain complete source code for this software and any
accompanying software that uses this software. The source code must
either be included in the distribution or be available for no more than
the cost of distribution plus a nominal fee, and must be freely
redistributable under reasonable conditions. For an executable file,
complete source code means the source code for all modules it contains.
It does not include source code for modules or files that typically
accompany the major components of the operating system on which the
executable file runs.
**This software is provided by the copyright holders and contributors
"as is" and any express or implied warranties, including, but not
limited to, the implied warranties of merchantability, fitness for a
particular purpose, or non-infringement, are disclaimed. In no event
shall the copyright holders and contributors be liable for any direct,
indirect, incidental, special, exemplary, or consequential damages
(including, but not limited to, procurement of substitute goods or
services; loss of use, data, or profits; or business interruption)
however caused and on any theory of liability, whether in contract,
strict liability, or tort (including negligence or otherwise) arising in
any way out of the use of this software, even if advised of the
possibility of such damage.**

494
mps/design/io/index.html
View file

@ -1,494 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<title>The design of the MPS i/o subsystem</title>
</head>
<body bgcolor="#FFFFFF" text="#000000" link="#000099" vlink="#660066" alink="#FF0000">
<div align="center">
<p>
<a href="/">Ravenbrook</a> /
<a href="/project/">Projects</a> /
<a href="/project/mps/">Memory Pool System</a> /
<a href="/project/mps/master/">Master Product Sources</a> /
<a href="/project/mps/master/design/">Design Documents</a>
</p>
<p><i><a href="/project/mps/">Memory Pool System Project</a></i></p>
<hr />
</div>
<pre>
THE DESIGN OF THE MPS I/O SUBSYSTEM
design.mps.io
incomplete design
richard 1996-08-30
INTRODUCTION
<a id="intro" name="intro">.intro</a>: This document is the design of the MPS I/O Subsystem, a part of the
plinth.
<a id="readership" name="readership">.readership</a>: This document is intended for MPS developers.
History
<a id="hist.1" name="hist.1">.hist.1</a>: Document created from paper notes by Richard Brooksby, 1996-08-30.
<a id="hist.2" name="hist.2">.hist.2</a>: Updated with mail.richard.1997-05-30.16-13 and subsequent discussion
in the Pool Hall at Longstanton. (See also mail.drj.1997-06-05.15-20.)
richard 1997-06-10
Background
<a id="bg" name="bg">.bg</a>: This design is partly based on the design of the Internet User Datagram
Protocol (UDP). Mainly I used this to make sure I hadn't left out anything
which we might need.
PURPOSE
<a id="purpose" name="purpose">.purpose</a>: The purpose of the MPS I/O Subsystem is to provide a means to
measure, debug, control, and test a memory manager build using the MPS.
<a id="purpose.measure" name="purpose.measure">.purpose.measure</a>: Measurement consists of emitting data which can be collected
and analysed in order to improve the attributes of application program, quite
possibly by adjusting parameters of the memory manager (see overview.mps.usage).
<a id="purpose.control" name="purpose.control">.purpose.control</a>: Control means adjusting the behaviour of the MM dynamically.
For example, one might want to adjust a parameter in order to observe the
effect, then transfer that adjustment to the client application later.
<a id="purpose.test" name="purpose.test">.purpose.test</a>: Test output can be used to ensure that the memory manager is
behaving as expected in response to certain inputs.
REQUIREMENTS
General
<a id="req.fun.non-hosted" name="req.fun.non-hosted">.req.fun.non-hosted</a>: The MPM must be a host-independent system.
<a id="req.attr.host" name="req.attr.host">.req.attr.host</a>: It should be easy for the client to set up the MPM for a
particular host (such as a washing machine).
Functional
<a id="req.fun.measure" name="req.fun.measure">.req.fun.measure</a>: The subsystem must allow the MPS to transmit quantitative
measurement data to an external tool so that the system can be tuned.
<a id="req.fun.debug" name="req.fun.debug">.req.fun.debug</a>: The subsystem must allow the MPS to transmit qualitative
information about its operation to an external tool so that the system can be
debugged.
<a id="req.fun.control" name="req.fun.control">.req.fun.control</a>: The subsystem must allow the MPS to receive control
information from an external tool so that the system can be adjusted while it
is running.
<a id="req.dc.env.no-net" name="req.dc.env.no-net">.req.dc.env.no-net</a>: The subsystem should operate in environments where there is
no networking available.
<a id="req.dc.env.no-fs" name="req.dc.env.no-fs">.req.dc.env.no-fs</a>: The subsystem should operate in environments where there is
no filesystem available.
ARCHITECTURE
<a id="arch.diagram" name="arch.diagram">.arch.diagram</a>:
- I/O Architecture Diagram
<a id="arch.int" name="arch.int">.arch.int</a>: The I/O Interface is a C function call interface by which the MPM
sends and receives "messages" to and from the hosted I/O module.
<a id="arch.module" name="arch.module">.arch.module</a>: The modules are part of the MPS but not part of the freestanding
core system (see design.mps.exec-env). The I/O module is responsible for
transmitting those messages to the external tools, and for receiving messages
from external tools and passing them to the MPM.
<a id="arch.module.example" name="arch.module.example">.arch.module.example</a>: For example, the "file implementation" might just
send/write telemetry messages into a file so that they can be received/read
later by an off-line measurement tool.
<a id="arch.external" name="arch.external">.arch.external</a>: The I/O Interface is part of interface to the freestanding core
system (see design.mps.exec-env). This is so that the MPS can be deployed in a
freestanding environment, with a special I/O module. For example, if the MPS
is used in a washing machine the I/O module could communicate by writing output
to the seven-segment display.
Example Configurations
<a id="example.telnet" name="example.telnet">.example.telnet</a>: This shows the I/O Subsystem communicating with a telnet
client over a TCP/IP connection. In this case, the I/O Subsystem is
translating the I/O Interface into an interactive text protocol so that the
user of the telnet client can talk to the MM.
<a id="example.file" name="example.file">.example.file</a>: This shows the I/O Subsystem dumping measurement data into a
file which is later read and analysed. In this case the I/O Subsystem is
simply writing out binary in a format which can be decoded.
<a id="example.serial" name="example.serial">.example.serial</a>: This shows the I/O Subsystem communicating with a graphical
analysis tool over a serial link. This could be useful for a developer who has
two machines in close proximity and no networking support.
<a id="example.local" name="example.local">.example.local</a>: In this example the application is talking directly to the I/O
Subsystem. This is useful when the application is a reflective development
environment (such as MLWorks) which wants to observe its own behaviour.
- MPS I/O Configuration Diagrams
INTERFACE
<a id="if.msg" name="if.msg">.if.msg</a>: The I/O interface is oriented around opaque binary "messages" which
the I/O module must pass between the MPM and external tools. The I/O module
need not understand or interpret the contents of those messages.
<a id="if.msg.opaque" name="if.msg.opaque">.if.msg.opaque</a>: The messages are opaque in order to minimize the dependency of
the I/O module on the message internals. It should be possible for clients to
implement their own I/O modules for unusual environments. We do not want to
reveal the internal structure of our data to the clients. Nor do we want to
burden them with the details of our protocols. We'd also like their code to be
independent of ours, so that we can expand or change the protocols without
requiring them to modify their modules.
<a id="if.msg.dgram" name="if.msg.dgram">.if.msg.dgram</a>: Neither the MPM nor the external tools should assume that the
messages will be delivered in finite time, exactly once, or in order. This
will allow the I/O modules to be implemented using unreliable transport layers
such as the Internet User Datagram Protocl (UDP). It will also give the I/O
module the freedom to drop information rather than block on a congested
network, or stop the memory manager when the disk is full, or similar events
which really shouldn't cause the memory manager to stop working. The protocols
we need to implement at the high level can be design to be robust against
lossage without much difficulty.
I/O Module State
<a id="if.state" name="if.state">.if.state</a>: The I/O module may have some internal state to preserve. The I/O
Interface defines a type for this state, "mps_io_t", a pointer to an incomplete
structure "mps_io_s". The I/O module is at liberty to define this structure.
typedef struct mps_io_s *mps_io_t;
Message Types
<a id="if.type" name="if.type">.if.type</a>: The I/O module must be able to deliver messages of several different
types. It will probably choose to send them to different destinations based on
their type: telemetry to the measurement tool, debugging output to the
debugger, etc.
typedef int mps_io_type_t;
enum {
MPS_IO_TYPE_TELEMETRY,
MPS_IO_TYPE_DEBUG
};
Limits
<a id="if.message-max" name="if.message-max">.if.message-max</a>: The interface will define an unsigned integral constant
"MPS_IO_MESSAGE_MAX" which will be the maximum size of messages that the MPM
will pass to "mps_io_send" (.if.send) and the maximum size it will expect to
receive from "mps_io_receive".
Interface Set-up and Tear-down
<a id="if.create" name="if.create">.if.create</a>: The MPM will call "mps_io_create" to set up the I/O module. On
success, this function should return "MPS_RES_OK". It may also initialize
"*mps_io_r" to a "state" value which will be passed to subsequent calls through
the interface.
extern mps_res_t mps_io_create(mps_io_t *mps_io_r);
<a id="if.destroy" name="if.destroy">.if.destroy</a>: The MPM will call "mps_io_destroy" to tear down the I/O module,
after which it guarantees that the state value "mps_io" will not be used
again. The "state" parameter is the state previously returned by
"mps_io_create" (.if.create).
extern void mps_io_destroy(mps_io_t mps_io);
Message Send and Receive
<a id="if.send" name="if.send">.if.send</a>: The MPM will call "mps_io_send" when it wishes to send a message to a
destination. The "state" parameter is the state previously returned by
"mps_io_create" (.if.create). The "type" parameter is the type (.if.type) of
the message. The "message" parameter is a pointer to a buffer containing the
message, and "size" is the length of that message, in bytes. The I/O module
must make an effort to deliver the message to the destination, but is not
expected to guarantee delivery. The function should return "MPS_RES_IO" only
if a serious error occurs that should cause the MPM to return with an error to
the client application. Failure to deliver the message does not count.
[Should there be a timeout parameter? What are the timing constraints?
mps_io_send shouldn't block.]
extern mps_res_t mps_io_send(mps_io_t state,
mps_io_type_t type,
void *message,
size_t size);
<a id="if.receive" name="if.receive">.if.receive</a>: The MPM will call "mps_io_receive" when it wants to see if a
message has been sent to it. The "state" parameter is the state previously
returned by "mps_io_create" (.if.create). The "buffer_o" parameter is a
pointer to a value which should be updated with a pointer to a buffer
containing the message received. The "size_o" parameter is a pointer to a
value which should be updated with the length of the message received. If
there is no message ready for receipt, the length returned should be zero.
(Should we be able to receive truncated messages? How can this be done neatly?)
extern mps_res_t mps_io_receive(mps_io_t state,
void **buffer_o,
size_t *size_o);
I/O MODULE IMPLEMENTATIONS
Routeing
The I/O module must decide where to send the various messages. A file-based
implementation could put them in different files based on their types. A
network-based implementation must decide how to address the messages. In
either case, any configuration must either be statically compiled into the
module, or else read from some external source such as a configuration file.
NOTES
The external tools should be able to reconstruct stuff from partial info. For
example, you come across a fragment of an old log containing just a few old
messages. What can you do with it?
Here's some completely untested code which might do the job for UDP.
---
#include "mpsio.h"
#include &lt;sys/types.h&gt;
#include &lt;sys/socket.h&gt;
#include &lt;netinet/in.h&gt;
#include &lt;arpa/inet.h&gt;
#include &lt;fcntl.h&gt;
#include &lt;errno.h&gt;
typedef struct mps_io_s {
int sock;
struct sockaddr_in mine;
struct sockaddr_in telemetry;
struct sockaddr_in debugging;
} mps_io_s;
static mps_bool_t inited = 0;
static mps_io_s state;
mps_res_t mps_io_create(mps_io_t *mps_io_o)
{
int sock, r;
if(inited)
return MPS_RES_LIMIT;
state.mine = /* setup somehow from config */;
state.telemetry = /* setup something from config */;
state.debugging = /* setup something from config */;
/* Make a socket through which to communicate. */
sock = socket(AF_INET, SOCK_DGRAM, 0);
if(sock == -1) return MPS_RES_IO;
/* Set socket to non-blocking mode. */
r = fcntl(sock, F_SETFL, O_NDELAY);
if(r == -1) return MPS_RES_IO;
/* Bind the socket to some UDP port so that we can receive messages. */
r = bind(sock, (struct sockaddr *)&amp;state.mine, sizeof(state.mine));
if(r == -1) return MPS_RES_IO;
state.sock = sock;
inited = 1;
*mps_io_o = &amp;state;
return MPS_RES_OK;
}
void mps_io_destroy(mps_io_t mps_io)
{
assert(mps_io == &amp;state);
assert(inited);
(void)close(state.sock);
inited = 0;
}
mps_res_t mps_io_send(mps_io_t mps_io, mps_type_t type,
void *message, size_t size)
{
struct sockaddr *toaddr;
assert(mps_io == &amp;state);
assert(inited);
switch(type) {
MPS_IO_TYPE_TELEMETRY:
toaddr = (struct sockaddr *)&amp;state.telemetry;
break;
MPS_IO_TYPE_DEBUGGING:
toaddr = (struct sockaddr *)&amp;state.debugging;
break;
default:
assert(0);
return MPS_RES_UNIMPL;
}
(void)sendto(state.sock, message, size, 0, toaddr, sizeof(*toaddr));
return MPS_RES_OK;
}
mps_res_t mps_io_receive(mps_io_t mps_io,
void **message_o, size_t **size_o)
{
int r;
static char buffer[MPS_IO_MESSAGE_MAX];
assert(mps_io == &amp;state);
assert(inited);
r = recvfrom(state.sock, buffer, sizeof(buffer), 0, NULL, NULL);
if(r == -1)
switch(errno) {
/* Ignore interrupted system calls, and failures due to lack */
/* of resources (they might go away.) */
case EINTR: case ENOMEM: case ENOSR:
r = 0;
break;
default:
return MPS_RES_IO;
}
*message_o = buffer;
*size_o = r;
return MPS_RES_OK;
}
ATTACHMENTS
"O Architecture Diagram"
"O Configuration Diagrams"
</pre>
<h2><a id="section-A" name="section-A">A. References</a></h2>
<!-- Template Entry
<table>
<tr valign="top">
<td>[<a id="ref-#REF#" name="ref-#REF#" href="#REF_URL#">#REF_NAME#</a>]</td>
<td>
"#REF_TITLE#";
#REF_AUTHOR#;
&lt;URL: <a href="#REF_URL#">#REF_URL#</a>&gt;;
#REF_DATE#.
</td>
</tr>
</table>
-->
<h2><a id="section-B" name="section-B">B. Document History</a></h2>
<table>
<tr valign="top">
<td>2002-06-07</td>
<td><a href="mailto:rb@ravenbrook.com">RB</a></td>
<td>Converted from MMInfo database design document.</td>
</tr>
</table>
<h2><a id="section-C" name="section-C">C. Copyright and License</a></h2>
<p> This document is copyright &copy; 1995-2002 <a href="http://www.ravenbrook.com/">Ravenbrook Limited</a>. All rights reserved. This is an open source license. Contact Ravenbrook for commercial licensing options. </p>
<p> Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: </p>
<ol>
<li> Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. </li>
<li> Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. </li>
<li> Redistributions in any form must be accompanied by information on how to obtain complete source code for the this software and any accompanying software that uses this software. The source code must either be included in the distribution or be available for no more than the cost of distribution plus a nominal fee, and must be freely redistributable under reasonable conditions. For an executable file, complete source code means the source code for all modules it contains. It does not include source code for modules or files that typically accompany the major components of the operating system on which the executable file runs. </li>
</ol>
<p> <strong> This software is provided by the copyright holders and contributors "as is" and any express or implied warranties, including, but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement, are disclaimed. In no event shall the copyright holders and contributors be liable for any direct, indirect, incidental, special, exemplary, or consequential damages (including, but not limited to, procurement of substitute goods or services; loss of use, data, or profits; or business interruption) however caused and on any theory of liability, whether in contract, strict liability, or tort (including negligence or otherwise) arising in any way out of the use of this software, even if advised of the possibility of such damage. </strong> </p>
<hr />
<div align="center">
<p><code>$Id$</code></p>
<p>
<a href="/">Ravenbrook</a> /
<a href="/project/">Projects</a> /
<a href="/project/mps/">Memory Pool System</a> /
<a href="/project/mps/master/">Master Product Sources</a> /
<a href="/project/mps/master/design/">Design Documents</a>
</p>
</div>
</body>
</html>

733
mps/design/locus.txt Normal file
View file

@ -0,0 +1,733 @@
.. mode: -*- rst -*-
MPS Configuration
=================
:Tag: design.mps.locus
:Author: Gavin Matthews
:Date: 1998-02-27
:Status: incomplete design
:Revision: $Id$
:Copyright: See `Copyright and License`_.
Introduction
------------
_`.intro`: The locus manager coordinates between the pools and takes
the burden of having to be clever about tract/group placement away
from the pools, preserving trace differentiability and contiguity
where appropriate.
_`.source`: mail.gavinm.1998-02-05.17-52(0), mail.ptw.1998-02-05.19-53(0),
mail.pekka.1998-02-09.13-58(0), and mail.gavinm.1998-02-09.14-05(0).
_`.readership`: Any MPS developer.
Overview
--------
The MPS manages three main resources:
1. storage;
2. address space;
3. time.
The locus manager manages address space at the arena level.
.. note:
Tucker was right: see mail.ptw.1998-11-02.14-25. Richard Kistruck,
2007-04-24.
When a pool wants some address space, it expresses some preferences to
the locus manager. The locus manager and the arena (working together)
try to honour these preferences, and decide what address space the
pool gets.
Preferences are expressed by the ``SegPref`` argument to
``SegAlloc()``. Note that, when they call ``SegAlloc()``, pools are
asking for address space and writeable storage simultaneously, in a
single call. There is currently no way for pools to reserve address
space without requesting storage.
Why is it important to manage address space?
............................................
1. Trace differentiability
Carefully chosen addresses are used by reference tracing systems
(ie. automatic pools), to categorise objects into clumps; and to
summarise and cheaply find references between clumps.
Different clumps will become worth collecting at different times
(the classic example, of course, is generations in a generational
collector). For these partial collections to be efficient, it must
be cheap to keep these clumps differentiable, cheap to condemn
(Whiten) a particular clump, and cheap to find a good conservative
approximation to all inward references to a clump (both initially
to construct the Grey set, and to make scanning the Grey set
efficient).
This is what the MPS zone mechanism is all about.
The locus manager manages the mapping from clumps to zones.
To specify a clump, pools use the ``SegPrefGen`` argument to
``SegPrefExpress()``.
.. note::
The name is misleading: generations are only one sort of clump.
Richard Kistruck, 2007-04-24.
2. Prevent address space fragmentation (within the arena)
Address space is not infinite.
In some use cases, the MPS is required to remain efficient when
using very nearly all available address space and storage. For
example, with the client-arena class, where the only address space
available is that of the storage available.
Even with the VM arena class, typical storage sizes (as of 2007)
can make 32-bit address space constrained: the client may need
several gigabytes, which leaves little spare address space.
Address space fragmentation incurs failure when there is no way to
allocate a big block of address space. The big block may be
requested via the MPS (by the client), or by something else in the
same process, such as a third-party graphics library, image
library, etc.
Address space fragmentation incurs cost when:
- desired large-block requests (such as for buffering) are denied,
causing them to be re-requested as a smaller block, or as several
smaller blocks;
- possible operating-system costs in maintaining a fragmented
mapping?
3. Prevent storage fragmentation (within tracts and segments)
Storage is not infinite: it is allocated in multiples of a
fixed-size tract. Small lonely objects, each retaining a whole
tract, cause storage fragmentation.
Non-moving pools manage this fragmentation with placement
strategies that use:
- co-located death (in space and time);
- segment merging and splitting.
These pool-level strategies always care about contiguity of object
storage. They also often care about the *ordering* of addresses,
because pool code uses an address-ordered search when choosing
where to place a new object. For these two reasons, the address
chosen (by the locus manager and arena) for new tracts is
important.
Certain specialised pools, and/or some client programs that use
them, have carefully tuned segment sizes, positioning, and search
order. Be careful: seemingly inconsequential changes can
catastrophically break this tuning.
Pools can specify a preference for High and Low ends of address
space, which implies a search-order. Pools could also specify
clumping, using either ``SegPrefGen`` or ``SegPrefZoneSet``.
Discovering the layout
......................
The locus manager is not given advance notice of how much address
space will be required with what preferences. Instead, the locus
manager starts with an empty layout, and adapts it as more requests
come in over time. It is attempting to discover a suitable layout by
successive refinement. This is ambitious.
Definitions
-----------
_`.note.cohort`: We use the word "cohort" in its usual sense here, but
we're particularly interested in cohorts that have properties relevant
to tract placement. It is such cohorts that the pools will try to
organize using the services of the locus manager. Typical properties
would be trace differentiability or (en masse) death-time
predictability. Typical cohorts would be instances of a
non-generational pool, or generations of a collection strategy.
_`.def.trace.differentiability`: Objects (and hence tracts) that are
collected, may or may not have "trace differentiability" from each
other, depending on their placement in the different zones. Objects
(or pointers to them) can also have trace differentiability (or not)
from non-pointers in ambiguous references; in practice, we will be
worried about low integers, that may appear to be in zones 0 or -1.
Requirements
------------
_`.req.cohort`: Tract allocations must specify the cohort they
allocate in. These kind of cohorts will be called loci, and they will
have such attributes as are implied by the other requirements.
Critical.
_`.req.counter.objects`: As a counter-requirement, pools are expected
to manage objects. Objects the size of a tract allocation request
(segment-sized) are exceptional. Critical.
_`.req.counter.objects.just`: This means the locus manager is not
meant to solve the problems of allocating large objects, and it isn't
required to know what goes on in pools.
_`.req.contiguity`: Must support a high level of contiguity within
cohorts when requested. This means minimizing the number of times a
cohort is made aware of discontiguity. Essential (as we've effectively
renegotiated this in SW, down to a vague hope that certain critical
cohorts are not too badly fragmented). _`.req.contiguity.just`: TSBA.
_`.req.contiguity.specific`: It should be possible to request another
allocation next to a specific tract on either side (or an extension in
that direction, as the case may be). Such a request can fail, if
there's no space there. Nice. It would also be nice to have one for
"next to the largest free block".
_`.req.differentiable`: Must support the trace differentiability of
segments that may be condemned separately. Due to the limited number
of zones, it must be possible to place several cohorts into the same
zone. Essential.
_`.req.differentiable.integer`: It must be possible to place
collectable allocations so that they are trace-differentiable from
small integers. Essential.
_`.req.disjoint`: Must support the disjointness of pages that have
different VM properties (such as mutable/immutable,
read-only/read-write, and different lifetimes). Optional.
.. note::
I expect the implementation will simply work at page or larger
granularity, so the problem will not arise, but Tucker insisted on
stating this as a requirement. Pekka P. Pirinen, 1998-10-28.
_`.req.low-memory`: The architecture of the locus manager must not
prevent the design of efficient applications that often use all
available memory. Critical. _`.req.low-memory.expl`: This basically
says it must be designed to perform well in low-memory conditions, but
that there can be configurations where it doesn't do as well, as long
as this is documented for the application programmer. Note that it
doesn't say all applications are efficient, only that if you manage to
design an otherwise efficient application, the locus manager will not
sink it.
_`.req.address`: Must conserve address space in VM arenas to a
reasonable extent. Critical.
_`.req.inter-pool`: Must support the association of sets of tracts in
different pools into one cohort. Nice.
_`.req.ep-style`: Must support the existing EP-style of allocation
whereby allocation is from one end of address space either upwards or
downwards (or a close approximation thereto with the same behavior).
_`.req.ep-style.just`: We cannot risk disrupting a policy with
well-known properties when this technology is introduced.
_`.req.attributes`: There should be a way to inform the locus manager
about various attributes of cohorts that might be useful for
placement: deathtime, expected total size, and so on. Optional. It's a
given that the cohorts must then have these attributes, within the
limits set in the contract of the appropriate interface.
_`.req.attributes.action`: The locus manager should use the attributes
to guide its placement decisions. Nice.
_`.req.blacklisting`: There should be a way of maintaining at least
one blacklist for pages (or some other small unit), that can
not/should not be allocated to collectable pools. Optional.
.. note::
How to do blacklist breaking for ambiguous refs?
_`.req.hysteresis`: There should be a way to indicate which cohorts
fluctuate in size and by how much, to guide the arena hysteresis to
hold on to suitable pages. Optional.
Analysis
--------
_`.anal.sw`: Almost any placement policy would be an improvement on
the current SW one.
_`.anal.cause-and-effect`: The locus manager doesn't usually need to
know *why* things need to be differentiable, disjoint, contiguous, and
so on. Abstracting the reason away from the interface makes it more
generic, more likely to have serendipitous new uses. Attributes
described by a quantity (deathtime, size, etc.) are an exception to
this, because we can't devise a common measure.
_`.anal.stable`: The strategy must be stable: it must avoid repeated
recomputation, especially the kind that switches between alternatives
with a short period (repeated "bites" out the same region or
flip-flopping between two regions).
_`.anal.fragmentation`: There's some call to avoid fragmentation in
cohorts that don't need strict contiguity, but this is not a separate
requirement, since fragmentation is a global condition, and can only
be ameliorated if there's a global strategy that clumps allocations
together.
_`.anal.deathtime`: Cohorts with good death-time clumping of their
objects could use some locality of tract allocation, because it
increases the chances of creating large holes in the address space
(for other allocation to use). OTOH. many cohorts will not do multiple
frees in short succession, or at least cannot reasonably be predicted
to do so. This locality is not contiguity, nor is it low
fragmentation, it's just the requirement to place the new tracts next
to the tract where the last object was allocated in the cohort. Note
that the placement of objects is under the control of the pool, and
the locus manager will not know it, therefore this requirement should
be pursued by requesting allocation next to a particular tract (which
we already have a requirement for).
_`.anal.asymmetrical`: The strategy has to be asymmetrical with
respect to cohorts growing and shrinking. The reason of this asymmetry
is that it can choose where to grow, but it cannot choose where to
shrink (except in a small way by growing with good locality).
Interface
---------
_`.interface.locus`: A cohort will typically reside on multiple tracts
(and the pools will avoid putting objects of other cohorts on them),
so there should be an interface to describe the properties of the
cohort, and associate each allocation request with the cohort. We
shall call such an object, created to represent a cohort, a locus (pl.
loci).
_`.interface.locus.pool`: Loci will usually be created by the pool
that uses it. Some of the locus attributes will be inherited from
client-specified pool attributes [this means there will be additional
pool attributes].
_`.interface.detail`: This describes interface in overview; for
details, see implementation section and code, or user doc.
Loci
....
``Res LocusCreate(Locus *locusReturn, LocusAttrs attrs, ZoneGroup zg,
LocusAllocDesc adesc)``
_`.function.create`: A function to create a locus: ``adesc`` contains
the information about the allocation sequences in the locus, ``zg`` is
used for zone differentiability, and ``attrs`` encodes the following:
- _`.locus.contiguity`: A locus can be contiguous. This means
performing as required in `.req.contiguity`_, non-contiguous
allocations can be freely placed anywhere (but efficiency dictates
that similar allocations are placed close together and apart from
others).
- _`.locus.blacklist`: Allocations in the locus will avoid blacklisted
pages (for collectable segments).
- _`.locus.zero`: Allocations in the locus are zero-filled.
.. note::
Other attributes will be added, I'm sure.
_`.interface.zone-group`: The locus can be made a member of a zone
group. Passing ``ZoneGroupNONE`` means it's not a member of any group
(allocations will be placed without regard to zone, except to keep
them out of stripes likely to be needed for some group).
.. note::
I propose no mechanism for managing zone groups at this time,
since it's only used internally for one purpose. Pekka P. Pirinen,
2000-01-17.
_`.interface.size`: An allocation descriptor (``LocusAllocDesc``)
contains various descriptions of how the locus will develop over time
(inconsistent specifications are forbidden, of course):
- _`.interface.size.typical-alloc`: Size of a typical allocation in
this locus, in bytes. This will mainly affect the grouping of
non-contiguous loci.
- _`.interface.size.large-alloc`: Typical large allocation that the
manager should try to allow for (this allows some relief from
`.req.counter.objects`_), in bytes. This will mainly affect the size
of gaps that will be allotted adjoining this locus.
- _`.interface.size.direction`: Direction of growth: up/down/none.
Only useful if the locus is contiguous.
- _`.interface.size.lifetime`: Some measure of the lifetime of tracts
(not objects) in the cohort.
.. note::
Don't know the details yet, probably only useful for placing
similar cohorts next to each other, so the details don't
actually matter. Pekka P. Pirinen, 2000-01-17.
- _`.interface.size.deathtime`: Some measure of the deathtime of
tracts (not objects) in the cohort.
.. note::
Ditto. Pekka P. Pirinen, 2000-01-17.
_`.function.init`: ``LocusInit()`` is like ``LocusCreate()``, but
without the allocation. This is the usual interface, since most loci
are embedded in a pool or something.
_`.function.alloc`: ``ArenaAlloc()`` to take a locus argument.
``ArenaAllocHere()`` is like it, plus it takes a tract and a
specification to place the new allocation immediately above/below a
given tract; if that is not possible, it returns ``ResFAIL`` (this
will make it useful for reallocation functionality).
``ArenaSetTotalLoci(Arena arena, Size nLoci, Size nZoneGroups)``
_`.function.set-total`: A function to tell the arena the expected
number of (non-miscible client) loci, and of zone groups.
Peaks
.....
``mps_res_t mps_peak_create(mps_peak_t*, mps_arena_t)``
_`.function.peak.create`: A function to create a peak. A newly-created
peak is open, and will not be used to guide the strategy of the locus
manager.
``mps_res_t mps_peak_describe_pool(mps_peak_t, mps_pool_t, mps_size_desc_t)``
_`.function.peak.add`: A function to add a description of the state of
one pool into the peak. Calling this function again for the same peak and pool instance will replace
the earlier description.
_`.function.peak.add.size`: The size descriptor contains a total size
in bytes or percent of arena size.
.. note::
Is this right? Pekka P. Pirinen, 2000-01-17.
_`.function.peak.add.remove`: Specifying a ``NULL`` size will remove
the pool from the peak. The client is not allowed to destroy a pool
that is mentioned in any peak; it must be first removed from the peak,
or the peak must be destroyed. This is to ensure that the client
adjusts the peaks in a manner that makes sense to the application; the
locus manager can't know how to do that.
``mps_res_t mps_peak_close(mps_peak_t)``
_`.function.peak.close`: A function to indicate that all the
significant pools have been added to the peak, and it can now be used
to guide the locus manager. For any pool not described in the peak,
the locus manager will take its current size at any given moment as
the best prediction of its size at the peak.
_`.function.peak.close.after`: It is legal to add more descriptions to
the peak after closing, but this will reopen the peak, and it will
have to be closed before the locus manager will use it again. The
locus manager uses the previous closed state of the peak, while this
is going on.
``void mps_peak_destroy(mps_peak_t)``
_`.function.peak.destroy`: A function to destroy a peak.
_`.interface.ep-style`: This satisfies `.req.ep-style`_ by allowing SW
to specify zero size for most pools (which will cause them to be place
next to other loci with the same growth direction).
.. note::
Not sure this is good enough, but we'll try it first. Pekka P.
Pirinen, 2000-01-17.
Architecture
------------
Data objects
............
_`.arch.locus`: To represent the cohorts, we have locus objects.
Usually a locus is embedded in a pool instance, but generations are
separate loci.
_`.arch.locus.attr`: contiguity, blacklist, zg, current region, @@@@
_`.arch.locus.attr.exceptional`: The client can define a typical large
allocation for the locus. Requests substantially larger than that are
deemed exceptional.
_`.arch.zone-group`: To satisfy `.req.condemn`_, we offer zone groups.
Each locus can be a member of a zone group, and the locus manager will
attempt to place allocations in this locus in different zones from all
the other zone groups. A zone-group is represented as @@@@.
_`.arch.page-table`: A page table is maintained by the arena, as usual
to track association between tracts, pools and segments, and mapping
status for VM arenas.
_`.arch.region`: All of the address space is divided into disjoint
regions, represented by region objects. These objects store their
current limits, and high and low watermarks of currently allocated
tracts (we hope there's usually a gap of empty space between regions).
The limits are actually quite porous and flexible.
_`.arch.region.assoc`: Each region is associated with one contiguous
locus or any number of non-contiguous loci (or none). We call the
first kind of region "contiguous". _`.arch.locus.assoc`: Each locus
remembers all regions where it has tracts currently, excepting the
badly-placed allocations (see below). It is not our intention that any
locus would have very many, or that loci that share regions would have
any reason to stop doing do.
_`.arch.region.more`: Various quantities used by the placement
computation are also stored in the regions and the loci. Regions are
created (and destroyed) by the placement recomputation. Regions are
located in stripes (if it's a zoned region), but they can extend into
neighboring stripes if an exceptionally large tract allocation is
requested (to allow for large objects).
_`.arch.chunk`: Arenas may allocate more address space in additional
chunks, which may be disjoint from the existing chunks. Inter-chunk
space will be represented by dummy regions. There are also sentinel
regions at both ends of the address space.
Overview of strategy
....................
_`.arch.strategy.delay`: The general strategy is to delay placement
decisions until they have to be made, but no later.
_`.arch.strategy.delay.until`: Hence, the locus manager only makes
placement decisions when an allocation is requested (frees and other
operations might set a flag to cause the next allocation to redecide).
This also allows the client to change the peak and pool configuration
in complicated ways without causing a lot of recomputation, by doing
all the changes without allocating in the middle (unless the control
pool needs more space because of the changes).
_`.arch.strategy.normal`: While we want the placement to be
sophisticated, we do not believe it is worth the effort to consider
all the data at each allocation. Hence, allocations are usually just
placed in one of the regions used previously (see `.arch.alloc`_)
without reconsidering the issues.
_`.arch.strategy.normal.limit`: However, the manager sets
precautionary limits on the regions to ensure that the placement
decisions are revisited when an irrevocable placement is about to be
made.
_`.arch.strategy.create`: The manager doesn't create new regions until
they are needed for allocation (but it might compute where they could
be placed to accommodate a peak).
Allocation
..........
_`.arch.alloc`: Normally, each allocation to a locus is placed in its
current region. New regions are only sought when necessary to fulfill
an allocation request or when there is reason to think the situation
has changed significantly (see `.arch.significant`_).
_`.arch.alloc.same`: An allocation is first attempted next to the
previous allocation in the same locus, respecting growth direction. If
that is not possible, a good place in the current region is sought.
_`.arch.alloc.same.hole`: At the moment, for finding a good place
within a region, we just use the current algorithm, limited to the
region. In future, the placement within regions will be more clever.
_`.arch.alloc.extend`: If there's no adequate hole in the current
region and the request is not exceptional, the neighboring regions are
examined to see if the region could be extended at one border. (This
will basically only be done if the neighbor has shrunk since the last
placement recomputation, because the limit was set on sophisticated
criteria, and should not be changed without justification.)
_`.arch.alloc.extend.here`: When an allocation is requested next to a
specific tract (``ArenaAllocHere()``), we try to extend a little
harder (at least for ``change_size``, perhaps not for locality).
_`.arch.alloc.other`: If no way can be found to allocate in the
current region, other regions used for this locus are considered in
the same way, to see if space can be found there. [Or probably look at
other regions before trying to extend anything?]
_`.arch.alloc.recompute`: When no region of this locus has enough
space for the request, or when otherwise required, region placement is
recomputed to find a new region for the request (which might be the
same region, after extension).
_`.arch.alloc.current`: This region where the allocation was placed
then becomes the current region for this locus, except when the
request was exceptional, or when the region chosen was "bad" (see
@@@@).
_`.arch.significant`: Significant changes to the parameters affecting
placement are deemed to have happened at certain client calls and when
the total allocation has changed substantially since the last
recomputation. Such conditions set a flag that causes the next
allocation to recompute even if its current region is not full
(possibly second-guess the decision to recompute after some
investigation of the current state?).
Deallocation
............
_`.arch.free`: Deallocation simply updates the counters in the region
and the locus. For some loci, it will make the region of the
deallocation the current region. _`.arch.free.remove`: If a region
becomes entirely empty, it is deleted (and the neighbors limits might
be adjusted).
.. note::
This is quite tricky to get right.
Region placement recomputation
..............................
_`.arch.gap`: When doing placement computations, we view the arena as
a sequence of alternating region cores and gaps (which can be small,
even zero-sized). Initially, we'll take the core of a region to be the
area between the high and low watermark, but in the future we might be
more flexible about that.
.. note::
Edge determination is actually a worthwhile direction to explore.
_`.arch.reach`: The gap between two cores could potentially end up
being allocated to either region, if they grow in that direction, or
one or neither, if they don't. The set of states that the region
assignment could reach by assigning the gaps to their neighbors is
called the reach of the current configuration.
_`.arch.placement.object`: The object of the recomputation is to find
a configuration of regions that is not too far from the current
configuration and that keeps all the peaks inside its reach; if that
is not possible, keep the nearest ones in the reach and then minimize
the total distance from the rest.
_`.arch.placement.hypothetical`: The configurations that are
considered will include hypothetical placements for new regions for
loci that cannot fit in their existing regions at the peak. This is
necessary to avoid choosing a bad alternative.
_`.arch.placement.interesting`: The computation will only consider new
regions of loci that are deemed interesting, that is, far from their
peak state. This will reduce the computational burden and avoid
jittering near a peak.
.. note::
Details missing.
Implementation
--------------
[missing]
Notes
-----
_`.idea.change`: Even after the first segment, be prepared to change
your mind, if by the second segment a lot of new loci have been
created.
_`.distance`: If the current state is far from a peak, there's time to
reassign regions and for free space to appear (in fact, under the
steady arena assumption, enough free space *will* appear).
_`.clear-pool`: Need to have a function to deallocate all objects in a
pool, so that ``PoolDestroy()`` won't have to be used for that
purpose.
Document History
----------------
- 1998-02-27 Gavin Matthews. Incomplete design. Originally written as
part of change.dylan.box-turtle.170569. Much developed since.
- 1998-10-28 Pekka P. Pirinen. Wrote the real requirements after some
discussion.
- 1998-12-15 Pekka P. Pirinen. Deleted Gavin's design and wrote a new one.
- 2002-06-07 RB_ Converted from MMInfo database design document.
- 2007-04-24 Richard Kistruck. Added Guide: Manage arena address
space, why, discover layout.
- 2013-05-23 GDR_ Converted to reStructuredText.
.. _RB: http://www.ravenbrook.com/consultants/rb/
.. _GDR: http://www.ravenbrook.com/consultants/gdr/
Copyright and License
---------------------
Copyright © 2013 Ravenbrook Limited. All rights reserved.
<http://www.ravenbrook.com/>. This is an open source license. Contact
Ravenbrook for commercial licensing options.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. Redistributions in any form must be accompanied by information on how
to obtain complete source code for this software and any
accompanying software that uses this software. The source code must
either be included in the distribution or be available for no more than
the cost of distribution plus a nominal fee, and must be freely
redistributable under reasonable conditions. For an executable file,
complete source code means the source code for all modules it contains.
It does not include source code for modules or files that typically
accompany the major components of the operating system on which the
executable file runs.
**This software is provided by the copyright holders and contributors
"as is" and any express or implied warranties, including, but not
limited to, the implied warranties of merchantability, fitness for a
particular purpose, or non-infringement, are disclaimed. In no event
shall the copyright holders and contributors be liable for any direct,
indirect, incidental, special, exemplary, or consequential damages
(including, but not limited to, procurement of substitute goods or
services; loss of use, data, or profits; or business interruption)
however caused and on any theory of liability, whether in contract,
strict liability, or tort (including negligence or otherwise) arising in
any way out of the use of this software, even if advised of the
possibility of such damage.**

645
mps/design/locus/index.html
View file

@ -1,645 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<title>The MPS Locus Manager</title>
</head>
<body bgcolor="#FFFFFF" text="#000000" link="#000099" vlink="#660066" alink="#FF0000">
<div align="center">
<p>
<a href="/">Ravenbrook</a> /
<a href="/project/">Projects</a> /
<a href="/project/mps/">Memory Pool System</a> /
<a href="/project/mps/master/">Master Product Sources</a> /
<a href="/project/mps/master/design/">Design Documents</a>
</p>
<p><i><a href="/project/mps/">Memory Pool System Project</a></i></p>
<hr />
<h1>The MPS Locus Manager</h1>
</div>
<p>This document contains a <a href="#guide">guide</a> to the MPS locus manager, followed by the historical <a href="#initial-design">initial design</a>. References, History, Copyright and License are <a href="#section-A">at the end</a>.</p>
<hr />
<h1> <a id="guide">Guide</a> </h1>
<p> Readership: any MPS developer. Not confidential. </p>
<p>The MPS manages three main resources:</p>
<ul>
<li>storage;</li>
<li>address-space;</li>
<li>time.</li>
</ul>
<h2> Management of arena address-space </h2>
<p>The locus manager manages address-space at the arena level.</p>
<p>[Tucker was right: <a href="http://info.ravenbrook.com/project/mps/mail/1998/11/02/14-25/0.txt">mail.ptw.1998-11-02.14-25</a>. RHSK 2007-04-24].</p>
<p>When a pool wants some address-space, it expresses some preferences to the locus manager. The locus manager and the arena (working together) try to honour these preferences, and decide what address-space the pool gets.</p>
<p>Preferences are expressed by the SegPref argument to SegAlloc().</p>
<p>Note that, when they call SegAlloc(), pools are asking for address-space and writeable storage simultaneously, in a single call. There is currently no way for pools to 'reserve' address space without requesting storage.</p>
<h2> Why is it important to manage address-space? </h2>
<h3> Trace differentiability </h3>
<p>Carefully chosen addresses are used by reference tracing systems (ie. automatic pools), to:</p>
<ul>
<li> categorise objects into clumps; </li>
<li> summarise and cheaply find references between clumps. </li>
</ul>
<p>Different clumps will become worth collecting at different times (the classic example, of course, is generations in a generational collector). For these partial collections to be efficient, it must be cheap to keep these clumps differentiable, cheap to condemn (Whiten) a particular clump, and cheap to find a good conservative approximation to all inward references to a clump (both initially to construct the Grey set, and to make scanning the Grey set efficient).</p>
<p>This is what the MPS zone mechanism is all about.</p>
<p>The locus manager manages the mapping from clumps to zones.</p>
<p>To specify a clump, pools use the "SegPrefGen" tag to SegPrefExpress. [The name is misleading: generations are only one sort of clump. RHSK 2007-04-24]</p>
<h3> Prevent address-space fragmentation (within arena) </h3>
<p>Address-space is not infinite.</p>
<p>In some use-cases, the MPS is required to remain efficient when using very nearly all available address-space and storage. (For example, with the client-arena class, where the only address-space available is that of the storage available).</p>
<p>Even with the VM arena class, typical storage sizes (in the year 2007) can make 32-bit address-space constrained: the problem may have Order(4GB) size, which leaves little 'spare' address-space.</p>
<p>Address-space fragmentation incurs failure when there is no way to allocate a big block of address-space. The big block may be requested via the MPS (by the client), or by something else in the same process, such as a 3rd-party graphics library, image library, etc.</p>
<p>Address-space fragmentation incurs cost when:</p>
<ul>
<li> desired large-block requests (such as for buffering) are denied, causing them to be re-requested as a smaller block, or as several smaller blocks; </li>
<li> possible operating-system costs in maintaining a fragmented mapping? </li>
</ul>
<h3> Prevent storage fragmentation (within tracts and segments) </h3>
<p>Storage is not infinite: it is allocated in multiples of a fixed-size tract. Small lonely objects, each retaining a whole tract, cause storage fragmentation. </p>
<p>Non-moving pools manage this fragmentation with placement strategies that use:</p>
<ul>
<li> co-located death (in space and time); </li>
<li> segment merging and splitting. </li>
</ul>
<p>These pool-level strategies always care about contiguity of object storage. They also often care about the <em>ordering</em> of addresses, because pool code uses an address-ordered search when choosing where to place a new object. For these two reasons, the address chosen (by the locus manager and arena) for new tracts is important.</p>
<p>Certain specialised pools, and/or some client programs that use them, have carefully tuned segment sizes, positioning, and search order. Be careful: seemingly inconsequential changes can catastrophically break this tuning.</p>
<p>Pools can specify a preference for High and Low ends of address-space, which implies a search-order. Pools could also specify clumping, using either SegPrefGen or SegPrefZoneSet.</p>
<h2> Discovering the layout </h2>
<p>The locus manager is not given advance notice of how much address-space will be required with what preferences. Instead, the locus manager starts with an empty 'layout', and adapts it as more requests come in over time. It is attempting to 'discover' a suitable layout by successive refinement. This is ambitious.</p>
<hr />
<h1> <a id="initial-design">Initial Design</a> </h1>
<pre>
THE DESIGN FOR THE LOCUS MANAGER
design.mps.locus
incomplete design
gavinm 1998-02-27
INTRODUCTION
<a id="intro" name="intro">.intro</a>: The locus manager coordinates between the pools and takes the burden of
having to be clever about tract/group placement away from the pools, preserving
trace differentiability and contiguity where appropriate.
<a id="source" name="source">.source</a>: mail.gavinm.1998-02-05.17-52(0), mail.ptw.1998-02-05.19-53(0),
mail.pekka.1998-02-09.13-58(0), and mail.gavinm.1998-02-09.14-05(0).
Document History
<a id="hist.0" name="hist.0">.hist.0</a>: Originally written as part of change.dylan.box-turtle.170569. Much
developed since. gavinm 1998-02-27
<a id="hist.1" name="hist.1">.hist.1</a>: Pekka wrote the real requirements after some discussion. pekka
1998-10-28
<a id="hist.2" name="hist.2">.hist.2</a>: Pekka deleted Gavin's design and wrote a new one. pekka 1998-12-15
DEFINITIONS
<a id="note.cohort" name="note.cohort">.note.cohort</a>: We use the word "cohort" in its usual sense here, but we're
particularly interested in cohorts that have properties relevant to tract
placement. It is such cohorts that the pools will try to organize using the
services of the locus manager. Typical properties would be trace
differentiability or (en masse) death-time
predictability. Typical cohorts would be instances of a non-generational pool,
or generations of a collection strategy.
<a id="def.trace.differentiability" name="def.trace.differentiability">.def.trace.differentiability</a>: Objects (and hence tracts) that are collected,
may or may not have "trace differentiability" from each other, depending on
their placement in the different zones. Objects (or pointers to them) can also
have trace differentiability (or not) from non-pointers in ambiguous
references; in practice, we will be worried about low integers, that may appear
to be in zones 0 or -1.
REQUIREMENTS
<a id="req.cohort" name="req.cohort">.req.cohort</a>: Tract allocations must specify the cohort they allocate in. These
kind of cohorts will be called loci, and they will have such attributes as are
implied by the other requirements. Critical.
<a id="req.counter.objects" name="req.counter.objects">.req.counter.objects</a>: As a counter-requirement, pools are expected to manage
objects. Objects the size of a tract allocation request (segment-sized) are
exceptional. Critical. <a id="req.counter.objects.just" name="req.counter.objects.just">.req.counter.objects.just</a>: This means the locus
manager is not meant to solve the problems of allocating large objects, and it
isn't required to know what goes on in pools.
<a id="req.contiguity" name="req.contiguity">.req.contiguity</a>: Must support a high level of contiguity within cohorts when
requested. This means minimizing the number of times a cohort is made aware of
discontiguity. Essential (as we've effectively renegotiated this in SW, down
to a vague hope that certain critical cohorts are not too badly fragmented).
<a id="req.contiguity.just" name="req.contiguity.just">.req.contiguity.just</a>: TSBA.
<a id="req.contiguity.specific" name="req.contiguity.specific">.req.contiguity.specific</a>: It should be possible to request another allocation
next to a specific tract on either side (or an extension in that direction, as
the case may be). Such a request can fail, if there's no space there. Nice.
[IWBN to have one for "next to the largest free block".]
<a id="req.differentiable" name="req.differentiable">.req.differentiable</a>: Must support the trace differentiability of segments that
may be condemned separately. Due to the limited number of zones, it must be
possible to place several cohorts into the same zone. Essential.
<a id="req.differentiable.integer" name="req.differentiable.integer">.req.differentiable.integer</a>: It must be possible to place collectable
allocations so that they are trace-differentiable from small integers.
Essential.
<a id="req.disjoint" name="req.disjoint">.req.disjoint</a>: Must support the disjointness of pages that have different VM
properties (such as mutable/immutable, read-only/read-write, and different
lifetimes). Optional. [I expect the implementation will simply work at page
or larger granularity, so the problem will not arise, but Tucker insisted on
stating this as a requirement. pekka 1998-10-28]
<a id="req.low-memory" name="req.low-memory">.req.low-memory</a>: The architecture of the locus manager must not prevent the
design of efficient applications that often use all available memory.
Critical. <a id="req.low-memory.expl" name="req.low-memory.expl">.req.low-memory.expl</a>: This basically says it must be designed to
perform well in low-memory conditions, but that there can be configurations
where it doesn't do as well, as long as this is documented for the application
programmer. Note that it doesn't say all applications are efficient, only that
if you manage to design an otherwise efficient application, the locus manager
will not sink it.
<a id="req.address" name="req.address">.req.address</a>: Must conserve address space in VM arenas to a reasonable extent.
Critical.
<a id="req.inter-pool" name="req.inter-pool">.req.inter-pool</a>: Must support the association of sets of tracts in different
pools into one cohort. Nice.
<a id="req.ep-style" name="req.ep-style">.req.ep-style</a>: Must support the existing EP-style of allocation whereby
allocation is from one end of address space either upwards or downwards (or a
close approximation thereto with the same behavior). <a id="req.ep-style.just" name="req.ep-style.just">.req.ep-style.just</a>: We
cannot risk disrupting a policy with well-known properties when this technology
is introduced.
<a id="req.attributes" name="req.attributes">.req.attributes</a>: There should be a way to inform the locus manager about
various attributes of cohorts that might be useful for placement: deathtime,
expected total size, [more in the future]. Optional. [It's a given that the
cohorts must then have these attributes, within the limits set in the contract
of the appropriate interface.] <a id="req.attributes.action" name="req.attributes.action">.req.attributes.action</a>: The locus manager
should use the attributes to guide its placement decisions. Nice.
<a id="req.blacklisting" name="req.blacklisting">.req.blacklisting</a>: There should be a way of maintaining at least one blacklist
for pages (or some other small unit), that can not/should not be allocated to
collectable pools. [How to do blacklist breaking for ambiguous refs?]
Optional.
<a id="req.hysteresis" name="req.hysteresis">.req.hysteresis</a>: There should be a way to indicate which cohorts fluctuate in
size and by how much, to guide the arena hysteresis to hold on to suitable
pages. Optional.
ANALYSIS
<a id="anal.sw" name="anal.sw">.anal.sw</a>: Almost any placement policy would be an improvement on the current SW
one.
<a id="anal.cause-and-effect" name="anal.cause-and-effect">.anal.cause-and-effect</a>: The locus manager doesn't usually need to know _why_
things need to be differentiable, disjoint, contiguous, etc. Abstracting the
reason away from the interface makes it more generic, more likely to have
serendipitous new uses. Attributes described by a quantity (deathtime, size,
etc.) are an exception to this, because we can't devise a common measure.
<a id="anal.stable" name="anal.stable">.anal.stable</a>: The strategy must be stable: it must avoid repeated
recomputation, especially the kind that switches between alternatives with a
short period (repeated "bites" out the same region or flip-flopping between two
regions).
<a id="anal.fragmentation" name="anal.fragmentation">.anal.fragmentation</a>: There's some call to avoid fragmentation in cohorts that
don't need strict contiguity, but this is not a separate requirement, since
fragmentation is a global condition, and can only be ameliorated if there's a
global strategy that clumps allocations together. <a id="anal.deathtime" name="anal.deathtime">.anal.deathtime</a>: Cohorts
with good death-time clumping of their objects could use some locality of tract
allocation, because it increases the chances of creating large holes in the
address space (for other allocation to use). OTOH. many cohorts will not do
multiple frees in short succession, or at least cannot reasonably be predicted
to do so. This locality is not contiguity, nor is it low fragmentation, it's
just the requirement to place the new tracts next to the tract where the last
object was allocated in the cohort. Note that the placement of objects is
under the control of the pool, and the locus manager will not know it,
therefore this requirement should be pursued by requesting allocation next to a
particular tract (which we already have a requirement for).
<a id="anal.asymmetrical" name="anal.asymmetrical">.anal.asymmetrical</a>: The strategy has to be asymmetrical with respect to cohorts
growing and shrinking. The reason of this asymmetry is that it can choose
where to grow, but it cannot choose where to shrink (except in a small way by
growing with good locality).
INTERFACE
<a id="interface.locus" name="interface.locus">.interface.locus</a>: A cohort will typically reside on multiple tracts (and the
pools will avoid putting objects of other cohorts on them), so there should be
an interface to describe the properties of the cohort, and associate each
allocation request with the cohort. We shall call such an object, created to
represent a cohort, a locus (pl. loci).
<a id="interface.locus.pool" name="interface.locus.pool">.interface.locus.pool</a>: Loci will usually be created by the pool that uses it.
Some of the locus attributes will be inherited from client-specified pool
attributes [this means there will be additional pool attributes].
<a id="interface.detail" name="interface.detail">.interface.detail</a>: This describes interface in overview; for details, see
implementation section and code, or user doc.
Loci
<a id="function.create" name="function.create">.function.create</a>: A function to create a locus:
Res LocusCreate(Locus *locusReturn, LocusAttrs attrs, ZoneGroup zg,
LocusAllocDesc adesc)
where adesc contains the information about the allocation sequences in the
locus, zg is used for zone differentiability, and attrs encodes the following:
<a id="locus.contiguity" name="locus.contiguity">.locus.contiguity</a>: A locus can be contiguous. This means performing as
required in .req.contiguity, non-contiguous allocations can be freely placed
anywhere (but efficiency dictates that similar allocations are placed close
together and apart from others).
<a id="locus.blacklist" name="locus.blacklist">.locus.blacklist</a>: Allocations in the locus will avoid blacklisted pages (for
collectable segments).
<a id="locus.zero" name="locus.zero">.locus.zero</a>: Allocations in the locus are zero-filled.
[Other attributes will be added, I'm sure.]
<a id="interface.zone-group" name="interface.zone-group">.interface.zone-group</a>: The locus can be made a member of a zone group. Passing
ZoneGroupNONE means it's not a member of any group (allocations will be placed
without regard to zone, except to keep them out of stripes likely to be needed
for some group). [I propose no mechanism for managing zone groups at this
time, since it's only used internally for one purpose. pekka 2000-01-17]
<a id="interface.size" name="interface.size">.interface.size</a>: An allocation descriptor (LocusAllocDesc) contains various
descriptions of how the locus will develop over time (inconsistent
specifications are forbidden, of course):
<a id="interface.size.typical-alloc" name="interface.size.typical-alloc">.interface.size.typical-alloc</a>: Size of a typical allocation in this locus, in
bytes. This will mainly affect the grouping of non-contiguous loci.
<a id="interface.size.large-alloc" name="interface.size.large-alloc">.interface.size.large-alloc</a>: Typical large allocation that the manager should
try to allow for (this allows some relief from .req.counter.objects), in
bytes. This will mainly affect the size of gaps that will be allotted
adjoining this locus.
<a id="interface.size.direction" name="interface.size.direction">.interface.size.direction</a>: Direction of growth: up/down/none. Only useful if
the locus is contiguous.
<a id="interface.size.lifetime" name="interface.size.lifetime">.interface.size.lifetime</a>: Some measure of the lifetime of tracts (not
objects) in the cohort. [Don't know the details yet, probably only useful for
placing similar cohorts next to each other, so the details don't actually
matter. pekka 2000-01-17]
<a id="interface.size.deathtime" name="interface.size.deathtime">.interface.size.deathtime</a>: Some measure of the deathtime of tracts (not
objects) in the cohort. [Ditto. pekka 2000-01-17]
<a id="function.init" name="function.init">.function.init</a>: LocusInit is like LocusCreate, but without the allocation.
This is the usual i/f, since most loci are embedded in a pool or something.
<a id="function.alloc" name="function.alloc">.function.alloc</a>: ArenaAlloc to take a locus arg. ArenaAllocHere is like it,
plus it takes a tract and a specification to place the new allocation
immediately above/below a given tract; if that is not possible, it returns
ResFAIL (this will make it useful for realloc functionality).
<a id="function.set-total" name="function.set-total">.function.set-total</a>: A function to tell the arena the expected number of
(non-miscible client) loci, and of zone groups:
ArenaSetTotalLoci(Arena arena, Size nLoci, Size nZoneGroups)
Peaks
<a id="function.peak.create" name="function.peak.create">.function.peak.create</a>: A function to create a peak:
mps_res_t mps_peak_create(mps_peak_t*, mps_arena_t)
A newly-created peak is open, and will not be used to guide the strategy of the
locus manager.
<a id="function.peak.add" name="function.peak.add">.function.peak.add</a>: A function to add a description of the state of one pool
into the peak:
mps_res_t mps_peak_describe_pool(mps_peak_t, mps_pool_t, mps_size_desc_t)
Calling this function again for the same peak and pool instance will replace
the earlier description. <a id="function.peak.add.size" name="function.peak.add.size">.function.peak.add.size</a>: The size descriptor contains
a total size in bytes or % of arena size [@@@@is this right?].
<a id="function.peak.add.remove" name="function.peak.add.remove">.function.peak.add.remove</a>: Specifying a NULL size will remove the pool from the
peak. The client is not allowed to destroy a pool that is mentioned in any
peak; it must be first removed from the peak, or the peak must be destroyed.
This is to ensure that the client adjusts the peaks in a manner that makes
sense to the application; the locus manager can't know how to do that.
<a id="function.peak.close" name="function.peak.close">.function.peak.close</a>: A function to indicate that all the significant pools
have been added to the peak, and it can now be used to guide the locus manager:
mps_res_t mps_peak_close(mps_peak_t)
For any pool not described in the peak, the locus manager will take its current
size at any given moment as the best prediction of its size at the peak.
<a id="function.peak.close.after" name="function.peak.close.after">.function.peak.close.after</a>: It is legal to add more descriptions to the peak
after closing, but this will reopen the peak, and it will have to be closed
before the locus manager will use it again. The locus manager uses the
previous closed state of the peak, while this is going on.
<a id="function.peak.destroy" name="function.peak.destroy">.function.peak.destroy</a>: A function to destroy a peak:
void mps_peak_destroy(mps_peak_t)
<a id="interface.ep-style" name="interface.ep-style">.interface.ep-style</a>: This satisfies .req.ep-style by allowing SW to specify
zero size for most pools (which will cause them to be place next to other loci
with the same growth direction). [Not sure this is good enough, but we'll try
it first. pekka 2000-01-17]
ARCHITECTURE
Data Objects
<a id="arch.locus" name="arch.locus">.arch.locus</a>: To represent the cohorts, we have locus objects. Usually a locus
is embedded in a pool instance, but generations are separate loci.
<a id="arch.locus.attr" name="arch.locus.attr">.arch.locus.attr</a>: contiguity, blacklist, zg, current region, @@@@
<a id="arch.locus.attr.exceptional" name="arch.locus.attr.exceptional">.arch.locus.attr.exceptional</a>: The client can define a typical large allocation
for the locus. Requests substantially larger than that are deemed exceptional.
<a id="arch.zone-group" name="arch.zone-group">.arch.zone-group</a>: To satisfy .req.condemn, we offer zone groups. Each locus
can be a member of a zone group, and the locus manager will attempt to place
allocations in this locus in different zones from all the other zone groups. A
zone-group is represented as @@@@.
<a id="arch.page-table" name="arch.page-table">.arch.page-table</a>: A page table is maintained by the arena, as usual to track
association between tracts, pools and segments, and mapping status for VM
arenas.
<a id="arch.region" name="arch.region">.arch.region</a>: All of the address space is divided into disjoint regions,
represented by region objects. These objects store their current limits, and
high and low watermarks of currently allocated tracts (we hope there's usually
a gap of empty space between regions). The limits are actually quite porous
and flexible.
<a id="arch.region.assoc" name="arch.region.assoc">.arch.region.assoc</a>: Each region is associated with one contiguous locus or any
number of non-contiguous loci (or none). We call the first kind of region
"contiguous". <a id="arch.locus.assoc" name="arch.locus.assoc">.arch.locus.assoc</a>: Each locus remembers all regions where it has
tracts currently, excepting the badly-placed allocations (see below). It is
not our intention that any locus would have very many, or that loci that share
regions would have any reason to stop doing do.
<a id="arch.region.more" name="arch.region.more">.arch.region.more</a>: Various quantities used by the placement computation are
also stored in the regions and the loci. Regions are created (and destroyed)
by the placement recomputation. Regions are located in stripes (if it's a
zoned region), but they can extend into neighboring stripes if an exceptionally
large tract allocation is requested (to allow for large objects).
<a id="arch.chunk" name="arch.chunk">.arch.chunk</a>: Arenas may allocate more address space in additional chunks, which
may be disjoint from the existing chunks. Inter-chunk space will be
represented by dummy regions. There are also sentinel regions at both ends of
the address space.
Overview of Strategy
<a id="arch.strategy.delay" name="arch.strategy.delay">.arch.strategy.delay</a>: The general strategy is to delay placement decisions
until they have to be made, but no later.
<a id="arch.strategy.delay.until" name="arch.strategy.delay.until">.arch.strategy.delay.until</a>: Hence, the locus manager only makes placement
decisions when an allocation is requested (frees and other operations might set
a flag to cause the next allocation to redecide). This also allows the client
to change the peak and pool configuration in complicated ways without causing a
lot of recomputation, by doing all the changes without allocating in the middle
(unless the control pool needs more space because of the changes).
<a id="arch.strategy.normal" name="arch.strategy.normal">.arch.strategy.normal</a>: While we want the placement to be sophisticated, we do
not believe it is worth the effort to consider all the data at each
allocation. Hence, allocations are usually just placed in one of the regions
used previously (see .arch.alloc) without reconsidering the issues.
<a id="arch.strategy.normal.limit" name="arch.strategy.normal.limit">.arch.strategy.normal.limit</a>: However, the manager sets precautionary limits on
the regions to ensure that the placement decisions are revisited when an
irrevocable placement is about to be made.
<a id="arch.strategy.create" name="arch.strategy.create">.arch.strategy.create</a>: The manager doesn't create new regions until they are
needed for allocation (but it might compute where they could be placed to
accommodate a peak).
Allocation
<a id="arch.alloc" name="arch.alloc">.arch.alloc</a>: Normally, each allocation to a locus is placed in its current
region. New regions are only sought when necessary to fulfill an allocation
request or when there is reason to think the situation has changed
significantly (see .arch.significant).
<a id="arch.alloc.same" name="arch.alloc.same">.arch.alloc.same</a>: An allocation is first attempted next to the previous
allocation in the same locus, respecting growth direction. If that is not
possible, a good place in the current region is sought. <a id="arch.alloc.same.hole" name="arch.alloc.same.hole">.arch.alloc.same.hole</a>:
ATM, for finding a good place within a region, we just use the current
algorithm, limited to the region. In future, the placement within regions will
be more clever.
<a id="arch.alloc.extend" name="arch.alloc.extend">.arch.alloc.extend</a>: If there's no adequate hole in the current region and the
request is not exceptional, the neighboring regions are examined to see if the
region could be extended at one border. (This will basically only be done if
the neighbor has shrunk since the last placement recomputation, because the
limit was set on sophisticated criteria, and should not be changed without
justification.) <a id="arch.alloc.extend.here" name="arch.alloc.extend.here">.arch.alloc.extend.here</a>: When an allocation is requested next
to a specific tract (ArenaAllocHere), we try to extend a little harder [at
least for change_size, perhaps not for locality].
<a id="arch.alloc.other" name="arch.alloc.other">.arch.alloc.other</a>: If no way can be found to allocate in the current region,
other regions used for this locus are considered in the same way, to see if
space can be found there. [Or probably look at other regions before trying to
extend anything?]
<a id="arch.alloc.recompute" name="arch.alloc.recompute">.arch.alloc.recompute</a>: When no region of this locus has enough space for the
request, or when otherwise required, region placement is recomputed to find a
new region for the request (which might be the same region, after extension).
<a id="arch.alloc.current" name="arch.alloc.current">.arch.alloc.current</a>: This region where the allocation was placed then becomes
the current region for this locus, except when the request was exceptional, or
when the region chosen was "bad" (see @@@@).
<a id="arch.significant" name="arch.significant">.arch.significant</a>: Significant changes to the parameters affecting placement
are deemed to have happened at certain client calls and when the total
allocation has changed substantially since the last recomputation. Such
conditions set a flag that causes the next allocation to recompute even if its
current region is not full [possibly second-guess the decision to recompute
after some investigation of the current state?].
Deallocation
<a id="arch.free" name="arch.free">.arch.free</a>: Deallocation simply updates the counters in the region and the
locus. For some loci, it will make the region of the deallocation the current
region.
<a id="arch.free.remove" name="arch.free.remove">.arch.free.remove</a>: If a region becomes entirely empty, it is deleted (and the
neighbors limits might be adjusted [quite tricky to get right, this]).
Region Placement Recomputation
<a id="arch.gap" name="arch.gap">.arch.gap</a>: When doing placement computations, we view the arena as a sequence
of alternating region cores and gaps (which can be small, even zero-sized).
Initially, we'll take the core of a region to be the area between the high and
low watermark, but in the future we might be more flexible about that. [Edge
determination is actually a worthwhile direction to explore.]
<a id="arch.reach" name="arch.reach">.arch.reach</a>: The gap between two cores could potentially end up being allocated
to either region, if they grow in that direction, or one or neither, if they
don't. The set of states that the region assignment could reach by assigning
the gaps to their neighbors is called the reach of the current configuration.
<a id="arch.placement.object" name="arch.placement.object">.arch.placement.object</a>: The object of the recomputation is to find a
configuration of regions that is not too far from the current configuration and
that keeps all the peaks inside its reach; if that is not possible, keep the
nearest ones in the reach and then minimize the total distance from the rest.
<a id="arch.placement.hypothetical" name="arch.placement.hypothetical">.arch.placement.hypothetical</a>: The configurations that are considered will
include hypothetical placements for new regions for loci that cannot fit in
their existing regions at the peak. This is necessary to avoid choosing a bad
alternative.
<a id="arch.placement.interesting" name="arch.placement.interesting">.arch.placement.interesting</a>: The computation will only consider new regions of
loci that are deemed interesting, i.e., far from their peak state. This will
reduce the computational burden and avoid jittering near a peak.
[details missing]
IMPLEMENTATION
[missing]
NOTES
<a id="idea.change" name="idea.change">.idea.change</a>: Even after the first segment, be prepared to change your mind, if
by the second segment a lot of new loci have been created.
<a id="distance" name="distance">.distance</a>: If the current state is far from a peak, there's time to reassign
regions and for free space to appear (in fact, under the steady arena
assumption, enough free space _will_ appear).
<a id="clear-pool" name="clear-pool">.clear-pool</a>: Need to have a function to deallocate all objects in a pool, so
that PoolDestroy won't have to be used for that purpose.
</pre>
<hr />
<h2><a id="section-A" name="section-A">A. References</a></h2>
<!-- Template Entry
<table>
<tr valign="top">
<td>[<a id="ref-#REF#" name="ref-#REF#" href="#REF_URL#">#REF_NAME#</a>]</td>
<td>
"#REF_TITLE#";
#REF_AUTHOR#;
&lt;URL: <a href="#REF_URL#">#REF_URL#</a>&gt;;
#REF_DATE#.
</td>
</tr>
</table>
-->
<h2><a id="section-B" name="section-B">B. Document History</a></h2>
<table>
<tr valign="top">
<td>2002-06-07</td>
<td><a href="mailto:rb@ravenbrook.com">RB</a></td>
<td>Converted from MMInfo database design document.</td>
</tr>
<tr valign="top">
<td>2007-04-24</td>
<td><a href="mailto:rhsk@ravenbrook.com">RHSK</a></td>
<td>add Guide: Manage arena address-space, why, discover layout.</td>
</tr>
</table>
<h2><a id="section-C" name="section-C">C. Copyright and License</a></h2>
<p> This document is copyright &copy; 1995-2002, 2007 <a href="http://www.ravenbrook.com/">Ravenbrook Limited</a>. All rights reserved. This is an open source license. Contact Ravenbrook for commercial licensing options. </p>
<p> Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: </p>
<ol>
<li> Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. </li>
<li> Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. </li>
<li> Redistributions in any form must be accompanied by information on how to obtain complete source code for the this software and any accompanying software that uses this software. The source code must either be included in the distribution or be available for no more than the cost of distribution plus a nominal fee, and must be freely redistributable under reasonable conditions. For an executable file, complete source code means the source code for all modules it contains. It does not include source code for modules or files that typically accompany the major components of the operating system on which the executable file runs. </li>
</ol>
<p> <strong> This software is provided by the copyright holders and contributors "as is" and any express or implied warranties, including, but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement, are disclaimed. In no event shall the copyright holders and contributors be liable for any direct, indirect, incidental, special, exemplary, or consequential damages (including, but not limited to, procurement of substitute goods or services; loss of use, data, or profits; or business interruption) however caused and on any theory of liability, whether in contract, strict liability, or tort (including negligence or otherwise) arising in any way out of the use of this software, even if advised of the possibility of such damage. </strong> </p>
<hr />
<div align="center">
<p><code>$Id$</code></p>
<p>
<a href="/">Ravenbrook</a> /
<a href="/project/">Projects</a> /
<a href="/project/mps/">Memory Pool System</a> /
<a href="/project/mps/master/">Master Product Sources</a> /
<a href="/project/mps/master/design/">Design Documents</a>
</p>
</div>
</body>
</html>

26
mps/design/protan.txt
View file

@ -50,22 +50,18 @@ exists. It satisfies `.req.test`_ and `.req.rapid-port`_ in this way.
Functions
---------
``void ProtSetup(void)``
_`.fun.protsetup`: ``ProtSetup()`` does nothing as there is nothing to
do (under UNIX we might expect the protection module to install one or
more signal handlers at this pointer, but that is not appropriate for
the ANSI implementation). Of course, we can't have an empty function
body, so there is a ``NOOP;`` here.
_`.fun.protsetup`: Does nothing as there is nothing to do (under UNIX
we might expect the protection module to install one or more signal
handlers at this pointer, but that is not appropriate for the ANSI
implementation). Of course, we can't have an empty function body, so
there is a ``NOOP;`` here.
``void ProtSync(Arena arena)``
_`.fun.sync`: Called to ensure that the actual protection of each
segment (as determined by the OS) is in accordance with the segments's
pm field. In the ANSI implementation we have no way of changing the
protection of a segment, so instead we generate faults on all
protected segments in the assumption that that will remove the
protection on segments.
_`.fun.sync`: ``ProtSync()`` is called to ensure that the actual
protection of each segment (as determined by the OS) is in accordance
with the segments's pm field. In the ANSI implementation we have no
way of changing the protection of a segment, so instead we generate
faults on all protected segments in the assumption that that will
remove the protection on segments.
_`.fun.sync.how`: Continually loops over all the segments until it
finds that all segments have no protection.

6
mps/manual/source/design/alloc-frame.rst Normal file
View file

@ -0,0 +1,6 @@
.. index::
pair: allocation frames; design
.. _design-alloc-frame:
.. include:: ../../converted/alloc-frame.rst

6
mps/manual/source/design/diag.rst Normal file
View file

@ -0,0 +1,6 @@
.. index::
pair: diagnostic feedback; design
.. _design-diag:
.. include:: ../../converted/diag.rst

7
mps/manual/source/design/guide.impl.c.format.rst Normal file
View file

@ -0,0 +1,7 @@
.. index::
pair: C language; formatting guide
pair: C language formatting; guide
.. _design-guide.impl.c.format:
.. include:: ../../converted/guide.impl.c.format.rst

1
mps/manual/source/design/index.rst
View file

@ -9,6 +9,7 @@ Design
config
critical-path
guide.hex.trans
guide.impl.c.format
keyword-arguments
ring
sig

6
mps/manual/source/design/interface-c.rst Normal file
View file

@ -0,0 +1,6 @@
.. index::
pair: C interface; design
.. _design-interface-c:
.. include:: ../../converted/interface-c.rst

6
mps/manual/source/design/io.rst Normal file
View file

@ -0,0 +1,6 @@
.. index::
pair: I/O subsystem; design
.. _design-io:
.. include:: ../../converted/io.rst

6
mps/manual/source/design/locus.rst Normal file
View file

@ -0,0 +1,6 @@
.. index::
pair: locus manager; design
.. _design-locus:
.. include:: ../../converted/locus.rst

5
mps/manual/source/design/old.rst
View file

@ -14,6 +14,7 @@ Old design
.. toctree::
:numbered:
alloc-frame
arena
arenavm
bt
@ -22,10 +23,14 @@ Old design
check
class-interface
collection
diag
finalize
fix
interface-c
io
lib
lock
locus
message
message-gc
object-debug

2
mps/manual/tool/convert.py
View file

@ -18,7 +18,7 @@ TYPES = '''
def main():
mode = re.compile(r'\.\. mode: .*\n')
prefix = re.compile(r'^:Tag: ([a-z][a-z.0-9-]*[a-z0-9])$')
rst_tag = re.compile(r'^:(?:Author|Date|Status|Revision|Copyright|Organization):.*\n')
rst_tag = re.compile(r'^:(?:Author|Date|Status|Revision|Copyright|Organization|Format):.*\n')
mps_tag = re.compile(r'_`\.([a-z][A-Za-z.0-9_-]*[A-Za-z0-9])`:')
mps_ref = re.compile(r'`(\.[a-z][A-Za-z.0-9_-]*[A-Za-z0-9])`_(?: )?')
funcdef = re.compile(r'^``([^`]*\([^`]*\))``$')