diff --git a/mps/manual/html/_sources/design/abq.txt b/mps/manual/html/_sources/design/abq.txt
new file mode 100644
index 00000000000..a6846871029
--- /dev/null
+++ b/mps/manual/html/_sources/design/abq.txt
@@ -0,0 +1,106 @@
+.. _design-abq:
+
+
+Queue design
+============
+
+.. mps:prefix:: design.mps.abq
+
+
+Introduction
+------------
+
+:mps:tag:`intro` This is the design of the ABQ module, which implements a
+fixed-length queue of small objects.
+
+:mps:tag:`readership` This document is intended for any MM developer.
+
+:mps:tag:`name` The name ABQ originally stood for "Available Block Queue" as
+the module is used by the MVT pool.
+
+
+Requirements
+------------
+
+:mps:tag:`req.push` Clients can efficiently push new elements onto the queue.
+
+:mps:tag:`req.pop` Clients can efficiently pop elements from the queue.
+
+:mps:tag:`req.empty` Clients can efficiently test whether the queue is empty.
+
+:mps:tag:`req.abstract` The ABQ module does not know anything about the
+elements in the queue other than their size.
+
+:mps:tag:`req.delete` Clients can delete elements from the queue. (Note: not necessarily efficiently.)
+
+:mps:tag:`req.iterate` Clients can iterate over elements in the queue.
+
+
+Interface
+---------
+
+.. c:type:: ABQStruct *ABQ
+
+:c:macro:`ABQ` is the type of a queue. It is an alias for ``ABQStruct *``.
+:c:type:`ABQStruct` is defined in the header so that it can be inlined in
+client structures: clients must not depend on its implementation
+details.
+
+.. c:function:: ABQInit(Arena arena, ABQ abq, void *owner, Count elements, Size elementSize)
+
+Initialize the queue ``abq``. The parameter ``arena`` is the arena
+whose control pool should be used to allocate the memory for the
+queue; ``owner`` is passed to :c:func:`MeterInit()` for the statistics;
+``elements`` is the maximum number of elements that can be stored in
+the queue; and ``elementSize`` is the size of each element.
+
+.. c:function:: void ABQFinish(Arena arena, ABQ abq)
+
+Finish ``abq`` and free all resources associated with it.
+
+.. c:function:: Res ABQPush(ABQ abq, void *element)
+
+If the queue is full, leave it unchanged and return ``ResFAIL``.
+Otherwise, push ``element`` on to the queue and return ``ResOK``.
+
+.. c:function:: Res ABQPop(ABQ abq, void *elementReturn)
+
+If the queue is empty, return ``ResFAIL``. Othwreise, copy the first
+element on the queue into the memory pointed to by ``elementReturn``,
+remove the element from the queue, and return ``ResOK``.
+
+.. c:function:: Res ABQPeek(ABQ abq, void *elementReturn)
+
+If the queue is empty, return ``ResFAIL``. Otherwise, copy the first
+element on the queue into the memory pointed to by ``elementReturn``
+and return ``ResOK``. (This is the same as :c:func:`ABQPop()` except that
+the queue is unchanged.)
+
+.. c:function:: Bool ABQIsEmpty(ABQ abq)
+
+If the queue is empty, return :c:macro:`TRUE`, otherwise return :c:macro:`FALSE`.
+
+.. c:function:: Bool ABQIsFull(ABQ abq)
+
+If the queue is full, return :c:macro:`TRUE`, otherwise return :c:macro:`FALSE`.
+
+.. c:function:: Count ABQDepth(ABQ abq)
+
+Return the number of elements in the queue.
+
+.. c:type:: Bool (*ABQIterateMethod)(Bool *deleteReturn, void *element, void *closureP, Size closureS)
+
+A callback function for :c:func:`ABQIterate()`. The parameter ``element`` is
+an element in the queue, and ``closureP`` and ``closureS`` are the
+values that were originally passed to :c:func:`ABQIterate()`. This function
+must set ``*deleteReturn`` to :c:macro:`FALSE` if ``element`` must be kept in
+the queue, or :c:macro:`TRUE` if ``element`` must be deleted from the queue.
+It must return :c:macro:`TRUE` if the iteration must continue, or :c:macro:`FALSE`
+if the iteration must stop after processing ``element``.
+
+.. c:function:: void ABQIterate(ABQ abq, ABQIterateMethod iterate, void *closureP, Size closureS)
+
+Call ``iterate`` for each elements in the queue, passing the element
+and ``closureP``. See :c:type:`ABQIterateMethod` for details.
+
+
diff --git a/mps/manual/html/_sources/design/boot.txt b/mps/manual/html/_sources/design/boot.txt
new file mode 100644
index 00000000000..ce4d55de18e
--- /dev/null
+++ b/mps/manual/html/_sources/design/boot.txt
@@ -0,0 +1,37 @@
+.. _design-boot:
+
+
+The MPS Bootstrap
+=================
+
+.. mps:prefix:: design.mps.boot
+
+
+Introduction
+------------
+The `Memory Pool System`_ starts with no memory, but must somehow
+allocate its own control structures in order to provide memory to the
+client program. The MPS is freestanding [ref?] and so it can't get its
+memory from the C library's ``malloc``. So how does it get off the
+ground? It pulls itself up by its own bootstraps. This document
+describes how.
+
+.. note::
+
+ This document was written as a prelude to reforming the bootstrap,
+ so it shouldn't be taken as advocating it as an amazing piece of
+ design.
+
+
+Pretty much the first call to the MPS is to ``ArenaCreate``, which calls the
+arena class specific ``init`` method. That must create an initialized arena,
+except for the "control pool", from which many MPS data structures will be
+allocated.
+
+In the case of the VM arena, ``VMArenaInit`` creates a VM large enough to hold
+a :c:type:`VMArenaStruct` (which contains the generic :c:type:`ArenaStruct`) and maps
+pages into it. It then calls ``ArenaInit`` to initialise the generic part,
+before filling in the VM-specific part. Having done that, it adds the initial
+``VMChunk`` -- a large area of address space -- that will be used to supply
+memory via ``ArenaAlloc``.
+
diff --git a/mps/manual/html/_sources/design/cbs.txt b/mps/manual/html/_sources/design/cbs.txt
new file mode 100644
index 00000000000..8cc005422f0
--- /dev/null
+++ b/mps/manual/html/_sources/design/cbs.txt
@@ -0,0 +1,329 @@
+.. _design-cbs:
+
+
+Coalescing block structure
+==========================
+
+.. mps:prefix:: design.mps.cbs
+
+
+Introduction
+------------
+
+:mps:tag:`intro` This is the design for impl.c.cbs, which implements a data
+structure for the management of non-intersecting memory ranges, with
+eager coalescence.
+
+:mps:tag:`readership` This document is intended for any MM developer.
+
+:mps:tag:`source` design.mps.poolmv2, design.mps.poolmvff.
+
+:mps:tag:`overview` The "coalescing block structure" is a set of addresses
+(or a subset of address space), with provision for efficient
+management of contiguous ranges, including insertion and deletion,
+high level communication with the client about the size of contiguous
+ranges, and detection of protocol violations.
+
+
+Definitions
+-----------
+
+:mps:tag:`def.range` A (contiguous) *range* of addresses is a semi-open
+interval on address space.
+
+:mps:tag:`def.isolated` A contiguous range is *isolated* with respect to
+some property it has, if adjacent elements do not have that property.
+
+
+Requirements
+------------
+
+:mps:tag:`req.set` Must maintain a set of addresses.
+
+:mps:tag:`req.fast` Common operations must have a low amortized cost.
+
+:mps:tag:`req.add` Must be able to add address ranges to the set.
+
+:mps:tag:`req.remove` Must be able to remove address ranges from the set.
+
+:mps:tag:`req.size` Must report concisely to the client when isolated
+contiguous ranges of at least a certain size appear and disappear.
+
+:mps:tag:`req.iterate` Must support the iteration of all isolated
+contiguous ranges. This will not be a common operation.
+
+:mps:tag:`req.protocol` Must detect protocol violations.
+
+:mps:tag:`req.debug` Must support debugging of client code.
+
+:mps:tag:`req.small` Must have a small space overhead for the storage of
+typical subsets of address space and not have abysmal overhead for the
+storage of any subset of address space.
+
+:mps:tag:`req.align` Must support an alignment (the alignment of all
+addresses specifying ranges) of down to ``sizeof(void *)`` without
+losing memory.
+
+
+Interface
+---------
+
+:mps:tag:`header` CBS is used through impl.h.cbs.
+
+
+External types
+..............
+
+.. c:type:: struct CBSStruct *CBS
+
+:mps:tag:`type.cbs` :c:macro:`CBS` is the main data structure for manipulating a
+CBS. It is intended that a :c:type:`CBSStruct` be embedded in another
+structure. No convenience functions are provided for the allocation or
+deallocation of the CBS.
+
+.. c:type:: Bool (*CBSIterateMethod)(CBS cbs, Range range, void *closureP, Size closureS)
+
+:mps:tag:`type.cbs.iterate.method` Type :c:type:`CBSIterateMethod` is a callback
+function that may be passed to :c:func:`CBSIterate()`. It is called for
+every isolated contiguous range in address order. The function must
+returns a :c:type:`Bool` indicating whether to continue with the iteration.
+
+
+External functions
+..................
+
+.. c:function:: Res CBSInit(Arena arena, CBS cbs, void *owner, Align alignment, Bool fastFind, ArgList args)
+
+:mps:tag:`function.cbs.init` :c:func:`CBSInit()` is the function that initialises
+the CBS structure. It performs allocation in the supplied arena. The
+parameter ``owner`` is passed to :c:func:`MeterInit()`, an ``alignment``
+indicates the alignment of ranges to be maintained. An initialised CBS
+contains no ranges.
+
+``fastFind``, if set, causes the CBS to maintain, for each subtree,
+the size of the largest block in that subtree. This must be true if
+any of the :c:func:`CBSFindFirst()`, :c:func:`CBSFindLast()`, or
+:c:func:`CBSFindLargest()` functions are going to be used on the CBS.
+
+:c:func:`CBSInit()` may take one keyword argument:
+
+* :c:macro:`MPS_KEY_CBS_EXTEND_BY` (type :c:type:`Size`; default 4096) is the size
+ of segment that the CBS will request from the arena in which to
+ allocate its ``CBSBlock`` structures.
+
+.. c:function:: void CBSFinish(CBS cbs)
+
+:mps:tag:`function.cbs.finish` :c:func:`CBSFinish()` is the function that finishes
+the CBS structure and discards any other resources associated with the
+CBS.
+
+.. c:function:: Res CBSInsert(Range rangeReturn, CBS cbs, Range range)
+
+:mps:tag:`function.cbs.insert` If any part of ``range`` is already in the
+CBS, then leave it unchanged and return ``ResFAIL``. Otherwise,
+attempt to insert ``range`` into the CBS. If the insertion succeeds,
+then update ``rangeReturn`` to describe the contiguous isolated range
+containing the inserted range (this may differ from ``range`` if there
+was coalescence on either side) and return ``ResOK``. If the insertion
+fails, return a result code indicating allocation failure.
+
+:mps:tag:`function.cbs.insert.fail` Insertion of a valid range (that is, one
+that does not overlap with any range in the CBS) can only fail if the
+new range is isolated and the allocation of the necessary data
+structure to represent it failed.
+
+
+.. c:function:: Res CBSDelete(Range rangeReturn, CBS cbs, Range range)
+
+:mps:tag:`function.cbs.delete` If any part of the range is not in the CBS,
+then leave the CBS unchanged and return ``ResFAIL``. Otherwise, update
+``rangeReturn`` to describe the contiguous isolated range that
+contains ``range`` (this may differ from ``range`` if there are
+fragments on either side) and attempt to delete the range from the
+CBS. If the deletion succeeds, return ``ResOK``. If the deletion
+fails, return a result code indicating allocation failure.
+
+:mps:tag:`function.cbs.delete.fail` Deletion of a valid range (that is, one
+that is wholly contained in the CBS) can only fail if there are
+fragments on both sides and the allocation of the necessary data
+structures to represent them fails.
+
+:mps:tag:`function.cbs.delete.return` :c:func:`CBSDelete()` returns the contiguous
+isolated range that contains ``range`` even if the deletion fails.
+This is so that the caller can try deleting the whole block (which is
+guaranteed to succeed) and managing the fragments using a fallback
+strategy.
+
+.. c:function:: void CBSIterate(CBS cbs, CBSIterateMethod iterate, void *closureP, Size closureS)
+
+:mps:tag:`function.cbs.iterate` :c:func:`CBSIterate()` is the function used to
+iterate all isolated contiguous ranges in a CBS. It receives a
+pointer, :c:type:`Size` closure pair to pass on to the iterator method,
+and an iterator method to invoke on every range in address order. If
+the iterator method returns :c:macro:`FALSE`, then the iteration is
+terminated.
+
+.. c:function:: Res CBSDescribe(CBS cbs, mps_lib_FILE *stream)
+
+:mps:tag:`function.cbs.describe` :c:func:`CBSDescribe()` is a function that prints
+a textual representation of the CBS to the given stream, indicating
+the contiguous ranges in order, as well as the structure of the
+underlying splay tree implementation. It is provided for debugging
+purposes only.
+
+.. c:function:: Bool CBSFindFirst(Range rangeReturn, Range oldRangeReturn, CBS cbs, Size size, FindDelete findDelete)
+
+:mps:tag:`function.cbs.find.first` Locate the first block (in address order)
+within the CBS of at least the specified size, update ``rangeReturn``
+to describe that range, and return :c:macro:`TRUE`. If there is no such
+block, it returns :c:macro:`FALSE`.
+
+In addition, optionally delete the top, bottom, or all of the found
+range, depending on the ``findDelete`` argument. This saves a separate
+call to :c:func:`CBSDelete()`, and uses the knowledge of exactly where we
+found the range. The value of ``findDelete`` must come from this
+enumeration::
+
+ enum {
+ FindDeleteNONE, /* don't delete after finding */
+ FindDeleteLOW, /* delete size bytes from low end of block */
+ FindDeleteHIGH, /* delete size bytes from high end of block */
+ FindDeleteENTIRE /* delete entire range */
+ };
+
+The original contiguous isolated range in which the range was found is
+returned via the ``oldRangeReturn`` argument. (If ``findDelete`` is
+``FindDeleteNONE`` or ``FindDeleteENTIRE``, then this will be
+identical to the range returned via the ``rangeReturn`` argument.)
+
+:c:func:`CBSFindFirst()` requires that ``fastFind`` was true when
+:c:func:`CBSInit()` was called.
+
+.. c:function:: Bool CBSFindLast(Range rangeReturn, Range oldRangeReturn, CBS cbs, Size size, FindDelete findDelete)
+
+:mps:tag:`function.cbs.find.last` Like :c:func:`CBSFindFirst()`, except that it
+finds the last block in address order.
+
+.. c:function:: Bool CBSFindLargest(Range rangeReturn, Range oldRangeReturn, CBS cbs, Size size, FindDelete findDelete)
+
+:mps:tag:`function.cbs.find.largest` Locate the largest block within the
+CBS, and if that block is at least as big as ``size``, return its
+range via the ``rangeReturn`` argument, and return :c:macro:`TRUE`. If there
+are no blocks in the CBS at least as large as ``size``, return
+:c:macro:`FALSE`. Pass 0 for ``size`` if you want the largest block
+unconditionally.
+
+Like :c:func:`CBSFindFirst()`, optionally delete the range (specifying
+``FindDeleteLOW`` or ``FindDeleteHIGH`` has the same effect as
+``FindDeleteENTIRE``). This feature requires that ``fastFind`` was
+true when :c:func:`CBSInit()` was called.
+
+
+Implementation
+--------------
+
+:mps:tag:`impl` This section is concerned with describing various aspects of
+the implementation. It does not form part of the interface definition.
+
+
+
+Splay tree
+..........
+
+:mps:tag:`impl.splay` The CBS is principally implemented using a splay tree
+(see design.mps.splay_). Each splay tree node is embedded in a
+``CBSBlock`` that represents a semi-open address range. The key passed
+for comparison is the base of another range.
+
+.. _design.mps.splay: splay
+
+:mps:tag:`impl.splay.fast-find` :c:func:`CBSFindFirst()` and :c:func:`CBSFindLast()` use
+the update/refresh facility of splay trees to store, in each
+``CBSBlock``, an accurate summary of the maximum block size in the
+tree rooted at the corresponding splay node. This allows rapid
+location of the first or last suitable block, and very rapid failure
+if there is no suitable block.
+
+:mps:tag:`impl.find-largest` :c:func:`CBSFindLargest()` simply finds out the size
+of the largest block in the CBS from the root of the tree, using
+:c:func:`SplayRoot()`, and does :c:func:`SplayFindFirst()` for a block of that
+size. This is O(log *n*) in the size of the free list, so it's about
+the best you can do without maintaining a separate priority queue,
+just to do :c:func:`CBSFindLargest()`.
+
+
+Low memory behaviour
+....................
+
+:mps:tag:`impl.low-mem` When the CBS tries to allocate a new ``CBSBlock``
+structure for a new isolated range as a result of either
+:c:func:`CBSInsert()` or :c:func:`CBSDelete()`, and there is insufficient memory
+to allocation the ``CBSBlock`` structure, then the range is not added
+to the CBS or deleted from it, and the call to :c:func:`CBSInsert()` or
+:c:func:`CBSDelete()` returns ``ResMEMORY``.
+
+
+The CBS block
+.............
+
+:mps:tag:`impl.cbs.block` The block contains a base-limit pair and a splay
+tree node.
+
+:mps:tag:`impl.cbs.block.special` The base and limit may be equal if the
+block is halfway through being deleted.
+
+:mps:tag:`impl.cbs.block.special.just` This conflates values and status, but
+is justified because block size is very important.
+
+
+Testing
+-------
+
+:mps:tag:`test` The following testing will be performed on this module:
+
+:mps:tag:`test.cbstest` There is a stress test for this module in
+impl.c.cbstest. This allocates a large block of memory and then
+simulates the allocation and deallocation of ranges within this block
+using both a :c:macro:`CBS` and a :c:type:`BT`. It makes both valid and invalid
+requests, and compares the :c:macro:`CBS` response to the correct behaviour
+as determined by the :c:type:`BT`. It also iterates the ranges in the
+:c:macro:`CBS`, comparing them to the :c:type:`BT`. It also invokes the
+:c:func:`CBSDescribe()` method, but makes no automatic test of the resulting
+output. It does not currently test the callbacks.
+
+:mps:tag:`test.pool` Several pools (currently MVT_ and MVFF_) are implemented
+on top of a CBS. These pool are subject to testing in development, QA,
+and are/will be heavily exercised by customers.
+
+.. _MVT: poolmvt
+.. _MVFF: poolmvff
+
+
+Notes for future development
+----------------------------
+
+:mps:tag:`future.not-splay` The initial implementation of CBSs is based on
+splay trees. It could be revised to use any other data structure that
+meets the requirements (especially :mps:ref:`.req.fast`).
+
+:mps:tag:`future.hybrid` It would be possible to attenuate the problem of
+:mps:ref:`.risk.overhead` (below) by using a single word bit set to represent
+the membership in a (possibly aligned) word-width of grains. This
+might be used for block sizes less than a word-width of grains,
+converting them when they reach all free in the bit set. Note that
+this would make coalescence slightly less eager, by up to
+``(word-width - 1)``.
+
+
+Risks
+-----
+
+:mps:tag:`risk.overhead` Clients should note that the current implementation
+of CBSs has a space overhead proportional to the number of isolated
+contiguous ranges. [Four words per range.] If the CBS contains every
+other grain in an area, then the overhead will be large compared to
+the size of that area. [Four words per two grains.] The CBS structure
+is thus suitable only for managing large enough ranges.
+
+
+
diff --git a/mps/manual/html/_sources/design/freelist.txt b/mps/manual/html/_sources/design/freelist.txt
new file mode 100644
index 00000000000..c8a12533345
--- /dev/null
+++ b/mps/manual/html/_sources/design/freelist.txt
@@ -0,0 +1,257 @@
+.. _design-freelist:
+
+
+Free list allocator
+===================
+
+.. mps:prefix:: design.mps.freelist
+
+
+Introduction
+------------
+
+:mps:tag:`intro` This document describes the free list allocator for the
+Memory Pool System.
+
+:mps:tag:`readership` Any MPS developer.
+
+
+Overview
+--------
+
+:mps:tag:`overview` The free list allocator is an "emergency" allocator. It
+is intended for use as a fallback allocation strategy in low memory
+situations, when memory is not available for the control structures
+needed by other allocators. In these situations the free list allocator
+ensures that memory is not lost, but with several disadvantages:
+
+1. operations on the free list are O(n) in the number of free blocks;
+2. the data structures are stored in client memory and so are
+ vulnerable to corruption;
+3. the data structures have poor locality (and thus potentially poor
+ cache performance).
+
+When memory becomes available again to allocate control structures,
+the free lists can be "flushed" back into the more efficient data
+structures.
+
+:mps:tag:`bg` The free list allocator was formerly part of the Coalescing
+Block Structure module (see design.mps.cbs) but it was split into its
+own module because this makes it:
+
+1. simpler (no need to interact with CBS) and thus more maintainable;
+2. possible to test directly (no need to create a CBS and then force
+ its control pool to run out of memory); and
+3. usable as a fallback allocator in other pools (not just in pools
+ that use CBS).
+
+
+Definitions
+-----------
+
+:mps:tag:`def.range` A (contiguous) *range* of addresses is a semi-open
+interval on address space.
+
+:mps:tag:`def.isolated` A contiguous range is *isolated* with respect to
+some property it has, if adjacent elements do not have that property.
+
+
+Requirements
+------------
+
+:mps:tag:`req.set` Must maintain a set of free address ranges.
+
+:mps:tag:`req.add` Must be able to add free address ranges to the set.
+
+:mps:tag:`req.remove` Must be able to remove address ranges from the set (in
+particular, when memory is allocated).
+
+:mps:tag:`req.iterate` Must support the iteration of all isolated contiguous
+ranges.
+
+:mps:tag:`req.protocol` Must detect protocol violations.
+
+:mps:tag:`req.align` Must support an alignment (the alignment of all
+addresses specifying ranges) of down to ``sizeof(void *)`` without
+losing memory.
+
+:mps:tag:`req.zero-overhead` Must have zero space overhead for the storage
+of any set of free blocks, so that it can be used to manage memory
+when no memory can be allocated for control structures.
+
+:mps:tag:`req.source` This set of requirements is derived from those of the
+CBS module (see design.mps.cbs.req), except that there is no
+equivalent of design.mps.cbs.req.fast, and design.mps.cbs.req.small
+has been replaced with :mps:ref:`.req.zero-overhead`.
+
+
+Interface
+---------
+
+
+Types
+.....
+
+.. c:type:: struct FreelistStruct *Freelist
+
+:mps:tag:`type.freelist` The type of free lists. The structure
+:c:type:`FreelistStruct` is declared in the header so that it can be inlined
+in other structures, but you should not depend on its details.
+
+.. c:type:: Bool (*FreelistIterateMethod)(Bool *deleteReturn, Freelist fl, Range range, void *closureP, Size closureS)
+
+:mps:tag:`type.iterate.method` A callback function that may be passed to
+:c:func:`FreelistIterate()`. It is called for every isolated contiguous
+range in address order, and with the closure arguments that were
+originally passed to :c:func:`FreelistIterate()`. It must update
+``*deleteReturn`` to :c:macro:`TRUE` if the range must be deleted from the
+free lists, or :c:macro:`FALSE` if the range must be kept. The function must
+return :c:macro:`TRUE` if the iteration must continue, and :c:macro:`FALSE` if the
+iteration must stop (after possibly deleting the current range).
+
+
+Functions
+.........
+
+.. c:function:: Res FreelistInit(Freelist fl, Align alignment)
+
+:mps:tag:`function.init` Initialize the ``Freelist`` structure pointed to by
+``fl``. The argument ``alignment`` is the alignment of address ranges
+to be maintained. An initialised free list contains no address ranges.
+
+.. c:function:: void FreelistFinish(Freelist fl)
+
+:mps:tag:`function.finish` Finish the free list pointed to by ``fl``.
+
+.. c:function:: Res FreelistInsert(Range rangeReturn, Freelist fl, Range range)
+
+:mps:tag:`function.insert` If any part of ``range`` is already in the free
+list ``fl``, then leave the free list unchanged and return
+``ResFAIL``. Otherwise, insert ``range`` into the free list ``fl``;
+update ``rangeReturn`` to describe the contiguous isolated range
+containing the inserted range (this may differ from ``range`` if there
+was coalescence on either side) and return ``ResOK``.
+
+.. c:function:: Res FreelistDelete(Range rangeReturn, Freelist fl, Range range)
+
+:mps:tag:`function.delete` If any part of the range is not in the free list,
+then leave the free list unchanged and return ``ResFAIL``. Otherwise,
+remove ``range`` from the free list and update ``rangeReturn`` to
+describe the contiguous isolated range that formerly contained the
+deleted range (this may differ from ``range`` if there were fragments
+left on either side), and return ``ResOK``.
+
+.. c:function:: void FreelistIterate(Freelist fl, FreelistIterateMethod iterate, void *closureP, Size closureS)
+
+:mps:tag:`function.iterate` Iterate all isolated contiguous ranges in the
+free list ``fl`` in address order, calling ``iterate`` for each one.
+See :c:type:`FreelistIterateMethod` for details.
+
+.. c:function:: Bool FreelistFindFirst(Range rangeReturn, Range oldRangeReturn, Freelist fl, Size size, FindDelete findDelete)
+
+:mps:tag:`function.find.first` Locate the first isolated contiguous range in
+address order, within the free list ``fl``, of at least ``size``
+bytes, update ``rangeReturn`` to that range, and return :c:macro:`TRUE`. If
+there is no such continuous range, return :c:macro:`FALSE`.
+
+In addition, optionally delete the found range from the free list,
+depending on the ``findDelete`` argument. This saves a separate call
+to :c:func:`FreelistDelete()`, and uses the knowledge of exactly where we
+found the range. The value of ``findDelete`` must come from this
+enumeration::
+
+ enum {
+ FindDeleteNONE, /* don't delete after finding */
+ FindDeleteLOW, /* delete size bytes from low end of block */
+ FindDeleteHIGH, /* delete size bytes from high end of block */
+ FindDeleteENTIRE /* delete entire range */
+ };
+
+The original contiguous isolated range in which the range was found is
+returned via the ``oldRangeReturn`` argument. (If ``findDelete`` is
+``FindDeleteNONE`` or ``FindDeleteENTIRE``, then this will be
+identical to the range returned via the ``rangeReturn`` argument.)
+
+.. c:function:: Bool FreelistFindLast(Range rangeReturn, Range oldRangeReturn, Freelist fl, Size size, FindDelete findDelete)
+
+:mps:tag:`function.find.last` Like :c:func:`FreelistFindFirst()`, except that it
+finds the last block in address order.
+
+.. c:function:: Bool FreelistFindLargest(Range rangeReturn, Range oldRangeReturn, Freelist fl, Size, size, FindDelete findDelete)
+
+:mps:tag:`function.find.largest` Locate the largest block within the free
+list ``fl``, and if that block is at least as big as ``size``, return
+its range via the ``rangeReturn`` argument, and return :c:macro:`TRUE`. If
+there are no blocks in the free list at least as large as ``size``,
+return :c:macro:`FALSE`. Pass 0 for ``size`` if you want the largest block
+unconditionally.
+
+Like :c:func:`FreelistFindFirst()`, optionally delete the range from the
+free list. (Always the whole range: specifying ``FindDeleteLOW`` or
+``FindDeleteHIGH`` has the same effect as ``FindDeleteENTIRE``).
+
+.. c:function:: void FreelistFlushToCBS(Freelist fl, CBS cbs)
+
+Remove free address ranges from the free list ``fl`` and add them to
+the Coalescing Block Structure ``cbs``. Continue until a call to
+:c:func:`CBSInsert()` fails, or until the free list is empty, whichever
+happens first.
+
+.. c:function:: Res FreelistDescribe(Freelist fl, mps_lib_FILE *stream)
+
+:mps:tag:`function.describe` Print a textual representation of the free
+list ``fl`` to the given stream, indicating the contiguous ranges in
+order. It is provided for debugging purposes only.
+
+
+
+Implementation
+--------------
+
+:mps:tag:`impl.list` The isolated contiguous free address ranges are kept on
+an address-ordered singly linked free list. (As in traditional
+:c:func:`malloc()` implementations.)
+
+:mps:tag:`impl.block` If the free address range is large enough to contain
+an inline block descriptor consisting of two pointers, then the two
+pointers stored are to the next free range in address order (or
+:c:macro:`NULL` if there are no more ranges), and to the limit of current
+free address range, in that order.
+
+:mps:tag:`impl.grain` Otherwise, the free address range must be large enough
+to contain a single pointer. The pointer stored is to the next free
+range in address order, or :c:macro:`NULL` if there are no more ranges.
+
+:mps:tag:`impl.tag` Grains and blocks are distinguished by a one-bit tag in
+the low bit of the first word (the one containing the pointer to the
+next range). Grains have this bit set; blocks have this bit reset.
+
+:mps:tag:`impl.invariant` The ranges stored in the free list are *isolated*:
+no two ranges are adjacent or overlapping.
+
+:mps:tag:`impl.merge` When a free address range is added to the free list,
+it is merged with adjacent ranges so as to maintain
+:mps:ref:`.impl.invariant`.
+
+:mps:tag:`impl.rule.break` The use of :c:macro:`NULL` to mark the end of the list
+violates the rule that exceptional values should not be used to
+distinguish exeptional situations. This infraction allows the
+implementation to meet :mps:ref:`.req.zero-overhead`. (There are other ways to
+do this, such as using another tag to indicate the last block in the
+list, but these would be more complicated.)
+
+
+Opportunities for improvement
+-----------------------------
+
+:mps:tag:`improve.length` When iterating over the list, we could check that
+the number of elements visited in the course of the iteration does not
+exceed the recorded size of the list.
+
+:mps:tag:`improve.maxsize` We could maintain the maximum size of any range
+on the list, and use that to make an early exit from
+:c:func:`FreelistFindLargest()`. It's not clear that this would actually be
+an improvement.
+
+
+
diff --git a/mps/manual/html/_sources/design/index.txt b/mps/manual/html/_sources/design/index.txt
index a0c98237228..3cf6719ac74 100644
--- a/mps/manual/html/_sources/design/index.txt
+++ b/mps/manual/html/_sources/design/index.txt
@@ -6,10 +6,14 @@ Design
.. toctree::
:numbered:
+ abq
+ cbs
config
critical-path
+ freelist
guide.hex.trans
guide.impl.c.format
keyword-arguments
+ range
ring
sig
diff --git a/mps/manual/html/_sources/design/poolmvff.txt b/mps/manual/html/_sources/design/poolmvff.txt
index ee4c005675c..c01308cf2f5 100644
--- a/mps/manual/html/_sources/design/poolmvff.txt
+++ b/mps/manual/html/_sources/design/poolmvff.txt
@@ -120,9 +120,10 @@ Implementation
--------------
:mps:tag:`impl.free-list` The pool stores its free list in a CBS (see
-`design.mps.cbs
.intro: This is the design of the ABQ module, which implements a +fixed-length queue of small objects.
+.readership: This document is intended for any MM developer.
+.name: The name ABQ originally stood for “Available Block Queue” as +the module is used by the MVT pool.
+.req.push: Clients can efficiently push new elements onto the queue.
+.req.pop: Clients can efficiently pop elements from the queue.
+.req.empty: Clients can efficiently test whether the queue is empty.
+.req.abstract: The ABQ module does not know anything about the +elements in the queue other than their size.
+.req.delete: Clients can delete elements from the queue. (Note: not necessarily efficiently.)
+.req.iterate: Clients can iterate over elements in the queue.
+ABQ is the type of a queue. It is an alias for ABQStruct *. +ABQStruct is defined in the header so that it can be inlined in +client structures: clients must not depend on its implementation +details.
+ + +Initialize the queue abq. The parameter arena is the arena +whose control pool should be used to allocate the memory for the +queue; owner is passed to MeterInit() for the statistics; +elements is the maximum number of elements that can be stored in +the queue; and elementSize is the size of each element.
+ + +Finish abq and free all resources associated with it.
+ + +If the queue is full, leave it unchanged and return ResFAIL. +Otherwise, push element on to the queue and return ResOK.
+ + +If the queue is empty, return ResFAIL. Othwreise, copy the first +element on the queue into the memory pointed to by elementReturn, +remove the element from the queue, and return ResOK.
+ + +If the queue is empty, return ResFAIL. Otherwise, copy the first +element on the queue into the memory pointed to by elementReturn +and return ResOK. (This is the same as ABQPop() except that +the queue is unchanged.)
+ + +If the queue is empty, return TRUE, otherwise return FALSE.
+ + +If the queue is full, return TRUE, otherwise return FALSE.
+ + +Return the number of elements in the queue.
+ + +A callback function for ABQIterate(). The parameter element is +an element in the queue, and closureP and closureS are the +values that were originally passed to ABQIterate(). This function +must set *deleteReturn to FALSE if element must be kept in +the queue, or TRUE if element must be deleted from the queue. +It must return TRUE if the iteration must continue, or FALSE +if the iteration must stop after processing element.
+Call iterate for each elements in the queue, passing the element +and closureP. See ABQIterateMethod for details.
+
+MPS Kit release 1.111.0
+All MPS Kit releases
+
+Known issues
+Issues fixed in release 1.111.0
+
The `Memory Pool System`_ starts with no memory, but must somehow +allocate its own control structures in order to provide memory to the +client program. The MPS is freestanding [ref?] and so it can’t get its +memory from the C library’s malloc. So how does it get off the +ground? It pulls itself up by its own bootstraps. This document +describes how.
+Note
+This document was written as a prelude to reforming the bootstrap, +so it shouldn’t be taken as advocating it as an amazing piece of +design.
+Pretty much the first call to the MPS is to ArenaCreate, which calls the +arena class specific init method. That must create an initialized arena, +except for the “control pool”, from which many MPS data structures will be +allocated.
+In the case of the VM arena, VMArenaInit creates a VM large enough to hold +a VMArenaStruct (which contains the generic ArenaStruct) and maps +pages into it. It then calls ArenaInit to initialise the generic part, +before filling in the VM-specific part. Having done that, it adds the initial +VMChunk – a large area of address space – that will be used to supply +memory via ArenaAlloc.
+
MPS Kit release 1.111.0
diff --git a/mps/manual/html/design/bt.html b/mps/manual/html/design/bt.html
index cf3c8ffbb48..c7338dea279 100644
--- a/mps/manual/html/design/bt.html
+++ b/mps/manual/html/design/bt.html
@@ -713,8 +713,8 @@ development.
.test.cbstest: cbstest.c. This was written as a test of the -CBS module (design.mps.cbs(2)). It compares the functional -operation of a CBS with that of a BT so is a good functional +CBS module (design.mps.cbs(2)). It compares the functional +operation of a CBS with that of a BT so is a good functional test of either module.
.test.mmqa.120: MMQA_test_function!210.c. This is used because it has a fair amount of segment allocation and freeing so exercises the arena diff --git a/mps/manual/html/design/cbs.html b/mps/manual/html/design/cbs.html new file mode 100644 index 00000000000..8242ce8331a --- /dev/null +++ b/mps/manual/html/design/cbs.html @@ -0,0 +1,422 @@ + + + + + + +
+ + +.intro: This is the design for impl.c.cbs, which implements a data +structure for the management of non-intersecting memory ranges, with +eager coalescence.
+.readership: This document is intended for any MM developer.
+.source: design.mps.poolmv2, design.mps.poolmvff.
+.overview: The “coalescing block structure” is a set of addresses +(or a subset of address space), with provision for efficient +management of contiguous ranges, including insertion and deletion, +high level communication with the client about the size of contiguous +ranges, and detection of protocol violations.
+.def.range: A (contiguous) range of addresses is a semi-open +interval on address space.
+.def.isolated: A contiguous range is isolated with respect to +some property it has, if adjacent elements do not have that property.
+.req.set: Must maintain a set of addresses.
+.req.fast: Common operations must have a low amortized cost.
+.req.add: Must be able to add address ranges to the set.
+.req.remove: Must be able to remove address ranges from the set.
+.req.size: Must report concisely to the client when isolated +contiguous ranges of at least a certain size appear and disappear.
+.req.iterate: Must support the iteration of all isolated +contiguous ranges. This will not be a common operation.
+.req.protocol: Must detect protocol violations.
+.req.debug: Must support debugging of client code.
+.req.small: Must have a small space overhead for the storage of +typical subsets of address space and not have abysmal overhead for the +storage of any subset of address space.
+.req.align: Must support an alignment (the alignment of all +addresses specifying ranges) of down to sizeof(void *) without +losing memory.
+.header: CBS is used through impl.h.cbs.
+.type.cbs: CBS is the main data structure for manipulating a +CBS. It is intended that a CBSStruct be embedded in another +structure. No convenience functions are provided for the allocation or +deallocation of the CBS.
+ + +.type.cbs.iterate.method: Type CBSIterateMethod is a callback +function that may be passed to CBSIterate(). It is called for +every isolated contiguous range in address order. The function must +returns a Bool indicating whether to continue with the iteration.
+.function.cbs.init: CBSInit() is the function that initialises +the CBS structure. It performs allocation in the supplied arena. The +parameter owner is passed to MeterInit(), an alignment +indicates the alignment of ranges to be maintained. An initialised CBS +contains no ranges.
+fastFind, if set, causes the CBS to maintain, for each subtree, +the size of the largest block in that subtree. This must be true if +any of the CBSFindFirst(), CBSFindLast(), or +CBSFindLargest() functions are going to be used on the CBS.
+CBSInit() may take one keyword argument:
+.function.cbs.finish: CBSFinish() is the function that finishes +the CBS structure and discards any other resources associated with the +CBS.
+ + +.function.cbs.insert: If any part of range is already in the +CBS, then leave it unchanged and return ResFAIL. Otherwise, +attempt to insert range into the CBS. If the insertion succeeds, +then update rangeReturn to describe the contiguous isolated range +containing the inserted range (this may differ from range if there +was coalescence on either side) and return ResOK. If the insertion +fails, return a result code indicating allocation failure.
+.function.cbs.insert.fail: Insertion of a valid range (that is, one +that does not overlap with any range in the CBS) can only fail if the +new range is isolated and the allocation of the necessary data +structure to represent it failed.
+ + +.function.cbs.delete: If any part of the range is not in the CBS, +then leave the CBS unchanged and return ResFAIL. Otherwise, update +rangeReturn to describe the contiguous isolated range that +contains range (this may differ from range if there are +fragments on either side) and attempt to delete the range from the +CBS. If the deletion succeeds, return ResOK. If the deletion +fails, return a result code indicating allocation failure.
+.function.cbs.delete.fail: Deletion of a valid range (that is, one +that is wholly contained in the CBS) can only fail if there are +fragments on both sides and the allocation of the necessary data +structures to represent them fails.
+.function.cbs.delete.return: CBSDelete() returns the contiguous +isolated range that contains range even if the deletion fails. +This is so that the caller can try deleting the whole block (which is +guaranteed to succeed) and managing the fragments using a fallback +strategy.
+.function.cbs.iterate: CBSIterate() is the function used to +iterate all isolated contiguous ranges in a CBS. It receives a +pointer, Size closure pair to pass on to the iterator method, +and an iterator method to invoke on every range in address order. If +the iterator method returns FALSE, then the iteration is +terminated.
+.function.cbs.describe: CBSDescribe() is a function that prints +a textual representation of the CBS to the given stream, indicating +the contiguous ranges in order, as well as the structure of the +underlying splay tree implementation. It is provided for debugging +purposes only.
+.function.cbs.find.first: Locate the first block (in address order) +within the CBS of at least the specified size, update rangeReturn +to describe that range, and return TRUE. If there is no such +block, it returns FALSE.
+In addition, optionally delete the top, bottom, or all of the found +range, depending on the findDelete argument. This saves a separate +call to CBSDelete(), and uses the knowledge of exactly where we +found the range. The value of findDelete must come from this +enumeration:
+enum {
+ FindDeleteNONE, /* don't delete after finding */
+ FindDeleteLOW, /* delete size bytes from low end of block */
+ FindDeleteHIGH, /* delete size bytes from high end of block */
+ FindDeleteENTIRE /* delete entire range */
+};
+The original contiguous isolated range in which the range was found is +returned via the oldRangeReturn argument. (If findDelete is +FindDeleteNONE or FindDeleteENTIRE, then this will be +identical to the range returned via the rangeReturn argument.)
+CBSFindFirst() requires that fastFind was true when +CBSInit() was called.
+.function.cbs.find.last: Like CBSFindFirst(), except that it +finds the last block in address order.
+.function.cbs.find.largest: Locate the largest block within the +CBS, and if that block is at least as big as size, return its +range via the rangeReturn argument, and return TRUE. If there +are no blocks in the CBS at least as large as size, return +FALSE. Pass 0 for size if you want the largest block +unconditionally.
+Like CBSFindFirst(), optionally delete the range (specifying +FindDeleteLOW or FindDeleteHIGH has the same effect as +FindDeleteENTIRE). This feature requires that fastFind was +true when CBSInit() was called.
+.impl: This section is concerned with describing various aspects of +the implementation. It does not form part of the interface definition.
+.impl.splay: The CBS is principally implemented using a splay tree +(see design.mps.splay). Each splay tree node is embedded in a +CBSBlock that represents a semi-open address range. The key passed +for comparison is the base of another range.
+.impl.splay.fast-find: CBSFindFirst() and CBSFindLast() use +the update/refresh facility of splay trees to store, in each +CBSBlock, an accurate summary of the maximum block size in the +tree rooted at the corresponding splay node. This allows rapid +location of the first or last suitable block, and very rapid failure +if there is no suitable block.
+.impl.find-largest: CBSFindLargest() simply finds out the size +of the largest block in the CBS from the root of the tree, using +SplayRoot(), and does SplayFindFirst() for a block of that +size. This is O(log n) in the size of the free list, so it’s about +the best you can do without maintaining a separate priority queue, +just to do CBSFindLargest().
+.impl.low-mem: When the CBS tries to allocate a new CBSBlock +structure for a new isolated range as a result of either +CBSInsert() or CBSDelete(), and there is insufficient memory +to allocation the CBSBlock structure, then the range is not added +to the CBS or deleted from it, and the call to CBSInsert() or +CBSDelete() returns ResMEMORY.
+.impl.cbs.block: The block contains a base-limit pair and a splay +tree node.
+.impl.cbs.block.special: The base and limit may be equal if the +block is halfway through being deleted.
+.impl.cbs.block.special.just: This conflates values and status, but +is justified because block size is very important.
+.test: The following testing will be performed on this module:
+.test.cbstest: There is a stress test for this module in +impl.c.cbstest. This allocates a large block of memory and then +simulates the allocation and deallocation of ranges within this block +using both a CBS and a BT. It makes both valid and invalid +requests, and compares the CBS response to the correct behaviour +as determined by the BT. It also iterates the ranges in the +CBS, comparing them to the BT. It also invokes the +CBSDescribe() method, but makes no automatic test of the resulting +output. It does not currently test the callbacks.
+.test.pool: Several pools (currently MVT and MVFF) are implemented +on top of a CBS. These pool are subject to testing in development, QA, +and are/will be heavily exercised by customers.
+.future.not-splay: The initial implementation of CBSs is based on +splay trees. It could be revised to use any other data structure that +meets the requirements (especially .req.fast).
+.future.hybrid: It would be possible to attenuate the problem of +.risk.overhead (below) by using a single word bit set to represent +the membership in a (possibly aligned) word-width of grains. This +might be used for block sizes less than a word-width of grains, +converting them when they reach all free in the bit set. Note that +this would make coalescence slightly less eager, by up to +(word-width - 1).
+.risk.overhead: Clients should note that the current implementation +of CBSs has a space overhead proportional to the number of isolated +contiguous ranges. [Four words per range.] If the CBS contains every +other grain in an area, then the overhead will be large compared to +the size of that area. [Four words per two grains.] The CBS structure +is thus suitable only for managing large enough ranges.
+
+MPS Kit release 1.111.0
+All MPS Kit releases
+
+Known issues
+Issues fixed in release 1.111.0
+
.intro: This document describes how the Memory Pool System source code is configured so that it can target different architectures, operating systems, build environments, varieties, and products.
.req.import: The MPS must be simple to include in third-party projects.
.req.arch: Allow architecture specific configurations of the MPS, so that we can vary the MPS according to the target architecture.
@@ -79,7 +79,7 @@ maintainability of the implementation..req.maint: Maintenance of the configuration and build system should not consume much developer time.
.req.prod: Allow product specific configurations of the MPS, so that
we can build variants of the MPS for use in different products. This
requirement has been retired on 2012-09-03 as part of work on the
@@ -89,7 +89,7 @@ reduce costs and increase reliability. See
- .def.platform: A platform is a combination of an architecture
(.def.arch), an operating system (.def.os), and a builder
(.def.builder). The set of supported platforms is maintained in the
@@ -117,7 +117,7 @@ as a dimension of configuration since .def.target: The target is the result of the build.1.3. Definitions¶
+3.3. Definitions¶
.import.source: The MPS can be simply included in client products as source code. Since version 1.110 we made it possible to simply include the file mps.c in a client’s build process, without @@ -139,9 +139,9 @@ general rules for implementation [ref?] that are enforced by MPS development procedures including code review and inspection.
.build.fun: The MPS implementation assumes only a simple “build function” that takes a set of sources, possibly in several languages, compiles them with a set of predefined preprocessor symbols, and links @@ -199,7 +199,7 @@ example, when building a test program, it might include the ANSI C library and an operating system interface library.
.file.dir: The MPS source code is arranged in a single directory called “code” containing all the sources for the whole family of targets.
@@ -213,7 +213,7 @@ indicates the source language. in their name. See .mod.impls..mod.unique: Each module has an identifier which is unique within the MPS.
.mod.impls: Each module has one or more implementations which may be in any language supported by the relevant build environment.
@@ -234,7 +234,7 @@ platform-independent C in ss.c architecture built with Microsoft Visual C..build.rat: This simple design makes it possible to build the MPS using many different tools. Microsoft Visual C and other graphical development tools do not support much in the way of generated sources, @@ -266,7 +266,7 @@ Publishing division of Harlequin.
.impl: The two implementation files config.h and mpstd.h can be seen as preprocessor programs which “accept” build parameters and “emit” configuration parameters (.fig.impl). The build parameters are @@ -302,7 +302,7 @@ identifers beginning “CONFIG_” should only appear in impl.h.config. Code may depend on configuration parameters in certain, limited ways, as defined below (.conf).
.pf: The target platform is “detected” by the preprocessor directives in mpstd.h.
.pf.form: This file consists of sets of directives of the form:
@@ -379,7 +379,7 @@ that is used to identify the target platform in.var: The target variety is handled by preprocessor directives in impl.h.config.
.var.form: The file contains sets of directives of the form:
@@ -403,7 +403,7 @@ should all begin with the prefix.conf: This section describes how the configuration may affect the source code of the MPS.
.conf.limit: The form of dependency allowed is carefully limited to @@ -412,7 +412,7 @@ ensure that code remains maintainable and portable (.req.impact).
.conf.params: The compilation of a module is parameterized by:
MPS_ARCH_<arch-code>
MPS_OS_<os-code>
@@ -422,7 +422,7 @@ be kept to a minimum in order to keep the system maintainable
.abs.caller: Basic principle: the caller musn’t be affected by configuration of a module. This reduces complexity and dependency of configuration. All callers use the same abstract interface. Caller @@ -490,7 +490,7 @@ For example, this sort of thing: