1
Fork 0
mirror of git://git.sv.gnu.org/emacs.git synced 2025-12-26 07:11:34 -08:00

New design document design.mps.an explains the design of generic modules.

Copied from Perforce
 Change: 187450
 ServerID: perforce.ravenbrook.com
This commit is contained in:
Gareth Rees 2014-11-02 16:54:28 +00:00
parent 70f065f187
commit e0658be613
8 changed files with 302 additions and 190 deletions

216
mps/design/an.txt Normal file
View file

@ -0,0 +1,216 @@
.. mode: -*- rst -*-
Generic modules
===============
:Tag: design.mps.an
:Author: Gareth Rees
:Date: 2014-11-02
:Status: complete design
:Revision: $Id$
:Copyright: See `Copyright and License`_.
:Index terms: pair: generic modules; design
Introduction
------------
_`.intro`: This is the design of generic modules in the MPS.
_`.readership`: Any MPS developer; anyone porting the MPS to a new
platform.
_`.overview`: Generic modules provide implementations of functional
modules using only the features of the Standard C Library. These
implementations are partially functional or non-functional, but
provide a basis for ports of the MPS to new platforms.
_`.name`: The name "ANSI" for the generic modules is historical: the C
language was originally standardized by the American National
Standards Institute, and so Standard C used to be known as "ANSI C".
Requirements
------------
_`.req.port`: The MPS must be portable to new platforms. (Otherwise we
can't meet the needs of customers using new platforms.)
_`.req.port.rapid`: The MPS should be portable to new platforms
rapidly.
_`.req.port.rapid.expert`: An expert MPS developer (who may be a
novice on the new platform) should be able to get a minimally useful
implementation of the MPS running on a new platform within a few
hours.
_`.req.port.rapid.novice`: A novice MPS developer (who is an expert on
the new platform) should be able to get the MPS running on a new
platform within a few days.
Design
------
_`.sol.modules`: Features of the MPS which can benefit from
platform-specific implementations are divided into *functional
modules*, with clean interfaces to the MPS and to each other. See
`.mod`_ for a list of these modules. (This helps meet `.req.port`_ by
isolating the platform dependencies, and it helps meet
`.req.port.rapid`_ because a porter can mix and match implementations,
using existing implementations where possible.)
_`.sol.generic`: Each functional module has a generic implementation
using only features of the Standard C Library. (This helps meet
`.req.port.rapid`_ because the MPS can be ported in stages, starting
with the generic modules and porting the modules needed to meet the
most urgent requirements. The generic implementations help meet
`.req.port.rapid.novice`_ by providing clear and illustrative
examples.)
_`.sol.fallback`: The interfaces to the modules are designed to make
it possible to implement `.sol.generic`_. When a platform-specific
feature is needed to meet performance (or other attribute)
requirements, the interface also makes it possible to meet the
functional requirements while missing the attribute requirements. See
`.sol.fallback.example`_ for an example. (This helps meet
`.req.port.rapid`_ by allowing the generic implementations to meet
many or most of the functional requirements.)
_`.sol.fallback.example`: The MPS normally uses incremental collection
to meet requirements on pause times, but this requires barriers. The
interface to the protection module is designed to make it possible to
write an implementation without barriers, via the function
``ProtSync()`` that synchronizes the mutator with the collector.
_`.sol.test`: There are makefiles for the pseudo-platforms ``anangc``,
``ananll`` and ``ananmv`` that compile and test the generic
implementations. See design.mps.config.opt_ for the configuration
options used to implement these platforms. (This supports
`.req.port.rapid`_ by making sure that the generic implementations are
working when it is time to use them.)
.. _design.mps.config.opt: config#opt
Modules
-------
_`.mod`: This section lists the functional modules in the MPS.
_`.mod.lock`: Locks. See design.mps.lock_.
_`.mod.prmc`: Protection mutator context. See design.mps.prmc_.
_`.mod.prot`: Memory protection. See design.mps.prot_.
_`.mod.ss`: Stack and register scanning. See design.mps.ss_.
_`.mod.sp`: Stack probe. See design.mps.sp_.
_`.mod.th`: Thread manager. See design.mps.thread-manager_.
_`.mod.vm`: Virtual mapping. See design.mps.vm_.
.. _design.mps.lock: lock
.. _design.mps.prot: prot
.. _design.mps.prmc: prmc
.. _design.mps.sp: sp
.. _design.mps.ss: ss
.. _design.mps.thread-manager: thread-manager
.. _design.mps.vm: vm
Limitations of generic implementations
--------------------------------------
_`.lim`: This section summarizes the limitations of the generic
implementations of the function modules.
_`.lim.lock`: Requires a single-threaded mutator (see
design.mps.lock.impl.an_).
_`.lim.prmc`: Does not support single-stepping of accesses (see
design.mps.prmc.impl.an.fault_) and requires a single-threaded mutator
(see design.mps.prmc.impl.an.suspend_).
_`.lim.prot`: Does not support incremental collection (see
design.mps.prot.impl.an.sync_) and is not compatible with
implementations of the protection mutator context module that support
single-stepping of accesses (see design.mps.prot.impl.an.sync.issue_).
_`.lim.sp`: Only suitable for use with programs that do not handle
stack overflow faults, or do not call into the MPS from the handler
(see design.mps.sp.issue.an_).
_`.lim.ss`: Overscans compared to a platform-specific implementation
(see design.mps.ss.impl.an_).
_`.lim.th`: Requires a single-threaded mutator (see
design.mps.thread-manager.impl.an.single_).
_`.lim.vm`: Maps all reserved addresses into main memory (see
design.mps.vm.impl.an.reserve_), thus using more main memory than a
platform-specific implementation.
.. _design.mps.lock.impl.an: lock#impl.an
.. _design.mps.prmc.impl.an.fault: prmc#impl.an.fault
.. _design.mps.prmc.impl.an.suspend: prmc#impl.an.suspend
.. _design.mps.prot.impl.an.sync: prot#impl.an.sync
.. _design.mps.prot.impl.an.sync.issue: prot#impl.an.sync.issue
.. _design.mps.sp.issue.an: sp#issue.an
.. _design.mps.ss.impl.an: ss#impl.an
.. _design.mps.thread-manager.impl.an.single: thread-manager#impl.an.single
.. _design.mps.vm.impl.an.reserve: vm#impl.an.reserve
Document History
----------------
- 2014-11-02 GDR_ Initial draft based on design.mps.protan.
.. _GDR: http://www.ravenbrook.com/consultants/gdr/
Copyright and License
---------------------
Copyright © 2014 Ravenbrook Limited. All rights reserved.
<http://www.ravenbrook.com/>. This is an open source license. Contact
Ravenbrook for commercial licensing options.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
#. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
#. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
#. Redistributions in any form must be accompanied by information on how
to obtain complete source code for this software and any
accompanying software that uses this software. The source code must
either be included in the distribution or be available for no more than
the cost of distribution plus a nominal fee, and must be freely
redistributable under reasonable conditions. For an executable file,
complete source code means the source code for all modules it contains.
It does not include source code for modules or files that typically
accompany the major components of the operating system on which the
executable file runs.
**This software is provided by the copyright holders and contributors
"as is" and any express or implied warranties, including, but not
limited to, the implied warranties of merchantability, fitness for a
particular purpose, or non-infringement, are disclaimed. In no event
shall the copyright holders and contributors be liable for any direct,
indirect, incidental, special, exemplary, or consequential damages
(including, but not limited to, procurement of substitute goods or
services; loss of use, data, or profits; or business interruption)
however caused and on any theory of liability, whether in contract,
strict liability, or tort (including negligence or otherwise) arising in
any way out of the use of this software, even if advised of the
possibility of such damage.**

View file

@ -41,6 +41,7 @@ Designs
====================== ================================================
abq_ Fixed-length queues
alloc-frame_ Allocation frame protocol
an_ Generic modules
arena_ Arena
arenavm_ Virtual memory arena
bt_ Bit tables
@ -82,7 +83,6 @@ poolmvt_ Manual Variable Temporal pool class
poolmvff_ Manual Variable First-Fit pool class
prmc_ Protection mutator context
prot_ Memory protection
protan_ ANSI implementation of protection module
protli_ Linux implementation of protection module
protocol_ Protocol inheritance
protsu_ SunOS 4 implementation of protection module
@ -117,6 +117,7 @@ writef_ The WriteF function
.. _abq: abq
.. _alloc-frame: alloc-frame
.. _an: an
.. _arena: arena
.. _arenavm: arenavm
.. _bt: bt
@ -158,7 +159,6 @@ writef_ The WriteF function
.. _poolmvff: poolmvff
.. _prmc: prmc
.. _prot: prot
.. _protan: protan
.. _protli: protli
.. _protocol: protocol
.. _protsu: protsu

View file

@ -166,7 +166,7 @@ implemented using the same mechanism as normal locks. (But an
operating system-specific mechanism is used, if possible, to ensure
that the global locks are initialized just once.)
_`.impl.ansi`: Single-threaded generic implementation ``lockan.c``:
_`.impl.an`: Single-threaded generic implementation ``lockan.c``:
- single-threaded;
- no need for locking;
@ -174,7 +174,7 @@ _`.impl.ansi`: Single-threaded generic implementation ``lockan.c``:
- provides checking in debug version;
- otherwise does nothing except keep count of claims.
_`.impl.win`: Windows implementation ``lockw3.c``:
_`.impl.w3`: Windows implementation ``lockw3.c``:
- supports Windows threads;
- uses critical section objects [cso]_;
@ -182,7 +182,7 @@ _`.impl.win`: Windows implementation ``lockw3.c``:
- recursive and non-recursive calls use the same Windows function;
- also performs checking.
_`.impl.posix`: POSIX implementation ``lockix.c``:
_`.impl.ix`: POSIX implementation ``lockix.c``:
- supports [POSIXThreads]_;
- locking structure contains a mutex, initialized to check for
@ -194,7 +194,7 @@ _`.impl.posix`: POSIX implementation ``lockix.c``:
success or ``EDEADLK`` (indicating a recursive claim);
- also performs checking.
_`.impl.linux`: Linux implementation ``lockli.c``:
_`.impl.li`: Linux implementation ``lockli.c``:
- supports [POSIXThreads]_;
- also supports [LinuxThreads]_, a partial implementation of POSIX Threads

View file

@ -116,9 +116,22 @@ _`.if.sync.noop`: ``ProtSync()`` is permitted to be a no-op if
Implementations
---------------
_`.impl.an`: Generic implementation. See design.mps.protan_.
_`.impl.an`: Generic implementation in ``protan.c``.
.. _design.mps.protan: protan
_`.impl.an.set`: ``ProtSet()`` does nothing.
_`.impl.an.sync`: ``ProtSync()`` has no way of changing the protection
of a segment, so it simulates faults on all segments that are supposed
to be protected, by calling ``TraceSegAccess()``, until it determines
that no segments require protection any more. This forces the trace to
proceed until it is completed, preventing incremental collection.
_`.impl.an.sync.issue`: This relies on the pool actually removing the
protection, otherwise there is an infinite loop here. This is
therefore not compatible with implementations of the protection
mutator context module that support single-stepping of accesses (see design.mps.prmc.req.fault.step_).
.. _design.mps.prmc.req.fault.step: prmc#req.fault.step
_`.impl.ix`: POSIX implementation.

View file

@ -1,135 +0,0 @@
.. mode: -*- rst -*-
ANSI implementation of protection module
========================================
:Tag: design.mps.protan
:Author: David Jones
:Date: 1997-03-19
:Status: incomplete document
:Revision: $Id$
:Copyright: See `Copyright and License`_.
:Index terms:
pair: ANSI; protection interface design
pair: ANSI protection interface; design
Introduction
------------
_`.readership`: Any MPS developer.
_`.intro`: This is the design for the ANSI implementation of the
protection module.
Requirements
------------
_`.req.test`: This module is required for testing. Particularly on
platforms where no real implementation of the protection module
exists.
_`.req.rapid-port`: This module is required for rapid porting. It
should enable a developer to port a minimally useful configuration of
the MPS to new platforms very quickly.
Overview
--------
_`.overview`: Most of the functions in the module do nothing. The
exception is ``ProtSync()`` which traverses over all segments in the
arena and simulates an access to each segment that has any protection
on it. This means that this module depends on certain fields in the
segment structure.
_`.overview.noos`: No operating system specific (or even ANSI hosted
specific) code is in this module. It can therefore be used on any
platform, particularly where no real implementation of the module
exists. It satisfies `.req.test`_ and `.req.rapid-port`_ in this way.
Functions
---------
_`.fun.protsetup`: ``ProtSetup()`` does nothing as there is nothing to
do (under UNIX we might expect the protection module to install one or
more signal handlers at this pointer, but that is not appropriate for
the ANSI implementation). Of course, we can't have an empty function
body, so there is a ``NOOP;`` here.
_`.fun.sync`: ``ProtSync()`` is called to ensure that the actual
protection of each segment (as determined by the OS) is in accordance
with the segments's pm field. In the ANSI implementation we have no
way of changing the protection of a segment, so instead we generate
faults on all protected segments in the assumption that that will
remove the protection on segments.
_`.fun.sync.how`: Continually loops over all the segments until it
finds that all segments have no protection.
_`.fun.sync.seg`: If it finds a segment that is protected then
``PoolAccess()`` is called on that segment's pool and with that
segment. The call to ``PoolAccess()`` is wrapped with a
``ShieldEnter()`` and ``ShieldLeave()`` thereby giving the pool the
illusion that the fault was generated outside the MM. This depends on
being able to determine the protection of a segment (using the ``pm``
field), on being able to call ``ShieldEnter()`` and ``ShieldLeave()``,
and on being able to call ``PoolAccess()``.
Document History
----------------
- 1997-03-19 David Jones. Incomplete document.
- 2002-06-07 RB_ Converted from MMInfo database design document.
- 2013-05-23 GDR_ Converted to reStructuredText.
.. _RB: http://www.ravenbrook.com/consultants/rb/
.. _GDR: http://www.ravenbrook.com/consultants/gdr/
Copyright and License
---------------------
Copyright © 2013-2014 Ravenbrook Limited. All rights reserved.
<http://www.ravenbrook.com/>. This is an open source license. Contact
Ravenbrook for commercial licensing options.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
#. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
#. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
#. Redistributions in any form must be accompanied by information on how
to obtain complete source code for this software and any
accompanying software that uses this software. The source code must
either be included in the distribution or be available for no more than
the cost of distribution plus a nominal fee, and must be freely
redistributable under reasonable conditions. For an executable file,
complete source code means the source code for all modules it contains.
It does not include source code for modules or files that typically
accompany the major components of the operating system on which the
executable file runs.
**This software is provided by the copyright holders and contributors
"as is" and any express or implied warranties, including, but not
limited to, the implied warranties of merchantability, fitness for a
particular purpose, or non-infringement, are disclaimed. In no event
shall the copyright holders and contributors be liable for any direct,
indirect, incidental, special, exemplary, or consequential damages
(including, but not limited to, procurement of substitute goods or
services; loss of use, data, or profits; or business interruption)
however caused and on any theory of liability, whether in contract,
strict liability, or tort (including negligence or otherwise) arising in
any way out of the use of this software, even if advised of the
possibility of such damage.**

View file

@ -7,6 +7,7 @@ Design
:numbered:
abq
an
cbs
config
critical-path

View file

@ -41,7 +41,6 @@ Old design
poolmv
poolmvt
poolmvff
protan
protli
protsu
protocol

View file

@ -38,29 +38,10 @@ usable.
See :ref:`design-lock` for the design, and ``lock.h`` for the
interface. There are implementations for Linux in ``lockli.c``,
POSIX in ``lockix.c``, and Windows in ``lockw3.c``. There is a
generic implementation in ``lockan.c``, which cannot actually take
any locks and so only works for a single thread.
POSIX in ``lockix.c``, and Windows in ``lockw3.c``.
#. The **thread manager** module suspends and resumes :term:`threads`,
so that the MPS can gain exclusive access to :term:`memory (2)`,
and so that it can scan the :term:`registers` and :term:`control
stack` of suspended threads.
See :ref:`design-thread-manager` for the design, and ``th.h`` for
the interface. There are implementations for POSIX in ``thix.c``
plus ``pthrdext.c``, OS X using Mach in ``thxc.c``, Windows in
``thw3.c``. There is a generic implementation in ``than.c``, which
necessarily only supports a single thread.
#. The **virtual mapping** module reserves :term:`address space` from
the operating system (and returns it), and :term:`maps <mapping>`
address space to :term:`main memory` (and unmaps it).
See :ref:`design-vm` for the design, and ``vm.h`` for the
interface. There are implementations for POSIX in ``vmix.c``, and
Windows in ``vmw3.c``. There is a generic implementation in
``vman.c``, which fakes virtual memory by calling :c:func:`malloc`.
There is a generic implementation in ``lockan.c``, which cannot
actually take any locks and so only works for a single thread.
#. The **memory protection** module applies :term:`protection` to
areas of :term:`memory (2)`, ensuring that attempts to read or
@ -70,10 +51,13 @@ usable.
See :ref:`design-prot` for the design, and ``prot.h`` for the
interface. There are implementations for POSIX in ``protix.c`` plus
``protsgix.c``, Linux in ``protli.c``, Windows in ``protw3.c``, and
OS X using Mach in ``protxc.c``. There is a generic implementation
in ``protan.c``, which can't provide memory protection, so it
forces memory to be scanned until that there is no further need to
protect it.
OS X using Mach in ``protxc.c``.
There is a generic implementation in ``protan.c``, which can't
provide memory protection, so it forces memory to be scanned until
that there is no further need to protect it. This means it can't
support incremental collection, and has no control over pause
times.
#. The **protection mutator context** module figures out what the
:term:`mutator` was doing when it caused a :term:`protection
@ -83,8 +67,10 @@ usable.
See :ref:`design-prmc` for the design, and ``prot.h`` for the
interface. There are implementations on Unix, Windows, and OS X for
IA-32 and x86-64. There is a generic implementation in
``prmcan.c``, which can't provide these features.
IA-32 and x86-64.
There is a generic implementation in ``prmcan.c``, which can't
provide these features, and so only supports a single thread.
#. The **stack probe** module checks that there is enough space on the
:term:`control stack` for the MPS to complete any operation that it
@ -93,8 +79,12 @@ usable.
See :ref:`design-sp` for the design, and ``sp.h`` for the
interface. There are implementations on Windows on IA-32 in
``spi3w3.c`` and x86-64 in ``spi6w3.c``. There is a generic
implementation in ``span.c``, which can't provide this feature.
``spi3w3.c`` and x86-64 in ``spi6w3.c``.
There is a generic implementation in ``span.c``, which can't
provide this feature, and so is only suitable for use with a client
program that does not handle stack overflow faults, or does not
call into the MPS from the handler.
#. The **stack and register scanning** module :term:`scans` the
:term:`registers` and :term:`control stack` of a thread.
@ -103,8 +93,34 @@ usable.
interface. There are implementations for POSIX on IA-32 in
``ssixi3.c`` and x86-64 in ``ssixi6.c``, and for Windows with
Microsoft Visual C/C++ on IA-32 in ``ssw3i3mv.c`` and x86-64 in
``ssw3i6mv.c``. There is a generic implementation in ``ssan.c``,
which calls :c:func:`setjmp` to spill the registers.
``ssw3i6mv.c``.
There is a generic implementation in ``ssan.c``, which calls
:c:func:`setjmp` to spill the registers and scans the whole jump
buffer, thus overscanning compared to a platform-specific
implementation.
#. The **thread manager** module suspends and resumes :term:`threads`,
so that the MPS can gain exclusive access to :term:`memory (2)`,
and so that it can scan the :term:`registers` and :term:`control
stack` of suspended threads.
See :ref:`design-thread-manager` for the design, and ``th.h`` for
the interface. There are implementations for POSIX in ``thix.c``
plus ``pthrdext.c``, OS X using Mach in ``thxc.c``, Windows in
``thw3.c``.
There is a generic implementation in ``than.c``, which necessarily
only supports a single thread.
#. The **virtual mapping** module reserves :term:`address space` from
the operating system (and returns it), and :term:`maps <mapping>`
address space to :term:`main memory` (and unmaps it).
See :ref:`design-vm` for the design, and ``vm.h`` for the
interface. There are implementations for POSIX in ``vmix.c``, and
Windows in ``vmw3.c``. There is a generic implementation in
``vman.c``, which fakes virtual memory by calling :c:func:`malloc`.
Platform detection
@ -185,15 +201,17 @@ Makefile
--------
Add a makefile even if you expect to use an integrated development
environment like Visual Studio or Xcode. Makefiles make it easier to
carry out continuous integration and delivery.
environment (IDE) like Visual Studio or Xcode. Makefiles make it
easier to carry out continuous integration and delivery, and are less
likely to stop working because of incompatibilities between IDE
versions.
The makefile must be named ``osarct.gmk``, and must define ``PFM`` to
be the platform code, ``MPMPF`` to be the list of platform modules
(the same files included by ``mps.c``), and ``LIBS`` to be the linker
options for any libraries required by the test cases. Then it must
include the compiler-specific makefile and ``comm.gmk``. For example,
``lii6ll.gmk`` looks like this::
On Unix platforms, the makefile must be named ``osarct.gmk``, and must
define ``PFM`` to be the platform code, ``MPMPF`` to be the list of
platform modules (the same files included by ``mps.c``), and ``LIBS``
to be the linker options for any libraries required by the test cases.
Then it must include the compiler-specific makefile and ``comm.gmk``.
For example, ``lii6ll.gmk`` looks like this::
PFM = lii6ll
@ -232,13 +250,13 @@ Then check that the "smoke tests" pass, by running::
make -f osarct.gmk testrun
Most or all of the test cases should pass at this point (if you're
Most or all of the test cases should pass at this point. If you're
using the generic threading implementation, then the multi-threaded
test cases ``amcssth`` and ``awlutth`` are expected to fail; and if
you're using the generic lock implementation, then the lock
utilization test case ``lockut`` is expected to fail). However,
performance will be very poor if you're using the generic memory
protection implementation.
test cases ``amcssth`` and ``awlutth`` are expected to fail. If you're
using the generic lock implementation, then the lock utilization test
case ``lockut`` is expected to fail. If you're using the generic
memory protection implementation, performance is expected to be poor,
as it does not support incremental collection.
Now that there is a working system to build on, porting the necessary
modules to the new platform can be done incrementally. It's a good