1
Fork 0
mirror of git://git.sv.gnu.org/emacs.git synced 2026-03-16 03:43:50 -07:00

Guide for new developers.

Documentation design.
Expand test case design.

Copied from Perforce
 Change: 195083
This commit is contained in:
Gareth Rees 2018-09-18 12:50:29 +01:00
parent 6780a47e2d
commit f6721224bf
7 changed files with 442 additions and 28 deletions

View file

@ -3,7 +3,7 @@
# comm.gmk: COMMON GNUMAKEFILE FRAGMENT
#
# $Id$
# Copyright (c) 2001-2016 Ravenbrook Limited. See end of file for license.
# Copyright (c) 2001-2018 Ravenbrook Limited. See end of file for license.
#
# DESCRIPTION
#
@ -308,11 +308,7 @@ all: $(ALL_TARGETS)
# == Automated test suites ==
#
# testrun = "smoke test", fast enough to run before every commit
# testci = continuous integration tests, must be known good
# testall = all test cases, for ensuring quality of a release
# testansi = tests that run on the generic ("ANSI") platform
# testpollnone = tests that run on the generic platform with CONFIG_POLL_NONE
# See design.mps.tests.target.
TEST_SUITES=testrun testci testall testansi testpollnone
@ -714,7 +710,7 @@ find-puns: phony
# C. COPYRIGHT AND LICENSE
#
# Copyright (c) 2001-2016 Ravenbrook Limited <http://www.ravenbrook.com/>.
# Copyright (c) 2001-2018 Ravenbrook Limited <http://www.ravenbrook.com/>.
# All rights reserved. This is an open source license. Contact
# Ravenbrook for commercial licensing options.
#

View file

@ -164,9 +164,7 @@ predefined when compiling the module sources::
CONFIG_VAR_<variety-code>
The variety-codes are the letter code that appears after "variety." in
the tag of the relevant variety document (see variety.*) converted to
upper case. Currently (2012-09-03):
_`.var`: The variety codes are as follows:
_`.var.hot`: ``HOT``
@ -621,7 +619,7 @@ Document History
Copyright and License
---------------------
Copyright © 2013-2016 Ravenbrook Limited <http://www.ravenbrook.com/>.
Copyright © 2013-2018 Ravenbrook Limited <http://www.ravenbrook.com/>.
All rights reserved. This is an open source license. Contact
Ravenbrook for commercial licensing options.

149
mps/design/doc.txt Normal file
View file

@ -0,0 +1,149 @@
.. mode: -*- rst -*-
Documentation
=============
:Tag: design.mps.doc
:Author: Gareth Rees
:Date: 2018-09-18
:Status: draft design
:Revision: $Id$
:Copyright: See section `Copyright and License`_.
:Index terms: pair: documentation; design
Introduction
------------
_`.intro`: This is the design of the documentation system for the
Memory Pool System.
_`.readership`: This document is intended for any MPS developer.
Types
-----
_`.type`: The MPS has multiple types of documentation, suitable for
different audiences.
_`.type.comment`: Comments in the code provide information that is
required in order for developers to make correct edits to nearby code.
(Audience: MPS developers editing nearby code.)
_`.type.design`: Design documentation lists requirements and explains
how the code meets the requirements. (Audience: MPS developers working
on a subsystem.)
_`.type.devguide`: Developer guides provide general guidance for
developers, not specific to any particular subsystem. (Audience: MPS
developers generally.)
_`.type.procedure`: Procedures list the steps for carrying out
development tasks. (Audience: MPS developers who need to carry out
particular tasks reliably.)
_`.type.tutorial`: Tutorials describe how to use the MPS to meet
client program requirements. (Audience: beginner client program
developers.)
_`.type.reference`: Reference documentation specifies the public
features of the MPS. (Audience: expert client program developers.)
_`.type.mmref`: The Memory Management Reference describes general
principles of memory management, with cross-references to the MPS
documentation. (Audience: the world.)
Requirements
------------
_`.req.source`: Derived from [RB_2013-05-09]_.
_`.req.easy`: It must be easy to read and write documentation must
using standard text editors. Barriers to documentation must be low.
_`.req.presentation`: It must be possible to process documentation
into presentation formats, for example web pages.
_`.req.single-source`: Documents must have a single source. Processing
into other formats must be automatic and not depend on hand editing or
maintaining parallel versions.
_`.req.durable`: The format of documents should be supported for the
foreseeable future. It must not require continual updating to keep up
with changes to processing software.
_`.req.design.ref`: It must be easy to reference points made in design
documents from the code.
_`.req.design.standalone`: Design documents must stand alone: they
must not require particular software to make them readable or
complete.
Implementation
--------------
TODO
References
----------
.. [RB_2013-05-09]
"MPS design document format and process";
Richard Brooksby; Ravenbrook Limited; 2013-05-09;
<https://info.ravenbrook.com/mail/2013/05/09/14-54-55/0/>.
Document History
----------------
- 2018-09-18 GDR_ Created based on [RB_2013-05-09]_.
.. _GDR: http://www.ravenbrook.com/consultants/gdr/
Copyright and License
---------------------
Copyright © 2018 Ravenbrook Limited <http://www.ravenbrook.com/>.
All rights reserved. This is an open source license. Contact
Ravenbrook for commercial licensing options.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
#. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
#. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
#. Redistributions in any form must be accompanied by information on how
to obtain complete source code for this software and any
accompanying software that uses this software. The source code must
either be included in the distribution or be available for no more than
the cost of distribution plus a nominal fee, and must be freely
redistributable under reasonable conditions. For an executable file,
complete source code means the source code for all modules it contains.
It does not include source code for modules or files that typically
accompany the major components of the operating system on which the
executable file runs.
**This software is provided by the copyright holders and contributors
"as is" and any express or implied warranties, including, but not
limited to, the implied warranties of merchantability, fitness for a
particular purpose, or non-infringement, are disclaimed. In no event
shall the copyright holders and contributors be liable for any direct,
indirect, incidental, special, exemplary, or consequential damages
(including, but not limited to, procurement of substitute goods or
services; loss of use, data, or profits; or business interruption)
however caused and on any theory of liability, whether in contract,
strict liability, or tort (including negligence or otherwise) arising in
any way out of the use of this software, even if advised of the
possibility of such damage.**

View file

@ -0,0 +1,100 @@
.. mode: -*- rst -*-
Review checklist
================
:Tag: guide.developer
:Status: draft documentation
:Author: Gareth Rees
:Organization: Ravenbrook Limited
:Date: 2018-09-18
:Revision: $Id$
:Copyright: See section `Copyright and License`_.
:Index terms: pair: developer; guide
Introduction
------------
_`.intro`: This is an introduction to the Memory Pool System (MPS) for
new developers.
What to read first
------------------
* `manual/build.txt`_ -- how to build the MPS.
* `manual/guide`_ -- tutorial for the public interface.
* `manual/topic`_ -- reference manual for the public interface.
* `manual/code-index`_ -- description and purpose of each file of source code.
* design.mps.config_ -- build configuration.
* design.mps.tests_ -- how to run test cases.
* design.mps.doc_ -- how to write and edit documentation.
.. _manual/build.txt: https://www.ravenbrook.com/project/mps/master/manual/build.txt
.. _manual/guide: https://www.ravenbrook.com/project/mps/master/manual/html/guide/index.html
.. _manual/topic: https://www.ravenbrook.com/project/mps/master/manual/html/topic/index.html
.. _manual/code-index: https://www.ravenbrook.com/project/mps/master/manual/html/code-index.html
.. _design.mps.config: config
.. _design.mps.tests: tests
.. _design.mps.doc: doc
References
----------
.. [APT_2018-09-17]
"Procedure for new developers";
Alistair Turnbull; Ravenbrook Limited; 2018-09-17;
<https://info.ravenbrook.com/mail/2018/09/17/12-36-45/0/>
Document History
----------------
2018-09-18 GDR_ Created based on [APT_22018-09-17]_.
.. _GDR: http://www.ravenbrook.com/consultants/gdr/
Copyright and License
---------------------
Copyright © 2015-2016 Ravenbrook Limited <http://www.ravenbrook.com/>.
All rights reserved. This is an open source license. Contact
Ravenbrook for commercial licensing options.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
#. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
#. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
#. Redistributions in any form must be accompanied by information on how
to obtain complete source code for this software and any
accompanying software that uses this software. The source code must
either be included in the distribution or be available for no more than
the cost of distribution plus a nominal fee, and must be freely
redistributable under reasonable conditions. For an executable file,
complete source code means the source code for all modules it contains.
It does not include source code for modules or files that typically
accompany the major components of the operating system on which the
executable file runs.
**This software is provided by the copyright holders and contributors
"as is" and any express or implied warranties, including, but not
limited to, the implied warranties of merchantability, fitness for a
particular purpose, or non-infringement, are disclaimed. In no event
shall the copyright holders and contributors be liable for any direct,
indirect, incidental, special, exemplary, or consequential damages
(including, but not limited to, procurement of substitute goods or
services; loss of use, data, or profits; or business interruption)
however caused and on any theory of liability, whether in contract,
strict liability, or tort (including negligence or otherwise) arising in
any way out of the use of this software, even if advised of the
possibility of such damage.**

View file

@ -54,11 +54,13 @@ collection_ Collection framework
config_ MPS configuration
critical-path_ The critical path through the MPS
diag_ Diagnostic feedback
doc_ Documentation
exec-env_ Execution environment
failover_ Fail-over allocator
finalize_ Finalization
fix_ The generic fix function
freelist_ Free list allocator
guide.developer_ Guide for new developers
guide.hex.trans_ Transliterating the alphabet into hexadecimal
guide.impl.c.format_ Coding standard: conventions for the general format of C source code in the MPS
guide.impl.c.naming_ Coding standard: conventions for internal names
@ -127,11 +129,13 @@ writef_ The WriteF function
.. _config: config
.. _critical-path: critical-path
.. _diag: diag
.. _doc: doc
.. _exec-env: exec-env
.. _failover: failover
.. _finalize: finalize
.. _fix: fix
.. _freelist: freelist
.. _guide.developer: guide.developer
.. _guide.hex.trans: guide.hex.trans
.. _guide.impl.c.format: guide.impl.c.format
.. _guide.impl.c.naming: guide.impl.c.naming
@ -227,7 +231,7 @@ Document History
Copyright and License
---------------------
Copyright © 2002-2016 Ravenbrook Limited <http://www.ravenbrook.com/>.
Copyright © 2002-2018 Ravenbrook Limited <http://www.ravenbrook.com/>.
All rights reserved. This is an open source license. Contact
Ravenbrook for commercial licensing options.

View file

@ -21,27 +21,100 @@ tests.
_`.readership`: This document is intended for any MPS developer.
Smoke tests
-----------
Running tests
-------------
_`.smoke`: The "smoke tests" provide quick checks that the MPS is
working. They run quickly enough for it to be practical to run them
every time the MPS is built.
_`.run`: Run these commands::
cd code
make -f <makefile> VARIETY=<variety> <target> # Unix
nmake /f <makefile> VARIETY=<variety> <target> # Windows
where ``<makefile>`` is the appropriate makefile for the platform (see
manual/build.txt_), ``<variety>`` is the variety (see
design.mps.config.var_) and ``<target>`` is the collection of tests
(see `.target`_ below). For example::
make -f lii6ll VARIETY=cool testrun
If ``<variety>`` is omitted, tests are run in both the cool and hot
varieties.
.. _design.mps.config.var: config#var
.. _manual/build.txt: https://www.ravenbrook.com/project/mps/master/manual/build.txt
Test targets
------------
_`.target`: The makefiles provide the following targets for common
sets of tests:
_`.target.testall`: The ``testall`` target runs all test cases (even
if known to fail).
_`.target.testrun`: The ``testrun`` target runs the "smoke tests".
This subset of tests are quick checks that the MPS is working. They
run quickly enough for it to be practical to run them every time the
MPS is built.
_`.target.testci`: The ``testci`` target runs the continuous
integration tests (all tests that are known good according to the test database).
_`.target.testansi`: The ``testansi`` target runs the subset of the
smoke tests that can pass in the generic ("ANSI") build configuration
(see design.mps.config.opt.ansi_).
_`.target.testpollnone`: The ``testpollnone`` target runs the subset
of the smoke tests that can pass in the generic ("ANSI") build
configuration (see design.mps.config.opt.ansi_) with the option
``CONFIG_POLL_NONE`` (see design.mps.config.opt.poll_).
_`.target.testratio`: The ``testratio`` target compares the
performance of the HOT and RASH varieties. See `.ratio`_.
.. _design.mps.config.opt.ansi: config#opt.ansi
.. _design.mps.config.opt.poll: config#opt.poll
Test features
-------------
_`.randomize`: Each time a test case is run, it randomly chooses some
of its parameters (for example, the sizes of objects, or how many
links to create in a graph of references). This allows a fast test
to cover many cases over time.
_`.randomize.repeatable`: The random numbers are chosen
_`.randomize.seed`: The random numbers are chosen
pseudo-randomly based on a seed initialized from environmental data
(the time and the processor cycle count). The seed is reported at test
startup. Each test can be run with a specified seed. This ensures
that the deterministic tests are repeatable.
startup, for example::
$ xci6ll/cool/apss
xci6ll/cool/apss: randomize(): choosing initial state (v3): 2116709187.
...
xci6ll/cool/apss: Conclusion: Failed to find any defects.
Here, the number 2116709187 is the random seed.
_`.randomize.specific-seed` Each test can be run with a specified seed
by passing the seed on the command line, for example::
code$ xci6ll/cool/apss 2116709187
xci6ll/cool/apss: randomize(): resetting initial state (v3) to: 2116709187.
...
xci6ll/cool/apss: Conclusion: Failed to find any defects.
_`.randomize.repeatable`: This ensures that the deterministic tests
are repeatable.
Smoke test list
...............
Test list
---------
See `manual/code-index`_ for the full list of automated test cases.
.. _manual/code-index: https://www.ravenbrook.com/project/mps/master/manual/html/code-index.html
_`.test.finalcv`: Registers objects for finalization, makes them
unreachable, deregisters them, etc. Churns to provoke minor (nursery)
@ -56,10 +129,53 @@ _`.test.zcoll`: Collection scheduling, and collection feedback.
_`.test.zmess`: Message lifecycle and finalization messages.
Test database
-------------
_`.db`: The automated tests are described in the test database
(tool/testcases.txt).
_`.db.format`: This is a self-documenting plain-text database which
gives for each test case its name and an optional set of features. For
example the feature ``P`` means that the test case requires polling to
succeed, and therefore is expected to fail in build configurations
without polling (see design.mps.config.opt.poll_).
_`.db.format.simple`: The format must be very simple because the test
runner on Windows is written as a batch file (.bat), in order to avoid
having to depend on any tools that are not standard with the operating
system, and batch files are inflexible.
_`.db.testrun`: The test runner (tool/testrun.sh on Unix or
tool/testrun.bat on Windows) parses the test database to work out
which tests to run according to the target. For example the
``testpollnone`` target must skip all test cases with the ``P``
feature.
Test runner
-----------
_`.runner.req.automated`: The test runner must execute without user
interaction, so that it can be used for continuous integration.
_`.runner.req.output.pass`: Test cases are expected to pass nearly all the
time, and in these cases we almost never want to see the output, so
the test runner must suppress the output for passing tests.
_`.runner.req.output.fail`: However, if a test case fails then the
test runner must preserve the output from the failing test, including
the random seed (see `.randomize.seed`_), so that this can be analyzed
and the test repeated. Moreover, it must print the output from the
failing test, so that if the test is being run on a continuous
integration system like Travis, then the output of the failing tests
is included in the failure report.
Performance test
----------------
_`.test.ratio`: The ``testratio`` target checks that the hot variety
_`.ratio`: The ``testratio`` target checks that the hot variety
is not too much slower than the rash variety. A failure of this test
usually is expected to indicate that there are assertions on the
critical path using ``AVER`` instead of ``AVER_CRITICAL`` (and so on).
@ -68,12 +184,13 @@ the MVFF pool class, in the hot variety and the rash variety,
computing the ratio of CPU time taken in the two varieties, and
testing that this falls under an acceptable limit.
Note that we don't use the elapsed time (as reported by the benchmark)
because we want to be able to run this test on continuous integration
machines that might be heavily loaded.
_`.ratio.cpu-time`: Note that we use the CPU time (reported by
``/usr/bin/time``) and not the elapsed time (as reported by the
benchmark) because we want to be able to run this test on continuous
integration machines that might be heavily loaded.
This target is currently supported only on Unix platforms using GNU
Makefiles.
_`.ratio.platform`: This target is currently supported only on Unix
platforms using GNU Makefiles.
Adding a new smoke test
@ -141,6 +258,54 @@ _`.new.manual`: Edit manual/source/code-index.rst and add the new test
case to the "Automated test cases" section.
Continuous integration
----------------------
_`.ci`: The MPS uses the following systems for continuous integration:
_`.ci.travis`: The commercial Travis_ continuous integration service
runs ``./configure && make install && make test`` (which exercises the
``testci`` target for platforms lii6gc, lii6ll, xci6ll, and
``testansi`` and ``testpollnone`` for platforms anangc and ananll)
whenever there is a commit to the `mps GitHub repository`_. See the
`Ravenbrook/mps`_ project on Travis for a build history.
.. _mps GitHub repository: https://github.com/ravenbrook/mps
.. _Travis: https://travis-ci.org
.. _Ravenbrook/mps: https://travis-ci.org/Ravenbrook/mps
_`.ci.travis.config`: Travis is configured using the .travis.yml file
at top level of the project.
_`.ci.jenkins`: An instance of Jenkins_ running on "gannet" (a Windows
PC at the Ravenbrook office) runs the ``testci`` target on platforms
w3i3mv and w3i6mv whenever there is a commit to the `mps GitHub
repository`_.
_`.ci.jenkins.config`: There are instructions for installing and
configuring Jenkins in [GDR_2016-04-14]_, [GDR_2016-04-15]_ and
[GDR_2016-04-20]_.
References
----------
.. [GDR_2016-04-14]
"Jenkins setup and configuration procedure";
Gareth Rees; Ravenbrook Limited; 2016-04-14;
<https://info.ravenbrook.com/mail/2016/04/14/17-19-08/0/>.
.. [GDR_2016-04-15]
"Re: Jenkins setup and configuration procedure";
Gareth Rees; Ravenbrook Limited; 2016-04-15;
<https://info.ravenbrook.com/mail/2016/04/15/09-03-53/0/>.
.. [GDR_2016-04-20]
"Re: Jenkins setup and configuration procedure";
Gareth Rees; Ravenbrook Limited; 2016-04-20;
<https://info.ravenbrook.com/mail/2016/04/20/13-26-33/0/>.
Document History
----------------

View file

@ -13,10 +13,12 @@ Design
clock
config
critical-path
doc
exec-env
failover
finalize
freelist
guide.developer
guide.hex.trans
guide.impl.c.format
guide.impl.c.naming