emacs/mps/design/io.txt

.. mode: -*- rst -*-

I/O subsystem
=============

:Tag: design.mps.io
:Author: Richard Brooksby
:Date: 1996-08-30
:Status: incomplete design
:Revision: $Id$
:Copyright: See `Copyright and License`_.
:Index terms: pair: I/O subsystem; design


Introduction
------------

_`.intro`: This document is the design of the MPS I/O Subsystem, a
part of the plinth.

_`.readership`: This document is intended for MPS developers.


Background
----------

_`.bg`: This design is partly based on the design of the Internet User
Datagram Protocol (UDP). Mainly I used this to make sure I hadn't left
out anything which we might need.


Purpose
-------

_`.purpose`: The purpose of the MPS I/O Subsystem is to provide a
means to measure, debug, control, and test a memory manager build
using the MPS.

_`.purpose.measure`: Measurement consists of emitting data which can
be collected and analysed in order to improve the attributes of
application program, quite possibly by adjusting parameters of the
memory manager (see overview.mps.usage).

_`.purpose.control`: Control means adjusting the behaviour of the MM
dynamically. For example, one might want to adjust a parameter in
order to observe the effect, then transfer that adjustment to the
client application later.

_`.purpose.test`: Test output can be used to ensure that the memory
manager is behaving as expected in response to certain inputs.


Requirements
------------

General
.......

_`.req.fun.non-hosted`: The MPM must be a host-independent system.

_`.req.attr.host`: It should be easy for the client to set up the MPM
for a particular host (such as a washing machine).

Functional
..........

_`.req.fun.measure`: The subsystem must allow the MPS to transmit
quantitative measurement data to an external tool so that the system
can be tuned.

_`.req.fun.debug`: The subsystem must allow the MPS to transmit
qualitative information about its operation to an external tool so
that the system can be debugged.

_`.req.fun.control`: The subsystem must allow the MPS to receive
control information from an external tool so that the system can be
adjusted while it is running.

_`.req.dc.env.no-net`: The subsystem should operate in environments
where there is no networking available.

_`.req.dc.env.no-fs`: The subsystem should operate in environments
where there is no filesystem available.


Architecture
------------

_`.arch.diagram`: I/O Architecture Diagram

[missing diagram]

_`.arch.int`: The I/O Interface is a C function call interface by
which the MPM sends and receives "messages" to and from the hosted I/O
module.

_`.arch.module`: The modules are part of the MPS but not part of the
freestanding core system (see design.mps.exec-env). The I/O module is
responsible for transmitting those messages to the external tools, and
for receiving messages from external tools and passing them to the
MPM.

_`.arch.module.example`: For example, the "file implementation" might
just send/write telemetry messages into a file so that they can be
received/read later by an off-line measurement tool.

_`.arch.external`: The I/O Interface is part of interface to the
freestanding core system (see design.mps.exec-env). This is so that
the MPS can be deployed in a freestanding environment, with a special
I/O module. For example, if the MPS is used in a washing machine the
I/O module could communicate by writing output to the seven-segment
display.


Example configurations
......................

_`.example.telnet`: This shows the I/O subsystem communicating with a
telnet client over a TCP/IP connection. In this case, the I/O
subsystem is translating the I/O Interface into an interactive text
protocol so that the user of the telnet client can talk to the MM.

[missing diagram]

_`.example.file`: This shows the I/O subsystem dumping measurement
data into a file which is later read and analysed. In this case the
I/O subsystem is simply writing out binary in a format which can be
decoded.

[missing diagram]

_`.example.serial`: This shows the I/O subsystem communicating with a
graphical analysis tool over a serial link. This could be useful for a
developer who has two machines in close proximity and no networking
support.

_`.example.local`: In this example the application is talking directly
to the I/O subsystem. This is useful when the application is a
reflective development environment (such as MLWorks) which wants to
observe its own behaviour.

[missing diagram]


Interface
---------

_`.if.msg`: The I/O interface is oriented around opaque binary
"messages" which the I/O module must pass between the MPM and external
tools. The I/O module need not understand or interpret the contents of
those messages.

_`.if.msg.opaque`: The messages are opaque in order to minimize the
dependency of the I/O module on the message internals. It should be
possible for clients to implement their own I/O modules for unusual
environments. We do not want to reveal the internal structure of our
data to the clients. Nor do we want to burden them with the details of
our protocols. We'd also like their code to be independent of ours, so
that we can expand or change the protocols without requiring them to
modify their modules.

_`.if.msg.dgram`: Neither the MPM nor the external tools should assume
that the messages will be delivered in finite time, exactly once, or
in order. This will allow the I/O modules to be implemented using
unreliable transport layers such as the Internet User Datagram Protocl
(UDP). It will also give the I/O module the freedom to drop
information rather than block on a congested network, or stop the
memory manager when the disk is full, or similar events which really
shouldn't cause the memory manager to stop working. The protocols we
need to implement at the high level can be design to be robust against
lossage without much difficulty.


I/O module state
................

_`.if.state`: The I/O module may have some internal state to preserve.
The I/O Interface defines a type for this state, ``mps_io_t``, a
pointer to an incomplete structure ``mps_io_s``. The I/O module is at
liberty to define this structure.


Message types
.............

_`.if.type`: The I/O module must be able to deliver messages of
several different types. It will probably choose to send them to
different destinations based on their type: telemetry to the
measurement tool, debugging output to the debugger, etc. ::

    typedef int mps_io_type_t;
    enum {
      MPS_IO_TYPE_TELEMETRY,
      MPS_IO_TYPE_DEBUG
    };


Limits
......

_`.if.message-max`: The interface will define an unsigned integral
constant ``MPS_IO_MESSAGE_MAX`` which will be the maximum size of
messages that the MPM will pass to ``mps_io_send()`` (`.if.send`_) and
the maximum size it will expect to receive from ``mps_io_receive()``.


Interface set-up and tear-down
..............................

_`.if.create`: The MPM will call ``mps_io_create()`` to set up the I/O
module. On success, this function should return ``MPS_RES_OK``. It may
also initialize a "state" value which will be passed to subsequent
calls through the interface.

_`.if.destroy`: The MPM will call ``mps_io_destroy()`` to tear down
the I/O module, after which it guarantees that the state value will
not be used again. The ``state`` parameter is the state previously
returned by ``mps_io_create()`` (`.if.create`_).

Message send and receive
........................

``extern mps_res_t mps_io_send(mps_io_t state, mps_io_type_t type, void *message, size_t size)``

_`.if.send`: The MPM will call ``mps_io_send()`` when it wishes to
send a message to a destination. The ``state`` parameter is the state
previously returned by ``mps_io_create()`` (`.if.create`_). The
``type`` parameter is the type (`.if.type`_) of the message. The
``message`` parameter is a pointer to a buffer containing the message,
and ``size`` is the length of that message, in bytes. The I/O module
must make an effort to deliver the message to the destination, but is
not expected to guarantee delivery. The function should return
``MPS_RES_IO`` only if a serious error occurs that should cause the
MPM to return with an error to the client application. Failure to
deliver the message does not count.

.. note::

    Should there be a timeout parameter? What are the timing
    constraints? ``mps_io_send()`` shouldn't block.

``extern mps_res_t mps_io_receive(mps_io_t state, void **buffer_o, size_t *size_o)``

_`.if.receive`: The MPM will call ``mps_io_receive()`` when it wants
to see if a message has been sent to it. The ``state`` parameter is
the state previously returned by ``mps_io_create()`` (`.if.create`_).
The ``buffer_o`` parameter is a pointer to a value which should be
updated with a pointer to a buffer containing the message received.
The ``size_o`` parameter is a pointer to a value which should be
updated with the length of the message received. If there is no
message ready for receipt, the length returned should be zero.

.. note::

    Should we be able to receive truncated messages? How can this be
    done neatly?


I/O module implementations
--------------------------

Routeing
........

The I/O module must decide where to send the various messages. A
file-based implementation could put them in different files based on
their types. A network-based implementation must decide how to address
the messages. In either case, any configuration must either be
statically compiled into the module, or else read from some external
source such as a configuration file.


Notes
-----

The external tools should be able to reconstruct stuff from partial
info. For example, you come across a fragment of an old log containing
just a few old messages. What can you do with it?

Here's some completely untested code which might do the job for UDP.
::

    #include "mpsio.h"

    #include <sys/types.h>
    #include <sys/socket.h>
    #include <netinet/in.h>
    #include <arpa/inet.h>
    #include <fcntl.h>
    #include <errno.h>

    typedef struct mps_io_s {
      int sock;
      struct sockaddr_in mine;
      struct sockaddr_in telemetry;
      struct sockaddr_in debugging;
    } mps_io_s;

    static mps_bool_t inited = 0;
    static mps_io_s state;


    mps_res_t mps_io_create(mps_io_t *mps_io_o)
    {
      int sock, r;

      if(inited)
        return MPS_RES_LIMIT;

      state.mine = /* setup somehow from config */;
      state.telemetry = /* setup something from config */;
      state.debugging = /* setup something from config */;

      /* Make a socket through which to communicate. */
      sock = socket(AF_INET, SOCK_DGRAM, 0);
      if(sock == -1) return MPS_RES_IO;

      /* Set socket to non-blocking mode. */
      r = fcntl(sock, F_SETFL, O_NDELAY);
      if(r == -1) return MPS_RES_IO;

      /* Bind the socket to some UDP port so that we can receive messages. */
      r = bind(sock, (struct sockaddr *)&state.mine, sizeof(state.mine));
      if(r == -1) return MPS_RES_IO;

      state.sock = sock;

      inited = 1;

      *mps_io_o = &state;
      return MPS_RES_OK;
    }


    void mps_io_destroy(mps_io_t mps_io)
    {
      assert(mps_io == &state);
      assert(inited);

      (void)close(state.sock);

      inited = 0;
    }


    mps_res_t mps_io_send(mps_io_t mps_io, mps_type_t type,
                          void *message, size_t size)
    {
      struct sockaddr *toaddr;

      assert(mps_io == &state);
      assert(inited);

      switch(type) {
        MPS_IO_TYPE_TELEMETRY:
        toaddr = (struct sockaddr *)&state.telemetry;
        break;

        MPS_IO_TYPE_DEBUGGING:
        toaddr = (struct sockaddr *)&state.debugging;
        break;

        default:
        assert(0);
        return MPS_RES_UNIMPL;
      }

      (void)sendto(state.sock, message, size, 0, toaddr, sizeof(*toaddr));

      return MPS_RES_OK;
    }


    mps_res_t mps_io_receive(mps_io_t mps_io,
                             void **message_o, size_t **size_o)
    {
      int r;
      static char buffer[MPS_IO_MESSAGE_MAX];

      assert(mps_io == &state);
      assert(inited);

      r = recvfrom(state.sock, buffer, sizeof(buffer), 0, NULL, NULL);
      if(r == -1)
        switch(errno) {
          /* Ignore interrupted system calls, and failures due to lack */
          /* of resources (they might go away.) */
          case EINTR: case ENOMEM: case ENOSR:
          r = 0;
          break;

          default:
          return MPS_RES_IO;
        }

      *message_o = buffer;
      *size_o = r;
      return MPS_RES_OK;
    }


Attachments
-----------

"O Architecture Diagram"
"O Configuration Diagrams"


Document History
----------------

- 1996-08-30 RB_ Document created from paper notes.

- 1997-06-10 RB_ Updated with mail.richard.1997-05-30.16-13 and
  subsequent discussion in the Pool Hall at Longstanton. (See also
  mail.drj.1997-06-05.15-20.)

- 2002-06-07 RB_ Converted from MMInfo database design document.

- 2013-05-23 GDR_ Converted to reStructuredText.

.. _RB: http://www.ravenbrook.com/consultants/rb/
.. _GDR: http://www.ravenbrook.com/consultants/gdr/


Copyright and License
---------------------

Copyright © 2013-2014 Ravenbrook Limited. All rights reserved.
<http://www.ravenbrook.com/>. This is an open source license. Contact
Ravenbrook for commercial licensing options.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:

#. Redistributions of source code must retain the above copyright
   notice, this list of conditions and the following disclaimer.

#. Redistributions in binary form must reproduce the above copyright
   notice, this list of conditions and the following disclaimer in the
   documentation and/or other materials provided with the distribution.

#. Redistributions in any form must be accompanied by information on how
   to obtain complete source code for this software and any
   accompanying software that uses this software.  The source code must
   either be included in the distribution or be available for no more than
   the cost of distribution plus a nominal fee, and must be freely
   redistributable under reasonable conditions.  For an executable file,
   complete source code means the source code for all modules it contains.
   It does not include source code for modules or files that typically
   accompany the major components of the operating system on which the
   executable file runs.

**This software is provided by the copyright holders and contributors
"as is" and any express or implied warranties, including, but not
limited to, the implied warranties of merchantability, fitness for a
particular purpose, or non-infringement, are disclaimed.  In no event
shall the copyright holders and contributors be liable for any direct,
indirect, incidental, special, exemplary, or consequential damages
(including, but not limited to, procurement of substitute goods or
services; loss of use, data, or profits; or business interruption)
however caused and on any theory of liability, whether in contract,
strict liability, or tort (including negligence or otherwise) arising in
any way out of the use of this software, even if advised of the
possibility of such damage.**