mirrors/emacs
1
Fork 0
mirror of git://git.sv.gnu.org/emacs.git synced 2025-12-25 23:10:47 -08:00
Code Issues Projects Releases Wiki Activity
emacs/mps/manual/html/guide/advanced.html
Gareth Rees 2ed2ff1893 Updated manual html 
Copied from Perforce
 Change: 181750
 ServerID: perforce.ravenbrook.com
2013-05-13 21:04:49 +01:00

936 lines
No EOL
92 KiB
HTML
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>6. Advanced topics &mdash; Memory Pool System 1.111.0 documentation</title>
<link rel="stylesheet" href="../_static/mps.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../',
VERSION: '1.111.0',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="../_static/jquery.js"></script>
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<link rel="copyright" title="Copyright" href="../copyright.html" />
<link rel="top" title="Memory Pool System 1.111.0 documentation" href="../index.html" />
<link rel="up" title="Guide" href="index.html" />
<link rel="next" title="Reference" href="../topic/index.html" />
<link rel="prev" title="5. Tuning the Memory Pool System for performance" href="perf.html" />
</head>
<body>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="../topic/index.html" title="Reference"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="perf.html" title="5. Tuning the Memory Pool System for performance"
accesskey="P">previous</a> |</li>
<li><a href="../index.html">Memory Pool System 1.111.0 documentation</a> &raquo;</li>
<li><a href="index.html" accesskey="U">Guide</a> &raquo;</li>
</ul>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body">
<div class="section" id="advanced-topics">
<span id="guide-advanced"></span><h1>6. Advanced topics<a class="headerlink" href="#advanced-topics" title="Permalink to this headline"></a></h1>
<div class="section" id="finalization">
<span id="index-0"></span><h2>6.1. Finalization<a class="headerlink" href="#finalization" title="Permalink to this headline"></a></h2>
<p>In Scheme, an open file is represented by a <em>port</em>. In the toy Scheme
interpreter, a port is a wrapper around a standard C file handle:</p>
<div class="highlight-c"><div class="highlight"><pre><span class="k">typedef</span> <span class="k">struct</span> <span class="n">port_s</span> <span class="p">{</span>
<span class="n">type_t</span> <span class="n">type</span><span class="p">;</span> <span class="cm">/* TYPE_PORT */</span>
<span class="n">obj_t</span> <span class="n">name</span><span class="p">;</span> <span class="cm">/* name of stream */</span>
<span class="kt">FILE</span> <span class="o">*</span><span class="n">stream</span><span class="p">;</span>
<span class="p">}</span> <span class="n">port_s</span><span class="p">;</span>
</pre></div>
</div>
<p>Operating systems limit the number of files that a process can have open
simultaneously, so to avoid running out of file handles, it is necessary
to close ports when you are done with them. If a Scheme program fails to
call <tt class="docutils literal"><span class="pre">close-input-file</span></tt>, then the underlying file handle should still
be closed when the port object <a class="reference internal" href="../glossary/d.html#term-dead"><em class="xref std std-term">dies</em></a>. This procedure is
known as <a class="reference internal" href="../glossary/f.html#term-finalization"><em class="xref std std-term">finalization</em></a>.</p>
<div class="admonition-note admonition">
<p class="first admonition-title">Note</p>
<p class="last">It&#8217;s generally a bad idea to depend on finalization to release your
resources (see the <a class="reference internal" href="../topic/finalization.html#topic-finalization-cautions"><em>Cautions</em></a> section in
<a class="reference internal" href="../topic/finalization.html#topic-finalization"><em>Finalization</em></a>). Treat it as a last resort when more
reliable mechanisms for releasing resources (like Scheme&#8217;s
<tt class="docutils literal"><span class="pre">with-open-input-file</span></tt>) aren&#8217;t available.</p>
</div>
<p>Any block in an <a class="reference internal" href="../glossary/a.html#term-automatic-memory-management"><em class="xref std std-term">automatically managed</em></a> <a class="reference internal" href="../glossary/p.html#term-pool"><em class="xref std std-term">pool</em></a> can be registered for finalization by calling
<a class="reference internal" href="../topic/finalization.html#mps_finalize" title="mps_finalize"><tt class="xref c c-func docutils literal"><span class="pre">mps_finalize()</span></tt></a>. In the toy Scheme interpreter, this can be done
in <tt class="docutils literal"><span class="pre">make_port</span></tt>:</p>
<div class="highlight-c"><div class="highlight"><pre> <span class="k">static</span> <span class="n">obj_t</span> <span class="nf">make_port</span><span class="p">(</span><span class="n">obj_t</span> <span class="n">name</span><span class="p">,</span> <span class="kt">FILE</span> <span class="o">*</span><span class="n">stream</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">mps_addr_t</span> <span class="n">port_ref</span><span class="p">;</span>
<span class="n">obj_t</span> <span class="n">obj</span><span class="p">;</span>
<span class="n">mps_addr_t</span> <span class="n">addr</span><span class="p">;</span>
<span class="kt">size_t</span> <span class="n">size</span> <span class="o">=</span> <span class="n">ALIGN</span><span class="p">(</span><span class="k">sizeof</span><span class="p">(</span><span class="n">port_s</span><span class="p">));</span>
<span class="k">do</span> <span class="p">{</span>
<span class="n">mps_res_t</span> <span class="n">res</span> <span class="o">=</span> <span class="n">mps_reserve</span><span class="p">(</span><span class="o">&amp;</span><span class="n">addr</span><span class="p">,</span> <span class="n">obj_ap</span><span class="p">,</span> <span class="n">size</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="n">res</span> <span class="o">!=</span> <span class="n">MPS_RES_OK</span><span class="p">)</span> <span class="n">error</span><span class="p">(</span><span class="s">&quot;out of memory in make_port&quot;</span><span class="p">);</span>
<span class="n">obj</span> <span class="o">=</span> <span class="n">addr</span><span class="p">;</span>
<span class="n">obj</span><span class="o">-&gt;</span><span class="n">port</span><span class="p">.</span><span class="n">type</span> <span class="o">=</span> <span class="n">TYPE_PORT</span><span class="p">;</span>
<span class="n">obj</span><span class="o">-&gt;</span><span class="n">port</span><span class="p">.</span><span class="n">name</span> <span class="o">=</span> <span class="n">name</span><span class="p">;</span>
<span class="n">obj</span><span class="o">-&gt;</span><span class="n">port</span><span class="p">.</span><span class="n">stream</span> <span class="o">=</span> <span class="n">stream</span><span class="p">;</span>
<span class="p">}</span> <span class="k">while</span><span class="p">(</span><span class="o">!</span><span class="n">mps_commit</span><span class="p">(</span><span class="n">obj_ap</span><span class="p">,</span> <span class="n">addr</span><span class="p">,</span> <span class="n">size</span><span class="p">));</span>
<span class="n">total</span> <span class="o">+=</span> <span class="k">sizeof</span><span class="p">(</span><span class="n">port_s</span><span class="p">);</span>
<span class="n">port_ref</span> <span class="o">=</span> <span class="n">obj</span><span class="p">;</span>
<span class="hll"> <span class="n">mps_finalize</span><span class="p">(</span><span class="n">arena</span><span class="p">,</span> <span class="o">&amp;</span><span class="n">port_ref</span><span class="p">);</span>
</span>
<span class="k">return</span> <span class="n">obj</span><span class="p">;</span>
<span class="p">}</span>
</pre></div>
</div>
<p>The MPS implements finalization by posting a <a class="reference internal" href="../glossary/m.html#term-message"><em class="xref std std-term">message</em></a> to the
arena&#8217;s <a class="reference internal" href="../glossary/m.html#term-message-queue"><em class="xref std std-term">message queue</em></a> when an object that has been registered
for finalization is about to die.</p>
<p>If you want to finalize your objects, you must first enable
finalization messages by calling <a class="reference internal" href="../topic/message.html#mps_message_type_enable" title="mps_message_type_enable"><tt class="xref c c-func docutils literal"><span class="pre">mps_message_type_enable()</span></tt></a>:</p>
<div class="highlight-c"><div class="highlight"><pre><span class="n">mps_message_type_enable</span><span class="p">(</span><span class="n">arena</span><span class="p">,</span> <span class="n">mps_message_type_finalization</span><span class="p">());</span>
</pre></div>
</div>
<p>You must then poll the arena&#8217;s message queue at times that are
convenient for you, call <a class="reference internal" href="../topic/message.html#mps_message_get" title="mps_message_get"><tt class="xref c c-func docutils literal"><span class="pre">mps_message_get()</span></tt></a> to pick up a
finalization message from the queue, call
<a class="reference internal" href="../topic/finalization.html#mps_message_finalization_ref" title="mps_message_finalization_ref"><tt class="xref c c-func docutils literal"><span class="pre">mps_message_finalization_ref()</span></tt></a> to access the finalized object,
and finally call <a class="reference internal" href="../topic/message.html#mps_message_discard" title="mps_message_discard"><tt class="xref c c-func docutils literal"><span class="pre">mps_message_discard()</span></tt></a> on the finalization
message. The finalized object is then subject to the normal rules of
life and death: it continues to live as long as it is strongly
reachable.</p>
<p>In the toy Scheme interpreter, the most convenient moment to process the
message queue is at the start of the read–eval–print loop. When a
finalization message is found, the associated file handle is closed
(unless it has been closed already), and the message is discarded.</p>
<div class="highlight-c"><div class="highlight"><pre> <span class="n">mps_message_type_t</span> <span class="n">type</span><span class="p">;</span>
<span class="k">while</span> <span class="p">(</span><span class="n">mps_message_queue_type</span><span class="p">(</span><span class="o">&amp;</span><span class="n">type</span><span class="p">,</span> <span class="n">arena</span><span class="p">))</span> <span class="p">{</span>
<span class="n">mps_message_t</span> <span class="n">message</span><span class="p">;</span>
<span class="n">mps_bool_t</span> <span class="n">b</span><span class="p">;</span>
<span class="n">b</span> <span class="o">=</span> <span class="n">mps_message_get</span><span class="p">(</span><span class="o">&amp;</span><span class="n">message</span><span class="p">,</span> <span class="n">arena</span><span class="p">,</span> <span class="n">type</span><span class="p">);</span>
<span class="n">assert</span><span class="p">(</span><span class="n">b</span><span class="p">);</span> <span class="cm">/* we just checked there was one */</span>
<span class="hll"> <span class="k">if</span> <span class="p">(</span><span class="n">type</span> <span class="o">==</span> <span class="n">mps_message_type_finalization</span><span class="p">())</span> <span class="p">{</span>
</span> <span class="n">mps_addr_t</span> <span class="n">port_ref</span><span class="p">;</span>
<span class="n">obj_t</span> <span class="n">port</span><span class="p">;</span>
<span class="hll"> <span class="n">mps_message_finalization_ref</span><span class="p">(</span><span class="o">&amp;</span><span class="n">port_ref</span><span class="p">,</span> <span class="n">arena</span><span class="p">,</span> <span class="n">message</span><span class="p">);</span>
</span> <span class="n">port</span> <span class="o">=</span> <span class="n">port_ref</span><span class="p">;</span>
<span class="n">assert</span><span class="p">(</span><span class="n">TYPE</span><span class="p">(</span><span class="n">port</span><span class="p">)</span> <span class="o">==</span> <span class="n">TYPE_PORT</span><span class="p">);</span>
<span class="k">if</span><span class="p">(</span><span class="n">port</span><span class="o">-&gt;</span><span class="n">port</span><span class="p">.</span><span class="n">stream</span><span class="p">)</span> <span class="p">{</span>
<span class="n">printf</span><span class="p">(</span><span class="s">&quot;Port to file </span><span class="se">\&quot;</span><span class="s">%s</span><span class="se">\&quot;</span><span class="s"> is dying. Closing file.</span><span class="se">\n</span><span class="s">&quot;</span><span class="p">,</span>
<span class="n">port</span><span class="o">-&gt;</span><span class="n">port</span><span class="p">.</span><span class="n">name</span><span class="o">-&gt;</span><span class="n">string</span><span class="p">.</span><span class="n">string</span><span class="p">);</span>
<span class="p">(</span><span class="kt">void</span><span class="p">)</span><span class="n">fclose</span><span class="p">(</span><span class="n">port</span><span class="o">-&gt;</span><span class="n">port</span><span class="p">.</span><span class="n">stream</span><span class="p">);</span>
<span class="n">port</span><span class="o">-&gt;</span><span class="n">port</span><span class="p">.</span><span class="n">stream</span> <span class="o">=</span> <span class="nb">NULL</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="cm">/* ... handle other message types ... */</span>
<span class="p">}</span>
<span class="hll"> <span class="n">mps_message_discard</span><span class="p">(</span><span class="n">arena</span><span class="p">,</span> <span class="n">message</span><span class="p">);</span>
</span> <span class="p">}</span>
</pre></div>
</div>
<p>Here&#8217;s an example session showing finalization taking place:</p>
<div class="highlight-none"><div class="highlight"><pre> MPS Toy Scheme Example
9960, 0&gt; (open-input-file &quot;scheme.c&quot;)
#[port &quot;scheme.c&quot;]
10064, 0&gt; (gc)
Collection started.
Why: Client requests: immediate full collection.
Clock: 3401
<span class="hll"> Port to file &quot;scheme.c&quot; is dying. Closing file.
</span> Collection finished.
live 10040
condemned 10088
not_condemned 0
clock: 3807
</pre></div>
</div>
<p>The toy Scheme interpreter <em class="dfn">definalizes</em> ports by calling
<a class="reference internal" href="../topic/finalization.html#mps_definalize" title="mps_definalize"><tt class="xref c c-func docutils literal"><span class="pre">mps_definalize()</span></tt></a> when they are closed. This is purely an
optimization: setting <tt class="docutils literal"><span class="pre">stream</span></tt> to <tt class="docutils literal"><span class="pre">NULL</span></tt> ensures that the file
handle wouldn&#8217;t be closed more than once, even if the port object were
later finalized.</p>
<div class="highlight-c"><div class="highlight"><pre><span class="k">static</span> <span class="kt">void</span> <span class="nf">port_close</span><span class="p">(</span><span class="n">obj_t</span> <span class="n">port</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">assert</span><span class="p">(</span><span class="n">TYPE</span><span class="p">(</span><span class="n">port</span><span class="p">)</span> <span class="o">==</span> <span class="n">TYPE_PORT</span><span class="p">);</span>
<span class="k">if</span><span class="p">(</span><span class="n">port</span><span class="o">-&gt;</span><span class="n">port</span><span class="p">.</span><span class="n">stream</span> <span class="o">!=</span> <span class="nb">NULL</span><span class="p">)</span> <span class="p">{</span>
<span class="n">mps_addr_t</span> <span class="n">port_ref</span> <span class="o">=</span> <span class="n">port</span><span class="p">;</span>
<span class="n">fclose</span><span class="p">(</span><span class="n">port</span><span class="o">-&gt;</span><span class="n">port</span><span class="p">.</span><span class="n">stream</span><span class="p">);</span>
<span class="n">port</span><span class="o">-&gt;</span><span class="n">port</span><span class="p">.</span><span class="n">stream</span> <span class="o">=</span> <span class="nb">NULL</span><span class="p">;</span>
<span class="hll"> <span class="n">mps_definalize</span><span class="p">(</span><span class="n">arena</span><span class="p">,</span> <span class="o">&amp;</span><span class="n">port_ref</span><span class="p">);</span>
</span> <span class="p">}</span>
<span class="p">}</span>
</pre></div>
</div>
<p>It&#8217;s still possible that the toy Scheme interpreter might run out of
open file handles despite having some or all of its port objects being
finalizable. That&#8217;s because the arena&#8217;s message queue is only polled
after evaluating an expression at top level: if the expression itself
opens too many file handles, the finalization messages will queue up and
not be processed in time. For example:</p>
<div class="highlight-none"><div class="highlight"><pre>MPS Toy Scheme Example
9960, 0&gt; (define (repeat n f _) (if (eqv? n 0) &#39;() (repeat (- n 1) f (f))))
repeat
10840, 0&gt; (repeat 300 (lambda () (open-input-file &quot;scheme.c&quot;)) 0)
open-input-file: cannot open input file
</pre></div>
</div>
<p>A less naïve interpreter might process finalization messages on a more
regular schedule, or might take emergency action in the event of running
out of open file handles by carrying out a full garbage collection and
processing any finalization messages that are posted as a result.</p>
<div class="admonition-topic admonition">
<p class="first admonition-title">Topics</p>
<p class="last"><a class="reference internal" href="../topic/finalization.html#topic-finalization"><em>Finalization</em></a>, <a class="reference internal" href="../topic/message.html#topic-message"><em>Messages</em></a>.</p>
</div>
</div>
<div class="section" id="location-dependency">
<span id="guide-advanced-location"></span><span id="index-1"></span><h2>6.2. Location dependency<a class="headerlink" href="#location-dependency" title="Permalink to this headline"></a></h2>
<p>The toy Scheme interpreter contains an address-based (<tt class="docutils literal"><span class="pre">eq?</span></tt>) hash
table implementation. It hashes the addresses of its keys, and so needs
to take account of the possibility that a <a class="reference internal" href="../glossary/m.html#term-moving-garbage-collector"><em class="xref std std-term">moving garbage
collector</em></a> might move the keys. If it fails to take account of this, the
hash table might become invalid after a garbage collection.</p>
<p>In the interaction shown below (with a naïve version of the code) you&#8217;ll
see that although the keys remain present in the table after garbage
collection, they cannot be found. This is because their locations (and
hence their hashes) have changed, but their positions in the table have
not been updated to match.</p>
<div class="highlight-none"><div class="highlight"><pre>MPS Toy Scheme Example
10240, 0&gt; (define ht (make-eq-hashtable))
ht
10584, 0&gt; (hashtable-set! ht &#39;one 1)
10768, 0&gt; (hashtable-set! ht &#39;two 2)
10952, 0&gt; (hashtable-set! ht &#39;three 3)
11136, 0&gt; ht
#[hashtable (two 2) (three 3) (one 1)]
11136, 0&gt; (hashtable-ref ht &#39;two #f)
2
11280, 0&gt; (gc)
11304, 1&gt; (hashtable-ref ht &#39;one #f)
#f
11448, 1&gt; (hashtable-ref ht &#39;two #f)
#f
11592, 1&gt; (hashtable-ref ht &#39;three #f)
#f
11736, 1&gt; ht
#[hashtable (two 2) (three 3) (one 1)]
</pre></div>
</div>
<p>The MPS solves this problem with its <em class="dfn">location dependency</em> feature:
a structure of type <a class="reference internal" href="../topic/location.html#mps_ld_s" title="mps_ld_s"><tt class="xref c c-type docutils literal"><span class="pre">mps_ld_s</span></tt></a> encapsulates a set of
dependencies on the locations of blocks. You add addresses to the
location dependency, and then test to see if it has been made
<em class="dfn">stale</em>: that is, if any of the blocks whose location has been
depended on might have moved since their location was depended upon.</p>
<p>You need to provide space for the <a class="reference internal" href="../topic/location.html#mps_ld_s" title="mps_ld_s"><tt class="xref c c-type docutils literal"><span class="pre">mps_ld_s</span></tt></a> structure. In the
case of a hash table, it is most convenient to inline it in the hash
table&#8217;s metadata:</p>
<div class="highlight-c"><div class="highlight"><pre><span class="k">typedef</span> <span class="k">struct</span> <span class="n">table_s</span> <span class="p">{</span>
<span class="n">type_t</span> <span class="n">type</span><span class="p">;</span> <span class="cm">/* TYPE_TABLE */</span>
<span class="n">hash_t</span> <span class="n">hash</span><span class="p">;</span> <span class="cm">/* hash function */</span>
<span class="n">cmp_t</span> <span class="n">cmp</span><span class="p">;</span> <span class="cm">/* comparison function */</span>
<span class="hll"> <span class="n">mps_ld_s</span> <span class="n">ld</span><span class="p">;</span> <span class="cm">/* location dependency */</span>
</span> <span class="n">obj_t</span> <span class="n">buckets</span><span class="p">;</span> <span class="cm">/* hash buckets */</span>
<span class="p">}</span> <span class="n">table_s</span><span class="p">;</span>
</pre></div>
</div>
<p>Before being used, the location dependency must be reset to indicate
that nothing is depended upon, by calling <a class="reference internal" href="../topic/location.html#mps_ld_reset" title="mps_ld_reset"><tt class="xref c c-func docutils literal"><span class="pre">mps_ld_reset()</span></tt></a>.</p>
<p>For example:</p>
<div class="highlight-c"><div class="highlight"><pre><span class="k">static</span> <span class="n">obj_t</span> <span class="nf">make_table</span><span class="p">(</span><span class="kt">size_t</span> <span class="n">length</span><span class="p">,</span> <span class="n">hash_t</span> <span class="n">hashf</span><span class="p">,</span> <span class="n">cmp_t</span> <span class="n">cmpf</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">obj_t</span> <span class="n">obj</span><span class="p">;</span>
<span class="n">mps_addr_t</span> <span class="n">addr</span><span class="p">;</span>
<span class="kt">size_t</span> <span class="n">l</span><span class="p">,</span> <span class="n">size</span> <span class="o">=</span> <span class="n">ALIGN</span><span class="p">(</span><span class="k">sizeof</span><span class="p">(</span><span class="n">table_s</span><span class="p">));</span>
<span class="k">do</span> <span class="p">{</span>
<span class="n">mps_res_t</span> <span class="n">res</span> <span class="o">=</span> <span class="n">mps_reserve</span><span class="p">(</span><span class="o">&amp;</span><span class="n">addr</span><span class="p">,</span> <span class="n">obj_ap</span><span class="p">,</span> <span class="n">size</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="n">res</span> <span class="o">!=</span> <span class="n">MPS_RES_OK</span><span class="p">)</span> <span class="n">error</span><span class="p">(</span><span class="s">&quot;out of memory in make_table&quot;</span><span class="p">);</span>
<span class="n">obj</span> <span class="o">=</span> <span class="n">addr</span><span class="p">;</span>
<span class="n">obj</span><span class="o">-&gt;</span><span class="n">table</span><span class="p">.</span><span class="n">type</span> <span class="o">=</span> <span class="n">TYPE_TABLE</span><span class="p">;</span>
<span class="n">obj</span><span class="o">-&gt;</span><span class="n">table</span><span class="p">.</span><span class="n">buckets</span> <span class="o">=</span> <span class="nb">NULL</span><span class="p">;</span>
<span class="p">}</span> <span class="k">while</span><span class="p">(</span><span class="o">!</span><span class="n">mps_commit</span><span class="p">(</span><span class="n">obj_ap</span><span class="p">,</span> <span class="n">addr</span><span class="p">,</span> <span class="n">size</span><span class="p">));</span>
<span class="n">total</span> <span class="o">+=</span> <span class="n">size</span><span class="p">;</span>
<span class="n">obj</span><span class="o">-&gt;</span><span class="n">table</span><span class="p">.</span><span class="n">hash</span> <span class="o">=</span> <span class="n">hashf</span><span class="p">;</span>
<span class="n">obj</span><span class="o">-&gt;</span><span class="n">table</span><span class="p">.</span><span class="n">cmp</span> <span class="o">=</span> <span class="n">cmpf</span><span class="p">;</span>
<span class="cm">/* round up to next power of 2 */</span>
<span class="k">for</span><span class="p">(</span><span class="n">l</span> <span class="o">=</span> <span class="mi">1</span><span class="p">;</span> <span class="n">l</span> <span class="o">&lt;</span> <span class="n">length</span><span class="p">;</span> <span class="n">l</span> <span class="o">*=</span> <span class="mi">2</span><span class="p">);</span>
<span class="n">obj</span><span class="o">-&gt;</span><span class="n">table</span><span class="p">.</span><span class="n">buckets</span> <span class="o">=</span> <span class="n">make_buckets</span><span class="p">(</span><span class="n">l</span><span class="p">);</span>
<span class="hll"> <span class="n">mps_ld_reset</span><span class="p">(</span><span class="o">&amp;</span><span class="n">obj</span><span class="o">-&gt;</span><span class="n">table</span><span class="p">.</span><span class="n">ld</span><span class="p">,</span> <span class="n">arena</span><span class="p">);</span>
</span> <span class="k">return</span> <span class="n">obj</span><span class="p">;</span>
<span class="p">}</span>
</pre></div>
</div>
<p><em>Before</em> the hash table becomes dependent on the location of a block,
the address of the block must be added to its location dependency by
calling <a class="reference internal" href="../topic/location.html#mps_ld_add" title="mps_ld_add"><tt class="xref c c-func docutils literal"><span class="pre">mps_ld_add()</span></tt></a>. In particular, you must call
<a class="reference internal" href="../topic/location.html#mps_ld_add" title="mps_ld_add"><tt class="xref c c-func docutils literal"><span class="pre">mps_ld_add()</span></tt></a> before computing the hash of the address. (If you
wait until afterwards, it might be too late: a garbage collection might
have taken place after the hash was computed but before you added the
dependency.)</p>
<p>In the toy Scheme interpreter, this is done just before the computation
of the hash of the address.</p>
<div class="highlight-c"><div class="highlight"><pre><span class="k">static</span> <span class="kt">unsigned</span> <span class="kt">long</span> <span class="nf">eq_hash</span><span class="p">(</span><span class="n">obj_t</span> <span class="n">obj</span><span class="p">,</span> <span class="n">mps_ld_t</span> <span class="n">ld</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">union</span> <span class="p">{</span><span class="kt">char</span> <span class="n">s</span><span class="p">[</span><span class="k">sizeof</span><span class="p">(</span><span class="n">obj_t</span><span class="p">)];</span> <span class="n">obj_t</span> <span class="n">addr</span><span class="p">;}</span> <span class="n">u</span><span class="p">;</span>
<span class="hll"> <span class="k">if</span> <span class="p">(</span><span class="n">ld</span><span class="p">)</span> <span class="n">mps_ld_add</span><span class="p">(</span><span class="n">ld</span><span class="p">,</span> <span class="n">arena</span><span class="p">,</span> <span class="n">obj</span><span class="p">);</span>
</span> <span class="n">u</span><span class="p">.</span><span class="n">addr</span> <span class="o">=</span> <span class="n">obj</span><span class="p">;</span>
<span class="k">return</span> <span class="n">hash</span><span class="p">(</span><span class="n">u</span><span class="p">.</span><span class="n">s</span><span class="p">,</span> <span class="k">sizeof</span><span class="p">(</span><span class="n">obj_t</span><span class="p">));</span>
<span class="p">}</span>
</pre></div>
</div>
<p>By adding the dependency at this point in the code, the implementation
avoids adding unnecessary dependencies on a location. For example, an
<tt class="docutils literal"><span class="pre">eqv?</span></tt> hash table does not need to depend on the location of numbers
and characters:</p>
<div class="highlight-c"><div class="highlight"><pre><span class="k">static</span> <span class="kt">unsigned</span> <span class="kt">long</span> <span class="nf">eqv_hash</span><span class="p">(</span><span class="n">obj_t</span> <span class="n">obj</span><span class="p">,</span> <span class="n">mps_ld_t</span> <span class="n">ld</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">switch</span><span class="p">(</span><span class="n">TYPE</span><span class="p">(</span><span class="n">obj</span><span class="p">))</span> <span class="p">{</span>
<span class="k">case</span> <span class="n">TYPE_INTEGER</span>:
<span class="k">return</span> <span class="n">obj</span><span class="o">-&gt;</span><span class="n">integer</span><span class="p">.</span><span class="n">integer</span><span class="p">;</span>
<span class="k">case</span> <span class="n">TYPE_CHARACTER</span>:
<span class="k">return</span> <span class="n">obj</span><span class="o">-&gt;</span><span class="n">character</span><span class="p">.</span><span class="n">c</span><span class="p">;</span>
<span class="nl">default:</span>
<span class="k">return</span> <span class="n">eq_hash</span><span class="p">(</span><span class="n">obj</span><span class="p">,</span> <span class="n">ld</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
</pre></div>
</div>
<p>and a <tt class="docutils literal"><span class="pre">string=?</span></tt> hash table does not need to depend on the location of
any of its keys.</p>
<div class="admonition-note admonition">
<p class="first admonition-title">Note</p>
<p>The garbage collector may run at any time, so the table may become
be stale at any time after calling <a class="reference internal" href="../topic/location.html#mps_ld_add" title="mps_ld_add"><tt class="xref c c-func docutils literal"><span class="pre">mps_ld_add()</span></tt></a>, perhaps
even before you&#8217;ve added the new key.</p>
<p class="last">It&#8217;s best to postpone worrying about this until this key is actually
looked up, when the staleness will be discovered. After all, it may
never be looked up.</p>
</div>
<p>If you look up a key in an address-based hash table and fail to find it
there, that might be because the table&#8217;s dependency on the location of
the key is stale: that is, if the garbage collector moved the key. The
function <a class="reference internal" href="../topic/location.html#mps_ld_isstale" title="mps_ld_isstale"><tt class="xref c c-func docutils literal"><span class="pre">mps_ld_isstale()</span></tt></a> tells you if any of the blocks whose
locations you depended upon since the last call to
<a class="reference internal" href="../topic/location.html#mps_ld_reset" title="mps_ld_reset"><tt class="xref c c-func docutils literal"><span class="pre">mps_ld_reset()</span></tt></a> might have moved.</p>
<div class="highlight-c"><div class="highlight"><pre><span class="k">static</span> <span class="n">obj_t</span> <span class="nf">table_ref</span><span class="p">(</span><span class="n">obj_t</span> <span class="n">tbl</span><span class="p">,</span> <span class="n">obj_t</span> <span class="n">key</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">struct</span> <span class="n">bucket_s</span> <span class="o">*</span><span class="n">b</span> <span class="o">=</span> <span class="n">buckets_find</span><span class="p">(</span><span class="n">tbl</span><span class="p">,</span> <span class="n">tbl</span><span class="o">-&gt;</span><span class="n">table</span><span class="p">.</span><span class="n">buckets</span><span class="p">,</span> <span class="n">key</span><span class="p">,</span> <span class="nb">NULL</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="n">b</span> <span class="o">&amp;&amp;</span> <span class="n">b</span><span class="o">-&gt;</span><span class="n">key</span> <span class="o">!=</span> <span class="nb">NULL</span> <span class="o">&amp;&amp;</span> <span class="n">b</span><span class="o">-&gt;</span><span class="n">key</span> <span class="o">!=</span> <span class="n">obj_deleted</span><span class="p">)</span>
<span class="k">return</span> <span class="n">b</span><span class="o">-&gt;</span><span class="n">value</span><span class="p">;</span>
<span class="hll"> <span class="k">if</span> <span class="p">(</span><span class="n">mps_ld_isstale</span><span class="p">(</span><span class="o">&amp;</span><span class="n">tbl</span><span class="o">-&gt;</span><span class="n">table</span><span class="p">.</span><span class="n">ld</span><span class="p">,</span> <span class="n">arena</span><span class="p">,</span> <span class="n">key</span><span class="p">))</span> <span class="p">{</span>
</span> <span class="n">b</span> <span class="o">=</span> <span class="n">table_rehash</span><span class="p">(</span><span class="n">tbl</span><span class="p">,</span> <span class="n">tbl</span><span class="o">-&gt;</span><span class="n">table</span><span class="p">.</span><span class="n">buckets</span><span class="o">-&gt;</span><span class="n">buckets</span><span class="p">.</span><span class="n">length</span><span class="p">,</span> <span class="n">key</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="n">b</span><span class="p">)</span> <span class="k">return</span> <span class="n">b</span><span class="o">-&gt;</span><span class="n">value</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">return</span> <span class="nb">NULL</span><span class="p">;</span>
<span class="p">}</span>
</pre></div>
</div>
<p>It&#8217;s important to test <a class="reference internal" href="../topic/location.html#mps_ld_isstale" title="mps_ld_isstale"><tt class="xref c c-func docutils literal"><span class="pre">mps_ld_isstale()</span></tt></a> only in case of failure.
The function tells you whether <em>any</em> of the dependencies is stale, not
whether a particular dependency is stale. So if <tt class="docutils literal"><span class="pre">key</span></tt> has not moved,
but some other keys have moved, then if you tested
<a class="reference internal" href="../topic/location.html#mps_ld_isstale" title="mps_ld_isstale"><tt class="xref c c-func docutils literal"><span class="pre">mps_ld_isstale()</span></tt></a> first, it would return true and so you&#8217;d end up
unnecessarily rehashing the whole table. (It&#8217;s crucial, however, to
actually test that <tt class="docutils literal"><span class="pre">key</span></tt> appears in the table, not just that some key
with the same hash does.)</p>
<p>When a table is rehashed, call <a class="reference internal" href="../topic/location.html#mps_ld_reset" title="mps_ld_reset"><tt class="xref c c-func docutils literal"><span class="pre">mps_ld_reset()</span></tt></a> to clear the
location dependency, and then <a class="reference internal" href="../topic/location.html#mps_ld_add" title="mps_ld_add"><tt class="xref c c-func docutils literal"><span class="pre">mps_ld_add()</span></tt></a> for each key before it is added back to the table.</p>
<div class="admonition-note admonition">
<p class="first admonition-title">Note</p>
<p class="last">Somewhat misleadingly, <a class="reference internal" href="../topic/location.html#mps_ld_isstale" title="mps_ld_isstale"><tt class="xref c c-func docutils literal"><span class="pre">mps_ld_isstale()</span></tt></a> takes an address as
its third argument. This address is not tested for staleness: it
appears in the <a class="reference internal" href="../glossary/t.html#term-telemetry-stream"><em class="xref std std-term">telemetry stream</em></a>, however, where it might be
useful for debugging.</p>
</div>
<div class="admonition-note admonition">
<p class="first admonition-title">Note</p>
<p>After <a class="reference internal" href="../topic/location.html#mps_ld_isstale" title="mps_ld_isstale"><tt class="xref c c-func docutils literal"><span class="pre">mps_ld_isstale()</span></tt></a> has returned true, and after
rehashing the table, I don&#8217;t just repeat the usual lookup by calling
<tt class="docutils literal"><span class="pre">buckets_find</span></tt>. That&#8217;s because the table might have become stale
again already.</p>
<p class="last">Instead, <tt class="docutils literal"><span class="pre">table_rehash</span></tt> finds and returns the bucket containing
<tt class="docutils literal"><span class="pre">key</span></tt>. (Since it has to loop over all the entries in the table
anyway, it might as well find this bucket too.)</p>
</div>
<p>By adding the line:</p>
<div class="highlight-c"><div class="highlight"><pre><span class="n">puts</span><span class="p">(</span><span class="s">&quot;stale!&quot;</span><span class="p">);</span>
</pre></div>
</div>
<p>after <a class="reference internal" href="../topic/location.html#mps_ld_isstale" title="mps_ld_isstale"><tt class="xref c c-func docutils literal"><span class="pre">mps_ld_isstale()</span></tt></a> returns true, it&#8217;s possible to see when
the location dependency becomes stale and the table has to be rehashed.</p>
<div class="highlight-none"><div class="highlight"><pre>MPS Toy Scheme Example
10240, 0&gt; (define ht (make-eq-hashtable))
ht
10584, 0&gt; (hashtable-set! ht &#39;one 1)
10768, 0&gt; ht
#[hashtable (one 1)]
10768, 0&gt; (gc)
10792, 1&gt; (hashtable-ref ht &#39;one #f)
stale!
1
11080, 1&gt; (hashtable-set! ht &#39;two 2)
11264, 1&gt; (gc)
11288, 2&gt; (hashtable-ref ht &#39;one #f)
stale!
1
11576, 2&gt; (hashtable-set! ht &#39;three 3)
11760, 2&gt; (hashtable-ref ht &#39;two #f)
2
11904, 2&gt; (gc)
11928, 3&gt; (hashtable-ref ht &#39;one #f)
<span class="hll">1
</span>12072, 3&gt; (hashtable-ref ht &#39;two #f)
<span class="hll">stale!
</span>2
12360, 3&gt; (hashtable-ref ht &#39;three #f)
3
</pre></div>
</div>
<div class="admonition-note admonition">
<p class="first admonition-title">Note</p>
<p class="last">In case you&#8217;re puzzled by the highlighted lines: the symbol
<tt class="docutils literal"><span class="pre">'one</span></tt> must not have been moved by the collection, and so was
found in the table at the correct location. Thus
<a class="reference internal" href="../topic/location.html#mps_ld_isstale" title="mps_ld_isstale"><tt class="xref c c-func docutils literal"><span class="pre">mps_ld_isstale()</span></tt></a> was not called. The symbol <tt class="docutils literal"><span class="pre">'two</span></tt> did
move in the collection, so it&#8217;s not found in the table, and that
causes <a class="reference internal" href="../topic/location.html#mps_ld_isstale" title="mps_ld_isstale"><tt class="xref c c-func docutils literal"><span class="pre">mps_ld_isstale()</span></tt></a> to be tested.</p>
</div>
<p>Don&#8217;t forget to check the location dependency for staleness if you are
about to delete a key from a hash table but discover that it&#8217;s not
there. In the toy Scheme interpreter, deletion looks like this:</p>
<div class="highlight-c"><div class="highlight"><pre><span class="k">static</span> <span class="kt">void</span> <span class="nf">table_delete</span><span class="p">(</span><span class="n">obj_t</span> <span class="n">tbl</span><span class="p">,</span> <span class="n">obj_t</span> <span class="n">key</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">struct</span> <span class="n">bucket_s</span> <span class="o">*</span><span class="n">b</span><span class="p">;</span>
<span class="n">assert</span><span class="p">(</span><span class="n">TYPE</span><span class="p">(</span><span class="n">tbl</span><span class="p">)</span> <span class="o">==</span> <span class="n">TYPE_TABLE</span><span class="p">);</span>
<span class="n">b</span> <span class="o">=</span> <span class="n">buckets_find</span><span class="p">(</span><span class="n">tbl</span><span class="p">,</span> <span class="n">tbl</span><span class="o">-&gt;</span><span class="n">table</span><span class="p">.</span><span class="n">buckets</span><span class="p">,</span> <span class="n">key</span><span class="p">,</span> <span class="nb">NULL</span><span class="p">);</span>
<span class="hll"> <span class="k">if</span> <span class="p">((</span><span class="n">b</span> <span class="o">==</span> <span class="nb">NULL</span> <span class="o">||</span> <span class="n">b</span><span class="o">-&gt;</span><span class="n">key</span> <span class="o">==</span> <span class="nb">NULL</span><span class="p">)</span> <span class="o">&amp;&amp;</span> <span class="n">mps_ld_isstale</span><span class="p">(</span><span class="o">&amp;</span><span class="n">tbl</span><span class="o">-&gt;</span><span class="n">table</span><span class="p">.</span><span class="n">ld</span><span class="p">,</span> <span class="n">arena</span><span class="p">,</span> <span class="n">key</span><span class="p">))</span> <span class="p">{</span>
</span> <span class="n">b</span> <span class="o">=</span> <span class="n">table_rehash</span><span class="p">(</span><span class="n">tbl</span><span class="p">,</span> <span class="n">tbl</span><span class="o">-&gt;</span><span class="n">table</span><span class="p">.</span><span class="n">buckets</span><span class="o">-&gt;</span><span class="n">buckets</span><span class="p">.</span><span class="n">length</span><span class="p">,</span> <span class="n">key</span><span class="p">);</span>
<span class="p">}</span>
<span class="k">if</span> <span class="p">(</span><span class="n">b</span> <span class="o">!=</span> <span class="nb">NULL</span> <span class="o">&amp;&amp;</span> <span class="n">b</span><span class="o">-&gt;</span><span class="n">key</span> <span class="o">!=</span> <span class="nb">NULL</span><span class="p">)</span> <span class="p">{</span>
<span class="n">b</span><span class="o">-&gt;</span><span class="n">key</span> <span class="o">=</span> <span class="n">obj_deleted</span><span class="p">;</span>
<span class="o">++</span> <span class="n">tbl</span><span class="o">-&gt;</span><span class="n">table</span><span class="p">.</span><span class="n">buckets</span><span class="o">-&gt;</span><span class="n">buckets</span><span class="p">.</span><span class="n">deleted</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
</pre></div>
</div>
<p>Again, by adding the line <tt class="docutils literal"><span class="pre">puts(&quot;stale!&quot;);</span></tt> after
<a class="reference internal" href="../topic/location.html#mps_ld_isstale" title="mps_ld_isstale"><tt class="xref c c-func docutils literal"><span class="pre">mps_ld_isstale()</span></tt></a> returns true, it&#8217;s possible to see when the
location dependency becomes stale and the table has to be rehashed:</p>
<div class="highlight-none"><div class="highlight"><pre>MPS Toy Scheme Example
13248, 0&gt; (define ht (make-eq-hashtable))
ht
13624, 0&gt; (hashtable-set! ht &#39;one 1)
13808, 0&gt; (gc)
13832, 1&gt; (hashtable-delete! ht &#39;one)
stale!
14112, 1&gt; ht
#[hashtable]
</pre></div>
</div>
<div class="admonition-topic admonition">
<p class="first admonition-title">Topic</p>
<p class="last"><a class="reference internal" href="../topic/location.html#topic-location"><em>Location dependency</em></a>.</p>
</div>
</div>
<div class="section" id="weak-hash-tables">
<span id="guide-advanced-weak"></span><span id="index-2"></span><h2>6.3. Weak hash tables<a class="headerlink" href="#weak-hash-tables" title="Permalink to this headline"></a></h2>
<p>A <a class="reference internal" href="../glossary/w.html#term-weak-key-hash-table"><em class="xref std std-term">weak-key hash table</em></a> has <a class="reference internal" href="../glossary/w.html#term-weak-reference-1"><em class="xref std std-term">weak references<sup>(1)</sup></em></a> to its
keys. If the key dies, the value corresponding to that key is
automatically deleted from the table too. Similarly, a
<a class="reference internal" href="../glossary/w.html#term-weak-value-hash-table"><em class="xref std std-term">weak-value hash table</em></a> has weak references to its values, and a
<a class="reference internal" href="../glossary/d.html#term-doubly-weak-hash-table"><em class="xref std std-term">doubly weak hash table</em></a> has weak references to both.</p>
<p>In this section, I&#8217;ll describe how to add all three types of weak hash
table to the toy Scheme interpreter. This requires a few far-reaching
changes to the code, so in order to keep the basic integration
understandable by newcomers to the MPS, I&#8217;ve made these changes in a
separate version of the code:</p>
<p><a class="reference download internal" href="../_downloads/scheme-advanced.c"><tt class="xref download docutils literal"><span class="pre">scheme-advanced.c</span></tt></a></p>
<blockquote>
<div>The Scheme interpreter after a number of &#8220;advanced&#8221; features,
including weak hash tables, have been implemented.</div></blockquote>
<p>The MPS supports weak references only in <a class="reference internal" href="../glossary/r.html#term-root"><em class="xref std std-term">roots</em></a> and in blocks
allocated in pools belonging to the <a class="reference internal" href="../pool/awl.html#pool-awl"><em>AWL (Automatic Weak Linked)</em></a> pool class. Roots
aren&#8217;t convenient for this use case: it&#8217;s necessary for hash tables
to be automatically reclaimed when they die. So AWL it is.</p>
<div class="admonition-note admonition">
<p class="first admonition-title">Note</p>
<p class="last">This isn&#8217;t a design limitation of the MPS: it&#8217;s just that up until
now the only uses our customers have had for weak references are the
ones supported by AWL. (In particular, AWL was designed around the
requirements of weak hash tables in <a class="reference external" href="http://opendylan.org/">Open Dylan</a>.) If you need more general handling of
weak references, <a class="reference internal" href="../contact.html#contact"><em>contact us</em></a>.</p>
</div>
<p>All the references in a <a class="reference internal" href="../glossary/f.html#term-formatted-object"><em class="xref std std-term">formatted object</em></a> belong to the same
<a class="reference internal" href="../glossary/r.html#term-rank"><em class="xref std std-term">rank</em></a>: that is, they are all <a class="reference internal" href="../glossary/e.html#term-exact-reference"><em class="xref std std-term">exact</em></a>,
<a class="reference internal" href="../glossary/w.html#term-weak-reference-1"><em class="xref std std-term">weak</em></a>, or <a class="reference internal" href="../glossary/a.html#term-ambiguous-reference"><em class="xref std std-term">ambiguous references</em></a>. In
AWL, the rank of references is specified when creating an
<a class="reference internal" href="../glossary/a.html#term-allocation-point"><em class="xref std std-term">allocation point</em></a>. This has consequences for the design of the
hash table data structure: in weak-key strong-value hash tables, the
keys need to be in one object and the values in another (and the same is
true in the strong-key weak-value case). So instead of having one vector
of buckets with alternate keys and values, hash tables must have two
vectors, one for the keys and the other for the values, to allow keys
and values to have different ranks.</p>
<p>These vectors will be allocated from an AWL pool with two allocation
points, one for strong references, and one for weak references:</p>
<div class="highlight-c"><div class="highlight"><pre><span class="k">static</span> <span class="n">mps_pool_t</span> <span class="n">buckets_pool</span><span class="p">;</span> <span class="cm">/* pool for hash table buckets */</span>
<span class="k">static</span> <span class="n">mps_ap_t</span> <span class="n">strong_buckets_ap</span><span class="p">;</span> <span class="cm">/* allocation point for strong buckets */</span>
<span class="k">static</span> <span class="n">mps_ap_t</span> <span class="n">weak_buckets_ap</span><span class="p">;</span> <span class="cm">/* allocation point for weak buckets */</span>
</pre></div>
</div>
<div class="admonition-note admonition">
<p class="first admonition-title">Note</p>
<p class="last">It&#8217;s not necessary to allocate the strong buckets from the same pool
as the weak buckets, but we&#8217;ll see below that they have to be
allocated in a <em>non-moving</em> pool such as AWL.</p>
</div>
<p>The MPS <em class="dfn">splats</em> a weak reference in a <a class="reference internal" href="../glossary/f.html#term-formatted-object"><em class="xref std std-term">formatted object</em></a> by
replacing it with a null pointer when it is <a class="reference internal" href="../glossary/f.html#term-fix"><em class="xref std std-term">fixed</em></a> by the object
format&#8217;s <a class="reference internal" href="../glossary/s.html#term-scan-method"><em class="xref std std-term">scan method</em></a>. So the scan method for the buckets is
going to have the following structure. (See below for the actual code.)</p>
<div class="highlight-c"><div class="highlight"><pre><span class="k">static</span> <span class="n">mps_res_t</span> <span class="nf">buckets_scan</span><span class="p">(</span><span class="n">mps_ss_t</span> <span class="n">ss</span><span class="p">,</span> <span class="n">mps_addr_t</span> <span class="n">base</span><span class="p">,</span> <span class="n">mps_addr_t</span> <span class="n">limit</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">MPS_SCAN_BEGIN</span><span class="p">(</span><span class="n">ss</span><span class="p">)</span> <span class="p">{</span>
<span class="k">while</span> <span class="p">(</span><span class="n">base</span> <span class="o">&lt;</span> <span class="n">limit</span><span class="p">)</span> <span class="p">{</span>
<span class="n">buckets_t</span> <span class="n">buckets</span> <span class="o">=</span> <span class="n">base</span><span class="p">;</span>
<span class="kt">size_t</span> <span class="n">length</span> <span class="o">=</span> <span class="n">buckets</span><span class="o">-&gt;</span><span class="n">length</span><span class="p">;</span>
<span class="k">for</span> <span class="p">(</span><span class="n">i</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span> <span class="n">i</span> <span class="o">&lt;</span> <span class="n">length</span><span class="p">;</span> <span class="o">++</span><span class="n">i</span><span class="p">)</span> <span class="p">{</span>
<span class="n">mps_addr_t</span> <span class="n">p</span> <span class="o">=</span> <span class="n">buckets</span><span class="o">-&gt;</span><span class="n">bucket</span><span class="p">[</span><span class="n">i</span><span class="p">];</span>
<span class="k">if</span> <span class="p">(</span><span class="n">MPS_FIX1</span><span class="p">(</span><span class="n">ss</span><span class="p">,</span> <span class="n">p</span><span class="p">))</span> <span class="p">{</span>
<span class="n">mps_res_t</span> <span class="n">res</span> <span class="o">=</span> <span class="n">MPS_FIX2</span><span class="p">(</span><span class="n">ss</span><span class="p">,</span> <span class="o">&amp;</span><span class="n">p</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="n">res</span> <span class="o">!=</span> <span class="n">MPS_RES_OK</span><span class="p">)</span> <span class="k">return</span> <span class="n">res</span><span class="p">;</span>
<span class="k">if</span> <span class="p">(</span><span class="n">p</span> <span class="o">==</span> <span class="nb">NULL</span><span class="p">)</span> <span class="p">{</span>
<span class="cm">/* TODO: key/value was splatted: splat value/key too */</span>
<span class="p">}</span>
<span class="n">buckets</span><span class="o">-&gt;</span><span class="n">bucket</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">=</span> <span class="n">p</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="n">base</span> <span class="o">=</span> <span class="p">(</span><span class="kt">char</span> <span class="o">*</span><span class="p">)</span><span class="n">base</span> <span class="o">+</span>
<span class="n">ALIGN</span><span class="p">(</span><span class="n">offsetof</span><span class="p">(</span><span class="n">buckets_s</span><span class="p">,</span> <span class="n">bucket</span><span class="p">)</span> <span class="o">+</span>
<span class="n">length</span> <span class="o">*</span> <span class="k">sizeof</span><span class="p">(</span><span class="n">buckets</span><span class="o">-&gt;</span><span class="n">bucket</span><span class="p">[</span><span class="mi">0</span><span class="p">]));</span>
<span class="p">}</span>
<span class="p">}</span> <span class="n">MPS_SCAN_END</span><span class="p">(</span><span class="n">ss</span><span class="p">);</span>
<span class="k">return</span> <span class="n">MPS_RES_OK</span><span class="p">;</span>
<span class="p">}</span>
</pre></div>
</div>
<p>But how can the corresponding key/value be splatted? A format method is
not normally allowed to access memory managed by the MPS in pools that
might protect their objects (see the <a class="reference internal" href="../topic/format.html#topic-format-cautions"><em>Cautions</em></a>
section in <a class="reference internal" href="../topic/format.html#topic-format"><em>Object formats</em></a>). The AWL pool class relaxes this
constraint by allowing each object in the pool to have a
<a class="reference internal" href="../glossary/d.html#term-dependent-object"><em class="xref std std-term">dependent object</em></a>. When <a class="reference internal" href="../glossary/s.html#term-scan"><em class="xref std std-term">scanning</em></a> an object in an
AWL pool, the MPS ensures that the dependent object is not protected.
The dependent object does not have to be in the same pool as the
original object, but must be in a non-moving pool. See
<a class="reference internal" href="../pool/awl.html#pool-awl-dependent"><em>Dependent objects</em></a>.</p>
<p>So the value buckets will be the dependent object of the key buckets,
and vice versa.</p>
<p>The AWL pool determines an object&#8217;s dependent object by calling a
function that you supply when creating the pool. This means that each
object needs to have a reference to its dependent object:</p>
<div class="highlight-c"><div class="highlight"><pre><span class="k">static</span> <span class="n">mps_addr_t</span> <span class="nf">buckets_find_dependent</span><span class="p">(</span><span class="n">mps_addr_t</span> <span class="n">addr</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">buckets_t</span> <span class="n">buckets</span> <span class="o">=</span> <span class="n">addr</span><span class="p">;</span>
<span class="k">return</span> <span class="n">buckets</span><span class="o">-&gt;</span><span class="n">dependent</span><span class="p">;</span>
<span class="p">}</span>
</pre></div>
</div>
<p>There&#8217;s one final requirement to take into account before revealing the
new buckets structure, which is that each word in an object in an AWL
pool must either be a valid word-aligned reference, or else the bottom
bits of the word must be non-zero so that it does not look like an
aligned pointer. So the sizes stored in the buckets structure (the
length of the array of buckets, and the counts of used and deleted
buckets) must be tagged so that they cannot be mistaken for pointers.
See the <a class="reference internal" href="../pool/awl.html#pool-awl-caution"><em>Caution</em></a> section in <a class="reference internal" href="../pool/awl.html#pool-awl"><em>AWL (Automatic Weak Linked)</em></a>.</p>
<p>A one-bit tag suffices here:</p>
<div class="highlight-c"><div class="highlight"><pre><span class="cp">#define TAG_SIZE(i) (((i) &lt;&lt; 1) | 1)</span>
<span class="cp">#define UNTAG_SIZE(i) ((i) &gt;&gt; 1)</span>
<span class="k">typedef</span> <span class="k">struct</span> <span class="n">buckets_s</span> <span class="p">{</span>
<span class="k">struct</span> <span class="n">buckets_s</span> <span class="o">*</span><span class="n">dependent</span><span class="p">;</span> <span class="cm">/* the dependent object */</span>
<span class="kt">size_t</span> <span class="n">length</span><span class="p">;</span> <span class="cm">/* number of buckets (tagged) */</span>
<span class="kt">size_t</span> <span class="n">used</span><span class="p">;</span> <span class="cm">/* number of buckets in use (tagged) */</span>
<span class="kt">size_t</span> <span class="n">deleted</span><span class="p">;</span> <span class="cm">/* number of deleted buckets (tagged) */</span>
<span class="n">obj_t</span> <span class="n">bucket</span><span class="p">[</span><span class="mi">1</span><span class="p">];</span> <span class="cm">/* hash buckets */</span>
<span class="p">}</span> <span class="n">buckets_s</span><span class="p">,</span> <span class="o">*</span><span class="n">buckets_t</span><span class="p">;</span>
</pre></div>
</div>
<p>Now the full details of the scan method can be given, with the revised
code highlighted:</p>
<div class="highlight-c"><div class="highlight"><pre><span class="k">static</span> <span class="n">mps_res_t</span> <span class="nf">buckets_scan</span><span class="p">(</span><span class="n">mps_ss_t</span> <span class="n">ss</span><span class="p">,</span> <span class="n">mps_addr_t</span> <span class="n">base</span><span class="p">,</span> <span class="n">mps_addr_t</span> <span class="n">limit</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">MPS_SCAN_BEGIN</span><span class="p">(</span><span class="n">ss</span><span class="p">)</span> <span class="p">{</span>
<span class="k">while</span> <span class="p">(</span><span class="n">base</span> <span class="o">&lt;</span> <span class="n">limit</span><span class="p">)</span> <span class="p">{</span>
<span class="n">buckets_t</span> <span class="n">buckets</span> <span class="o">=</span> <span class="n">base</span><span class="p">;</span>
<span class="hll"> <span class="kt">size_t</span> <span class="n">i</span><span class="p">,</span> <span class="n">length</span> <span class="o">=</span> <span class="n">UNTAG_SIZE</span><span class="p">(</span><span class="n">buckets</span><span class="o">-&gt;</span><span class="n">length</span><span class="p">);</span>
</span><span class="hll"> <span class="n">FIX</span><span class="p">(</span><span class="n">buckets</span><span class="o">-&gt;</span><span class="n">dependent</span><span class="p">);</span>
</span><span class="hll"> <span class="k">if</span><span class="p">(</span><span class="n">buckets</span><span class="o">-&gt;</span><span class="n">dependent</span> <span class="o">!=</span> <span class="nb">NULL</span><span class="p">)</span>
</span><span class="hll"> <span class="n">assert</span><span class="p">(</span><span class="n">buckets</span><span class="o">-&gt;</span><span class="n">dependent</span><span class="o">-&gt;</span><span class="n">length</span> <span class="o">==</span> <span class="n">buckets</span><span class="o">-&gt;</span><span class="n">length</span><span class="p">);</span>
</span> <span class="k">for</span> <span class="p">(</span><span class="n">i</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span> <span class="n">i</span> <span class="o">&lt;</span> <span class="n">length</span><span class="p">;</span> <span class="o">++</span><span class="n">i</span><span class="p">)</span> <span class="p">{</span>
<span class="n">mps_addr_t</span> <span class="n">p</span> <span class="o">=</span> <span class="n">buckets</span><span class="o">-&gt;</span><span class="n">bucket</span><span class="p">[</span><span class="n">i</span><span class="p">];</span>
<span class="k">if</span> <span class="p">(</span><span class="n">MPS_FIX1</span><span class="p">(</span><span class="n">ss</span><span class="p">,</span> <span class="n">p</span><span class="p">))</span> <span class="p">{</span>
<span class="n">mps_res_t</span> <span class="n">res</span> <span class="o">=</span> <span class="n">MPS_FIX2</span><span class="p">(</span><span class="n">ss</span><span class="p">,</span> <span class="o">&amp;</span><span class="n">p</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="n">res</span> <span class="o">!=</span> <span class="n">MPS_RES_OK</span><span class="p">)</span> <span class="k">return</span> <span class="n">res</span><span class="p">;</span>
<span class="k">if</span> <span class="p">(</span><span class="n">p</span> <span class="o">==</span> <span class="nb">NULL</span><span class="p">)</span> <span class="p">{</span>
<span class="hll"> <span class="cm">/* key/value was splatted: splat value/key too */</span>
</span><span class="hll"> <span class="n">p</span> <span class="o">=</span> <span class="n">obj_deleted</span><span class="p">;</span>
</span><span class="hll"> <span class="n">buckets</span><span class="o">-&gt;</span><span class="n">deleted</span> <span class="o">+=</span> <span class="mi">2</span><span class="p">;</span> <span class="cm">/* tagged */</span>
</span><span class="hll"> <span class="k">if</span> <span class="p">(</span><span class="n">buckets</span><span class="o">-&gt;</span><span class="n">dependent</span> <span class="o">!=</span> <span class="nb">NULL</span><span class="p">)</span> <span class="p">{</span>
</span><span class="hll"> <span class="n">buckets</span><span class="o">-&gt;</span><span class="n">dependent</span><span class="o">-&gt;</span><span class="n">bucket</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">=</span> <span class="n">p</span><span class="p">;</span>
</span><span class="hll"> <span class="n">buckets</span><span class="o">-&gt;</span><span class="n">dependent</span><span class="o">-&gt;</span><span class="n">deleted</span> <span class="o">+=</span> <span class="mi">2</span><span class="p">;</span> <span class="cm">/* tagged */</span>
</span><span class="hll"> <span class="p">}</span>
</span> <span class="p">}</span>
<span class="n">buckets</span><span class="o">-&gt;</span><span class="n">bucket</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">=</span> <span class="n">p</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="n">base</span> <span class="o">=</span> <span class="p">(</span><span class="kt">char</span> <span class="o">*</span><span class="p">)</span><span class="n">base</span> <span class="o">+</span>
<span class="n">ALIGN</span><span class="p">(</span><span class="n">offsetof</span><span class="p">(</span><span class="n">buckets_s</span><span class="p">,</span> <span class="n">bucket</span><span class="p">)</span> <span class="o">+</span>
<span class="n">length</span> <span class="o">*</span> <span class="k">sizeof</span><span class="p">(</span><span class="n">buckets</span><span class="o">-&gt;</span><span class="n">bucket</span><span class="p">[</span><span class="mi">0</span><span class="p">]));</span>
<span class="p">}</span>
<span class="p">}</span> <span class="n">MPS_SCAN_END</span><span class="p">(</span><span class="n">ss</span><span class="p">);</span>
<span class="k">return</span> <span class="n">MPS_RES_OK</span><span class="p">;</span>
<span class="p">}</span>
</pre></div>
</div>
<div class="admonition-note admonition">
<p class="first admonition-title">Notes</p>
<ol class="last arabic">
<li><p class="first">There&#8217;s no need to dispatch on the type of the buckets object (or
even to store a type at all) because buckets are the only objects
to be stored in this pool.</p>
</li>
<li><p class="first">The dependent object must be <a class="reference internal" href="../glossary/f.html#term-fix"><em class="xref std std-term">fixed</em></a>, and because the
reference to it might be weak, it might be splatted. This means
that even if you are confident that you will always initialize
this field, you still have to guard access to it, as here.</p>
</li>
<li><p class="first">This hash table implementation uses <tt class="docutils literal"><span class="pre">NULL</span></tt> to mean &#8220;never used&#8221;
and <tt class="docutils literal"><span class="pre">obj_deleted</span></tt> to mean &#8220;formerly used but then deleted&#8221;. So
when a key is splatted it is necessary to replace it with
<tt class="docutils literal"><span class="pre">obj_deleted</span></tt>. (It would simplify the code slightly to turn the
implementation around and use <tt class="docutils literal"><span class="pre">obj_unused</span></tt>, say, for &#8220;never
used&#8221;, and <tt class="docutils literal"><span class="pre">NULL</span></tt> for &#8220;deleted&#8221;.)</p>
</li>
<li><p class="first">The updating of the tagged sizes has been abbreviated from:</p>
<div class="highlight-c"><div class="highlight"><pre><span class="n">buckets</span><span class="o">-&gt;</span><span class="n">deleted</span> <span class="o">=</span> <span class="n">TAG_SIZE</span><span class="p">(</span><span class="n">UNTAG_SIZE</span><span class="p">(</span><span class="n">buckets</span><span class="o">-&gt;</span><span class="n">deleted</span><span class="p">)</span> <span class="o">+</span> <span class="mi">1</span><span class="p">)</span>
</pre></div>
</div>
<p>to <tt class="docutils literal"><span class="pre">buckets-&gt;deleted</span> <span class="pre">+=</span> <span class="pre">2</span></tt>.</p>
</li>
</ol>
</div>
<p>The <a class="reference internal" href="../glossary/s.html#term-skip-method"><em class="xref std std-term">skip method</em></a> is straightforward:</p>
<div class="highlight-c"><div class="highlight"><pre><span class="k">static</span> <span class="n">mps_addr_t</span> <span class="nf">buckets_skip</span><span class="p">(</span><span class="n">mps_addr_t</span> <span class="n">base</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">buckets_t</span> <span class="n">buckets</span> <span class="o">=</span> <span class="n">base</span><span class="p">;</span>
<span class="kt">size_t</span> <span class="n">length</span> <span class="o">=</span> <span class="n">UNTAG_SIZE</span><span class="p">(</span><span class="n">buckets</span><span class="o">-&gt;</span><span class="n">length</span><span class="p">);</span>
<span class="k">return</span> <span class="p">(</span><span class="kt">char</span> <span class="o">*</span><span class="p">)</span><span class="n">base</span> <span class="o">+</span>
<span class="n">ALIGN</span><span class="p">(</span><span class="n">offsetof</span><span class="p">(</span><span class="n">buckets_s</span><span class="p">,</span> <span class="n">bucket</span><span class="p">)</span> <span class="o">+</span>
<span class="n">length</span> <span class="o">*</span> <span class="k">sizeof</span><span class="p">(</span><span class="n">buckets</span><span class="o">-&gt;</span><span class="n">bucket</span><span class="p">[</span><span class="mi">0</span><span class="p">]));</span>
<span class="p">}</span>
</pre></div>
</div>
<p>as is the object format, since AWL only calls the scan and skip
methods:</p>
<div class="highlight-c"><div class="highlight"><pre><span class="k">struct</span> <span class="n">mps_fmt_A_s</span> <span class="n">buckets_fmt_s</span> <span class="o">=</span> <span class="p">{</span>
<span class="n">ALIGNMENT</span><span class="p">,</span>
<span class="n">buckets_scan</span><span class="p">,</span>
<span class="n">buckets_skip</span><span class="p">,</span>
<span class="nb">NULL</span><span class="p">,</span> <span class="cm">/* Obsolete copy method */</span>
<span class="nb">NULL</span><span class="p">,</span> <span class="cm">/* fwd method not used by AWL */</span>
<span class="nb">NULL</span><span class="p">,</span> <span class="cm">/* isfwd method not used by AWL */</span>
<span class="nb">NULL</span> <span class="cm">/* pad method not used by AWL */</span>
<span class="p">};</span>
</pre></div>
</div>
<p>Finally, we can create the buckets pool and its allocation points:</p>
<div class="highlight-c"><div class="highlight"><pre><span class="cm">/* Create the buckets format. */</span>
<span class="n">res</span> <span class="o">=</span> <span class="n">mps_fmt_create_A</span><span class="p">(</span><span class="o">&amp;</span><span class="n">buckets_fmt</span><span class="p">,</span> <span class="n">arena</span><span class="p">,</span> <span class="o">&amp;</span><span class="n">buckets_fmt_s</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="n">res</span> <span class="o">!=</span> <span class="n">MPS_RES_OK</span><span class="p">)</span> <span class="n">error</span><span class="p">(</span><span class="s">&quot;Couldn&#39;t create buckets format&quot;</span><span class="p">);</span>
<span class="cm">/* Create an Automatic Weak Linked (AWL) pool to manage the hash table</span>
<span class="cm"> buckets. */</span>
<span class="n">MPS_ARGS_BEGIN</span><span class="p">(</span><span class="n">args</span><span class="p">)</span> <span class="p">{</span>
<span class="n">MPS_ARGS_ADD</span><span class="p">(</span><span class="n">args</span><span class="p">,</span> <span class="n">MPS_KEY_FORMAT</span><span class="p">,</span> <span class="n">buckets_fmt</span><span class="p">);</span>
<span class="n">MPS_ARGS_ADD</span><span class="p">(</span><span class="n">args</span><span class="p">,</span> <span class="n">MPS_KEY_AWL_FIND_DEPENDENT</span><span class="p">,</span> <span class="n">buckets_find_dependent</span><span class="p">);</span>
<span class="n">MPS_ARGS_DONE</span><span class="p">(</span><span class="n">args</span><span class="p">);</span>
<span class="n">res</span> <span class="o">=</span> <span class="n">mps_pool_create_k</span><span class="p">(</span><span class="o">&amp;</span><span class="n">buckets_pool</span><span class="p">,</span> <span class="n">arena</span><span class="p">,</span> <span class="n">mps_class_awl</span><span class="p">(),</span> <span class="n">args</span><span class="p">);</span>
<span class="p">}</span> <span class="n">MPS_ARGS_END</span><span class="p">(</span><span class="n">args</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="n">res</span> <span class="o">!=</span> <span class="n">MPS_RES_OK</span><span class="p">)</span> <span class="n">error</span><span class="p">(</span><span class="s">&quot;Couldn&#39;t create buckets pool&quot;</span><span class="p">);</span>
<span class="cm">/* Create allocation points for weak and strong buckets. */</span>
<span class="n">MPS_ARGS_BEGIN</span><span class="p">(</span><span class="n">args</span><span class="p">)</span> <span class="p">{</span>
<span class="n">MPS_ARGS_ADD</span><span class="p">(</span><span class="n">args</span><span class="p">,</span> <span class="n">MPS_KEY_RANK</span><span class="p">,</span> <span class="n">mps_rank_exact</span><span class="p">());</span>
<span class="n">MPS_ARGS_DONE</span><span class="p">(</span><span class="n">args</span><span class="p">);</span>
<span class="n">res</span> <span class="o">=</span> <span class="n">mps_ap_create_k</span><span class="p">(</span><span class="o">&amp;</span><span class="n">strong_buckets_ap</span><span class="p">,</span> <span class="n">buckets_pool</span><span class="p">,</span> <span class="n">args</span><span class="p">);</span>
<span class="p">}</span> <span class="n">MPS_ARGS_END</span><span class="p">(</span><span class="n">args</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="n">res</span> <span class="o">!=</span> <span class="n">MPS_RES_OK</span><span class="p">)</span> <span class="n">error</span><span class="p">(</span><span class="s">&quot;Couldn&#39;t create strong buckets allocation point&quot;</span><span class="p">);</span>
<span class="n">MPS_ARGS_BEGIN</span><span class="p">(</span><span class="n">args</span><span class="p">)</span> <span class="p">{</span>
<span class="n">MPS_ARGS_ADD</span><span class="p">(</span><span class="n">args</span><span class="p">,</span> <span class="n">MPS_KEY_RANK</span><span class="p">,</span> <span class="n">mps_rank_weak</span><span class="p">());</span>
<span class="n">MPS_ARGS_DONE</span><span class="p">(</span><span class="n">args</span><span class="p">);</span>
<span class="n">res</span> <span class="o">=</span> <span class="n">mps_ap_create_k</span><span class="p">(</span><span class="o">&amp;</span><span class="n">weak_buckets_ap</span><span class="p">,</span> <span class="n">buckets_pool</span><span class="p">,</span> <span class="n">args</span><span class="p">);</span>
<span class="p">}</span> <span class="n">MPS_ARGS_END</span><span class="p">(</span><span class="n">args</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="n">res</span> <span class="o">!=</span> <span class="n">MPS_RES_OK</span><span class="p">)</span> <span class="n">error</span><span class="p">(</span><span class="s">&quot;Couldn&#39;t create weak buckets allocation point&quot;</span><span class="p">);</span>
</pre></div>
</div>
<p>By adding the line:</p>
<div class="highlight-c"><div class="highlight"><pre><span class="n">puts</span><span class="p">(</span><span class="s">&quot;splat!&quot;</span><span class="p">);</span>
</pre></div>
</div>
<p>at the point in <tt class="docutils literal"><span class="pre">buckets_scan</span></tt> where the splatting of a weak reference
is detected, we can see this happening:</p>
<div class="highlight-none"><div class="highlight"><pre>MPS Toy Scheme Example
24624, 0&gt; (define ht (make-doubly-weak-hashtable string-hash string=?))
ht
25264, 0&gt; (hashtable-set! ht &quot;one&quot; 1)
25456, 0&gt; (hashtable-set! ht &quot;two&quot; 2)
25648, 0&gt; (hashtable-set! ht &quot;three&quot; 3)
25840, 0&gt; ht
#[hashtable (&quot;two&quot; 2) (&quot;one&quot; 1) (&quot;three&quot; 3)]
25864, 0&gt; (gc)
splat!
splat!
splat!
25912, 1&gt; ht
#[hashtable]
</pre></div>
</div>
<div class="admonition-topic admonition">
<p class="first admonition-title">Topics</p>
<p class="last"><a class="reference internal" href="../topic/weak.html#topic-weak"><em>Weak references</em></a>, <a class="reference internal" href="../pool/awl.html#pool-awl"><em>AWL (Automatic Weak Linked)</em></a>.</p>
</div>
</div>
<div class="section" id="global-symbol-table">
<span id="index-3"></span><h2>6.4. Global symbol table<a class="headerlink" href="#global-symbol-table" title="Permalink to this headline"></a></h2>
<p>In the original (non-MPS) version of the toy Scheme interpreter, the
global symbol table was implemented as a key-only hash table, and each
symbol stored its own name.</p>
<p>But now that we have weak hash tables, it makes sense to re-implement
the global symbol table as a strong-key weak-value hash table mapping
strings to symbols. Each symbol will now contain a reference to its name
as a string object, instead of containing the name itself.</p>
<div class="figure align-center">
<img alt="Diagram: Global symbol table design (weak references shown as dashed lines)." src="../_images/symbol-table.svg" /><p class="caption">Global symbol table design (weak references shown as dashed lines).</p>
</div>
<p>This design depends on the string object containing the symbol name
being immutable. As it happens, all strings are immutable, because the
toy Scheme interpreter doesn&#8217;t implement <tt class="docutils literal"><span class="pre">string-set!</span></tt>, but if it did
then some care would need to be taken. (Either by marking these strings
as immutable in some way, or by ensuring that these strings are
&#8220;private&#8221;: that is, that Scheme programs never get hold of references to
them.)</p>
<p>When there are no more strong references to a symbol:</p>
<ol class="arabic simple">
<li>the reference to the symbol from the &#8220;values&#8221; array may be splatted;</li>
<li>that&#8217;s detected by the buckets scan method, which deletes the
corresponding entry in the &#8220;keys&#8221; array;</li>
<li>which may in turn cause the symbol name to die, unless there are
other strong references keeping it alive.</li>
</ol>
<p>Here&#8217;s the new symbol structure:</p>
<div class="highlight-c"><div class="highlight"><pre><span class="k">typedef</span> <span class="k">struct</span> <span class="n">symbol_s</span> <span class="p">{</span>
<span class="n">type_t</span> <span class="n">type</span><span class="p">;</span> <span class="cm">/* TYPE_SYMBOL */</span>
<span class="n">obj_t</span> <span class="n">name</span><span class="p">;</span> <span class="cm">/* its name (a string) */</span>
<span class="p">}</span> <span class="n">symbol_s</span><span class="p">;</span>
</pre></div>
</div>
<p>and the new implementation of <tt class="docutils literal"><span class="pre">intern</span></tt>:</p>
<div class="highlight-c"><div class="highlight"><pre><span class="k">static</span> <span class="n">obj_t</span> <span class="nf">intern_string</span><span class="p">(</span><span class="n">obj_t</span> <span class="n">name</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">obj_t</span> <span class="n">symbol</span><span class="p">;</span>
<span class="n">assert</span><span class="p">(</span><span class="n">TYPE</span><span class="p">(</span><span class="n">name</span><span class="p">)</span> <span class="o">==</span> <span class="n">TYPE_STRING</span><span class="p">);</span>
<span class="n">symbol</span> <span class="o">=</span> <span class="n">table_ref</span><span class="p">(</span><span class="n">symtab</span><span class="p">,</span> <span class="n">name</span><span class="p">);</span>
<span class="k">if</span><span class="p">(</span><span class="n">symbol</span> <span class="o">==</span> <span class="nb">NULL</span><span class="p">)</span> <span class="p">{</span>
<span class="n">symbol</span> <span class="o">=</span> <span class="n">make_symbol</span><span class="p">(</span><span class="n">name</span><span class="p">);</span>
<span class="n">table_set</span><span class="p">(</span><span class="n">symtab</span><span class="p">,</span> <span class="n">name</span><span class="p">,</span> <span class="n">symbol</span><span class="p">);</span>
<span class="p">}</span>
<span class="k">return</span> <span class="n">symbol</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">static</span> <span class="n">obj_t</span> <span class="nf">intern</span><span class="p">(</span><span class="kt">char</span> <span class="o">*</span><span class="n">string</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">return</span> <span class="n">intern_string</span><span class="p">(</span><span class="n">make_string</span><span class="p">(</span><span class="n">strlen</span><span class="p">(</span><span class="n">string</span><span class="p">),</span> <span class="n">string</span><span class="p">));</span>
<span class="p">}</span>
</pre></div>
</div>
<p>The symbol table now becomes a very simple <a class="reference internal" href="../glossary/r.html#term-root"><em class="xref std std-term">root</em></a>, that only has
to be registered once (not <a class="reference internal" href="lang.html#guide-lang-root"><em>every time it is rehashed</em></a>, as previously):</p>
<div class="highlight-c"><div class="highlight"><pre><span class="n">mps_addr_t</span> <span class="n">ref</span><span class="p">;</span>
<span class="n">symtab</span> <span class="o">=</span> <span class="nb">NULL</span><span class="p">;</span>
<span class="n">ref</span> <span class="o">=</span> <span class="o">&amp;</span><span class="n">symtab</span><span class="p">;</span>
<span class="n">res</span> <span class="o">=</span> <span class="n">mps_root_create_table</span><span class="p">(</span><span class="o">&amp;</span><span class="n">symtab_root</span><span class="p">,</span> <span class="n">arena</span><span class="p">,</span> <span class="n">mps_rank_exact</span><span class="p">(),</span> <span class="mi">0</span><span class="p">,</span>
<span class="n">ref</span><span class="p">,</span> <span class="mi">1</span><span class="p">);</span>
<span class="k">if</span><span class="p">(</span><span class="n">res</span> <span class="o">!=</span> <span class="n">MPS_RES_OK</span><span class="p">)</span> <span class="n">error</span><span class="p">(</span><span class="s">&quot;Couldn&#39;t register symtab root&quot;</span><span class="p">);</span>
<span class="n">symtab</span> <span class="o">=</span> <span class="n">make_table</span><span class="p">(</span><span class="mi">16</span><span class="p">,</span> <span class="n">string_hash</span><span class="p">,</span> <span class="n">string_equalp</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">);</span>
</pre></div>
</div>
<div class="admonition-note admonition">
<p class="first admonition-title">Note</p>
<p class="last">The order of operations is important here. The global variable
<tt class="docutils literal"><span class="pre">symtab</span></tt> must be registered as a root before creating the symbol
table, otherwise the symbol table might be collected in the
interval between creation and registration. But we must also ensure
that <tt class="docutils literal"><span class="pre">symtab</span></tt> is valid (that is, scannable) before registering it
(in this case, by setting it to <tt class="docutils literal"><span class="pre">NULL</span></tt>).</p>
</div>
<p>By printing <tt class="docutils literal"><span class="pre">splat!</span></tt> when the splatting of a weak reference is
detected by the scan method, we can see when symbols are dying:</p>
<div class="highlight-none"><div class="highlight"><pre>MPS Toy Scheme Example
24624, 0&gt; (define a 1)
a
24832, 0&gt; &#39;(a b c d)
(a b c d)
25144, 0&gt; (gc)
splat!
splat!
splat!
</pre></div>
</div>
<p>Here, the symbols <tt class="docutils literal"><span class="pre">b</span></tt>, <tt class="docutils literal"><span class="pre">c</span></tt> and <tt class="docutils literal"><span class="pre">d</span></tt> died, but <tt class="docutils literal"><span class="pre">a</span></tt> was kept alive
by the reference from the environment.</p>
</div>
<div class="section" id="segregation-of-objects">
<span id="guide-advanced-segregation"></span><span id="index-4"></span><h2>6.5. Segregation of objects<a class="headerlink" href="#segregation-of-objects" title="Permalink to this headline"></a></h2>
<p>When objects of different types have different properties (different
sizes, lifetimes, references, layouts) it makes sense to segregate
them into pools of appropriate classes.</p>
<p>For example, the toy Scheme interpreter has a mixture of object types,
some of which contain references to other objects (for example, pairs)
that must be <a class="reference internal" href="../glossary/s.html#term-scan"><em class="xref std std-term">scanned</em></a>, and some of which do not (for
example, strings). If the <a class="reference internal" href="../glossary/l.html#term-leaf-object"><em class="xref std std-term">leaf objects</em></a> are segregated into a
pool of an appropriate class, the cost of scanning them can be
avoided.</p>
<p>Here the appropriate class is <a class="reference internal" href="../pool/amcz.html#pool-amcz"><em>AMCZ (Automatic Mostly-Copying Zero-rank)</em></a>, and the necessary code
changes are straightforward. First, global variables for the new pool
and its <a class="reference internal" href="../glossary/a.html#term-allocation-point"><em class="xref std std-term">allocation point</em></a>:</p>
<div class="highlight-c"><div class="highlight"><pre><span class="k">static</span> <span class="n">mps_pool_t</span> <span class="n">leaf_pool</span><span class="p">;</span> <span class="cm">/* pool for leaf objects */</span>
<span class="k">static</span> <span class="n">mps_ap_t</span> <span class="n">leaf_ap</span><span class="p">;</span> <span class="cm">/* allocation point for leaf objects */</span>
</pre></div>
</div>
<p>Second, the leaf objects must be allocated on <tt class="docutils literal"><span class="pre">leaf_ap</span></tt> instead of
<tt class="docutils literal"><span class="pre">obj_ap</span></tt>. And third, the pool and its allocation point must be created:</p>
<div class="highlight-c"><div class="highlight"><pre><span class="cm">/* Create an Automatic Mostly-Copying Zero-rank (AMCZ) pool to</span>
<span class="cm"> manage the leaf objects. */</span>
<span class="n">MPS_ARGS_BEGIN</span><span class="p">(</span><span class="n">args</span><span class="p">)</span> <span class="p">{</span>
<span class="n">MPS_ARGS_ADD</span><span class="p">(</span><span class="n">args</span><span class="p">,</span> <span class="n">MPS_KEY_CHAIN</span><span class="p">,</span> <span class="n">obj_chain</span><span class="p">);</span>
<span class="n">MPS_ARGS_ADD</span><span class="p">(</span><span class="n">args</span><span class="p">,</span> <span class="n">MPS_KEY_FORMAT</span><span class="p">,</span> <span class="n">obj_fmt</span><span class="p">);</span>
<span class="n">MPS_ARGS_DONE</span><span class="p">(</span><span class="n">args</span><span class="p">);</span>
<span class="n">res</span> <span class="o">=</span> <span class="n">mps_pool_create_k</span><span class="p">(</span><span class="o">&amp;</span><span class="n">leaf_pool</span><span class="p">,</span> <span class="n">arena</span><span class="p">,</span> <span class="n">mps_class_amcz</span><span class="p">(),</span> <span class="n">args</span><span class="p">);</span>
<span class="p">}</span> <span class="n">MPS_ARGS_END</span><span class="p">(</span><span class="n">args</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="n">res</span> <span class="o">!=</span> <span class="n">MPS_RES_OK</span><span class="p">)</span> <span class="n">error</span><span class="p">(</span><span class="s">&quot;Couldn&#39;t create leaf pool&quot;</span><span class="p">);</span>
<span class="cm">/* Create allocation point for leaf objects. */</span>
<span class="n">res</span> <span class="o">=</span> <span class="n">mps_ap_create_k</span><span class="p">(</span><span class="o">&amp;</span><span class="n">leaf_ap</span><span class="p">,</span> <span class="n">leaf_pool</span><span class="p">,</span> <span class="n">mps_args_none</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="n">res</span> <span class="o">!=</span> <span class="n">MPS_RES_OK</span><span class="p">)</span> <span class="n">error</span><span class="p">(</span><span class="s">&quot;Couldn&#39;t create leaf objects allocation point&quot;</span><span class="p">);</span>
</pre></div>
</div>
<p>Note that the new pool shared a <a class="reference internal" href="../glossary/g.html#term-generation-chain"><em class="xref std std-term">generation chain</em></a> with the old
pool. This is important, because the leaf objects live and die along
with the non-leaf objects of similar ages.</p>
<p>As an initial step in making this change, the new pool uses the same
<a class="reference internal" href="../glossary/o.html#term-object-format"><em class="xref std std-term">object format</em></a>. However, we normally wouldn&#8217;t stop there: we&#8217;d
take advantage of the segregation to simplify the scanning of the
objects that have been left behind.</p>
<div class="admonition-topic admonition">
<p class="first admonition-title">Topic</p>
<p class="last"><a class="reference internal" href="../pool/amcz.html#pool-amcz"><em>AMCZ (Automatic Mostly-Copying Zero-rank)</em></a>.</p>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="sphinxsidebar">
<div class="sphinxsidebarwrapper">
<p class="logo"><a href="../index.html">
<img class="logo" src="../_static/logo.png" alt="Logo"/>
</a></p>
<h3><a href="../index.html">Table Of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">6. Advanced topics</a><ul>
<li><a class="reference internal" href="#finalization">6.1. Finalization</a></li>
<li><a class="reference internal" href="#location-dependency">6.2. Location dependency</a></li>
<li><a class="reference internal" href="#weak-hash-tables">6.3. Weak hash tables</a></li>
<li><a class="reference internal" href="#global-symbol-table">6.4. Global symbol table</a></li>
<li><a class="reference internal" href="#segregation-of-objects">6.5. Segregation of objects</a></li>
</ul>
</li>
</ul>
<h4>Previous topic</h4>
<p class="topless"><a href="perf.html"
title="previous chapter">5. Tuning the Memory Pool System for performance</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="../topic/index.html"
title="next chapter">Reference</a></p><h4>Downloads</h4>
<p class="topless">
<a href="http://www.ravenbrook.com/project/mps/release/1.111.0/">MPS Kit release 1.111.0</a><br>
<a href="http://www.ravenbrook.com/project/mps/release/">All MPS Kit releases</a>
</p>
<h4>Issues</h4>
<p class="topless">
<a href="http://www.ravenbrook.com/project/mps/issue/?action=list&amp;view=status%3dopen&amp;display=Job:Priority:Title&amp;sort=Priority">Known issues</a><br>
<a href="http://www.ravenbrook.com/project/mps/issue/?action=fixed&release_fixed=1.111.0">Issues fixed in release 1.111.0</a>
</p><h4>Contact us</h4>
<p class="topless"><a href="mailto:mps-questions@ravenbrook.com">mps-questions@ravenbrook.com</a></p>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../genindex.html" title="General Index"
>index</a></li>
<li class="right" >
<a href="../topic/index.html" title="Reference"
>next</a> |</li>
<li class="right" >
<a href="perf.html" title="5. Tuning the Memory Pool System for performance"
>previous</a> |</li>
<li><a href="../index.html">Memory Pool System 1.111.0 documentation</a> &raquo;</li>
<li><a href="index.html" >Guide</a> &raquo;</li>
</ul>
</div>
<div class="footer">
&copy; <a href="../copyright.html">Copyright</a> 2013, Ravenbrook Limited.
Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 1.1.3.
</div>
</body>
</html>
Reference in a new issue View git blame Copy permalink