Mps wiki: gc story: what triggers a gc?

Copied from Perforce
 Change: 161058
 ServerID: perforce.ravenbrook.com

mps/manual/wiki/gc_story.html

@@ -56,26 +56,118 @@
 
 <h2>Concepts and Datastructures</h2>
 
+<dl>
+  <dt>Zone</dt>
+  <dd>Stripe of memory.</dd>
+</dl>
+
 
 <h2>Set-up</h2>
 
-<p>Mutator creates an array of mps_gen_param_s structs (for example: [ { 100KB, 90% mortality }, { 100K, 50% mortality } ] ), passes it to mps_chain_create, and uses this chain to create a new AMC pool.</p>
+<p>Say the mutator creates an array of 2 mps_gen_param_s structs:</p>
+<ul>
+  <li> { 100KB, 90% mortality }, </li>
+  <li> { 200KB, 50% mortality } </li>
+</ul>
 
-<p>The Chain contains an array of two GenDescs: numbers 0 and 1.  The AMC pool creates *three* PoolGens: 0 and 1 are linked to the corresponding GenDesc, and 2 is linked to the arena-wide "topGen" GenDesc.</p>
+<p>It passes this array to mps_chain_create, and then uses the chain to create a new AMC pool.</p>
 
-<p>The GenDesc zonesets are empty.  The PoolGen newSizes are zero.</p>
+<p>The Chain contains an array of two GenDescs: numbers 0 and 1.  The AMC pool creates *three* PoolGens:</p>
+<ul>
+  <li> PoolGen 0 is linked to GenDesc 0;</li>
+  <li> PoolGen 1 is linked to GenDesc 1;</li>
+  <li> PoolGen 2 is linked to the arena-wide "topGen" GenDesc.</li>
+</ul>
+
+<p>The PoolGen newSizes are zero.  The GenDesc zonesets are empty.</p>
 
 
 <h2>Accumulating objects</h2>
 
+<p>Mutator creates and uses an allocation buffer, making new objects accumulate in the nursery generation.</p>
 
 
-<h2>Triggering a major collection</h2>
+<h3>Placement</h3>
+
+<p>When AMCBufferFill asks for new memory segments, it passes the PoolGen's "nr" generation number (0 for mutator allocation, 1 or 2 for preserved objects) as a segment-placement preference (with SegPrefGen).</p>
+
+<p>ArenaVM tries hard to keep all segments for this SegPrefGen-number together in the same zone or zones, and separate from the zones used for all other things.  (Such as: zones with other generations, blacklist zones, and as-yet unused zones).</p>
+
+
+<h3>How big, and where, is this generation?</h3>
+
+<p>AMCBufferFill does this accounting:
+  
+<p>The segment's size is added into the PoolGen's newSize.</p>
+
+<p>The segment's zoneset is unioned into the GenDesc zoneset (by calling  PoolGenUpdateZones).</p>
+
+
+<h2>Call paths that may trigger a collection</h2>
+
+<p>All collections start from ArenaStep().  There are two routes into ArenaStep: an explicit call to mps_arena_step(), or an implicit one from the time-stealing ArenaPolls in mps_alloc, mps_reserve, and mps_alloc_pattern_end/reset.</p>
+
+
+<h2>Triggering a full collection</h2>
+
+
+<h3>Condition</h3>
+
+<p>There are two trigger conditions:</p>
+
+<p>Firstly, lots of "spare time".  An explicit call to mps_arena_step() can specify non-zero interval and multiplier.  If (interval x multiplier) is big enough, and it's been long enough since the last one, start a full collection.</p>
+
+<p>Secondly (when ArenaStep calls TracePoll) the infamous "dynamic criterion".  The plan is to start a full collection soon enough so that we don't completely run out of memory.  I hope that the idea of this is:</p>
+
+<ol>
+  <li> look ahead to how much extra forwarding-space would be required for a full collection;</li>
+  <li> add how much extra client-allocation would occur during collection;</li>
+  <li> and compare it against ArenaAvail.</li>
+</ol>
+
+<p>Both trigger conditions call traceStartCollectAll().</p>
+
+
+<h3>What to condemn</h3>
+
+<p>traceStartCollectAll() finds all chains, all the PoolGens in Gen 0 of those chains, all the pools those PoolGens are part of, all the segments of those pools, and condemns all those segments:</p>
+
+<pre>traceStartCollectAll():
+  traceCondemnAll()
+    for chain in all chains:
+      ChainCondemnAll(chain)</pre>
+
+<pre>ChainCondemnAll(chain):
+  for PoolGen in GenDesc 0 of chain:
+    for Seg in (PoolGen->pool)->SegRing:
+      TraceWhiten(Seg)</pre>
+
+<p>Note that AMS pools have a Gen-0-only chain (and so get condemned).</p>
+
+<p>Note that LO and AWL pools also have a Gen-0-only chain (and so get condemned).  [This is despite their segment-placement preference being hardwired to SegPrefGen-number 1; yuk!  RHSK 2006-12-01]</p>
 
 
 <h2>Triggering a minor collection</h2>
 
 
+<h3>Condition</h3>
+
+<p>A minor collection is triggered if there's a chain whose GenDesc 0 is 'over capacity': the sum of the PoolGen 0 newSizes exceeds the GenDesc's capacity.  (If there's a choice, pick the chain whose Gen 0 is most over capacity).</p>
+
+<p>[Note that we only look at "newSize".  I don't understand what this means, or how it differs from totalSize.  (It may be a consequence of nailing, perhaps?).  RHSK 2006-12-01]</p>
+
+
+<h3>What to condemn</h3>
+
+<p>For the triggering chain, ChainCondemnAuto() finds the list of GenDescs to condemn: GenDesc 0 and each higher GenDesc that's also over its capacity.  (That is: where the sum of newSizes exceeds capacity, as before).</p>
+
+<p>These GenDescs have been recording the zoneset of all the segments ever added into that GenDesc, as long as the pool noted it by calling PoolGenUpdateZones(PoolGen, Seg).</p>
+
+<p>ChainCondemnAuto() calls TraceCondemnZones() to condemn the full zoneset ever touched by any segment in any of the condemned GenDescs.</p>
+
+<p>TraceCondemnZones() uses the SegFirst/SegNext() iterator, and for every segment that is wholly within the condemned zones, it calls TraceAddWhite(seg).</p>
+
+
 <h2>Progress of a collection</h2>
 
   
@@ -83,6 +175,7 @@
 
 <pre>
   2006-11-30  RHSK  Created, incomplete.
+  2006-12-01  RHSK  What triggers a GC?
 </pre>