TEST_HEADER
 id = $HopeName$
 summary = SW performance no worse than last release
 language = english
END_HEADER

EP have a requirement that each release of ScriptWorks be no slower
than the last in any typical operation. EPQA test this by running
various test suites, with various combinations of settings, and
seeing if anything gets slower. It's not just the overall time
for a suite that's important, but the times for individual jobs,
because each job could be typical for a particular user.

Of course, SW performance is mostly out of our hands; however,
we should check that each release of the MM is as fast as releases
that have previously gone out to customers. Here's a performance
comparison.

1. Check out a version of SW known to be stable and reliable. It's
best to opt for one which corresponds to an actual external release,
e.g. SW_B11, which is 4.5r1.

2. Build a release version using the old memory manager and one using
the new memory manager. Bear in mind that in this context, "memory
manager" means not just the mmsw library, but the glue code. If the
distance between the current SW trunk and the branch you chose is
great enough, you may well need to edit the new glue code to make it
compatible with the old SW.

3. If on a Mac, set the memory of the two versions of SW (with
File Info) so that the RIP gets the same amount of memory in
the end (e.g. 30000K is a reasonable number).

4. Use either executable to set up device and page settings. Then
quit, remove LOGFILE from the SW folder, and backup the SW folder.

5. Start one executable, run tests, quit, keep LOGFILE. Bear in mind
that the times reported in log file are clock-on-the-wall times, and
will be affected by other processes running on your machine.

6. Delete SW folder and restore from backup.

7. Start other executable, run same tests, quit, keep LOGFILE.

8. Compare performance by running "difflogs" on the two LOGFILEs. Look for
jobs which
 - used to run but now fail -- this could be serious.
 - now run slower than before (total or interpretation time)
 - produce unexpected differences in output, e.g. checksums.
When a job looks like it might be a problem, try running it again, both
on its own and within the suite. There is significant random variation
in the times recorded in SW LOGFILEs, so it's important to check that
what looks like a difference really is one.

If time permits, try on a range of platforms with a range of ScriptWorks
settings, including memory settings. Here are some examples of suitable
settings to try:

TestHqn capstan device. 1270 dpi, produce separations, HPS on, lots of
grey levels, single (if required) mode.

EasyTrap on.

Something with Harlequin Colour Management (make sure to use a colour
device, or produce separations).


rit 1998-03-13