Calibration preflight runs multiple smoke test iterations. If one
run fails (e.g. a scenario times out), skip its data and use
results from successful runs. Only fail if ALL runs fail.
Cucumber step timeout = 120s (safety net, rarely reached).
Playwright element timeout = 10s (fast fail for missing elements).
Playwright system operations = 120s (process launch, window management).
Log marker wait = 120s (generous ceiling with internal retries).
Quick failures come from PLAYWRIGHT_TIMEOUT (10s), not cucumber.
System operations have room to complete without timeouts.
electron.launch, firstWindow, waitForLoadState are system-level
operations (process creation, window management), not test assertions.
They need generous timeouts independent of measured step time.
Cucumber step timeout still catches truly hung test steps.
electron.launch() is a system-level process creation, not a test step.
It needs a generous timeout independent of measured step time. The
cucumber step timeout (measured from calibration) handles step-level
hangs, but the Playwright-level launch needs room for process startup.
All 16 failures were app launch timing out at 5.5s. Two scenarios × 2 runs
= 4 launch samples missed the 25%+ of launches that exceed 5.5s. Add a
minimal 3rd scenario (launch + page load check) to capture 6 samples
total and better cover the launch time distribution.
Calibration now classifies steps by type and stores per-type max durations:
- stepMs: all steps → CUCUMBER_GLOBAL_TIMEOUT
- launchMs: launch/browser-view steps → HEAVY_PLAYWRIGHT_TIMEOUT
- waitMs: wait/log/SSE/watch-fs steps → LOG_MARKER_WAIT_TIMEOUT
Every internal timeout now comes from measurement. Removed all hardcoded
subtractions (500, 4000, 5000 from LOG_MARKER_WAIT_TIMEOUT).
With measured step timeouts (5.5s), the old formula gave LOG_MARKER_WAIT
= max(5000, 5500-5000) = 5000ms. Markers appearing at 5.2s would fail
the internal wait before the cucumber step timeout. Reduce buffer to 500ms
so internal timeouts stay close to measured worst-case step duration.
Watcher re-index time scales with accumulated file state. Three file
additions before restart simulate the loaded state that causes 10-20×
slowdown in the full test suite vs. fresh calibration.
The real bottleneck in the main test is NOT first-time watcher init
but watcher re-initialization after wiki restart (re-indexing files).
Add restart + watch-fs wait AFTER file operations to measure this.
Single create/modify/delete only captures first-launch watcher latency
(6.4s), but main test watcher degrades 3-10× under multi-scenario load.
Add second file cycle to measure performance after sustained watcher use.
The smoke test's heaviest step was 2.6s ('wait for SSE and watch-fs')
— but the main test's same step takes 10-40s under multi-scenario load.
Add create/modify/delete file + watch-fs detection to the calibration
scenario so maxStepMs reflects real worst-case watch-fs latency.
Calibration now extracts actual max individual step duration from
cucumber JSON output — no multipliers, REFERENCE, caps, or floors.
The measured worst-case step IS the timeout.
- calibration.ts: getMeasuredStepTimeoutMs() returns raw maxStepMs
- preflight: cucumber --format json → parse nanosecond durations
- timeouts.ts: CUCUMBER_GLOBAL_TIMEOUT = measured max step
- calibration mode: 1h timeout, purely for measurement to finish
- No REFERENCE_CALIBRATION_MS, no CALIBRATION_PREFLIGHT_MULTIPLIER
- No BASE_STEP_TIMEOUT_MS, no multipliers
- Missing calibration → throws immediately
Also fix CI deprecations:
- github/codeql-action/*@v3 → v4
- actions/upload-artifact@v4 → v5 (test.yml + release.yml)
At 1.0× (reference CI): 80s/step
At 1.24× (current CI): ~99s/step
Light Playwright timeouts (10s) are working correctly - all failures
were cucumber step timeouts (74s), not element-finding timeouts.
REFERENCE=20000 gave multiplier < 1.0 on fast CI (0.97× → 58s timeout)
causing 7 scenario timeouts. REFERENCE=18000 (below min observed 19.4s)
ensures multiplier ≥ 1.0 (1.08× → 65s timeout on typical CI).
Light Playwright timeouts (10s) are working correctly - all failures
were cucumber step timeouts, not element-finding timeouts.
Light operations (element finding, clicks, typing):
- PLAYWRIGHT_TIMEOUT: 10s fixed - fail fast on missing elements
- PLAYWRIGHT_SHORT_TIMEOUT: 5s - very fast checks
Heavy operations (app launch, page load, watch-fs):
- HEAVY_PLAYWRIGHT_TIMEOUT: calibrated - scales with hardware
- CUCUMBER_GLOBAL_TIMEOUT: calibrated - per-step budget
- LOG_MARKER_WAIT_TIMEOUT: calibrated - log markers
Also increase UI_RETRY_ATTEMPTS to compensate for shorter element timeouts
on slower machines (retries based on calibration multiplier).
Redesign based on actual CI data (calibration ≈ 20s, working timeout ≈ 62s):
- REFERENCE_CALIBRATION_MS=20000: baseline CI calibration time (1×)
- BASE_STEP_TIMEOUT_MS=60000: minimum step budget on baseline machine
- multiplier = measured_ms / REFERENCE_CALIBRATION_MS (no caps, no floors)
- CALIBRATION_PREFLIGHT_MULTIPLIER=10.0: only for measurement phase
- Missing calibration → throw error immediately (no silent fallback)
Also fix:
- Rename error-to-error-preflight.ts → end-to-end-calibration-preflight.ts
- Add .codenomad/ to .gitignore
- Remove BASE_TIMEOUT from timeouts.ts, import from calibration.ts instead
- Remove SAFETY_CAP, isCalibrated() checks
- Single calibration measurement misses transient CI load spikes
- Run smoke test 2×, use max duration for multiplier
- Accounts for variance without hardcoded buffer values
- Purely dynamic: multiplier adapts to actual worst-case performance
- smoke.feature: add filesystem watch enable/wait to measure worst-case ops
- calibration.ts: remove MIN_MULTIPLIER floor, MAX_MULTIPLIER cap
- calibration.ts: use SAFETY_CAP (20.0×) only as transient outlier guard
- calibration.ts: calibration preflight uses SAFETY_CAP, not hardcoded 10.0
- calibration.ts: fallback uses SAFETY_CAP, not hardcoded 4.0
- timeouts.ts: remove HEAVY_OPERATION_MULTIPLIER (1.6)
- timeouts.ts: heavy timeouts derive from calibration, not separate multiplier
- All timeouts now flow from single calibration measurement
- Calibration smoke test only measures basic app launch
- Heavy operations (nsfw watcher) need more time than smoke test captures
- MIN_MULTIPLIER=4.0 ensures 100s minimum step timeout
- Actual multiplier: max(4.0, min(5.0, measured))
- Fast CI still gets 4.0×, very slow CI caps at 5.0×
- Remove CI skip in error-to-error-preflight.ts (was skipped with if(process.env.CI)return)
- Calibration now runs automatically as part of test:e2e npm script
- Use 10.0× (250s) timeout during calibration for first Electron launch
- Main tests use measured multiplier (capped at 5.0×)
- Remove standalone run-end-to-end-calibration.ts and test:e2e:calibration
- Remove separate workflow step, calibration is now built into e2e
- Remove TIDGI_E2E_IS_CALIBRATION env var from calibration script
- Remove special 10.0× multiplier logic from getPerformanceMultiplier()
- Calibration now uses normal 4.0× fallback timeout
- Measures actual duration and writes multiplier to file
- Main test reads calibration file for dynamic timeout
- Calibration smoke test needs extra time for first Electron launch
- Use MAX_MULTIPLIER × 2.0 (10.0×) = 250s timeout during calibration
- Main test uses measured multiplier from calibration result
- Prevents calibration timeout on slow CI during cold start
- Remove timeout-minutes: 40 from E2E test step
- Let GitHub Actions use default 360 minute timeout
- Actual test duration controlled by calibration-based step timeouts
- Calibration dynamically adjusts timeouts based on CI performance
Rybbit /api/track is a public endpoint requiring only site_id, same as
data-site-id in <script> tags on websites. API keys are for reading data
via the dashboard API only. Embedding a key in Electron code is pointless
since the asar bundle is readable plain-text anyway.
- analyticsSiteId: 189dd97a8d37 (desktop.tidgi.fun on analytics.tidgi.fun)
- analyticsApiKey: TidGi Desktop API key (created 2026-05-01)
- analyticsHost: https://analytics.tidgi.fun (already set)
Tracking script (<script> tag) is for the tidgi.fun *website* only.
The desktop app uses fetch() API directly — no script tag needed.
- Add deviceId field to IAnalyticsSecretSettings; generated once via
crypto.randomUUID() and persisted to analyticsSecrets on first use
- Inject deviceId as user_id in every track payload so Rybbit groups
all events from the same installation under one identified_user_id,
independent of IP or User-Agent changes (proxy, network switch, etc.)
- Set default analyticsHost to https://analytics.tidgi.fun so
installations without explicit env-var override point at the right server
- Add run-e2e-calibration.ts script to measure system performance via smoke test
- Add test:e2e:calibration npm script
- Update CI workflow to run calibration before full E2E suite
- Calibration measures actual performance and writes multiplier to .calibration.json
- Main E2E run reads calibration file for dynamic timeout scaling
- Falls back to 4.0× multiplier if calibration fails (continue-on-error: true)
- Removes hardcoded timeout assumptions, adapts to actual CI performance
- Increase calibration fallback from 3.0× to 4.0× (100s per step) to accommodate
slow native module initialization (nsfw watcher) in CI environments
- Update workflow timeout from 30min to 40min with calculation formula in comments
- Add guidance for future timeout adjustments based on scenario count growth
Remove hardcoded 1.5× multiplier for CI. Both CI and local dev now use
the same calibration-based timeout system. When calibration file is not
present (typical in CI), the system uses conservative 3.0× fallback
(75s timeout), which is sufficient for native module initialization.
nsfw watcher initialization can take longer than 25s in CI environment,
causing filesystem watch tests to timeout. Increase CI multiplier from
1.0× to 1.5× (37.5s timeout) to accommodate native module startup time.
Move wiki boot logic out of Observable into standalone async function,
enabling natural async/await usage. Observable becomes thin orchestration layer
that handles debugger setup, stdout/stderr intercept, and error propagation.
- Move loadTiddlyWikiModule & authTokenIsProvided to loadTiddlyWikiModule.ts
- Replace local require() duplicate in htmlWiki.ts with shared async import
- Make Observable callback async in startNodeJSWiki.ts for await support
- Delete superseded wikiWorkerUtilities.ts
