The owner ran the installed 0.1.0-beta.26 (sessions 64-71) and asked for the results to be checked, the two flaky tests fixed, bugs fixed, and a release if there were bugs. Read from a backup copy of the ledger and the logs (docs/spike-notes/23-owner-runs-2026-10-20.md). - FgWindow.IsNearWholeMultiplier: a window is a state's only within 10 % of a whole multiplier of two or more (FgSlices.Slice.IsActive); FgMixRecord.Parse reads stored rows by the same rule. Silent Hill 2 under Dynamic MFG listed x2.52 for 12.8 s - windows where the multiplier stepped; it now reads x2.0, x3.0, none and 7.36 % not classified. - NgxDriverState / CaptureSession: the driver is not asked after the game has gone, and CREATED | EVALUATE stand once a reading carried them. The last NGX reading, taken after the target exited, replaced every in-game one: none of the 14 answered sessions kept the two bits, so DLSS (driver-reported) and the NVIDIA App's mode (D63) never showed on an NGX-direct title. - CaptureOrchestrator: a launch says it ended on its own (Recorded) with the pids it recorded (Targets), so a process turned away while it ran - the game Steam starts after Lies of P hands itself over - is recorded when it ends; the launch's own process never is. - SessionSummaryOpener: one window per session; opening it again brings it forward. - DisplayTally: the sizes are the mode's last sample that saw the window (a game leaving exclusive fullscreen closes it first; five sessions stored none). - DlssText.Line: no line that says only "DLSS". - Tests: the guard's scan set includes the test host above the harness, which under a full gate releases modules while it starts its other classes - ModuleScanFailed (reproduced 1 run in 12 of the assembly with coverage; 15 of 15 green after), ProcessUnreadable on 2026-10-19. ShmDrainIntegrationTests.InjectAsync asks again on those two reasons only (4 times at most, 250 ms apart); the pause case waits for READY; every injection suite waits for the harness's flushed "presenting" line instead of 800 ms (tests/Shared/HarnessReady.cs). - Docs: 03_METRICS (the mix, the driver-reported rung, display sizes), 04_CAPTURE, 06_DATA_MODEL, 08_UI, 14_TESTING, 18_GPU_VENDOR_APIS, 19_SAFETY (direct starts counted by the crash safety - measured, the policy unchanged), LIMITATIONS, guide (troubleshooting, results), spike-notes section 23, HANDOFF items 20 and 21, CHANGELOG. Then, on the owner's word about Steam games (Play > Start directly says a Steam game may hand itself to Steam - the notice, the tooltip and Settings (en/vi/ja); why a note rather than a fixed Steam rule (beta.27 PR-1)): The owner, 2026-10-20: Steam games hand themselves to Steam; a note in the UI, or a fixed setting that Steam's games always go through the Steam client - the choice left open. The note: - GamePlayer: PlayResult.SteamGame - the game the agent starts (Start directly, or an App running as administrator, or a Steam record with no link) is one Steam's own records name; the notice then reads Play_Started_Steam_Format / Play_Elevated_Steam_Format. Store records are read once a press. - Strings: Play_ToolTip_Direct and Settings_PlayMode_Body say a Steam game may hand itself to Steam and the game Steam then starts is recorded; Settings adds that a few will not start outside Steam at all. en/vi/ja, ja marked review; Strings.Designer regenerated. - Why not the fixed rule: three of the five Steam games started directly on 2026-10-20 ran and were measured from their first frame, and a Vulkan game is measured only when FrameLedger starts it (Everhex Demo) - recorded in 08_UI. - Docs: 08_UI, guide/08-settings, LIMITATIONS, CHANGELOG. - Tests: GamePlayerTests (Steam started directly and elevated say it; a loose or GOG game does not). Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
312 lines
15 KiB
C#
312 lines
15 KiB
C#
// SPDX-License-Identifier: GPL-3.0-only
|
|
// Copyright (C) 2026 poli0981 - additional terms under GPLv3 section 7: see NOTICE
|
|
|
|
using System.Diagnostics;
|
|
using System.IO.MemoryMappedFiles;
|
|
using FluentAssertions;
|
|
using FrameLedger.Application.Capture;
|
|
using FrameLedger.Application.Consent;
|
|
using FrameLedger.Application.Persistence;
|
|
using FrameLedger.Domain.Consent;
|
|
using FrameLedger.Domain.Sessions;
|
|
using FrameLedger.Infrastructure.Persistence;
|
|
using FrameLedger.Infrastructure.Settings;
|
|
using FrameLedger.Testing;
|
|
|
|
namespace FrameLedger.Agent.Tests;
|
|
|
|
/// <summary>
|
|
/// The shipped Agent as a separate process (P2 PR-F): <c>--console capture</c> against our own harness, with a
|
|
/// scratch ledger under <c>--data-dir</c>, ends as ONE <c>sessions</c> row — the milestone's technical half,
|
|
/// from outside the process.
|
|
/// </summary>
|
|
/// <remarks>
|
|
/// <para>
|
|
/// <b>hook-harness is the only thing this suite ever injects into</b>, and
|
|
/// <c>TheTargetAndTheConsentRecordAreBothOurOwnHarness</c> asserts that on the suite itself, exactly as the
|
|
/// capture host's suite does. The consent record is written through the real store rather than through
|
|
/// <c>consent grant</c>, which refuses redirected stdin by design; the test is standing in for a human, so the
|
|
/// self-constraint pins WHICH executable it may stand in for.
|
|
/// </para>
|
|
/// <para>
|
|
/// <b>The scratch ledger is this test's, never the profile's.</b> <c>--data-dir</c> is the whole reason decision
|
|
/// D6 exists; every Agent this class starts is given one, and the directory is deleted with the test.
|
|
/// </para>
|
|
/// </remarks>
|
|
// ONE COLLECTION FOR EVERY CLASS THAT STARTS hook-harness, so they never run at the same time. xUnit
|
|
// parallelises across classes but never within a collection, and two harnesses of the same image are exactly
|
|
// the ambiguity TargetResolver refuses on ("2 processes share the image and none can be singled out") —
|
|
// measured 2026-09-10, when this class and its sibling first ran concurrently. Refusing was correct; the
|
|
// suites were wrong to be concurrent.
|
|
[Trait("Category", "Integration")]
|
|
[Collection("agent-harness")]
|
|
public sealed class AgentEndToEndTests : IDisposable
|
|
{
|
|
private static string Harness => Path.Combine(AppContext.BaseDirectory, "hook-harness.exe");
|
|
|
|
private static string Agent => Path.Combine(AppContext.BaseDirectory, "FrameLedger.Agent.exe");
|
|
|
|
private readonly string _dataDir = Path.Combine(Path.GetTempPath(), "fl-agent-e2e-" + Guid.NewGuid().ToString("N"));
|
|
|
|
/// <summary>
|
|
/// The ONE executable this suite may write a consent record for: our harness, copied under a run-unique name
|
|
/// (beta.16). The Agent resolves its target BY NAME (<c>TargetResolver</c>), and <c>build.ps1</c> runs the test projects as
|
|
/// parallel processes: a <c>hook-harness.exe</c> another project was still starting - its module list a partial copy -
|
|
/// is an unreadable process of the same name, which the resolver must count, and it refused <c>TargetAmbiguous</c>
|
|
/// (#269's first CI run). No other test starts a process of this name.
|
|
/// </summary>
|
|
private string ConsentedExecutable => Path.Combine(_dataDir, "hook-harness-" + Path.GetFileName(_dataDir) + ".exe");
|
|
|
|
private string Ledger => Path.Combine(_dataDir, LedgerPaths.DatabaseFileName);
|
|
|
|
public void Dispose()
|
|
{
|
|
try
|
|
{
|
|
Directory.Delete(_dataDir, recursive: true);
|
|
}
|
|
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException)
|
|
{
|
|
}
|
|
}
|
|
|
|
[Fact]
|
|
public void TheTargetAndTheConsentRecordAreBothOurOwnHarness()
|
|
{
|
|
StageTarget();
|
|
Path.GetFileName(Harness).Should().Be("hook-harness.exe");
|
|
Path.GetDirectoryName(Harness).Should().Be(AppContext.BaseDirectory.TrimEnd(Path.DirectorySeparatorChar));
|
|
Path.GetFileName(ConsentedExecutable).Should().StartWith("hook-harness-");
|
|
Path.GetDirectoryName(ConsentedExecutable).Should().Be(_dataDir, "the copy lives in this run's scratch directory and nowhere else");
|
|
File.ReadAllBytes(ConsentedExecutable).Should().Equal(File.ReadAllBytes(Harness), "the target IS hook-harness, byte for byte, under another name");
|
|
Path.GetFileName(Agent).Should().Be("FrameLedger.Agent.exe");
|
|
}
|
|
|
|
/// <summary>This run's copy of hook-harness, once per test; its overlay logs are this assembly's to remove (<c>17_HOOK_ENGINE</c> §Native logging).</summary>
|
|
private void StageTarget()
|
|
{
|
|
if (File.Exists(ConsentedExecutable))
|
|
{
|
|
return;
|
|
}
|
|
|
|
File.Exists(Harness).Should().BeTrue(
|
|
"hook-harness.exe must be staged beside the test binary (FrameLedger.DrainFixtures.targets). "
|
|
+ "This FAILS rather than skipping: an integration test that quietly does nothing when its fixture "
|
|
+ "is absent is a gate that cannot fail.");
|
|
Directory.CreateDirectory(_dataDir);
|
|
HarnessOverlayLogSweep.Own(_dataDir);
|
|
File.Copy(Harness, ConsentedExecutable);
|
|
}
|
|
|
|
private async Task<ConsentWriteOutcome> GrantConsentAsync()
|
|
{
|
|
StageTarget();
|
|
ExecutableFingerprint fingerprint = FrameLedger.Infrastructure.Io.ExecutableIdentity.Read(ConsentedExecutable)!.Value;
|
|
Directory.CreateDirectory(_dataDir);
|
|
LedgerDatabase db = await LedgerDatabase.OpenAsync(Ledger, ct: TestContext.Current.CancellationToken).ConfigureAwait(false);
|
|
await using (db.ConfigureAwait(false))
|
|
{
|
|
return await new SqliteGameConsentStore(db).RecordOperatorAcknowledgementAsync(new OperatorAcknowledgement
|
|
{
|
|
Fingerprint = fingerprint,
|
|
DisclosureVersion = OperatorDisclosure.AgentConsoleVersion,
|
|
AcknowledgedAt = DateTimeOffset.UtcNow,
|
|
Provenance = ConsentProvenance.AgentConsoleOperator,
|
|
}, TestContext.Current.CancellationToken).ConfigureAwait(false);
|
|
}
|
|
}
|
|
|
|
private async Task SetKillSwitchAsync(bool engaged)
|
|
{
|
|
Directory.CreateDirectory(_dataDir);
|
|
LedgerDatabase db = await LedgerDatabase.OpenAsync(Ledger, ct: TestContext.Current.CancellationToken).ConfigureAwait(false);
|
|
await using (db.ConfigureAwait(false))
|
|
{
|
|
await new SettingsKillSwitch(new SqliteSettingsStore(db)).SetAsync(engaged, TestContext.Current.CancellationToken).ConfigureAwait(false);
|
|
}
|
|
}
|
|
|
|
private async Task<Process> StartHarnessAsync(string arguments)
|
|
{
|
|
StageTarget();
|
|
File.Exists(Agent).Should().BeTrue("the Agent must be built and copied beside this test");
|
|
var p = Process.Start(new ProcessStartInfo(ConsentedExecutable, arguments) { UseShellExecute = false, CreateNoWindow = true, RedirectStandardOutput = true })!;
|
|
bool presenting = await HarnessReady.WaitUntilPresentingAsync(p, TestContext.Current.CancellationToken).ConfigureAwait(false);
|
|
if (!presenting && !p.HasExited)
|
|
{
|
|
p.Kill(entireProcessTree: true);
|
|
}
|
|
|
|
presenting.Should().BeTrue("the harness must reach its present loop before anything looks for it or is injected into it (HarnessReady)");
|
|
p.HasExited.Should().BeFalse("the harness must be running before the Agent looks for it");
|
|
return p;
|
|
}
|
|
|
|
private Process StartAgent(params string[] consoleArgs)
|
|
{
|
|
var psi = new ProcessStartInfo(Agent) { UseShellExecute = false, CreateNoWindow = true, RedirectStandardOutput = true, RedirectStandardError = true };
|
|
psi.ArgumentList.Add("--console");
|
|
psi.ArgumentList.Add("--data-dir");
|
|
psi.ArgumentList.Add(_dataDir);
|
|
foreach (string a in consoleArgs)
|
|
{
|
|
psi.ArgumentList.Add(a);
|
|
}
|
|
|
|
return Process.Start(psi)!;
|
|
}
|
|
|
|
private static async Task<(string Stdout, int Exit)> FinishAsync(Process p)
|
|
{
|
|
Task<string> err = p.StandardError.ReadToEndAsync(TestContext.Current.CancellationToken);
|
|
string stdout = await p.StandardOutput.ReadToEndAsync(TestContext.Current.CancellationToken).ConfigureAwait(false);
|
|
await p.WaitForExitAsync(TestContext.Current.CancellationToken).ConfigureAwait(false);
|
|
return (stdout + "\n---- stderr ----\n" + await err.ConfigureAwait(false) + $"\n---- exit {p.ExitCode} ----\n", p.ExitCode);
|
|
}
|
|
|
|
private static void Kill(Process p)
|
|
{
|
|
try
|
|
{
|
|
if (!p.HasExited)
|
|
{
|
|
p.Kill(entireProcessTree: true);
|
|
}
|
|
}
|
|
catch (InvalidOperationException)
|
|
{
|
|
}
|
|
|
|
p.Dispose();
|
|
}
|
|
|
|
[Fact]
|
|
public async Task AConsentedBoundedCaptureLandsAsOneHookedRowInTheScratchLedger()
|
|
{
|
|
// THE MILESTONE'S TECHNICAL HALF, from outside the process: the shipped Agent, our harness, one row.
|
|
(await GrantConsentAsync()).Should().Be(ConsentWriteOutcome.Written);
|
|
|
|
Process harness = await StartHarnessAsync("--real --hold-presenting 40");
|
|
try
|
|
{
|
|
using Process agent = StartAgent("capture", "--exe", ConsentedExecutable, "--seconds", "8");
|
|
(string output, int exit) = await FinishAsync(agent);
|
|
|
|
exit.Should().Be(0, output);
|
|
output.Should().Contain("ledger: session ").And.Contain("SAVED as sessions.id=", output);
|
|
output.Should().Contain("telemetry=l1", output);
|
|
|
|
LedgerDatabase db = await LedgerDatabase.OpenAsync(Ledger, ct: TestContext.Current.CancellationToken).ConfigureAwait(true);
|
|
await using (db.ConfigureAwait(true))
|
|
{
|
|
var sessions = new SqliteSessionRepository(db);
|
|
SessionRow row = (await sessions.ListRecentAsync(1, TestContext.Current.CancellationToken).ConfigureAwait(true)).Should().ContainSingle().Subject;
|
|
row.Tier.Should().Be(CaptureTier.Hooked);
|
|
row.Mode.Should().Be(CaptureMode.Attach);
|
|
row.ExitStatus.Should().Be(ExitStatus.Normal, "the harness was still presenting when the bounded capture ended");
|
|
row.FrameCount.Should().BeGreaterThan(100, "8 s at ~120/s");
|
|
row.Api.Should().Be("d3d11");
|
|
row.GuardTicksPublished.Should().BeGreaterThanOrEqualTo(1);
|
|
row.OverlayBuildId.Should().NotBeNullOrEmpty();
|
|
row.TelemetrySource.Should().Contain("l1", "the Agent composes L1 + L2 + L3 for every session");
|
|
FrameBlobs? frames = await sessions.FindFramesAsync(row.Id, TestContext.Current.CancellationToken).ConfigureAwait(true);
|
|
frames.Should().NotBeNull();
|
|
frames!.SampleCount.Should().Be(row.FrameCount);
|
|
}
|
|
|
|
Directory.EnumerateFiles(Path.Combine(_dataDir, "tmp"), "*.partial").Should().BeEmpty("finalize deletes the .partial");
|
|
}
|
|
finally
|
|
{
|
|
Kill(harness);
|
|
}
|
|
}
|
|
|
|
[Fact]
|
|
public async Task WithTheKillSwitchOnAConsentedGameIsRefusedAndNothingIsEverInjected()
|
|
{
|
|
// FR-2.4 from outside the process (decision D7): consent is present and enabled, the switch is on, the
|
|
// gate refuses with its own reason, and no ring ever exists in the target.
|
|
(await GrantConsentAsync()).Should().Be(ConsentWriteOutcome.Written);
|
|
await SetKillSwitchAsync(engaged: true);
|
|
|
|
Process harness = await StartHarnessAsync("--real --hold-presenting 20");
|
|
try
|
|
{
|
|
using Process agent = StartAgent("capture", "--exe", ConsentedExecutable, "--seconds", "3");
|
|
(string output, int exit) = await FinishAsync(agent);
|
|
|
|
output.Should().Contain(nameof(SessionEndReason.RefusedKillSwitch), output);
|
|
exit.Should().NotBe(0, output);
|
|
|
|
harness.HasExited.Should().BeFalse();
|
|
Action open = () => MemoryMappedFile.OpenExisting($@"Local\FrameLedger.Ring.{harness.Id}", MemoryMappedFileRights.Read).Dispose();
|
|
open.Should().Throw<FileNotFoundException>("no ring means no Overlay means no injection");
|
|
}
|
|
finally
|
|
{
|
|
Kill(harness);
|
|
}
|
|
}
|
|
|
|
/// <summary>
|
|
/// beta.16: the collision #269's first CI run hit. Another hook-harness.exe running beside this suite's run - another
|
|
/// test project's, or one left over - shares neither the copy's name nor its path, so the Agent's resolver never counts
|
|
/// it. Until beta.16 this suite consented to the staged hook-harness.exe itself, and a second one was a second match:
|
|
/// TargetAmbiguous, exit 6.
|
|
/// </summary>
|
|
[Fact]
|
|
public async Task AnotherHookHarnessRunningBesideTheSuiteDoesNotMakeTheTargetAmbiguous()
|
|
{
|
|
(await GrantConsentAsync()).Should().Be(ConsentWriteOutcome.Written);
|
|
Process stranger = Process.Start(new ProcessStartInfo(Harness, "--real --hold-presenting 30") { UseShellExecute = false, CreateNoWindow = true, RedirectStandardOutput = true })!;
|
|
Process harness = await StartHarnessAsync("--real --hold-presenting 30");
|
|
try
|
|
{
|
|
using Process agent = StartAgent("capture", "--exe", ConsentedExecutable, "--seconds", "3");
|
|
(string output, int exit) = await FinishAsync(agent);
|
|
|
|
exit.Should().Be(0, output);
|
|
output.Should().Contain("SAVED as sessions.id=", output);
|
|
output.Should().NotContain("TargetAmbiguous", output);
|
|
}
|
|
finally
|
|
{
|
|
Kill(harness);
|
|
Kill(stranger);
|
|
}
|
|
}
|
|
|
|
[Fact]
|
|
public async Task ConsentGrantRefusesRedirectedStdinAndWritesNothing()
|
|
{
|
|
StageTarget();
|
|
using Process agent = StartAgent("consent", "grant", "--exe", ConsentedExecutable);
|
|
(string output, int exit) = await FinishAsync(agent);
|
|
|
|
exit.Should().Be(2, output);
|
|
output.Should().Contain("stdin is redirected", output);
|
|
output.Should().Contain("OPERATOR SURFACE", "the Agent's disclosure names its own surface first");
|
|
|
|
using Process list = StartAgent("consent", "list");
|
|
(string listed, int listExit) = await FinishAsync(list);
|
|
listExit.Should().Be(0, listed);
|
|
listed.Should().Contain("nothing is enabled", listed);
|
|
}
|
|
|
|
[Fact]
|
|
public async Task TheAppsDiagFlagAnswersNotThisBinarysWithExitTwo()
|
|
{
|
|
// The four maintenance flags are real since P3 PR-8b and run against the PRODUCT directory (a registration,
|
|
// a scheduled task), so an end-to-end test never invokes them; --diag is the App's and still exits 2 here.
|
|
var psi = new ProcessStartInfo(Agent) { UseShellExecute = false, CreateNoWindow = true, RedirectStandardOutput = true, RedirectStandardError = true };
|
|
psi.ArgumentList.Add("--diag");
|
|
using Process agent = Process.Start(psi)!;
|
|
(string output, int exit) = await FinishAsync(agent);
|
|
|
|
exit.Should().Be(2, output);
|
|
output.Should().Contain("not this binary", output);
|
|
}
|
|
}
|