Part VII · Reach and shipRelease operations
Chapter 38source checked

Release, update, and observability: three safety boundaries

T3 Code's release graph publishes an exact CLI runtime before clients can require it. Desktop, service, and mobile each cross an update boundary differently, while analytics and diagnostics distinguish opt-out product telemetry from local traces, optional OTLP export, authenticated browser ingestion, and demand-driven native resource history.

What this chapter resolves
  • Follow stable and nightly release-channel resolution through the exact-version publication invariant.
  • Compare the desktop, boot-service, and Expo update boundaries without treating them as one protocol.
  • Locate the point at which a server update becomes reversible, committed, and ready for client correlation.
  • Separate anonymous product analytics, local traces, optional remote exporters, authenticated browser traces, and ephemeral resource diagnostics.

A release is not one artifact copied to three clients. It is a dependency graph with a particular invariant: a client can offer to update a connected server to its own version only after the exact t3@<version> runtime exists. The mechanisms that consume that release then have different owners and different safe points: Electron exits and installs an application, a boot-service launcher trials an immutable server runtime around a SQLite snapshot, and Expo swaps a compatible JavaScript bundle only after mobile persistence and lifecycle checks.

The same precision matters for diagnostics. Product analytics, server traces, browser traces, metrics, and native resource samples are related operational signals, but they do not share a destination, retention model, or authorization boundary.

Act I — release graph: publish the runtime before exposing the client

The release workflow resolves two channels. A scheduled or explicitly nightly run uses the next patch version plus a UTC date and run number, publishes a GitHub prerelease, and uses npm’s nightly dist-tag. A stable run requires a semver-like version and produces a v<version> tag. The nightly lane is deliberately not the latest stable release and has its own desktop updater channel.

.github/workflows/release.yml:103–155 ↗verbatim · bash · 14a9355c
      - id: release_meta
        name: Resolve release version
        shell: bash
        env:
          DISPATCH_CHANNEL: ${{ github.event.inputs.channel }}
          DISPATCH_VERSION: ${{ github.event.inputs.version }}
          NIGHTLY_DATE: ${{ github.run_started_at }}
          NIGHTLY_SHA: ${{ github.sha }}
          NIGHTLY_RUN_NUMBER: ${{ github.run_number }}
        run: |
          if [[ "${GITHUB_EVENT_NAME}" == "schedule" || ( "${GITHUB_EVENT_NAME}" == "workflow_dispatch" && "${DISPATCH_CHANNEL:-stable}" == "nightly" ) ]]; then
            nightly_date="$(date -u -d "$NIGHTLY_DATE" +%Y%m%d)"
 
            node scripts/resolve-nightly-release.ts \
              --date "$nightly_date" \
              --run-number "$NIGHTLY_RUN_NUMBER" \
              --sha "$NIGHTLY_SHA" \
              --github-output
 
            echo "release_channel=nightly" >> "$GITHUB_OUTPUT"
            echo "cli_dist_tag=nightly" >> "$GITHUB_OUTPUT"
            echo "is_prerelease=true" >> "$GITHUB_OUTPUT"
            echo "make_latest=false" >> "$GITHUB_OUTPUT"
          else
            if [[ "${GITHUB_EVENT_NAME}" == "workflow_dispatch" ]]; then
              raw="${DISPATCH_VERSION}"
              if [[ -z "$raw" ]]; then
                echo "workflow_dispatch stable releases require the version input." >&2
                exit 1
              fi
            else
              raw="${GITHUB_REF_NAME}"
            fi
 
            version="${raw#v}"
            if [[ ! "$version" =~ ^[0-9]+\.[0-9]+\.[0-9]+([.-][0-9A-Za-z.-]+)?$ ]]; then
              echo "Invalid release version: $raw" >&2
              exit 1
            fi
 
            echo "release_channel=stable" >> "$GITHUB_OUTPUT"
            echo "version=$version" >> "$GITHUB_OUTPUT"
            echo "tag=v$version" >> "$GITHUB_OUTPUT"
            echo "name=T3 Code v$version" >> "$GITHUB_OUTPUT"
            echo "cli_dist_tag=latest" >> "$GITHUB_OUTPUT"
            if [[ "$version" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
              echo "is_prerelease=false" >> "$GITHUB_OUTPUT"
              echo "make_latest=true" >> "$GITHUB_OUTPUT"
            else
              echo "is_prerelease=true" >> "$GITHUB_OUTPUT"
              echo "make_latest=false" >> "$GITHUB_OUTPUT"
            fi
          fi
Read this as: The release workflow chooses nightly from scheduled or explicitly selected runs; otherwise it validates and resolves a stable version.

The important edge is downstream of naming. Connected servers are asked to install the client’s exact version, not “whatever npm currently calls latest.” The operations guide therefore requires publish_cli before release, and release before the hosted web deployment. This prevents a newly visible client from issuing an update request for a package version that cannot yet be fetched.

Figure 38.1 · Release availability is ordered around the exact server runtimeFollow the solid arrows. A client may be discoverable only after the exact runtime it can request is published.
Release dependency graph and exact-version server update invariantDiagram loading

Zoom with the controls, +/, or Ctrl/ + trackpad scroll. Enable Pan to drag, use two-finger scrolling, or use the arrow keys. 0 fits the diagram; Esc leaves Pan or expanded view.

Release dependency graph and exact-version server update invariant
Text equivalent

A channel resolver feeds exact CLI publication. CLI publication gates the GitHub release. The GitHub release gates hosted-web deployment and supplies desktop updater metadata. A desktop or hosted client can request a connected boot-service server to install that exact npm version. A nightly channel is separate metadata and a prerelease, not a different server update algorithm.

Figure 38.1. Stable and nightly first resolve a version/channel. The CLI package for that exact version publishes to npm before the GitHub release can expose desktop artifacts and updater metadata. The hosted client follows the GitHub release. Connected server self-update requests are version-pinned, so they resolve the same package—not an npm dist-tag at update time.
docs/operations/release.md:174–187 ↗verbatim · markdown · 22eb4423
## Server self-update release invariant
 
Connected servers update to the client's exact version, not to an npm dist-tag. Every released
desktop or hosted client version must therefore have a matching `t3@<version>` package available on
npm before users can receive that client.
 
The workflow enforces this ordering:
 
1. `publish_cli` publishes the exact stable or nightly version to npm.
2. `release` depends on `publish_cli` before exposing desktop artifacts in GitHub Releases.
3. `deploy_web` depends on `release` before moving the hosted channel to the new client.
 
Preserve these dependencies when changing the release graph. Publishing a client first would leave
the **Update server** action targeting a package version that does not exist yet.
Read this as: The release operations guide names the exact-version invariant and the CLI → release → web dependency order.

This is a release-time guarantee, not a claim that every machine is immediately updated. A disconnected server can remain on an earlier immutable runtime; a desktop client can decline to download; a mobile binary may be incompatible with an OTA bundle. The graph makes versions available in a safe order. The updater on each machine chooses whether and when to cross its own boundary.

Act II — three update architectures cross three different boundaries

1. Desktop: check automatically, but download and install by user action

The packaged desktop app configures an Electron updater for the selected channel. It may check on startup and on an interval, but it does not silently download or install. A check moves update state toward available; a user action starts the download; a later user action starts quit-and-install. A single active action reservation prevents overlapping check, download, channel-change, and install work.

apps/desktop/src/updates/DesktopUpdates.ts:334–458 ↗verbatim · typescript · 446f64f0
  const applyAutoUpdaterChannel = Effect.fn("desktop.updates.applyAutoUpdaterChannel")(function* (
    channel: DesktopUpdateChannel,
  ) {
    yield* Effect.annotateCurrentSpan({ channel });
    const allowsPrerelease = channel === "nightly";
    yield* electronUpdater.setChannel(channel);
    yield* electronUpdater.setAllowPrerelease(allowsPrerelease);
    yield* electronUpdater.setAllowDowngrade(allowsPrerelease);
    yield* electronUpdater.setFullChangelog(allowsPrerelease);
    yield* logUpdaterInfo("using update channel", {
      channel,
      allowPrerelease: allowsPrerelease,
      allowDowngrade: allowsPrerelease,
      fullChangelog: allowsPrerelease,
    });
  });
 
  const shouldEnableAutoUpdates = resolveDisabledReason.pipe(Effect.map(Option.isNone));
 
  const checkForUpdates = Effect.fn("desktop.updates.checkForUpdates")(function* (
    reason: string,
    actionReservation: "acquire" | "held" = "acquire",
  ) {
    yield* Effect.annotateCurrentSpan({ reason });
    if (yield* Ref.get(desktopState.quitting)) return false;
    if (!(yield* Ref.get(updaterConfiguredRef))) return false;
 
    const state = yield* Ref.get(updateStateRef);
    if (state.status === "downloading") {
      yield* logUpdaterInfo("skipping update check while update is active", {
        reason,
        status: state.status,
      });
      return false;
    }
 
    if (actionReservation === "acquire" && !(yield* tryStartUpdateAction("check"))) return false;
 
    const check = Effect.gen(function* () {
      const checkedAt = yield* currentIsoTimestamp;
      yield* setState(reduceDesktopUpdateStateOnCheckStart(state, checkedAt));
      yield* logUpdaterInfo("checking for updates", { reason });
 
      return yield* electronUpdater.checkForUpdates.pipe(
        Effect.as(true),
        Effect.catchTags({
          ElectronUpdaterCheckForUpdatesError: Effect.fn(
            "desktop.updates.handleCheckForUpdatesFailure",
          )(function* (error) {
            const failedAt = yield* currentIsoTimestamp;
            yield* updateState((current) =>
              reduceDesktopUpdateStateOnCheckFailure(current, error.message, failedAt),
            );
            yield* logUpdaterError(error.message, {
              errorTag: error._tag,
              channel: error.channel,
            });
            return true;
          }),
        }),
      );
    });
 
    return yield* actionReservation === "held"
      ? check
      : check.pipe(Effect.ensuring(finishUpdateAction("check")));
  });
 
  const downloadAvailableUpdate = Effect.gen(function* () {
    const state = yield* Ref.get(updateStateRef);
    if (!(yield* Ref.get(updaterConfiguredRef)) || state.status !== "available") {
      return { accepted: false, completed: false };
    }
 
    if (!(yield* tryStartUpdateAction("download"))) {
      return { accepted: false, completed: false };
    }
 
    return yield* Effect.gen(function* () {
      yield* setState(reduceDesktopUpdateStateOnDownloadStart(state));
      yield* electronUpdater.setDisableDifferentialDownload(
        isArm64HostRunningIntelBuild(environment.runtimeInfo),
      );
      yield* logUpdaterInfo("downloading update");
      yield* electronUpdater.downloadUpdate;
      return { accepted: true, completed: true };
    }).pipe(
      Effect.catchTags({
        ElectronUpdaterDownloadUpdateError: Effect.fn("desktop.updates.handleDownloadFailure")(
          function* (error) {
            yield* updateState((current) =>
              reduceDesktopUpdateStateOnDownloadFailure(current, error.message),
            );
            yield* logUpdaterError(error.message, {
              errorTag: error._tag,
              channel: error.channel,
            });
            return { accepted: true, completed: false };
          },
        ),
      }),
      Effect.onInterrupt(() =>
        updateState((current) => (current.status === "downloading" ? state : current)).pipe(
          Effect.asVoid,
        ),
      ),
      Effect.catchCause((cause) => {
        if (Cause.hasInterruptsOnly(cause)) {
          return Effect.failCause(cause);
        }
        const error = new DesktopUpdateUnexpectedActionError({ action: "download", cause });
        return Effect.gen(function* () {
          yield* updateState((current) =>
            reduceDesktopUpdateStateOnDownloadFailure(current, error.message),
          );
          yield* logUpdaterError(error.message, {
            errorTag: error._tag,
            action: error.action,
          });
          return { accepted: true, completed: false };
        });
      }),
      Effect.ensuring(finishUpdateAction("download")),
    );
  }).pipe(Effect.withSpan("desktop.updates.downloadAvailableUpdate"));
Read this as: Desktop checks are guarded against incompatible state; downloading is accepted only from an available update and owns a single update action reservation.

Install is a process-lifecycle boundary, not merely a file replacement. Before quitAndInstall, the app stops every backend in the pool—including parallel WSL and Windows instances—with a grace budget, destroys windows, then yields control to the updater. That avoids relying on an application shutdown cascade after the OS has begun quitting it.

apps/desktop/src/updates/DesktopUpdates.ts:465–575 ↗verbatim · typescript · c192a656
  const installDownloadedUpdate = Effect.gen(function* () {
    const state = yield* Ref.get(updateStateRef);
    const hasInstallableDownload =
      state.downloadedVersion !== null &&
      (state.status === "downloaded" ||
        (state.status === "error" &&
          (state.errorContext === null || state.errorContext === "install")));
    if (
      (yield* Ref.get(desktopState.quitting)) ||
      !(yield* Ref.get(updaterConfiguredRef)) ||
      !hasInstallableDownload
    ) {
      return { accepted: false, completed: false };
    }
 
    if (!(yield* tryStartUpdateAction("install"))) {
      return { accepted: false, completed: false };
    }
 
    yield* Ref.set(desktopState.quitting, true);
 
    return yield* Effect.gen(function* () {
      // Stop every backend in the pool, not just the primary. With
      // parallel WSL + Windows backends, leaving the WSL instance up
      // means quitAndInstall's app.quit() exits before the pool's
      // scope cascade has a chance to run its stop finalizer, so the
      // WSL child gets hard-killed by the OS instead of receiving
      // SIGTERM + grace. Stops run concurrently with the same 5s
      // budget the primary had on its own.
      const instances = yield* pool.list;
      yield* Effect.forEach(
        instances,
        (instance) => instance.stop({ timeout: Duration.seconds(5) }),
        { concurrency: "unbounded" },
      );
      yield* electronWindow.destroyAll;
      yield* electronUpdater.quitAndInstall({
        isSilent: true,
        isForceRunAfter: true,
      });
      return { accepted: true, completed: false };
    }).pipe(
      Effect.catchTags({
        ElectronUpdaterQuitAndInstallError: Effect.fn("desktop.updates.handleInstallFailure")(
          function* (error) {
            yield* resetInstallAction;
            yield* updateState((current) =>
              reduceDesktopUpdateStateOnInstallFailure(current, error.message),
            );
            yield* logUpdaterError(error.message, {
              errorTag: error._tag,
              channel: error.channel,
              isSilent: error.isSilent,
              isForceRunAfter: error.isForceRunAfter,
            });
            return { accepted: true, completed: false };
          },
        ),
      }),
      Effect.onInterrupt(() => resetInstallAction),
      Effect.catchCause((cause) =>
        Effect.gen(function* () {
          if (Cause.hasInterruptsOnly(cause)) {
            return yield* Effect.failCause(cause);
          }
          yield* resetInstallAction;
          const error = new DesktopUpdateUnexpectedActionError({ action: "install", cause });
          yield* updateState((current) =>
            reduceDesktopUpdateStateOnInstallFailure(current, error.message),
          );
          yield* logUpdaterError(error.message, {
            errorTag: error._tag,
            action: error.action,
          });
          return { accepted: true, completed: false };
        }),
      ),
    );
  }).pipe(Effect.withSpan("desktop.updates.installDownloadedUpdate"));
 
  const startUpdatePollers: Effect.Effect<void, never, Scope.Scope> = Effect.gen(function* () {
    yield* Effect.sleep(AUTO_UPDATE_STARTUP_DELAY).pipe(
      Effect.andThen(checkForUpdates("startup")),
      Effect.catchCause((cause) => {
        if (Cause.hasInterruptsOnly(cause)) {
          return Effect.void;
        }
        const error = new DesktopUpdatePollerError({ poller: "startup", cause });
        return logUpdaterError(error.message, {
          errorTag: error._tag,
          poller: error.poller,
        });
      }),
      Effect.forkScoped,
    );
    yield* Effect.sleep(AUTO_UPDATE_POLL_INTERVAL).pipe(
      Effect.andThen(checkForUpdates("poll")),
      Effect.forever,
      Effect.catchCause((cause) => {
        if (Cause.hasInterruptsOnly(cause)) {
          return Effect.void;
        }
        const error = new DesktopUpdatePollerError({ poller: "poll", cause });
        return logUpdaterError(error.message, {
          errorTag: error._tag,
          poller: error.poller,
        });
      }),
      Effect.forkScoped,
    );
  }).pipe(Effect.withSpan("desktop.updates.startPollers"));
Read this as: The desktop installer first marks the application as quitting, stops all pool instances, destroys windows, and then calls the Electron updater.

2. Boot-service server: stage, preflight, trial, then have the launcher commit

The remote server path is intentionally stricter because it can migrate the state that serves a remote client. It only exists for a server running under the supported background-service launcher; a desktop-managed server tells the user to update its desktop app, and an unsupported process shape gives manual-install guidance.

The active child first downloads an immutable exact-version runtime into staging and runs its __service-preflight. The preflight result must name the requested version and report readiness before the runtime is placed into its version directory. Only then does the child ask the stable launcher to activate the prepared version.

apps/server/src/cloud/selfUpdate.ts:64–169 ↗verbatim · typescript · 2024b45f
  const update: ServerSelfUpdate["Service"]["update"] = Effect.fn(
    "cloud.server_self_update.update",
  )(function* (input, reportProgress = () => Effect.void) {
    if (capability === "desktop-managed") {
      return yield* failWith(
        "This server is managed by the T3 Code desktop app on its machine; update the desktop app to update it.",
      );
    }
    if (capability === null) {
      return yield* failWith(
        "Remote updates require the T3 Code background service. Run `t3 service install` on the server machine.",
      );
    }
 
    const targetVersion = input.targetVersion.trim();
    if (!isExactServiceVersion(targetVersion)) {
      return yield* failWith(`'${targetVersion}' is not an exact t3 version.`);
    }
    if (yield* Ref.getAndSet(inFlight, true)) {
      return yield* failWith("A server update is already in progress.");
    }
 
    return yield* Effect.gen(function* () {
      yield* reportProgress("downloading");
      const paths = yield* ensurePinnedRuntimeInstalled({
        baseDir: serverConfig.baseDir,
        version: targetVersion,
        fs,
        path,
        runner,
        validate: (runtime) =>
          runner
            .run({
              command: execPath,
              args: [
                runtime.entryPath,
                "__service-preflight",
                "--database-path",
                serverConfig.dbPath,
                "--launcher-protocol",
                String(SERVICE_LAUNCHER_PROTOCOL),
              ],
              timeout: PREFLIGHT_TIMEOUT,
            })
            .pipe(
              Effect.mapError(
                (cause) =>
                  new PinnedRuntimeInstallError({
                    step: "running the staged service preflight",
                    cause,
                  }),
              ),
              Effect.flatMap(
                (
                  result,
                ): Effect.Effect<
                  void,
                  PinnedRuntimeInstallError | PinnedRuntimePreflightBlockedError
                > => {
                  if (result.code !== 0) {
                    return Effect.fail(
                      new PinnedRuntimeInstallError({
                        step: "running the staged service preflight",
                        exitCode: Number(result.code),
                        stdoutLength: result.stdout.length,
                        stderrLength: result.stderr.length,
                      }),
                    );
                  }
                  let parsed: unknown;
                  try {
                    parsed = JSON.parse(result.stdout.trim());
                  } catch (cause) {
                    return Effect.fail(
                      new PinnedRuntimeInstallError({
                        step: "decoding the staged service preflight",
                        cause,
                      }),
                    );
                  }
                  const preflight = decodeServicePreflightResult(parsed);
                  if (preflight === undefined || preflight.version !== targetVersion) {
                    return Effect.fail(
                      new PinnedRuntimeInstallError({
                        step: "verifying the staged service preflight",
                      }),
                    );
                  }
                  return preflight.status === "ready"
                    ? Effect.void
                    : Effect.fail(
                        new PinnedRuntimePreflightBlockedError({
                          version: targetVersion,
                          reason: preflight.reason,
                        }),
                      );
                },
              ),
            ),
      }).pipe(
        Effect.mapError((error) =>
          error._tag === "PinnedRuntimePreflightBlockedError"
            ? failWith(error.reason, error)
            : failWith(`Could not prepare t3@${targetVersion}.`, error),
        ),
      );
Read this as: The service update path rejects non-exact targets, prevents concurrent updates, stages the runtime, and requires a version-matching preflight result before handoff.

The stable launcher, rather than the child it is about to stop, owns durable active selection. It writes a pending trial, stops the old server after response-flush grace, snapshots SQLite database/WAL/shared-memory files, starts the target under an activation gate, and waits for prepared. Only the launcher commits the new active version and removes the snapshot. A failed or timed-out trial restores the snapshot and restarts the old version.

old child:  stage V → preflight V → request-update(V)
launcher:   record pending → stop old → snapshot SQLite → start V as trial
trial V:    migrate + bind + prepare roots → report prepared
launcher:   commit V → discard snapshot → let V open activation gate
apps/server/src/cloud/selfUpdate.ts:171–191 ↗verbatim · typescript · ce552ec0
      yield* reportProgress("installing");
      const updateId = yield* launcher
        .requestUpdate({ targetVersion, dbPath: serverConfig.dbPath })
        .pipe(
          Effect.mapError((error) =>
            failWith(
              error._tag === "ServiceLauncherRejectedError"
                ? error.reason
                : "Could not ask the service launcher to activate the prepared update.",
              error,
            ),
          ),
        );
 
      yield* Effect.logInfo("Server update prepared; handing off to the service launcher.", {
        updateId,
        targetVersion,
        runtimePath: paths.entryPath,
      });
      return { targetVersion, method: "boot-service" as const, updateId };
    }).pipe(Effect.onError(() => Ref.set(inFlight, false)));
Read this as: After staging succeeds, the active server reports progress and hands a target version/database path to the service launcher; the launcher returns a correlation update id.

The snapshot is a bounded rollback tool, not a universal undo. It covers the database, WAL, and shared-memory files around the trial; attachments and other state-directory files are outside that boundary. A launcher protocol that is too old blocks a target that needs this safety path. After a commit, ordinary service manager restart policy applies; there is no promise of a second automatic rollback for every later runtime failure.

3. Mobile: OTA eligibility is native compatibility plus a safe teardown moment

Mobile’s Expo Updates configuration uses a fingerprint runtime version. Inference: because the fingerprint is derived from native-project inputs, it is a compatibility gate between an OTA JavaScript bundle and the installed native binary—not a release label intended to synchronize mobile with npm.

apps/mobile/app.config.ts:159–180 ↗verbatim · typescript · 8fca9ce8
const config: ExpoConfig = {
  name: variant.appName,
  slug: "t3-code",
  platforms: ["ios", "android"],
  scheme: variant.scheme,
  version: "1.0.4",
  runtimeVersion: {
    // Fingerprint (not appVersion) so an OTA only reaches binaries whose native
    // project — native deps, config plugins, AND patches/ — matches the update.
    // With appVersion, every 0.1.0 build shares a runtime version, so a JS update
    // could land on a binary missing the native changes it needs and crash.
    policy: process.env.MOBILE_VERSION_POLICY ?? "fingerprint",
  },
  orientation: "portrait",
  icon: variant.assets.appIcon,
  userInterfaceStyle: "automatic",
  updates: {
    enabled: true,
    url: "https://u.expo.dev/d763fcb8-d37c-41ea-a773-b54a0ab4a454",
    checkAutomatically: "ON_LOAD",
    fallbackToCacheTimeout: 0,
  },
Read this as: The mobile app configuration sets Expo runtimeVersion to the fingerprint policy and declares OTA update settings.

After an eligible bundle downloads, automatic application is deferred to a safe background transition. The update coordinator flushes draft and outbox persistence, then checks that the app is still backgrounded and is not behind an app-initiated foreground handoff such as a picker. A failed flush or unsafe lifecycle state keeps the current runtime alive and rearms a later transition. Explicit user-requested update application is a separate immediate path.

Figure 38.2 · The updater owner changes with the artifact being replacedUse the three columns as different state machines, not steps of one universal updater.
Desktop, server, and mobile update boundariesDiagram loading

Zoom with the controls, +/, or Ctrl/ + trackpad scroll. Enable Pan to drag, use two-finger scrolling, or use the arrow keys. 0 fits the diagram; Esc leaves Pan or expanded view.

Desktop, server, and mobile update boundaries
Text equivalent

Desktop: check, user download, user install, stop backend pool, quit/install. Server: exact runtime stage, preflight, launcher pending state, SQLite snapshot, target trial, prepared signal, commit or rollback. Mobile: fingerprint-compatible OTA check/download, flush drafts/outbox, confirm safe true background, reload or defer.

Figure 38.2. Desktop keeps the user in charge of download and installation, then stops pooled children before application replacement. A boot-service server has a stable launcher trial an exact immutable runtime around a SQLite snapshot before commit. Mobile accepts only fingerprint-compatible OTA bundles and waits for persistence plus a safe lifecycle window before reloading JavaScript.

Act III — observability and privacy are separate lanes, not one data pipe

The server’s anonymous PostHog analytics layer is configured on by default but can be disabled with T3CODE_TELEMETRY_ENABLED. Events remain buffered in memory and are sent in batches only when delivery is enabled; the payload explicitly disables PostHog person profiles. The service supplies platform, architecture, client type, and T3 Code version alongside the event properties.

apps/server/src/telemetry/AnalyticsService.ts:31–44 ↗verbatim · typescript · f3c3ac2e
const TelemetryEnvConfig = Config.all({
  posthogKey: Config.string("T3CODE_POSTHOG_KEY").pipe(
    Config.withDefault("phc_XOWci4oZP4VvLiEyrFqkFjP4CZn55mjYYBMREK5Wd6m"),
  ),
  posthogHost: Config.string("T3CODE_POSTHOG_HOST").pipe(
    Config.withDefault("https://us.i.posthog.com"),
  ),
  enabled: Config.boolean("T3CODE_TELEMETRY_ENABLED").pipe(Config.withDefault(true)),
  flushBatchSize: Config.number("T3CODE_TELEMETRY_FLUSH_BATCH_SIZE").pipe(Config.withDefault(20)),
  maxBufferedEvents: Config.number("T3CODE_TELEMETRY_MAX_BUFFERED_EVENTS").pipe(
    Config.withDefault(1_000),
  ),
  wslDistroName: Config.string("WSL_DISTRO_NAME").pipe(Config.option),
});
Read this as: Telemetry configuration supplies default PostHog settings, an enabled switch, and bounded in-memory buffering settings.
apps/server/src/telemetry/AnalyticsService.ts:69–134 ↗verbatim · typescript · 80a1888a
export const make = Effect.gen(function* () {
  const telemetryConfig = yield* TelemetryEnvConfig;
  const httpClient = yield* HttpClient.HttpClient;
  const serverConfig = yield* ServerConfig.ServerConfig;
  const identifier = yield* getTelemetryIdentifier;
  const bufferRef = yield* Ref.make<ReadonlyArray<BufferedAnalyticsEvent>>([]);
  const clientType = serverConfig.mode === "desktop" ? "desktop-app" : "cli-web-client";
  const hostPlatform = yield* HostProcessPlatform;
  const hostArchitecture = yield* HostProcessArchitecture;
 
  const enqueueBufferedEvent = (event: string, properties?: Readonly<Record<string, unknown>>) =>
    Effect.flatMap(DateTime.now, (now) =>
      Ref.modify(bufferRef, (current) => {
        const appended = [
          ...current,
          {
            event,
            ...(properties ? { properties } : {}),
            capturedAt: DateTime.formatIso(now),
          } satisfies BufferedAnalyticsEvent,
        ];
 
        const next =
          appended.length > telemetryConfig.maxBufferedEvents
            ? appended.slice(appended.length - telemetryConfig.maxBufferedEvents)
            : appended;
 
        return [
          {
            size: next.length,
            dropped: next.length !== appended.length,
          } as const,
          next,
        ] as const;
      }),
    );
 
  const sendBatch = Effect.fn("AnalyticsService.sendBatch")(function* (
    events: ReadonlyArray<BufferedAnalyticsEvent>,
  ) {
    if (!telemetryConfig.enabled || !identifier) return;
 
    const payload = {
      api_key: telemetryConfig.posthogKey,
      batch: events.map((event) => ({
        event: event.event,
        distinct_id: identifier,
        properties: {
          ...event.properties,
          $process_person_profile: false,
          platform: hostPlatform,
          wsl: Option.getOrUndefined(telemetryConfig.wslDistroName),
          arch: hostArchitecture,
          t3CodeVersion: packageJson.version,
          clientType,
        },
        timestamp: event.capturedAt,
      })),
    };
 
    yield* HttpClientRequest.post(`${telemetryConfig.posthogHost}/batch/`).pipe(
      HttpClientRequest.bodyJson(payload),
      Effect.flatMap(httpClient.execute),
      Effect.flatMap(HttpClientResponse.filterStatusOk),
    );
  });
Read this as: Batch delivery exits when telemetry is disabled or no identifier is available, and marks the PostHog person profile as disabled.

Identity selection is an ordered, hashed ladder: a Codex account id if readable, else a Claude user id, else an installation-scoped anonymous UUID in the T3 home directory. The code hashes the chosen value with its source namespace before returning it. That reduces direct identifier exposure to the analytics service; it does not mean all event properties are automatically anonymous, so the payload schema remains an operational privacy boundary.

apps/server/src/telemetry/Identify.ts:248–299 ↗verbatim · typescript · faea0eb8
/**
 * getTelemetryIdentifier - Users are "identified" by finding the first match of the following, then hashing the value.
 * 1. ~/.codex/auth.json tokens.account_id
 * 2. ~/.claude.json userID
 * 3. ~/.t3/telemetry/anonymous-id
 */
export const getTelemetryIdentifierForHome = Effect.fn("getTelemetryIdentifierForHome")(
  function* (homeDirectory: string) {
    const codexAccountId = yield* getCodexAccountId(homeDirectory).pipe(
      Effect.catchTags({
        TelemetryIdentityReadError: (error) =>
          logTelemetryIdentityError(error).pipe(Effect.as(Option.none<string>())),
        TelemetryIdentityDecodeError: (error) =>
          logTelemetryIdentityError(error).pipe(Effect.as(Option.none<string>())),
      }),
    );
    if (Option.isSome(codexAccountId)) {
      return yield* hash("codex", codexAccountId.value);
    }
 
    const claudeUserId = yield* getClaudeUserId(homeDirectory).pipe(
      Effect.catchTags({
        TelemetryIdentityReadError: (error) =>
          logTelemetryIdentityError(error).pipe(Effect.as(Option.none<string>())),
        TelemetryIdentityDecodeError: (error) =>
          logTelemetryIdentityError(error).pipe(Effect.as(Option.none<string>())),
      }),
    );
    if (Option.isSome(claudeUserId)) {
      return yield* hash("claude", claudeUserId.value);
    }
 
    const anonymousId = yield* upsertAnonymousId.pipe(
      Effect.map(Option.some),
      Effect.catchTags({
        TelemetryIdentityReadError: (error) =>
          logTelemetryIdentityError(error).pipe(Effect.as(Option.none<string>())),
        TelemetryAnonymousIdGenerationError: (error) =>
          logTelemetryIdentityError(error).pipe(Effect.as(Option.none<string>())),
        TelemetryAnonymousIdPersistenceError: (error) =>
          logTelemetryIdentityError(error).pipe(Effect.as(Option.none<string>())),
      }),
    );
    if (Option.isSome(anonymousId)) {
      return yield* hash("anonymous", anonymousId.value);
    }
 
    return null;
  },
  Effect.tapError(logTelemetryIdentityError),
  Effect.orElseSucceed(() => null),
);
Read this as: Identifier resolution documents and implements the Codex → Claude → persisted anonymous-id fallback, hashing the first available source.

Tracing has a different default. The server creates a bounded local file trace sink and applies HTTP-header redaction. When an OTLP trace URL is configured, the local tracer also delegates to the remote exporter; metrics, by contrast, exist only when an OTLP metrics URL is configured. This is a local-plus-optional-remote trace lane, not a claim that metrics are retained locally.

apps/server/src/observability/Layers/Observability.ts:19–93 ↗verbatim · typescript · 17644127
export const ObservabilityLive = Layer.unwrap(
  Effect.gen(function* () {
    const config = yield* ServerConfig.ServerConfig;
    const attribution = yield* ResourceAttribution.ResourceAttribution;
 
    const traceReferencesLayer = Layer.mergeAll(
      Layer.succeed(Tracer.MinimumTraceLevel, config.traceMinLevel),
      Layer.succeed(References.TracerTimingEnabled, config.traceTimingEnabled),
      httpHeaderRedactionLayer,
    );
 
    const tracerLayer = Layer.unwrap(
      Effect.gen(function* () {
        const sink = yield* makeTraceSink({
          filePath: config.serverTracePath,
          maxBytes: config.traceMaxBytes,
          maxFiles: config.traceMaxFiles,
          batchWindowMs: config.traceBatchWindowMs,
          onFlush: (stats) =>
            attribution.record({
              component: "server-trace",
              operation: "append",
              logicalWriteBytes: stats.logicalWriteBytes,
              count: stats.count,
              durationMs: stats.durationMs,
            }),
        });
        const delegate =
          config.otlpTracesUrl === undefined
            ? undefined
            : yield* OtlpTracer.make({
                url: config.otlpTracesUrl,
                exportInterval: `${config.otlpExportIntervalMs} millis`,
                resource: {
                  serviceName: config.otlpServiceName,
                  attributes: {
                    "service.runtime": "t3-server",
                    "service.mode": config.mode,
                  },
                },
              });
 
        const tracer = yield* makeLocalFileTracer({
          filePath: config.serverTracePath,
          maxBytes: config.traceMaxBytes,
          maxFiles: config.traceMaxFiles,
          batchWindowMs: config.traceBatchWindowMs,
          sink,
          ...(delegate ? { delegate } : {}),
        });
 
        return Layer.mergeAll(
          Layer.succeed(Tracer.Tracer, tracer),
          BrowserTraceCollector.layer(sink),
        );
      }),
    ).pipe(Layer.provide(OtlpExporter.layerFlusher), Layer.provideMerge(otlpSerializationLayer));
 
    const metricsLayer =
      config.otlpMetricsUrl === undefined
        ? Layer.empty
        : OtlpMetrics.layer({
            url: config.otlpMetricsUrl,
            exportInterval: `${config.otlpExportIntervalMs} millis`,
            resource: {
              serviceName: config.otlpServiceName,
              attributes: {
                "service.runtime": "t3-server",
                "service.mode": config.mode,
              },
            },
          }).pipe(Layer.provideMerge(otlpSerializationLayer));
 
    return Layer.mergeAll(ServerLoggerLive, traceReferencesLayer, tracerLayer, metricsLayer);
  }),
Read this as: The observability layer builds local file tracing with redaction and optional OTLP delegation; metrics layer creation depends on an OTLP metrics endpoint.

Browser OTLP records follow another route: the server authenticates the raw request at the operate scope, then attempts to decode and record it locally. A decode or local-collection failure is logged and recovered; it does not prevent the original JSON body from being forwarded when a remote trace endpoint is configured. With no remote URL, the route returns no content after that best-effort local attempt. The authentication boundary is strict, while local recording and remote forwarding are deliberately independent delivery lanes.

apps/server/src/http.ts:146–199 ↗verbatim · typescript · 220e3509
export const otlpTracesProxyRouteLayer = HttpRouter.add(
  "POST",
  OTLP_TRACES_PROXY_PATH,
  Effect.gen(function* () {
    yield* authenticateRawRouteWithScope(AuthOrchestrationOperateScope);
    const request = yield* HttpServerRequest.HttpServerRequest;
    const config = yield* ServerConfig.ServerConfig;
    const otlpTracesUrl = config.otlpTracesUrl;
    const browserTraceCollector = yield* BrowserTraceCollector.BrowserTraceCollector;
    const httpClient = yield* HttpClient.HttpClient;
    const bodyJson = cast<unknown, OtlpTracer.TraceData>(yield* request.json);
 
    yield* Effect.try({
      try: () => decodeOtlpTraceRecords(bodyJson),
      catch: (cause) => new DecodeOtlpTraceRecordsError({ cause, bodyJson }),
    }).pipe(
      Effect.flatMap((records) => browserTraceCollector.record(records)),
      Effect.catch((cause) =>
        Effect.logWarning("Failed to decode browser OTLP traces", {
          cause,
          bodyJson,
        }),
      ),
    );
 
    if (otlpTracesUrl === undefined) {
      return HttpServerResponse.empty({ status: 204 });
    }
 
    return yield* httpClient
      .post(otlpTracesUrl, {
        body: HttpBody.jsonUnsafe(bodyJson),
      })
      .pipe(
        Effect.flatMap(HttpClientResponse.filterStatusOk),
        Effect.as(HttpServerResponse.empty({ status: 204 })),
        Effect.tapError((cause) =>
          Effect.logWarning("Failed to export browser OTLP traces", {
            cause,
            otlpTracesUrl,
          }),
        ),
        Effect.orElseSucceed(() =>
          HttpServerResponse.text("Trace export failed.", { status: 502 }),
        ),
      );
  }).pipe(
    Effect.catchTags({
      EnvironmentAuthInvalidError: HttpServerRespondable.toResponse,
      EnvironmentInternalError: HttpServerRespondable.toResponse,
      EnvironmentScopeRequiredError: HttpServerRespondable.toResponse,
    }),
  ),
);
Read this as: The browser trace proxy authenticates with the operate scope, attempts local decoding and recording without making that attempt fatal, and independently forwards the original JSON only when OTLP is configured.

Finally, native resource telemetry is diagnostics-oriented rather than an event archive. Its native sidecar keeps a one-hour in-memory, bounded ring. Periodic streaming is off until a diagnostics subscription is retained; explicit refresh still works. The model samples counters and process trees, so processes that begin and end between samples may not be seen. It is explicitly not syscall, eBPF, ETW, or endpoint-security tracing.

docs/internals/resource-telemetry.md:129–170 ↗verbatim · markdown · 5bc90ad0
### Native history and streaming
 
Every native sample is appended to a one-hour in-memory ring bounded to 3,600
snapshots, 20,000 retained process rows, and 64 MiB of retained history bytes.
History stays in the sidecar until a `readHistory` request and is returned in
bounded chunks. The first bound reached wins, so high process counts or large
process names and command lines shorten the effective history window.
 
Periodic snapshot streaming is disabled by default. The server enables it only
while at least one diagnostics subscription is retained. `sampleNow` remains
available for explicit refreshes and identity validation.
 
The server adjusts native sampling without restarting the sidecar:
 
- suspended, locked, low-power, or serious/critical thermal state: 15 seconds;
- battery: 5 seconds;
- normal AC: 1 second;
- unknown or stale power: 5 seconds in the background and 1 second while live
  diagnostics is open.
 
### Sampling limits
 
This is counter sampling, not syscall tracing.
 
- A process that starts and exits entirely between samples may not be observed.
- Cumulative CPU and I/O counters still provide accurate deltas for processes
  that survive across samples.
- Exact file paths, individual write syscalls, ETW events, eBPF events, and
  Endpoint Security events are outside this implementation.
 
Those deeper tracing systems can be added later as opt-in diagnostic modes
without changing the public `ResourceTelemetry` model.
 
## I/O semantics
 
The monitor preserves platform semantics instead of presenting all counters as
equivalent:
 
- Unix-like platforms report storage I/O counters exposed by `sysinfo`.
- Windows reports all process I/O bytes, not only disk bytes.
- Operating-system caches can prevent logical application reads or writes from
  appearing as physical storage bytes.
Read this as: The resource telemetry design specifies bounded in-memory native history, subscriber-driven streaming, sampleNow behavior, and counter-sampling limits.
Interactive operations lab · Chapter 38

One release, three replacement boundaries, four signal lanes

Choose a lens, select a teaching case, and move the marker yourself. The model shows the point at which each owner may proceed—and the boundary that keeps it from doing so.

Release graph · stable exact-version path

Publishing the CLI unblocks every client that can request a server update

The graph does not update a machine. It only makes a matching, exact runtime available before a client can advertise itself.

Guard
publish `t3@V` before a client at V is exposed
Owner now
release workflow
Failure boundary
a client cannot target a package that has not published

Release graph. Step 1.

Static operations ledger
FlowProceed only afterOwnerIf it cannot proceed
Exact server update`t3@<version>` publishes before client releaserelease workflow, then service launcherclient version cannot be offered before the matching runtime exists
Desktop applicationuser chooses download and then install; pooled backends stopdesktop updaterstate records an error or remains downloaded for a later install
Boot-service serverexact runtime preflight, SQLite snapshot, target reports preparedstable launcherrollback restores snapshot and selects the old runtime
Mobile OTAfingerprint-compatible bundle, flushed intent, safe true backgroundmobile update coordinatordefer to a later lifecycle transition
Diagnosticsdestination and authorization vary by laneanalytics/tracer/resource serviceslocal traces can remain local; resource history stays bounded in memory

The operational reading is therefore concise: release ordering makes an exact runtime available; each platform crosses its own replacement boundary; diagnostics retain, export, and authorize different things on purpose.

T3
Source-locked editionRead against fa219001d · 23 Aug 2026
Book search

Find a concept, module, or source path

Type two or more characters.