Reconnect, environments, notifications, and version skew
T3 Code recovers reachability one environment at a time: a registry leases a single supervisor and RPC session per environment, projections reconcile independently after a new generation, background work follows declared demand, notifications wake attention rather than synchronize state, and capability plus exact-version recovery makes skew explicit.
What this chapter resolves
- Distinguish a connection attempt, a session lease, a retry supervisor, and environment-scoped projections.
- Explain why a reconnect is not evidence that a shell or thread has synchronized.
- Trace the awareness relay and APNs path without treating a notification as authoritative state delivery.
- Choose capability checks, exact-version guidance, and server-update paths that remain safe under version skew.
A remote client does not have one global connection to “T3 Code.” It has a catalog of environments, and each environment is a separate authority with its own endpoint, credentials, session, caches, shell, threads, and failure history. That is the unit of recovery.
The distinction prevents two tempting but incorrect shortcuts:
- a new socket is not proof that a shell or thread is current; and
- a notification is not a state update merely because it mentions a thread.
Both are signals to begin or focus reconciliation. The environment server remains the authority, and the client becomes current through a new session plus the normal snapshot/subscription paths.
1. The catalog is a map of authorities, not a pool of interchangeable servers
The shared runtime loads persisted targets into a map keyed by EnvironmentId.
The registry holds environment-scoped service scopes and a per-environment lease
lock. Starting the runtime acquires supervisors for persisted targets, but a caller
asking to run an operation still resolves the specific environment at that moment.
This is the boundary that lets a phone or browser display several environments
without cross-wiring their sessions.
const entries =
yield* SubscriptionRef.make<ReadonlyMap<EnvironmentId, ConnectionCatalogEntry>>(initialEntries);
const networkStatus = yield* SubscriptionRef.make(yield* connectivity.status);
const serviceScopes = yield* SubscriptionRef.make<
ReadonlyMap<EnvironmentId, EnvironmentServiceScope>
>(new Map());
const platformEnvironmentIds = yield* Ref.make<ReadonlySet<EnvironmentId>>(new Set());
const persistedTargetsByEnvironment = yield* Ref.make<
ReadonlyMap<EnvironmentId, ConnectionTarget>
>(new Map(persistedTargets.map((target) => [target.environmentId, target])));
interface LeaseLock {
readonly semaphore: Semaphore.Semaphore;
readonly users: number;
}
const leaseLocks = yield* Ref.make<ReadonlyMap<EnvironmentId, LeaseLock>>(new Map());
const leaseLocksGuard = yield* Semaphore.make(1);
const started = yield* Ref.make(false);
const withLeaseLock = <A, E, R>(
environmentId: EnvironmentId,
effect: Effect.Effect<A, E, R>,
): Effect.Effect<A, E, R> =>
Effect.acquireUseRelease(
leaseLocksGuard.withPermits(1)(
Effect.gen(function* () {
const current = yield* Ref.get(leaseLocks);
const existing = current.get(environmentId);
if (existing !== undefined) {
yield* Ref.set(
leaseLocks,
new Map(current).set(environmentId, {
semaphore: existing.semaphore,
users: existing.users + 1,
}),
);
return existing.semaphore;
}
const semaphore = yield* Semaphore.make(1);
yield* Ref.set(leaseLocks, new Map(current).set(environmentId, { semaphore, users: 1 }));
return semaphore;
}),
),
(semaphore) => semaphore.withPermits(1)(effect),
(semaphore) =>
leaseLocksGuard.withPermits(1)(
Ref.update(leaseLocks, (current) => {
const existing = current.get(environmentId);
if (existing === undefined || existing.semaphore !== semaphore) {
return current;
}
const next = new Map(current);
if (existing.users === 1) {
next.delete(environmentId);
} else {
next.set(environmentId, {
semaphore,
users: existing.users - 1,
});
}
return next;
}),
),
).pipe(Effect.withSpan("EnvironmentRegistry.withLeaseLock"));
const getEntry = Effect.fn("EnvironmentRegistry.getEntry")(function* (
environmentId: EnvironmentId,
) {
const entry = (yield* SubscriptionRef.get(entries)).get(environmentId);
if (entry === undefined) {
return yield* new EnvironmentNotRegisteredError({
environmentId,
});
}
return entry;
});
const closeServiceScope = Effect.fn("EnvironmentRegistry.closeServiceScope")(function* (
environmentId: EnvironmentId,
) {
const current = yield* SubscriptionRef.get(serviceScopes);
const lease = current.get(environmentId);
if (lease === undefined) {
return;
}
const next = new Map(current);
next.delete(environmentId);
yield* SubscriptionRef.set(serviceScopes, next);
yield* Scope.close(lease.scope, Exit.void);
});
const createServiceScope = Effect.fn("EnvironmentRegistry.createServiceScope")(
(entry: ConnectionCatalogEntry) =>
Effect.uninterruptible(
Effect.gen(function* () {
const environmentId = entry.target.environmentId;
const scope = yield* Scope.make();
const supervisor = yield* EnvironmentSupervisor.make(entry, {
initiallyDesired: false,
}).pipe(
Effect.provideService(Connectivity.Connectivity, connectivity),
Effect.provideService(ConnectionDriver.ConnectionDriver, driver),
Effect.provideService(ConnectionWakeups.ConnectionWakeups, wakeups),
Scope.provide(scope),
Effect.onError(() => Scope.close(scope, Exit.void)),
);
yield* supervisor.connect;
yield* SubscriptionRef.update(serviceScopes, (current) => {
const next = new Map(current);
next.set(environmentId, { entry, supervisor, scope });
return next;
});
return supervisor;
}),
),
);
const acquireSupervisor = Effect.fn("EnvironmentRegistry.acquireSupervisor")(function* (
environmentId: EnvironmentId,
) {
return yield* withLeaseLock(
environmentId,
Effect.gen(function* () {
const entry = yield* getEntry(environmentId);
const existing = (yield* SubscriptionRef.get(serviceScopes)).get(environmentId);
if (existing !== undefined) {
if (Equal.equals(existing.entry, entry)) {
return existing.supervisor;
}
yield* closeServiceScope(environmentId);
}
return yield* createServiceScope(entry);
}),
);An environment id must travel with an object reference. The contracts make this
concrete for projects and threads: a ScopedThreadRef is (environmentId, threadId), not a thread id that can be looked up globally. A superficially
similar repository or a reused thread-shaped identifier in another environment
does not authorize a merge.
/** How a server can replace itself with another version when asked over RPC.
New servers only advertise the stable launcher-backed "boot-service" path;
"respawn" remains decodable for compatibility with older servers. */
export const ServerSelfUpdateMethod = Schema.Literals(["boot-service", "respawn"]);
export type ServerSelfUpdateMethod = typeof ServerSelfUpdateMethod.Type;
/** What update path a client should offer for a server: one of the RPC
self-update methods above, or "desktop-managed" when the backend's
version belongs to the T3 Code desktop app supervising it — updating the
app on that machine is the only way to update the server. */
export const ServerSelfUpdateCapability = Schema.Literals([
"boot-service",
"respawn",
"desktop-managed",
]);
export type ServerSelfUpdateCapability = typeof ServerSelfUpdateCapability.Type;
export const ExecutionEnvironmentCapabilities = Schema.Struct({
repositoryIdentity: Schema.Boolean.pipe(Schema.withDecodingDefault(Effect.succeed(false))),
connectionProbe: Schema.optionalKey(Schema.Boolean),
/** Server exposes the pull-request list, detail, activity, diff, and mutation APIs. Absent on
servers from before the pull-request workspace shipped, so clients must not probe them. */
pullRequests: Schema.optionalKey(Schema.Boolean),
/** Server understands thread.settle / thread.unsettle commands. Absent on
pre-settlement servers, so clients treat missing as unsupported and
never send the commands under version skew. */
threadSettlement: Schema.optionalKey(Schema.Boolean),
/** Server understands thread.snooze / thread.unsnooze commands. Same
version-skew contract as threadSettlement. */
threadSnooze: Schema.optionalKey(Schema.Boolean),
/** Server understands thread.pin / thread.unpin commands. Same
version-skew contract as threadSettlement. */
threadPinning: Schema.optionalKey(Schema.Boolean),
/** Server understands thread.pin.reorder (and orderKey on thread.pin).
Same version-skew contract as threadSettlement. */
threadPinReorder: Schema.optionalKey(Schema.Boolean),
/** Server understands regenerateTitle on thread.meta.update. Absent on
older servers, so clients hide the action instead of sending it. */
threadTitleRegeneration: Schema.optionalKey(Schema.Boolean),
/** The update path clients should offer for this server. Absent on
servers that must be relaunched manually (dev checkouts, Windows
foreground runs, pre-update servers). */
serverSelfUpdate: Schema.optionalKey(ServerSelfUpdateCapability),
/** Server can stream self-update progress before acknowledging the
restart. Clients fall back to server.updateServer when absent. */
serverSelfUpdateProgress: Schema.optionalKey(Schema.Boolean),
/** Agent-activity publishes (push notifications and Live Activities)
currently leave this environment: the publish opt-in is enabled and the
relay link credentials exist. Clients skip seeding a Live Activity when
this is false — no update would ever repaint it. Absent on older
servers, which may still publish, so only an explicit false skips. */
agentActivityPublishing: Schema.optionalKey(Schema.Boolean),
});
export type ExecutionEnvironmentCapabilities = typeof ExecutionEnvironmentCapabilities.Type;
export const ExecutionEnvironmentDescriptor = Schema.Struct({
environmentId: EnvironmentId,
label: TrimmedNonEmptyString,
platform: ExecutionEnvironmentPlatform,
serverVersion: TrimmedNonEmptyString,
capabilities: ExecutionEnvironmentCapabilities,
});
export type ExecutionEnvironmentDescriptor = typeof ExecutionEnvironmentDescriptor.Type;
export const EnvironmentConnectionState = Schema.Literals([
"connecting",
"connected",
"disconnected",
"error",
]);
export type EnvironmentConnectionState = typeof EnvironmentConnectionState.Type;
export const RepositoryIdentityLocator = Schema.Struct({
source: Schema.Literal("git-remote"),
remoteName: TrimmedNonEmptyString,
remoteUrl: TrimmedNonEmptyString,
});
export type RepositoryIdentityLocator = typeof RepositoryIdentityLocator.Type;
export const RepositoryIdentity = Schema.Struct({
canonicalKey: TrimmedNonEmptyString,
locator: RepositoryIdentityLocator,
rootPath: Schema.optionalKey(TrimmedNonEmptyString),
displayName: Schema.optionalKey(TrimmedNonEmptyString),
provider: Schema.optionalKey(TrimmedNonEmptyString),
owner: Schema.optionalKey(TrimmedNonEmptyString),
name: Schema.optionalKey(TrimmedNonEmptyString),
});
export type RepositoryIdentity = typeof RepositoryIdentity.Type;
export const ScopedProjectRef = Schema.Struct({
environmentId: EnvironmentId,
projectId: ProjectId,
});
export type ScopedProjectRef = typeof ScopedProjectRef.Type;
export const ScopedThreadRef = Schema.Struct({
environmentId: EnvironmentId,
threadId: ThreadId,
});Multi-environment presentation is a merge of views, not truth
The client can combine shell summaries from every catalog entry into one sidebar, connection list, or “needs attention” count. That is a presentation merge. Each input remains owned by its environment’s cache and its own snapshot/stream state. Do not de-duplicate two threads because their labels or Git remotes look alike, and do not use an environment A session to repair environment B cache.
This constraint is useful even when a user has one desktop server exposed through direct LAN, relay, and SSH-assisted routes: access and launch can differ while the saved environment identity decides which runtime scope owns recovery. Chapter 34 separates access from authority; this chapter follows the recovery consequence.
Zoom with the controls, +/−, or Ctrl/⌘ + trackpad scroll. Enable Pan to drag, use two-finger scrolling, or use the arrow keys. 0 fits the diagram; Esc leaves Pan or expanded view.
Text equivalent
The catalog contains Environment A and Environment B. Each has a separate supervisor, session generations, shell cache, and thread cache. Both can contribute summary information to one user interface, but their authoritative state does not merge. An awareness relay sends an APNs notification containing an environment and thread route. The notification causes the client to navigate and synchronize that environment; it is not itself a state snapshot.
packages/client-runtime/src/connection/registry.ts:156–289 ↗packages/client-runtime/src/connection/supervisor.ts:490–615 ↗packages/client-runtime/src/connection/supervisor.ts:643–755 ↗packages/client-runtime/src/state/shell.ts:138–254 ↗packages/client-runtime/src/state/threads.ts:212–338 ↗packages/contracts/src/environment.ts:31–130 ↗apps/mobile/src/features/agent-awareness/notificationPayload.ts:37–105 ↗2. One supervisor owns retry; a session performs one attempt
The terms describe deliberately different responsibilities.
| Layer | Owns | Does not own |
|---|---|---|
| Connection attempt / RPC session | one prepared endpoint, transport, initial configuration, probe, and close signal | retry ladder or another session after close |
| Environment supervisor | desired state, network/lifecycle inputs, the current lease, generation, backoff, and replacement | shell or thread reducer semantics |
| Environment shell/thread state | cached snapshot, synchronization status, sequence/cursor rules, and domain errors | raw transport creation or independent retry loops |
| UI component | selection and presentation | a socket, timer, or RPC client |
The supervisor clears the prepared connection and session together when a lease
ends. A successful attempt publishes one connected state with its generation,
then monitors either the session close or a wakeup probe. The main loop advances
the generation only for an established attempt; offline releases a lease and waits
for a signal rather than consuming retry attempts.
const runAttempt = Effect.fnUntraced(function* (
attempt: number,
generation: number,
lastFailure: ConnectionAttemptError | null,
pendingRetry: Option.Option<PendingRetryTrace>,
) {
yield* SubscriptionRef.set(prepared, Option.none());
const establishment = yield* Effect.raceAllFirst([
exitUnlessInterrupted(
establishTracedConnection(attempt, generation, lastFailure, pendingRetry),
).pipe(
Effect.map(
(exit): EstablishmentEvent => ({
_tag: "Completed",
exit,
}),
),
),
waitForEstablishmentInterrupt().pipe(
Effect.map(
(resetRetry): EstablishmentEvent => ({
_tag: "Interrupted",
resetRetry,
}),
),
),
Effect.sleep(CONNECTION_ESTABLISHMENT_TIMEOUT).pipe(
Effect.as<EstablishmentEvent>({ _tag: "TimedOut" }),
),
]);
if (establishment._tag === "Interrupted") {
return {
_tag: "Interrupted",
established: false,
stable: false,
resetRetry: establishment.resetRetry,
} satisfies AttemptOutcome;
}
if (establishment._tag === "TimedOut") {
return {
_tag: "Failure",
established: false,
stable: false,
failure: {
error: new ConnectionTransientError({
reason: "timeout",
detail: `${target.label} did not respond during connection setup.`,
}),
attemptSpan: Option.none(),
},
} satisfies AttemptOutcome;
}
if (Exit.isFailure(establishment.exit)) {
const isUnexpectedDefect =
!Cause.hasInterruptsOnly(establishment.exit.cause) &&
!establishment.exit.cause.reasons.some(Cause.isFailReason);
const outcome = failureFromExit(target, establishment.exit, false, false);
if (isUnexpectedDefect) {
const defect = establishment.exit.cause.reasons.find(Cause.isDieReason)?.defect;
yield* Effect.logError("Connection attempt failed with an unexpected defect.").pipe(
Effect.annotateLogs({
"environment.id": target.environmentId,
"environment.label": target.label,
"cause.reason_count": establishment.exit.cause.reasons.length,
...safeErrorLogAttributes(defect),
}),
);
}
return outcome;
}
const active = establishment.exit.value;
const currentIntent = yield* Ref.get(intent);
if (!currentIntent.desired || currentIntent.network === "offline") {
return {
_tag: "Interrupted",
established: false,
stable: false,
resetRetry: false,
} satisfies AttemptOutcome;
}
const connectedAt = yield* Clock.currentTimeMillis;
yield* SubscriptionRef.set(prepared, Option.some(active.lease.prepared));
yield* SubscriptionRef.set(session, Option.some(active.lease.session));
yield* setState({
desired: true,
network: currentIntent.network,
phase: "connected",
stage: null,
attempt,
generation,
lastFailure: null,
retryAt: null,
});
const connectedExit = yield* Effect.raceFirst(
active.lease.session.closed.pipe(
Effect.mapError(
(error): TracedAttemptFailure => ({
error,
attemptSpan: active.attemptSpan,
}),
),
),
monitorConnectedLease(active.lease).pipe(
Effect.mapError(
(error): TracedAttemptFailure => ({
error,
attemptSpan: active.attemptSpan,
}),
),
),
).pipe(exitUnlessInterrupted);
const connectedForMs = (yield* Clock.currentTimeMillis) - connectedAt;
if (Exit.isSuccess(connectedExit)) {
return {
_tag: "Interrupted",
established: true,
stable: connectedForMs >= BACKOFF_RESET_AFTER_MS,
resetRetry: connectedExit.value,
} satisfies AttemptOutcome;
}
return failureFromExit(target, connectedExit, true, connectedForMs >= BACKOFF_RESET_AFTER_MS);
}, Effect.ensuring(clearLease)); const run = Effect.fnUntraced(function* () {
let failureCount = 0;
let generation = 0;
let latestFailure: ConnectionAttemptError | null = null;
let pendingRetry = Option.none<PendingRetryTrace>();
const resetRetryLadder = () => {
failureCount = 0;
pendingRetry = Option.none();
};
for (;;) {
if (yield* Ref.getAndSet(resetRetryState, false)) {
failureCount = 0;
latestFailure = null;
pendingRetry = Option.none();
}
const currentIntent = yield* Ref.get(intent);
if (!currentIntent.desired) {
resetRetryLadder();
latestFailure = null;
yield* clearLease;
yield* setState(availableState(currentIntent, generation));
yield* waitForSignal;
continue;
}
if (currentIntent.network === "offline") {
yield* clearLease;
yield* setState(offlineState(currentIntent, generation, failureCount + 1, latestFailure));
const applicationActivated = yield* waitForSignal;
if (applicationActivated) {
resetRetryLadder();
}
continue;
}
const attempt = failureCount + 1;
const nextGeneration = generation + 1;
const outcome: AttemptOutcome = yield* Effect.scoped(
runAttempt(attempt, nextGeneration, latestFailure, pendingRetry),
);
// Consumed on every iteration so a stale marker can never leak into a
// later, unrelated failure.
const failedWakeProbe = yield* Ref.getAndSet(wakeProbeFailed, false);
if (outcome.established) {
generation = nextGeneration;
if (outcome.stable) {
resetRetryLadder();
latestFailure = null;
}
}
if (outcome._tag === "Interrupted") {
if (outcome.resetRetry) {
resetRetryLadder();
}
continue;
}
const attemptSpan: Option.Option<Tracer.Span> = outcome.failure.attemptSpan;
const error: ConnectionAttemptError = outcome.failure.error;
latestFailure = error;
if (error._tag === "ConnectionBlockedError") {
const blockedIntent = yield* Ref.get(intent);
yield* setState({
desired: blockedIntent.desired,
network: blockedIntent.network,
phase: "blocked",
stage: null,
attempt,
generation,
lastFailure: error,
retryAt: null,
});
const applicationActivated = yield* waitForSignal;
if (applicationActivated) {
resetRetryLadder();
}
continue;
}
if (failedWakeProbe) {
// The wake probe found a dead transport while the user is returning to
// the app, so reconnect immediately instead of sleeping the first
// backoff rung. Only this first attempt skips the ladder; if it fails
// too, normal backoff resumes.
resetRetryLadder();
yield* setState(connectingState(yield* Ref.get(intent), generation, 1, error));
continue;
}
failureCount += 1;
const delayMs = retryDelayMs(failureCount - 1);
pendingRetry = Option.map(attemptSpan, (previousAttempt) => ({
previousAttempt,
failureCount,
delayMs,
reason: error.reason,
}));
const failedIntent = yield* Ref.get(intent);
yield* setState({
desired: failedIntent.desired,
network: failedIntent.network,
phase: "backoff",
stage: null,
attempt,
generation,
lastFailure: error,
retryAt: (yield* Clock.currentTimeMillis) + delayMs,
});
const applicationActivated = yield* waitForRetrySignal(delayMs);
if (applicationActivated) {
resetRetryLadder();
}
}There can therefore be one reconnect owner per environment, not one per panel, route, subscription, or command. A second retry loop is harmful: it can race a close against a new session, reset backoff inconsistently, or attach a subscription to a lease that another loop is about to discard. Domain streams wait for the supervisor’s replacement session; expected domain failures can be handled without declaring the transport dead.
3. Reconciliation follows the replacement lease, domain by domain
The shell starts from a cache when available, but caches are presentation
continuity—not an authority handoff. On a new subscription session, the shell
loader obtains an authoritative HTTP snapshot before attempting cursor resume.
If that refresh fails, it omits the cached cursor so the socket fallback sends a
complete snapshot. A later synchronized marker is what upgrades a retained
snapshot to live when that protocol feature is available.
const applyItem = Effect.fn("EnvironmentShellState.applyItem")(function* (
item: OrchestrationShellStreamItem,
) {
if (item.kind === "synchronized") {
yield* Ref.set(awaitingCompletion, false);
yield* SubscriptionRef.update(state, (current) =>
Option.isSome(current.snapshot)
? { ...current, status: "live" as const, error: Option.none() }
: current,
);
return;
}
const current = yield* SubscriptionRef.get(state);
const nextSnapshot =
item.kind === "snapshot"
? item.snapshot
: Option.match(current.snapshot, {
onNone: () => null,
onSome: (snapshot) =>
item.sequence > snapshot.snapshotSequence
? applyShellStreamEvent(snapshot, item)
: snapshot,
});
if (nextSnapshot === null) {
return;
}
const waiting = yield* Ref.get(awaitingCompletion);
yield* SubscriptionRef.set(state, {
snapshot: Option.some(nextSnapshot),
status: waiting ? "synchronizing" : "live",
error: Option.none(),
});
if (item.kind === "snapshot") {
const session = yield* Ref.get(activeSubscriptionSession);
if (session !== null) {
yield* Ref.set(lastAuthoritativeSession, session);
}
}
yield* Queue.offer(persistence, nextSnapshot);
});
const foregroundResubscriptions = Option.match(wakeups, {
onNone: () => Stream.never,
onSome: (service) =>
service.changes.pipe(Stream.filter(ConnectionWakeups.shouldResubscribeAfterWakeup)),
});
yield* setSynchronizing;
yield* Effect.forkScoped(
subscribeDynamic(
ORCHESTRATION_WS_METHODS.subscribeShell,
Effect.fn("EnvironmentShellState.makeSubscribeInput")(function* (session) {
yield* Ref.set(activeSubscriptionSession, session);
const supportsCompletionMarker = yield* session.initialConfig.pipe(
Effect.map((config) => config.shellResumeCompletionMarker === true),
Effect.orElseSucceed(() => false),
);
yield* Ref.set(awaitingCompletion, supportsCompletionMarker);
yield* setSynchronizing;
// Foreground resubscriptions on the same live session can resume from
// the in-memory cursor. A new session reloads the authoritative HTTP
// snapshot so a valid cursor cannot preserve incomplete cached data.
const hasAuthoritativeSnapshot = (yield* Ref.get(lastAuthoritativeSession)) === session;
let canResume = hasAuthoritativeSnapshot;
let current = yield* SubscriptionRef.get(state);
if (!hasAuthoritativeSnapshot || Option.isNone(current.snapshot)) {
const prepared = yield* SubscriptionRef.get(supervisor.prepared).pipe(
Effect.flatMap(
Option.match({
onSome: Effect.succeed,
onNone: () =>
SubscriptionRef.changes(supervisor.prepared).pipe(
Stream.filter(Option.isSome),
Stream.map((value) => value.value),
Stream.runHead,
Effect.map(Option.getOrThrow),
),
}),
),
);
const httpSnapshot = yield* snapshotLoader.load(prepared);
if (Option.isSome(httpSnapshot)) {
yield* applyItem({ kind: "snapshot", snapshot: httpSnapshot.value });
canResume = true;
current = yield* SubscriptionRef.get(state);
}
}
// If the authoritative refresh failed, omit the cached cursor so the
// socket fallback sends a complete snapshot for this new session.
if (!canResume || Option.isNone(current.snapshot)) {
return supportsCompletionMarker ? { requestCompletionMarker: true as const } : {};
}
if (!supportsCompletionMarker) {
// Without a completion marker there is no synchronized signal for a
// resumed subscription, so report live immediately, like threads.
yield* SubscriptionRef.update(state, (value) => ({
...value,
status: "live" as const,
error: Option.none(),
}));
}
return {
afterSequence: current.snapshot.value.snapshotSequence,
...(supportsCompletionMarker ? { requestCompletionMarker: true as const } : {}),
};
}),
{
onExpectedFailure: (cause) => setStreamError(Cause.squash(cause)),
retryExpectedFailureAfter: "250 millis",
resubscribe: foregroundResubscriptions,
},
).pipe(Stream.runForEach(applyItem)),
);Thread detail has the same separation, with an additional history race. A fresh thread snapshot replaces all loaded history and advances a local history epoch. An older-page request captured before that epoch cannot merge afterward. The rule is intentionally local to a scoped thread: it protects one detail window without pretending to establish a global event order across all threads or environments.
const setSynchronizing = SubscriptionRef.update(state, (current) =>
current.status === "deleted"
? current
: {
...current,
status: "synchronizing" as const,
error: Option.none(),
},
);
const setReady = SubscriptionRef.update(state, (current) =>
current.status === "live" || current.status === "deleted"
? current
: {
...current,
status: "synchronizing" as const,
error: Option.none(),
},
);
const setDisconnected = Effect.gen(function* () {
yield* Ref.set(awaitingCompletion, false);
// The capability belongs to the session that advertised it. During a
// reconnect, a new prepared connection can exist before the new session's
// config arrives; leaving the old value would let loadOlderTurns send
// window parameters to a server that may not accept them (review
// finding). makeSubscribeInput re-sets it from the next session's config.
yield* Ref.set(paginationSupported, false);
yield* SubscriptionRef.update(state, (current) => ({
...current,
status: current.status === "deleted" ? current.status : statusWithoutLiveData(current.data),
}));
});
const setStreamError = (cause: Cause.Cause<unknown>) =>
Ref.set(awaitingCompletion, false).pipe(
Effect.andThen(
SubscriptionRef.update(state, (current) => ({
...current,
status:
current.status === "deleted" ? current.status : statusWithoutLiveData(current.data),
error: Option.some(formatThreadError(cause)),
})),
),
);
const setThread = Effect.fn("EnvironmentThreadState.setThread")(function* (
thread: OrchestrationThread,
// "keep" preserves the current page state (live events touch only loaded
// recent turns); a snapshot or merged page passes its own page state.
page: Option.Option<EnvironmentThreadPageState> | "keep",
) {
const waiting = yield* Ref.get(awaitingCompletion);
yield* SubscriptionRef.update(state, (current) => ({
data: Option.some(thread),
status: waiting ? ("synchronizing" as const) : ("live" as const),
error: Option.none(),
page: page === "keep" ? current.page : page,
}));
// Active threads can update many times per second and retain large tool
// payloads. The server remains the source of truth while a turn is active;
// persist once it settles so cache encoding stays off the streaming path.
if (shouldPersistThread(thread)) {
const snapshotSequence = yield* SubscriptionRef.get(lastSequence);
const currentPage = yield* SubscriptionRef.get(state).pipe(Effect.map((value) => value.page));
yield* Queue.offer(persistence, {
snapshotSequence,
thread,
// Persist the window boundary with the window's content so a cache
// restore can keep paging from where the loaded history ends.
...Option.match(currentPage, {
onNone: () => ({}),
onSome: (value) =>
({
page: {
beforeCursor: value.beforeCursor,
hasMore: value.hasMore,
snapshotSequence,
},
}) as const,
}),
});
}
});
const setDeleted = Effect.fn("EnvironmentThreadState.setDeleted")(function* () {
yield* Ref.set(awaitingCompletion, false);
yield* Ref.update(historyEpoch, (epoch) => epoch + 1);
yield* SubscriptionRef.set(state, {
data: Option.none(),
status: "deleted",
error: Option.none(),
page: Option.none(),
});
yield* cache.removeThread(environmentId, threadId).pipe(
Effect.catch((error) =>
Effect.logWarning("Could not remove the cached thread.").pipe(
Effect.annotateLogs({
environmentId,
threadId,
error: error.message,
}),
),
),
);
});
// Body of applyItem, running under applyLock.
const applyItemLocked = Effect.fn("EnvironmentThreadState.applyItemLocked")(function* (
item: OrchestrationThreadStreamItem,
) {
if (item.kind === "synchronized") {
yield* Ref.set(awaitingCompletion, false);
yield* SubscriptionRef.update(state, (current) =>
Option.isSome(current.data) && current.status !== "deleted"
? { ...current, status: "live" as const, error: Option.none() }
: current,
);
return;
}
if (item.kind === "snapshot") {
// A fresh snapshot replaces all loaded history, including older
// pages: a turn reverted while disconnected would otherwise survive
// in the preserved history with no event left to remove it. The
// epoch bump discards any older-page fetch racing this snapshot.
yield* Ref.update(historyEpoch, (epoch) => epoch + 1);
yield* SubscriptionRef.set(lastSequence, item.snapshot.snapshotSequence);
yield* setThread(item.snapshot.thread, pageStateFromSnapshot(item.snapshot.page));
return;The safe recovery order is therefore:
- retain the selected
(environmentId, threadId)address and any cache marked cached; - let that environment’s supervisor establish or retain a healthy lease;
- let shell and thread factories attach to the replacement session;
- replace or resume using their snapshot/sequence rules; and
- only then present the relevant projection as live.
No step says “replay every command because the connection returned.” Commands resolve their current environment runtime at execution time. Durable command and outbox semantics belong to Chapter 24 and Chapter 33; reconnect supplies reachability, not permission to duplicate intent.
4. Background work has demand and policy, not a hidden always-on connection
The phone reports client activity for each registered environment. Its baseline report includes visibility, focus, recent interaction, app state, and declared background scopes; retained subscriptions add only known scope types such as diagnostics or VCS status, reference-counted per environment. This lets the server choose what remains useful while the app is backgrounded without equating “installed” with “keep every stream alive.”
export const mobileBackgroundActivityReporterLayer = Layer.effectDiscard(
Effect.gen(function* () {
const registry = yield* EnvironmentRegistry;
const storage = yield* MobileStorage.MobileStorage;
const clientId = yield* storage.loadOrCreateAgentAwarenessDeviceId.pipe(
Effect.map((deviceId) => `mobile-${deviceId}`),
Effect.orElseSucceed(() => "ephemeral-mobile-client"),
);
const reportRequests = yield* Queue.sliding<void>(1);
const requestReport = () => Queue.offerUnsafe(reportRequests, undefined);
let appState = AppState.currentState;
const report = Effect.gen(function* () {
const observedAtMs = yield* Clock.currentTimeMillis;
const active = appState === "active";
const entries = yield* SubscriptionRef.get(registry.entries);
yield* Effect.forEach(
entries.keys(),
(environmentId) =>
registry
.run(
environmentId,
request(WS_METHODS.serverReportClientActivity, {
environmentId: environmentId as EnvironmentId,
clientId,
clientKind: "mobile",
visible: active,
focused: active,
recentlyInteracted: active,
appState: normalizeAppState(appState),
scopes: [
...BASELINE_SCOPES,
...retainedMobileBackgroundScopes(environmentId as EnvironmentId),
],
ttlMs: LEASE_TTL_MS,
observedAt: DateTime.makeUnsafe(observedAtMs),
}),
)
.pipe(Effect.ignore),
{ concurrency: "unbounded", discard: true },
);
}).pipe(Effect.withSpan("mobile.backgroundActivity.report"));
yield* Effect.acquireRelease(
Effect.sync(() => {
const removeScopeListener = onRetainedMobileBackgroundScopesChange(requestReport);
const subscription = AppState.addEventListener("change", (nextState) => {
appState = nextState;
requestReport();
});
return { removeScopeListener, subscription };
}),
({ removeScopeListener, subscription }) =>
Effect.sync(() => {
removeScopeListener();
subscription.remove();
}),
);
yield* SubscriptionRef.changes(registry.entries).pipe(
Stream.runForEach(() => Effect.sync(requestReport)),
Effect.forkScoped,
);
yield* Stream.fromQueue(reportRequests).pipe(
Stream.debounce("250 millis"),
Stream.runForEach(() => report),
Effect.forkScoped,
);
yield* Effect.sync(requestReport).pipe(
Effect.repeat(Schedule.spaced(`${REPORT_INTERVAL_MS} millis`)),
Effect.forkScoped,
);
}),function stableScopeKey(environmentId: EnvironmentId, scope: BackgroundScope): string {
switch (scope.type) {
case "server-config":
case "diagnostics":
return JSON.stringify([environmentId, scope.type]);
case "provider-status":
return JSON.stringify([environmentId, scope.type, scope.instanceId ?? null]);
case "vcs-status":
case "git-refs":
return JSON.stringify([environmentId, scope.type, scope.cwd]);
case "thread":
return JSON.stringify([environmentId, scope.type, scope.threadId]);
}
}
function scopeForSubscription(
observation: EnvironmentRpcSubscriptionObservation,
): BackgroundScope | null {
if (observation.method === WS_METHODS.subscribeResourceTelemetry) {
return { type: "diagnostics" };
}
if (observation.method !== WS_METHODS.subscribeVcsStatus) {
return null;
}
const input = observation.input as { readonly cwd?: unknown };
return typeof input.cwd === "string" ? { type: "vcs-status", cwd: input.cwd } : null;
}
export function retainedMobileBackgroundScopes(
environmentId: EnvironmentId,
): ReadonlyArray<BackgroundScope> {
return Array.from(retainedScopes.values(), (entry) =>
entry.environmentId === environmentId ? entry.scope : null,
).filter((scope): scope is BackgroundScope => scope !== null);
}
export function observeMobileBackgroundActivitySubscription(
observation: EnvironmentRpcSubscriptionObservation,
): Effect.Effect<Effect.Effect<void>> {
const scope = scopeForSubscription(observation);
if (scope === null) return Effect.succeed(Effect.void);
return Effect.sync(() => {
const environmentId = observation.environmentId as EnvironmentId;
const key = stableScopeKey(environmentId, scope);
const current = retainedScopes.get(key);
if (current) {
current.refCount += 1;
} else {
retainedScopes.set(key, { environmentId, scope, refCount: 1 });
notify();
}
return Effect.sync(() => {
const retained = retainedScopes.get(key);
if (!retained) return;
retained.refCount -= 1;
if (retained.refCount <= 0) {
retainedScopes.delete(key);
notify();
}
});
});This is a policy boundary. A background report can help the server decide whether to retain an eligible activity, but it does not authorize an arbitrary client timer, guarantee OS execution time, or make a notification channel a durable subscription. The current mobile reporter does have its own reporting cadence; that operational mechanism is not a reason for UI or labs to create autonomous reconnect timers.
5. Awareness relay → APNs is an attention path, not state synchronization
When agent-awareness publishing is enabled and relay credentials exist, the environment reads its projected thread and project shell, derives a deliberately small awareness state, signs a short-lived environment proof, and publishes it to the relay. The server skips an unchanged projected state. These are useful suppression and privacy properties: the payload is not a transcript, and the relay does not need a client to pretend that every provider stream event is a push notification.
export function shouldPublishAgentAwarenessEvent(event: OrchestrationEvent): boolean {
switch (event.type) {
case "thread.message-sent":
case "thread.turn-start-requested":
// These events express intent to start work, but the shell still contains
// the previous turn's terminal state until the provider acknowledges the
// new turn. Publishing that snapshot can queue a fresh "Done" alert just
// before the real running state arrives. Provider lifecycle events publish
// the authoritative starting/running state instead.
return false;
case "thread.proposed-plan-upserted":
case "thread.runtime-mode-set":
case "thread.interaction-mode-set":
return false;
case "thread.activity-appended":
return (
event.payload.activity.kind === "approval.requested" ||
event.payload.activity.kind === "approval.resolved" ||
event.payload.activity.kind === "provider.approval.respond.failed" ||
event.payload.activity.kind === "user-input.requested" ||
event.payload.activity.kind === "user-input.resolved" ||
event.payload.activity.kind === "runtime.error"
);
default:
return true;
}
}
export function agentAwarenessPublishIdentity(state: RelayAgentActivityState | null): string {
if (state === null) {
return "null";
}
const { updatedAt: _updatedAt, ...meaningfulState } = state;
return JSON.stringify(meaningfulState);
}
export function isAgentActivityPublishingEnabled(value: string | null): boolean {
return isAgentActivityPublishingEnabledValue(value);
}
export function resolveAgentActivityPublishingStartupState(input: {
readonly relayConfigured: boolean;
readonly publishEnabled: boolean;
}): "waiting-for-link" | "disabled" | "enabled" {
if (!input.relayConfigured) {
return "waiting-for-link";
}
return input.publishEnabled ? "enabled" : "disabled";
}
const RELAY_AGENT_ACTIVITY_DETAIL_MAX_LENGTH = 160;
const REDACTED_RELAY_AGENT_FAILURE_DETAIL = "The agent run failed.";
export function sanitizeRelayAgentActivityState(
state: RelayAgentActivityState | null,
): RelayAgentActivityState | null {
if (state === null) {
return null;
}
const { detail: _detail, ...rest } = state;
const detail = (state.phase === "failed" ? REDACTED_RELAY_AGENT_FAILURE_DETAIL : state.detail)
?.trim()
.slice(0, RELAY_AGENT_ACTIVITY_DETAIL_MAX_LENGTH)
.trim();
return detail ? { ...rest, detail } : rest; const publishThreadUnsafe = Effect.fn("publishThreadUnsafe")(function* (threadId: ThreadId) {
const publishAgentActivity = yield* readPublishAgentActivityEnabled.pipe(
Effect.orElseSucceed(() => false),
);
if (!publishAgentActivity) {
yield* Effect.logDebug("agent activity publish skipped; publication disabled", {
threadId,
});
return;
}
const relayConfig = yield* readRelayConfig.pipe(Effect.orElseSucceed(() => null));
if (!relayConfig) {
yield* Effect.logDebug("agent activity publish skipped; relay link credentials unavailable", {
threadId,
});
return;
}
const relayClient = yield* makeRelayClient(relayConfig);
const environmentId = yield* serverEnvironment.getEnvironmentId;
const publishState = (input: {
readonly projectId: string | null;
readonly state: RelayAgentActivityState | null;
readonly reason: string;
}) =>
Effect.gen(function* () {
const proof = yield* makePublishProof({
privateKey: cloudLinkKeyPair.privateKey,
relayIssuer: relayConfig.issuer,
environmentId,
threadId,
state: input.state,
jti: yield* crypto.randomUUIDv4,
});
yield* Effect.logInfo("publishing agent activity for thread", {
environmentId,
threadId,
projectId: input.projectId,
statePhase: input.state?.phase ?? null,
hasState: input.state !== null,
reason: input.reason,
});
const response = yield* relayClient.server.publishAgentActivity({
params: {
environmentId,
threadId,
},
payload: {
state: input.state,
proof,
},
});
yield* Effect.logInfo("agent activity publish completed", {
environmentId,
threadId,
ok: response.ok,
deliveries: deliveryStats(response.deliveries),
});
});
const thread = yield* snapshotQuery.getThreadShellById(threadId);
const project = Option.isSome(thread)
? yield* snapshotQuery.getProjectShellById(thread.value.projectId)
: Option.none<OrchestrationProjectShell>();
const snapshot = resolveAgentAwarenessRelayPublishSnapshot({
environmentId,
threadId,
thread,
project,
});
const publishIdentity = agentAwarenessPublishIdentity(snapshot.state);
const publishedStateByThread = yield* Ref.get(publishedStateByThreadRef);
if (publishedStateByThread.get(threadId) === publishIdentity) {
// The projection is back at (or never left) the last published state, so
// any pending deferred confirmation is moot. Leaving the deadline in
// place would let a much later transient null find it already expired
// and publish a tombstone immediately, skipping the deferral window.
publishConfirmDeadlines.delete(threadId);
yield* Effect.logDebug("agent activity publish skipped; projected state unchanged", {The relay’s APNs work is delivery work. It may queue, succeed, or report a failure; iOS may also throttle delivery. A device registration is not treated as locally enabled merely because notification permission exists: the mobile module records whether the relay actually accepted the registration. It briefly collapses bursty re-registration, yet preserves a later registration/replay opportunity so foreground reconciliation can repair drifted Live Activity presentation.
function aggregateNeedsAttention(aggregate: RelayAgentActivityAggregateState): boolean {
return aggregate.activities.some(
(row) => row.phase === "waiting_for_approval" || row.phase === "waiting_for_input",
);
}
function isAttentionPhase(phase: string): boolean {
return phase === "waiting_for_approval" || phase === "waiting_for_input";
}
// Honors the same per-event notification switches the push channel uses; a
// missing/corrupt preferences blob only disables nothing (matching how the
// liveActivitiesEnabled check treats it), since every registration writes one.
function alertAllowedForPhase(
preferences: RelayAgentAwarenessPreferences | null,
phase: string,
): boolean {
if (preferences === null) {
return true;
}
switch (phase) {
case "waiting_for_approval":
return preferences.notifyOnApproval;
case "waiting_for_input":
return preferences.notifyOnInput;
case "completed":
return preferences.notifyOnCompletion;
case "failed":
return preferences.notifyOnFailure;
default:
return false;
}
}
// Alert copy for an update whose aggregate contains threads that were NOT in an
// attention phase in the previously delivered aggregate. A null previous
// aggregate means there is no known baseline (fresh registration, replay after
// data loss) — alerting there would buzz on reconnect, not on a transition.
export function alertForAttentionTransition(input: {
readonly previousAggregate: RelayAgentActivityAggregateState | null;
readonly nextAggregate: RelayAgentActivityAggregateState;
readonly preferences: RelayAgentAwarenessPreferences | null;
}): ApnsLiveActivityAlert | null {
if (input.previousAggregate === null) {
return null;
}
const previouslyAttention = new Set(
input.previousAggregate.activities
.filter((row) => isAttentionPhase(row.phase))
.map((row) => row.threadId),
);
const newlyAttention = input.nextAggregate.activities.filter(
(row) =>
isAttentionPhase(row.phase) &&
!previouslyAttention.has(row.threadId) &&
alertAllowedForPhase(input.preferences, row.phase),
);
const first = newlyAttention[0];
if (!first) {
return null;
}
if (newlyAttention.length === 1) {
return { title: first.threadTitle, body: `${first.status}: ${first.projectTitle}` };
}
return {
title: `${newlyAttention.length} agents need attention`,
body: newlyAttention.map((row) => row.threadTitle).join(", "),
};
}
// Alert copy for an update whose aggregate contains threads that finished
// (Done/Failed) since the previously delivered aggregate — the mid-flight
// completion buzz while other agents keep the activity alive. Requires the
// thread to have been present and non-terminal before, so a baseline-less
// replay or a row that merely fell off the display cap never rings.
function newlyTerminalRows(
previousAggregate: RelayAgentActivityAggregateState | null,
nextAggregate: RelayAgentActivityAggregateState,
): ReadonlyArray<RelayAgentActivityAggregateState["activities"][number]> {
if (previousAggregate === null) {
return [];
}
const previousPhases = new Map(
previousAggregate.activities.map((row) => [row.threadId, row.phase]),
);
return nextAggregate.activities.filter((row) => {
if (row.phase !== "completed" && row.phase !== "failed") {
return false;
}
const previousPhase = previousPhases.get(row.threadId);
return (
previousPhase !== undefined && previousPhase !== "completed" && previousPhase !== "failed"
);
});
}
function isFreshTerminalRow(
row: RelayAgentActivityAggregateState["activities"][number],
nowMs: number,
): boolean {
const updatedAtMs = Option.match(DateTime.make(row.updatedAt), {
onNone: () => null,
onSome: (dt) => dt.epochMilliseconds,
});
return updatedAtMs !== null && nowMs - updatedAtMs <= TERMINAL_NOTIFICATION_FRESHNESS_MS;
}
export function alertForNewlyTerminal(input: {
readonly previousAggregate: RelayAgentActivityAggregateState | null;
readonly nextAggregate: RelayAgentActivityAggregateState;
readonly preferences: RelayAgentAwarenessPreferences | null;
readonly nowMs: number;
}): ApnsLiveActivityAlert | null {
const newlyTerminal = newlyTerminalRows(input.previousAggregate, input.nextAggregate).filter(
(row) =>
alertAllowedForPhase(input.preferences, row.phase) &&
// Replays of old aggregates (server restarts, redeliveries) repaint
// state without ringing; only fresh completions buzz.
isFreshTerminalRow(row, input.nowMs),
);
const first = newlyTerminal[0];
if (!first) {
return null;
}
if (newlyTerminal.length === 1) {
return { title: first.threadTitle, body: `${first.status}: ${first.projectTitle}` };
}
return {
title: `${newlyTerminal.length} agents finished`,
body: newlyTerminal.map((row) => row.threadTitle).join(", "),
};
}
// Alert copy for an end event carrying a terminal (Done/Failed) aggregate.
export function alertForTerminalAggregate(input: {
readonly aggregate: RelayAgentActivityAggregateState | null;
readonly preferences: RelayAgentAwarenessPreferences | null;
}): ApnsLiveActivityAlert | null {
const row = input.aggregate?.activities[0];
if (!row || (row.phase !== "completed" && row.phase !== "failed")) {
return null;
}
if (!alertAllowedForPhase(input.preferences, row.phase)) {
return null;
}
return { title: row.threadTitle, body: `${row.status}: ${row.projectTitle}` };
}
function shouldUpdateLiveActivity(input: {
readonly previousAggregate: RelayAgentActivityAggregateState | null;
readonly nextAggregate: RelayAgentActivityAggregateState;
readonly lastDeliveryAt: string | null;
readonly nowMs: number;
}): boolean {
if (!input.previousAggregate) {
return true;
}
if (JSON.stringify(input.previousAggregate) === JSON.stringify(input.nextAggregate)) {
return false;
}
if (input.previousAggregate.activeCount !== input.nextAggregate.activeCount) {
return true;
}
if (aggregateNeedsAttention(input.nextAggregate)) {
return true;
}
// A thread finishing must never be throttled away: when a completion and a
// new start land in the same window, activeCount is unchanged and the Done
// transition (and its alert) would otherwise be suppressed.
if (newlyTerminalRows(input.previousAggregate, input.nextAggregate).length > 0) {
return true;
}
const lastDeliveryAtMs =
input.lastDeliveryAt === null
? null
: Option.match(DateTime.make(input.lastDeliveryAt), {
onNone: () => Number.NaN,
onSome: (dt) => dt.epochMilliseconds,
});
return (
lastDeliveryAtMs === null ||
Number.isNaN(lastDeliveryAtMs) ||
input.nowMs - lastDeliveryAtMs >= MIN_LIVE_ACTIVITY_UPDATE_INTERVAL_MS
);const environmentConnections = new Map<EnvironmentId, SavedRemoteConnection>();
const activityPushTokenListeners = new WeakSet<LiveActivity<AgentActivityProps>>();
// Activity tokens the relay recently accepted, by acceptance time. The refresh
// runs on sign-in, every app foreground, and every environment-connection
// update, which arrive in bursts and spammed identical registrations. But the
// registration is not a pure no-op: the relay replays the current aggregate to
// this device on every accepted registration, and that replay is the
// foreground reconciliation that repairs drifted or orphaned activities. So
// dedupe only within a short window — bursts collapse to one request, while a
// foreground after real time away still triggers a replay. Cleared on
// sign-out/identity change alongside the device registration state.
const ACTIVITY_TOKEN_REREGISTER_INTERVAL_MS = 60_000;
const registeredActivityPushTokens = new Map<string, number>();
let pushTokenSubscription: { remove: () => void } | null = null;
let appStateSubscription: { remove: () => void } | null = null;
// Whether the relay has actually accepted this device's registration. The
// notification/Live Activity settings toggles must reflect this rather than
// only local iOS permission or saved preferences: if the registration request
// never succeeded, the device cannot receive anything, so the switches must
// not read as enabled.
export type AgentAwarenessRegistrationStatus = "unknown" | "pending" | "registered" | "failed";The notification response consumer protects the route boundary too. It validates the deep link, deduplicates a delivered response identifier, and only navigates. It does not mutate a thread from untrusted notification data. Once routed, the selected environment’s normal state runtime establishes what remains pending, completed, or deleted.
function encodeThreadDeepLink(input: {
readonly environmentId: string;
readonly threadId: string;
}): string | null {
if (input.environmentId.length === 0 || input.threadId.length === 0) {
return null;
}
return `/threads/${encodeURIComponent(input.environmentId)}/${encodeURIComponent(input.threadId)}`;
}
function normalizeThreadDeepLink(value: string): string | null {
if (
value.trim() !== value ||
value.startsWith("//") ||
value.includes("?") ||
value.includes("#")
) {
return null;
}
const parts = value.split("/");
if (parts.length !== 4 || parts[0] !== "" || parts[1] !== "threads") {
return null;
}
try {
return encodeThreadDeepLink({
environmentId: decodeURIComponent(parts[2] ?? ""),
threadId: decodeURIComponent(parts[3] ?? ""),
});
} catch {
return null;
}
}
export function extractAgentNotificationDeepLink(response: unknown): string | null {
const data = dataFromNotificationResponse(response);
const deepLink = data?.deepLink;
if (typeof deepLink === "string") {
const normalizedDeepLink = normalizeThreadDeepLink(deepLink);
if (normalizedDeepLink) {
return normalizedDeepLink;
}
}
const environmentId = data?.environmentId;
const threadId = data?.threadId;
if (typeof environmentId === "string" && typeof threadId === "string") {
return encodeThreadDeepLink({ environmentId, threadId });
}
return null;
}
export function routeAgentNotificationResponseOnce(input: {
readonly handledResponseIds: Set<string>;
readonly response: unknown;
readonly navigate: (deepLink: string) => void;
}): void {
const responseId = identifierFromNotificationResponse(input.response);
if (responseId && input.handledResponseIds.has(responseId)) {
return;
}
if (responseId) {
input.handledResponseIds.add(responseId);
}
const deepLink = extractAgentNotificationDeepLink(input.response);
if (deepLink) {
input.navigate(deepLink);
}A signal can wake recovery; it cannot replace synchronization
Choose a track and a deliberate event. The ledger shows which owner acts, what may be inferred, and the next authoritative read.
Connection supervision
One environment lease has closed
The environment-scoped supervisor clears its old session, records a new generation only after establishment, and owns any retry.
- Owner
- Environment supervisor for this environment
- Signal means
- The active RPC lease is no longer usable.
- Do not infer
- That shell or thread projections are current.
- Authoritative next step
- Obtain a replacement session, then refresh/subscribe the scoped projections.
Connection supervision: session closes.
Static recovery ledger
| Track | Event | Owner | Safe conclusion | Required follow-up |
|---|---|---|---|---|
| Connection | Socket closes | One environment supervisor | The old lease is unusable; retry policy belongs to the supervisor. | A replacement session changes generation; shell and thread synchronize independently. |
| Connection | Device goes offline | One environment supervisor | Release the active lease and wait for an external signal; do not spend retry attempts. | Reconnect only after online/wakeup input permits it. |
| Connection | Foreground wakeup | One environment supervisor | A healthy lease can be probed instead of discarded. | A failed probe leads into the same owned recovery path. |
| Notification | Approval push arrives | Mobile navigation and the selected environment state | The notification names a route-worthy event, not a thread snapshot. | Navigate, connect if needed, then reconcile shell/thread state from the environment. |
| Notification | Relay registration is accepted | Relay/device-registration boundary | The relay can replay its aggregate; a local permission alone was never delivery proof. | Use normal synchronization to repair state and remove stale UI assumptions. |
| Notification | APNs delivery is delayed or absent | Normal connection and synchronization path | There is no promised notification receipt or durable state transfer. | The app later converges through its environment supervisor and projections. |
6. Capability and exact-version recovery solve different kinds of skew
The environment descriptor contains a server version and capability record. Optional capability keys allow a newer client to hide or avoid a command that an older server never implemented; for example, an absent settlement or pinning capability means the client does not optimistically send that command. This is feature-level compatibility, not a claim that every pair of versions behaves the same.
/** How a server can replace itself with another version when asked over RPC.
New servers only advertise the stable launcher-backed "boot-service" path;
"respawn" remains decodable for compatibility with older servers. */
export const ServerSelfUpdateMethod = Schema.Literals(["boot-service", "respawn"]);
export type ServerSelfUpdateMethod = typeof ServerSelfUpdateMethod.Type;
/** What update path a client should offer for a server: one of the RPC
self-update methods above, or "desktop-managed" when the backend's
version belongs to the T3 Code desktop app supervising it — updating the
app on that machine is the only way to update the server. */
export const ServerSelfUpdateCapability = Schema.Literals([
"boot-service",
"respawn",
"desktop-managed",
]);
export type ServerSelfUpdateCapability = typeof ServerSelfUpdateCapability.Type;
export const ExecutionEnvironmentCapabilities = Schema.Struct({
repositoryIdentity: Schema.Boolean.pipe(Schema.withDecodingDefault(Effect.succeed(false))),
connectionProbe: Schema.optionalKey(Schema.Boolean),
/** Server exposes the pull-request list, detail, activity, diff, and mutation APIs. Absent on
servers from before the pull-request workspace shipped, so clients must not probe them. */
pullRequests: Schema.optionalKey(Schema.Boolean),
/** Server understands thread.settle / thread.unsettle commands. Absent on
pre-settlement servers, so clients treat missing as unsupported and
never send the commands under version skew. */
threadSettlement: Schema.optionalKey(Schema.Boolean),
/** Server understands thread.snooze / thread.unsnooze commands. Same
version-skew contract as threadSettlement. */
threadSnooze: Schema.optionalKey(Schema.Boolean),
/** Server understands thread.pin / thread.unpin commands. Same
version-skew contract as threadSettlement. */
threadPinning: Schema.optionalKey(Schema.Boolean),
/** Server understands thread.pin.reorder (and orderKey on thread.pin).
Same version-skew contract as threadSettlement. */
threadPinReorder: Schema.optionalKey(Schema.Boolean),
/** Server understands regenerateTitle on thread.meta.update. Absent on
older servers, so clients hide the action instead of sending it. */
threadTitleRegeneration: Schema.optionalKey(Schema.Boolean),
/** The update path clients should offer for this server. Absent on
servers that must be relaunched manually (dev checkouts, Windows
foreground runs, pre-update servers). */
serverSelfUpdate: Schema.optionalKey(ServerSelfUpdateCapability),
/** Server can stream self-update progress before acknowledging the
restart. Clients fall back to server.updateServer when absent. */
serverSelfUpdateProgress: Schema.optionalKey(Schema.Boolean),
/** Agent-activity publishes (push notifications and Live Activities)
currently leave this environment: the publish opt-in is enabled and the
relay link credentials exist. Clients skip seeding a Live Activity when
this is false — no update would ever repaint it. Absent on older
servers, which may still publish, so only an explicit false skips. */
agentActivityPublishing: Schema.optionalKey(Schema.Boolean),
});
export type ExecutionEnvironmentCapabilities = typeof ExecutionEnvironmentCapabilities.Type;
export const ExecutionEnvironmentDescriptor = Schema.Struct({
environmentId: EnvironmentId,
label: TrimmedNonEmptyString,
platform: ExecutionEnvironmentPlatform,
serverVersion: TrimmedNonEmptyString,
capabilities: ExecutionEnvironmentCapabilities,
});For broader incompatibility, the web compares normalized exact client and server version strings and stores dismissals per environment plus that version pair. It then chooses guidance from the advertised update capability: boot service/respawn, desktop-managed update, or manual relaunch. A dismissal is scoped to that observed pair, so a changed environment version can surface a new warning.
function normalizeVersion(version: string | null | undefined): string | null {
const trimmed = version?.trim();
return trimmed && trimmed.length > 0 ? trimmed : null;
}
export function resolveVersionMismatch(
serverVersion: string | null | undefined,
): VersionMismatch | null {
const normalizedClientVersion = normalizeVersion(APP_VERSION);
const normalizedServerVersion = normalizeVersion(serverVersion);
if (
!normalizedClientVersion ||
!normalizedServerVersion ||
normalizedClientVersion === normalizedServerVersion
) {
return null;
}
return {
clientVersion: normalizedClientVersion,
serverVersion: normalizedServerVersion,
hint: "Version mismatch. Try syncing the client and server to the same T3 Code version.",
};
}
export function resolveServerConfigVersionMismatch(
serverConfig: Pick<ServerConfig, "environment"> | null | undefined,
): VersionMismatch | null {
return resolveVersionMismatch(serverConfig?.environment.serverVersion);
}
/** The update path the connected server offers, or null when it only
supports a manual relaunch (older servers, dev checkouts, Windows). */
export function resolveServerSelfUpdateCapability(
serverConfig: Pick<ServerConfig, "environment"> | null | undefined,
): ServerSelfUpdateCapability | null {
return serverConfig?.environment.capabilities.serverSelfUpdate ?? null;
}
/** The command to hand users whose server cannot update itself. */
export function manualServerUpdateCommand(targetVersion: string): string {
return `npx t3@${targetVersion}`;
}
/** One sentence telling the user how to resolve version skew for a server,
matched to the update path it offers. */
export function serverUpdateGuidance(
capability: ServerSelfUpdateCapability | null,
serverLabel: string,
): string {
switch (capability) {
case "boot-service":
case "respawn":
return `Update the ${serverLabel} so they stay in sync.`;
case "desktop-managed":
return `Update the desktop app that runs the ${serverLabel}.`;
default:
return `Relaunch the ${serverLabel} with the copied command to sync them.`;
}
}
export function buildVersionMismatchDismissalKey(
environmentId: EnvironmentId,
mismatch: Pick<VersionMismatch, "clientVersion" | "serverVersion">,
): string {
return `${environmentId}:${mismatch.clientVersion}:${mismatch.serverVersion}`;
}
function readVersionMismatchDismissals(): VersionMismatchDismissals {
try {
return (
getLocalStorageItem(
VERSION_MISMATCH_DISMISSALS_STORAGE_KEY,
VersionMismatchDismissalsSchema,
) ?? { keys: [] }
);
} catch (error) {
console.error("Could not read version-mismatch dismissals.", error);
return { keys: [] };
}
}
function writeVersionMismatchDismissals(document: VersionMismatchDismissals): void {
try {
setLocalStorageItem(
VERSION_MISMATCH_DISMISSALS_STORAGE_KEY,
document,
VersionMismatchDismissalsSchema,
);
} catch (error) {
console.error("Could not persist version-mismatch dismissals.", error);
}
}
export function isVersionMismatchDismissed(dismissalKey: string | null | undefined): boolean {
if (!dismissalKey) {
return false;
}
return readVersionMismatchDismissals().keys.includes(dismissalKey);
}
export function dismissVersionMismatch(dismissalKey: string | null | undefined): void {
if (!dismissalKey) {
return;
}
const document = readVersionMismatchDismissals();
if (document.keys.includes(dismissalKey)) {
return;
}
writeVersionMismatchDismissals({When a launcher-managed server can self-update, it requires an exact target version, stages that pinned runtime, runs a preflight, and only then asks the launcher to activate it. The update naturally closes the old connection; the same environment supervisor handles the involuntary close and re-establishes the lease. Do not make the transport layer a process manager merely to disguise that restart.
export function resolveServerSelfUpdateCapability(input: {
readonly desktopManaged: boolean;
readonly launcherManaged: boolean;
}): ServerSelfUpdateCapability | null {
if (input.desktopManaged) return "desktop-managed" as const;
return input.launcherManaged ? ("boot-service" as const) : null;
}
export class ServerSelfUpdate extends Context.Service<
ServerSelfUpdate,
{
readonly update: (
input: ServerSelfUpdateInput,
reportProgress?: (stage: ServerSelfUpdateProgressStage) => Effect.Effect<void>,
) => Effect.Effect<ServerSelfUpdateResult, ServerSelfUpdateError>;
}
>()("t3/cloud/selfUpdate/ServerSelfUpdate") {}
export const make = Effect.fn("cloud.server_self_update.make")(function* () {
const serverConfig = yield* ServerConfig.ServerConfig;
const launcher = yield* ServiceLauncherClient.ServiceLauncherClient;
const runner = yield* ProcessRunner.ProcessRunner;
const fs = yield* FileSystem.FileSystem;
const path = yield* Path.Path;
const execPath = yield* HostProcessExecutablePath;
const inFlight = yield* Ref.make(false);
const capability: ServerSelfUpdateCapability | null =
serverConfig.mode === "desktop" ? "desktop-managed" : launcher.managed ? "boot-service" : null;
const failWith = (reason: string, cause?: unknown) =>
cause === undefined
? new ServerSelfUpdateError({ reason })
: new ServerSelfUpdateError({ reason, cause });
const update: ServerSelfUpdate["Service"]["update"] = Effect.fn(
"cloud.server_self_update.update",
)(function* (input, reportProgress = () => Effect.void) {
if (capability === "desktop-managed") {
return yield* failWith(
"This server is managed by the T3 Code desktop app on its machine; update the desktop app to update it.",
);
}
if (capability === null) {
return yield* failWith(
"Remote updates require the T3 Code background service. Run `t3 service install` on the server machine.",
);
}
const targetVersion = input.targetVersion.trim();
if (!isExactServiceVersion(targetVersion)) {
return yield* failWith(`'${targetVersion}' is not an exact t3 version.`);
}
if (yield* Ref.getAndSet(inFlight, true)) {
return yield* failWith("A server update is already in progress.");
}
return yield* Effect.gen(function* () {
yield* reportProgress("downloading");
const paths = yield* ensurePinnedRuntimeInstalled({
baseDir: serverConfig.baseDir,
version: targetVersion,
fs,
path,
runner,
validate: (runtime) =>
runner
.run({
command: execPath,
args: [
runtime.entryPath,
"__service-preflight",
"--database-path",
serverConfig.dbPath,
"--launcher-protocol",
String(SERVICE_LAUNCHER_PROTOCOL),
],
timeout: PREFLIGHT_TIMEOUT,
})
.pipe(
Effect.mapError(
(cause) =>
new PinnedRuntimeInstallError({
step: "running the staged service preflight",
cause,
}),
),
Effect.flatMap(
(
result,
): Effect.Effect<
void,
PinnedRuntimeInstallError | PinnedRuntimePreflightBlockedError
> => {
if (result.code !== 0) {
return Effect.fail(
new PinnedRuntimeInstallError({
step: "running the staged service preflight",
exitCode: Number(result.code),
stdoutLength: result.stdout.length,
stderrLength: result.stderr.length,
}),
);
}
let parsed: unknown;
try {
parsed = JSON.parse(result.stdout.trim());
} catch (cause) {
return Effect.fail(
new PinnedRuntimeInstallError({
step: "decoding the staged service preflight",
cause,
}),
);
}
const preflight = decodeServicePreflightResult(parsed);
if (preflight === undefined || preflight.version !== targetVersion) {
return Effect.fail(
new PinnedRuntimeInstallError({
step: "verifying the staged service preflight",
}),
);
}
return preflight.status === "ready"
? Effect.void
: Effect.fail(
new PinnedRuntimePreflightBlockedError({
version: targetVersion,
reason: preflight.reason,
}),
);
},
),
),
}).pipe(
Effect.mapError((error) =>
error._tag === "PinnedRuntimePreflightBlockedError"
? failWith(error.reason, error)
: failWith(`Could not prepare t3@${targetVersion}.`, error),
),
);
yield* reportProgress("installing");
const updateId = yield* launcher
.requestUpdate({ targetVersion, dbPath: serverConfig.dbPath })
.pipe(
Effect.mapError((error) =>
failWith(
error._tag === "ServiceLauncherRejectedError"
? error.reason
: "Could not ask the service launcher to activate the prepared update.",
error,
),
),
);
yield* Effect.logInfo("Server update prepared; handing off to the service launcher.", {
updateId,
targetVersion,
runtimePath: paths.entryPath,
});
return { targetVersion, method: "boot-service" as const, updateId };7. The compact rule set
Keep these rules together when extending any client surface:
- Address every cache, operation, notification route, and selection with its environment.
- Give each environment one supervisor and let that supervisor own session replacement and retry policy.
- Treat connection generation as a session-lease boundary, then let shell and thread state independently become live.
- Merge multi-environment summaries in presentation only; do not blend their authorities or caches.
- Declare background demand narrowly and accept that OS delivery is policy, not a durable stream.
- Treat relay/APNs activity as an attention signal; navigate and synchronize afterward.
- Gate new features by capability, surface exact-version skew honestly, and recover a server update through the ordinary reconnect owner.
That model is modest by design. It does not promise exactly-once notification delivery, a global merged event log, or a perpetual mobile socket. It does make the important recovery path legible: one environment, one lease owner, current state re-established from the authority.