DEV Community: StareBrain

Building an Android AI Agent, Part 12: AgentContext — Knowing What the User Is Doing Before They Ask

StareBrain — Mon, 02 Nov 2026 03:58:55 +0000

Building an Android AI Agent, Part 12: AgentContext — Knowing What the User Is Doing Before They Ask

In Part 11 we handled mid-task replanning — sending the current screen snapshot to the LLM when a step fails and the agent needs a new plan for the remaining steps.

That covers recovery. This part covers anticipation.

When a user opens StareBrain mid-task, the agent shouldn't need to ask "what are you trying to do?" It should already have a working model of the current context: which app is active, what the user was doing before the interrupt, and what they were likely about to do next.

That working model is AgentContext.

What AgentContext holds

data class AgentContext(
    val activePackage: String?,
    val previousPackage: String?,
    val activeScreenSnapshot: ScreenSnapshot?,
    val recentAppSwitches: List<AppSwitch>,
    val pendingTask: AgentTask?,
    val lastUserIntent: String?,
    val confidenceLevel: ContextConfidence,
    val capturedAt: Long = System.currentTimeMillis()
)

data class AppSwitch(
    val fromPackage: String?,
    val toPackage: String?,
    val timestamp: Long
)

enum class ContextConfidence { HIGH, MEDIUM, LOW }

recentAppSwitches is the key field. A sequence of app switches is a behavioral signal — someone who went from their email app to their calendar app to a task manager in quick succession is probably scheduling a meeting. That pattern is readable before the user says anything.

confidenceLevel determines how the agent surfaces its inference. HIGH auto-fills context into the next step. MEDIUM suggests and waits for confirmation. LOW asks explicitly.

Building the context from accessibility events

class AgentContextBuilder(
    private val maxSwitches: Int = 10
) {
    private val switches = ArrayDeque<AppSwitch>(maxSwitches)
    private var activePackage: String? = null
    private var activeSnapshot: ScreenSnapshot? = null

    fun onAppSwitch(fromPackage: String?, toPackage: String?) {
        if (switches.size >= maxSwitches) switches.removeFirst()
        switches.addLast(
            AppSwitch(
                fromPackage = fromPackage,
                toPackage = toPackage,
                timestamp = System.currentTimeMillis()
            )
        )
        activePackage = toPackage
    }

    fun onScreenUpdate(snapshot: ScreenSnapshot) {
        activeSnapshot = snapshot
    }

    fun build(
        pendingTask: AgentTask? = null,
        lastUserIntent: String? = null
    ): AgentContext {
        return AgentContext(
            activePackage = activePackage,
            previousPackage = switches.lastOrNull()?.fromPackage,
            activeScreenSnapshot = activeSnapshot,
            recentAppSwitches = switches.toList(),
            pendingTask = pendingTask,
            lastUserIntent = lastUserIntent,
            confidenceLevel = inferConfidence()
        )
    }

    private fun inferConfidence(): ContextConfidence {
        val recentCount = switches.count {
            System.currentTimeMillis() - it.timestamp < 60_000
        }
        return when {
            recentCount >= 3 -> ContextConfidence.HIGH   // active pattern
            recentCount >= 1 -> ContextConfidence.MEDIUM // some signal
            else -> ContextConfidence.LOW                // cold start
        }
    }
}

Three or more app switches in the last 60 seconds is a HIGH confidence signal — the user is actively doing something, and the pattern is readable. One switch is MEDIUM. A cold start with no recent activity is LOW.

Serializing context for the LLM

fun AgentContext.toPrompt(): String = buildString {
    appendLine("Current app: ${activePackage ?: "unknown"}")
    if (previousPackage != null) {
        appendLine("Previous app: $previousPackage")
    }
    if (recentAppSwitches.isNotEmpty()) {
        appendLine("Recent app switches:")
        recentAppSwitches.takeLast(5).forEach { switch ->
            appendLine("  ${switch.fromPackage} → ${switch.toPackage}")
        }
    }
    activeScreenSnapshot?.let {
        appendLine("Current screen:")
        appendLine(it.toPromptSummary())
    }
    lastUserIntent?.let {
        appendLine("Last known intent: $it")
    }
    appendLine("Context confidence: $confidenceLevel")
}

The last 5 app switches are enough. More than that adds noise without improving the inference. The LLM uses this to generate a step list that's already oriented toward the likely task rather than starting from a blank slate.

Using confidence level at the surface

fun buildInitialPrompt(context: AgentContext, userInstruction: String?): String {
    return when (context.confidenceLevel) {
        ContextConfidence.HIGH -> {
            // Auto-fill context, proceed directly
            "${context.toPrompt()}\n\nUser instruction: ${userInstruction ?: "continue current task"}"
        }
        ContextConfidence.MEDIUM -> {
            // Suggest and confirm
            "${context.toPrompt()}\n\nUser instruction: ${userInstruction ?: "unknown — suggest most likely next step and ask for confirmation"}"
        }
        ContextConfidence.LOW -> {
            // Ask explicitly
            userInstruction?.let {
                "${context.toPrompt()}\n\nUser instruction: $it"
            } ?: "No context available. Ask the user what they'd like to do."
        }
    }
}

The LOW case with no user instruction is the only moment the agent asks an open question. Every other path has enough signal to generate at least a suggested first step.

Privacy consideration

AgentContext observes everything the accessibility service can see. That's a lot. The context is never persisted beyond the current session — it lives in memory and is rebuilt on each service start. The capturedAt timestamp enforces a staleness check: context older than 5 minutes is treated as LOW confidence regardless of the switch count, because the user's situation may have changed completely.

fun AgentContext.isStale(thresholdMs: Long = 5 * 60 * 1000): Boolean {
    return System.currentTimeMillis() - capturedAt > thresholdMs
}

What's next

Part 13: wiring AgentContext into the full request pipeline — from accessibility event to LLM call to task execution, end to end.

Building an Android AI Agent, Part 11: Mid-Task Replanning Without Starting Over

StareBrain — Sun, 01 Nov 2026 04:15:07 +0000

Building an Android AI Agent, Part 11: Mid-Task Replanning Without Starting Over

In Part 10 we added an obstruction detector that handles spinners, dismissible overlays, and permission dialogs inline. When none of those apply and the expected screen simply isn't there, the executor returns a FAILED step with a reason.

That's where this part picks up. The task hasn't failed — two of four steps completed successfully. What failed is the plan for step three. The agent needs a new plan for the remaining steps, built on top of what already happened.

The problem with restarting from scratch

The naive recovery is to send the original instruction back to the LLM and regenerate the full step list. This has two problems.

First, it replays completed steps. If step one opened the right app and step two filled in a form, regenerating from scratch may produce a plan that tries to do both again — which either fails (the form is already filled) or causes a side effect (it submits twice).

Second, it throws away context. The LLM that generated the original plan didn't know what the screen looks like right now. The replanning call does — and that's the only reason to make it.

The right call: send the completed steps, the failed step, the failure reason, and the current screen snapshot. Ask for a revised plan for the remaining steps only.

The replan request

data class ReplanRequest(
    val originalInstruction: String,
    val completedSteps: List<AgentStep>,
    val failedStep: AgentStep,
    val failureReason: String,
    val currentSnapshot: ScreenSnapshot,
    val remainingStepCount: Int
)

data class ReplanResponse(
    val revisedSteps: List<AgentStep>,
    val explanation: String
)

The completedSteps tell the LLM what has already happened and must not be repeated. The failedStep and failureReason tell it what went wrong. The currentSnapshot tells it what the screen actually looks like right now. The remainingStepCount gives it a soft target — how many steps were originally planned for the rest of the task, so it doesn't over-engineer the recovery.

Serializing the snapshot for the LLM

The ScreenSnapshot needs to be readable by the LLM without overwhelming the context window. A flat summary works better than a full node dump.

fun ScreenSnapshot.toPromptSummary(): String {
    val sb = StringBuilder()
    sb.appendLine("Current screen: ${packageName ?: "unknown"}")
    if (visibleTexts.isNotEmpty()) {
        sb.appendLine("Visible text (first 10):")
        visibleTexts.take(10).forEach { sb.appendLine("  - $it") }
    }
    if (interactableIds.isNotEmpty()) {
        sb.appendLine("Interactable elements (first 10):")
        interactableIds.take(10).forEach { sb.appendLine("  - $it") }
    }
    return sb.toString()
}

Capping at 10 items each keeps the prompt tight. The LLM doesn't need the full accessibility tree — it needs enough to understand what's on screen and what's tappable.

The replan call

suspend fun requestReplan(
    request: ReplanRequest,
    apiClient: StareBrainApiClient
): ReplanResponse {
    val prompt = buildString {
        appendLine("Original instruction: ${request.originalInstruction}")
        appendLine()
        appendLine("Completed steps:")
        request.completedSteps.forEachIndexed { i, step ->
            appendLine("  ${i + 1}. ${step.instruction} [DONE]")
        }
        appendLine()
        appendLine("Failed step: ${request.failedStep.instruction}")
        appendLine("Failure reason: ${request.failureReason}")
        appendLine()
        appendLine("Current screen state:")
        appendLine(request.currentSnapshot.toPromptSummary())
        appendLine()
        appendLine("Generate revised steps for the remaining task only.")
        appendLine("Do not repeat completed steps.")
        appendLine("Target approximately ${request.remainingStepCount} steps.")
    }
    return apiClient.replan(prompt)
}

Merging the revised steps back into the task

fun AgentTask.withRevisedSteps(revised: List<AgentStep>): AgentTask {
    val kept = steps.take(currentStepIndex) // completed steps stay
    val merged = kept + revised
    return copy(
        steps = merged,
        status = TaskStatus.IN_PROGRESS
    )
}

The completed steps are preserved exactly. The revised steps replace everything from the failure point forward. The executor picks up from currentStepIndex as normal — it doesn't know a replan happened.

When not to replan

Not every failure deserves a replan. Two cases where replanning is the wrong call:

Repeated failures on the same step. If the revised plan hits the same failure on the same screen twice, the screen state is genuinely unexpected and the LLM isn't going to reason its way out of it. Surface the failure to the user with enough context to decide whether to continue or abandon.

Confirmed steps that failed. If the user confirmed an action and it still failed, that's a product bug, not a planning problem. Replanning would just generate another confirmation prompt for the same broken action.

fun shouldReplan(step: AgentStep, task: AgentTask): Boolean {
    if (step.result?.confirmedBy != null) return false // confirmed action failed — surface to user
    val previousAttempts = task.steps.count { 
        it.instruction == step.instruction && it.status == StepStatus.FAILED 
    }
    return previousAttempts < 2
}

What's next

Part 12: the AgentContext system — how StareBrain builds and maintains a model of what the user is doing across app switches, so the agent has context before the user says anything.

Building an Android AI Agent, Part 10: Handling Unexpected Screens Mid-Task

StareBrain — Sat, 31 Oct 2026 05:30:17 +0000

Building an Android AI Agent, Part 10: Handling Unexpected Screens Mid-Task

In Part 9 we built a flat task executor that walks a step list with a cursor, pauses for confirmation, and keeps state on the client.

The assumption baked into that executor: when the agent arrives at a step, the expected screen is there.

It often isn't.

A permission dialog appears. The app updated overnight and the UI changed. A loading spinner is still running. A previous step succeeded but triggered a side effect — an ad, a rate prompt, a "what's new" overlay.

The agent needs to handle this without asking the user every time, and without silently doing the wrong thing.

What "unexpected" actually means

There are three distinct cases:

Transient obstruction — a dialog, overlay, or spinner that wasn't part of the plan but can be dismissed or waited out. The underlying screen is still there.
Navigation drift — the app moved to a different screen than expected, either because a previous step had a side effect or because the app's flow changed.
Hard failure — the target element doesn't exist on this screen and there's no recoverable path forward without a new LLM decision.

Treating all three the same way is the mistake. A spinner should be waited out. A permission dialog should be handled inline. A hard failure needs a new plan.

Detecting the current screen state

data class ScreenSnapshot(
    val packageName: String?,
    val activityName: String?,
    val visibleTexts: List<String>,
    val interactableIds: List<String>
)

fun captureSnapshot(rootNode: AccessibilityNodeInfo?): ScreenSnapshot {
    val texts = mutableListOf<String>()
    val ids = mutableListOf<String>()
    fun traverse(node: AccessibilityNodeInfo?) {
        node ?: return
        node.text?.toString()?.let { texts.add(it) }
        node.viewIdResourceName?.let { ids.add(it) }
        for (i in 0 until node.childCount) traverse(node.getChild(i))
    }
    traverse(rootNode)
    return ScreenSnapshot(
        packageName = rootNode?.packageName?.toString(),
        activityName = null, // filled from AccessibilityEvent if available
        visibleTexts = texts,
        interactableIds = ids
    )
}

Before executing each step, the executor captures a snapshot and compares it against what the step expects.

The obstruction detector

object ObstructionDetector {
    private val DISMISS_TEXTS = setOf(
        "not now", "maybe later", "skip", "no thanks",
        "dismiss", "close", "cancel", "got it", "ok"
    )
    private val PERMISSION_TEXTS = setOf(
        "allow", "deny", "while using the app",
        "only this time", "don't allow"
    )
    private val LOADING_IDS = setOf(
        "android:id/progress", "progressBar", "loadingSpinner"
    )

    sealed class ObstructionType {
        object None : ObstructionType()
        object Loading : ObstructionType()
        data class Dismissible(val dismissText: String) : ObstructionType()
        data class Permission(val allowText: String) : ObstructionType()
        object Unknown : ObstructionType()
    }

    fun detect(snapshot: ScreenSnapshot): ObstructionType {
        if (snapshot.interactableIds.any { id -> LOADING_IDS.any { id.contains(it) } }) {
            return ObstructionType.Loading
        }
        val lower = snapshot.visibleTexts.map { it.lowercase() }
        val permission = PERMISSION_TEXTS.firstOrNull { p -> lower.any { it.contains(p) } }
        if (permission != null) {
            val allowText = snapshot.visibleTexts.firstOrNull { 
                it.lowercase().contains("allow") 
            } ?: "Allow"
            return ObstructionType.Permission(allowText)
        }
        val dismiss = DISMISS_TEXTS.firstOrNull { d -> lower.any { it.contains(d) } }
        if (dismiss != null) {
            val dismissText = snapshot.visibleTexts.firstOrNull { 
                it.lowercase().contains(dismiss) 
            } ?: dismiss
            return ObstructionType.Dismissible(dismissText)
        }
        return ObstructionType.None
    }
}

Handling obstructions inline

suspend fun handleObstruction(
    obstruction: ObstructionDetector.ObstructionType,
    service: StareBrainService
): Boolean {
    return when (obstruction) {
        is ObstructionDetector.ObstructionType.Loading -> {
            delay(1500)
            true // retry the step
        }
        is ObstructionDetector.ObstructionType.Dismissible -> {
            service.clickText(obstruction.dismissText)
            delay(300)
            true
        }
        is ObstructionDetector.ObstructionType.Permission -> {
            // Permissions are always user decisions — don't auto-allow
            false // escalate to confirmation flow
        }
        is ObstructionDetector.ObstructionType.Unknown -> {
            false // escalate to LLM
        }
        is ObstructionDetector.ObstructionType.None -> true
    }
}

Permissions are never auto-handled. The agent dismissing a "allow microphone access" dialog on behalf of the user is a trust violation, even if it's technically possible. It escalates to confirmation instead.

Plugging it into the executor


kotlin
suspend fun executeStep(step: AgentStep): StepResult {
    val snapshot = captureSnapshot(service.rootNode)
    val obstruction = ObstructionDetector.detect(snapshot)

    val canProceed = handleObstruction(obstruction, service)
    if (!canProceed) {
        return StepResult(
            status = StepStatus.FAILED,
            reason = "Unexpected screen: ${obstruction::class.simpleName}"
        )

Building an Android AI Agent, Part 9: Multi-Step Task Execution Without a State Machine

StareBrain — Fri, 30 Oct 2026 03:01:48 +0000

Building an Android AI Agent, Part 9: Multi-Step Task Execution Without a State Machine

In Part 8 we added context-aware confirmation — the agent now knows whether it's operating inside a banking app or a notes app and adjusts its confirmation threshold accordingly.

Now the harder problem: what happens when a task requires more than one action across more than one app?

"Send the invoice to the client I just messaged, then mark the project as done in my task app."

That's three steps, two apps, one instruction. The agent needs to execute them in order, recover if one fails, and not lose track of where it is.

The temptation: a state machine

The obvious solution is a state machine. Define states, define transitions, track current state, handle errors per state.

We didn't do that. Here's why.

A state machine works well when the set of possible states is known at design time. An AI agent operating across arbitrary apps on an arbitrary device doesn't have that property. The apps change. The user's workflow changes. A rigid state machine becomes a maintenance problem the moment a new app enters the picture.

Instead, we use a flat task list with a cursor.

The data model

data class AgentTask(
    val id: String = UUID.randomUUID().toString(),
    val description: String,
    val steps: List<AgentStep>,
    val currentStepIndex: Int = 0,
    val status: TaskStatus = TaskStatus.PENDING
)

data class AgentStep(
    val id: String = UUID.randomUUID().toString(),
    val instruction: String,
    val targetPackage: String?,
    val status: StepStatus = StepStatus.PENDING,
    val result: StepResult? = null
)

enum class TaskStatus { PENDING, IN_PROGRESS, COMPLETED, FAILED, AWAITING_CONFIRMATION }
enum class StepStatus { PENDING, IN_PROGRESS, COMPLETED, FAILED, SKIPPED }

The AgentTask holds the full list of steps. currentStepIndex is the cursor. The LLM generates the step list upfront; the executor walks it.

The executor

class TaskExecutor(
    private val accessibilityService: StareBrainService,
    private val confirmationGuard: ConfirmationGuard,
    private val scope: CoroutineScope
) {
    suspend fun execute(task: AgentTask): AgentTask {
        var current = task.copy(status = TaskStatus.IN_PROGRESS)

        for (i in current.currentStepIndex until current.steps.size) {
            val step = current.steps[i]

            if (confirmationGuard.shouldConfirm(
                    step.instruction,
                    null,
                    null,
                    step.targetPackage
                )
            ) {
                return current.copy(
                    currentStepIndex = i,
                    status = TaskStatus.AWAITING_CONFIRMATION
                )
            }

            val result = executeStep(step)
            val updatedStep = step.copy(status = result.status, result = result)
            val updatedSteps = current.steps.toMutableList()
            updatedSteps[i] = updatedStep
            current = current.copy(
                steps = updatedSteps,
                currentStepIndex = i + 1
            )

            if (result.status == StepStatus.FAILED) {
                return current.copy(status = TaskStatus.FAILED)
            }
        }

        return current.copy(status = TaskStatus.COMPLETED)
    }

    private suspend fun executeStep(step: AgentStep): StepResult {
        // Dispatches to accessibility service actions
        return accessibilityService.performAction(step)
    }
}

Two things worth noting:

Confirmation pauses mid-task. If step 2 of 4 requires confirmation, the executor returns with AWAITING_CONFIRMATION and the current index. The UI shows the confirmation prompt. When the user approves, execution resumes from where it stopped — no replay from the beginning.
Failure stops the chain. If a step fails, the task halts and the UI surfaces which step failed and why. The user can retry that step or abandon the task. We don't silently skip failures.

Resuming after confirmation

suspend fun resume(task: AgentTask, approved: Boolean): AgentTask {
    if (!approved) {
        return task.copy(status = TaskStatus.FAILED)
    }
    // Skip the confirmation check for the step we just approved
    val step = task.steps[task.currentStepIndex]
    val result = executeStep(step)
    val updatedStep = step.copy(status = result.status, result = result)
    val updatedSteps = task.steps.toMutableList()
    updatedSteps[task.currentStepIndex] = updatedStep

    val resumed = task.copy(
        steps = updatedSteps,
        currentStepIndex = task.currentStepIndex + 1,
        status = TaskStatus.IN_PROGRESS
    )
    return execute(resumed)
}

The resume function re-enters the executor from the current index with the confirmation check bypassed for that one step. Everything after it still runs normally — including any further confirmation checks further down the list.

Why the client holds state

The backend generates the step list. The client owns execution state. This is intentional.

If the backend owned execution state, every step would require a round trip to persist progress. On a mobile device with unreliable connectivity, that's a reliability problem. The client already has the accessibility context, the current app state, and the user's confirmation response. Keeping the task state on the client means the agent can continue executing even if the backend is momentarily unreachable.

The backend only gets called again if a step requires a new LLM decision — for example, if an unexpected screen appears mid-task and the agent needs to figure out what to do next.

What's next

Part 10: handling unexpected screens mid-task — when the agent reaches a step and the expected UI isn't there.

When “Send” Isn’t Risky: Context-Aware Confirmation in an Android AI Agent

StareBrain — Thu, 29 Oct 2026 03:06:16 +0000

When "Send" Isn't Risky: Context-Aware Confirmation in an Android AI Agent

In the last post I covered the two-layer confirmation guard: the LLM's requires_confirmation flag plus a keyword guard as a dumb-but-reliable backstop. A reader raised the right follow-up: "send" in a banking app and "send" in a notes app are not the same risk. The guard shouldn't treat them the same way.

This post covers the next layer — making the confirmation guard context-aware by weighing app identity and action type together.

The Problem With a Flat Keyword List

A flat list of risky keywords has two failure modes.

False negatives (the dangerous kind): the guard misses a risky action because the label is in another language, uses a synonym, or comes from a resource ID the guard never sees.

False positives (the usability kind): the guard fires on "payload", "sender", "shareholder" — or on "send" in an app where sending is so routine it carries no real risk. If confirmations fire too often on benign actions, users train themselves to tap through without reading. When the real confirmation arrives, it gets tapped through too.

The second failure mode undermines the first. A guard that cries wolf enough times stops being a guard.

App Context Is Already in the Request

The AgentStepRequest already carries current_package — the package name of the app currently in focus:

class AgentStepRequest(BaseModel):
    task: str
    screen_description: str
    current_package: Optional[str] = None  // e.g. "com.google.android.gm"
    ...

This is the signal that makes context-aware risk assessment possible. The same action means different things in different apps.

Risk Tiers by Package

enum class AppRiskTier {
    HIGH,    // Banking, payments, email clients, messaging
    MEDIUM,  // Social, productivity, calendar
    LOW      // Notes, utilities, entertainment
}

object AppRiskClassifier {

    private val HIGH_RISK_PACKAGES = setOf(
        "com.google.android.gm",           // Gmail
        "com.android.mms",                  // SMS
        "org.telegram.messenger",           // Telegram
        "com.whatsapp",                     // WhatsApp
        "com.paypal.android.p2pmobile",    // PayPal
        "com.google.android.apps.nbu.paisa.user" // GPay
    )

    private val LOW_RISK_PACKAGES = setOf(
        "com.google.android.keep",         // Keep Notes
        "com.spotify.music",               // Spotify
        "com.google.android.youtube"       // YouTube
    )

    fun tierFor(packageName: String?): AppRiskTier {
        if (packageName == null) return AppRiskTier.MEDIUM
        return when {
            HIGH_RISK_PACKAGES.any { packageName.startsWith(it) } -> AppRiskTier.HIGH
            LOW_RISK_PACKAGES.any { packageName.startsWith(it) } -> AppRiskTier.LOW
            else -> AppRiskTier.MEDIUM
        }
    }
}

Context-Aware Keyword Guard

object ConfirmationGuard {

    private val HIGH_RISK_HINTS = setOf(
        "send", "pay", "transfer", "purchase", "buy",
        "confirm payment", "place order", "checkout"
    )

    private val MEDIUM_RISK_HINTS = setOf(
        "delete", "remove", "uninstall", "submit",
        "sign out", "log out", "deactivate",
        "cancel subscription", "share"
    )

    private val LOW_RISK_HINTS = setOf(
        "post", "publish", "save"
    )

    fun shouldConfirm(
        text: String?,
        packageName: String?
    ): Boolean {
        if (text == null) return false
        val lower = text.lowercase().trim()
        val tier = AppRiskClassifier.tierFor(packageName)

        return when (tier) {
            AppRiskTier.HIGH ->
                HIGH_RISK_HINTS.any { matchesWord(lower, it) } ||
                MEDIUM_RISK_HINTS.any { matchesWord(lower, it) }

            AppRiskTier.MEDIUM ->
                HIGH_RISK_HINTS.any { matchesWord(lower, it) } ||
                MEDIUM_RISK_HINTS.any { matchesWord(lower, it) }

            AppRiskTier.LOW ->
                HIGH_RISK_HINTS.any { matchesWord(lower, it) }
        }
    }

    // Whole-word match to avoid "pay" in "payload", "send" in "sender"
    private fun matchesWord(text: String, keyword: String): Boolean {
        val pattern = Regex("\\b${Regex.escape(keyword)}\\b")
        return pattern.containsMatchIn(text)
    }
}

Two changes from the flat version: whole-word matching instead of substring matching, and threshold variation by app tier. "Send" in a notes app (LOW tier) doesn't trigger confirmation. "Send" in Gmail (HIGH tier) does.

Localization: Use Resource IDs

A reader flagged the language gap in the last post — a phone set to French or Arabic renders "Envoyer" or "إرسال", neither of which matches "send". The visible label isn't reliable across locales.

The fix: use the view's contentDescription or resource ID alongside the label. Resource IDs are set by the developer and almost always stay in English regardless of locale:

fun shouldConfirm(
    visibleText: String?,
    contentDescription: String?,
    resourceId: String?,
    packageName: String?
): Boolean {
    // Check visible label, contentDescription, and resource ID
    // Resource IDs travel across locales; visible text does not
    return listOf(visibleText, contentDescription, resourceId)
        .any { text ->
            shouldConfirmText(text, packageName)
        }
}

The Accessibility API surfaces all three. Using all three means the guard fires correctly regardless of the device language.

The Updated ConfirmationSource

enum class ConfirmationSource {
    LLM_FLAGGED,
    KEYWORD_GUARD,
    CONTEXT_AWARE_GUARD,
    BOTH
}

CONTEXT_AWARE_GUARD in the log means the flat guard would have missed it (LOW tier, medium-risk keyword) or the flat guard would have over-fired (LOW tier, low-risk keyword suppressed). Both are worth tracking separately — one shows where context is adding value, the other shows where it's reducing noise.

The Rule

A confirmation guard that fires on everything trains users to ignore it. A guard that knows which app it's running in fires less often and means more when it does. Context doesn't weaken the protection — it concentrates it where the risk is real.

Series so far:

Part 1: Two-axis state model + DENIED_UNRESOLVED
Part 2: Context propagation + confidence thresholds
Part 3: Late confirmation race + terminal state check
Part 4: Provider capability tiers
Part 5: Stale plan protection with FreshCtx
Part 6: Stateless backend architecture
Part 7: Defense-in-depth confirmation guard
Part 8: Context-aware confirmation ← today

Why We Check Twice Before a Risky Action: Defense-in-Depth in an Android AI Agent

StareBrain — Wed, 28 Oct 2026 02:52:31 +0000

Introduction

When an AI agent is about to send a message, make a payment, or delete something, one layer of protection isn’t enough.

StareBrain uses two independent checks before any risky action reaches the user for confirmation: the LLM’s own requires_confirmation flag in the response, and a client-side keyword guard that runs regardless of what the LLM said.

Neither layer trusts the other. That’s the point.

Why Two Layers

The LLM decides requires_confirmation based on its understanding of the action. Most of the time it gets this right. But LLMs can be wrong — a prompt that’s slightly unusual, a new action type the model hasn’t seen in context, a subtle misclassification. If the only gate is the model’s own judgment, a missed classification means an unconfirmed destructive action goes through.

The keyword guard is dumb by design. It doesn’t understand the action — it just scans the button label or action text for known risky patterns. It can’t be confused by context. It doesn’t hallucinate. It catches what the LLM missed.

Together they form a defense-in-depth: the LLM catches the cases it understands, the keyword guard catches the cases it doesn’t.

The Confirmation Guard

`object ConfirmationGuard {

    private val RISKY_TEXT_HINTS = listOf(
        "send", "pay", "transfer", "purchase", "buy", "delete",
        "remove", "uninstall", "submit", "confirm payment",
        "place order", "checkout", "sign out", "log out",
        "deactivate", "cancel subscription", "share"
    )

    fun isRiskyActionText(text: String?): Boolean {
        if (text == null) return false
        val lower = text.lowercase()
        return RISKY_TEXT_HINTS.any { hint -> lower.contains(hint) }
    }
}`

Simple substring match on the lowercased text. No ML, no context, no state. If the button label contains “send”, “pay”, “delete”, or any other hint in the list, it’s flagged as risky.

Wiring It Into the Agent Loop

The agent loop checks both signals before dispatching:

`fun shouldRequireConfirmation(
    response: AgentStepResponse
): Boolean {
    // LLM explicitly flagged it
    if (response.requires_confirmation) return true

    // Keyword guard catches what the LLM missed
    if (ConfirmationGuard.isRiskyActionText(response.text_to_type)) return true
    if (ConfirmationGuard.isRiskyActionText(response.status_text)) return true

    return false
}`

Either signal is sufficient. The user sees the confirmation dialog if the LLM flagged it OR if the keyword guard flagged it. Both conditions don’t need to be true.

What the User Sees

When either layer triggers, the agent pauses and surfaces the action to the user before dispatching:

`data class ConfirmationRequest(
    val actionId: String,
    val actionSummary: String,
    val reason: String,
    val confirmedBy: ConfirmationSource
)

enum class ConfirmationSource {
    LLM_FLAGGED,
    KEYWORD_GUARD,
    BOTH
}`

The confirmedBy field is logged for analysis — if KEYWORD_GUARD appears frequently without LLM_FLAGGED, that’s a signal the LLM is systematically missing a class of risky actions and the prompt needs updating.

Keeping the List Current

The keyword list is a living document. New action types get added when:

A risky action slips through in testing
A new integration is added (new app, new capability)
Analysis shows KEYWORD_GUARD catching things LLM_FLAGGED missed

It’s not a replacement for improving the LLM’s classification — it’s the safety net that buys you time while you improve it.

The Rule

For any action that can’t be undone or involves real-world consequences — money, messages, deletions, account changes — confirmation should never depend on a single system getting it right. The keyword guard exists precisely for the cases where the smarter system fails.

Series so far:

Part 1: Two-axis state model + DENIED_UNRESOLVED
Part 2: Context propagation + confidence thresholds
Part 3: Late confirmation race + terminal state check
Part 4: Provider capability tiers
Part 5: Stale plan protection with FreshCtx
Part 6: Stateless backend architecture
Part 7: Defense-in-depth confirmation guard ← today

Why StareBrain’s AI Backend Writes Nothing to a Database: A Stateless Agent Architecture

StareBrain — Tue, 27 Oct 2026 05:12:18 +0000

Introduction

Most backend services own state. They write to a database, track sessions, remember where you are in a flow.

StareBrain’s backend does none of that. Every request to /next-step is self-contained. The backend reads what the client sends, decides the next action, and returns it. Nothing is written. Nothing is stored. The session ends when the response leaves.

This is a deliberate architectural choice — and it has real consequences for how the agent handles plans, step history, and recovery.

The Request and Response

The full schema for a single agent step:

`class AgentStepRequest(BaseModel):
    task: str
    screen_description: str
    current_package: Optional[str] = None
    steps_so_far: list[str] = []
    step_count: int
    known_facts: str = ""
    plan: list[str] = []
    current_subgoal: int = 0
    screenshot_b64: Optional[str] = None
    capabilities: list[str] = []

class AgentStepResponse(BaseModel):
    reasoning: Optional[str] = None
    action: str
    target_index: Optional[int] = None
    text_to_type: Optional[str] = None
    app_name: Optional[str] = None
    setting_name: Optional[str] = None
    status_text: str
    final_answer: Optional[str] = None
    remember: Optional[RememberFact] = None
    plan: Optional[list[str]] = None
    advance_subgoal: bool = False
    replan: Optional[list[str]] = None
    requires_confirmation: bool = False
    confirmation_reason: Optional[str] = None
    question: Optional[str] = None
    options: Optional[list[str]] = None`

Notice what’s in the request: steps_so_far, plan, current_subgoal, known_facts. The client sends the entire history of the session on every call. The backend never looks anything up — it receives the full context and decides from that alone.

Why Stateless

Three reasons drove this decision:

1. The client already has the state.
The Android app is executing actions, reading screens, tracking what happened. It has the most accurate picture of session state by definition. Duplicating that in a backend database adds a consistency problem: which one is right when they diverge?

2. Resumability is free.
If the backend crashes mid-session, the client resumes by sending the same request again. There’s no session to recover, no checkpoint to restore. The next call picks up exactly where the last confirmed step left off.

3. Scaling is trivial.
A stateless backend has no affinity — any instance handles any request. No sticky sessions, no shared cache, no distributed state to synchronize.

What the Client Owns

Because the backend is stateless, the client is the source of truth for:

`data class AgentSession(
    val task: String,
    val stepsSoFar: MutableList<String>,
    val plan: MutableList<String>,
    val currentSubgoal: Int,
    val knownFacts: String,
    val stepCount: Int
)`

After each response, the client updates its local session and sends the updated state on the next call. The backend never needs to remember the previous step — it’s in the request.

Plan and Replan

The response can carry a plan (initial plan, set on the first step) or a replan (replacement plan mid-session). The client applies whichever arrives:

`fun applyResponse(session: AgentSession, response: AgentStepResponse) {
    response.plan?.let { session.plan.addAll(it) }
    response.replan?.let {
        session.plan.clear()
        session.plan.addAll(it)
    }
    if (response.advance_subgoal) {
        session.currentSubgoal += 1
    }
    response.remember?.let {
        session.knownFacts += "\n${it.key}: ${it.value}"
    }
    session.stepsSoFar.add(response.action)
    session.stepCount += 1
}`

The backend proposes. The client applies and owns. On the next request, the updated plan goes back up in AgentStepRequest.plan.

The Tradeoff

Stateless is not free. The request payload grows with session length — steps_so_far and known_facts accumulate. Long sessions send more data per call.

The mitigation: steps_so_far is summarized rather than appended verbatim after a threshold, and known_facts is structured as key-value pairs so the backend can read the relevant facts without parsing a growing narrative.

The other tradeoff: if the client loses state (app killed, device rebooted mid-task), the session is gone. There’s no server-side recovery. For the current use case — short agentic tasks on a mobile device — this is acceptable. For long-running tasks that span hours, it would need rethinking.

Series so far:

Stale Plan Protection in an AI Agent: How FreshCtx Catches Context Drift Before Dispatch

StareBrain — Mon, 26 Oct 2026 03:39:19 +0000

The Problem

An AI agent builds a plan from the context it sees at planning time. By the time the user confirms and the action dispatches, that context may have changed.

A slot that was available is now taken. A contact that existed has been removed. A setting that was on is now off.

If the agent doesn’t revalidate before dispatch, it executes a plan built on stale information — and the user confirmed something that no longer reflects reality.

FreshCtx

FreshCtx is a Python library built to catch this. The core idea: fingerprint the relevant fields of a context record before planning, then revalidate that fingerprint immediately before invocation. If the fingerprint has changed, block the dispatch.

`import hashlib
import json
from pathlib import Path

def fingerprint(slot_path: str) -> str:
    record = json.loads(Path(slot_path).read_text())
    # Only fingerprint fields that matter for this action
    relevant = {"slot_id": record["slot_id"], "status": record["status"]}
    return hashlib.sha256(
        json.dumps(relevant, sort_keys=True).encode()
    ).hexdigest()[:16]`

The key design decision: field-level fingerprinting, not record-level. You don’t want a dispatch blocked because an unrelated field changed — a scheduler tag updated, a last-modified timestamp rotated. You want it blocked when the fields your action depends on have changed.

Three-Path Comparison

We ran a fixture with three evidence files against the same base:

`# v1 — base state
v1 = {
    "slot_id": "slot-42",
    "time": "2026-10-23T10:00:00Z",
    "status": "available",
    "last_modified_by": "scheduler-v1"
}

# v2 — status changed (available → unavailable)
v2 = {
    "slot_id": "slot-42",
    "time": "2026-10-23T10:00:00Z",
    "status": "unavailable",
    "last_modified_by": "scheduler-v1"
}

# v3 — unrelated field changed (scheduler-v1 → scheduler-v2)
v3 = {
    "slot_id": "slot-42",
    "time": "2026-10-23T10:00:00Z",
    "status": "available",
    "last_modified_by": "scheduler-v2"
}`

Results:

v1 and v3 produce identical fingerprints because last_modified_by is not in the relevant field set. The dispatch only blocks when status changes — exactly the field the booking action depends on.

Wiring It Into the Agent

In StareBrain, the calendar action uses Intent.ACTION_INSERT — the system calendar opens pre-filled and the user saves manually. FreshCtx’s boundary sits before the intent is dispatched, not before a write, because StareBrain never makes the calendar write itself.

`// Before dispatching the intent
val currentFingerprint = freshCtx.fingerprint(slotPath)

if (currentFingerprint != plan.capturedFingerprint) {
    return DispatchResult.StalePlan(
        reason = "slot status changed since plan was built",
        capturedAt = plan.capturedAt,
        detectedAt = Instant.now()
    )
}

// Fingerprint matches — safe to dispatch
dispatchCalendarIntent(plan)`

The capturedFingerprint is stored on the plan at planning time. At dispatch time, FreshCtx recomputes it and compares. If they match, proceed. If they don’t, return StalePlan — the agent surfaces this to the user before anything is dispatched.

Why This Matters

Without stale plan protection, the agent is confident about information it has no business being confident about. The user confirmed a plan. The plan was built on a snapshot. The snapshot is now wrong.

FreshCtx makes the boundary explicit: the plan is valid as of the moment it was built, and that validity is checked again at the last possible moment before dispatch. Everything between planning and dispatch is treated as a window where the world could have moved.

Series so far:

Part 1: Two-axis state model + DENIED_UNRESOLVED
Part 2: Context propagation + confidence thresholds
Part 3: Late confirmation race + terminal state check
Part 4: Provider capability tiers
Part 5: Stale plan protection with FreshCtx ← today

Not Every Backend Tells You What Happened: Provider Capability Tiers in an AI Agent

StareBrain — Sun, 25 Oct 2026 04:54:59 +0000

Introduction

When an AI agent dispatches an action — send a message, book a slot, toggle a setting — it needs to know whether the action succeeded. But not every backend gives you the same quality of answer.

Some tell you exactly what happened. Some give you enough to figure it out. Some give you nothing at all.

If your agent treats all backends the same way, it will make wrong decisions when a provider goes quiet. The fix is to model capability explicitly before you dispatch — not after.

The Four Tiers

`enum class ProviderCapabilityTier {
    FULL,           // Confirms delivery + acceptance
    RECONCILABLE,   // No direct confirmation; can query state after dispatch
    PARTIAL,        // Confirms delivery only; acceptance unknown
    BLIND           // No confirmation, no queryable state
}`

FULL — The provider sends back a confirmed result. You know delivery happened and the action was accepted. Calendar providers with a write API, messaging platforms with delivery receipts.

RECONCILABLE — No direct confirmation on dispatch, but you can query the provider’s state afterward and compare against a pre-dispatch snapshot. The diff tells you what changed.

PARTIAL — The provider acknowledges receipt but can’t tell you whether the action was actually carried out. You know it arrived; you don’t know if it landed.

BLIND — Fire and forget. No receipt, no queryable state. You dispatched. That’s all you know.

Why It Matters Before Dispatch

The tier determines what your reconciliation strategy needs to be — and that decision has to be made before you dispatch, not after you’re already waiting for a response that isn’t coming.

`data class DispatchStrategy(
    val tier: ProviderCapabilityTier,
    val reconciliationKey: String?,
    val eligibleForReconciliationAt: Instant,
    val requiresManualVerification: Boolean
)

fun buildDispatchStrategy(
    tier: ProviderCapabilityTier,
    now: Instant
): DispatchStrategy {
    return when (tier) {
        ProviderCapabilityTier.FULL -> DispatchStrategy(
            tier = tier,
            reconciliationKey = null,
            eligibleForReconciliationAt = now,
            requiresManualVerification = false
        )
        ProviderCapabilityTier.RECONCILABLE -> DispatchStrategy(
            tier = tier,
            reconciliationKey = generateReconciliationKey(),
            eligibleForReconciliationAt = now.plusSeconds(CONSISTENCY_WINDOW_SECONDS),
            requiresManualVerification = false
        )
        ProviderCapabilityTier.PARTIAL -> DispatchStrategy(
            tier = tier,
            reconciliationKey = generateReconciliationKey(),
            eligibleForReconciliationAt = now.plusSeconds(CONSISTENCY_WINDOW_SECONDS),
            requiresManualVerification = true
        )
        ProviderCapabilityTier.BLIND -> DispatchStrategy(
            tier = tier,
            reconciliationKey = null,
            eligibleForReconciliationAt = Instant.MAX,
            requiresManualVerification = true
        )
    }
}

private const val CONSISTENCY_WINDOW_SECONDS = 30L`

Wiring It Into the Action Record

From the Oct 22 post, the ActionRecord carries delivery and acceptance state. The tier slots in alongside it:

`data class ActionRecord(
    val actionId: String,
    val dispatchId: String,
    val providerObjectId: String?,
    val tier: ProviderCapabilityTier,
    val dispatchStrategy: DispatchStrategy,
    val delivery: DeliveryState,
    val acceptance: AcceptanceState,
    val dispatchedAt: Instant,
    val confirmedAt: Instant?,
    val eligibleForReconciliationAt: Instant
)`

For a BLIND provider, eligibleForReconciliationAt is Instant.MAX — the record never becomes eligible for automated reconciliation. It stays DENIED_UNRESOLVED until a human or a higher-level signal resolves it.

For a RECONCILABLE provider, the consistency window enforces a minimum age before the reconciliation job touches the record — the same replication lag protection from the Oct 22 post.

The Rule

Model the tier at registration time, not at failure time. By the time dispatch has failed silently, it’s too late to decide how you were going to handle it.

Every provider your agent talks to gets a declared tier. Every action record carries the strategy that tier implies. Reconciliation and confirmation handling downstream just read the record — they don’t need to know which provider they’re dealing with.

Series so far:

Part 1: Two-axis state model + DENIED_UNRESOLVED
Part 2: Context propagation + confidence thresholds
Part 3: Late confirmation race + terminal state check
Part 4: Provider capability tiers ← today

Late Confirmation Race: Handling the Gap Between User Approval and Action Dispatch in Android AI Agents

StareBrain — Sat, 24 Oct 2026 03:56:17 +0000

There’s a timing problem in confirm-before-execute flows that’s easy to overlook until it causes a duplicate action: the window between when the user confirms and when the action actually dispatches.

In most implementations, the flow looks like this:

Agent plans an action
User is shown a confirmation prompt
User taps confirm
Action dispatches

The assumption is that step 3 leads directly to step 4. But in practice, there’s a gap — and things can happen in that gap.

This post covers what we call the late confirmation race in StareBrain, and how we handle it without introducing false blocks or silent duplicates.

What Can Go Wrong in the Gap

The most common case: the action was already dispatched by an earlier path before the confirmation came back.

How does that happen? A few ways:

The user confirmed on one device/session while a background task had already triggered the same action
A retry fired because the first dispatch timed out, but the timeout was too aggressive — the original dispatch was still in-flight
The auto-confirm threshold was met (for low-risk actions) at the same time the manual confirm came back

In all three cases, a naive implementation dispatches twice. One dispatch was right. The other is a duplicate.

Checking Terminal State Before Replacement Dispatch

The fix is straightforward: before honoring a confirmation by dispatching, check whether the action is already in a terminal delivery state.

`sealed class ConfirmationOutcome {
    data class Dispatched(val actionId: String) : ConfirmationOutcome()
    data class AlreadyCompleted(val actionId: String, val state: DeliveryState) : ConfirmationOutcome()
    data class Superseded(val actionId: String) : ConfirmationOutcome()
    data class Blocked(val actionId: String, val reason: String) : ConfirmationOutcome()
}

class ConfirmationHandler(
    private val actionRepository: ActionRepository,
    private val dispatcher: ActionDispatcher
) {
    fun handleConfirmation(actionId: String): ConfirmationOutcome {
        val record = actionRepository.find(actionId)
            ?: return ConfirmationOutcome.Blocked(actionId, "action record not found")

        // Terminal check before dispatch
        return when (record.delivery) {
            DeliveryState.CONFIRMED -> {
                ConfirmationOutcome.AlreadyCompleted(actionId, DeliveryState.CONFIRMED)
            }
            DeliveryState.FAILED -> {
                ConfirmationOutcome.AlreadyCompleted(actionId, DeliveryState.FAILED)
            }
            DeliveryState.PENDING -> {
                // Safe to dispatch — not yet in a terminal state
                dispatcher.dispatch(record)
                ConfirmationOutcome.Dispatched(actionId)
            }
        }
    }
}`

If the delivery state is already CONFIRMED or FAILED, the confirmation is honored as a no-op. The user’s tap is acknowledged, but no second dispatch fires.

The DENIED_UNRESOLVED Case

The terminal check above handles CONFIRMED and FAILED cleanly. The harder case is DENIED_UNRESOLVED — the action dispatched, but the outcome is unknown.

This is not a terminal state in the sense that it’s resolved, but it is a state where a replacement dispatch is dangerous: if the first dispatch actually landed on the provider side and just failed to confirm back, a second dispatch is a duplicate.

`fun handleConfirmation(actionId: String): ConfirmationOutcome {
    val record = actionRepository.find(actionId)
        ?: return ConfirmationOutcome.Blocked(actionId, "action record not found")

    return when {
        record.delivery == DeliveryState.CONFIRMED ->
            ConfirmationOutcome.AlreadyCompleted(actionId, DeliveryState.CONFIRMED)

        record.delivery == DeliveryState.FAILED ->
            ConfirmationOutcome.AlreadyCompleted(actionId, DeliveryState.FAILED)

        record.actionStatus == ActionStatus.DENIED_UNRESOLVED -> {
            // Do NOT dispatch. Surface for reconciliation first.
            ConfirmationOutcome.Blocked(
                actionId,
                "prior dispatch unresolved — reconcile before retry"
            )
        }

        record.delivery == DeliveryState.PENDING ->
            dispatcher.dispatch(record).let {
                ConfirmationOutcome.Dispatched(actionId)
            }

        else -> ConfirmationOutcome.Blocked(actionId, "unexpected state")
    }
}`

The DENIED_UNRESOLVED case blocks replacement dispatch and routes to reconciliation. The user sees that the previous attempt’s outcome is unknown and needs to be checked before proceeding — not a silent failure, not a silent retry.

Idempotency on the Dispatch Side

The terminal check on confirmation prevents duplicate dispatches from the confirmation handler. But the dispatch path itself should also be idempotent, because the confirmation handler isn’t the only entry point.

`class ActionDispatcher(
    private val repository: ActionRepository,
    private val provider: ActionProvider
) {
    fun dispatch(record: ActionRecord): DispatchResult {
        // Idempotency key scoped to action + attempt number
        val idempotencyKey = "${record.actionId}:${record.attemptNumber}"

        // Re-check delivery state inside dispatch — the terminal check
        // in ConfirmationHandler is a fast path, not the only guard
        if (record.delivery != DeliveryState.PENDING) {
            return DispatchResult.Skipped(record.actionId, "non-pending state at dispatch")
        }

        return try {
            val result = provider.execute(record, idempotencyKey)
            repository.markDelivered(record.actionId, result.providerObjectId)
            DispatchResult.Succeeded(record.actionId)
        } catch (e: TimeoutException) {
            repository.markUnresolved(record.actionId)
            DispatchResult.Unresolved(record.actionId)
        } catch (e: Exception) {
            repository.markFailed(record.actionId)
            DispatchResult.Failed(record.actionId, e.message)
        }
    }
}

sealed class DispatchResult {
    data class Succeeded(val actionId: String) : DispatchResult()
    data class Failed(val actionId: String, val reason: String?) : DispatchResult()
    data class Unresolved(val actionId: String) : DispatchResult()
    data class Skipped(val actionId: String, val reason: String) : DispatchResult()
}`

The idempotency key scoped to actionId + attemptNumber means a retry on the same attempt doesn’t produce a second provider-side effect. A new attempt gets a new key.

What the User Sees

The goal is that the user never sees a duplicate action and never gets a silent failure. What they see instead:

Normal path: confirm → action runs → done
Already completed: confirm acknowledged, brief notice that it already ran
DENIED_UNRESOLVED: confirm blocked, clear message that the previous attempt’s outcome is unknown and needs to be checked

The third case is the uncomfortable one to surface. But a user who sees “we’re not sure if the last attempt ran” is in a better position than a user who just had a duplicate booking made silently.

What This Closes

The late confirmation race is a narrow window but it’s a real one — especially in mobile contexts where network conditions vary and the same action can be triggered from multiple paths. The terminal check before dispatch, combined with idempotent dispatch, closes both the happy-path duplicate and the unresolved-state duplicate without adding meaningful latency to the normal flow.

Context Propagation in Android AI Agents: Resolving Device State Before Action Dispatch

StareBrain — Fri, 23 Oct 2026 03:30:01 +0000

An AI agent on a phone has access to something a server-side agent doesn’t: the device itself. Calendar events, saved locations, recent contacts, active notifications — all of it is locally available before any network call is made.

The question isn’t whether to use that context. It’s how to resolve it into action parameters without asking the user for information the phone already has, and without silently filling in details the phone only partially knows.

This post covers the context propagation layer in StareBrain — how device state gets resolved into action parameters, how confidence thresholds decide when to auto-fill vs. suggest vs. ask, and what happens when resolution fails.

The Resolution Problem

When a user says “book a slot for my dentist appointment,” the agent needs to resolve:

Which calendar? (probably the default one)
Which slot? (depends on availability)
Which location? (saved as “dentist” in the location store, or needs to be asked)
Which contact to notify, if any? (ambiguous without more context)

Some of these can be resolved from device state with high confidence. Others can be suggested. Others need to be asked explicitly. Collapsing all of them into a single “ask the user for everything” flow is safe but slow. Auto-filling everything without surfacing uncertainty is fast but fragile.

The resolution layer sits between intent parsing and action dispatch, and its job is to decide which parameters fall into which bucket.

Modeling Device Context

`data class DeviceContext(
    val upcomingEvents: List<CalendarEvent>,
    val savedLocations: List<SavedLocation>,
    val recentContacts: List<Contact>,
    val recentApps: List<String>,
    val activeNotifications: List<NotificationSummary>
)

data class CalendarEvent(
    val eventId: String,
    val title: String,
    val startTime: Instant,
    val location: String?,
    val attendees: List<String>
)

data class SavedLocation(
    val label: String,
    val address: String,
    val coordinates: Pair<Double, Double>
)`

The context is captured at intent-parse time — a snapshot of device state at the moment the user expressed the intent. This matters: if resolution runs against live state at dispatch time, a contact added or an event modified after the user spoke could produce parameters the user never approved.

Confidence-Gated Resolution

Each parameter resolution produces a confidence level that determines what happens next:

`enum class ResolutionConfidence { HIGH, MEDIUM, LOW, UNRESOLVABLE }

data class ParameterValue(
    val value: Any?,
    val confidence: ResolutionConfidence,
    val source: ContextSource
)

sealed class ContextSource {
    object DeviceCalendar : ContextSource()
    object SavedLocations : ContextSource()
    object RecentContacts : ContextSource()
    object UserProvided : ContextSource()
    object Inferred : ContextSource()
}

fun ParameterValue.shouldAutoFill(): Boolean =
    confidence == ResolutionConfidence.HIGH

fun ParameterValue.shouldSuggest(): Boolean =
    confidence == ResolutionConfidence.MEDIUM

fun ParameterValue.shouldAsk(): Boolean =
    confidence == ResolutionConfidence.LOW ||
    confidence == ResolutionConfidence.UNRESOLVABLE`

HIGH → auto-fill silently. The device has this unambiguously (default calendar, a location saved under an exact label match).

MEDIUM → surface as a suggestion. The device has a candidate but it’s not certain (a location match on partial label, a contact with a common name).

LOW / UNRESOLVABLE → ask the user explicitly before proceeding.

The Resolver

`class ContextResolver(
    private val calendarProvider: CalendarProvider,
    private val locationStore: LocationStore,
    private val contactStore: ContactStore
) {
    fun resolve(
        intent: ActionIntent,
        context: DeviceContext
    ): Map<String, ParameterValue> {
        return intent.requiredParameters.associate { param ->
            param.name to resolveParameter(param, context)
        }
    }

    private fun resolveParameter(
        param: Parameter,
        context: DeviceContext
    ): ParameterValue {
        return when (param) {
            is Parameter.Location -> resolveLocation(param, context)
            is Parameter.Contact -> resolveContact(param, context)
            is Parameter.TimeSlot -> resolveTimeSlot(param, context)
            is Parameter.Calendar -> resolveCalendar(param, context)
        }
    }

    private fun resolveLocation(
        param: Parameter.Location,
        context: DeviceContext
    ): ParameterValue {
        val exactMatch = context.savedLocations
            .firstOrNull { it.label.equals(param.hint, ignoreCase = true) }

        if (exactMatch != null) {
            return ParameterValue(
                value = exactMatch,
                confidence = ResolutionConfidence.HIGH,
                source = ContextSource.SavedLocations
            )
        }

        val partialMatch = context.savedLocations
            .firstOrNull { it.label.contains(param.hint, ignoreCase = true) }

        if (partialMatch != null) {
            return ParameterValue(
                value = partialMatch,
                confidence = ResolutionConfidence.MEDIUM,
                source = ContextSource.SavedLocations
            )
        }

        return ParameterValue(
            value = null,
            confidence = ResolutionConfidence.UNRESOLVABLE,
            source = ContextSource.SavedLocations
        )
    }
}`

Building the Agent Context

The resolved parameters get packaged into an AgentContext that travels with the action through to dispatch:

`data class AgentContext(
    val actionIntent: ActionIntent,
    val deviceContext: DeviceContext,
    val resolvedParameters: Map<String, ParameterValue>,
    val capturedAt: Instant
)

fun AgentContext.pendingParameters(): List<String> {
    return resolvedParameters
        .filter { (_, value) -> value.shouldAsk() }
        .map { (name, _) -> name }
}

fun AgentContext.suggestedParameters(): Map<String, ParameterValue> {
    return resolvedParameters
        .filter { (_, value) -> value.shouldSuggest() }
}

fun AgentContext.autoFilledParameters(): Map<String, Any?> {
    return resolvedParameters
        .filter { (_, value) -> value.shouldAutoFill() }
        .mapValues { (_, value) -> value.value }
}`

Before the action proceeds to the confirmation step, the agent checks pendingParameters(). If anything is unresolvable, the user is asked. If there are suggestions, they’re surfaced with the option to confirm or override. Only HIGH confidence parameters skip the user entirely.

Why capturedAt Matters

The capturedAt timestamp on AgentContext is the snapshot boundary. Resolution runs once, against the device state at intent time. If the user takes three minutes to confirm the action, the context doesn’t re-resolve — the parameters they’re approving are the ones they were shown.

This is intentional. Re-resolving at confirmation time could change the suggested contact or location between when the user read the suggestion and when they tapped confirm, which is a different version of the stale-plan problem at the resolution layer rather than the dispatch layer.

The resolution snapshot is the contract. The dispatch layer’s job (covered in earlier posts) is to verify that the external world still matches that contract at execution time.

What This Gives You

The resolver separates three categories of knowledge: what the device knows for certain, what it can reasonably guess, and what it genuinely doesn’t have. Surfacing that distinction explicitly — rather than flattening it into “fill what you can and ask for the rest” — keeps the user in the loop on the decisions that affect them and out of the loop on the ones that don’t.

Why AI Agents Need a Reconciliation Layer: Modeling the Gap Between Dispatch and Confirmation

StareBrain — Thu, 22 Oct 2026 03:35:41 +0000

When an AI agent dispatches an action — booking a slot, sending a message, writing a file — there are two questions that need answering, and they’re not the same question:

Did the action reach the provider?
Did the provider accept it?

Most agent implementations collapse these into one. The action fires, something comes back, and the outcome is recorded as success or failure. That works until the network gets involved, the provider is slow, or the response never arrives.

This post covers the reconciliation layer we built in StareBrain to handle the gap between dispatch and confirmation — and why collapsing those two questions into one creates failure modes that are invisible until they’re not.

The Two-Axis State Model

Every dispatchable action has two independent state progressions:

`enum class DeliveryState { PENDING, CONFIRMED, FAILED }
enum class AcceptanceState { PENDING, ACCEPTED, REJECTED }`

DeliveryState tracks whether the action reached the provider. AcceptanceState tracks whether the provider acted on it. These are orthogonal — delivery can be confirmed while acceptance is still pending, and acceptance can never move out of PENDING until delivery is CONFIRMED.

`data class ActionRecord(
    val actionId: String,
    val dispatchId: String,
    val providerObjectId: String?,
    val delivery: DeliveryState,
    val acceptance: AcceptanceState,
    val dispatchedAt: Instant,
    val confirmedAt: Instant?,
    val eligibleForReconciliationAt: Instant
)

fun ActionRecord.tryAccept(verdict: AcceptanceState): ActionRecord {
    check(delivery == DeliveryState.CONFIRMED) {
        "Cannot evaluate acceptance before delivery is confirmed."
    }
    return copy(acceptance = verdict)
}`

The structural invariant — acceptance cannot move until delivery is confirmed — prevents a class of race conditions where an optimistic acceptance gets recorded before the provider has even received the request.

DENIED_UNRESOLVED: A First-Class Outcome

The failure mode most systems miss: the action dispatched, the provider was reached, but the outcome is unknown. Network timeout after dispatch. Provider returned a non-terminal status. Response arrived but was malformed.

This isn’t a failure. It isn’t a success. It’s an unresolvable state — and it needs to stay that way permanently.

`enum class ActionStatus { CONFIRMED, FAILED, DENIED_UNRESOLVED }`

DENIED_UNRESOLVED never decays into success on timeout. It never gets silently retried. It sits in the record as a permanent signal that this action’s outcome is unknown and requires explicit reconciliation before any replacement dispatch.

The reason this matters: if a booking dispatch returned DENIED_UNRESOLVED and the agent silently retries, the provider may have already processed the first request. The result is a duplicate booking the user never approved.

Provider Capability Tiers

Not every provider supports the same reconciliation capabilities. We model this explicitly:

`enum class ProviderCapabilityTier {
    FULL,           // Native webhook + queryable by our reference
    RECONCILABLE,   // No webhook, but queryable by client reference
    PARTIAL,        // Queryable by provider ID only (we must store their ID)
    BLIND           // No query capability — outcome unknowable after dispatch
}`

The tier determines what reconciliation is possible after a DENIED_UNRESOLVED:

FULL: reconcile via webhook or active query using our own reference
RECONCILABLE: requires that our client reference was transmitted in the request AND is queryable by the provider — both conditions must hold
PARTIAL: we can query, but only if we captured the provider’s object ID at dispatch time
BLIND: reconciliation is impossible — surface to user for manual resolution

`data class ReconciliationPolicy(
    val tier: ProviderCapabilityTier,
    val minAgeBeforeEligible: Duration,
    val maxAttempts: Int,
    val clientReferenceKey: String?
)

fun ActionRecord.isEligibleForReconciliation(): Boolean {
    return Instant.now().isAfter(eligibleForReconciliationAt)
}`

The minAgeBeforeEligible gate prevents reconciliation attempts against records that are still within the provider’s normal processing window — querying too early produces false negatives.

The Pre-Dispatch Reconciliation Key

For RECONCILABLE and FULL tiers, the client reference must be generated before dispatch — not after. If it’s generated after, a network failure between generation and transmission means we have a reference that doesn’t match anything on the provider side.

`data class DispatchContext(
    val actionId: String,
    val reconciliationKey: String,  // generated before dispatch
    val providerTier: ProviderCapabilityTier,
    val dispatchedAt: Instant
)

fun buildDispatchContext(actionId: String, tier: ProviderCapabilityTier): DispatchContext {
    return DispatchContext(
        actionId = actionId,
        reconciliationKey = UUID.randomUUID().toString(),
        providerTier = tier,
        dispatchedAt = Instant.now()
    )
}`

The key is stored locally before the network call. If the call fails before transmitting the key, the record is DENIED_UNRESOLVED with no matching provider-side reference — which is the correct state, because reconciliation by reference isn’t possible.

Idempotency Scoping

Retries on non-terminal states need idempotency keys to prevent duplicate execution. We scope the key to event ID + action type, not just event ID:

`fun buildIdempotencyKey(eventId: String, actionType: String): String {
    return "$eventId:$actionType"
}`

Scoping to event ID alone fails when the same event produces multiple actions of different types — the second action gets rejected as a duplicate of the first. Scoping to event ID + action type keeps retries safe without blocking legitimate parallel actions.

What This Closes

The two-axis model makes it impossible to record acceptance before delivery is confirmed. DENIED_UNRESOLVED as a permanent state prevents silent retry on unknown outcomes. Provider capability tiers make reconciliation strategy explicit at the architectural level rather than scattered across action handlers. Pre-dispatch key generation closes the reference mismatch on network failure.

The invisible failure mode — action dispatched, outcome unknown, agent moves on — becomes a named, queryable, reconcilable state.