The chart is almost right. You correct one label, then ask the agent to change the colors. It regenerates the chart and your label correction disappears.
That is a product bug, even if both generations look impressive. The user was editing an object; the system treated each turn as a fresh request for a picture.
This note uses a generalized AI-analyst interface to explain the design choices behind an editable result. Tool names and event payloads are illustrative.
Give the artifact an identity and a revision
A chart should be a structured object with an id. A later request should refer to that object and apply a bounded change. Changing its colors should not silently replace its data, labels, or filters.
An id alone is not enough when human edits and agent updates can race. Track the revision too. An update based on revision 3 should not blindly overwrite the user's revision 4.
type ChartPatch = {
chartId: string;
expectedRevision: number;
changes: {
title?: string;
colors?: string[];
};
};The server validates the permitted changes and rejects a stale revision. The interface can then show what changed and ask the agent to reconcile against the current chart. That is easier to understand than quietly accepting whichever response arrived last.
This also changes export. Keep charts editable where the target format supports it, and let users inspect the actual export artifact. A flattened image is useful for sharing a finished picture; it is frustrating as the only representation of a chart someone is still working on.
Streaming needs events with meaning
Appending text chunks works for prose. It becomes fragile once a response also contains tool progress, citations, chart updates, and errors. Use typed events and a reducer so each event has an explicit effect on the interface.
type StreamEvent =
| { type: "message_start"; messageId: string }
| { type: "text_delta"; messageId: string; delta: string }
| { type: "tool_start"; toolCallId: string; summary: string }
| { type: "tool_result"; toolCallId: string; status: "ok" | "error" }
| { type: "chart_patch"; patch: ChartPatch }
| { type: "message_end"; messageId: string; status: "complete" | "partial" }
| { type: "stream_error"; messageId: string; detail: string };Show a tool's activity when it starts, then its outcome when it finishes. The summary should explain the action to the user without dumping credentials or raw internal payloads into the chat.
Preserve partial work when a stream fails, but keep it visibly incomplete. A
network disconnect must not have the same meaning as message_end. If a stream
can resume, use event identities or sequence numbers to prevent replayed deltas
from duplicating text and repeated results from reapplying edits.
stateDiagram-v2
[*] --> Streaming: message_start
Streaming --> Streaming: text, tools, artifact updates
Streaming --> Complete: message_end complete
Streaming --> Partial: message_end partial
Streaming --> Interrupted: connection lost or stream_error
Interrupted --> Streaming: verified resume
Complete --> [*]
Partial --> [*]Keep candidates separate from committed values
An autofill result begins as a proposal: a value plus the source used to obtain it. Accepting it commits the value and its provenance. Rejecting it leaves the field unchanged. The pending state needs to look different from both an empty field and an accepted value.
A confidence number can look reassuring without being calibrated. I would rather show the passage, its date, and any conflicting evidence than put an unexplained percentage beside the candidate. The citation note covers why even a real source link still needs a support check.
Make rejecting cheap. A review flow that makes acceptance one click and rejection a form is encouraging a particular decision instead of helping the user inspect the result.
Approval applies to the revision reviewed
An export preview is only meaningful if it corresponds to what gets exported. Bind the approval record to the artifact revision, and invalidate it when the content changes. Otherwise a person can approve one chart while the background job exports its replacement.
The server must enforce that relationship. Disabling an export button in the browser is presentation, not authorization. The same boundary should enforce which tools can run and which sources the user can access. Provider credentials stay behind the application API.
For long exports, retain the job id, the approved revision, and a readable failure reason. A retry should make clear whether it will reuse the reviewed content or requires a new review.
Test the moments where work can be lost
Most of this behavior is deterministic. A scripted gateway can replay the exact sequences that are difficult to reproduce with a live model:
- a user edit followed by a stale agent patch;
- a citation arriving after its text;
- the same event delivered twice after reconnect;
- a failed tool call while other output continues streaming;
- an export requested after the approved artifact changes.
Reducer tests can check the state transitions. A few end-to-end flows can check that the user sees the conflict, retains the edit, and exports the revision they reviewed. Those checks say more about whether the interface is usable than a snapshot of a perfect generated answer.
The next prompt should continue the user's work. Keeping that promise takes object identity, revision checks, and explicit states underneath the animation.