> ## Documentation Index
> Fetch the complete documentation index at: https://docs.komaa.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Attachments in chat
> Turning a pasted screenshot, a dragged-in file or a voice note into one ChatTurn your agent can answer, in the StandIn TypeScript SDK.
A Microsoft Teams message can carry a pasted screenshot, a file dragged in from disk, or a voice note. Without any of this, your handler gets `InboundMessage.attachments` as raw objects, so the best it can do is read a JSON blob to a model and the worst is answer a message about a picture as though nothing had been sent.
## One call does the whole message
```ts theme={null}
import { buildChatTurn, chatImageDataUrl } from "@komaa/standin-sdk";
const turn = await buildChatTurn(message, { transcribe: myStt });
const answer = await agent.ask(turn.query, turn.images.map(chatImageDataUrl));
```
`buildChatTurn` takes the `InboundMessage` your [chat handler](/typescript-sdk/chat) was given and returns a `ChatTurn`: the text to ask, the pictures to hand over, and a plain sentence naming anything that came with the message.
| Field | Type | What it is |
| ---------------- | ------------- | ------------------------------------------------------------------ |
| `query` | `string` | The text to put in front of the model, including every note below. |
| `images` | `ChatImage[]` | Pictures that were fetched and are ready for a vision model. |
| `voiceNote` | `string` | What the voice notes said, or empty. |
| `attachmentNote` | `string` | The "what was attached" sentence, on its own. |
Everything but the message arrives in one `ChatTurnOptions` object, and every field of it is optional:
```ts theme={null}
const turn = await buildChatTurn(message, {
origin: undefined, // leave it off to derive it; see the origin pin below
images: true, // false to skip image fetching entirely
transcribe: undefined, // your speech-to-text function, or no transcription
});
```
`images: false` and no `transcribe` make it a pure text assembler that opens no sockets at all. A handler that fetches nothing pays nothing. There is a fourth option, `fetchFn`, which substitutes the `fetch` every request goes through: it exists so a test can reach every edge of this page without a socket.
This lives in `attachments.ts`, deliberately **not** in `chat.ts`. That module owns the socket, the duplicate check and the per-conversation ordering, and none of that changes here. Fetching is optional work that must never be able to wedge the transport, so it sits beside it rather than inside it. Everything on this page is re-exported at the package root, with exactly one exception, and that exception is under [Images](#images) below.
## The order is the order a person would say it in
`query` is assembled in a fixed order, and the order is not arbitrary: it is the order somebody would have said the same thing out loud.
1. **What they typed.** `message.text`, trimmed.
2. **What they pressed.** The submit payload of an `Action.Submit` on a card this agent sent, as `cardActionNote`. A card message arrives with **empty text**, so without this the agent is asked nothing at all and answers as though the person said nothing. The payload is serialized and truncated at `CARD_PAYLOAD_MAX_CHARS`, which is 4096 characters, and that bounds this agent's own card template rather than a stranger's message.
3. **What they said out loud.** The transcribed voice notes, under `[They sent a voice message]`.
4. **What they attached.** The note from `attachmentsNote`, under `[Attached to this message]`.
Each part is omitted when it is empty, and the parts are joined by blank lines. A message with nothing attached produces a `query` that is exactly the text the person typed.
## It never throws
`buildChatTurn` does not throw. Anything that will not load is **named in the note** instead of failing the turn, because a failed picture is not a reason to leave a question unanswered.
The note distinguishes the two outcomes per attachment:
```text theme={null}
[Attached to this message]
- plan.png [image] (attached)
- q3.xlsx [file] (unreadable)
```
That distinction is the point of the note. A model told a picture was attached and could not be opened says something useful. A model told nothing answers as if the message were empty. And a model told a picture was attached, when nothing was actually fetched, answers **about a picture it has never seen**, which is the worst of the three.
The note is bounded at `ATTACHMENT_NOTE_MAX_LINES` entries, which is ten, followed by a count of the rest, so a message with forty attachments does not become the whole prompt. `attachmentsNote(attachments, status)` is callable on its own, where `status` is a `Map` keyed by the attachment's index in the message. Both that bound and `CARD_PAYLOAD_MAX_CHARS` above are exported from the package root here. The Python twin keeps its two out of its public surface entirely, so that page spells the numbers rather than naming a constant to import.
## Images
```ts theme={null}
import type { ChatImage } from "@komaa/standin-sdk";
```
| Field | Type | What it is |
| ------------ | --------------------- | --------------------------------------------------------------- |
| `dataBase64` | `string` | The image as it arrived, base64. |
| `mime` | `string` | The real media type. The response's own header wins; see below. |
| `name` | `string \| undefined` | The filename, when there was one. |
| `sizeBytes` | `number` | How big it was. |
`ChatImage` is an interface, and an interface carries no accessors, so the two conveniences the Python twin exposes as properties are plain functions here:
```ts theme={null}
import { chatImageData, chatImageDataUrl } from "@komaa/standin-sdk";
chatImageData(image); // Buffer, for an API that uploads a file
chatImageDataUrl(image); // "data:image/png;base64,..." for most vision APIs
```
`fetchChatImages(attachments, { origin })` is the piece underneath, for a handler that wants the pictures without the rest of the turn. It is best-effort per attachment: one that will not load costs that attachment, never the answer.
The caps are at the package root, with one missing on purpose:
```ts theme={null}
import {
IMAGE_FETCH_ATTEMPTS, // 8
IMAGE_FETCH_TIMEOUT_MS, // 10_000
MAX_IMAGES, // 4
} from "@komaa/standin-sdk";
```
`MAX_IMAGES` is the **accept** cap: four pictures are kept from one message, because each becomes a base64 blob in front of a model. `IMAGE_FETCH_TIMEOUT_MS` is how long one image has to arrive, **in milliseconds**. The Python twin spells the same two budgets in seconds, as `IMAGE_FETCH_TIMEOUT_S` and `CLIP_FETCH_TIMEOUT_S`, so a number carried across unchanged is out by a factor of a thousand. Override either cap on a `fetchChatImages` call with its `maxImages` and `timeoutMs` options.
`IMAGE_FETCH_ATTEMPTS` is a **separate** cap, and it is the one that keeps the worst case arithmetic. The accept cap counts only successes, so without it a message naming fifty attachments that all time out still costs fifty timeouts and blows the whole turn budget while nothing is ever accepted. The attempt cap counts requests whatever the outcome: eight tries at ten seconds, and that is the ceiling.
The one number you cannot import is how much one image may weigh. It is 4 MiB, it is the default for the `maxBytes` option, and it matches the per-attachment ceiling StandIn applies anyway, so a larger local number could never be reached.
`MAX_IMAGE_BYTES` **is** exported from the package root, and it is **not** this cap. The root one comes from `vision.ts` and is `1_400_000`: the **outbound** limit on an image you draw on the bot's video tile. The inbound attachment cap is a different number in a different direction of travel, and it keeps its own `MAX_IMAGE_BYTES` inside `attachments.ts` rather than colliding with the other at the root.
Two consequences. Passing the root `MAX_IMAGE_BYTES` as `fetchChatImages`'s `maxBytes` compiles cleanly, is a perfectly real number, and quietly caps inbound attachments at 1.4 MB instead of 4 MiB. And there is no deep import that reaches the other one: the package's `exports` map lists the root and the plugin subpaths only, so `@komaa/standin-sdk/attachments` does not resolve. Leave `maxBytes` unset to take the 4 MiB default, or pass a number of your own. `buildChatTurn` never takes one at all, so a turn assembled through it always uses the default. See [Vision](/typescript-sdk/vision#fetching-an-image-a-model-chose) for the outbound one.
Two more things the fetch does, both about what arrives rather than what was claimed. The media type is judged **before a byte is read**, and the response's own header wins, because an error page would otherwise be base64'd in front of a model as though it were a picture. What the message declared is the fallback, and only when it is itself a media type: a dragged-in file declares a bare extension, and taking that literally would fail every `image/` and `audio/` check. And the size cap holds **while** the body is read rather than after it, because a `content-length` that lies, or is simply absent, otherwise gets to allocate whatever it likes before a later check objects.
## A dragged-in picture is a file, not an image
A pasted screenshot arrives with `kind` of `image`. The **same file** dragged in from disk arrives with `kind` of `file` and a `contentType` that is a bare extension like `png` rather than a media type.
So gating on the declared kind alone is the trap here, and it is a quiet one: the picture is never fetched, but the note still says one was attached, and the model answers about a picture nobody gave it.
An attachment is therefore worth a request when its kind matches **or** when its kind is exactly `file` and its filename's extension, or a declared type that is itself a bare extension, is one of `png`, `jpg`, `jpeg`, `gif`, `webp`, `bmp`, `heic`, `heif`. The widening is `file` only: it exists to rescue the dragged-in case, not to override a kind the message stated outright. What the file then turns out to be is decided by the response, not by the extension that got it tried.
An attachment StandIn marked as not relayable is skipped without a request.
## Voice notes
```ts theme={null}
import type { ChatAudio, Transcriber } from "@komaa/standin-sdk";
```
`ChatAudio` is one clip before anything has transcribed it: `data` as a `Buffer`, `mime`, and `name`. `fetchChatAudio` returns them. `video/` types are accepted as well as `audio/`, because some clients label a voice note with a container type that speech-to-text reads perfectly well, and the same extension widening applies: a voice note relays as a file in practice, so a plugin gating on `kind === "audio"` transcribes nothing, ever.
`Transcriber` is the function **you** supply:
```ts theme={null}
type Transcriber = (data: Buffer, mime: string) => Promise;
```
The core ships none and reads no provider key. Deciding which speech vendor somebody's voice is sent to is not a decision an SDK gets to make on their behalf, and a default would make it silently.
```ts theme={null}
import { transcribeVoiceMessages } from "@komaa/standin-sdk";
const said = await transcribeVoiceMessages(attachments, {
origin,
transcribe: myStt,
});
```
It returns the clips as one block of text, and empty when there are none, when no transcriber was supplied, or when every one failed. With no transcriber it fetches nothing at all rather than downloading audio that has nowhere to go. A transcriber that throws costs that clip and nothing else.
The clip budgets are deliberately larger than the image ones, and all four are at the package root:
```ts theme={null}
import {
CLIP_FETCH_ATTEMPTS, // 4
CLIP_FETCH_TIMEOUT_MS, // 20_000
MAX_CLIPS, // 2
MAX_CLIP_BYTES, // 16 MiB
} from "@komaa/standin-sdk";
```
A voice note is minutes of audio where a picture is one screen, so it needs four times the bytes and twice the wall clock to arrive. The attempt cap is halved to pay for that: four tries at twenty seconds is the same eighty second worst case as eight tries at ten. And two clips rather than four, because each accepted clip is a transcription call on top of the download.
## An engine that only takes a path
Plenty of transcription engines will not take bytes. `spoolClip` writes the clip to a temporary file, named with the extension its media type implies, and hands your callback the path:
```ts theme={null}
import { spoolClip, type Transcriber } from "@komaa/standin-sdk";
const myStt: Transcriber = async (data, mime) =>
spoolClip({ data, mime, name: "" }, (path) => engine.transcribeFile(path));
```
It resolves to whatever your callback resolved to, and it rethrows whatever your callback threw. The file is removed **on the way out, on every path**, including the one where the engine threw. That is not tidiness. A transcription engine is being handed somebody's voice, and a clip left behind in a temporary directory is a copy nobody decided to keep, sitting on disk long after the conversation it came from ended.
This is a callback rather than a context manager, which is the shape TypeScript has for "hold this open for exactly this long". The other difference from the Python twin is that this one always spools into the system temporary directory: there is no directory argument here.
## The origin pin
Every fetch on this page is pinned to the one origin the messages themselves arrived from:
```ts theme={null}
import { chatAttachmentOrigin } from "@komaa/standin-sdk";
chatAttachmentOrigin(); // from STANDIN_CHAT_URL, else the default
chatAttachmentOrigin("wss://example.com/api/chat/channel"); // "https://example.com"
chatAttachmentOrigin("ws://127.0.0.1:9444/x"); // "http://127.0.0.1:9444"
```
It is derived from the chat channel's own URL rather than configured separately, so it is right by construction for a self-hosted or local setup and there is no second setting to get wrong. `ws` maps to `http` and `wss` to `https`, and a default port is not spelled out, so `wss://host:443` and `wss://host` compare equal.
The name is `chatAttachmentOrigin` here and `attachment_origin` in Python. That is the one name on this page that is not a straight casing translation of its twin, so a search for `attachmentOrigin` finds nothing.
It **fails closed**. A URL it cannot resolve, and anything that is not `http`, `https`, `ws` or `wss`, returns `undefined`, and `undefined` means fetch nothing at all.
That direction is the whole design. An attachment URL is signed, but not by you and not for you: it arrives inside a message somebody else wrote, and the pin is the only thing bounding where this worker can be told to go. An unset origin read as "anywhere" would turn a configuration typo into a fetcher that a stranger's message can point at any address it likes, an internal service or a cloud metadata endpoint included.
Two consequences worth knowing. An attachment whose URL is not on that origin is refused **without a request**, so the log line is the only cost. And no redirect is followed: the URL is same-origin and signed, so a redirect off it is already anomalous, and following one would reopen the door the pin just closed, since the pin is checked on the URL in the message and not on wherever a `302` points.
Pass `origin` explicitly when your handler already knows it. Otherwise `buildChatTurn` derives it for you.
## Next
The channel these messages arrive on, and what goes back out.
The signing lanes, and the rest of the fetch posture.