Deepgram plugin

standin.plugins.deepgram puts Deepgram on a real Microsoft Teams call. StandIn answers the call and dials your worker; this plugin answers that dial, opens one session per call, and relays the audio both ways.

Install and run

There is nothing to install beyond the SDK. Deepgram is reached over an ordinary WebSocket, so this plugin adds no dependency and needs no extra.

pip install standin-sdk

export STANDIN_SECRET=...          # your StandIn connection secret
export DEEPGRAM_API_KEY=...
python -m standin.plugins.deepgram

Expose port 9442 at the /msteams/calling path and register the public wss:// URL as your StandIn identity’s agent voice URL. The full walkthrough is in the Quickstart, and there is a runnable example at examples/deepgram-msteams-connector.

The audio format

Audio is pinned to linear16 at 16 kHz in both directions when the session starts. That is exactly what a Microsoft Teams call carries, so the hot path is a copy and nothing resamples anything.

What the agent can do about the call

These are declared automatically when the call starts, so there is nothing to set up on the provider’s side. Nothing to implement: the plugin answers them.

Tool	Parameters	What it does
`end_call`	none	Hang up.
`express`	`emotion`	Set the avatar’s expression on the bot’s tile. See The avatar.
`show_image`	`url`, `caption?`, `display?`	Put a picture on the bot’s video tile.
`look`	`source?`, `question?`	Look at the caller’s screen share or camera.
`look_back`	`question?`	Look again at something the caller has already moved past. Recorded calls only.

show_image takes a public url, fetched through the SDK’s guard: public hosts only, no private or link-local addresses, and the address is re-checked at connect time to close the DNS rebind window. display is fullscreen or overlay, and this plugin routes the call through CallTools.dispatch, so the model’s choice is honoured. See Vision and the avatar for what each mode is for, and Sending a picture for putting a picture into chat instead. A Voice Agent hears but does not see, so look needs a vision model of your choosing. Set STANDIN_VISION_API_URL and STANDIN_VISION_MODEL to any OpenAI-compatible endpoint that accepts images, including one you run yourself. The frame is sent for inference and not stored, and only the description comes back. Without it the agent is told plainly that looking is unavailable, rather than being left with silence. The whole surface is on Vision and the avatar. Pass describer= to point this plugin at a different vision model from the one the environment names, which is how one worker answers two identities with two vision budgets. Build it with FrameDescriber.from_env() or its constructor, both on Vision and the avatar.

How context reaches the agent

The Voice Agent API has no non-interrupting context message, so this plugin folds participant counts, key presses and recording changes into the agent’s prompt and pushes it with an update. That has a cost worth knowing: the prompt is resent in full on every change, so the context section is a rolling window of the last eight notes rather than a transcript. A meeting where people join and leave repeatedly keeps the recent notes and drops the old ones, which is the right trade when the alternative is a prompt that grows for the length of the call. Context that arrives before the agent socket is open is held rather than dropped, then folded in once it exists. The “there are N people here, stay quiet” line and the recording change both land in exactly that gap.

Tools of your own

A custom tool runs in your worker, so it can reach whatever your worker can reach:

from standin import CallServer
from standin.plugins.deepgram import CustomTool, DeepgramConfig, DeepgramHandler

async def open_ticket(params, ctx):
    # ctx.call is the live call; ctx.recording says whether it is being recorded.
    return f"opened ticket for {params['summary']}"

tools = [
    CustomTool(
        name="open_ticket",
        description="Open a support ticket for the caller.",
        handler=open_ticket,
        parameters={
            "type": "object",
            "properties": {"summary": {"type": "string"}},
            "required": ["summary"],
        },
    )
]

config = DeepgramConfig.from_env()
server = CallServer(handler_factory=lambda: DeepgramHandler(config, tools=tools))
await server.start()

The description is what the model reads to decide whether to call it, so write it for a model rather than for a developer. Keep the handler fast: the caller is waiting in silence while it runs. A custom tool may not shadow a built-in one. That is refused when the handler is built rather than at the first call, because a shadowed end_call is an agent that has quietly lost the ability to hang up.

Inside your own worker

The plugin is a CallHandler like any other, so you can build it yourself instead of running the module:

from standin import CallServer
from standin.plugins.deepgram import DeepgramConfig, DeepgramHandler

config = DeepgramConfig.from_env()
server = CallServer(handler_factory=lambda: DeepgramHandler(config))
await server.start()

Read the configuration once, then close over it. handler_factory runs once per call, so handler_factory=DeepgramHandler would call DeepgramConfig.from_env() again on every call, and a key removed from the environment after startup would fail the next caller rather than failing you. server.start() returns as soon as the listener is bound, so a script that ends there exits before a single call arrives. serve(), which is what python -m standin.plugins.deepgram runs, is the sample above plus an await asyncio.Event().wait() to hold the process open and an await server.aclose() in a finally, which drains live calls and releases the port on the way out.

Configuration

Variable	What it is
`DEEPGRAM_API_KEY`	Your API key. Never logged, never sent to StandIn.

Optional settings

Variable	What it is
`DEEPGRAM_LISTEN_MODEL`	Speech to text. Defaults to `nova-3`.
`DEEPGRAM_SPEAK_MODEL`	Text to speech. Defaults to `aura-2-thalia-en`.
`DEEPGRAM_THINK_PROVIDER`	Which vendor reasons. Defaults to `open_ai`.
`DEEPGRAM_THINK_MODEL`	Defaults to `gpt-4o-mini`.
`DEEPGRAM_THINK_ENDPOINT_URL`	Reason with your own model instead.
`DEEPGRAM_THINK_ENDPOINT_HEADERS`	A JSON object of headers for that endpoint. Carries your model credentials, so it is never logged. Anything but a JSON object is refused when the configuration is read.
`DEEPGRAM_INSTRUCTIONS`	The agent’s base prompt.
`DEEPGRAM_GREETING`	What the agent says first.
`DEEPGRAM_LANGUAGE`	Defaults to `en`.
`DEEPGRAM_AGENT_HOST`	The Voice Agent socket host. Defaults to `agent.deepgram.com`. Must be a deepgram.com host.
`DEEPGRAM_API_HOST`	The host the REST calls go to. Defaults to `api.deepgram.com`. Must be a deepgram.com host.
`DEEPGRAM_LOG_TRANSCRIPTS`	Exactly `true` to log turns. Gated a second time on the call being recorded.

The configuration is read once when the worker starts, not per call, so a missing key stops the worker at startup rather than surprising the first caller. Both hosts are checked against the deepgram.com suffix, because your API key travels to them: a wrong host would be credential leakage rather than a failed call. The reasoning behind that shape is on Configuration.

Realtime providers

The startup buffer, the echo guard and barge-in this plugin sits on.

Vision and the avatar

What look, show_image and express reach.

Get started

The SDK

Plugins

Deepgram plugin

Install and run

The audio format

What the agent can do about the call

How context reaches the agent

Tools of your own

Inside your own worker

Configuration

Next

Realtime providers

Vision and the avatar

​Install and run

​The audio format

​What the agent can do about the call

​How context reaches the agent

​Tools of your own

​Inside your own worker

​Configuration

​Next

Realtime providers

Vision and the avatar

Install and run

The audio format

What the agent can do about the call

How context reaches the agent

Tools of your own

Inside your own worker

Configuration

Next