Checking the install

Every plugin has to answer one question before anybody trusts it with a real call: does this actually work here? Until now the only way to find out was to place a real call, which costs a provider bill, a public tunnel, a Microsoft Teams tenant and somebody’s afternoon. run_smoke answers it on loopback in under a second. It binds an ephemeral listener, connects a real client that speaks the real call wire, streams a few frames of silence, and reports what came back.

import asyncio

from standin.smoke import report, run_smoke

from my_plugin import MyHandler

async def main() -> int:
    result = await run_smoke(MyHandler)
    print(report(result))
    return 0 if result.ok else 1

raise SystemExit(asyncio.run(main()))

handler_factory is the same callable CallServer takes, called once. A handler class with a no-argument constructor works as it is; anything else goes in a lambda.

The printer is on the package as smoke_report, not report. The module names the function report and the package renames it on the way out, so from standin import report does not resolve. SyntheticCall, run_smoke, SmokeCheck and SmokeResult keep their own names.

What it prints

standin: ok
  ok   secret (generated for this run)
  ok   listener (127.0.0.1:55064/msteams/calling)
  ok   call (10 frames in 235 ms)
  ok   audio (10 frames came back)

And when the handler never sent anything back:

standin: NOT ok
  ok   secret (generated for this run)
  ok   listener (127.0.0.1:55073/msteams/calling)
  ok   call (4 frames in 93 ms)
  FAIL audio (0 frames came back)
       without it: the caller would hear nothing

Every failed check prints what stops working without it. A report that names a missing surface and not the consequence sends the operator to the source to find out whether it matters. report returns the string rather than printing it, so a plugin can put it in a log line, a tool result or an HTTP response. When the run recorded an error, a final error: ... line follows the checks.

The four checks

They run in this order, and each one is the next thing that breaks in the field.

Check	What it proves	What the failure costs
`secret`	A credential exists for this run.	nothing could authenticate
`listener`	The worker bound its socket and has a call path. The detail is the bound address.	no call could ever reach this worker
`call`	Connect, handshake, `session.start`, the frame loop and hang-up all completed.	a real call would fail the same way, or would hang the same way if the run timed out
`audio`	Frames came back out of the handler.	the caller would hear nothing

All four are mandatory, and result.ok is true only when every mandatory check passed. ok is never true when nothing echoed. The whole point of the run is that audio made the round trip, so a run that proved nothing must not read as a pass. A listener that binds, authenticates and then says nothing is exactly the install people ship by accident. The run sends recording.status: active before the first frame. Handlers commonly gate their output on the call being recorded, and without it a recording-gated handler stays silent and the run reports a false negative. Frames are PCM16 mono at 16 kHz, 20 ms each, which is FRAME_BYTES (640) at the cadence a real call arrives at. frames defaults to 10. If the listener cannot bind, the run returns there and then with two checks, a failed listener carrying the bind error, and echo_frames at 0. There is nothing left to ring. The two results are plain dataclasses:

@dataclass(frozen=True)
class SmokeCheck:
    name: str
    ok: bool
    detail: str = ""
    cost: str = ""
    required: bool = True


@dataclass
class SmokeResult:
    checks: list[SmokeCheck] = field(default_factory=list)
    echo_frames: int = 0
    error: str = ""

    @property
    def ok(self) -> bool: ...

It never borrows your configuration

The run generates its own connection secret, secrets.token_hex(16), for that run only. It never reads STANDIN_SECRET. Two reasons, and both are real failures rather than tidiness. A fixed string would be a predictable credential on a live listener. The check is meant to be safe to run anywhere, including on a host that is answering calls. Borrowing the operator’s secret makes the run pass or fail for reasons that have nothing to do with the wire, and it makes the check impossible to run before the secret is configured. That is exactly when people run it.

Loopback, on port 0

The listener binds 127.0.0.1 on port 0, and the port it actually got is read back after it is listening. Loopback because a verification run has no business being reachable from the network. Port 0 because the alternative is picking a free port in advance with a throwaway socket, which races whoever binds it next. Two runs in the same CI job, or a run next to a real listener, and the loser fails for a reason that has nothing to do with the install.

Nothing that delivers is started

This is the most important property on the page. No pending-message sweep, no no-answer reaper, no outbound caller, no chat lane. Nothing that delivers is started, and the listener’s own stale-call reaper is off for the run (stale_call_reaper_seconds=0).A verification command that quietly resumed durable jobs would place real calls and post real messages as a side effect of somebody checking an install.

Durable work is durable on purpose. Parked messages, background-task promises and unanswered outbound legs all survive a restart, because the person they are for is still owed the answer. That is correct for a worker coming up, and completely wrong for a check somebody typed to find out whether an install works. run_smoke starts a listener and a handler, and nothing else. The listener is closed with aclose() in a finally, on every path including the failing ones, so a run leaves nothing bound behind it.

A plugin’s own checks

extra runs after the call and returns more checks, so a plugin can prove its own surfaces in the same report.

from standin.smoke import SmokeCheck, run_smoke


async def my_checks() -> list[SmokeCheck]:
    return [
        SmokeCheck("model", await model_reachable(), cost="the agent could not answer"),
        SmokeCheck(
            "browser",
            await browser_reachable(),
            detail="no display",
            cost="show_page would apologise",
            required=False,
        ),
    ]


result = await run_smoke(MyHandler, extra=my_checks)

Mandatory unless declared advisory. required defaults to True, so a check you add counts towards ok unless you pass required=False. Set it false for anything the call itself survives without: a degraded extra is worth reporting and is not worth failing an install over. Give every check a cost. It is the sentence printed under a failure, and it is what tells the operator whether to act now or after lunch.

  FAIL browser (no display)
       without it: show_page would apologise

An extra that raises is recorded, not fatal. The report gains a non-required plugin checks entry carrying the exception text, because a broken check must not take down the report of the call that did work.

Pointing one at your own listener

SyntheticCall is the client run_smoke uses. It is exposed because a plugin with its own listener may want to point one at that instead.

from standin.smoke import SyntheticCall

call_id = "smoke-local"
call = SyntheticCall(
    f"http://127.0.0.1:{port}/msteams/calling/{call_id}",
    secret,
    call_id,
    frames=10,
)
await call.run()
print(call.echo_frames)

run() connects, greets, streams, hangs up, and raises on anything that fails. echo_frames is how many audio frames came back. The constructor takes url, secret, call_id, then frames (10, clamped to at least 1) and connect_timeout_s (5.0). Three things it does that a hand-rolled client usually gets wrong:

The handshake is signed freshly for each connect. The listener enforces single use inside the freshness window, so a retry that reused the headers would loop on 401 and read as a wrong secret.
The callId in session.start is identical to the one in the path. A start that disagrees with the authenticated path is refused, and the refusal reads as a wire fault.
The socket is drained between sends, with a bound. A blocking read stalls the 20 ms cadence and makes the call look idle to the server’s watchdogs. A socket that is never drained leaves the echo count at zero for ever, so “audio came back” could never be proven. Anything it cannot parse is not an echo.

The socket is closed in a finally, on every path. An error path that skipped the close leaks a client session per run inside a long-lived host that exposes the check.

The client lives in the SDK, next to the wire it speaks, deliberately. A copy kept inside a plugin drifts, and the copy that used to exist kept passing against a call path that no longer existed. That is the exact failure a smoke check is for.

Timeouts

The whole run is bounded at timeout_s, 15 seconds by default, and the connect at 5. A listener that accepts and then stops talking would otherwise hang CI, and hang a status tool call for ever. A run that times out is a recorded failure, not a hang. The call check fails with the call did not finish within 15 seconds, and the same sentence becomes result.error.

async def run_smoke(
    handler_factory: Callable[[], CallHandler],
    frames: int = 10,
    extra: Callable[[], Awaitable[list[SmokeCheck]]] | None = None,
    timeout_s: float = RUN_TIMEOUT_S,  # 15.0
) -> SmokeResult: ...

In a plugin’s own command

Wire it to a subcommand and exit on result.ok, so CI fails on a bad install. The Hermes plugin does exactly this:

hermes msteams-bridge smoke

It needs no network, no tenant and no credentials, so it also belongs in the test suite, next to the unit tests. Four frames is enough to prove a round trip and keeps the test well under a second.

Get started

The SDK

Plugins

Checking the install

What it prints

The four checks

It never borrows your configuration

Loopback, on port 0

Nothing that delivers is started

A plugin’s own checks

Pointing one at your own listener

Timeouts

In a plugin’s own command

​What it prints

​The four checks

​It never borrows your configuration

​Loopback, on port 0

​Nothing that delivers is started

​A plugin’s own checks

​Pointing one at your own listener

​Timeouts

​In a plugin’s own command

What it prints

The four checks

It never borrows your configuration

Loopback, on port 0

Nothing that delivers is started

A plugin’s own checks

Pointing one at your own listener

Timeouts

In a plugin’s own command