Get image and video generation working
Wire an image or video provider so a generation completes end to end: request variables, array parser paths, status polling, storage of the result, and the picture or clip shown in the agent conversation.
An image or video generation has four stages before a user sees the result: the provider request, the provider's response (often a task ID), the status query that completes the task, and file storage. This guide configures each one for the image-creation and video-creation scenes, then shows the same result published by the agent. For the model behind it, see AI channels, scenes and task lifecycle.
Before you begin
- An account with an image or video provider, its API key, and a real example of its submit response and its status response.
- File storage configured in Settings → Storage Config. Without it a task still succeeds, but nothing is copied into the user's files and the agent has no file to publish. See Set up file storage and uploads.
- An admin account with
ai.readandai.write. - For the agent part only: the agent starter running, with a model for
agent.general.
Wire the provider
Create the provider and its request
On Settings → AI Config (/admin/config/ai), create the provider with its base URL and API key as described in Connect an AI provider. In Request parameters, build the body the provider expects from these variables:
| Variable | Value |
|---|---|
{{prompt}} | The prompt of the request |
{{input}} | The request's input; the prompt when none is given |
{{metadata.<name>}} | Any other field of the request, such as size, aspect_ratio or duration |
{{externalTaskId}} | The provider's task ID, once it is known. Used in the status URL |
{{sceneKey}}, {{channelKey}}, {{mediaType}} | The scene, the provider and the output type of the call |
{{secret.<key>}} | An admin config setting classified as secret |
A text value is inserted as it is and any other value as JSON, so a number such as a duration arrives as a number. Any other variable is rejected with Unsupported template variable. A template that contains a pasted key is also rejected, so reference a secret instead.
Fields sent to POST /api/<scene>/generate other than reserved names such as prompt and mediaType, and the fields of input in an agent's model.generate call, are the request's metadata. For example, a frame chosen in the image agent reaches the provider as {{metadata.aspect_ratio}}.
Add one provider scene per model
Add a provider scene with output type Image for the image model and one with Video for the video model. In Provider scene endpoint, enter either a path starting with /, which is appended to the provider's base URL, or a full https:// URL. Set its pricing as in Route scenes and set pricing.
Point the parser at the provider's fields
Fill in Upstream task ID path, Status path, Output path and Error path from the real responses. Paths start at $ and may index arrays:
{ "code": 200, "data": [{ "status": "submitted", "task_id": "task_1" }] }| Field | Path |
|---|---|
| Upstream task ID path | $.data[0].task_id |
| Status path | $.data[0].status |
{ "data": { "status": "completed", "result": { "images": [{ "url": ["https://cdn.example.com/a.png"] }] } } }| Field | Path |
|---|---|
| Status path | $.data.status |
| Output path | $.data.result.images[0].url[0] |
An index past the end of an array, or an index applied to an object, is treated as a missing value. Submit and status responses usually differ in shape, so set the paths for the status query separately under Status response fields.
Map every status the provider returns
In Status mapping, list which provider strings mean Queued, Running, Succeeded, Failed and Canceled, comma-separated. Matching ignores case. Include the status of the submit response too (submitted above belongs under Queued). A status that is in no list fails the task with Unmapped AI channel status, even when the provider accepted the work.
Paste a real response and click Test parser before saving.
Set how the result arrives
Most image and video APIs return a task ID. Under Result handling, choose Status polling and fill in Status URL template with {{externalTaskId}} in it, plus Interval sec.
On a provider scene, the status URL may be a path starting with /. It is then resolved against the provider's base URL, the same way as the endpoint:
Provider base URL https://api.example.com
Status URL template /v1/tasks/{{externalTaskId}}
Queried https://api.example.com/v1/tasks/task_1A full https:// URL is used as it is. For providers that call back instead, see Receive async results.
Route the scenes
Route image-creation to the image provider scene and video-creation to the video provider scene. A scene only routes to an active, ready provider scene of the same output type.
How a running task finishes
A task that the provider accepted stays running until its status is queried. Three triggers query it, and each one settles the credits when the provider reports that the task ended:
| Trigger | When |
|---|---|
A client reading GET /api/status?taskId=… | Whenever the polling interval has passed. The /app/ai/<taskId> page does this while it is open. |
The scheduled job ai-channel-tasks | Once a minute, for tasks nobody is watching. See Background jobs. |
The agent's model.collect | When a run picks up a generation it started with async: true. |
When the task succeeds, every URL in its output is downloaded and stored as a private file in the user's /generated folder, marked as coming from that task. The task's output then lists those stored files.
Check the storage endpoint
Generated files are written through the storage provider you configured. For an S3-compatible provider, the address of an object follows the S3 endpoint:
| Endpoint | Object address |
|---|---|
https://s3.example.com | https://s3.example.com/<bucket>/<key> (path style) |
| The host already starts with the bucket name | https://<bucket>.storage.example.com/<key> |
A host ending in .aliyuncs.com | https://<bucket>.oss-<region>.aliyuncs.com/<key> (virtual-hosted style) |
Requests and presigned URLs are signed for the same address, so enter the endpoint exactly as the provider documents it. For Alibaba Cloud OSS, enter the region endpoint without the bucket.
Publish from the agent
The agent generates through the same scenes and needs no further configuration. In a run:
- A skill calls
model.generatewithsceneKey: "image-creation"or"video-creation"and asaveTopath. Withasync: trueit gets a task handle and later callsmodel.collectwith the samesaveTo. - The file written at
saveTois a small manifest that names the stored file. The picture or clip itself is already in storage. - The skill calls
artifact.publishwithkind: "image"or"video"and that path. The artifact's revision then points at the stored file, and nothing is downloaded or uploaded again. This applies when the manifest names exactly one file, the file belongs to the user, came from an AI task, and its content type matches the kind. - The conversation shows the artifact in the message: a picture as a preview, a clip as a player with controls. The side panel can still open it.
Preview grants last one minute. A clip is read in ranges while it plays, so when a load fails on an expired grant the player fetches a new grant once and resumes at the same position. When a load fails while the grant is still valid, the card shows its open and download actions instead of the player.
For pictures with a frame picker and a per-run limit, install the image agent.
Verify
-
Start an image task from
/app/ai, or with the API:curl -X POST https://<your-domain>/api/image-creation/generate \ -H "Authorization: Bearer $API_KEY" \ -H "Content-Type: application/json" \ -d '{"prompt":"A lighthouse at dusk","mediaType":"image"}' -
The task moves from
runningtosucceededon/app/ai/<taskId>, and its events includeGenerated output imported to storage. -
The picture is listed among the user's generated files on
/app/assets. -
Repeat with
video-creationand"mediaType":"video". -
With the agent installed, ask
/app/agentfor a picture or a clip. The result appears inside the message, and a clip still plays after the page has been open for more than a minute. -
In the repository,
pnpm validate:generation-pipelinechecks parser paths, the relative status URL and object addressing.pnpm validate:agent-generation, which ships with the agent starter, checks publishing and the players. Neither calls a provider.
Troubleshooting
The task fails although the provider accepted it
The submit response's status is in no Status mapping list, or Status path points at the wrong field. The task's error names the unmapped status. Add it under Queued or Running.
The task stays running
Check that Upstream task ID path reads the ID from the submit response (use an index when the provider returns an array), that polling is enabled with a Status URL template, and that background jobs run. /setup warns when five minutes pass without a round.
The task succeeds but no file is stored
The task's events show Generated output storage import skipped. with the reason for each URL. A 403 from an S3-compatible provider usually means the endpoint style does not match the table above.
The agent's artifact says it could not be loaded
The published path was not the saveTo manifest of a generation in this run, or the kind does not match the file. Publish a clip as video and a picture as image.