Video generation

There are two routes. Generate a clip directly (available by default), or enable the Movie plugin for full narrative production (opt-in). Both ultimately call the same video tool to render frames; Movie adds an orchestration layer and a timeline on top.

The three actions of the `video` tool

video belongs to profy-creative (user_selectable: false), so it is available in every conversation.

Action	Purpose	Consumes credits
`generate`	Text-to-video or image-to-video, produces one clip	Yes
`generate_batch`	Generate several clips concurrently (max concurrency 20)	Yes, billed per clip
`read`	Understand an existing video, returns a text description	Billed as tokens on the multimodal model

Parameters for generate

Parameter	Type	Default	Meaning
`prompt`	string	`""`	Description of the shot
`resolution`	string	`"720p"`	Must be a tier the current model supports
`duration`	float	`5.0`	Length in seconds
`reference_image_url`	string	`""`	First-frame reference; supplying it makes this image-to-video

The default duration is 5 seconds. Say nothing about length and you get 5 seconds — ask for “a 10-second clip” explicitly if you want more.

Parameters are pre-validated against model capabilities

Each video model declares which parameter tiers it supports. When your resolution or duration is outside that model’s allowed set, the tool does not call the provider and fail — it returns param_unsupported immediately:

The requested resolution '4k' is not supported by the current video model.
Supported values: 480p, 720p, 1080p.
Please ask the user which value they'd like to use, or suggest the closest supported option.

The expert will come back and ask you which tier to use rather than deciding for you. This is the good outcome — without validation you would wait out a full round only to get a vague provider-side error. Separately: if the model’s configuration sets a default for a parameter, that default overrides whatever you and the expert agreed on. If you asked for 1080p and got 720p, check the model configuration first.

Generating directly

Describe the shot in the conversation:

Generate a 10-second sunset beach timelapse, slow lateral camera move,
warm color grade, 1080p.

For image-to-video, send the reference image first, then:

Turn this photo into a slow push-in, 5 seconds. Keep the person still,
only the clouds move.

Be explicit about camera motion (push-in / pan / orbit / locked off) and speed (slow / fast). Those two are what distinguishes video from a still image, and they matter far more than piling on adjectives.

Video understanding (read)

read uses the platform-configured multimodal model, not a video generation model:

Watch this video and tell me what it covers — pull out any figures mentioned.

Constraints: the description is capped at 2048 tokens per analysis, with a 120-second HTTP timeout. If no default multimodal model is configured in the backend:

No multimodal model configured. Please set a default multimodal model in the admin panel.

What happens while it runs

Same async pipeline as music generation:

Stage	Value
Total wait budget per generation	600 seconds (10 minutes)
Progress poll interval	3 seconds
Progress curve	`1 - e^(-t/60)`, capped at 95% (an estimate)
Automatic provider retries	Up to 3 attempts
Fast failures only	A failure that took ≥ 60 seconds is not retried
`generate_batch` concurrency	20

You can send the job to the background while it runs (not on desktop — no local Redis channel).

Movie: seven-phase orchestration

Movie is user_selectable: true and exclusive: true — you enable it in plugin settings, and once enabled it takes over the conversation so other optional plugins stay inactive.

The movie tool itself calls no external API. It emits structured data that drives frontend rendering; the frames still come from the video tool. Its seven actions are the seven phases:

Phase	Action	Output
1. Brief	`brief`	Structured summary of topic, audience, length, style
2. Script	`script`	Narrative structure and dialogue
3. Storyboard	`storyboard`	Shot list; also emits the `video_storyboard_init` event that drives the panel
4. Visual strategy	`strategy`	Style, grade, consistency constraints
5. Assemble	`assemble`	Clips laid out in storyboard order
6. Captions	`caption`	Subtitle data from ASR
7. Export	`export`	Export specification envelope (no actual encoding)

The storyboard phase assigns stable IDs to every element / segment / shot / audio item (prefixes sb- / el- / seg- / shot-). Later timeline operations address clips by those IDs.

Timeline editing

The timeline_edit tool lets the expert modify the timeline on instruction. Ten actions:

Action	Required parameters
`insert_clip`	trackId, clipId, startTime, duration, mediaUrl, type
`remove_clip`	trackId, clipId
`trim`	trackId, clipId, trimStart, trimEnd
`split`	trackId, clipId, splitTime
`reorder`	trackId, clipIds
`add_track`	type, name
`set_volume`	trackId, clipId, volume
`add_caption`	trackId, text, startTime, duration, style
`export`	format, quality
`get_state`	none

Allowed values: clip types video / audio / image; track types video / audio / caption / music; export formats mp4 / mov; export quality high / medium / low. Invalid parameters produce an error listing the valid values — the tool never silently substitutes a near match. So you can say “cut the back half of shot 3”, “drop the music to 30% volume”, “export mp4 at high quality”, and the expert translates that into the corresponding timeline_edit calls.

Failures and what to do

Symptom	Cause	Fix
`No video model selected`	No video model in the selector	Select one and retry
`param_unsupported`	Resolution/duration outside the model’s allowed set	Pick one of the values listed in the error
Asked for 1080p, got 720p	The model configuration sets a `default` that overrides the request	Use a model that does not pin that parameter
`This model is currently unavailable...`	Provider credential/config problem (raw errors mentioning keys are replaced)	Switch models
`Media generation timed out after 600s`	Exceeded the total wait budget	Shorten the clip, lower the resolution, or switch models
`No multimodal model configured`	`read` has no multimodal model to use	Ask an administrator to configure one
Only got 5 seconds	That is the `duration` default	State the length explicitly

Billing

The billing unit is set by the model’s pricing configuration in the backend; what the tool reports is only a hint. The common case is duration_second: ceil(output_seconds × per_second_rate) — billed on the length of the output, not the time spent generating it (waiting 3 minutes for a 5-second clip costs 5 seconds’ worth). Some models use cost_matrix, where the combination of resolution, frame rate, and duration tier sets the price; the tool reports the parameters it actually used so the right tier is matched.

Failed generations are not billed (usage is only reported for non-failed results)
generate_batch bills per clip — 20 clips is 20 charges
At most 100 creative usage events are settled per conversation stream
In Movie, movie and timeline_edit incur no media cost themselves; the cost comes from the video / music / image calls they orchestrate

Per-model rates are operations data — the model selector is authoritative. The algorithms are fixed; see Billing formulas.

A full Movie run makes many video generation calls — one per shot. Settle the shot count and per-shot length before generation starts: ten 5-second shots is ten charges.

Keep reading

Music creation

The same tool Movie calls during scoring

Image generation

Where storyboard reference frames usually come from

Billing formulas

Full algorithms for duration_second and cost_matrix

Plugin catalog

Which plugins are exclusive

Get Started

Core Concepts

Guides

Platform & Account

Deep Dive

Video generation

The three actions of the `video` tool

Parameters for generate

Parameters are pre-validated against model capabilities

Generating directly

Video understanding (read)

What happens while it runs

Movie: seven-phase orchestration

Timeline editing

Failures and what to do

Billing

Keep reading

Music creation

Image generation

Billing formulas

Plugin catalog

​The three actions of the video tool

​Parameters for generate

​Parameters are pre-validated against model capabilities

​Generating directly

​Video understanding (read)

​What happens while it runs

​Movie: seven-phase orchestration

​Timeline editing

​Failures and what to do

​Billing

​Keep reading

Music creation

Image generation

Billing formulas

Plugin catalog

The three actions of the `video` tool

Parameters for generate

Parameters are pre-validated against model capabilities

Generating directly

Video understanding (read)

What happens while it runs

Movie: seven-phase orchestration

Timeline editing

Failures and what to do

Billing

Keep reading