Runway Video Prompt Extractor: Reference Clips to Renderable Prompts

Runway has spent years building tools around how working creators actually direct shots, and its models reward prompts written the same way: short, physical, and often only half the input — because with a reference image or clip alongside the text, you should not spend your word budget describing what the model can already see.

Runway’s prompting dialect

• Economy. One or two sentences outperform a paragraph; the model is tuned on tight production descriptions.
• Action over adjectives. Name what happens, then how it is shot.
• Explicit camera control where the product exposes it (movement and framing controls, motion brush style tools) — meaning the prompt can stop carrying that job.
• Reference-aware. Where a still or clip is supplied as conditioning, the text should describe the change, not restate the picture.

Runway’s positioning is explicitly toward professional post-production workflows, which is why its prompting style differs from the parameter-list dialect used by several short-form models.

Extract and compress

Reference clip into SocialToPrompt, take the structured fields — subject, camera, lighting, pacing, style — then compress hard. The 27-platform link support plus MP4/MOV/WebM upload means any reference works; the run takes the stated 20–40 seconds and returns something typically 50–70 words, which for Runway is usually twice as long as it needs.

Extracted:  A barista tamps espresso at a marble counter in a bright specialty café, slow push-in
            from chest height, soft window light from camera-left, warm neutral grade, shallow
            depth of field, steam visible, 6 seconds, single continuous shot.
Runway shape: Barista tamping espresso, shoulders turning into frame. Slow push-in, chest height.
            Keep the window light and grade from the reference image. Ends with the tamper lifted.

Half the description moved into the reference asset, and the text now says only what the picture cannot: the action and the camera move.

Camera and control language

Whatever the interface exposes as a control should not be duplicated in prose — that is the classic double-instruction conflict, and the result is usually a mushy average of the two. Rule of thumb: controls for geometry, text for behaviour. So camera path, zoom and framing go in the control surface; the subject’s action, the physical cause, and the moment of change go in the prompt.

For the underlying vocabulary — the difference between a dolly and a zoom, or a pan and a track — see extracting camera movement from a video. Verifying the extractor’s camera verb against the source matters more here than anywhere, because if the control and the text disagree, one silently wins.

Using the clip as a reference, not just a prompt

Three ways a reference clip does work a prompt cannot, and how to exploit each:

What you want	Prompt carries	Reference carries
A look / grade	Nothing, or one palette word	Frames, entirely
A camera path	The verb	The speed and easing
A subject’s consistency	A short noun	A still from the clip
An action beat	All of it	Nothing — text only

This is also the workflow that saves the most words: extract once so you understand the shot, then hand the model the picture and describe only the change.

Iteration habits

Short-prompt iteration is cheap and fast, so the loop is: lock the reference, change one clause of behaviour, render, compare. Keep the extracted full-length version in your .txt as an annotation of the shot — it is the record of what the reference actually contained, which is useful the day you want to reuse the mechanism on a different model with a different dialect (Seedance, Kling, Veo, Sora).

The general method for reducing a reference to transferable mechanism is video prompt reverse engineering, and the multi-shot version is extracting shot lists and prompts. Access note: generation itself sits under Runway’s product and API terms, so a portable prompt is a useful hedge against any single platform’s pricing or policy change.

Frequently asked questions

How should a prompt for Runway differ from other models?

Shorter, and less descriptive of what a reference already shows. Runway's dialect favours one or two tight sentences naming the action and the camera behaviour, with look and geometry handled by the supplied image or clip and by its own controls.

Should I describe the look if I attach a reference?

No, that is the most common mistake. If a still or clip is conditioning the generation, describe only what should change: the action, the timing, the direction of movement. Restating the picture in words competes with it and usually weakens both.

Where do camera controls and prompt text conflict?

When you write geometry in prose that the interface also exposes as a control — zoom amount, pan direction, framing. Use the control for geometry and the text for behaviour, otherwise the two instructions average out into mush.

Do extracted prompts work across different generators?

As a source of truth, yes; as literal text, no. Each family has a dialect: parameter clauses for Seedance and Kling, prose for Sora, compressed behaviour for Runway. Keep the extraction in your notes and rewrite per destination.