Scene format

A scene is one JSON document: a master size, the sizes it renders at, its assets, shared tokens, styles and components, and a tree of layers. This page is the reference for every field. The ideas behind it are in Concepts, and the tools that write and render it in Tools and replies.

{
  "sizes": ["instagram-square", "iab-skyscraper"],
  "tokens": {"brand": "#D0202E"},
  "layers": [
    {"type": "rect", "width": "fill", "height": "fill", "fill": "#FFF4E0"},
    {"type": "text", "text": "Cold Brew <b>Season</b>", "fontSize": 96, "fontWeight": 800,
     "color": "{{brand}}", "width": "80%", "place": "center", "media": {"tall": {"fontSize": 64}}}
  ]
}

Contents: Conventions · Document · Layers · Layout · Text · Paint · Reuse · Motion · Template files · Validation and limits · Example · Names

Conventions

Field names and values follow what models already know:

1. CSS names and values wherever CSS has the concept, camelCase as in React Native: fontWeight, textAlign, justifyContent, alignItems, gap, padding, borderRadius, filter. Figma words for sizing, pinning and clipping (hug, fill, constraints, clipsContent), SVG for shapes and strokes (fill, stroke, markerEnd), GSAP for motion, HTML media for video, and SwiftUI only where none of those has the concept (firstFit, layoutPriority, minimumScaleFactor).
2. CSS names, keyline's own layout. The names and enum values are CSS's, and colors are any CSS color (rgba(0,0,0,.25), hsl(…)). The layout model is keyline's, documented on this page: close to flexbox and grid where it borrows from them, but not browser-exact.
3. Short forms. A compound field has a one-value short form: fill: "#fff", padding: 24, borderRadius: 12, stroke: "#000". CSS written CSS's way reads as meant: padding: "48px 40px" (one to four values) and paddingTop…paddingLeft, "12px" for gap, borderRadius and fontSize, a box-shadow string as shadow, border: "2px solid #fff" as stroke, and gradient stops by position. A stack is a frame laid out as a column, a rectangle a rect, and layout: "row" (or "column", "horizontal", "vertical") is flexDirection. A frame with padding, gap or alignment but no direction is a column, as a padded <div> stacks its children, when none of its children is placed by x, y or place, no two fill it (layers over each other) and it has no style; otherwise those fields need a flexDirection.
4. Every field has a default, and defaults are omitted in what the agent sends and in what the server stores.
5. Unknown fields are rejected, with the nearest known name suggested.

Value types

The tables below use these types.

Resolution order

A layer's final values are built in this order; each step wins over the one before:

1. Tokens, when the layer is written: every {{name}} becomes the token's value.
2. Components, for a use layer: {{prop}} is filled from the instance's props (a prop wins over a token of the same name; the component's other {{name}}s are tokens), and the use layer's own fields replace the component root's, field by field.
3. Styles, in the order listed; a later style wins. The layer's own fields win over all styles.
4. media, per size: aspect classes broadest first, then the size id.

media and layer_update's set merge like a JSON merge patch: nested objects (an object enter, constraints) merge field by field, while lists (fill, ranges, children) and plain values are replaced whole, and null resets a field. Styles and a use layer's fields replace whole top-level fields, except media: a style's media and the layer's own merge, size by size and field by field, the layer's winning.

Document

Sizes

A size is an object, a preset name, or "WxH" (its id is that string: "300x600").

Presets

A preset's id is its name.

Aspect classes

With r = width ÷ height of the size:

A size is in one of the first three and may also be wide or tall.

Layers

Every layer has a type, the common fields, and its type's own fields.

Common fields

In a stack, children also take the stack child fields; in a grid, the grid child fields.

frame

With neither flexDirection nor a grid template, children are placed freely, by their own position.

{"type": "shot", "duration": 3, "transition": "fade", …} is a frame that plays in turn with the other shots (Shots and transitions).

text

See Text.

image

SVGs are drawn at their drawn size, so they stay sharp.

video

A clip added with asset_add, drawn like an image and under any layers above it: titles, captions, logos. Decoding it needs ffmpeg (Tools and replies).

{"type": "video", "asset": "beach", "width": "fill", "height": "fill", "trimStart": 2, "playbackRate": 0.5, "muted": true}

A clip's sound plays with its pictures: from trimStart, at playbackRate, looping with it, and only while its shot is on; several clips' sounds are mixed. Stills (time, or a scene at rest) show the clip's frame at that moment.

Shapes

A named shape is a real path, so every paint applies: a photo in a blob, a gradient ribbon, a dashed speech bubble.

icon

An icon is 24 px tall unless sized.

spacer

firstFit

Draws the first of its children that fits its box at this size with nothing wrong anywhere inside it: no text overflowing, truncated or shrunk to fit, no stack squeezed. When none fits, it draws the last. Typical uses: a long and a short headline, a row CTA and a stacked CTA. scene_describe shows what was chosen at each size (→ short).

use

See Components.

Layout

A frame lays out its children in one of three ways: free (each child's position and constraints), stack (a row or column, CSS flexbox) or grid (CSS grid). Children of a stack or grid ignore x, y and constraints unless they set position: "absolute", which places them like a free child (a badge over a card's corner).

Sizing

In a grid, a child with a px size keeps it and sits at its cell's start; otherwise it fills its cell. Min and max clamps apply last.

Free layout

Free children use the parent's whole box; its padding doesn't apply. A "N%" x or y is that share of the parent at every size and moves the layer without resizing it. A free frame sized by a stack with fill or % places its children on the box the stack gave it.

Stacks

A frame with flexDirection is a stack, like CSS flexbox (display: "flex" alone makes a row, as in CSS):

{"type": "frame", "flexDirection": "row", "gap": 16, "padding": [24, 32], "alignItems": "center", "justifyContent": "space-between", "children": ["…"]}

padding, gap, justifyContent, alignItems and flexWrap on a frame without flexDirection or a grid template are an error that says so.

As in CSS, text and frames in a column never shrink below their content's height, and text without a width wraps at a column's width rather than run past it; a stack whose children don't fit even then is reported as !overflow needs W×H, the size it needs.

Stack children

Grids

A frame with a grid template is a grid, like CSS grid:

{"type": "frame", "gridTemplateColumns": "2fr 1fr", "gridTemplateRows": "2fr 1fr", "gridTemplateAreas": ["photo side", "cta side"], "gap": 24, "children": ["…"]}

A size can rearrange the whole grid by changing only its templates in media. A grid whose tracks don't fit it is reported as !overflow needs W×H, as a stack is.

Grid children

Per size

media changes a layer's fields for some sizes: "media": {"sky": {"fontSize": 20}, "tall": {"hidden": true}}. Its keys are size ids or aspect classes, applied broadest first: landscape/square/portrait, then wide/tall, then the size id. Its values merge into the layer (Resolution order). media can't change a layer's id, type, children, media or style; a key that isn't a size or class is an error.

Text

Fields

stroke outlines the letters (fill: [] with a stroke makes outlined text) and shadow follows their shapes.

Fitting

The box decides how text fits:

• Width and height: the font shrinks until the text fits, down to minimumScaleFactor, then ends with an ellipsis.
• Width only: the text wraps and the box grows down. In a column, alignItems: stretch gives text the column's width, so it wraps.
• Neither: one line, as wide as the text; in a column narrower than that, it wraps at the column's width.

Text that is cut is reported as !truncated needs W×H, with the box it needs; nothing changes silently.

Inline markup

Models miscount character offsets, so text takes a small HTML subset instead:

{"type": "text", "style": "h1", "text": "Proven <accent>RESULTS</accent> for <accent>WILLOWMERE</accent> Families"}
{"type": "text", "text": "<s>$49</s> <b>$29</b><sup>99</sup> today"}

• Tags: <b> (or <strong>), <i> (or <em>), <u>, <s>, <sup>, <sub>, <br>, a style's name as a tag (<accent>), and <span style="…"> with CSS: color, font-weight, font-style, font-size, font-family, text-decoration, background-color (a highlight). Values can be tokens (style="color:{{red}}"). A style's name as a tag carries the style's color, fontWeight, fontStyle, fontSize, fontFamily, textDecoration and highlight (a brush too).
• A < that doesn't open a known tag is text; <, >, & and " are entities.
• A <span> whose style doesn't parse is an error, not text.

Ranges

ranges styles parts of the displayed text by character offset; markup is usually easier.

Paint

Frames, shapes, images, text and icons take the same paint fields.

Fills

fill is one paint or a list, bottom to top; [] fills nothing (outlined text, for example). Every paint takes opacity (0–1, default 1) and blendMode (default normal).

An image fill works on any shape: a photo in a circle is {"type": "ellipse", "fill": {"image": "photo"}}.

Gradients

Patterns and grain

rays are hard-edged sectors, so over a photo they cut across its detail and read much stronger than over a flat color: keep them to about 3–4% there ("color": "#FFFFFF0A" with blendMode: "screen"), or put them on the flat areas.

Filters

filter on an image, video or image fill: CSS filter functions on their CSS scales:

Strokes

stroke is one stroke or a list, SVG-style; "#000" is a 1 px black stroke.

Shadows

shadow is one shadow or a list, like CSS box-shadow. A shadow follows the layer's shape: it hugs a cutout photo or the letters of a text. A glow is a shadow at x: 0, y: 0; a hard offset shadow has blur: 0.

Blur

blur (px) blurs the layer; backdropBlur (px) blurs what's behind it within its shape (frosted glass).

Masks

Edges

Blend modes

normal, multiply, screen, overlay, darken, lighten, color-dodge, color-burn, hard-light, soft-light, difference, exclusion, hue, saturation, color, luminosity.

Reuse

Tokens

tokens holds named values, used as {{name}} (a letter or _, then letters, digits, _, . and -), as in Mustache and Handlebars:

"tokens": {"brand": "#D0202E", "h1": 64, "name": "Mia", "photo": "cat"}
{"type": "text", "text": "Meet {{name}}, 7 months old", "fontSize": "{{h1}}", "color": "{{brand}}"}
{"type": "image", "asset": "{{photo}}"}
{"type": "text", "text": "Proven <span style=\"color:{{brand}}\">RESULTS</span>"}

• A string that is only {{name}} takes the token's value as it is: a number stays a number.
• Inside a string (a sentence, a markup attribute), the value is spliced into the text; a token spliced in is a string, number or boolean.
• The server remembers which fields came from which token, and a sentence's template, so changing a token in layer_update (or a render's rows) changes every field bound to it; setting such a field to a value of its own unbinds it.
• A string without {{name}} is never touched, so "$29" and "{curly}" stay text. An unknown name is an error that lists the tokens there are.
• {{n}} is the counting number (Counting), never a token.
• A whole value written "$name", for a token that exists, reads as {{name}}. Text with $name in it is left alone (it may be a price), and the edit's reply adds hint: did you mean {{name}}?.

Styles

styles hold any layer fields, media included: a card style can carry fill, borderRadius and shadow. A layer with its own media keeps the style's too: {"style": "ink", "media": {"story": {"fontSize": 40}}} still gets ink's a4-portrait color, and where both set a field for one size, the layer's wins. A layer's style takes one name or a list; a later style wins where they overlap, and the layer's own fields win over all of them. Changing a style through layer_update changes every layer that uses it. A style can't set id, text or children; a type in it is dropped.

Components

"components": {
  "candidate": {"type": "frame", "flexDirection": "column", "gap": 4, "alignItems": "center", "children": [
    {"type": "text", "role": "name", "text": "{{name}}", "style": "name"},
    {"type": "text", "role": "office", "text": "{{office}}", "style": "office"}]}
}

{"type": "use", "id": "c", "component": "candidate", "each": [
  {"name": "Dana Levi", "office": "Mayor"},
  {"name": "Omar Haddad", "office": "Council"}]}

• {{prop}} (or {{ prop }}) in any string of a component is filled from the instance's props, as in Mustache; a string that is only {{prop}} takes the value as is (a number stays a number). {{n}} stays the counting number, never a prop. A prop wins over a token of the same name; any other {{name}} in the component is a token.
• The use layer's own fields (width, constraints, media…) apply to each instance's root.
• Instances stay linked: changing the component changes every instance. Their layers are named by the use id, the instance number and the inner layer's id or role, e.g. c.1.name, in replies. To change one, target the component ({"component": "candidate", "role": "name"}), or detach the use layer into plain layers.
• Components may place other components, up to 8 levels deep.

Motion

A scene with a duration, or made of shots, moves. Only fields that don't change layout animate, so the layout is the same at every moment and every check holds throughout. A scene without motion fields is drawn at rest. A still of a moving scene (PNG, JPEG, WebP, PDF, without time) is also at rest: each layer as written, before its tracks (a "scale": 1.12 written for a pan's room shows at 1.12), except draw and count, which show where they end. Output formats are in Tools and replies.

Scene timing

Enter and exit

{"type": "text", "text": "…", "enter": "fade-up"}
{"type": "frame", "enter": {"effect": "pop", "delay": 1.2, "ease": "back.out"}, "exit": {"effect": "fade", "delay": 7}}

enter and exit take an effect name, or an object. A layer is hidden before it enters and gone after it exits. A directional effect is named for the way the layer moves: fade-left moves left into place.

Keyframes

animate sets values over time, GSAP-style: one track or a list of them.

"animate": {"scale": [1, 1.06, 1], "duration": 1.6, "repeat": -1, "ease": "sine.inOut"}
"animate": {"rotate": {"from": "random(-90, 90)"}, "translate": {"from": [0, -80]}, "duration": 0.8, "ease": "back.out"}

A number may be "random(lo, hi)", as in GSAP: each target (each layer, or each piece of split text) gets its own value, seeded from its id, the same on every render. Values hold before a track starts and after it ends. Layout fields (width, fontSize, text, padding…) can't animate; to make something grow, animate scale.

Drawing strokes

draw draws a share of a layer's strokes, like After Effects' trim paths: a progress ring, a line or underline drawing itself, line art signing itself.

{"type": "ellipse", "width": 120, "height": 120, "stroke": {"width": 10, "color": "#0AE448", "cap": "round"}, "animate": {"draw": [0, 0.72], "duration": 1.2, "ease": "power2.out"}}
{"type": "path", "d": "M0 40 C 40 0, 80 80, 120 40", "stroke": "#000", "animate": {"draw": {"from": 0}, "duration": 2}}

It works on every stroke: paths, shapes, lines, ellipses and text outlines. A line draws from its start to its end, an ellipse clockwise from the top, a path from its first point, and text letter by letter; dashes and end markers follow the drawn part, and an ellipse's dashes start at the top too. Stills at a time show the share drawn then; at rest, the share where the track ends (all of it without a draw track).

Counting

count puts a number that counts into a text layer, wherever its text says {{n}}:

{"type": "text", "text": "{{n}}+ teams", "animate": {"count": [0, 1250], "separator": ",", "duration": 1.5, "ease": "power2.out"}}

The text is measured with its widest value (usually the last), so its box holds still and nothing around it moves while the number changes; digits use the font's tabular figures when it has them. At rest, and in scene_describe, the text shows that value. A text that counts needs {{n}} in it.

Easing

GSAP's names: none, power1 … power4, sine, expo, circ, back, elastic, bounce, each with .in, .out or .inOut (a family alone is .out), and steps(n). CSS's ease, ease-in, ease-out and ease-in-out, and smooth, snappy and bouncy, read as the nearest of those.

Stagger and split

{
  "type": "text",
  "text": "Animate Anything",
  "fontSize": 96,
  "split": "chars",
  "stagger": 0.05,
  "animate": {
    "translate": {
      "from": [
        "random(-400, 400)",
        "random(-250, 250)"
      ]
    },
    "rotate": {
      "from": "random(-180, 180)"
    },
    "opacity": {
      "from": 0
    },
    "duration": 1.1,
    "ease": "back.out"
  }
}

Frames clip their children, animated ones included: give a frame clipsContent: false when its children move beyond its edges.

Soundtrack

audio puts music (or any sound) under an MP4 or WebM, mixed with the clips' own sound. It starts with the video and is cut to its length.

{"asset": "song", "volume": 0.8, "trimStart": 12, "fadeIn": 0.5, "fadeOut": 1.5}

Set it with layer_update's {target: {scene: true}, set: {audio: …}}; music, soundtrack or an asset id alone read as it, and null removes it. A sound shorter than the video ends in silence. Animated PNG and GIF have no sound; render's muted leaves it out of video.

Shots and transitions

A layer of type: "shot" is a shot: a full-size frame that plays in turn with the other shots instead of stacking. Times inside a shot (enter, animate, a clip's delay) count from the shot's own start. Layers that aren't shots, such as a logo or a caption bar, stay on across all of them. At rest, the first shot shows. Every shot is checked at every size; to keep one out of a size that never plays it (an A4 page of a video's first shot), hide it there: "media": {"a4-portrait": {"hidden": true}}. Its checks and facts then skip that size.

{"id": "s1", "type": "shot", "duration": 3, "children": ["…"]}
{"id": "s2", "type": "shot", "duration": 3, "transition": "push-left", "children": ["…"]}

Each shot starts where the one before ends minus its transition. The scene's length is where the last shot ends unless duration says otherwise; a longer duration holds the last shot to the end. A transition can't be longer than either shot it joins.

Template files

A template is a scene file that scene_create loads by URL or path (Tools and replies). It has a scene's fields, and its assets name files instead of hashes: a URL, or a path relative to the template. Its tokens are its variables.

{
  "sizes": ["instagram-square", "iab-medium-rectangle"],
  "tokens": {"headline": "Spring sale", "price": "$29", "accent": "#D0202E"},
  "assets": {"photo": "photo.jpg", "logo": "https://example.com/logo.svg"},
  "layers": [
    {"type": "image", "asset": "photo", "width": "fill", "height": "fill"},
    {"type": "text", "text": "{{headline}}", "fontSize": 64, "fontWeight": 800, "color": "{{accent}}", "place": "center"}
  ]
}

A scene keyline saved (its assets by sha256) loads as a template too.

Validation and limits

A change that breaks any of these rules is refused whole, with a one-line error (Tools and replies); what the layout does at each size (overflow, clipping, contrast) is never refused, but reported as problems.

• Unknown fields, in layers and in every object inside them.
• Values of the wrong type or out of range: opacity 0–1, fontWeight 100–900 in 100s, minimumScaleFactor above 0 and at most 1, flexGrow ≥ 0, polygon sides ≥ 3, video playbackRate 0.01–100, sizes at least 1 px with scale above 0.
• Values whose unit was likely mistaken: a lineHeight above 4 (px, not × fontSize), a motion duration of 100 or more that's longer than the scene (ms, not seconds).
• Layout fields on a frame that isn't a stack or grid (padding without flexDirection), and margin without place.
• An unknown token, style, component, asset, parent frame or media key; a duplicate layer id.
• Shots that aren't top level, have no positive duration, or whose transition is longer than a shot it joins; split on anything but text; count on a layer whose text has no {{n}}; draw or count on split text.
• An audio asset without sound, or a negative volume, trimStart, fadeIn or fadeOut; a sound used as an image or video.

Asset and file limits are in Tools and replies.

Example

The reference ad from the end-to-end tests, in one layer_add: tokens, styles, two components placed with each, and a layout that adapts to a portrait post, a wide banner and a skyscraper without per-size positions.

{
  "tokens": {
    "navy": "#1B2A5C",
    "red": "#D0202E",
    "grey": "#6B7280"
  },
  "styles": {
    "accent": {
      "color": "{{red}}"
    },
    "name": {
      "fontSize": 36,
      "fontWeight": 700,
      "color": "{{navy}}",
      "textAlign": "center"
    },
    "office": {
      "fontSize": 30,
      "fontWeight": 500,
      "color": "{{grey}}",
      "textAlign": "center"
    }
  },
  "components": {
    "candidate": {
      "type": "frame",
      "flexDirection": "column",
      "gap": 4,
      "alignItems": "center",
      "children": [
        {
          "type": "text",
          "role": "name",
          "text": "{{name}}",
          "style": "name"
        },
        {
          "type": "text",
          "role": "office",
          "text": "{{office}}",
          "style": "office"
        }
      ]
    },
    "step": {
      "type": "frame",
      "width": "fill",
      "flexDirection": "row",
      "gap": 16,
      "alignItems": "center",
      "children": [
        {
          "type": "image",
          "asset": "check",
          "width": 40,
          "height": 40
        },
        {
          "type": "text",
          "role": "step",
          "text": "{{text}}",
          "fontSize": 32,
          "fontWeight": 500,
          "color": "{{navy}}",
          "width": "fill"
        }
      ]
    }
  },
  "layers": [
    {
      "id": "page",
      "type": "frame",
      "width": "fill",
      "height": "fill",
      "flexDirection": "column",
      "gap": 28,
      "padding": [
        48,
        0
      ],
      "alignItems": "flex-start",
      "children": [
        {
          "id": "headline",
          "type": "text",
          "width": "fill",
          "padding": [
            0,
            40
          ],
          "fontSize": 64,
          "fontWeight": 800,
          "color": "{{navy}}",
          "textAlign": "center",
          "textWrap": "balance",
          "text": "Proven <accent>RESULTS</accent> for <accent>WILLOWMERE</accent> Families"
        },
        {
          "id": "photo",
          "type": "image",
          "asset": "photo",
          "width": "fill",
          "height": "fill",
          "minHeight": 120
        },
        {
          "id": "cands",
          "type": "frame",
          "width": "fill",
          "flexDirection": [
            "row",
            "column"
          ],
          "gap": 12,
          "alignItems": "flex-start",
          "justifyContent": "space-evenly",
          "children": [
            {
              "id": "c",
              "type": "use",
              "component": "candidate",
              "each": [
                {
                  "name": "Dana Levi",
                  "office": "Mayor"
                },
                {
                  "name": "Omar Haddad",
                  "office": "Council"
                },
                {
                  "name": "Ruth Cohen",
                  "office": "Council"
                }
              ]
            }
          ]
        },
        {
          "id": "cta",
          "type": "frame",
          "width": "fill",
          "fill": "{{red}}",
          "flexDirection": "row",
          "gap": 16,
          "padding": 22,
          "alignItems": "center",
          "justifyContent": "center",
          "children": [
            {
              "type": "icon",
              "name": "mail",
              "color": "#FFFFFF",
              "width": 48,
              "height": 48
            },
            {
              "type": "text",
              "text": "VOTE BY MAIL",
              "fontSize": 48,
              "fontWeight": 800,
              "color": "#FFFFFF"
            }
          ]
        },
        {
          "id": "steps",
          "type": "frame",
          "width": "fill",
          "flexDirection": "column",
          "gap": 12,
          "padding": [
            0,
            60
          ],
          "alignItems": "flex-start",
          "children": [
            {
              "id": "s",
              "type": "use",
              "component": "step",
              "each": [
                {
                  "text": "Request your ballot by October 20"
                },
                {
                  "text": "Fill it out at home"
                },
                {
                  "text": "Mail it back by November 3"
                }
              ]
            }
          ]
        },
        {
          "id": "footer",
          "type": "text",
          "width": "fill",
          "text": "Paid for by Willowmere Forward · willowmereforward.org",
          "fontSize": 20,
          "color": "{{grey}}",
          "textAlign": "center",
          "media": {
            "tall": {
              "hidden": true
            }
          }
        }
      ]
    }
  ]
}

The photo takes whatever height is left at each size, the candidates switch to a column where a row doesn't fit, and the footer is dropped on tall sizes.

Appendix: names

This page is built from docs/scene.md; also as Markdown.