Idea Sketch #4: The Temporal Dimension in Moodboards

This idea grew out of a couple of small, persistent observations I made over the past few years.

One of them came from climbing. I’ve spent some time building a tool called Replay to look at my own climbing videos, mostly just to understand what I was doing wrong. I soon realized how much I depended on being able to move through a video slowly and precisely. A shift in weight, a hesitation, a slightly off trajectory—these things can happen in just a few frames, and if you can’t really feel your way through those frames, they’re easy to miss.

In the browser, that kind of interaction has always felt a bit off. Scrubbing never quite has the same immediacy as it does in something like macOS's QuickTime player. The native HTML <video> element is built for watching, not for inspecting. It works, but it doesn’t invite you to linger or look closely.

The other observation came from how I collect creative references.

They’re scattered everywhere—Pinterest, Instagram Saved, YouTube, and links across different platforms. Pinterest works well for images. Instagram is where many of my interesting motion references live, but they quickly disappear into collections and accounts. Even when I save a video, I’m often interested in only a few seconds of it.

That led me to a broader thought: moodboards are mostly spatial, but motion references can often communicate ideas that static images cannot. When a concept involves sequence—such as animation, transitions, timing, or camera movement—video makes it much easier to understand how the idea unfolds. Yet many creative tools still treat video primarily as an embedded player, rather than as material that can be examined, broken apart, and composed with.

I wanted to explore what might change if video could be broken into moments and arranged alongside other references while still preserving its relationship to the original sequence.

Scope of the experiment

This project is a small interaction probe rather than a complete product. I kept the scope narrow and focused on a few behaviors that seemed most important:

• Moving through video at a fine temporal resolution.
• Pulling out individual frames or short clips.
• Placing those pieces into a shared space.
• Letting them sit alongside images, notes, and connections.

At this stage, I’m mainly interested in whether these interactions feel natural and whether they change how people inspect, compare, and reuse creative references.

Use cases

A temporal moodboard

The first place this shows up is in a kind of moodboard that can hold time-based media. I can drop images, videos, extracted frames, and short clips into a single workspace. Instead of saving an entire video because of one transition, I can isolate that moment and place it next to something else—a still image, another clip, or a note.

I can also loop several clips at once. The result may look a little chaotic, but it makes rhythm, pacing, and movement easier to compare across references. The board holds on to more of what made the original material interesting: the video is neither flattened into a thumbnail nor locked away behind a play button.

A climbing notebook

The second use case is closer to where this project started.

With a climbing video, I can pull out the moments around a crux, sketch the path of movement, and line up a sequence of frames. It starts off messy—just fragments and notes—but over time, it becomes something more structured.

It’s not quite just a moodboard, and it’s not quite just a storyboard either. It combines aspects of a moodboard and a storyboard: a place to collect references, externalize observations, and arrange them into sequences.

How it is built

The project has two main views. The /demo page isolates the FrameFlow component and includes a debug overlay for inspecting scrubbing behavior. The main Storyboard Playground wraps FrameFlow inside media cards on a React Flow canvas:

• FrameFlow $\to$ Media card $\to$ React Flow canvas

At the center of the Storyboard Playground is FrameFlow, a small React component for interacting with video.

The goal was simply to make video feel more like a surface you can explore. You can scrub directly across it, pause on a moment, and pull frames or clips into the surrounding canvas.

FrameFlow uses a hidden HTML video element as its playback and decoding engine, while a visible canvas displays the current frame. It uses requestVideoFrameCallback to detect when the browser presents a decoded frame, then paints that frame onto the canvas.

Because videos do not always expose a reliable frame rate, FrameFlow estimates one by sampling the timing between presented frames. The estimate is needed to translate between playback timestamps and approximate frame positions during scrubbing.

This is not a fully frame-accurate decoding pipeline, but it provides enough temporal precision for the interaction I am exploring.

During scrubbing, it also measures the speed and direction of the pointer. Slow drags advance in small frame-sized steps, while faster drags skip farther through the video. This creates a more responsive interaction without treating every gesture the same way.

FrameFlow exposes playback, frame position, seeking, scrubbing, and frame capture through a React context. The surrounding storyboard uses those controls to support frame extraction, clip selection, looping, and restoring the last viewed position.

Internally, the component has three layers:

Looking forward

Collect from anywhere

Right now, collecting references is still manual. A browser extension could bring back the familiar “Pin It” interaction, allowing people to save images, videos, or even specific moments directly into a workspace while preserving a link to the original source. We can even build a connector layer that imports previous references from Pinterest or Instagram.

Better temporal composition

The current prototype only begins to explore how clips might relate to one another. Synchronized playback, more precise trimming, timestamped annotations, and clearer sequencing could all be useful, but their value should emerge from real creative workflows rather than speculative feature-building.

Visual search

Another major piece I have not explored is making the collection queryable. Tools such as mymind, Pinterest, and newer visual knowledge systems like Dotient suggest that images and videos could be retrieved semantically instead of found by manually browsing a board. A similar search layer could eventually sit behind this prototype.

Composing references for AI generation

Because the board already combines images, videos, extracted frames, annotations, and relationships, it could also serve as a structured input for generative tools. StyleFrame uses arranged keyframes to guide motion, Luma brings references and generated variations together inside project boards, and Morphic combines a creative canvas with workflows for turning selected frames into video and composing scenes into larger stories.

A system like FrameFlow could complement those workflows by helping creators assemble source material more deliberately—for example, extracting movement from a video, pairing it with the composition of an image, annotating a camera direction, and arranging those elements into a sequence. The board could then act as a visual brief for image or video generation.

I believe in a future where we create tools not to automate creativity away, but to give creative people more expressive ways to shape, communicate, and refine their ideas.

Beyond two-dimensional media

A future version might also support 3D objects, spatial scenes, or audio, allowing each medium to preserve the qualities that make it useful rather than reducing everything to a static preview.

FrameFlow is still an early prototype, but it has already helped me think more clearly about how time-based media might fit into a creative workspace.

References

1. "Perform efficient per-video-frame operations" by Thomas Steiner.