How I Made This AI Video

What we're making

A short AI-generated cinematic video built from a strong starting frame and then animated using a detailed motion prompt.

The basic workflow is:

What you'll need

Don't get stuck trying to use the “perfect” stack. The idea, starting image, motion, and storytelling matter more than having every tool.

01. Start With One Clear Visual Idea

Before touching the prompt, decide what the viewer should understand in the first second.

For this style of video, I try to describe the shot as one sentence before adding technical details.

Example:

“Two neighboring houses with completely opposite personalities sitting side by side on the same quiet suburban street.”

That becomes the foundation.

Only after the visual idea is clear do we add:

• environment
• lighting
• composition
• lens
• textures
• characters
• mood
• continuity details

02. Build the Starting Frame

The first frame does a huge amount of the work.

Video models are much more predictable when you give them a strong composition before asking them to create motion.

Photorealistic cinematic establishing shot of two neighboring suburban houses positioned directly beside one another, clearly visible within the same composition. The two homes have dramatically contrasting personalities and visual design while still existing naturally within the same believable neighborhood.

Wide eye-level composition from across the residential street, both houses fully readable, realistic sidewalks, driveway details, landscaping, trees, parked neighborhood elements and subtle lived-in imperfections.

Natural daylight, realistic architectural materials, believable scale and perspective, detailed windows, roofing, siding, concrete and vegetation. The environment should feel like a real photograph rather than an illustration or exaggerated 3D render.

Documentary-style realism, restrained cinematic composition, realistic dynamic range, natural shadows, fine environmental texture, subtle depth, physically believable lighting, no text, no logos, no warped architecture, no duplicate objects, no surreal geometry.

03. Change the Details

This is where I wouldn't recommend simply copying my image.

Keep the structure, but change something meaningful.

Try changing:

• location
• architecture
• colors
• weather
• time of day
• season
• characters
• camera angle
• emotional tone

The goal isn't to recreate somebody else's frame perfectly.

Use the prompt as scaffolding and push the idea somewhere that feels like you.

Reference the technique. Change the story.

04. Bring the Frame Into Higgsfield

Once the starting image feels right, bring it into your video generator.

For this workflow, Higgsfield makes it convenient to experiment with different video models without rebuilding the entire workflow somewhere else.

Open Higgsfield ↗

When working image-to-video, I describe four things separately:

05. Write Motion, Not Another Image Description

One of the biggest mistakes I made when I started was spending most of the video prompt redescribing the image.

The model can already see the image.

Use the video prompt to explain what happens next.

One continuous realistic cinematic shot.

The neighborhood remains visually consistent with the provided starting frame. Preserve the architecture, house positions, street layout, landscaping, proportions, lighting direction and overall identity of the scene.

Begin with a calm establishing composition.

Subtle natural environmental movement: tree branches and small leaves move gently in the breeze, distant foliage shifts naturally, soft atmospheric movement gives the environment life.

Maintain physically realistic motion and believable environmental physics.

Camera movement is extremely controlled: a slow subtle cinematic push forward toward the two houses with minimal lateral drift. Camera remains at approximately human eye level. No sudden pan, no dramatic orbit, no artificial drone movement.

Preserve the geometry and identity of both houses throughout the entire shot.

Natural photographic motion blur only when appropriate.

No morphing architecture.
No melting structures.
No changing windows.
No disappearing objects.
No duplicated objects.
No sudden camera jumps.
No random people appearing.
No text.
No logos.
No surreal transformations.
No artificial time-lapse.
No unnecessary exaggerated movement.

Photorealistic live-action appearance.

Why This Prompt Works

Preserve identity

Telling the model what needs to stay consistent can be just as important as describing the movement.

Separate movement

Subject, environment, and camera motion are easier to control when they aren't mashed into one vague sentence.

Keep the camera simple

Complex camera choreography introduces more opportunities for the generation to break.

Use negative constraints strategically

Don't create an enormous negative prompt. Call out the mistakes that would actually ruin this particular shot.

06. Generate More Than Once

AI video is still probabilistic.

Even a strong prompt can produce several completely different interpretations.

Instead of endlessly rewriting a good prompt after one bad generation:

1. Run a few variations.
2. Compare motion consistency.
3. Look closely at the first and last frames.
4. Choose the generation with the most believable motion.
5. Only rewrite the prompt if the same problem keeps happening.

07. The AI Generation Isn't the Finished Video

Once I have the clips I like, I bring them into an editor.

My finishing process is intentionally simple:

• choose the strongest take
• trim awkward beginnings or endings
• tighten pacing
• add sound design
• add music only when it supports the story
• make small color adjustments when necessary
• export for the platform I'm posting on

AI gives you footage.

Editing gives it intention.

If Your Generation Looks Wrong

Want to Try Making It Yourself?

Hi, I'm Key Lee.

I make things at the intersection of AI, filmmaking, technology, music, and storytelling.

I'm figuring a lot of this out too — and when I find something useful, I try to break it down so you can experiment with it without opening twenty tabs first.

You bring the idea. We'll figure out the rest. 🤍

More Guides