Video Embeddings Fresh 🌱

Embed video content directly into vector space for semantic search, classification, and retrieval.

Overview

Specifications

Parameter	Value
Model	gemini-embedding-2-preview
Max duration	128 seconds
Supported formats	MP4, MOV
Supported codecs	H264, H265, AV1, VP9
Default dimensions	3,072
Pricing (Standard)	$12.00 / 1M tokens (~$0.00079 per fps)
Pricing (Batch)	$6.00 / 1M tokens (~$0.000395 per fps)

Step 1: Embed a Short Video (Python)

from google import genai
from google.genai import types

with open('clip.mp4', 'rb') as f:
    video_bytes = f.read()

client = genai.Client()

result = client.models.embed_content(
    model='gemini-embedding-2-preview',
    contents=[
        types.Part.from_bytes(
            data=video_bytes,
            mime_type='video/mp4',
        ),
    ]
)

embedding = result.embeddings[0].values
print(f"Dimensions: {len(embedding)}")  # 3072

Step 2: Embed via cURL

VIDEO_BASE64=$(base64 -w 0 clip.mp4)

curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-embedding-2-preview:embedContent" \
    -H "Content-Type: application/json" \
    -H "x-goog-api-key: ${GEMINI_API_KEY}" \
    -d '{
        "content": {
            "parts": [{
                "inline_data": {
                    "mime_type": "video/mp4",
                    "data": "'"${VIDEO_BASE64}"'"
                }
            }]
        }
    }'

Step 3: Handle Long Videos (Chunking)

For videos longer than 128 seconds, split into overlapping segments.

import subprocess
import json
from google import genai
from google.genai import types

def get_duration(path):
    """Get video duration in seconds using ffprobe."""
    cmd = ['ffprobe', '-v', 'quiet', '-print_format', 'json',
           '-show_format', path]
    result = subprocess.run(cmd, capture_output=True, text=True)
    info = json.loads(result.stdout)
    return float(info['format']['duration'])

def chunk_video(input_path, chunk_duration=120, overlap=10):
    """Split video into overlapping chunks."""
    duration = get_duration(input_path)
    chunks = []
    start = 0
    idx = 0

    while start < duration:
        end = min(start + chunk_duration, duration)
        output = f'chunk_{idx:03d}.mp4'
        subprocess.run([
            'ffmpeg', '-i', input_path,
            '-ss', str(start), '-to', str(end),
            '-c', 'copy', output, '-y'
        ], capture_output=True)
        chunks.append(output)
        start += chunk_duration - overlap
        idx += 1

    return chunks

def embed_video_chunks(video_path):
    """Embed a long video by chunking and embedding each segment."""
    client = genai.Client()
    chunks = chunk_video(video_path)
    embeddings = []

    for chunk_path in chunks:
        with open(chunk_path, 'rb') as f:
            result = client.models.embed_content(
                model='gemini-embedding-2-preview',
                contents=[types.Part.from_bytes(
                    data=f.read(), mime_type='video/mp4'
                )]
            )
            embeddings.append(result.embeddings[0].values)

    return embeddings  # List of vectors, one per chunk

Step 4: Search Videos with Text

from google import genai
from google.genai import types
import numpy as np

client = genai.Client()

# Embed search query
query = client.models.embed_content(
    model='gemini-embedding-2-preview',
    contents='person riding a bicycle',
    config=types.EmbedContentConfig(task_type='RETRIEVAL_QUERY')
)
query_vec = np.array(query.embeddings[0].values)

# Embed video
with open('cycling.mp4', 'rb') as f:
    video = client.models.embed_content(
        model='gemini-embedding-2-preview',
        contents=[types.Part.from_bytes(data=f.read(), mime_type='video/mp4')]
    )
video_vec = np.array(video.embeddings[0].values)

# Cosine similarity
sim = np.dot(query_vec, video_vec) / (np.linalg.norm(query_vec) * np.linalg.norm(video_vec))
print(f"Similarity: {sim:.4f}")

Verification Checklist

• [ ] Short video (under 128s) embedded successfully
• [ ] Output vector has 3072 dimensions
• [ ] Long video chunking implemented with overlapping segments
• [ ] MP4 and MOV formats tested
• [ ] Cross-modal text-to-video search working
• [ ] Codec compatibility verified (H264/H265/AV1/VP9)

Troubleshooting

Issue	Solution
Video too long	Chunk into segments under 128 seconds with 10s overlap
Unsupported codec	Re-encode with ffmpeg -i input.avi -c:v libx264 output.mp4
Unsupported format	Convert to MP4: ffmpeg -i input.webm -c copy output.mp4
Large file size	Compress: ffmpeg -i input.mp4 -crf 28 -preset fast output.mp4
Request timeout	Reduce video duration or quality; check network

See Also

• Audio Embeddings — Embed audio tracks separately
• Multimodal Embeddings — Combine video with text annotations
• Semantic Search — Cross-modal video search workflow
• Pricing — Video embedding costs