VideoTextureScrub Op

Technical Report: Optimized Video Scrubbing in WebGL

1. Core Concept

This operator is a WebGL video texture player specifically engineered for fluid, zero-latency scrubbing on the web.

• The Video Format (GOP = 1): Standard videos compress data by calculating changes between frames, which causes heavy lag when scrubbing. Encoding the video with a GOP of 1 saves every frame as a complete picture, guaranteeing instant visual access.
• The Memory Trick (RAM Caching): Rapid scrubbing floods the browser with cancelled network requests. This operator solves this by downloading the entire video into local memory (RAM) upfront, bypassing the internet entirely during interaction.

2. Code Modifications

• Blob Fetching: Replaced the standard video URL with a fetch() command to download the entire file into memory as a Blob.
• Virtual URL: Used URL.createObjectURL() to link the video player directly to this local memory chunk.
• Memory Cleanup: Added URL.revokeObjectURL() to delete the memory chunk when the video is changed or removed, preventing memory leaks.
• Command Throttling: Added a check (videoElement.seeking) to ignore new slider movements if the player is still decoding the previous frame.
• Auto-Catchup: Added an onSeeked() event to instantly snap the video to the latest mouse position once the player finishes its current task.

Understanding WebGL Video Scrubbing

When building interactive media experiences on the web, users expect instant, fluid control when scrubbing through a video timeline. However, standard HTML5 video elements are designed for linear playback. When a user rapidly drags a time slider, standard streaming creates a bottleneck of cancelled network requests and heavy decoding lag.

To achieve frame-accurate, zero-latency scrubbing, two critical pieces must align:

1. The Video Encoding (GOP = 1): Standard codecs save space by calculating changes between frames, forcing the browser to perform heavy math during a seek. By encoding the video with a GOP of 1 (All-Intra), every single frame becomes an independent keyframe, allowing instant visual access.
2. The Memory Strategy (RAM Caching): Instead of relying on live streaming which stutters under rapid input, downloading the entire file upfront into a local memory Blob completely bypasses network latency.

By combining a properly encoded MP4 file with an optimized WebGL texture operator (VideoTextureScrub), developers can achieve perfectly smooth, responsive video control in real-time.