Reel2RealView source
Meta AI Developer Hackathon · Prototype demo

From watching to doing.

A short tutorial becomes a step guide. Muse Spark plans the sequence; SAM 3.1 checks the objects for the current step in a workspace frame. The learner controls progress.

2:03 · English synthetic narration and burned-in captions. Recorded Samples, generated QA media, virtual-camera input and separate live runs are labelled in the edit. One planning wait is shortened and disclosed.

What runs in the demo

  • A recorded lesson with saved instructions, saved SAM output and creator credit.
  • A real Muse Spark API call planning a complete, owned six-second generated clip, including its later blue-card step.
  • Real SAM checks from a virtual camera: missed objects in one run; a red-paper box and fresh recheck in a separate goal-plus-still run.
  • Next, Skip and a recap that preserve the learner’s choices.

Current boundaries

  • Detection can miss visible objects. These test runs include missed card/tray and paper edge/corner checks.
  • An object box does not verify the action, technique, order or general safety.
  • This is a generated-media test, not physical learner footage. Phone trials, useful physical-task correction and learner outcome studies remain to be done.
  • Recorded Samples replay supplied results; they do not make live inference calls.
Media provenance and credits

The red/blue lesson and red-square image are original procedural QA media created for this project. Camera input comes from Chromium’s virtual video device. The screencast records the hosted application without intercepting or replacing model responses. Narration uses the synthetic flite SLT voice.

The main recorded lesson uses Pexels 5480225 under the Pexels License. Gallery thumbnails retain the project’s full source and licence credits. Stock footage is not presented as self-filmed.