Types and schema

Every exported type of @meeeetup/camera-core@0.6.0, plus the React surface on top of it. Defaults are the values the constructor resolves when a field is omitted, not suggestions.

Session

class FaceCaptureSession {
  constructor(opts: FaceCaptureSessionOptions);

  readonly trackedCount: number;
  readonly pendingFaces: SelectedFace[];
  readonly activeTracks: readonly TrackedFace[];

  processDetections(detections: Detection[], frame: FrameBuffer): void;
  tick(): void;
  rearm(): void;
  flushBatch(): Promise<void>;
  dispose(): void;
}

FaceCaptureSessionOptions

interface FaceCaptureSessionOptions {
  sessionType: "interactive" | "passive";

  batchIntervalMs?: number;     // 10_000
  maxBufferedFaces?: number;    // 200
  maxFacesPerRequest?: number;  // 50
  cooldownMs?: number;          // 10_000
  minDetectionScore?: number;   // 0.5
  minFaceHeight?: number;       // 0
  minFrontalness?: number;      // 50
  captureWindowMs?: number;     // 0 — off
  noseLandmark?: NoseLandmark;  // "noseTip"

  onSelect?: (face: SelectedFace) => void;
  onTrackRemoved?: (trackId: string) => void;
  onBatchSent?: (count: number) => void;
  onBatchError?: (error: unknown) => void;
  onBatchCapture?: (faces: SelectedFace[]) => void | Promise<void>;
}

Callbacks all default to no-ops; onBatchCapture is required in practice for passive mode, since it is the only output path.

Detector contract

Everything entering the pipeline is normalised to 0–1 against the frame, whatever the detector natively reports.

interface Detection {
  boundingBox: {
    originX: number;  // left edge,  0–1
    originY: number;  // top edge,   0–1
    width: number;    // 0–1
    height: number;   // 0–1
  };
  /** [0] rightEye · [1] leftEye · [2] noseTip (or noseBase on ML Kit) */
  keypoints: Array<{ x: number; y: number }>;
  score: number;      // detector confidence, 0–1
}

The keypoint order is a contract, not a convention: frontalness and direction both index it directly.

Frame contract

FrameBuffer is the only platform seam — the core never touches a DOM or native image.

interface FrameBuffer {
  readonly width: number;
  readonly height: number;
  toJpegBase64(quality?: number): string;
  cropFaceJpeg(
    cx: number, cy: number,
    bw: number, bh: number,
    quality?: number,
  ): string | null;
}

@meeeetup/camera-web implements it over a canvas and @meeeetup/camera-react-native over a cached snapshot file; implement it yourself only when wiring a detector or surface the SDK does not ship.

What you receive

interface SelectedFace {
  trackId: string;
  dataUrl: string;      // 256×256 JPEG data URI
  frontalness: number;  // 0–100
  lastSentAt: number;   // Unix ms
  createdAt: number;    // Unix ms, when the face was first seen
  final: boolean;       // false while a window may still replace it
}

interface LiveFacePreview {
  trackId: string;
  dataUrl: string;      // best crop so far
  frontalness: number;  // smoothed, 0–100
}

Tracked-face state

session.activeTracks exposes the tracker’s own state — read-only, and stable only within a frame. Overlays and progress indicators are what it is for.

interface TrackedFace {
  id: string;
  createdAt: number;
  cx: number; cy: number;                    // centroid, normalised
  dispCx: number; dispCy: number;            // last drawn position
  dispW: number; dispH: number;
  bestJpeg: string | null;                   // all-time best crop
  bestFrontalness: number;
  selectedJpeg: string | null;               // what the current window holds
  selectedFrontalness: number;
  currentFrontalness: number;                // this frame
  smoothedFrontalness: number;               // EMA, settle path only
  lastSeenAt: number; lastSentAt: number;
  pendingJpeg: string | null;                // settle path only
  pendingFrontalness: number;
  pendingStaleFrames: number;
  pendingConfirmedFrames: number;
  windowState: "idle" | "open" | "closed";
  windowElapsedMs: number;                   // window time actually accrued
  windowTickAt: number;
}

A windowed track fills selectedJpeg and leaves pendingJpeg null; a settle-path track does the opposite.

Geometry

function getFrontalness(
  kps: Array<{ x: number; y: number }>,
  faceWidth: number,
  faceHeight: number,
  opts?: { noseLandmark?: "noseTip" | "noseBase" },
): number; // 0–100

function getFaceDirection(
  kps: Array<{ x: number; y: number }>,
  faceWidth: number,
  opts?: { deadzone?: number }, // default 0.35
): FaceDirection;

interface FaceDirection {
  yaw: number;    // −1 viewer's left … +1 viewer's right
  pitch: number;  // −1 chin up … +1 chin down
  label: FaceDirectionLabel;
}

type FaceDirectionLabel =
  | "front" | "left" | "right" | "up" | "down"
  | "up-left" | "up-right" | "down-left" | "down-right";

Frontalness answers how usable is this frame; direction answers which way do I ask them to turn. Frontalness is pure pose and is never multiplied by Detection.score.

Constants

Exported so a consumer can reason about the pipeline without hard-coding numbers.

Tracker primitives

Below the session, for consumers running their own loop:

function updateTracks(
  detections: Detection[],
  frame: FrameBuffer,
  tracks: TrackedFace[],
  onCommit: (track: TrackedFace) => void,
  options?: UpdateTracksOptions,
): void;

interface UpdateTracksOptions {
  cooldownMs?: number;       // TRACK_COOLDOWN_MS
  noseLandmark?: NoseLandmark;
  minFrontalness?: number;   // PENDING_CONFIRM_THRESHOLD
  captureWindowMs?: number;  // 0
  onSelect?: (face: SelectedFace) => void; // required with a window
}

function flushStale(
  tracks: TrackedFace[],
  onCommit: (track: TrackedFace) => void,
  onRemove: (trackId: string) => void,
): TrackedFace[];

function commitPending(track: TrackedFace, onSelect: (face: SelectedFace) => void): void;
function resetCaptureWindow(track: TrackedFace): void;
function finalizeCaptureWindow(track: TrackedFace, onSelect: (face: SelectedFace) => void): void;
function nmsFilter(faces: Detection[], iouThreshold?: number): Detection[];
function boxIoU(a: Detection["boundingBox"], b: Detection["boundingBox"]): number;

React surface

type MeeeetUpCamProviderProps =
  | InteractiveMeeeetUpCamProviderProps
  | PassiveCaptureOnlyMeeeetUpCamProviderProps;

interface PassiveCaptureOnlyMeeeetUpCamProviderProps {
  mode: "passive";
  children: React.ReactNode;
  onBatchCapture: (faces: SelectedFace[]) => void | Promise<void>;
  minDetectionScore?: number;
  minFaceHeight?: number;
  minFrontalness?: number;
  captureWindowMs?: number;
  cooldownMs?: number;
  mediapipeWasmUrl?: string;
}

interface InteractiveMeeeetUpCamProviderProps {
  mode: "interactive";
  children: React.ReactNode;
  onCapture: (faceImage: string, fullCircleImage: string) => void | Promise<void>;
  alignmentCaptureDelayMs?: number; // 2000
  devFaces?: string[];
}

useMeeeetUpCam() returns a union discriminated on mode:

interface PassiveState {
  mode: "passive";
  ready: boolean;
  error: string | null;
  videoRef: React.RefObject<HTMLVideoElement | null>;
  overlayRef: React.RefObject<HTMLCanvasElement | null>;
  devices: MediaDeviceInfo[];
  currentDeviceId: string | undefined;
  setDeviceId: (id: string | undefined) => void;
  currentFacingMode: "user" | "environment";
  toggleCamera: () => void;
  trackedCount: number;
  totalCount: number;
  selectedFaces: SelectedFace[];
  livePreviews: LiveFacePreview[];
  flushBatch: () => Promise<void>;
  rearm: () => void;
}

The interactive half carries the alignment-ring state instead: isFaceAligned, captureProgress, alignmentCountdown, faceBoundingBox, captureImage(), toggleFlip().

React Native surface

interface UseFaceCaptureSessionOptions
  extends Omit<FaceCaptureSessionOptions, "noseLandmark"> {
  cameraRef: React.RefObject<CameraLike | null>;
  detectionFps?: number;        // 15
  photoCadenceMs?: number;      // 1000
  snapshotQuality?: number;     // 60
  windowWidth?: number;
  windowHeight?: number;
  cameraFacing?: "front" | "back" | "external"; // "front"
}

interface UseFaceCaptureSessionReturn {
  ready: boolean;
  error: string | null;
  trackedCount: number;
  lastDetections: Detection[];
  lastSnapshot: SnapshotInfo | null;
  lastTimings: PipelineTimings | null;
  frameProcessor: ReadonlyFrameProcessor;
  flushBatch: () => Promise<void>;
}

noseLandmark is omitted on purpose: the hook pins it to "noseBase" because ML Kit has no nose tip, and overriding it would break score parity with the web SDK.