Skip to content

Types and schema

Every exported type of @meeeetup/camera-core@0.6.0, plus the React surface on top of it. Defaults are the values the constructor resolves when a field is omitted, not suggestions.

class FaceCaptureSession {
constructor(opts: FaceCaptureSessionOptions);
readonly trackedCount: number;
readonly pendingFaces: SelectedFace[];
readonly activeTracks: readonly TrackedFace[];
processDetections(detections: Detection[], frame: FrameBuffer): void;
tick(): void;
rearm(): void;
flushBatch(): Promise<void>;
dispose(): void;
}
Member When to call it
processDetections Once per video frame, with that frame’s detections and pixels.
tick Once per animation frame or timer: reaps tracks unseen for TRACK_STALE_MS, finalising anything they still held.
rearm Re-arms every capture window. No-op without captureWindowMs.
flushBatch Hands buffered faces to onBatchCapture now — call it on page hide. Passive mode also calls it on batchIntervalMs.
dispose Stops timers, drops tracks and buffer. Call on unmount.
interface FaceCaptureSessionOptions {
sessionType: "interactive" | "passive";
batchIntervalMs?: number; // 10_000
maxBufferedFaces?: number; // 200
maxFacesPerRequest?: number; // 50
cooldownMs?: number; // 10_000
minDetectionScore?: number; // 0.5
minFaceHeight?: number; // 0
minFrontalness?: number; // 50
captureWindowMs?: number; // 0 — off
noseLandmark?: NoseLandmark; // "noseTip"
onSelect?: (face: SelectedFace) => void;
onTrackRemoved?: (trackId: string) => void;
onBatchSent?: (count: number) => void;
onBatchError?: (error: unknown) => void;
onBatchCapture?: (faces: SelectedFace[]) => void | Promise<void>;
}
Field Type Default Meaning
sessionType "interactive" | "passive" required Passive buffers and batches; interactive hands you one face at a time.
batchIntervalMs number 10_000 Passive send cadence.
maxBufferedFaces number 200 Buffer ceiling while offline; oldest drop first.
maxFacesPerRequest number 50 Batch size; must not exceed the backend’s cap.
cooldownMs number 10_000 Minimum gap between captures of the same track; the re-arm delay with a window.
minDetectionScore number 0.5 Detector confidence floor, 0–1.
minFaceHeight number 0 Face-box height floor as a fraction of frame height.
minFrontalness number 50 Pose floor, 0–100.
captureWindowMs number 0 Best-shot window per track; 0 keeps the settle gate.
noseLandmark "noseTip" | "noseBase" "noseTip" Which landmark sits in keypoint slot [2].

Callbacks all default to no-ops; onBatchCapture is required in practice for passive mode, since it is the only output path.

Everything entering the pipeline is normalised to 0–1 against the frame, whatever the detector natively reports.

interface Detection {
boundingBox: {
originX: number; // left edge, 0–1
originY: number; // top edge, 0–1
width: number; // 0–1
height: number; // 0–1
};
/** [0] rightEye · [1] leftEye · [2] noseTip (or noseBase on ML Kit) */
keypoints: Array<{ x: number; y: number }>;
score: number; // detector confidence, 0–1
}

The keypoint order is a contract, not a convention: frontalness and direction both index it directly.

FrameBuffer is the only platform seam — the core never touches a DOM or native image.

interface FrameBuffer {
readonly width: number;
readonly height: number;
toJpegBase64(quality?: number): string;
cropFaceJpeg(
cx: number, cy: number,
bw: number, bh: number,
quality?: number,
): string | null;
}

@meeeetup/camera-web implements it over a canvas and @meeeetup/camera-react-native over a cached snapshot file; implement it yourself only when wiring a detector or surface the SDK does not ship.

interface SelectedFace {
trackId: string;
dataUrl: string; // 256×256 JPEG data URI
frontalness: number; // 0–100
lastSentAt: number; // Unix ms
createdAt: number; // Unix ms, when the face was first seen
final: boolean; // false while a window may still replace it
}
interface LiveFacePreview {
trackId: string;
dataUrl: string; // best crop so far
frontalness: number; // smoothed, 0–100
}

session.activeTracks exposes the tracker’s own state — read-only, and stable only within a frame. Overlays and progress indicators are what it is for.

interface TrackedFace {
id: string;
createdAt: number;
cx: number; cy: number; // centroid, normalised
dispCx: number; dispCy: number; // last drawn position
dispW: number; dispH: number;
bestJpeg: string | null; // all-time best crop
bestFrontalness: number;
selectedJpeg: string | null; // what the current window holds
selectedFrontalness: number;
currentFrontalness: number; // this frame
smoothedFrontalness: number; // EMA, settle path only
lastSeenAt: number; lastSentAt: number;
pendingJpeg: string | null; // settle path only
pendingFrontalness: number;
pendingStaleFrames: number;
pendingConfirmedFrames: number;
windowState: "idle" | "open" | "closed";
windowElapsedMs: number; // window time actually accrued
windowTickAt: number;
}

A windowed track fills selectedJpeg and leaves pendingJpeg null; a settle-path track does the opposite.

function getFrontalness(
kps: Array<{ x: number; y: number }>,
faceWidth: number,
faceHeight: number,
opts?: { noseLandmark?: "noseTip" | "noseBase" },
): number; // 0–100
function getFaceDirection(
kps: Array<{ x: number; y: number }>,
faceWidth: number,
opts?: { deadzone?: number }, // default 0.35
): FaceDirection;
interface FaceDirection {
yaw: number; // −1 viewer's left … +1 viewer's right
pitch: number; // −1 chin up … +1 chin down
label: FaceDirectionLabel;
}
type FaceDirectionLabel =
| "front" | "left" | "right" | "up" | "down"
| "up-left" | "up-right" | "down-left" | "down-right";

Frontalness answers how usable is this frame; direction answers which way do I ask them to turn. Frontalness is pure pose and is never multiplied by Detection.score.

Exported so a consumer can reason about the pipeline without hard-coding numbers.

Constant Value Meaning
TRACK_MATCH_THRESHOLD 0.20 Max normalised centroid distance for a detection to join a track.
TRACK_STALE_MS 3000 Unseen longer than this and the track is reaped.
TRACK_CONFIRM_MS 300 A track must exist this long before it can be captured.
BOX_DISPLAY_MS 150 How long an overlay box outlives its last sighting.
PENDING_CONFIRM_THRESHOLD 50 Default minFrontalness.
MIN_PENDING_CONFIRMED_FRAMES 3 Frames above the floor before a capture may happen.
DEFAULT_MIN_DETECTION_SCORE 0.5 Default minDetectionScore.
TRACK_COOLDOWN_MS 10_000 Default cooldownMs.
WINDOW_RESUME_GAP_MS 250 Longest frame gap that still counts as continuous window time.
DEFAULT_MAX_BUFFERED_FACES 200 Default maxBufferedFaces.
DEFAULT_MAX_FACES_PER_REQUEST 50 Default maxFacesPerRequest.

Below the session, for consumers running their own loop:

function updateTracks(
detections: Detection[],
frame: FrameBuffer,
tracks: TrackedFace[],
onCommit: (track: TrackedFace) => void,
options?: UpdateTracksOptions,
): void;
interface UpdateTracksOptions {
cooldownMs?: number; // TRACK_COOLDOWN_MS
noseLandmark?: NoseLandmark;
minFrontalness?: number; // PENDING_CONFIRM_THRESHOLD
captureWindowMs?: number; // 0
onSelect?: (face: SelectedFace) => void; // required with a window
}
function flushStale(
tracks: TrackedFace[],
onCommit: (track: TrackedFace) => void,
onRemove: (trackId: string) => void,
): TrackedFace[];
function commitPending(track: TrackedFace, onSelect: (face: SelectedFace) => void): void;
function resetCaptureWindow(track: TrackedFace): void;
function finalizeCaptureWindow(track: TrackedFace, onSelect: (face: SelectedFace) => void): void;
function nmsFilter(faces: Detection[], iouThreshold?: number): Detection[];
function boxIoU(a: Detection["boundingBox"], b: Detection["boundingBox"]): number;
type MeeeetUpCamProviderProps =
| InteractiveMeeeetUpCamProviderProps
| PassiveCaptureOnlyMeeeetUpCamProviderProps;
interface PassiveCaptureOnlyMeeeetUpCamProviderProps {
mode: "passive";
children: React.ReactNode;
onBatchCapture: (faces: SelectedFace[]) => void | Promise<void>;
minDetectionScore?: number;
minFaceHeight?: number;
minFrontalness?: number;
captureWindowMs?: number;
cooldownMs?: number;
mediapipeWasmUrl?: string;
}
interface InteractiveMeeeetUpCamProviderProps {
mode: "interactive";
children: React.ReactNode;
onCapture: (faceImage: string, fullCircleImage: string) => void | Promise<void>;
alignmentCaptureDelayMs?: number; // 2000
devFaces?: string[];
}

useMeeeetUpCam() returns a union discriminated on mode:

interface PassiveState {
mode: "passive";
ready: boolean;
error: string | null;
videoRef: React.RefObject<HTMLVideoElement | null>;
overlayRef: React.RefObject<HTMLCanvasElement | null>;
devices: MediaDeviceInfo[];
currentDeviceId: string | undefined;
setDeviceId: (id: string | undefined) => void;
currentFacingMode: "user" | "environment";
toggleCamera: () => void;
trackedCount: number;
totalCount: number;
selectedFaces: SelectedFace[];
livePreviews: LiveFacePreview[];
flushBatch: () => Promise<void>;
rearm: () => void;
}

The interactive half carries the alignment-ring state instead: isFaceAligned, captureProgress, alignmentCountdown, faceBoundingBox, captureImage(), toggleFlip().

interface UseFaceCaptureSessionOptions
extends Omit<FaceCaptureSessionOptions, "noseLandmark"> {
cameraRef: React.RefObject<CameraLike | null>;
detectionFps?: number; // 15
photoCadenceMs?: number; // 1000
snapshotQuality?: number; // 60
windowWidth?: number;
windowHeight?: number;
cameraFacing?: "front" | "back" | "external"; // "front"
}
interface UseFaceCaptureSessionReturn {
ready: boolean;
error: string | null;
trackedCount: number;
lastDetections: Detection[];
lastSnapshot: SnapshotInfo | null;
lastTimings: PipelineTimings | null;
frameProcessor: ReadonlyFrameProcessor;
flushBatch: () => Promise<void>;
}

noseLandmark is omitted on purpose: the hook pins it to "noseBase" because ML Kit has no nose tip, and overriding it would break score parity with the web SDK.