GPU image features

TypeCV includes an experimental single-scale feature frontend implemented with TypeShade compute kernels. extractFeaturesGpu converts RGBA pixels to grayscale, scores FAST-9 corners, keeps up to four high-scoring candidates in each 24 × 24 tile, estimates orientation from the intensity centroid, and computes a deterministic 256-bit rotated BRIEF descriptor. matchFeaturesGpu calculates Hamming distances with countOneBits, applies a nearest/second nearest ratio in both directions, and returns mutual matches. Extraction supports frames from 48 × 48 through the active device's storage and dispatch limits, with a maximum of 2,000 selected features per frame. Matching rejects inputs above four million pairwise descriptor comparisons to bound GPU work.

import { ShadeDevice, extractFeaturesGpu, matchFeaturesGpu } from 'typecv/gpu';

const device = await ShadeDevice.open({ backend: 'webgpu' });
try {
  const first = await extractFeaturesGpu(device, firstRgba, width, height);
  const second = await extractFeaturesGpu(device, secondRgba, width, height);
  const correspondences = await matchFeaturesGpu(device, first, second);
  // correspondences.matches is [firstFeature, secondFeature, ...].
} finally {
  await device.dispose();
}

The matcher returns final feature indices and Hamming distances to the host; frame decoding, RGBA packing, camera graph control, and current mapper inputs are host work. The pixel tests and descriptor math run on the selected TypeShade tier. The browser video lab explicitly requests WebGPU and does not retry on CPU. The existing incremental mapper then runs GPU geometric verification, PnP, triangulation, filtering and bundle adjustment.

This is an initial CV operator, not a COLMAP-equivalent feature frontend. It is single-scale, does not model lens distortion, and has not been benchmarked on physical GPUs. Textureless scenes, repeated patterns, rolling shutter, motion blur, weak parallax, inaccurate focal assumptions, or very long video can produce too few matches, unregistered frames, or misleading scale. Inspect the reported reprojection error, camera path, unregistered views, and source video; the sparse viewer is not a dense surface reconstruction.