TECHNICAL REPORT / SL–01INTERACTIVE MODEL · 2026

Structured-lightdepth reconstruction

Camera–projector correspondence, triangulation, and failure modes—examined in a live synthetic scene.

ABSTRACT

This report traces one structured-light measurement from projected code to metric depth. The apparatus below exposes the optical and geometric assumptions directly: change a pattern, material, scene, or calibration variable, then compare the captured image with the reconstructed result.

APPARATUS: Calibrated camera + projector
OUTPUT: Depth, point cloud, and error
SCOPE: Instructional synthetic model

01 / APPARATUS

Interactive reconstruction model

Hold the defaults as a control, then change one variable at a time. Capture, decoding, and triangulation update from the same calibrated scene.

1
EncodeSelect a projected code and surface response.
2
ConfigureSet scene geometry and calibrated rig parameters.
3
ObserveCompare capture, depth, point cloud, and error.

PatternSurface

Drag to orbit · scroll to zoom

RECONSTRUCTIONSOLVING

Adjust modelOcclusion portrait · B 120 mm+

Scene

Curved foreground surfaces and a rear receiver expose projector shadows, self-occlusion, and grazing-angle dropout.

↥ LOAD SCENE PHOTO

Scene photo

Maps a PNG, JPEG, or WebP onto a calibrated planar target. Files are limited to 8 MB and 16 MP decoded, then resized to a 2048 px / 2 MP GPU budget. It changes albedo; it does not infer geometry.

Projection

Spatial frequency

Sets the number of sinusoidal periods across the projector. More periods improve phase sensitivity but increase wrapping ambiguity and sensitivity to blur.

18 cycles

Phase offset

Moves the starting fringe phase in the temporal sequence. The decoder subtracts the known offset, so geometry should remain stable; only sampling, clipping, and noise produce small changes.

32 °

Exposure

Scales collected pattern energy. Longer exposure improves modulation SNR until pixels saturate; in a real moving scene it also increases motion blur.

78 %

Laser optical power

Scales emitted optical power before inverse-square spreading and surface reflection. More power improves range and SNR but can saturate reflective regions.

100 %

Geometry

Baseline B

Camera–projector separation. A larger baseline creates more disparity and improves depth precision, but also enlarges occluded regions and correspondence search distance.

120 mm

Working distance Z

Moves both calibrated devices away from the target. Irradiance falls approximately with inverse square distance, while depth uncertainty grows approximately with Z².

850 mm

Camera focal f

Controls camera field of view and disparity scale. More focal pixels narrow the view and improve metric depth sensitivity for the same baseline.

1420 px

Sensor

Read noise σr

Adds signal-independent electronic noise to every exposure. Raising it destabilizes phase and correlation most strongly in dark or distant regions.

2 e⁻ RMS

Peak signal

Sets the sensor’s photon-to-electron signal budget. More electrons improve shot-noise SNR; reflective highlights can still reach the simulated full-well limit.

8000 e⁻

Ambient level

Adds visible hemispheric illumination and background electrons. It brightens the scene but lowers coded-pattern contrast and increases photon shot noise.

8 %

ADC depth

Sets digitizer quantization levels. Low bit depth rounds small intensity differences, damaging phase and Gray decisions even when optical SNR is high.

12 bit

Result diagnostics

Depth σ1.51mm

Density2.08M pts

Confidence96%

Valid surface—%

Disparity200.5px

Modulation SNR40.1:1

Correspondence σ0.356px

Relative irradiance34.6%

Signal electrons2,104e⁻

MODEL QUALITY

NOMINAL

ACTIVE SCENEOCCLUSION PORTRAIT

PINHOLE TRIANGULATIONZ = f B / dDepth grows with focal length and baseline; uncertainty grows approximately with Z².

FORWARD + INVERSE MODEL. Every geometry preset is rendered into the same calibrated camera/projector rig and rear receiver screen. Curved surfaces, holes, sharp depth steps, self-occlusion, and projector shadows therefore alter the captured codes and reconstructed output—not just the display mesh. Irradiance follows inverse-square spreading and surface foreshortening; ground truth remains a separate camera depth pass, while estimated points come only from captured and decoded frames.

02 / CODED LIGHT

Pattern library

Pattern design balances precision, ambiguity, capture speed, surface reflectance, and motion. There is no universal winner.

Pattern	How it works	Frames	Precision	Motion	Try pattern
Phase shiftSUBPIXEL	Sinusoidal fringes encode projector phase continuously. Excellent precision, but wrapped phase repeats every period.	3–12	0.05–0.2 px	Low	↗
Gray codeABSOLUTE	Binary stripes identify every projector column unambiguously. Adjacent codes differ by one bit, limiting edge errors.	20–24	0.5–1 px	Low	↗
Gray + phaseRECOMMENDED · CURRENT	Gray code resolves fringe order; phase shifting supplies subpixel correspondence. The metrology workhorse.	14–28	0.05–0.2 px	Low	↗
Coded IR dotsACTIVE STEREO	A fixed pseudo-random near-IR texture is matched against a calibrated reference with local normalized correlation. One exposure supports motion, but precision depends on local texture uniqueness.	1	0.15–1 px	High	↗

03 / HOW RECONSTRUCTION WORKS

From pattern to 3D point

Every method has the same goal: pair one camera pixel with one calibrated projector coordinate. Temporal codes decode that coordinate across several frames; coded dots recover it by matching one local texture patch. Triangulation is identical after correspondence.

KNOWN AFTER CALIBRATION

K_c, K_p — camera and projector intrinsics

R, t — rigid pose from camera to projector

(u_c, v_c) ↔ u_p — decoded correspondence

DEPTH UNCERTAINTYσ_Z ≈ Z² σ_d / fBDouble the range → roughly 4× the depth uncertainty.

MULTI-FRAMEPhase / Gray / hybrid

Decode the projector column from a synchronized temporal intensity sequence.

ONE FRAMECoded dots / static texture

Match a unique camera patch to the calibrated reference along one epipolar row.

PROJECT

Encode a known coordinate in light.

CAPTURE

Observe how the surface deforms it.

DECODE

Recover projector ↔ camera matches.

TRIANGULATE

Intersect rays to solve 3D points.

Confidence gate: shadowed, saturated, flat, repetitive, or geometrically unstable matches remain holes—no projector correspondence means no measured point.

SIDE VIEW / ONE CORRESPONDENCE

How one camera pixel becomes one 3D point

Live example: B = 120 mm · d = 200.5 px · Z = 850 mm

01DecodeThe captured temporal code identifies projector column u_p at camera pixel (u_c, v_c).

02CastCalibration turns those two coordinates into known directions from O_c and O_p.

03IntersectThe camera ray meets the projector sheet at X. Its camera-axis coordinate is the metric depth Z.

PER-PIXEL PIPELINE

Complete per-pixel equations

One camera pixel and one decoded projector column become one metric 3D point.

I_kdecodeu_pcalibrater_c + Π_pintersectX

01 / DECODE
Intensity → projector column
Three phase frames recover the wrapped phase; Gray code supplies the fringe order.
I_k = a + b cos(φ + 2πk/3), k ∈ {0,1,2}φ = atan2[√3(I₂ − I₁), 2I₀ − I₁ − I₂]u_p = (W_p/N) [m + φ/(2π)]
02 / CAMERA RAY
Pixel → line in 3D
Undistort the camera pixel, apply calibration, and express its ray in the common frame.
x̃_c = K_c⁻¹[u_c, v_c, 1]^Tr_c = R_c^Tx̃_c / ‖R_c^Tx̃_c‖X(λ) = O_c + λr_c
03 / PROJECTOR PLANE
Column → sheet of light
Top and bottom rays through the decoded column define one projector plane.
r_t = ray_p(u_p, 0), r_b = ray_p(u_p, H_p−1)n_p = (r_t × r_b) / ‖r_t × r_b‖n_p · (X − O_p) = 0
04 / INTERSECT
Ray ∩ plane → metric XYZ
Solve one scalar distance along the camera ray, then recover the 3D point.
λ = n_p · (O_p − O_c) / (n_p · r_c)X = O_c + λr_c

Return no point when modulation is weak, a frame saturates, the rays are nearly parallel, λ falls behind the camera, or reprojection fails.

04 / VISIBILITY + LIGHT TRANSPORT

Visibility + light transport

A surface point is measurable only if it is simultaneously visible to the camera and illuminated by the projector. The rear screen in the live scene makes the missing projector rays visible as real shadows.

MEASUREMENT MASKM(X) = V_c(X) · V_p(X) · FOV(X) · 𝟙[SNR(X) > τ]V_c and V_p are independent camera and projector visibility tests.

PROJECTOR

CAMERA

OCCLUDER

REAR SCREENPROJECTOR SHADOW

CASE 01

Camera occlusion

V_c = 0. Another object is nearer on the camera ray, so the camera never records the hidden point—even if the projector illuminates it.

CASE 02

Projector shadow

V_c = 1 but V_p = 0. The camera sees the background, but an object blocks its coded illumination. Modulation collapses and reconstruction must return a hole.

CASE 03

Mutual visibility

V_c = V_p = 1. The temporal code arrives, the camera decodes u_p, and triangulation can produce a measured 3D point.

PROJECTED PATTERN / DIVERGING RAYS

Power spreads over area.

E(X) = P η T(u_p) max(0, n·l) / (Ω r_p²)

The live shader uses this inverse-square projector model: doubling projector-to-surface range gives approximately one quarter of the irradiance before camera exposure compensation. Oblique surfaces receive less through the cosine term.

TRUE GAUSSIAN LASER BEAM

Collimated does not mean lossless.

I(r,z) = 2P/[πw(z)²] · exp[−2r²/w(z)²]w(z) = w₀√[1 + (z/z_R)²]

Near its waist a laser may stay nearly collimated; farther away diffraction expands the spot and lowers peak irradiance. Absorption and scattering add medium-dependent losses beyond geometric spreading.

WHY DEPTH FAILS BEFORE LIGHT DISAPPEARS

Fewer detected photons → noisier phase.

N_e ∝ E(X) · ρ · t_exp · T_lens/N_f²

For a resolved Lambertian surface, camera image irradiance follows scene radiance and lens f-number—there is not a second independent 1/r² penalty per pixel. Shot noise grows as √N while signal grows as N, so the pattern can remain faintly visible yet fall below the modulation threshold for reliable correspondence.

05 / LIVE WORKED EXAMPLE

Rectified depth calculator

This compact form assumes a rectified camera–projector pair, so corresponding rows align. Both coordinates use one common rectified pixel grid; f_x = f_y = 1420 px and the principal point is (960, 600) px. It is the familiar stereo shortcut to the full ray–plane calculation above.

Camera u₍c₎

Selects the horizontal camera pixel in the worked triangulation example. Together with projector u, it determines disparity and therefore depth.

1120 px

Camera v₍c₎

Selects the vertical camera pixel in the worked example. It changes reconstructed Y after the camera ray is unprojected.

620 px

Projector u₍p₎

Selects the decoded projector column. Moving it changes disparity d = u_c − u_p; small disparity means far or invalid depth.

920 px

1 / CORRESPONDENCE

camera (1120, 620) px ↔ projector column 920 px

2 / DISPARITYd = u_c − u_p = 200 px

3 / DEPTHZ = f_xB/d = 1420 × 120 / 200 = 852.0 mm

4 / LATERAL POSITION

X = (u_c−c_x)Z/f_x · Y = (v_c−c_y)Z/f_y

TRIANGULATED CAMERA-FRAME POINT[ 96.0, 12.0, 852.0 ] mmPredicted 1σ depth uncertainty: 1.52 mm

06 / BETWEEN VALID SAMPLES

Interpolation policy

Triangulation returns an organized depth image with holes—not permission to connect every neighbor. Interpolation is justified only inside a locally continuous surface and should carry a separate “inferred” mask.

A / SHORT SMOOTH HOLE

Confidence-weighted bilinear

ẑ(p) = Σ_iw_ic_iz_i / Σ_iw_ic_i

Use the four cell corners only when their depths agree within a surface threshold. w_i is bilinear distance weight; c_i is decode confidence.

B / TEXTURED OR CURVED PATCH

Edge-aware weighting

w_i = exp(−‖p−p_i‖²/2σ_s²) exp(−|I−I_i|²/2σ_r²)c_i

Spatial and image-intensity similarity prevent foreground depth from bleeding across a visible silhouette.

C / LOCALLY PLANAR PATCH

Weighted plane fit

min_n,δ Σ_iw_i(n · X_i + δ)²

Fit a plane to valid 3D neighbors, then intersect the missing pixel’s camera ray with it. Reject if residuals or normal spread are large.

D / OCCLUSION OR DISCONTINUITY

Leave the hole

|z_left − z_right| > τ_z ⇒ invalid

Do not bridge foreground/background edges, projector shadows, specular dropouts, or long gaps. Multi-view fusion or learned completion may fill them, but those points are estimates—not structured-light measurements.

VALID MEASUREMENT→CONFIDENCE + EDGE TEST→LOCAL MODEL→INFERRED DEPTH + MASK