SPECTRA

LA Hacks 2026 · Stanford University

SPECTRA

Sparse-to-dense dEpth CompleTion via RGB-guided upsampling.
Making $25 consumer LiDAR as useful as $10,000 industrial sensors.

Ajay ShahWilliam WangBenjamin JiangJunfeng Lin

The Problem

LiDAR is either accurate or affordable.
Never both.

Industrial LiDAR delivers dense, reliable depth — at $8,000–$75,000 per unit.[1]

Apple changed the cost curve. The iPhone 15 Pro ships a solid-state LiDAR scanner at roughly $3–$25.[2] But at 192×256 pixels, the raw output is too sparse for any real scene understanding.[3]

SPECTRA closes this gap — dense metric depth from cheap hardware.

Industrial LiDAR

$10,000+

Dense, long-range, accurate. Inaccessible to home robotics. [1]

Consumer LiDAR (iPhone Pro)

$3–$25

Cheap, but sparse and noisy — unusable as-is for scene understanding. [2]

The Gap SPECTRA Closes

Sparse ≠ usable
for robotics

Raw consumer depth misses thin objects and has no spatial resolution for path planning or grasping. [3]

Our Solution

Dense indoor depth.
From sparse consumer LiDAR.

SPECTRANet fuses the iPhone's sparse LiDAR with its RGB camera — producing a dense, edge-accurate, metric depth map at full camera resolution.

Trained on ARKitScenes with Faro laser scanner ground truth. [4] Scoped to 0.5–10m indoors. Runs entirely on-device via CoreML at ~2MB.

Sparse LiDAR192×256 · metric · noisy

+

RGB CameraFull-res · sharp edges

↓

SPECTRANet MobileNetV2 RGB encoder + depth CNN + FuseUpBlock decoder at 5 scales + residual correction
⊕ Expand Architecture

↓

Dense metric depth768×1024 px · float32 · meters · CoreML · ~2MB

Scope: Indoor, 0.5–10m. Not designed for outdoor or long-range use.

SPECTRANet

Click any block in the diagram to learn how that stage works.

Input 1RGB Image3×768×1024

Input 2LiDAR Depth192×256 sparse

Input 3Confidence{0,1,2}/pixel

↓

RGB EncoderMobileNetV24 scales · s2/4/8/16

Depth EncoderLightweight CNN5 scales · depth+conf

↓

Fusion DecoderFuseUpBlocksbot→up3→up2→up1→up0

↓

Residual HeadΔ correctionscale 0.2×

OutputDense Depth768×1024 · meters

Live Demo

See SPECTRALive
in real time.

iPhone 15 Pro streams RGB + sparse LiDAR into SPECTRANet on-device via ZETIC Melange. Dense metric depth out. No cloud, no latency.

~2MBmodel size

4×resolution

NPUon-device

ARKitScenes Validation Benchmarks

Method	RMSE↓	MAE↓	δ1↑	δ2↑	δ3↑
Bicubic baseline	0.2171	0.0730	0.9476	0.9550	0.9630
Marigold-DC (zero-shot)	0.0989	0.0724	0.9730	0.9973	0.9995
Marigold-DC (LoRA)	0.1007	0.0735	0.9714	0.9970	0.9995
SPECTRANet (L1 only)	0.0709	0.0257	0.9851	0.9953	0.9984
SPECTRANet (ours)	0.0552	0.0216	0.9909	0.9973	0.9991

        δ1=0.9909 → 99.1% of pixels within ±25% of ground truth.

        75% lower RMSE vs bicubic · 1.8× better than Marigold-DC on RMSE · 3.4× better MAE.

Home Robotics Deep Dive

Raw LiDAR vs SPECTRA — same scene, same robot

The Roomba is blind to 3D geometry. [5] A $10 solid-state LiDAR module helps — but only if the depth is dense enough to see a sock, a chair leg, or a pet. [3]

Raw LiDAR — 192×256 sparse · 10 scan rays

Sparse rays miss thin obstacles. Robot collides with objects between scan lines.

SPECTRANet dense — 768×1024 · 60 rays equiv.

Dense depth detects every obstacle. Clean path planning around all objects.

Infrastructure

Built to run at full speed.
Deployed to run offline.

SPECTRA uses two pieces of hardware for two distinct jobs — training at maximum efficiency on the GX10, then deploying the result entirely on-device via ZETIC Melange. Once deployed, the iPhone needs nothing else.

Training — ASUS Ascent GX10

1 petaFLOP · 128GB unified memory

SPECTRANet was trained on the full ARKitScenes dataset on the GX10's NVIDIA GB10 Grace Blackwell Superchip. The 128GB unified memory pool meant no gradient checkpointing hacks — full batch sizes, full resolution, full speed. Training that would have required cloud A100 time ran entirely local.

NVIDIA GB10 Grace Blackwell 128GB LPDDR5x Ubuntu + PyTorch ARKitScenes · full dataset

Deployment — ZETIC Melange + iPhone 15 Pro

NPU-optimized · fully offline · 3 lines of code

After training, the model was uploaded to ZETIC Melange — which automatically converted it to an NPU-optimized binary for Apple's Neural Engine and generated the Swift SDK. SPECTRALive runs the full inference pipeline on the iPhone itself. No GX10, no internet, no cloud. Private by default.

Apple Neural Engine (A17 Pro) ZETIC Melange SDK ~2MB on-device zero cloud dependency

Applications

What dense depth from cheap hardware unlocks

🤖

Home Robotics

Full 3D obstacle map — furniture legs, pets, dropped items — from a $10 sensor upgraded by SPECTRANet.

Roomba · Home Nav

🥽

AR Occlusion

Per-pixel metric depth anchors virtual objects to real surfaces. Correct occlusion needs SPECTRA's precision.

ARKit · RealityKit

♿

Accessibility

Real-time metric warnings — "obstacle 0.8m ahead" — from your phone, powered by SPECTRALive.

Navigation · Assistive

✊

Robot Grasping

Edge-accurate depth at object boundaries enables reliable grasp pose estimation from cheap hardware via SPECTRANet.

Pick-and-place

🏠

Smart Home Security

Consumer cameras upgraded with SPECTRALive detect intruders by 3D shape rather than motion — distinguishing a pet from a person.

Security · Presence Detection

🦾

Prosthetic Vision

Embedding SPECTRANet in a wearable gives amputees and prosthetic arm users real-time depth feedback for grasp control without a dedicated sensor rig.

Neuroprosthetics · Rehab

The Bigger Picture

Every iPhone Pro since 2020

ships with a LiDAR scanner.

Apple's component cost: under $25. [2]

The same sensor standalone: $10,000. [1]

The bottleneck was never the sensor.

It was the software to make sparse depth usable.

SPECTRA is that software layer.

Dense. Metric. On-device. Free.

The Team

Built in 36 hours at LA Hacks 2026

Ajay Shah

Stanford · CS + EE

William Wang

Stanford · CS

Benjamin Jiang

Stanford · CS

Junfeng Lin

Stanford · CS

References

[1]Velodyne "HDL-64E," velodynelidar.com; Ouster "OS1," ouster.com. Industrial LiDAR retail $8k–$75k (2020–2023).

[2]Frumusanu, A. AnandTech, 2020; iFixit teardown. Apple solid-state LiDAR BOM est. $3–$25 at scale.

[3]Song et al. "Depth Completion with Twin Surface Extrapolation." CVPR 2021. Documents sparse LiDAR failure modes.

[4]Dehghan et al. "ARKitScenes." NeurIPS Datasets & Benchmarks, 2021.

[5]iRobot. "Roomba Product Specifications." irobot.com. Primary sensing: infrared cliff + bump sensors only.