ECCV 2026 Workshop on Wearable AI

BEAT: Bottleneck-Enhanced Adaptive Temporal Learning for Heart Rate Estimation from Egocentric NIR Eye Videos

Juha Park¹ Sang Jun Lee^1,*

¹Jeonbuk National University ^*Corresponding author

BEAT architecture: IMU-conditioned NIR video encoder, location-wise bottleneck DW-TCN adapter, temporal decoder, and MPPL supervision — **Figure 1. BEAT architecture.** The PulseFormer-derived video/IMU backbone and temporal decoder are retained. BEAT inserts a shared location-wise DW-TCN at the spatially structured bottleneck before global pooling and adds Multi-Lag Pulse-Phase Progression Loss (MPPL). Full-resolution figure (PDF).

Refine each bottleneck location's temporal trajectory before spatial aggregation.

Abstract

Egocentric near-infrared eye videos expose only a small periocular region, where subtle pulse-related variation is mixed with spatially uneven disturbance from eyelids, gaze changes, occlusion, and headset motion. Global spatial pooling can merge stable and corrupted trajectories before their distinct temporal behavior is modeled.

BEAT preserves location identity at the encoder bottleneck. A lightweight residual depthwise temporal adapter independently refines all 24 spatial trajectories with shared parameters before global pooling. MPPL complements waveform and spectral supervision by aligning local analytic-phase progression at multiple temporal lags. The retained evaluation snapshot reports 7.20 BPM MAE, 9.68 BPM RMSE, 8.65% MAPE, and a Pearson correlation of 0.88 on egoPPG-DB.

Method

01 · WHERE

Pre-pooling temporal adaptation

BEAT receives the bottleneck tensor B × 64 × 32 × 3 × 8, preserves its 24 spatial positions, and applies the same temporal operator independently to every length-32 trajectory.

B × 64 × 32 × 3 × 8→(B × 24) × 64 × 32

02 · HOW

Residual depthwise TCN

Four non-causal depthwise Conv1D stages use kernel 3 and dilations [1, 2, 4, 8], giving a 31-token receptive field. GroupNorm, SiLU, pointwise GLU, layer scaling, and residual output projection stabilize refinement.

03 · OBJECTIVE

Multi-lag pulse phase

MPPL reconstructs a band-limited pulse and aligns analytic-phase increments at lags {1, 2, 4}, weighted by the target envelope to reduce supervision from ambiguous low-amplitude intervals.

L_total = L_wave + 0.1 L_spec + λ(t)L_MPPL

Main Results

Heart-rate estimation on egoPPG-DB. Lower MAE, RMSE, and MAPE and higher Pearson r are better.

Table 1. Comparison with existing rPPG methods on egoPPG-DB.
Model	MAE ↓	RMSE ↓	MAPE (%) ↓	r ↑
Yue et al.	29.63	32.99	37.86	0.10
DeepPhys	28.26	31.97	36.68	0.08
TS-CAN	26.32	32.39	29.13	0.11
ContrastPhys+	19.12	24.13	22.57	0.21
RhythmMamba	15.05	19.78	17.46	−0.16
Signal processing (eyes)	14.60	18.18	18.37	0.20
PhysMamba	13.94	16.86	17.76	0.61
RhythmFormer	13.13	17.43	14.73	0.51
Signal processing (skin)	12.40	15.54	15.29	0.50
PhysNet	12.09	15.43	15.14	0.66
PhysFormer	10.71	13.97	12.69	0.72
FactorizePhys	10.07	13.43	12.36	0.67
PulseFormer (published)	7.67	10.69	9.45	0.85
PulseFormer (our reproduction)	9.12	12.15	11.19	0.79
BEAT (ours)	7.20	9.68	8.65	0.88

Evaluation provenance. Values are transcribed from Table 1 of the paper. “PulseFormer (published)” and the other literature baselines are reported egoPPG references; “PulseFormer (our reproduction)” is the separate reproduction reported in our paper. The displayed BEAT values also match the retained local evaluation snapshot; this page update does not establish a new repeated-run aggregate. The controlled component table below isolates the adapter and MPPL under the common waveform–spectral objective.

Efficiency

Table 2. Computational comparison on an NVIDIA B200, as reported in the paper.
Model	T	Params (M)	MACs (G)	Latency (ms) ↓	kFPS ↑
PulseFormer	128	12.08	49.69	4.01	31.94
BEAT	128	12.10	49.70	4.61	27.77

Protocol reported in the paper: FP32 inference, batch size 4, 128-frame clips, NVIDIA B200; latency reported as the median over seven runs. Table values and this protocol description are reproduced from Table 2.

Component Ablation

The adapter and MPPL are toggled within the same training and evaluation pipeline.

Table 3. Individual and combined effects of the TCN adapter and MPPL.
Configuration	Adapter	MPPL	MAE ↓	RMSE ↓	MAPE (%) ↓	r ↑
Backbone control	—	—	8.83	12.63	11.65	0.83
Adapter component	✓	—	7.84	10.62	9.41	0.87
MPPL component	—	✓	7.82	11.07	9.96	0.86
Full BEAT	✓	✓	7.20	9.68	8.65	0.88

“Backbone control” is the retained architecture trained with the common waveform-plus-spectral objective; it is not an official reproduction of the published PulseFormer result.

Scope

Current evidence is limited to egoPPG-DB. The adapter is non-causal and is therefore not a strictly streaming implementation. MPPL supervises local phase progression, but direct beat-timing, short-window HR, and motion-stratified validation should be reported separately before making those empirical claims.

Code & Reproduction

The source-only BEAT implementation includes the shared bottleneck DW-TCN adapter, MPPL, reproducible configurations, training and evaluation entry points, and synthetic CPU tests.

The original PulseFormer backbone, MITA, preprocessing and HR evaluator are used through a separately obtained, pinned official egoPPG checkout. The adapter and loss extend this backbone; they do not replace MITA.

No trained weights, checkpoints, optimizer state, raw or preprocessed recordings, participant metadata, or experiment logs are distributed. Public initialization weights and egoPPG-DB must be obtained independently under their applicable terms.

Installation & training · Implementation details · Release scope

The legacy code identifier cpv denotes the paper's Multi-Lag Pulse-Phase Progression Loss (MPPL). Hyperparameter settings are included for reproduction; learned parameter tensors are not.

BibTeX

@inproceedings{park2026beat,
  title     = {BEAT: Bottleneck-Enhanced Adaptive Temporal Learning
               for Heart Rate Estimation from Egocentric NIR Eye Videos},
  author    = {Park, Juha and Lee, Sang Jun},
  booktitle = {ECCV Workshop on Wearable AI},
  year      = {2026}
}