Flow4R: Unifying 4D Reconstruction and Tracking with Scene Flow

Shenhan Qian1,2 Ganlin Zhang1,2 Shangzhe Wu3† Daniel Cremers1,2†
1 TU Munich 2 MCML 3 University of Cambridge

† Equal advising.

ECCV 2026

page arxiv 

  









    

  Flow4R teaser
  

    Given each image pair, Flow4R predicts for each image the point position $\mathbf{P}$, scene flow $\mathbf{F}$, pose weight $\mathbf{W}$, and confidence $\mathbf{C}$. Central to our framework, the scene flow $\mathbf{F}$ captures motion of points relative to the camera, thus is independent of the choice of coordinate system. Based on the pose weight $\mathbf{W}$, the scene flow $\mathbf{F}$ can be accurately decomposed into camera motion and object motion, enabling stable reconstruction and flexible tracking in both static and dynamic scenarios.
  





Abstract



Reconstructing and tracking dynamic 3D scenes remains a fundamental challenge in computer vision. Existing approaches often decouple geometry from motion: multi-view reconstruction methods assume static scenes, while dynamic tracking frameworks rely on explicit camera pose estimation or separate motion models. We propose Flow4R, a unified framework that treats camera-space scene flow as the central representation linking 3D structure, object motion, and camera motion. Flow4R predicts a minimal per-pixel property set—3D point position, scene flow, pose weight, and confidence—from two-view inputs using a Vision Transformer. This flow-centric formulation allows local geometry and bidirectional motion to be inferred symmetrically with a shared decoder in a single forward pass, without requiring explicit pose regressors or bundle adjustment. Trained jointly on static and dynamic datasets, Flow4R achieves state-of-the-art performance on 4D reconstruction and tracking tasks, demonstrating the effectiveness of the flow-central representation for spatiotemporal scene understanding.



Pipeline




  Flow4R pipeline
  

    Flow4R takes two images as input at a time and predicts a pixel-aligned property set, including point position $\mathbf{P}$, scene flow $\mathbf{F}$, pose weight $\mathbf{W}$, and confidence $\mathbf{C}$ (omitted in this figure), from which various downstream predictions can be deduced.
  





Evaluation



World Coordinate 3D Point Tracking



  

    

      
        

          Method
          All Points
          Dynamic Points
          # param.
(B)		
        

        

          ADT
          DR
          PO
          PS
          ADT
          DR
          PO
        

      
      
        

          MonST3R
          74.4
          58.1
          33.5
          51.3
          67.9
          51.9
          39.4
          0.7
        

        

          SpaTracker
          45.7
          54.9
          38.5
          62.6
          67.7
          58.7
          51.2
          0.2
        

        

          POMATO
          57.2
          68.4
          49.7
          64.9
          78.1
          62.7
          58.1
          0.7
        

        

          St4RTrack
          76.0
          73.7
          68.0
          69.7
          75.3
          68.1
          68.7
          0.7
        

        

          Flow4R
          78.6
          78.5
          71.1
          64.3
          70.9
          77.2
          72.9
          0.4
        

      
    

  

  

    We report the performance on four datasets, Aria Digital Twin (ADT), Dynamic Replica (DR), and Point Odyssey (PO), and Panoptic Studio (PS) using the Average Points under Distance (APD3D$\uparrow$) metric for all points and dynamic points after global median alignment. We also compare the model sizes in the last column. The best and second-best results are marked in bold and underlined.
  





World Coordinate 3D Reconstruction



  

    

      
        

          Category
          Method
          Point Odyssey
          TUM-Dynamics
        

        

          APD↑
          EPE↓
          APD↑
          EPE↓
        

      
      
        

          w/ Global Align.
          DUSt3R+GA
          43.90
          0.609
          70.49
          0.315
        

        

          MASt3R+GA
          60.44
          0.403
          68.38
          0.519
        

        

          MonST3R+GA
          72.31
          0.263
          63.87
          0.343
        

        

          Feedforward
          DUSt3R
          45.79
          0.639
          72.26
          0.289
        

        

          MASt3R
          56.90
          0.464
          66.22
          0.551
        

        

          MonST3R
          68.25
          0.304
          61.38
          0.365
        

        

          POMATO
          66.50
          0.385
          49.80
          0.509
        

        

          St4RTrack
          78.73
          0.205
          83.42
          0.185
        

        

          Flow4R
          81.00
          0.182
          79.87
          0.202
        

      
    

  

  

    We report performance on Point Odyssey and TUM-Dynamics after global median scaling. The best and second-best results are marked in bold and underlined.
  





Visualization



2D Visualization




  2D visualization example A
  2D visualization example B
  2D visualization example C
  2D visualization example D
  

    The point position map $\mathbf{P}$ captures scene geometry in the local space. The scene flow map $\mathbf{F}$ describes how each point moves from the current image to its pair, capturing both camera and object motions. The pose weight map $\mathbf{W}$ indicates which pixels are reliable for camera pose estimation. The confidence map $\mathbf{C}$ indicates the uncertainty of the predictions.
  





3D Visualization




  3D visualization
  

    We render the reconstructed and tracked points within a global coordinate system. Both dynamic elements (such as the train and human figures) and stationary structures (including the tree, ladder, wall, and table) exhibit high 3D consistency, demonstrating the effectiveness of our flow-centric tracking and reconstruction pipeline.
  






Our work is closely related to




Moving away from dedicated decoders for specific coordinate systems or timestamps, Flow4R adopts a symmetrical architecture that predicts local geometry and relative motion using a shared decoder and head.



Acknowledgments


This work was supported by the ERC Advanced Grant “SIMULACRON” (agreement #884679), the GNI Project “AI4Twinning”, the DFG project CR 250/26-1 “4DYoutube”, the Leibniz Supercomputing Centre (LRZ), and the UKRI AIRR programme. We would like to thank Weirong Chen, Dominik Muhle, and Linus Härenstam-Nielsen for valuable discussions throughout the project.