hongchi commited on
Commit
da94067
·
verified ·
1 Parent(s): 0e4e95f

Add current teaser and method figures

Browse files
.gitattributes CHANGED
@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ assets/teaser.png filter=lfs diff=lfs merge=lfs -text
37
+ assets/method_overview.png filter=lfs diff=lfs merge=lfs -text
38
+ assets/hcvae.png filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -26,6 +26,29 @@ The inference code and installation instructions are maintained at
26
  inputs are hosted at
27
  [datasets/hongchi/Fire3D](https://huggingface.co/datasets/hongchi/Fire3D).
28
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
29
  ## Model Bundle
30
 
31
  The repository contains the scene perception checkpoint, the three
@@ -54,8 +77,7 @@ frozen TRELLIS.2 SC-VAE decoding path.
54
  | TRELLIS.2 PBR decoder | `external/trellis2/tex_dec_next_dc_f16c32_fp16.safetensors` |
55
 
56
  Public filenames intentionally do not encode private training iteration
57
- numbers. The current method and HC-VAE figures are available in the
58
- [GitHub release](https://github.com/xiahongchi/Fire3D#method).
59
 
60
  The default protocols use classifier-free guidance 3, flow and VAE batches of
61
  16, and the batch CuMesh postprocessor. The iTHOR, Imaginarium, and ScanNet++
 
26
  inputs are hosted at
27
  [datasets/hongchi/Fire3D](https://huggingface.co/datasets/hongchi/Fire3D).
28
 
29
+ ## Overview
30
+
31
+ ![Fire3D reconstructs compositional, simulation-ready 3D scenes from RGB-D observations.](assets/teaser.png)
32
+
33
+ **Teaser.** Fire3D reconstructs a compositional 3D scene from a single RGB
34
+ image or casual RGB video. The output contains object-level pose, geometry,
35
+ and material assets that can be rendered, edited, and used in simulation.
36
+
37
+ ![Fire3D end-to-end perception and reconstruction method.](assets/method_overview.png)
38
+
39
+ **Method overview.** Fire3D lifts RGB-D observations into a shared 3D scene
40
+ representation, predicts object instances and 6-DoF poses, and reconstructs
41
+ the sparse structure, shape, and PBR fields with three cascaded flow-matching
42
+ models. Batched decoders and mesh post-processing produce the final textured
43
+ scene assets.
44
+
45
+ ![Fire3D Hierarchical Compression VAE architecture.](assets/hcvae.png)
46
+
47
+ **HC-VAE.** The Hierarchical Compression VAE reduces each sparse SC-VAE shape
48
+ or material field from a `32^3 x 32` representation to an `8^3 x 64` latent.
49
+ This 32x reduction makes scene-level batched flow sampling practical while
50
+ retaining the frozen TRELLIS.2 decoding path.
51
+
52
  ## Model Bundle
53
 
54
  The repository contains the scene perception checkpoint, the three
 
77
  | TRELLIS.2 PBR decoder | `external/trellis2/tex_dec_next_dc_f16c32_fp16.safetensors` |
78
 
79
  Public filenames intentionally do not encode private training iteration
80
+ numbers.
 
81
 
82
  The default protocols use classifier-free guidance 3, flow and VAE batches of
83
  16, and the batch CuMesh postprocessor. The iTHOR, Imaginarium, and ScanNet++
assets/hcvae.png ADDED

Git LFS Details

  • SHA256: 67ed55cff8b3fd41b40387954c107ed11af50e9901c61bde1a992383a71fbe5e
  • Pointer size: 131 Bytes
  • Size of remote file: 178 kB
assets/method_overview.png ADDED

Git LFS Details

  • SHA256: cebb1538f46e5cd2adf56adc029108fbe9f936392476d13c791e90f321f81bb3
  • Pointer size: 131 Bytes
  • Size of remote file: 726 kB
assets/teaser.png ADDED

Git LFS Details

  • SHA256: 7c44cbab6ca92382f1fb2d56b3628b48471e7b7501f5a7c15632aef7d8aca2b2
  • Pointer size: 131 Bytes
  • Size of remote file: 656 kB