Add current teaser and method figures
Browse files- .gitattributes +3 -0
- README.md +24 -2
- assets/hcvae.png +3 -0
- assets/method_overview.png +3 -0
- assets/teaser.png +3 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
assets/teaser.png filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
assets/method_overview.png filter=lfs diff=lfs merge=lfs -text
|
| 38 |
+
assets/hcvae.png filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -26,6 +26,29 @@ The inference code and installation instructions are maintained at
|
|
| 26 |
inputs are hosted at
|
| 27 |
[datasets/hongchi/Fire3D](https://huggingface.co/datasets/hongchi/Fire3D).
|
| 28 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 29 |
## Model Bundle
|
| 30 |
|
| 31 |
The repository contains the scene perception checkpoint, the three
|
|
@@ -54,8 +77,7 @@ frozen TRELLIS.2 SC-VAE decoding path.
|
|
| 54 |
| TRELLIS.2 PBR decoder | `external/trellis2/tex_dec_next_dc_f16c32_fp16.safetensors` |
|
| 55 |
|
| 56 |
Public filenames intentionally do not encode private training iteration
|
| 57 |
-
numbers.
|
| 58 |
-
[GitHub release](https://github.com/xiahongchi/Fire3D#method).
|
| 59 |
|
| 60 |
The default protocols use classifier-free guidance 3, flow and VAE batches of
|
| 61 |
16, and the batch CuMesh postprocessor. The iTHOR, Imaginarium, and ScanNet++
|
|
|
|
| 26 |
inputs are hosted at
|
| 27 |
[datasets/hongchi/Fire3D](https://huggingface.co/datasets/hongchi/Fire3D).
|
| 28 |
|
| 29 |
+
## Overview
|
| 30 |
+
|
| 31 |
+

|
| 32 |
+
|
| 33 |
+
**Teaser.** Fire3D reconstructs a compositional 3D scene from a single RGB
|
| 34 |
+
image or casual RGB video. The output contains object-level pose, geometry,
|
| 35 |
+
and material assets that can be rendered, edited, and used in simulation.
|
| 36 |
+
|
| 37 |
+

|
| 38 |
+
|
| 39 |
+
**Method overview.** Fire3D lifts RGB-D observations into a shared 3D scene
|
| 40 |
+
representation, predicts object instances and 6-DoF poses, and reconstructs
|
| 41 |
+
the sparse structure, shape, and PBR fields with three cascaded flow-matching
|
| 42 |
+
models. Batched decoders and mesh post-processing produce the final textured
|
| 43 |
+
scene assets.
|
| 44 |
+
|
| 45 |
+

|
| 46 |
+
|
| 47 |
+
**HC-VAE.** The Hierarchical Compression VAE reduces each sparse SC-VAE shape
|
| 48 |
+
or material field from a `32^3 x 32` representation to an `8^3 x 64` latent.
|
| 49 |
+
This 32x reduction makes scene-level batched flow sampling practical while
|
| 50 |
+
retaining the frozen TRELLIS.2 decoding path.
|
| 51 |
+
|
| 52 |
## Model Bundle
|
| 53 |
|
| 54 |
The repository contains the scene perception checkpoint, the three
|
|
|
|
| 77 |
| TRELLIS.2 PBR decoder | `external/trellis2/tex_dec_next_dc_f16c32_fp16.safetensors` |
|
| 78 |
|
| 79 |
Public filenames intentionally do not encode private training iteration
|
| 80 |
+
numbers.
|
|
|
|
| 81 |
|
| 82 |
The default protocols use classifier-free guidance 3, flow and VAE batches of
|
| 83 |
16, and the batch CuMesh postprocessor. The iTHOR, Imaginarium, and ScanNet++
|
assets/hcvae.png
ADDED
|
Git LFS Details
|
assets/method_overview.png
ADDED
|
Git LFS Details
|
assets/teaser.png
ADDED
|
Git LFS Details
|