Matterport3D: Learning from RGB-D Data in Indoor Environments
1. Paper overview
In one sentence: Globally aligned panoramic scans turn whole buildings into reusable perception supervision, but their value depends on viewpoint coverage, annotation choices and the quality of evaluation targets. e-acquisitione-annotatione-keypointe-overlape-normal-resultse-normal-qualitativee-region
| At a glance | What to know |
|---|---|
| Research problem | Source description Indoor RGB-D learning was constrained by limited rooms, viewpoints, coverage and image quality. The paper asks what becomes learnable when whole buildings are scanned from stationary panoramic viewpoints and observations share a global geometric frame. Its contribution is a dataset and baseline study. e-identitye-properties |
| Core mechanism | Source description The release contains 194,400 RGB-D images from 10,800 panoramas, plus camera poses, skyboxes and textured reconstructions; the reported scene collection spans 2,056 rooms. e-acquisitione-properties |
| A key reported result | Surface-normal estimation on NYUv2: Mean 20.89°; median 13.79°; 42.29%, 67.82%, 77.16% below 11.25°, 22.5°, 30°. Mean/median angular error (degrees, lower better); pixels below angular thresholds (%, higher better).. NYUv2 test images; sequential SUNCG → Matterport3D → NYUv2 training. SUNCG → NYUv2: mean 22.07°, median 14.79°, thresholds 39.61%, 65.63%, 75.25%. The additional real-data stage improves every listed metric. The authors separately judge SUNCG → Matterport3D outputs visually cleaner; metric agreement with noisy targets and visual quality differ. e-normal-resultse-normal-qualitative |
| Reading caution | Author claim Registration accuracy is not objectively measured: the authors lack ground-truth poses and subjectively estimate average surface correspondence error of 1 cm or less, with rare errors of 10 cm or more. e-properties |
Core contributions
- Source description
The release contains 194,400 RGB-D images from 10,800 panoramas, plus camera poses, skyboxes and textured reconstructions; the reported scene collection spans 2,056 rooms. e-acquisitione-properties
- Source description
Region annotation followed by object-instance painting yields 50,811 instances. Freeform labels are consolidated into 40 canonical object categories mapped to WordNet synsets. e-annotation
- Source description
Five separate tasks probe correspondence, overlap retrieval, normals, room categories and semantic voxels. These experiments test distinct uses of the data rather than a jointly trained architecture. e-keypointe-overlape-normal-methode-regione-voxel
Figure 24. One reconstructed building supports several distinct layers of human annotation. Original paper, p. 23 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the four panels in the order printed: textured mesh, object instances, raw categories and canonical categories. The top-right colors separate individual objects; the bottom panels group surfaces by semantic category. Appendix A.5 explains the transition: workers paint and name objects, spelling and synonyms are consolidated, then narrower categories are collapsed into a common vocabulary. The caption identifies chair variants as an example of consolidation. Before this object-level work, annotators define room-like regions using floor polygons, as described in Section 3.2. The graphic presents supervision attached to existing geometry; its panels are not successive predictions from a neural network. e-annotatione-voxel
What it supports. The dataset offers both instance identity and category identity on a shared reconstruction. Canonicalization makes heterogeneous freeform annotations usable for a common label space. That distinction matters when selecting a benchmark: the dataset has 40 canonical object categories, while the demonstrated voxel network uses 20.
Where the evidence stops. These are annotations, not measured segmentation performance. The figure does not establish inter-annotator agreement or show the full mapping from raw labels to the evaluated 20-class subset.
2. Motivation
2.1 The problem and the proposed response
Indoor RGB-D learning was constrained by limited rooms, viewpoints, coverage and image quality. The paper asks what becomes learnable when whole buildings are scanned from stationary panoramic viewpoints and observations share a global geometric frame. Its contribution is a dataset and baseline study. e-identitye-properties
2.2 What this reading follows
Matterport3D is best understood as a system for producing supervision. The same registered building can supply matching image patches, overlapping camera views, surface-normal targets and semantic labels. Its five baseline tasks ask different questions about that supervision: whether descriptors transfer, whether retrieval benefits from geometric overlap, whether cleaner depth teaches better normals, and whether broader visual context helps semantic understanding. Read the figures as a chain from data construction to experimental evidence. The strongest lesson is conditional: added data or context often helps, yet noisy reference normals and views across room boundaries can change what a favorable score means. e-acquisitione-annotatione-keypointe-overlape-normal-resultse-normal-qualitativee-region
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Datasets |
| Architecture | Not applicable |
| Prediction paradigm | Not applicable |
| Quadrant | Not applicable |
3.1 Evidence-based assessment
Supports the recorded classification
The recorded dataset classification is supported by the acquisition and annotation contribution. Architecture, prediction paradigm and quadrant are not applicable: the paper evaluates separate perception baselines, without joint future/action prediction, inverse dynamics or an executable world-action model. e-acquisitione-annotatione-keypointe-overlape-normal-methode-regione-voxel
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Use the reconstruction to understand where supervision comes from
Begin with capture rather than with the neural networks. Each panorama collects multiple RGB-D views around one tripod location, and global registration places observations from different locations in one building coordinate frame. That shared frame allows the authors to derive matching keypoints and shared-surface overlap targets. Human effort supplies a different kind of supervision: room polygons define region identity, while painted mesh triangles define object instances and categories. The dataset therefore combines geometric targets derived from observations with semantic targets supplied by annotators. At learning time, separate baseline networks consume patches, frames, panoramas or voxel grids. At inference time, those networks output descriptors, rankings, normals or labels. No step in this chain turns a prediction into an executed robot action; the scientific contribution is the reusable perception supervision and its tested benefits. e-acquisitione-annotatione-keypointe-overlape-normal-methode-regione-voxel
Figure 9. Registered geometry supplies matching-patch supervision despite changes in viewpoint. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the left examples: the red links mark corresponding locations viewed from different camera positions. Then inspect the separate patch panel on the right. Its caption identifies the first two columns as matching patches and the third as a non-match. Section 4.1 specifies how training matches are obtained: SIFT locations must back-project within 0.02 m and have world normals within 100°. A ResNet-50 then maps each patch into a 512-dimensional descriptor, trained with an L2 hinge embedding loss in a triplet Siamese arrangement. The image therefore illustrates how a registered reconstruction provides labels for an image-based representation. e-keypointe-properties
What it supports. The useful ingredient is correspondence across changing views, rather than visual similarity alone. On the separate SUN3D evaluation in Table 1, Matterport3D pretraining followed by SUN3D fine-tuning gives 9.2% false positives at 95% recall, compared with 10.5% for SUN3D-only training.
Where the evidence stops. The displayed examples do not measure matching accuracy. Registration errors can affect these derived labels, and the reported transfer comparison does not independently isolate data volume from viewpoint diversity.
5.2 Separate the benefit of more training data from the benefit of a better target
The overlap experiment contains two comparisons that should be read separately. A comparison across SUN3D test rows changes the training data. A comparison across the final two columns within one row changes the training objective. The latter is the closer test of the proposed regression term: triplets teach relative ordering, whereas overlap regression also supervises the degree of shared visibility for positive pairs. The table shows improvement for every listed row, making this a useful mechanism-level result. Reader interpretation: it is still not a complete causal account, because the paper does not report repeated trials, full loss weights or matched compute. Keep the test distribution fixed when judging a training change. The much lower Matterport3D test scores are discussed as evidence of harder viewpoints, not as directly comparable transfer losses on SUN3D. e-overlape-training-gapse-overlap-appendix
5.3 Ask what the evaluation label rewards
For normals, the model sees color during inference but is judged against depth-derived orientation targets. The paper argues that noise in those targets can reward predictions that look less geometrically regular. This explains why Table 3 and Figure 11 must be interpreted together: the best measured model includes final NYUv2 fine-tuning, while the authors visually prefer the preceding Matterport3D stage. A related issue appears in region classification. The label belongs to the room containing the camera, even when a panorama shows neighboring rooms more clearly. Reader interpretation: adding useful visual information can still create conflict with a particular target definition. Neither example licenses discarding the metric. Instead, preserve the score, state the label semantics, and propose an independent diagnostic capable of distinguishing better prediction from closer agreement with a noisy or context-dependent reference. e-normal-methode-normal-resultse-normal-qualitativee-region
5.4 Training and inference
During training
All Matterport3D experiments use 61 training, 11 validation and 18 test scenes. The appendix illustrates these scene groups. Keypoint transfer additionally uses 17 SUN3D training scenes and eight held-out SUN3D test scenes. e-splite-keypoint
The keypoint ResNet-50 maps patches to 512-dimensional descriptors and trains as a triplet Siamese network with an L2 hinge embedding loss. The overlap ResNet-50 uses a distance-ratio triplet loss plus overlap regression for positive pairs whose overlap exceeds 0.1. e-keypointe-overlap
Normal prediction uses a VGG-16-based encoder, symmetric decoder, high-resolution shortcuts and matching pooling/unpooling masks. Training compares sequences of SUNCG, Matterport3D and NYUv2; only horizontal and downward Matterport3D views are used. Region classification trains ResNet-50 on 12 common categories, while voxel labeling follows the ScanNet architecture. e-normal-methode-regione-voxel
During inference
Keypoint inference produces descriptors for matching; overlap retrieval sorts candidate frames by learned feature distance, with smaller L2 distance representing greater overlap. Evaluation excludes candidate pairs with travel distance at most 0.5 m. The paper suggests future SLAM integration but reports retrieval quality, not a deployed loop-closure system. e-keypointe-overlap
The other networks predict normals from color, the region containing an image's camera viewpoint, or object labels from voxelized scenes. They provide perception outputs without a policy, action extraction stage or feedback controller. e-normal-methode-regione-voxel
5.5 Implementation flow
- Capture and align observations
A tripod rig uses three RGB and three depth cameras with different tilts and six rotational orientations, producing 18 RGB-D images per panorama. Integrated depth is aligned to 1280×1024 color images. Matterport processing stitches skyboxes, estimates six-degree-of-freedom poses through global bundle adjustment and reconstructs textured meshes. e-acquisition
- Turn geometry into semantic supervision
Annotators draw floor polygons and select region categories; the tool snaps boundaries to surfaces and extrudes them to ceilings. Workers paint and name object instances on region meshes, followed by expert correction. Spell checking, synonym consolidation and category collapsing produce canonical labels. e-annotatione-region-annotation
- Derive geometric learning targets
SIFT locations form matching patch pairs when their back-projections lie within 0.02 m and their world normals within 100°. View-overlap supervision instead counts shared surface observations within 5 cm. These are geometric training targets, not action labels. e-keypointe-overlap
- Construct semantic voxel examples
The baseline uses 20 classes and up-aligned subvolumes, rejecting samples below 2% occupancy or below 70% valid labels among occupied voxels. Eight rotations expand 52,355 samples to 418,840. The stated physical and grid dimensions are inconsistent and require resolution before implementation. e-voxel
6. Experiments & results
Matterport3D converts panoramic RGB-D scans of 90 buildings into registered images, reconstructed surfaces and semantic annotations. Its five separate perception baselines test whether broad viewpoint coverage and cleaner geometry improve matching, retrieval, normal prediction and semantic understanding. The results support useful supervision and transfer, while leaving action execution and learned dynamics untested (e-acquisition, e-annotation, e-keypoint, e-overlap, e-normal-results, e-region, e-voxel).
6.1 Read the original evidence
Table 3. Adding Matterport3D between synthetic pretraining and NYUv2 fine-tuning improves the reported normal metrics. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Treat Train Set 1, 2 and 3 as an ordered training sequence; a dash means there is no stage in that column. MP abbreviates Matterport3D. Mean and median angular errors should decrease, while the three rightmost columns report percentages of pixels below 11.25°, 22.5° and 30° and should increase. Compare the second row, SUNCG followed by NYUv2, with the last row, which inserts Matterport3D before the final NYUv2 stage. Section 4.3 specifies the encoder-decoder architecture and restricts the Matterport3D training views to horizontal and downward directions. Every row in this table is evaluated on NYUv2. e-normal-methode-normal-results
What it supports. The final schedule reports 20.89° mean error and 42.29% of pixels below 11.25°, compared with 22.07° and 39.61% for SUNCG→NYUv2. It leads all five displayed metrics. This supports the usefulness of an intermediate real-data stage under the reported protocol.
Where the evidence stops. Do not confuse the MP-only row with SUNCG→MP. Also preserve the source's minor discrepancy: the SUNCG→MP 22.5° entry is 53.36 here but 53.35 in Table 4. No uncertainty accompanies either value.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Keypoint matching on SUN3D Eight held-out SUN3D scenes; Matterport3D pretraining followed by SUN3D fine-tuning. | 9.2% False-positive rate at 95% recall; lower is better. | SUN3D-only ResNet-50: 10.5%; Matterport3D-only: 10.6%; SIFT: 37.8%; SURF: 46.8%. Transfer reduces error by 1.3 percentage points versus SUN3D-only training. No uncertainty or repeated-run statistics are reported. e-keypoint |
| View-overlap retrieval and regression-loss ablation Table 2 training/test combinations; candidates separated by more than 0.5 m travel distance. | Matterport3D + SUN3D → SUN3D: 85.45 with triplet + regression. Normalized discounted cumulative gain, on the paper's displayed scale; higher is better. | Same data, triplet only: 84.10; SUN3D-only with regression: 83.34. Matterport3D → Matterport3D improves from 48.8 to 53.6 when regression is added. Within-row comparisons support the added loss; cross-test-dataset scores have different candidate distributions and are not interchangeable measures of transfer. e-overlap |
| Surface-normal estimation on NYUv2 NYUv2 test images; sequential SUNCG → Matterport3D → NYUv2 training. | Mean 20.89°; median 13.79°; 42.29%, 67.82%, 77.16% below 11.25°, 22.5°, 30°. Mean/median angular error (degrees, lower better); pixels below angular thresholds (%, higher better). | SUNCG → NYUv2: mean 22.07°, median 14.79°, thresholds 39.61%, 65.63%, 75.25%. The additional real-data stage improves every listed metric. The authors separately judge SUNCG → Matterport3D outputs visually cleaner; metric agreement with noisy targets and visual quality differ. e-normal-resultse-normal-qualitative |
| Region-type classification with wider field of view Matterport3D held-out test scenes; 12 common region categories, single images versus panoramas. | Panorama: hallway 66.6%, bedroom 62.9%, bathroom 65.4%. Per-region-category accuracy (% correct). | Single image: hallway 44.7%, bedroom 43.7%, bathroom 44.0%. Lounge falls from 21.7% to 15.4%; familyroom from 16.7% to 11.4%. Wider context helps most categories but can expose neighboring regions. The paper proposes this confusion as an explanation, without a controlled boundary analysis. e-region |
| Semantic voxel labeling on Matterport3D 20-class baseline evaluated on Matterport3D test scenes. | Total 70.3%; floor 92.6%; wall 78.8%; sofa 0.004%. Reported voxel-label prediction accuracy. | Table 6 supplies per-class results but no competing model or matched ablation. The total is not reported as mean class accuracy or IoU. Large class differences prevent treating it as uniform object recognition quality. e-voxel |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 2. Overlap regression improves retrieval in every listed training/test combination. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read across a row first, holding the listed training and test datasets fixed. The last two columns compare the distance-ratio triplet objective with that objective plus regression of geometric overlap. Higher normalized discounted cumulative gain means a better retrieval ordering; keep the numbers on the paper's displayed scale. Only then compare training sets within the SUN3D test rows. The bottom row uses Matterport3D for testing and therefore changes the candidate distribution. Section 4.2 defines the target from shared surface pixels and limits evaluation to image pairs more than 0.5 m apart in travel distance. This is a ranked-retrieval experiment. e-overlape-training-gaps
What it supports. For Matterport3D testing, adding regression raises the reported score from 48.8 to 53.6. With Matterport3D + SUN3D training and SUN3D testing, it raises 84.10 to 85.45. The within-row improvements support using overlap magnitude as supervision beyond a relative triplet ordering.
Where the evidence stops. The table does not report uncertainty, repeat counts or complete loss weighting. A higher retrieval score does not establish more accurate SLAM trajectories, and bottom-row scores should not be compared as if they shared the SUN3D test distribution.
Figure 11. Visual regularity and agreement with the benchmark reference can favor different training schedules. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow each scene from its color image to the Ground Truth column, then compare the four model outputs. The labels name their training histories: synthetic SUNCG alone, SUNCG→NYU, SUNCG→MP and SUNCG→MP→NYU. Compare broad walls with smaller boundaries around the sink, furniture and wall-mounted objects. The reference column itself contains substantial texture-like variation in planar regions. The authors use these examples to argue that Matterport3D fine-tuning produces cleaner planes while retaining detail. Table 3 must be read alongside the image: the model further fine-tuned on NYUv2 has the best numerical score, despite the authors' preference for the preceding model's visual quality. e-normal-methode-normal-resultse-normal-qualitative
What it supports. The source distinguishes two judgments. SUNCG→MP is described as visually cleaner, whereas SUNCG→MP→NYU achieves better measured agreement with NYUv2 normals. The supplementary comparisons repeat this concern about noisy depth supervision. Visual plausibility and benchmark accuracy are therefore separate observations in this experiment.
Where the evidence stops. These selected images are not a blinded visual study, and smooth-looking normals are not automatically physically correct. The colors encode surface orientation rather than an error heatmap; do not read a color difference as a calibrated error magnitude.
Table 5. Panoramic context improves most region categories while hurting lounge and familyroom classification. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Compare the single and pano rows within each region column. The target is the category of the region containing the camera, not necessarily the most visually prominent room. Section 4.4 trains a ResNet-50 classifier using the 12 most common categories and labels images from the manually defined regions. Hallway, bedroom and bathroom show substantial gains with a panorama. Lounge and familyroom move in the opposite direction. That pattern is scientifically informative because broader field of view can reveal an adjacent room without changing the camera's label. Read the table as twelve category-specific comparisons; it supplies no aggregate accuracy or weighting rule. e-regione-region-annotatione-training-gaps
What it supports. Hallway accuracy increases from 44.7% to 66.6%, and bathroom from 44.0% to 65.4%. Lounge drops from 21.7% to 15.4%, while familyroom drops from 16.7% to 11.4%. These exceptions qualify any blanket claim that more context is always beneficial.
Where the evidence stops. The neighboring-room explanation is the authors' hypothesis, not an isolated causal test. The PDF does not give a full panorama input-processing configuration or uncertainty for these per-category comparisons.
7. Analysis & limitations
7.1 What the evidence leaves open
Registration accuracy is not objectively measured: the authors lack ground-truth poses and subjectively estimate average surface correspondence error of 1 cm or less, with rare errors of 10 cm or more. e-properties
Stationary, mostly-home captures are valuable perception data but do not establish generalization to moving sensors, dynamic environments or action-conditioned prediction. The benchmarks also lack reported uncertainty. e-propertiese-keypointe-overlape-normal-resultse-regione-voxel
Section 4.5 prints “2 cm³ voxels” alongside 1.5×1.5×3 m subvolumes of 31×31×62 voxels; these do not define a consistent resolution. Table 3 gives 53.36 for SUNCG→MP at the 22.5° threshold, whereas Table 4 gives 53.35 for the corresponding cross-dataset entry. Neither discrepancy is silently corrected. e-voxele-normal-results
Figure 19's caption says the descriptor network was trained for keypoint matching on the Matterport3D test set, while Section 4.1 states training on 61 scenes. Its t-SNE visualization cannot resolve that wording ambiguity or establish an independent test protocol. e-keypointe-keypoint-appendix
7.2 Questions for discussion
- Would equal-size, equal-compute training preserve the benefit attributed to Matterport3D's viewpoint diversity?
- How should normal estimators be ranked when a noisy reference favors visually less regular predictions?
- Does panoramic region confusion concentrate near manually defined region boundaries?
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction requires the registered RGB-D images, depth-derived supervision, canonical label mappings and scene split identifiers, plus the relevant SUN3D/SUNCG/NYUv2 data. Split figures show buildings but do not supply machine-readable scene identifiers in the PDF. e-acquisitione-annotatione-splite-normal-method
The retained paper omits complete optimizer settings, learning-rate schedules, batch sizes, loss weights, seeds, hardware and runtimes. It cites baseline architectures without specifying a complete executable training configuration. Resolve the voxel dimensions before testing that baseline. e-training-gapse-voxel
The authors state that data and code are public and describe academic-research distribution permission for home imagery. Those statements do not specify a complete license or prove current artifact availability. e-access
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Hold retrieval data fixed while testing overlap regression
Reader-proposed check, not executed: use the documented Matterport3D scene split and a single fixed set of training triplets to compare triplet-only and triplet-plus-overlap-regression models. Match architecture, initialization, training steps and candidate lists, tune loss weighting only on validation scenes, and repeat across seeds. Keep the >0.5 m travel-distance rule and report nDCG overall and by geometric-overlap bin. The hypothesis predicts a consistent gain from regression on held-out retrieval rankings. If that gain disappears under fixed data and compute, Table 2's mechanism interpretation would need qualification. Document the ranking cutoff and gain formula because the PDF does not fully specify them. e-splite-overlape-training-gaps
Check 2: Test whether final NYUv2 fine-tuning improves geometry or reference agreement
Reader-proposed check, not executed: start from one SUNCG→Matterport3D checkpoint and compare it with a copy fine-tuned on NYUv2, using fixed test images and the same valid-depth masks. Report the paper's angular metrics separately from a preregistered diagnostic of planar consistency and boundary preservation, with reference regions selected independently of model outputs. Include both NYUv2 and Matterport3D test scenes to expose domain effects. The source's interpretation predicts better NYUv2 angular scores after fine-tuning but possible loss of planar regularity. If both independent geometry diagnostics and angular scores improve, that would weaken the claim that the final stage mainly introduces undesirable noise. e-normal-methode-normal-resultse-normal-qualitative
8.3 Reading coverage
Visual audit: All 25 original PDF pages were rendered and visually inspected, including the title/version block, Figures 1–26, Tables 1–6, every split montage on pp. 13–17, and all appendix explanations. All six final crops were separately viewed. Table headers, inequality thresholds, normal-model column ordering and annotation-panel meanings were cross-checked against the source text and captions. Figure 24 supplies the dataset-method visual; no new architecture diagram was created. The voxel-dimension inconsistency, the Table 3/4 rounding discrepancy and Figure 19's test-set wording are retained in the base report. External artifacts and experiments remain outside this reading.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Title, authors, abstract and arXiv version stamp (p. 1)
- Sections 1–2: Introduction and Background and Related Work (pp. 1–2)
- Sections 3.1–3.3: Acquisition, Semantic Annotation and Dataset Properties (pp. 2–5)
- Sections 4.1–4.5: All five learning tasks (pp. 5–9)
- Sections 5–6: Conclusion and Acknowledgements (p. 8)
- References (pp. 9–10)
- Appendix A and A.1–A.5 (pp. 11–12)
- Supplementary Figures 14–26, including every split illustration (pp. 13–25)
Outside the original text pass
- Text extraction does not reconstruct figure pixels or preserve equation/table layout; downstream reading must inspect the retained original PDF.
- Floorplan and split illustrations on PDF pages 13–17 are preserved in the original PDF and require visual reading.
- Availability of external supplements, code, datasets and videos beyond the retained PDF has not been fully verified.
- Identity: the observed title matches exactly. The inspected edition is arXiv:1709.06158v1 [cs.CV], 18 September 2017. The title page prints Angel Chang and Thomas Funkhouser; the catalog adds X. and A. respectively. All other author identities agree. The catalog's 3DV publication was not separately inspected, so equivalence to the venue edition is not asserted.
- Text extraction does not reconstruct figure pixels or preserve equation/table layout; the original PDF was therefore visually inspected on all 25 pages.
- Floorplan and split illustrations on PDF pages 13–17 are preserved in the original PDF and were visually inspected.
- Availability of external supplements, code, datasets and videos beyond the retained PDF has not been fully verified. No external material or code was inspected and no experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1, title block, version stamp, Abstract and Section 1
Exact title and nine authors are printed, with three university affiliations. The artifact identifies arXiv:1709.06158v1, 18 September 2017, and introduces a dataset for indoor perception.
Go to primary source ↓e-acquisitionPDF pp. 2–3, Section 3.1 and Figure 2
Three color/depth camera pairs, six orientations and 18 RGB-D images per panorama; integrated 1280×1024 depth, HDR color, global bundle adjustment, skyboxes and meshes. Totals: 90 buildings, 10,800 panoramas and 194,400 RGB-D images.
Go to primary source ↓e-annotationPDF p. 3, Section 3.2 and Figures 3–4; p. 12, A.5; p. 23, Figure 24
Region polygons precede crowdsourced object painting and expert verification. The 50,811 instance annotations have 1,659 freeform labels consolidated to 40 categories; Figure 24 separates texture, instance, raw-category and canonical-category views.
Go to primary source ↓e-propertiesPDF pp. 3–5, Section 3.3 and Figures 5–8
The source describes static tripod capture, mainly private homes, 2,056 rooms, near-uniform viewpoints and repeated observations. It explicitly lacks ground-truth camera poses and calls its ≤1 cm average registration estimate subjective; occasional ≥10 cm misalignments occur.
Go to primary source ↓e-splitPDF p. 5, Section 4 opening; p. 11, Appendix A opening; pp. 13–17, Figures 14–18
Experiments split scenes 61/11/18 for training/validation/testing. The appendix shows the three training montages and separate validation/test montages, without printed machine-readable scene IDs.
Go to primary source ↓e-keypointPDF p. 5, Section 4.1 and Figure 9; p. 6, Table 1, all rows
SIFT matches use 0.02 m proximity and normals within 100°. ResNet-50 yields 512-dimensional descriptors under triplet L2 hinge training. Eight held-out SUN3D scenes give errors 46.8, 37.8, 10.6, 10.5 and 9.2 percent at 95% recall for the listed baselines.
Go to primary source ↓e-overlapPDF p. 6, Section 4.2, unnumbered overlap equation and Table 2, all rows/columns
Overlap uses the minimum of directed shared-surface pixel counts within 5 cm. ResNet-50 triplet distance-ratio training adds regression for overlap >0.1. Retrieval evaluation uses travel distance >0.5 m and nDCG; Table 2 reports the training/test combinations and ablations.
Go to primary source ↓e-normal-methodPDF pp. 6–8, Section 4.3; p. 11, A.3
A VGG-16-based encoder and symmetric decoder predict normals from RGB, using shortcuts and pooling masks. Training compares synthetic SUNCG, Matterport3D and NYUv2 stages; Matterport3D uses horizontal/downward views.
Go to primary source ↓e-normal-resultsPDF p. 7, Table 3, all rows; Table 4 and adjoining cross-dataset paragraph
SUNCG→MP→NYUv2 reaches 20.89°/13.79° mean/median and 42.29/67.82/77.16 percent at the three thresholds. Table 4 models both start with synthetic pretraining. Tables 3 and 4 print 53.36 and 53.35 for the corresponding SUNCG→MP evaluation threshold.
Go to primary source ↓e-normal-qualitativePDF p. 7, Figure 11 and caption; p. 8, Section 4.3 continuation; pp. 11–12, A.3; pp. 20–22, Figures 21–23
The authors judge SUNCG→MP normals cleaner on planar areas and detailed objects; subsequent NYUv2 fine-tuning improves numerical agreement but can add noise. Supplementary columns explicitly identify datasets and show missing-depth regions.
Go to primary source ↓e-regionPDF p. 8, Section 4.4; p. 9, Table 5, all columns
ResNet-50 predicts the camera-containing region among 12 common categories. Panoramas improve most per-class accuracies; lounge and familyroom decline, attributed by the authors to neighboring-region confusion.
Go to primary source ↓e-region-annotationPDF p. 12, A.4; pp. 24–25, Figures 25–26
Manually specified region polygons and categories are snapped/extruded into mesh region labels; the figures distinguish category from instance segmentation.
Go to primary source ↓e-voxelPDF p. 8, Section 4.5 and Figure 12; p. 9, Table 6
The 20-class ScanNet-style baseline uses occupancy/annotation rejection, eight rotations and 52,355 original samples. Physical/grid dimensions are printed inconsistently. Table 6 reports total accuracy 70.3%, floor 92.6%, wall 78.8% and sofa 0.004%, with no competing baseline.
Go to primary source ↓e-keypoint-appendixPDF p. 11, A.1 and Figure 13; p. 18, Figure 19 and caption
The appendix illustrates viewpoint variation and a t-SNE descriptor embedding. Figure 19's caption uses the phrase trained for keypoint matching on the Matterport3D test set, unlike the explicit training split in Section 4.1.
Go to primary source ↓e-overlap-appendixPDF p. 11, A.2; p. 19, Figure 20 and caption
Eight selected cameras are yellow; other camera marks range from thin cyan at no overlap to thick red at ≥20% overlap. These illustrate geometric overlap within one scene rather than retrieval accuracy.
Go to primary source ↓e-training-gapsPDF pp. 5–8, Sections 4.1–4.5; pp. 11–12, Appendix A.1–A.5
The task descriptions give architectures, some objectives, geometric thresholds and selected data-generation settings, but no complete optimizer schedule, loss weighting, hardware/runtime account or executable configuration.
Go to primary source ↓e-accessPDF p. 2, footnote 2; pp. 4–5, Personal Living Spaces; p. 8, Conclusion and Acknowledgements
The authors link public data/code and state permission to distribute private-home imagery for academic research; the data were gifted by Matterport for the academic community.
Go to primary source ↓8.5 Primary sources
Matterport3D: Learning from RGB-D Data in Indoor Environments ↗
PDF · 8,676 extracted words
Source fingerprint
010065bd3541b099bc3cbb484fa25ccdb09d7995fabe5b9a180df19d76b103a6