Habitat: A Platform for Embodied AI Research
1. Paper overview
In one sentence: A modular, fast simulator makes training scale and dataset choice experimentally visible, while the navigation conclusions remain conditional on idealized sensing and a specific benchmark protocol. e02e03e04e05e13e14e16e18
| At a glance | What to know |
|---|---|
| Research problem | Source description Embodied-agent comparisons were fragmented by simulator-specific datasets, tasks, sensor configurations and slow rendering. Habitat makes these choices explicit and interchangeable so researchers can investigate training scale and transfer under a common protocol. e02 |
| Core mechanism | |
| A key reported result | PointGoal navigation on Gibson and Matterport3D test scenes: Depth PPO: Gibson 0.79 SPL / 0.89 success; Matterport3D 0.54 / 0.69. Mean SPL and episode success rate; higher is better. Same-dataset training/testing; idealized sensors and noise-free actions; PPO trained for 75 million steps, with best validation checkpoints. RGBD SLAM: 0.51 / 0.62 and 0.39 / 0.47. RGBD PPO: 0.70 / 0.80 and 0.42 / 0.53. RGB PPO: 0.46 / 0.64 and 0.30 / 0.42; Blind PPO: 0.42 / 0.62 and 0.25 / 0.35. Depth PPO beats this SLAM baseline by 0.28 and 0.15 absolute SPL, calculated from Table 2. Both algorithm and visual sensors differ; RGBD PPO provides a sensor-matched comparison. Table 2 prints no per-cell uncertainty. e06e07e11e14e18 |
| Reading caution |
Core contributions
Figure 1. Habitat organizes embodied-AI experiments into reusable layers. Original paper, p. 2 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the right-hand Habitat box from bottom to top. Generic Dataset Support provides the scene assets that Habitat-Sim loads and renders; Habitat-API supplies the task-facing layer above it. The upward arrows express software layering, not a neural forward pass or an action-feedback loop. The three blue bands on the left place existing datasets, simulators and tasks at corresponding levels. Section 3 makes the separation concrete: scenes become a common hierarchical graph, while Task defines permitted observations, termination and scoring. An agent's policy operates within this environment; it is not the object represented by the three stacked labels. e02e03e04e16
What it supports. The reusable contribution is a common route from 3D assets to evaluable embodied tasks. This organization supports changing datasets or agent configurations without redefining every layer, which is why the later cross-dataset comparisons are a meaningful demonstration of the platform's intended use.
Where the evidence stops. The illustrated task examples situate the platform in prior work; they are not a table of tasks experimentally solved in this paper. The reported empirical study concerns PointGoal navigation, and the arrows do not depict a learned world model.
2. Motivation
2.1 The problem and the proposed response
Embodied-agent comparisons were fragmented by simulator-specific datasets, tasks, sensor configurations and slow rendering. Habitat makes these choices explicit and interchangeable so researchers can investigate training scale and transfer under a common protocol. e02
2.2 What this reading follows
Habitat is easiest to understand as a controlled experimental environment. It turns scanned indoor spaces into sensor observations, gives agents a consistent action interface, and separates that machinery from the rules used to score a task. The paper then uses this infrastructure to revisit a substantive question: does learned navigation still lag a classical SLAM pipeline when training continues much longer? Follow the stack diagram before the speed table, then distinguish validation learning curves from held-out test results. Finally, read the transfer matrix with its test column fixed. That sequence reveals both the depth agent's strength and the limits of attributing it to a single cause. e02e03e04e05e13e14e16e18
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Benchmarks & simulators |
| Architecture | Not applicable |
| Prediction paradigm | Not applicable |
| Quadrant | Not applicable |
3.1 Evidence-based assessment
Conflicts with the recorded classification
Benchmarks & simulators and Navigation benchmarks are supported. Architecture, prediction paradigm and quadrant are correctly Not applicable: Habitat supplies simulation infrastructure, and its PPO baseline is a policy rather than a joint future/action model. Physics simulation overstates this edition's demonstrated scope: object-physics integration is future work. This is a partial conflict; the recorded snapshot remains unchanged. e02e03e10e17
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Build the environment before choosing the learner
Begin with the distinction between scene state and agent knowledge. Habitat-Sim holds a scene graph so it can render observations and resolve movement, but the PointGoal agent is not given a ground-truth map. Habitat-API then adds the goal, episode rules and success measures around those observations. This arrangement lets a PPO policy and a classical navigation pipeline interact with a common simulator while retaining different internal representations. Figure 1's upward arrows therefore describe the organization of software, not a sequence of neural modules. As a reader interpretation, the platform's scientific value is that it makes experimental assumptions easier to hold fixed or deliberately change. It does not make a benchmark automatically realistic: the chosen sensor and collision settings still determine what problem the agent actually solves. e02e03e04e06e07e12
5.2 Separate privileged training reward from deployed feedback
The static goal is paired with idealized GPS and compass, so even a Blind agent can relate its current position to the target. The PPO baseline combines its relative goal vector with a visual embedding when vision is available, using a GRU actor and linear critic. During training, the simulator additionally supplies geodesic progress reward: decreasing distance helps, every step costs 0.01, and reaching the goal adds 10. This reward signal is restricted to training environments. During evaluation the agent issues actions, observes their consequences and must learn when to stop. Collision sliding matters because a forward command need not produce its nominal displacement. The SPL equation then rewards successful short paths; it cannot be replaced with success alone without losing the efficiency distinction visible in Table 2. e06e07e08e10e11e14
5.3 Ask what each comparison holds constant
Figure 3 varies the amount of training experience while evaluating on validation scenes; Table 2 instead reports held-out test performance after checkpoint selection. Those two forms of evidence jointly support a budget-dependent comparison with the chosen SLAM baseline. Figure 5 introduces another axis: training dataset. Read down a fixed test column to compare training sources before reading across a row to compare target distributions. Gibson-trained Depth scoring better on MP3D than MP3D-trained Depth motivates the authors' easier-episode explanation, because Gibson episodes are shorter. That explanation remains a hypothesis: the matrix does not match episode difficulty, and changing training data changes more than one factor. My interpretation is that the study establishes a useful empirical interaction among training scale, sensing and data, rather than one universal ranking of navigation algorithms. e09e11e13e14e16e18
5.4 Training and inference
During training
PPO uses geodesic progress reward, a success bonus of 10 and a per-step penalty of −0.01. Training partitions scenes over eight Gibson or six Matterport3D workers, shuffles blocks of 500 episodes per scene, and accumulates 75 million steps across workers. Stop is learned; evaluation uses five random seeds and validation-selected checkpoints. e10e11e13e14
No pretrained or frozen module or staged pretraining procedure is specified in the main paper. Detailed network and implementation settings are deferred to the separate supplement. e10e20
During inference
At each simulated step, the actor uses observations and the goal representation to issue an action; simulation applies motion/collisions and returns new observations. No ground-truth map is supplied. Reward is restricted to training environments. Success requires stop within 0.2 m geodesic distance; episodes fail if unsuccessful after 500 actions. e06e07e08e10
5.5 Implementation flow
- Normalize scenes and render observations
Habitat-Sim loads datasets into a hierarchical scene graph. Its C++ backend uses Magnum and a multi-attachment shader to produce RGB, depth and semantic masks in one render pass when sensor parameters permit sharing. These are rendered observations, not learned future predictions. e03
- Separate the task from simulation
Habitat-API's Task extends observations/actions and supplies termination and success measures with read-only simulator and episode-data access. Episode records scene, initial pose and goal, optionally a shortest path; Environment gathers the task context. e04
- Specify embodiment and sensing
The PointGoal agent is a cylinder 0.2 m in diameter and 1.5 m tall. Turns are 10 degrees and forward commands request 0.25 m. Actions are noise-free, but collisions permit partial progress and sliding. RGB/depth use 256×256 images and a 90-degree field of view; even Blind receives GPS and compass. e06e07
- Construct nontrivial episodes
Scenes do not overlap across train/validation/test. Gibson is curated to 106 of 572 scenes; Matterport3D uses public splits. Start-to-goal geodesic distance is restricted to 1–30 m. Rejection sampling reduces nearly straight episodes, with geodesic/Euclidean distance in [1, 1.1], from 37% to 10% in Gibson. e09
- Map observations to actions
PPO combines a CNN visual embedding and relative goal vector with a GRU actor and linear critic. The CNN uses 8×8, 4×4 and 3×3 convolutions, ReLUs and a linear layer. The comparison SLAM agent uses the Mishkin et al. RGBD pipeline with ORB-SLAM2 localization. e10e12
6. Experiments & results
Habitat separates 3D scene ingestion, fast sensor simulation and task evaluation. Its PointGoal experiments show that longer PPO training changes the comparison with one SLAM baseline, with depth-only agents strongest under idealized sensing. Cross-dataset results expose training-distribution effects. The contribution is experimental infrastructure, not a learned world-action predictor (e02, e03, e04, e13, e14, e16, e18).
The supplied main paper provides sensor comparisons, training-budget curves and transfer diagnostics rather than an isolated network-component ablation. Further noisy-depth experiments and behavioral examples are deferred to a separate supplement that was not supplied; the edition therefore uses the main-paper diagnostics and makes no claim about those supplemental experiments. e13e14e15e16e20
6.1 Read the original evidence
Table 1. Sensor choice, resolution and concurrency determine simulator throughput. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. First choose the sensor row, then compare the same resolution under the two process-count headers. These columns report frames per second, so larger values indicate faster rendering. The caption specifies one Matterport3D scene, 17DRP5sb8fy, an Intel Xeon E5-2690 v4 CPU and an Nvidia Titan Xp GPU. The five processes share that GPU. The headline 10,592 fps belongs to 128-pixel RGB rendering; the navigation experiments use 256×256 visual inputs. For a closer sensor/resolution reference, read the RGB+depth row at 256: it gives 1,042 fps for one process and 1,774 for five. e03e05e07e11
What it supports. The table demonstrates substantial simulator throughput under specified conditions, and it shows why a single headline fps number is incomplete. Adding depth or raising resolution reduces throughput within each process configuration. Faster observation generation provides room for longer learning experiments, without measuring their full optimization cost.
Where the evidence stops. The crop excludes the long caption; its scene and hardware conditions are retained in this guide. It measures simulation on one scene, not training updates per second, policy inference latency or physical-robot performance.
Table 2. Depth PPO leads the reported held-out test comparison. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Each dataset has two columns: SPL measures successful path efficiency, while Succ is the average success rate. The bold Depth row is PPO with depth input; the SLAM row lies inside the RGBD sensor group. All agents also have idealized GPS and compass, including rows labeled Blind. The paper trains PPO for 75 million steps and chooses the best checkpoint on validation before testing. Compare Depth to SLAM for the headline result, then compare RGBD PPO to RGBD SLAM to hold the visual modality fixed. Finally compare PPO rows to examine the sensor intervention under the common training protocol. e07e08e11e12e14
What it supports. Depth PPO reaches 0.79 SPL and 0.89 success on Gibson, versus SLAM's 0.51 and 0.62; on MP3D it reaches 0.54 and 0.69, versus 0.39 and 0.47. Sensor-matched RGBD PPO also exceeds SLAM here, though by smaller margins than the Depth comparison.
Where the evidence stops. Using the table, Depth-minus-RGBD SPL is 0.09/0.12 and Depth-minus-RGB is 0.33/0.24 for Gibson/MP3D. The nearby prose instead gives ranges 0.09–0.16 and 0.13–0.33. These explanations follow the visible cells; no per-cell error bars are printed.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| PointGoal navigation on Gibson and Matterport3D test scenes Same-dataset training/testing; idealized sensors and noise-free actions; PPO trained for 75 million steps, with best validation checkpoints. | Depth PPO: Gibson 0.79 SPL / 0.89 success; Matterport3D 0.54 / 0.69. Mean SPL and episode success rate; higher is better | RGBD SLAM: 0.51 / 0.62 and 0.39 / 0.47. RGBD PPO: 0.70 / 0.80 and 0.42 / 0.53. RGB PPO: 0.46 / 0.64 and 0.30 / 0.42; Blind PPO: 0.42 / 0.62 and 0.25 / 0.35. Depth PPO beats this SLAM baseline by 0.28 and 0.15 absolute SPL, calculated from Table 2. Both algorithm and visual sensors differ; RGBD PPO provides a sensor-matched comparison. Table 2 prints no per-cell uncertainty. e06e07e11e14e18 |
| Navigation performance as training experience increases Gibson and Matterport3D validation episodes; Figure 3, five seeds. | The authors report Depth reaching SLAM performance at approximately 10 million steps on Gibson and 30 million on Matterport3D, then improving. Average SPL versus accumulated training steps | A five-million-step cutoff would favor SLAM; Blind improves early and saturates. Training budget changes the conclusion for these implementations. Bands show standard error over five seeds, not test-set confidence intervals; crossing times are approximate prose summaries. e13e18 |
| Cross-dataset PointGoal transfer to Matterport3D Hold the Matterport3D test column fixed in Figure 5; compare Gibson-trained and Matterport3D-trained agents under the paper's training budget. | Depth trained on Gibson: 0.68; Depth trained on Matterport3D: 0.54. Average test SPL | RGBD: 0.53 versus 0.42; RGB: 0.40 versus 0.30; Blind: 0.34 versus 0.25. Gibson training improves this target score for every tested modality. The easier-episode/curriculum explanation is an author hypothesis. Comparing different test columns additionally changes difficulty and cannot isolate transfer degradation. e09e11e16 |
| Habitat-Sim sensor rendering throughput Matterport3D scene 17DRP5sb8fy; Intel Xeon E5-2690 v4 CPU and Nvidia Titan Xp GPU; square images, one or five GPU-sharing simulator processes. | 128-pixel RGB: 4,093 fps with one process, 10,592 with five. Frames per second; higher is better | 128-pixel RGB+depth: 2,050 and 5,223 fps. At 256 pixels, RGB+depth is 1,042 and 1,774 fps. These are simulator measurements on one scene/configuration, not end-to-end training throughput or real-robot control rates. e05 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Figure 3. The training budget changes the learned-versus-classical comparison. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each panel horizontally as more experience is collected; the vertical coordinate is validation SPL. Orange is RGB, teal Depth, red RGBD and blue Blind, while the dashed line is the fixed SLAM baseline. The shaded regions show standard error over five seeds. Early rankings do not reliably describe the end of training: Blind improves rapidly and then levels off, while Depth continues upward. The accompanying prose summarizes Depth matching SLAM around 10 million steps on Gibson and 30 million on Matterport3D. These are approximate source summaries, not precisely digitized crossings. Keep this validation analysis separate from the test scores in Table 2. e11e13e14e18
What it supports. The learning curves support the paper's central scaling lesson: terminating training early can favor the classical baseline even when a longer run eventually favors Depth PPO. They also show that adding visual channels does not uniformly improve this architecture at the stated budget.
Where the evidence stops. The prose calls the panels top/bottom, but they are left/right. The MP3D dashed line appears slightly above the prose's 0.42 reference. Curves and crossing times should therefore not be treated as exact tabulated values; Table 2 separately reports test results.
Figure 4. Example paths illustrate the behavior behind the sensory comparison. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Treat the left and right halves as two separate validation examples, not repeated trials whose scores can be averaged into a benchmark. Within each half, compare the four sensor conditions on the same scene and episode. The caption identifies a blue start dot, red goal dot and blue arrow for the final pose. The path changes from blue through green toward red as the action budget is consumed; it is not a heat map of prediction confidence or distance to the goal. Blind takes circuitous routes near walls, whereas the displayed Depth routes are more direct. The printed SPL values summarize these particular trajectories. e06e07e14e15
What it supports. In these examples, Depth obtains 0.98 SPL on Gibson and 0.94 on MP3D, compared with Blind's 0.28 and 0.35. This is qualitative support for the authors' wall-following account of Blind behavior and for efficient depth-guided navigation, consistent with the aggregate test ranking.
Where the evidence stops. These are selected examples, not an estimate of failure frequency or proof that all Blind policies implement the same heuristic. The maps are diagnostic visualizations; the agents do not receive the displayed ground-truth floor plans.
Figure 5. Read transfer with a fixed target before interpreting a performance drop. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Every sensor group contains a Gibson-trained row and an MP3D-trained row. The two top headers name the test datasets. To ask which training source produces a better MP3D agent, stay in the right column and compare the two rows within a sensor group: Depth gives 0.68 after Gibson training versus 0.54 after MP3D training. A horizontal comparison asks something else: how the same trained agent performs on two different target distributions. That also changes scene and episode difficulty. The darker colors encode larger SPL, but use the printed numbers for precise comparisons; the figure provides no cellwise uncertainty. e09e11e16e17e18
What it supports. At the fixed MP3D target, Gibson training improves every tested modality: Blind 0.34 versus 0.25, RGB 0.40 versus 0.30, Depth 0.68 versus 0.54, and RGBD 0.53 versus 0.42. Depth is strongest in absolute transferred performance; the authors suggest easier Gibson episodes help bootstrap learning.
Where the evidence stops. The paper's depth-generalization message should not become a claim that other modalities never transfer. Blind changes relatively little, and test difficulty differs across columns. The proposed curriculum mechanism is not isolated by matching episode distributions in this figure.
7. Analysis & limitations
7.1 What the evidence leaves open
GPS/compass and depth are idealized; actions are noise-free. The study demonstrates simulated navigation without a physical deployment experiment. Physics-based object interaction is explicitly future work. e06e07e17
The authors compare representative learning and SLAM implementations and reject intrinsic family-superiority claims. Depth's advantage is attributed to free-space information and possible RGB overfitting, but these explanations are not isolated. e18
Table 2 implies Depth-minus-RGBD differences of 0.09/0.12 and Depth-minus-RGB differences of 0.33/0.24, unlike prose ranges 0.09–0.16 and 0.13–0.33. This report follows the table. Training cost is reported as 2,267 GPU-hours over 'all three datasets', although the main experiment names two; that allocation remains unresolved. e11e14
Figure 2 and its caption show two datasets, whereas the rendering paragraph says three. Its visible examples support Matterport3D and Replica only; broader dataset support is separately described. e19
7.2 Questions for discussion
- Does depth remain strongest when localization and sensing are noisy?
- Would matching training-episode difficulty remove Gibson's advantage on Matterport3D?
- How much of Blind performance depends on GPS/compass and collision-enabled wall following?
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction needs the exact curated scenes/episodes, isolated splits, collision behavior, static goal plus GPS/compass convention, stop criterion and reward. Preserve difficulty filtering and report SPL and success; small protocol changes can alter rankings. e06e07e08e09e10
The main paper gives kernel sizes, workers and training steps but defers detailed architecture, API examples, further throughput and noisy-depth results to the supplement. PPO optimizer settings, network widths and an exact executable release are not specified here; SLAM parameters refer to prior work. e10e11e12e20
Reader-proposed checks: track PPO/SLAM rankings across the full training budget using fixed validation episodes, and compare Gibson/MP3D training after matching episode difficulty on a fixed test set. These test scaling and curriculum interpretations; neither was reproduced here. e09e13e16
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Locate the budget at which the ranking changes
Reader-proposed, not performed: train Depth PPO and RGBD PPO under the documented PointGoal setup for five seeds, saving checkpoints at 5, 10, 30 and 75 million accumulated steps. Evaluate every checkpoint on the same held-out validation episodes, and run the fixed RGBD SLAM baseline on those episodes. Preserve GPS/compass, collisions, episode filtering and stopping rules; report SPL, success and seed variability. Depth-versus-SLAM tests the headline contrast, while RGBD-versus-SLAM controls the visual modality. The scaling explanation predicts an early SLAM advantage followed by a reproducible crossover. If neither learned agent overtakes it under the restored protocol, the claimed ranking reversal has not reproduced. e06e07e08e09e11e12e13e14
Check 2: Test whether easier training episodes explain Gibson's advantage
Reader-proposed, not performed: compare Depth PPO trained on Gibson or MP3D with identical architecture, reward, step budget and seeds. Include both the original episode distributions and versions matched for start-to-goal geodesic distance and geodesic/Euclidean distance ratio; preserve disjoint training/test scenes. Evaluate all models on one fixed MP3D test set and record how often positive success reward appears early in training. The curriculum account predicts that matching difficulty reduces Gibson's SPL advantage and its early-reward advantage. A persistent gap would weaken that explanation and motivate studying other dataset differences. The exact scene lists and implementation settings require the missing supplementary details. e09e10e11e16e20
8.3 Reading coverage
Visual audit: All nine supplied PDF pages were rendered and actually viewed. Page 1 verifies title, author order, institutions and CVF edition notice; page 9 completes printed pagination and references. Pages 2–4 support the stack, scene graph, rendering, API and hardware claims. Pages 5–6 support embodiment, sensors, evaluation, episode filtering, policy, reward and training details. Pages 7–8 support all reported results, source inconsistencies, trajectories, transfer interpretation and physics roadmap. All five figures and both tables were visually read. Six final original crops were viewed individually; Figure 2 was inspected on its page but not cropped. No separate supplement, video, code or other edition was inspected. Narrow crops were rendered from the PDF at higher DPI, without recreating or upscaling charts.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9. Appendix coverage: not present.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- PDF p. 1: title, authors, Abstract and Introduction
- PDF p. 2: Introduction continued, Figure 1 and Related Work
- PDF pp. 3–4: Related Work continued and Section 3, Habitat Platform
- PDF pp. 5–6: Section 4, PointGoal Navigation at Scale
- PDF pp. 6–8: Section 5, Results and Findings
- PDF p. 8: Section 6, Future Work and Acknowledgments
- PDF p. 9: References
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- All four supplied text chunks were read individually and all nine PDF pages visually inspected. No appendix occurs within this PDF; the repeatedly cited separate supplement and video were not supplied or read. Separate supplemental material availability has not been fully verified.
- The supplied extraction does not reconstruct figure images. This limitation was addressed by inspecting the retained PDF and all final original crops.
- Identity notes (e01): the title exactly matches the catalog. The same twelve authors appear in a different order; metadata follows the PDF order. The CVF watermark identifies an open-access copy identical to the accepted version except for the watermark, and points to IEEE Xplore for the final proceedings version. That edition was not inspected; no revision identifier or revision date is established.
- Identity notes (e01): printed pages in this PDF are 9339–9347, whereas the catalog BibTeX records 9338–9346. This report cites actual PDF page numbers.
- Code, linked resources, separate supplements and other editions were not inspected; no experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title/author block, equal-contribution footnote and CVF watermark; PDF pp. 1 and 9, printed page numbers
Exact title and twelve authors are visible. Savva, Kadian and Maksymets carry equal-contribution marks. Six institutions are listed. The watermark describes the accepted-version open-access copy; printed pagination is 9339–9347.
Go to primary source ↓e02PDF p. 2, Figure 1/caption and platform contribution list; PDF p. 3, Section 2, platform shortcomings
The stack separates datasets, simulators and tasks. Fragmentation, hard-coded configurations and slow rendering motivate a common platform.
Go to primary source ↓e03PDF p. 4, Section 3, Design overview, Generic 3D dataset API using scene graphs and Rendering engine
Habitat-Sim loads scenes into a hierarchical graph, configures sensors, simulates motion and returns observations; C++/Magnum and a multi-attachment shader share rendering work.
Go to primary source ↓e04PDF p. 4, Section 3, Habitat-API paragraph and Task/Episode/Environment bullets
Task controls task observations, termination and success with read-only simulator/episode access. Episode specifies scene, pose and goal, optionally shortest path; Environment provides the combined context.
Go to primary source ↓e05PDF p. 4, Table 1, RGB/RGB+depth rows, 1-process/5-process columns, and caption
Fps are reported for scene 17DRP5sb8fy on Xeon E5-2690 v4 and Titan Xp. At resolution 128, RGB gives 4093/10592 and RGB+depth 2050/5223; resolution 256 RGB+depth gives 1042/1774.
Go to primary source ↓e06PDF p. 5, Section 4, Task definition, Agent embodiment and action space, Collision dynamics
No map is supplied. The cylinder has diameter 0.2 m and height 1.5 m; commands turn 10 degrees or request 0.25 m movement. Noise-free experiments retain collisions and sliding.
Go to primary source ↓e07PDF p. 5, Section 4, Goal specification and Sensory input
Static PointGoal is paired with idealized GPS/compass. RGB/depth share position/orientation, 256×256 resolution and 90-degree field of view. Blind lacks visual input, not localization information.
Go to primary source ↓e08PDF p. 5, Section 4, Episode specification and Evaluation, inline SPL definition
Episodes allow 500 actions. Success requires stop within 0.2 m geodesic distance. SPL is S times l/max(p,l), using binary success, shortest distance and traveled distance.
Go to primary source ↓e09PDF p. 6, Section 4, Episode dataset preparation
Scenes are split without overlap, MP3D uses public splits, and Gibson is curated to 106/572 scenes. GDSP is 1–30 m; rejection sampling reduces Gibson episodes with GDSP/Euclidean ratio in [1,1.1] from 37% to 10%. Gibson scenes and episodes are smaller/shorter.
Go to primary source ↓e10PDF p. 6, Section 4, RL (PPO) baseline and unnumbered reward equation
CNN visual embedding and relative goal feed a GRU actor and linear critic. Kernel sizes are 8×8, 4×4 and 3×3 with ReLUs and a linear layer. Training-only reward uses distance progress, bonus 10 and time penalty −0.01.
Go to primary source ↓e11PDF p. 6, Section 4, Training procedure
Eight Gibson/six MP3D workers shuffle 500-episode blocks per scene; training totals 75 million steps and learns stop. Costs are 320/566/475/906 GPU-hours for Blind/RGB/Depth/RGBD, totaling 2267, described ambiguously as summed over three datasets.
Go to primary source ↓e12PDF p. 6, Section 4, Baselines, Random/Forward only/Goal follower and SLAM bullets
Hand-coded baselines include GPS-based stopping. RGBD SLAM uses ORB-SLAM2 and parameters from Mishkin et al.; the pipeline supplies localization, mapping and planning.
Go to primary source ↓e13PDF p. 7, Figure 3/caption and Learning vs SLAM paragraph
Validation curves show four sensory variants and constant SLAM, with standard-error bands over five seeds. Prose summarizes Depth matching SLAM at about 10M Gibson/30M MP3D frames. Rendered panels are left/right despite top/bottom wording.
Go to primary source ↓e14PDF p. 7, Table 2, all rows/columns and caption; adjacent checkpoint-selection and seed discussion
Test SPL/success: Depth 0.79/0.89 and 0.54/0.69; SLAM 0.51/0.62 and 0.39/0.47; RGBD PPO 0.70/0.80 and 0.42/0.53; RGB PPO 0.46/0.64 and 0.30/0.42; Blind PPO 0.42/0.62 and 0.25/0.35. Checkpoints are validation-selected. Some prose difference ranges disagree with table subtractions.
Go to primary source ↓e15PDF p. 8, Figure 4/caption; PDF p. 7, final trajectory-analysis paragraph
Two validation examples compare four sensory variants. Start/goal are blue/red dots, final pose a blue arrow, and trajectory color progresses blue–green–red with steps. Depth example SPL is 0.98 Gibson/0.94 MP3D, versus Blind 0.28/0.35.
Go to primary source ↓e16PDF p. 8, Figure 5/caption, all train/test cells; Section 5, Generalization across datasets
Rows identify sensor and training dataset; columns identify test dataset. On MP3D, Gibson-trained/MP3D-trained SPL: Blind 0.34/0.25, RGB 0.40/0.30, Depth 0.68/0.54, RGBD 0.53/0.42. Easier Gibson training and curriculum learning are proposed explanations.
Go to primary source ↓e17PDF p. 8, Section 6, Future Work and Feature roadmap
Physics simulation and mobile-agent/object interaction, procedural environment generation, and distributed multi-agent settings are future directions.
Go to primary source ↓e18PDF p. 6, Section 5, opening caveat; PDF p. 7, Learning vs SLAM interpretation
Authors avoid intrinsic-superiority claims about broad learning/SLAM families. Free-space relevance of depth and RGB overfitting are hypotheses for sensory ranking.
Go to primary source ↓e19PDF p. 3, Figure 2/caption; PDF p. 4, Section 3, Rendering engine; PDF p. 2, Habitat-Sim contribution
The visible sensor-example rows are Matterport3D and Replica, while p. 4 says the figure depicts three datasets. Platform text separately lists Matterport3D, Gibson and Replica support.
Go to primary source ↓e20PDF p. 4, Table 1 caption and Section 3 final paragraph; PDF p. 6, CNN baseline paragraph; PDF p. 7, noisy-depth/trajectory references
The main paper refers to separate supplemental material for further throughput, architecture/API/CNN details, noisy-depth experiments and behavioral examples.
Go to primary source ↓8.5 Primary sources
Habitat: A Platform for Embodied AI Research ↗
PDF · 7,089 extracted words
Source fingerprint
2ed8928308ad57fd152deb4a2e7178e9ead835b7ad29ecba09c7d493523518fa