Argoverse: 3D Tracking and Forecasting With Rich Maps
1. Paper overview
In one sentence: Argoverse makes road geometry an explicit prior for tracking and forecasting, while its baseline protocols leave some map benefits entangled with candidate generation and metric choice. e-resourcee-mapse-trackere-featurese-tracking-protocole-multimodal
| At a glance | What to know |
|---|---|
| Research problem | Source description Sensor-only observations leave ambiguities about road surfaces, vehicle heading and possible future routes. Argoverse supplies aligned map context so researchers can test these ambiguities in real Pittsburgh and Miami driving data. The experimental tasks estimate other vehicles’ tracks and future positions; they do not evaluate an ego-vehicle control policy. e-resourcee-trackere-forecast-task |
| Core mechanism | |
| A key reported result | Three-second vehicle trajectory forecasting: LSTM ED+map: 2.25 / 4.67. ADE / FDE, lower is better; Table 3 does not label units.. Argoverse-Forecasting, two-second history and three-second future. Table 3 does not explicitly identify the evaluated split. Map models use best-of-K hypotheses; non-map models use one. LSTM ED: 2.27 / 5.19; NN+map: 2.28 / 4.80; NN: 2.46 / 5.60; constant velocity: 3.55 / 7.89. The map LSTM has the lowest reported three-second errors. Map features and hypothesis multiplicity change together, so this is not an isolated map-feature gain. No uncertainty is reported. e-forecast-resultse-multimodale-forecast-task |
| Reading caution | Source description Mined forecasting trajectories contain acknowledged noise and errors because exhaustive verification is infeasible. Interesting-event selection and vehicle-only forecast targets limit what the benchmark represents; pedestrians and stationary vehicles appear only as context. e-forecast-data |
Core contributions
- Source description
Tracking-Beta provides 100 annotated logs, 10,572 objects and 17 categories, split into 60/20/20 training/validation/test logs. Forecasting supplies 333,441 mined sequences, split 211,691/41,146/80,604 across disjoint geographic areas. These are different annotation products. e-tracking-datae-forecast-data
- Author claim
The authors present rich maps as a resource for perception and map automation, and release tracking and forecasting baselines. Their broad novelty statements concern the contemporary dataset landscape; the experiments test selected uses of the maps. e-resourcee-tracking-resultse-forecast-results
Figure 1. Sensor observations and road structure share a spatial reference. Original paper, p. 1 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the central point cloud, then identify the green cuboids placed around individual vehicles. The magenta lines encode lane centerlines, while the orange markings indicate the driveable region. The surrounding camera images provide complementary views of the same environment; the paired images at middle right are the forward stereo views. Finally, the small lower-right map places the local scene in a wider road network. This is an overview of aligned data products, rather than a neural-network computation graph. Its relevance to the baselines is the connection between an observed object and the road geometry around it. e-resourcee-sensorse-mapse-forecast-datae-trackere-features
What it supports. The dataset supplies more than images with object labels: road structure is available alongside observations and tracks. That alignment enables the later tracker to consult ground and lane information, and lets forecasting models express a vehicle’s motion relative to candidate centerlines.
Where the evidence stops. This illustration does not quantify localization or annotation error. Its sensor-rich tracking view should also not be mistaken for the forecasting model’s input: the forecasting benchmark uses mined planar trajectories and optional context.
2. Motivation
2.1 The problem and the proposed response
Sensor-only observations leave ambiguities about road surfaces, vehicle heading and possible future routes. Argoverse supplies aligned map context so researchers can test these ambiguities in real Pittsburgh and Miami driving data. The experimental tasks estimate other vehicles’ tracks and future positions; they do not evaluate an ego-vehicle control policy. e-resourcee-trackere-forecast-task
2.2 What this reading follows
A vehicle’s recent motion does not uniquely determine its next turn, and sparse LiDAR can leave its heading uncertain. Argoverse supplies a shared spatial reference for both problems: detailed lane graphs, ground height and driveable area aligned with sensor observations. Read this paper as the introduction of a dataset and two baseline experiments. The tracking experiment tests map-assisted filtering and orientation; the forecasting experiment expresses motion relative to possible lanes. The original visuals show where those choices help, while the tables reveal two boundaries: centroid metrics overlook orientation, and map-based forecasts receive more chances under best-of-K evaluation. e-resourcee-mapse-trackere-featurese-tracking-protocole-multimodal
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Datasets |
| Architecture | Not applicable |
| Prediction paradigm | Not applicable |
| Quadrant | Not applicable |
3.1 Evidence-based assessment
Supports the recorded classification
The recorded dataset category is supported: sensor observations, map representations, annotations and evaluation protocols are the primary contribution. Separate tracking and forecasting baselines do not establish a unified world–action architecture, joint future/action prediction or inverse-dynamics control. Architecture, prediction-paradigm and quadrant labels are therefore not applicable here, rather than unverified model classifications. e-resourcee-trackere-models
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 From a map cell to a tracking hypothesis
Begin with a LiDAR sweep in a moving vehicle’s coordinate frame. Calibration and localization connect it to the city map, whose ground-height raster supplies a reference surface and whose driveable-area raster supplies a spatial filter. The tracking pipeline then combines point clustering, image-based vehicle filtering, temporal assignment, ICP transformations and Kalman pose estimation. Lane direction adds another kind of information: it constrains heading when the object is sparsely observed outside intersections. These are separate uses of a map, so the ablations should also be read separately. My interpretation is that the paper demonstrates several practical insertion points for a road prior, rather than one inseparable learned architecture. Table 2 particularly shows that the image filter remains crucial even when map context is present. e-sensorse-mapse-trackere-tracking-results
Figure 4. Mapped ground height handles road shape that a single plane cannot express. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the left panel as a spatial overview of the LiDAR returns. The colors represent surface normals, not semantic object categories. In the upper street views, road points remain visible; the caption highlights the differently colored slope in the middle column as evidence that the ground is not uniformly planar. Compare each upper view with the one beneath it. The lower row applies map tools to remove ground returns and points beyond the driveable area, leaving a more focused set of potential objects. Section 3.1 supplies the representation behind this operation: a rasterized ground-height map and a separate driveable-area raster. e-grounde-mapse-trackere-tracking-results
What it supports. The image explains why a stored terrain surface can be useful before object tracking: road geometry need not be inferred as one plane from the current scan. It illustrates a filtering mechanism whose tracking effect is tested quantitatively in Table 2.
Where the evidence stops. The lower row combines ground removal and driveable-area filtering. This visual alone cannot attribute every removed point to one operation, measure detection accuracy, or establish robustness to map or localization errors.
5.2 Learn motion relative to a possible road
The forecasting target is a sequence of planar vehicle positions after an observed history. A plain predictor sees normalized x/y motion; a map predictor instead describes each position by distance along a centerline and offset from it. In the map LSTM, those coordinates and map features enter the encoder-decoder, and its future distance/offset outputs are converted back to city coordinates for evaluation. The useful intuition is that the centerline already specifies a possible curved route, leaving the predictor to model progress and deviation relative to it. That is a reader interpretation of the representation, not a separately measured decomposition. Training details remain incomplete in the supplied paper. At inference, route ambiguity is handled with multiple hypothesized centerlines rather than a claim that one observed history determines a unique turn. e-featurese-modelse-forecast-taske-multimodale-reproduction
5.3 Separate route knowledge from evaluation privilege
The word oracle appears in two distinct roles. In the named oracle-centerline baselines, the model’s reference lane is selected using the future trajectory, so the inference input is privileged. In top-K evaluation, the metric uses the realized future to identify a successful candidate after predictions have been generated. The ordinary map baselines receive the latter evaluation opportunity without the named oracle centerline input. They can even score better than an oracle-centerline row because several plausible predictions offer more opportunities to match the target. This does not mean future information is intrinsically harmful. My reading is that Table 3 establishes the performance of the complete candidate-generating pipeline, while a clean attribution to lane coordinates requires a comparison with matched candidate counts and the same information available at prediction time. e-multimodale-modelse-forecast-results
5.4 Training and inference
During training
Forecasting defines paired observed and future trajectories, comparing nearest-neighbor regression with learned LSTM encoder-decoders. The supplied paper does not specify LSTM loss, optimizer, dimensions, schedule or compute, nor nearest-neighbor weights/count. No joint tracker–forecaster training or frozen-module regimen is established. e-forecast-taske-modelse-reproduction
During inference
Map-based forecasting emits a variable candidate set, averaging 5.9 hypotheses; non-map baselines emit one. The named oracle-centerline variants use future coordinates to choose their reference path and are diagnostic, not deployable predictors. Best-of-K scoring also uses the realized future, but differs from this oracle input privilege. e-multimodal
Tracking updates estimates from successive observations. Forecasting is evaluated offline against future positions; the paper supplies no action extraction, closed-loop feedback controller or executed driving-success measurement. e-trackere-forecast-taske-forecast-results
5.5 Implementation flow
- Represent road structure explicitly
The vector map stores centerline polylines and predecessor/successor links, turn direction, intersection membership and traffic-control flags. Ground height and driveable area use 1-meter rasters. Driveable area includes physically traversable shoulders; it is not a legality label. Annotation ROI adds a 5-meter margin. e-maps
- Build two kinds of supervision
Human tracking labels follow visible LiDAR objects, mark occlusion and undergo manual quality review. Forecast mining selects intersections, turns, lane changes and dense traffic from 1,006 driving hours. Five-second sequences sample centroids at 10 Hz; one challenging vehicle is the forecast target, while other trajectories provide context. e-tracking-datae-forecast-data
- Track with geometric and visual filtering
Cluster LiDAR returns, prune non-vehicle candidates with Mask R-CNN, associate clusters using Hungarian assignment, estimate transformations with ICP and estimate pose with a Kalman filter. Maps restrict driveable area and remove ground; lane direction adjusts sparse-point vehicle orientation outside intersections. e-tracker
- Choose a coordinate system for prediction
Without maps, translate each history to the origin and rotate its endpoint onto the positive x axis. With maps, convert city coordinates into distance along a centerline and perpendicular offset. Social features summarize front/back distances and neighbor count. e-features
- Predict along candidate routes
Weighted nearest neighbors and LSTM encoder-decoders forecast coordinates. Map LSTMs input along-lane distance, offset and map features, output future distance/offset, then convert back to city coordinates. Breadth-first search explores the lane graph; choosing the vehicle’s possible reference lanes remains ambiguous. e-modelse-multimodal
6. Experiments & results
Argoverse makes detailed maps usable alongside autonomous-driving observations: lane geometry guides trajectory forecasts, while ground height and driveable area constrain tracking. Its contribution is a dataset and benchmark with illustrative baselines. Reported improvements support map-aware perception, but unequal forecast hypothesis budgets and orientation-insensitive tracking metrics limit causal conclusions (e-resource, e-maps, e-tracking-protocol, e-multimodal, e-forecast-results).
6.1 Read the original evidence
Table 3. Map-conditioned forecasting leads the three-second table under unequal hypothesis budgets. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. First separate the one-second and three-second column groups. ADE summarizes displacement along the forecast and FDE concerns its endpoint; both are errors, so smaller values are better. Within the three-second group, compare the plain nearest-neighbor and LSTM rows with their map variants. Next identify rows explicitly labeled oracle: those choose a centerline using future coordinates. Ordinary map rows instead generate several candidate centerlines and receive best-of-K evaluation. This distinction is essential when reading their advantage over single-output models. The table itself supplies neither an evaluated-split label nor units or uncertainty intervals; those omissions remain unresolved here. e-forecast-resultse-multimodale-modelse-forecast-task
What it supports. LSTM ED+map reports three-second ADE/FDE of 2.25/4.67 versus 2.27/5.19 for plain LSTM ED. NN+map reaches 2.28/4.80 versus 2.46/5.60 for NN. These comparisons support the usefulness of the combined map-and-candidate pipeline, while leaving its separate contributions unresolved.
Where the evidence stops. Map baselines average 5.9 hypotheses; non-map baselines emit one. Their errors therefore do not isolate map features alone. Future-informed oracle rows are diagnostics, and best-of-K success does not show that the model ranks the realized future highest.
Figure 8. Candidate centerlines turn road connectivity into structured future possibilities. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the legend before following any path: blue dots are the observed history, red dots are predictions, green dots are the target, and dashed black lines are top-K centerlines. The pale gray road geometry provides context. Start at the blue history near the vehicle marker and trace the forecast into the road ahead. The upper panels illustrate straight travel and a smooth right turn. The lower-left panel is especially informative because multiple candidate routes remain visible around an intersection. The lower-right panel follows a lane with a slight left/right bend. Axes are map coordinates; each panel depicts a different location and scale. e-forecast-visuale-multimodale-modelse-features
What it supports. The examples show how the lane graph restricts plausible geometry without forcing every forecast to share a single destination. A predictor can follow several feasible centerlines and then estimate progress and offset along them. This is the qualitative mechanism behind the map-based forecasting rows.
Where the evidence stops. These are selected examples, not a frequency estimate of success or failure. Dashed centerlines are hypotheses, not probability contours. Agreement between some red and green points does not establish calibration, collision avoidance or executed driving performance.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Three-second vehicle trajectory forecasting Argoverse-Forecasting, two-second history and three-second future. Table 3 does not explicitly identify the evaluated split. Map models use best-of-K hypotheses; non-map models use one. | LSTM ED+map: 2.25 / 4.67. ADE / FDE, lower is better; Table 3 does not label units. | LSTM ED: 2.27 / 5.19; NN+map: 2.28 / 4.80; NN: 2.46 / 5.60; constant velocity: 3.55 / 7.89. The map LSTM has the lowest reported three-second errors. Map features and hypothesis multiplicity change together, so this is not an isolated map-feature gain. No uncertainty is reported. e-forecast-resultse-multimodale-forecast-task |
| Vehicle tracking across sensing ranges 20 Argoverse-Tracking-Beta test logs; vehicle-only centroid matching at 2.25 m; separate 30/50/100 m ego-distance filters. | Full baseline: 73.02 at 30 m, 52.74 at 50 m, 37.98 at 100 m. MOTA, higher is better, reported table scale. | At 100 m: 16.42 without Mask R-CNN, 37.95 without lane information, 37.36 with plane-fitting ground removal. Image filtering dominates these ablations; map ground removal adds a modest gain. Range changes the evaluated population, and centroid scoring is insensitive to much of the orientation benefit. e-tracking-protocole-tracking-resultse-tracker |
| Social-feature forecasting comparison Table 3, three-second future; single-output LSTM ED with or without the paper’s social features. Evaluated split not explicitly labeled. | LSTM ED+social: 2.29 / 5.22. ADE / FDE, lower is better; units not printed. | LSTM ED: 2.27 / 5.19. These particular social features do not improve three-second errors. This does not establish that traffic interactions are uninformative; no statistical significance is supplied. e-featurese-forecast-results |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Figure 7. Lane direction improves visible box alignment that centroid scoring largely misses. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Compare corresponding cuboids between panel (a), without lane information, and panel (b), with it. The green ground-truth boxes provide the reference; the other colored boxes show the tracking output. Concentrate on each box’s long axis rather than just its center. The text describes a conditional rule: use lane direction when the object has too few LiDAR points and is outside an intersection. That condition matters because an intersection may contain multiple overlapping lane segments. The displayed result is qualitative evidence about heading initialization and alignment. It should be read together with the nearly unchanged lane-ablation rows in Table 2. e-trackere-mapse-tracking-protocole-tracking-resultse-reproduction
What it supports. Road direction can disambiguate vehicle orientation when a sparse scan supplies weak geometric evidence. The figure makes that benefit visible even though the paper’s centroid-distance tracking score changes very little when lane information is disabled; heading and center accuracy are different properties.
Where the evidence stops. The paper gives no numerical orientation-error evaluation here and does not specify the sparse-point threshold in the supplied body. The original raster supports a qualitative comparison, not precise per-box angle measurements.
Table 2. Image filtering has the largest ablation effect; mapped ground gives smaller gains. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each four-row block as one controlled range setting. The first row enables Mask R-CNN, lane information and map ground removal. The next rows remove image filtering, disable lane information, or replace map ground removal with plane fitting. Compare configurations within a block before comparing ranges: each range threshold selects which annotations and predictions enter evaluation. MOTA is higher-is-better, whereas the paper uses lower-is-better MOTP. The false-positive and false-negative columns help explain the behavior behind the aggregate score. All these results use vehicle tracking on the twenty Beta test logs, with centroid matching rather than box overlap. e-tracking-resultse-tracking-protocole-trackere-reproduction
What it supports. At 100 meters, MOTA is 37.98 for the full baseline, 16.42 without Mask R-CNN, 37.95 without lane information and 37.36 with plane fitting. The reported false-positive entry rises from 105.40 to 1339.95 when image filtering is removed. This makes the detector filter the dominant change in this comparison.
Where the evidence stops. Range blocks evaluate different populations, and centroid matching does not measure orientation directly. Preserve the table’s scales as printed; no uncertainty intervals are supplied, and detailed metric specifications are deferred to the absent supplement.
7. Analysis & limitations
7.1 What the evidence leaves open
Mined forecasting trajectories contain acknowledged noise and errors because exhaustive verification is infeasible. Interesting-event selection and vehicle-only forecast targets limit what the benchmark represents; pedestrians and stationary vehicles appear only as context. e-forecast-data
Geographically disjoint city regions reduce direct spatial overlap, but both cities remain represented across the split. This is not a leave-one-city-out test. Map-conditioned best-of-K errors do not establish calibrated probabilities, off-map robustness or closed-loop safety. e-forecast-datae-multimodale-forecast-results
Orientation snapping has a visual benefit that centroid-based evaluation largely misses. Forecasting changes candidate count together with map use; oracle rows have future information. Both tasks require protocol-aware interpretation. e-trackere-tracking-protocole-multimodal
7.2 Questions for discussion
- How much forecasting improvement survives an equal candidate budget for map and non-map predictors?
- Would an orientation-sensitive tracking metric reveal gains that centroid MOTA misses?
- How does mined-label noise vary between turns, intersections and dense traffic?
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction requires matching Tracking-Beta and forecasting splits, per-log calibration/localization, maps, preprocessing and city-coordinate evaluation. The paper states a Creative Commons release but leaves the license variant unspecified; code availability is an author statement. e-tracking-datae-forecast-datae-sensorse-featurese-reproduction
Recover deferred coordinate, tracker and metric specifications before claiming faithful reproduction. Resolve LSTM optimization, neighbor weighting, candidate selection, forecasting evaluation split and metric units. Proposed checks should equalize forecast candidate budgets and separately measure tracking orientation; neither is a reported experiment. e-reproductione-forecast-resultse-multimodale-tracking-protocol
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Equalize the forecast candidate budget
Reader-proposed check, not a reported experiment: train or construct paired nearest-neighbor predictors using the published training partition, and evaluate on the geographic validation partition with two seconds observed and three seconds predicted. Compare normalized x/y and lane-relative features at fixed budgets K=1 and K=6; each method must produce the same number of distinct candidates using observed history only. Hold the training database, neighbor weighting and evaluation coordinates fixed, and report candidate duplication alongside ADE/FDE. Exclude future-informed centerline selection. If the map advantage shrinks sharply after matching K, multiplicity explains part of the original comparison; a persistent gap supports a geometric contribution. Document the chosen ranking and weighting rules because the supplied paper does not fully specify them. e-forecast-datae-featurese-forecast-taske-multimodale-modelse-forecast-resultse-reproduction
Check 2: Measure the heading effect that centroid MOTA misses
Reader-proposed check, not a reported experiment: rerun the tracking pipeline with lane snapping enabled and disabled on the same Beta test logs, holding the detector, ground filtering, association settings and range thresholds fixed. Record both the paper’s centroid metrics and an added absolute wrapped heading-error metric on matched vehicle boxes. Stratify sparse-point objects outside intersections from dense-point objects and intersection cases, and sweep an explicitly documented sparse-point threshold. A heading improvement concentrated in eligible sparse objects with little centroid change would support the mechanism illustrated in Figure 7. No heading improvement, or substantial deterioration for lane-inconsistent vehicles, would bound that explanation. The additional heading metric and threshold study are proposed extensions, not values reported by the paper. e-trackere-tracking-datae-tracking-protocole-tracking-resultse-reproduction
8.3 Reading coverage
Visual audit: All ten primary-PDF pages were rendered and visually inspected. The title/author block and CVF edition notice, Figures 1–8, Tables 1–3, method and evaluation text, and reference pages were checked. Six original crops were inspected individually: Figures 1, 4, 7 and 8 and Tables 2 and 3. Legends, axes, panel labels and table headers are retained; captions and body prose are excluded. Figure 7’s snapping condition and Figure 8’s legend were cross-checked against Sections 4 and 5. No claim-relevant graphic/text conflict was found. The paper contains no appendix; the separately referenced supplement was not supplied or inspected. All retained method, numerical and proposed-check premises are supported on the declared pages.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10. Appendix coverage: not present.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract
- 1. Introduction
- 2. Related Work
- 3. The Argoverse Dataset, including Sensors
- 3.1. Maps
- 3.2. 3D Track Annotations
- 3.3. Mined Trajectories for Motion Forecasting
- 4. 3D Object Tracking
- 4.1. Evaluation
- 5. Forecasting: preprocessing, coordinate systems and features
- 5.1. Problem Description
- 5.2. Multimodal Evaluation
- 5.3. Results
- 6. Discussion and Acknowledgements
- References, PDF pp. 9–10
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout. This acquisition caveat was addressed by visually inspecting all ten PDF pages and every final crop.
- Separate supplemental material availability has not been fully verified. No separate supplement was supplied or read; no appendix is present in the ten-page primary PDF.
- No code, external resource or later dataset release was inspected, and no experiment was reproduced.
- Identity/edition note: the supplied CVPR 2019 CVF open-access paper covers proceedings pp. 8748–8757 and Argoverse-Tracking-Beta. The title page uses lowercase “with”; supplied observedTitle uses “With”. Authors match, with Sławomir Bąk printed using diacritics. The watermark states equivalence to the accepted version except for itself; no independent comparison with IEEE or later revisions was made (e-identity).
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1 (proceedings p. 8748), title, author block, equal-contribution footnote and CVF watermark; p. 4, Section 3.2 footnote
The title prints “Argoverse: 3D Tracking and Forecasting with Rich Maps”; all eleven catalog authors appear, including Sławomir Bąk with diacritics. Affiliations are Argo AI, Carnegie Mellon University and Georgia Institute of Technology. The first four authors share equal contribution. The watermark describes an open-access copy identical to the accepted version except for the watermark; the tracking release is explicitly Beta.
Go to primary source ↓e-resourcePDF pp. 1–3, Abstract, Section 1 contributions, Section 3 opening and Figure 1
The contribution is aligned sensor data, maps and annotations for tracking and forecasting. Figure 1 combines camera views, LiDAR, lane centerlines, driveable region and cuboid tracks. Collection spans Pittsburgh and Miami, seasons, weather and times of day.
Go to primary source ↓e-sensorsPDF p. 3, Section 3 “Sensors”; p. 4, Figure 3 and caption
Two VLP-32 LiDARs, seven overlapping ring cameras at 30 Hz and two front stereo cameras at 5 Hz are described, with 6-DOF localization, per-log calibration and procedural face/license-plate blurring. Figure 3 distinguishes vehicle, camera and LiDAR frames.
Go to primary source ↓e-mapsPDF p. 4, Section 3.1; p. 3, Table 1 Argoverse rows
Maps comprise lane-centerline polylines with intersection, traffic-control, turn and predecessor/successor attributes, plus ground-height and driveable-area rasters at 1-meter resolution. Driveable means physically possible, not necessarily legal. Annotation ROI extends 5 meters beyond it. Table 1 reports 204 km of Miami lanes and 86 km of Pittsburgh lanes.
Go to primary source ↓e-groundPDF p. 4, Figure 4 and caption; Section 3.1 “Rasterized Ground Height Map”
Surface-normal colors differ across uneven road surfaces. The lower image row removes ground points and points beyond driveable area using map tools; the comparison does not isolate those two operations.
Go to primary source ↓e-tracking-dataPDF pp. 4–5, Section 3.2 and Figure 5
Tracking-Beta contains 100 human-annotated logs of 15–60 seconds, 10,572 objects and 17 categories. More than 70% are vehicles. Labels receive manual quality review; occlusions are marked. The train/validation/test division is 60/20/20 logs.
Go to primary source ↓e-forecast-dataPDF p. 5, Section 3.3 and Figure 6
Mining 1,006 driving hours identifies interesting behavior in 320 hours and yields 333,441 five-second sequences of 2D centroids at 10 Hz. Targets emphasize intersections, turns, lane changes and dense traffic. Splits contain 211,691/41,146/80,604 train/validation/test sequences from disjoint city areas. Pedestrians and stationary vehicles remain as context but are not forecast targets. Exhaustive verification is infeasible and some trajectories contain noise/errors.
Go to primary source ↓e-trackerPDF pp. 5–6, Section 4, pipeline paragraph and map-attribute paragraphs; p. 6, Figure 7
Vehicle-only tracking clusters LiDAR points, filters non-vehicles with Mask R-CNN, associates clusters with the Hungarian algorithm, estimates transformations with ICP and pose with a Kalman filter. Maps constrain driveable area, remove ground and adjust orientation outside intersections when too few LiDAR points are available. Figure 7 compares orientation snapping; green cuboids are ground truth.
Go to primary source ↓e-tracking-protocolPDF p. 6, Section 4.1
Tracking evaluates 20 Beta test logs at 30, 50 and 100 meters from the ego vehicle. Matching uses centroid distance with a 2.25-meter missed-track threshold instead of IoU. MOTP follows the lower-is-better CLEAR MOT convention. The authors note that centroid metrics do not capture the orientation benefit.
Go to primary source ↓e-tracking-resultsPDF p. 7, Table 2, MOTA and #FP columns, full-baseline and three ablation rows in every range block
Full-baseline MOTA is 37.98/52.74/73.02 at 100/50/30 m. At 100 m, removing Mask R-CNN gives 16.42 MOTA and #FP 1339.95 versus 105.40; disabling lane information gives 37.95 MOTA; plane fitting gives 37.36. At 50 m, full/map versus plane MOTA is 52.74 versus 52.05. No uncertainty intervals are reported.
Go to primary source ↓e-featuresPDF pp. 6–7, Section 5, “Forecasting Coordinate System and Normalization” and “Feature Engineering”
No-map histories are translated to start at the origin and rotated to end on the positive x axis. Map features express location as distance along and offset from a centerline. Social features use front/back minimum distances and neighbor count; social and map features are denoted s and m with trajectory/time indices.
Go to primary source ↓e-forecast-taskPDF p. 7, Section 5.1
For vehicle trajectory V_i, X_i contains observed coordinates at times 1 through T_obs; Y_i contains future coordinates from T_obs+1 through T_pred. Experiments observe 20 frames (2 seconds) and forecast 10–30 frames (1–3 seconds). Context can include other object histories and maps.
Go to primary source ↓e-multimodalPDF p. 7, Section 5.2
The vector map is a semantic graph; localization/hypothesis generation and breadth-first graph search produce candidate paths. Evaluation uses oracle error over top-K forecasts with ADE/FDE. Map baselines average K=5.9 hypotheses, and fewer than 2% of scenarios exceed 25; other baselines emit one. The separately named oracle variants access future coordinates to select a reference centerline, which can still be imperfect.
Go to primary source ↓e-modelsPDF pp. 7–8, Section 5.3 baseline definitions
Baselines include mean constant velocity, weighted nearest-neighbor regression and LSTM encoder-decoder models. Map LSTMs consume along-lane distance, lateral offset and map features and output distance/offset, subsequently mapped back to city coordinates. Oracle and top-K centerline variants are distinct; social features can be appended.
Go to primary source ↓e-forecast-resultsPDF p. 8, Table 3, 3 SECONDS ADE/FDE columns and Section 5.3 discussion; p. 7, Section 5.2
Three-second ADE/FDE are 2.25/4.67 for LSTM ED+map, 2.27/5.19 for LSTM ED, 2.29/5.22 for LSTM ED+social, 2.28/4.80 for NN+map, 2.46/5.60 for NN and 3.55/7.89 for constant velocity. NN+map(oracle) is 2.39/5.05. Table 3 does not print units, uncertainty or an explicit evaluated-split label; the comparison uses unequal hypothesis budgets.
Go to primary source ↓e-forecast-visualPDF p. 8, Figure 8, four panels, legends and caption
Legends mark observations blue, predictions red, targets green and top-K centerlines dashed black. Panels illustrate straight motion, a smooth right turn, several intersection possibilities and a slight left/right lane bend. These are selected qualitative examples of LSTM ED+map.
Go to primary source ↓e-reproductionPDF p. 3, Section 3; p. 6, Sections 4 and 4.1; pp. 7–8, Sections 5.1–5.3; p. 8, Section 6
The paper states data, annotations and API are available under a Creative Commons license and later states code is available. The license variant is unspecified. Coordinate-system, tracker and tracking-metric details are deferred to supplemental material. The supplied baseline descriptions do not specify LSTM loss, optimizer, dimensions, schedule, compute, nearest-neighbor weights/count or complete hypothesis-selection thresholds.
Go to primary source ↓8.5 Primary sources
Argoverse: 3D Tracking and Forecasting With Rich Maps ↗
PDF · 7,706 extracted words
Source fingerprint
2fad6f37efc0e41cfec80b79a38df8182f599302eb2d0a2ff6e75fb5bdb93511