Language-guided zero-shot segmentation with multi-angle reprojection for point cloud analysis
Bibliographic record
Abstract
• Language-guided zero-shot 3D segmentation and object extraction. • User-guided multi-angle orthophoto generation for improved scene coverage. • GroundingDINO-based text-prompted detection for intuitive object detection. • Confidence-weighted fusion ensuring accurate and consistent 3D reprojection. • Flexible viewpoint selection and iterative refinement for enhanced usability. Virtual Reality applications increasingly demand accurate 3D representations of real-world environments. While LiDAR point clouds capture physical spaces with high fidelity, they typically lack semantic labels, limiting their direct use for tasks such as object recognition, interaction modeling, and automation in immersive environments or digital twin systems. We present a LAnguage-guided zero-shot 3D SEgmentation and Reprojection tool (LASER), an engineered zero-shot segmentation tool that extends the state of the art by introducing language-guided 3D object detection for enhanced usability and accuracy. Unlike its predecessors, LASER uses an ensemble of GroundingDINO and Segment Anything Model as its backbone to process natural language queries and user-specified object categories, automated multi-view orthophoto generation with dynamic angles for optimal view selection, a confidence-weighted fusion algorithm for efficient 2D-3D reprojection, and a semantically labelled mesh output. The LASER pipeline begins by collecting point cloud data using LiDAR sensors, filtering the point cloud into ground and non-ground components, improving segmentation efficiency. It then generates multi-angle 2D orthophotos and perspective views, incorporating a user-guided angle selection module to optimise scene coverage. Then GroundingDINO detects objects based on textual descriptions, and Segment Anything Model subsequently refines these into segmentation masks. The core innovation of LASER lies in its confidence-weighted reprojection algorithm, which fuses multiple 2D segmentation results back into 3D space, ensuring higher segmentation accuracy and spatial consistency. The resulting semantically labelled assets can be exported in standard formats or iteratively refined through viewpoint adjustments or text prompt modifications. Our application of LASER to real-world 3D scans of construction sites demonstrates its effectiveness in delivering high segmentation precision, enhanced user interactivity, and seamless integration into virtual reality workflows. To comprehensively evaluate the proposed tool on diverse point cloud scans, we also presented the performance on four different test cases using two different scans (3DSES and Toronto3D) with both indoor and outdoor scenes. The results show consistent performance across scans. Finally, feature-based comparison with state-of-the-art approaches shows that LASER is an optimised tool for enriching static, open-world 3D scans with semantic labels, offering an alternative to existing state-of-the-art methods for niche applications.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.004 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.002 |
| Bibliometrics | 0.002 | 0.001 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.002 | 0.002 |
| Open science | 0.002 | 0.003 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.008 | 0.005 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".