Business context
The client works in 3D reconstruction for the reality capture and survey market. A plain reconstructed mesh is one surface with no meaning: it does not know which part is which object. For the client, the value lay in 3D data that analysis and downstream tools can query object by object.
The challenge
The client needed segmented 3D meshes reconstructed from images, with each object identified, for use in analysis and downstream tools. Two hard problems meet here. Labels found in 2D images must carry over correctly to 3D geometry. Each object also needs enough point density for an accurate surface, without making the whole scene too heavy to process.
Our solution
We built a pipeline in Python that combines an image segmentation model with adaptive point cloud generation and Poisson surface reconstruction. OpenCV handles the image side and Open3D the point cloud and mesh side, so every object keeps its class from image to final mesh.
- Segmentation and classification of objects in images
- Point clouds generated for each segment
- Adaptive point cloud resolution, so detail goes where each object needs it
- Poisson reconstruction of the surface mesh, with Open3D
- Semantic classification of the 3D mesh objects
Business impact
Semantic segmentation of 3D objects and geometrically accurate meshes.
- Analysts select, count and measure individual objects instead of editing one merged surface by hand
- Downstream tools receive 3D data that already carries object classes, which cuts manual labeling
- The client gains an ML building block it can apply to new capture projects and object types
- Classified 3D objects give AI and analytics workflows structured spatial data to work with
