top of page

Real-Time 3D object reconstruction

 

New augmented and virtual reality (AR/VR) glasses enable the creation of completely novel and immersive user experiences. To improve immersion, the integration of real content is promising and desired. In recent years, real-time 3D scanning technology has developed significantly and is now able to capture large environments with considerable accuracy. In this presentation, we talk about various technologies used to reconstruct the 3D objects along with various applications of them. All briefly.

 

Seene’s next generation of technology enables dense 3D scene reconstruction, providing full 3D geometry and texturing that is comparable to dedicated hardware scanners – all on standard mobile devices in real-time, without the need for processing in the cloud.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Seene has created a pipeline for high-resolution 3D face scanning on standard mobile devices. All processing is performed on the device without specialized hardware, so anyone can scan their face and see the result in seconds. The raw scan data is then fitted to a canonical base mesh and uploaded to our cloud platform, ready to be applied to virtual experiences and products.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

  • A rigid base mesh allows for facial animation that is powered by a single static scan.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

  • The process standardizes 3D user data for use across various products. It is already being used by some of the leading technology companies.

 

  • For people, scanning the human body helps create highly realistic representations.  This has been used in VR games, where people were scanned to create highly realistic characters, and are used to create ‘hero’ avatars to give automated tours through spaces or greet visitors as they enter a building.  In the future, more people may opt to retain scans of themselves that could be used when shopping for apparel or in multi-player / social virtual environments such as VR Chat.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

There are numerous cases where an accurate representation of physical reality is exactly what is needed to create a virtual reality experience.

 

Any time you’re planning to intervene or alter a physical space, there are compelling reasons to map the space in 3D first.  By absorbing reality into a virtual environment, you can gain a much deeper understanding of the space in a very holistic way.  You can interrogate the model to discover structural and thermal inefficiencies.  It can also be used to inform the design process, helping to convey a very accurate understanding of exactly how a design alteration will affect it.

 

It can even be an effective means of archiving physical spaces and objects so they can be digitally preserved, such as in a museum setting, enabling remote viewing of spaces or objects that are difficult or impractical to access physically.  The possibilities are endless.

 

When a scan is complete, you can bring it into Unity3D where it can be augmented with layers of interactivity and additional 3D objects can be blended in.

 

 

Real Estate:

 

For selling real estate, buyers can save a lot of time previewing properties in an immersive virtual reality.  Anyone who’s ever searched for residential or commercial real estate can attest to the fact that there’s always a significant disconnect between the way photographs on a website convey a listing vs. what it’s actually like to be there  Part of this is due to the inherent distortion associated with translating a 3D space into a 2D photograph.  By exploring a space in virtual reality, the buyer gains a much more holistic understanding of what space is actually like, in a way that is more accurate in scale and proportion.  This can be enormously useful in weeding out undesirable listings and qualifying properties to spend time visiting in reality.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Matterport: https://matterport.com/try-it/

 

Architecture

 

For architectural design, where a renovation or addition to an existing building is required, capturing 3D scans of the existing space is a no-brainer addition to this process that saves a lot of time and money.  The scanned 3D model can be used as a template to draft ‘as-built’ model, yielding a much more accurate result that is free of errors.  This model can then be manipulated to help visualize design ideas, set within the scan of the real environment to help gain a better understanding of context and how the new design will work with the existing space

 

Construction

 

For construction, there are numerous use cases for scanning.  In fact, 3D scanning has been an integral part of the modern construction process for many years.  However, it has recently become feasible to bring those scans into real-time engines like Unity and UE4 where they can be used to verify construction accuracy.

ArchVirtual: We recently created a virtual reality visualization of a large headquarters for a new tech company, where Revit model was used as a basis to create the visualization.  After construction was complete, we scanned the space and layered it over the virtual Revit model to discover that it overlapped flawlessly.  Not only did this verify construction accuracy, but it also helped validate the realism and accuracy we’re able to achieve with arch viz VR applications by comparing the before and after models.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Prototyping

 

For rapid prototyping, 3D scanning can be used for things like testing material choices and making furniture selections.  The existing space can be scanned, then manipulated in Unity to test different paint colors and material types.  We can also introduce 3D objects of proposed furniture types to see how they will fit and look within the space.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Look at ArchVirtual(http://archvirtual.com/) who are doing some amazing work harnessing the 3D object and scene reconstruction technology to provide the above-discussed applications.

 

 

 

Other Technologies:

 

Structure+Bridge Headset: 

Link: https://structure.io/

 

When used as a 3D scanner, Structure Sensor allows you to capture dense geometry in real-time. This enables you to simulate real-world physics and create high-fidelity 3D models with high-resolution textures in seconds. The possibilities are incredible.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Canvas

Link: https://canvas.io/

 

 

Canvas creates a scale-accurate 3D model of a home in minutes all on an iPad. You can measure, review, or share your model instantly, or convert it into simplified CAD format for programs like SketchUp, AutoCAD, or Revit.

 

With this Scan To CAD service, you can convert your scan into an editable, properly-layered, CAD file as if you had made it from scratch. That means you can hide walls, extract floor plans, and use any other feature available in tools like SketchUp, AutoCAD, Revit, 3D Studio Max, and other popular CAD programs. You can skip the set-up and jump right into the design. It saves lots of time.

 

 

 

 

 

 

 

 

 

 

 

 

 

​

Youtube Link: http://www.youtube.com/watch?v=XA7FMoNAK9M

 

Papers and Articles that describe how it made possible to bring this tech into reality with various techniques and optimizations.

 

 

 

Information/Video Sources: ArchVirtual, Seene, Structure

bottom of page