ARMobile (Experimental)

ARMobile is an extension designed to work with augmented reality experiences, so you will need compatible devices to run these kinds of applications. Currently, the only supported platform is Android (ARCore), but we also plan to support iOS (ARKit) in the future. ARMobile is based on the XR Platform.

Note

Please note that the Evergine.ARMobile extension is in an experimental state, so it has some limitations and known issues described in the Known issues section.

Create a Project Using ARMobile

To make it easier to create ARMobile projects, we have developed a new project template for Android ARCore. It works with OpenGL as the graphics backend due to limitations related to ARCore, which does not support Vulkan.

Graphics

You also need to open the Evergine Studio add-ons manager and add the Evergine.ARMobile add-on.

Graphics

Once Evergine Studio loads the default scene, you need to make some changes to ensure it works properly with ARMobile.

  • For the scene's 3D camera, set the background color to #00000000 and remove the FreeCameraBehavior component.
  • Disable the SkyAtmosphere entity.
  • Go to the Scene Managers tab and add the ARMobileManager scene manager.

Graphics Graphics

Built-in ARMobile Add-on Prefabs

The ARMobile add-on provides a set of prefabs that you can use for common AR scenarios. Expand Dependencies > Evergine.ARMobile > Prefabs to find the following prefabs: Cursor, Face, FeaturePoints, Image, and Planes.

Graphics

Cursor Prefab

This prefab draws a cursor whose orientation changes depending on surface detection. Drag and drop the Cursor prefab into your scene and set its position to (0, 0, 0).

Graphics

If you select the prefab entity, you will notice that apart from plane mesh components, there is an ARMobileHitTestBehavior that lets you choose how plane collision is detected: every frame or just when the user taps on the screen.

Face Prefab

The Face prefab allows you to use face tracking capabilities from the AR platform. Drag and drop the Face prefab to your scene and set its position to (0, 0, 0). Then, go to Scene Managers and check FaceTrackingEnabled.

When using face tracking detection, the rear camera is no longer used, and the front camera is used instead. Because of this, any other AR mode like image tracking or plane detection will not work.

Graphics

If you select the prefab entity, you can use the attached MaterialComponent to change the face material. You will also notice a component called XRFaceTrackingMesh that creates a face mesh from native buffers.

Feature Points Prefab

This prefab uses LineBatch3D to render feature points in green, orange, or red, corresponding to more confident to less confident results. Confidence is saved in the W field of the Points property in XRFeaturePoints. Drag and drop the FeaturePoints prefab to your scene, set its position to (0, 0, 0), and enable feature points detection in the ARMobileManager scene manager.

Graphics

Image Prefab

With the image prefab, you can easily track images and place 3D objects when those images are detected, and their real-world position changes. It includes a MaterialComponent that you can change to use a custom material for the plane that appears once the target image has been detected. Drag and drop the Image prefab to your scene, set its position to (0, 0, 0), and enable image detection in the ARMobileManager scene manager.

Graphics

Under Dependencies > Evergine.ARMobile > AR, you will find the default images dataset. It includes a single image named arimage.jpg. The dataset file is in CSV format:

  • Each line represents a single image from the dataset.
  • The first column is the relative path to the image file. The second column is the expected image size in the real world, measured in meters. For the default image, we expect it to be 20 centimeters wide.
  • For each of the images, we use the file name (without extension) as the image name, which we internally register in the dataset database. This is why the default ImageName value for the XRImageTracking component in the Image prefab is arimage, as this is the file name for the default image in the Evergine.ARMobile add-on.

If you want a custom set of images, create your own dataset within your project folders. Then, change the ImageDataSetPath value in the ARMobileManager scene manager to match your dataset path. To ensure your images are optimal for tracking, please follow platform recommendations:

  • For Android: Use the arcoreimg tool to evaluate your images' quality.

Planes Prefab

We provide this prefab to visualize detected planes. Drag and drop the Planes prefab into your scene and set its position to (0, 0, 0). Then, go to Scene Managers and select a plane detection mode.

The prefab includes a XRPlaneRenderer that lets you change the material used to draw the planes.

Known Issues

As the ARMobile extension is in an experimental state, you may encounter some issues while running applications. Here is a list of known issues:

  • The application crashes when returning from the background.
  • In Debug configuration, you may see some Google.AR.Core.Exceptions.DeadlineExceededException exceptions written to the output console, depending on the ARMobileManager configuration.
  • When dragging AR prefabs into the scene, ensure that their Transform3D location is set to (0, 0, 0). Any other position could cause an OpenGL rendering issue resulting in a full-screen green texture.