Image tracking with ARFoundation
ARFoundation is a framework that allows to create cross-platform Augmented Reality (AR) experiences within Unity. It includes core features from ARCore, ARKit, Magic Leap and HoloLens. Today we will go step-by-step through the setup of a work environment, and the development of a simple 2D image tracking system.
Step 1: Setup of Unity
For this walkthrough, we are using Unity 2022.1.15f1, which can be downloaded here. Make sure to download the iOS Build Support or Android Build Support, according to the mobile device you would be using for testing the application at the end of this guide.
Next, create a new project using the AR template provided by Unity. This process could take a few minutes.
Step 2: Environment Setup
Now that you have created the project, we are ready to start configuring our work environment. Open the project you have just created, and let’s have a look at the SampleScene that Unity has initialised for us.
In the hierarchy tab, the ARSessionOrigin component contains an ARCamera and any GameObject(s) created from detected features. The ARSession controls the whole lifecycle and configuration options for an AR session.
Next, open the ARSession component and remove all the components except for AR Session Origin (Script) and AR Tracked Image Manager (Script). After this step, add the AR Tracked Image Manager component to the ARSession component.
Finally, create two folders, Images and Prefabs, and import in the former the 2D images that you want to track, and in the latter the 3D content to display when image tracking occurs.
Step 3: Create a Reference Image Library
From the Assets->Create->XR tab, create a new ReferenceImageLibrary. Click on the created ReferenceImageLibrary, and proceed by adding your images. Bear in mind that your images and prefabs need to have the exact same name.
For each image, specify the real-world size. Unity uses meters as unit of measure (if your image is 8cm x 10cm, in Unity it would be 0.08 x 0.1). The same logic applies to the dimension of your prefabs.
The final step is to bind the ReferenceImageLibrary with the AR Tracked Image Manager. Click on the AR Session Origin and drag the ReferenceImageLibrary in the AR Tracked Image Manager, under SerializedLibrary. The filed Max Number Of Moving Images defined the number of images that we want to track simultaneously.
Step 4: Let’s do some coding
Now that we have everything in place, we need to recognise our 2D images and show the corresponding 3D object (prefab). Let’s start by creating a new C# script and open it with your preferred IDE (I’m using Rider, but you can use Visual Studio if you are more acquainted with it).
The first step will by importing the UnityEngine.XR.ARFoundation and the UnityEngine.XR.ARSubsystems namespaces before the declaration of our MonoBehaviour class
using UnityEngine.XR.ARFoundation;
using UnityEngine.XR.ARSubsystems; In our class we need to create two private variables that we will use to keep track of the ARTrackedImageManager and the prefabs we want to display. Next, we need a set of dictionaries to store the prefabs corresponding to the image we want to track, and to manage the state of the prefab (active=true, disabled=false).
public class MarkerTrackingManager : MonoBehaviour
{
[SerializeField] private ARTrackedImageManager aRTrackedImageManager;
[SerializeField] private GameObject[] aRModelsToAnchor;
private Dictionary<string, GameObject> _aRModels = new Dictionary<string, GameObject>();
private Dictionary<string, bool> _modelState = new Dictionary<string, bool>();
// Start is called before the first frame update
void Start()
{
}
// Update is called once per frame
void Update()
{
}
}The Start method is called on the frame when a script is enabled and is called exactly once in the lifetime of the script. Let’s iterate over our prefabs and instantiate them, then set their state to false as we don’t need to display them until we find the corresponding 2D image that will trigger their activation.
void Start()
{
foreach (var aRModel in aRModelsToAnchor)
{
GameObject newARModel = Instantiate(aRModel, Vector3.zero, Quaternion.identity);
newARModel.name = aRModel.name;
_aRModels.Add(newARModel.name, newARModel);
newARModel.SetActive(false);
_modelState.Add(newARModel.name, false);
}
}
When a 2D image is tracked by the ARCamera, we want to the display the corresponding prefab. When the 2D image is not the field of view of the ARCamera, we need to disable the prefab. Let’s define two methods: OnEnable() and OnDisplay() to achieve this interaction.
private void OnEnable()
{
aRTrackedImageManager.trackedImagesChanged += ImageFound;
}
private void OnDisable()
{
aRTrackedImageManager.trackedImagesChanged -= ImageFound;
}Our IDE will prompt us with an error, suggesting to create a method for ImageFound. This method will manage the detection of images based on the tracking state, and eventually spawn or hide our 3D prefabs.
private void ImageFound(ARTrackedImagesChangedEventArgs obj)
{
foreach (var trackedImage in obj.added)
{
SpawnARModel(trackedImage);
}
foreach (var trackedImage in obj.updated)
{
if (trackedImage.trackingState == TrackingState.Tracking)
{
SpawnARModel(trackedImage);
}
else if (trackedImage.trackingState == TrackingState.Limited)
{
HideARModel(trackedImage);
}
}
}Finally, let’s create the SpawnARModel and HideARModel methods to update the state of the prefabs in the dictionaries we had defined at the beginning.
private void SpawnARModel(ARTrackedImage trackedImage)
{
bool isActive = _modelState[trackedImage.referenceImage.name];
if (!isActive)
{
GameObject aRModel = _aRModels[trackedImage.referenceImage.name];
aRModel.transform.position = trackedImage.transform.position;
aRModel.SetActive(true);
_modelState[trackedImage.referenceImage.name] = true;
}
else
{
GameObject aRModel = _aRModels[trackedImage.referenceImage.name];
aRModel.transform.position = trackedImage.transform.position;
}
}
private void HideARModel(ARTrackedImage trackedImage)
{
bool isActive = _modelState[trackedImage.referenceImage.name];
if (isActive)
{
GameObject aRModel = _aRModels[trackedImage.referenceImage.name];
aRModel.SetActive(false);
_modelState[trackedImage.referenceImage.name] = false;
}
}Step 5: Attach the created C# Script to the AR Session Origin
Now that we have our C# Script, we need to reference it in Unity, by dragging it on the AR Session Origin. From the Inspector tab, under our Script we need to bind the AR Models to Anchor by simply dragging all our prefabs. Finally, under AR Tracked Image Manager, select AR Session Origin.
Now we are ready to build our application. Unity will ask us to create a new folder within the project where we want to store our build files. Open the Xcode project that was generated with our build, and build the application onto our iOS device (you would need to use Android Studio for Android devices).
Ensure that you have an Apple Developer account and that signing is enabled on Xcode.
Step 6: Demo
If you followed all the steps, you would end up with this final result!
