Jun 17, 2024

Augmented reality (AR) enhances the real world with an additional layer of information. This technology has great potential and applications in various contexts and domains. However, designers and practitioners have limited guidelines and principles for successfully designing and implementing AR experiences.

Most users in my experience were unfamiliar with augmented reality; some confused it with virtual reality. On my side, I have almost five years working with UX and UI experiences for augmented reality platforms and projects close to Goopy Augmented Reality, and I would like to share some insights and tools that I learned during my journey working with that.

Now, with the evolution of ARKit and Arcore, The Meta Quest 3 managing Mixed Reality, and the launch this year of the Apple Vision Pro in February, I think UX designers now have to take a look at the Augmented and Mixed reality world.  

Here I list some insights and design patterns and tools that you should learn if you want start to work with UX design for Aumented and mixed reality.

Now in 2024 you have to forget of format about 3D files like ObJ and FBX etc. If you wanna work in the augmented reality world you have to know about the following format of 3D files.

glTF
(Graphics Library Transmission Format or GL Transmission Format and formerly known as WebGL Transmissions Format or WebGL TF) is a standard file format for three-dimensional scenes and models.

GLB
GLB is a standardized file format for sharing 3D data used in virtual reality, augmented reality, games, and online applications. GLB files are a binary form of the GL Transmission Format (glTF) file, which employs JSON (JavaScript Object Notation) encoding.

USDZ
USDZ format is a single, self-contained file format that includes all the necessary information to display a 3D object or scene without any external dependencies.

Video Chroma (MOV-MP4)
Also, if you want to create good AR Experiences, In that case, you should learn about Chroma key compositing because, with this technique, you can use holograms and animated graphics elements to increase the UX/UI AR experiences.

PNG Transparent
It would help if you learned to generate cutting-edge PNG with Figma, illustration, or Photoshop.

 

It is not necessary to learn the technical aspects of the AR framework, but if you want to understand the pattern of the principal frameworks, you should know the following.

ARKit
ARKit is an application programming interface (API) for iOS, iPadOS and VisionOS which lets third-party developers build augmented reality apps, taking advantage of a device's camera, CPU, GPU, and motion sensors. The ARKit functionality is only available to users of devices with Apple A9 and later processors.

ARCore
ARCore, is a software development kit developed by Google that allows for augmented reality (AR) applications to be built. ARCore has been integrated into a multitude of devices. ARCore uses a few key technologies to integrate virtual content with the real world as seen through the camera of a smartphone or tablet. Each of these technologies can be utilized by developers to create a high-quality, immersive AR experience.

Unity
Unity is a cross-platform game engine developed by Unity Technologies, first announced and released in June 2005 at Apple Worldwide Developers Conference as a Mac OS X game engine. The engine has since been gradually extended to support a variety of desktop, mobile, console, augmented reality, and virtual reality platforms. It is particularly popular for iOS and Android mobile game development, is considered easy to use for beginner developers, and is popular for indie game development.

Sketchfab
Sketchfab is a 3D asset website used to publish, share, discover, buy and sell 3D, VR and AR content. It provides a viewer based on the WebGL and WebXR technologies that allows users to display 3D models on the web, to be viewed on any mobile browser, desktop browser or Virtual Reality headset.

Vectary
Vectary, is an online platform for creating and managing interactive 3D and AR designs that inspire, educate and solve problems. ‍No downloads, no-code: all in the browser.

Adobe After Effects
If you are working with augmented reality, it would be good to learn a little animation with After Effects to create holograms and animations in augmented reality. 

Figma Apple Design Resources – visionOS
Apple’s initial visionOS design kit for Figma contains a comprehensive set of UI components, views, system interfaces, text styles, color styles, and materials. All of the core ingredients you need to quickly create highly realistic visionOS app designs.

WebGL
WebGL (short for Web Graphics Library) is a JavaScript API for rendering interactive 2D and 3D graphics within any compatible web browser without the use of plug-ins.WebGL is fully integrated with other web standards, allowing GPU-accelerated usage of physics, image processing, and effects in the HTML canvas.

Three.js
Three.js is a cross-browser JavaScript library and application programming interface (API) used to create and display animated 3D computer graphics in a web browser using WebGL. The source code is hosted in a repository on GitHub.
https://threejs.org/