At WWDC 2017, Apple unveiled ARKit, a framework to build augmented reality (AR) apps for iOS. ARKit aims to allow for accurate and realistic immersion of virtual content on top of real-world scenes.
At its heart, ARKit provides support for basic key capabilities such as motion tracking, horizontal plane detection, and scale and ambient light estimation.
Motion tracking is powered by the CoreMotion framework and specialized co-processors coming with Apple mobile devices. It uses visual inertial odometry to estimate the 3D orientation and movement of the device based on the data coming from the camera. This, according to Apple, ensures a high degree of accuracy without requiring any additional calibration. Thanks to the ability of detecting horizontal planes, such as floors and tables, ARKit also makes it possible to place objects precisely within a physical scene and track their movement. Ambient light estimation is key to realistic shadowing and lightning of virtual objects.
The ARKit API supports two main configurations: ARSessionConfiguration, to track only a device’s orientation, and ARWorldTrackingSessionConfiguration, which also tracks a device’s position and detects real-world surfaces. Any detected surface is associated to an ARPlaneAnchor object and can be further described using ARHitTestResult, which provides its distance from the camera, position, and orientation. An ARPlaneAnchor is a special case of an ARAnchor, which can be added to an AR scene to place virtual objects within it. ARKit easily integrates with SceneKit or SpriteKit to handle 3D or 2D virtual content thanks to ARSCView and ARSKView. Alternatively, you can use Metal, updated to version 2 at WWDC, to take full control of scene rendering.
On the Mac the aforementioned Metal 2 also adds support for virtual reality (VR) rendering and external GPUs, and provides new APIs and optimizations to help developers embrace these emerging technologies within their apps, though customer availability of external GPU support isn't planned until Spring 2018.
This is just a quick overview of Apple ARKit. Make sure you read the introduction to AR and the official API reference if you need additional information.
While Apple did not disclose its future AR plans, Apple CEO Tim Cook stated in an interview to Bloomberg’s Emily Chang that ARKit provides the foundations for future Apple products.
ARKit is in beta and included in Xcode 9, which is already available to registered developers. Apple also provided a sample project showing the basic mechanics of placing objects into an AR scene.