What's New in iOS 9: New SDK Frameworks

At WWDC 2015, Apple introduced iOS 9. Although the new SDK does not introduce as many new or enhanced features as iOS 8, which included more than 4,000 new APIs, it does still provide a wealth of new functionality and enhancements. Along with the new SDK, iOS 9 is also marked by new developer tools to support some of its features, and new releases of Apple’s major programming languages, Swift and Objective-C.

This series aims at introducing all that is essential for developers to know about building apps for the latest release of Apple’s mobile OS. It comprises five articles that will cover what’s new in iOS 9 SDK, new features in Swift, Objective-C, and developer tools, and Apple’s new bitcode.

This InfoQ article is part of the series “IOS 9 For Developers ”. You can subscribe to receive notifications via RSS.

At WWDC 2015, Apple introduced iOS 9. Although the new SDK does not introduce as many new or enhanced features as iOS 8, which included more than 4,000 new APIs, it does still provide a wealth of new functionality and enhancements. In this article, the first in a series focusing on iOS 9, we are going to review a number of new frameworks that Apple has included with its new mobile OS. This review is based on the documentation that Apple made available and on video sessions from WWDC 2015.

Contacts

The new Contacts API is a much needed replacement for the previous AddressBook framework. It most notably provides the following improvements:

• partial thread-safety, whereas AddressBook’s objects were not meant to be shared across threads;
• optimized performance for the read-only case, the most common according to Apple;
• a higher-level API, whereas AddressBook was based on a few opaque types such as ABRecordRef, ABPropertyID, etc.

The cornerstone of the new API is the CNContact class, which is thread-safe and immutable (i.e. it has value semantics in Swift parlance) and provides access to contact properties, such as the contact's name, image, or phone numbers.

CNContact has a mutable subclass named CNMutableContact that allows to modify contact properties. You can create a CNMutableContact instance either by copying an existing CNContact or by providing a predicate to filter contacts to a class factory method. Contrary to CNContact, CNMutableContact is not thread-safe.

Contact properties are represented through CNLabeledValue, another immutable, thread-safe class that combines in a tuple a contact property value with a label such as Home, Work, iPhone, etc. Contact properties that can have multiple values, such as phone numbers or email addresses, are represented through an array of CNLabeledValue. The Contacts framework provides some predefined labels such as CNLabelHome, CNLabelEmailiCloud etc., and you can create your own custom labels.

The user’s contacts database is represented by (CNContactStore). This class is also thread-safe and is responsible for fetching and saving of contacts and groups. Apple suggests to execute all contact store methods on background threads, since they are synchronous and could otherwise block, albeit shortly, the main thread. Running fetches on a background thread has no drawback thanks to fetch results being thread-safe, so they can be safely passed back to the main thread. If fetch results are cached, the app is responsible to refetch them whenever the contact store is changed and the CNContactStoreDidChangeNotification notification posted.

The contact store is also used to save changes to the Contacts framework objects. The CNSaveRequest class enables save operations and allows batching of changes to multiple contacts and groups into a single operation.

When it comes to contacts, an extremely important feature is enabling to customize their format and localization. For example, you can correctly format a contact name using CNContactFormatter or format an international postal address using CNPostalAddressFormatter.

As it was the case with the old AddressBook framework, users shall actively grant access to contact data for any app that tries to use CNContactStore methods. Thus, on the first attempt to use one of CNContactStore’s methods, the app will block waiting for the user to grant or deny access, following the usual iOS pattern to grant or deny priviledges to an app.

Contacts UI

The Contacts UI framework provides controllers that facilitate displaying, editing, selecting, and creating contacts on your iOS devices. It replaces the old AddressBookUI framework.

The main functionality is supported by the following controllers:

• CNContactPickerViewController, which creates a controller object that manages the contacts picker view.
• CNContactViewController, which implements the view to display a contact.
• CNContactPickerDelegate, which describes the interface CNContactPickerViewController delegates must adopt to respond to contact-picker user events.
• CNContactViewControllerDelegate, which declares the interface that CNContactViewController delegates must implement.

CoreSpotlight

The importance that Apple attributes to CoreSpotlight cannot be underestimated, since it appears high in the list of new features iOS 9 provides to developers. In short, CoreSpotLight provides the means for apps to integrate with the system-wide search functionality and have their content surface in Spotlight search results, Handoff, or Siri reminders.

Furthermore, thanks to deep linking, when the user taps on a search result, the app will be launched with enough information to display the most appropriate content for that search result. Another usage of deep links that is made possible by CoreSpotlight is implementing app history. This feature will enable apps to provide a kind of history stack similar to a browser’s navigation history and any app that is registered with Search will get it for free.

CoreSpotlight is available not only to native apps but also to web apps. A specific markup will allow to indicate that a web site can be accessed through Spotlight searches. This means that when navigating to the deep link associated to a search result, the user will be either taken to the website, or, if a specific app is installed, to that app.

To adopt those new possibilities, developers can use:

• The NSUserActivity class, which provides ways to index activities and app states so they are displayed in search results.
• The new CoreSpotlight framework, which provides the means to index your app content and enable deep link to that content. This can be used with persistent user data, such as documents, photos, etc.
• To make content searchable with CoreSpotlight, you can create an attribute set, e.g. a CSSearchableItemAttributeSet object, that specify the metadata to display in a search result:

// Create an attribute set for an item that represents an image.
CSSearchableItemAttributeSet* attributeSet =
    [[CSSearchableItemAttributeSet alloc] initWithItemContentType:(NSString *)kUTTypeImage];
//
// Set properties that describe attributes of the item such as title, description, and image.
attributeSet.title = @"...";
attributeSet.contentDescription = @"...";

• This is then wrapped into a CSSearchableItem object, which contains a unique identifier for your app and a domain identifier which is useful to group together attribute sets, e.g. to delete them as a whole.
• Once you have the CSSearchableItem object, you index it through CSSearchableIndex’s indexSearchableItems method, and deindex it through deleteSearchableItemsWithIdentifiers method:

// Create a searchable item, specifying its ID, associated domain,
// and the attribute set you created earlier.
CSSearchableItem *item;
item = [[CSSearchableItem alloc] initWithUniqueIdentifier:@"123456"
    domainIdentifier:@"domain1.subdomainA" attributeSet:attributeSet];
// Index the item.
[[CSSearchableIndex defaultSearchableIndex] indexSearchableItems:@[item]
    completionHandler: ^(NSError * __nullable error) {
        NSLog(@"Search item indexed");
}];

• The way you can make your app ready to respond to user action on the Spotlight search result is by implementing application:continueUserActivity:restorationHandler: in the app delegate. This method will be called when the app should respond to the activity requested by the user.

• Within application:continueUserActivity:restorationHandler:, the continueUserActivity parameter represents the activity that triggered the app launch. If it has the type CSSearchableItemActionType, this means the app was launched from a Spotlight search, so we unwrap the CSSearchableItemActivityIdentifier that was passed in to find the unique identifier of the indexed item that was tapped:

  (BOOL)application:(UIApplication *)application
        continueUserActivity:(NSUserActivity *)userActivity
        restorationHandler:(void(^)(NSArray *restorableObjects))restorationHandler {
   
        if ([[userActivity activityType] isEqualToString:CSSearchableItemActionType]) {
        // This activity represents an item indexed using Core Spotlight, so restore the context related to the unique identifier.
        // The unique identifier of the Core Spotlight item is set in the activity’s userInfo for the key CSSearchableItemActivityIdentifier.
            NSString *uniqueIdentifier = [activity.userInfo objectForKey:CSSearchableItemActivityIdentifier];
        }
  }
  

GamePlayKit

GameplayKit aims to provide the foundations to build a game’s logic. GameplayKit is not a game engine and it is meant to be used in conjunction with one, such as SpriteKit for 2D games, or SceneKit for 3D games, third-party game engines, or even no game engine at all for less demanding games that can rely uniquely on UIKit for the graphical requirements.

GameplayKit covers several areas of game design and development:

• Handling game objects types and their associated behavior through the Entity-component architecture, which aims to enforce a composable and reusable architecture. In this architecture, each entity (GKEntity) is an object relevant to the game, while a component (GKComponent) handles a number of behavioural traits of entities, such as visual representation in a given state, handling firing, movement, health level, etc. After creating entities and components, you can attach components to entities thus defining what that entity does during game play, while the overall component hierarchy defines the overall game logic. At runtime, in the update/render loop, you can choose either to dispatch an updateWithDeltaTime: method on a per-entity base, or on a per-component base. In the former case, each messaged entity will then forward the update message to all of its components. In the latter case, more suitable to games with complex logic, the messages are dispatched through the component hierarchy (CGComponentSystem), so it is not required to know at each moment what entities are there.
• Creating and handling a state machine to represent the different states of a game’s entities and the transition between them. A state machine is an instance of GKStateMachine that has exactly one current state, represented by an instance of GKState, at any one time. In the update/render loop of your app you typically call:
  • updateWithDeltaTime: on the state machine, so the current state is forwarded the update message;
  • enterState: to choose a new state, which is sent the didEnterWithPreviousState: message. Each concrete state is represented by a subclass of GKState that implements state-specific behaviour. As mentioned, the current state receives the updateWithDeltaTime: at each iteration of the update/render loop, giving it a chance to transition to a different state.
• Dealing with chance and probability, which are key ingredients of a game. To make it easier for the developer to choose the right random number generator, GameplayKit provides a host of randomization classes that aim to satisfy different goals, such as randomness, performance, known distribution, etc. All provided randomizers conform to the GKRandom protocol. Examples of randomizers are: GKRandomDistribution for a uniform distribution; GKGaussianDistribution for a Gaussian distribution, etc. Alternatively, you can use one of several subclasses of GKRandomSource, such as GKARC4RandomSource, GKLinearCongruentialRandomSource, etc.
• Implementing “intelligent” game play by using any of several complex algorithms such as minmax, which is suited to adversarial turn-based games; pathfinding, useful to model navigability through a game world; agent simulation, to let game characters move themselves based on high-level goals; or rule systems that separate gameplay design from code and allow fuzzy logic reasoning and emergent behaviour.

Apple is providing several Xcode projects that illustrate the use of various features of GameplayKit:

• Maze: A simplified classic arcade game, using many of the design and gameplay features in GameplayKit.
• FourInARow: A simple board game built using UIKit and using the GKMinmaxStrategist class and related protocols, discussed in The Minmax Strategist.
• AgentsCatalog: A demonstration of GKAgent class and several of the individual goals an agent can follow, as well as how to combine goals into complex behaviors.
• DemoBots: A full-featured game, using nearly all of the systems in GameplayKit.

Model I/O

Model I/O is a framework for handling 3D assets and data. It aims to provide a set of data structures that describes the physics of cameras, the physics of materials, the physics of light and make all of this more easily accessible. You can use it to:

• import and export 3D assets: Model I/O allows to import and export 3D asset files from a variety of industry standard file formats such as Alembic (.abc), Polygon (.ply), Wawefront (.obj), and others.
• describe in a physically realistic way lighting, materials, environments
• process and generate assets.

MetalKit

The MetalKit framework aims to reduce the effort required to implement a Metal application. It provides features that can be grouped in three key areas:

• Texture loading, through the MTKTextureLoader class, which allows your application to easily load textures from a variety of sources, such as a URL, a CGImage, its in-memory representation, etc. Textures can even be loaded asynchronously by specifing a completion handler for the operation.

• Model handling, which provides functionality to easily interface Metal with Model I/O models through the following classes:

  • The MTKMesh class provides a container for the vertex data of a mesh and its submeshes.
  • The MTKSubmesh class provides a container for a segment of mesh data that can be rendered in a single draw call. Actual submesh vertex data resides in the submesh's parent mesh.
  • The MTKMeshBuffer class provides a Metal buffer to back vertex and index data for Model I/O meshes and submeshes.

• View management, through the MTKView, which provides an implementation of a Metal-aware view. It has an optional drawing delegate, which allows to customize the drawing of a MTKView without subclassing it and thus reducing the amount of required code.

MetalPerformanceShaders

The MetalPerformanceShaders framework provides a collection of data arallel algorithms for image processing tasks such as convolution, for blur and edge detection; resampling, for scaling; and morphology, for dilate and erode. Available algorithms include also histogram, thresholding, integral, and others. This framework aims to provide highly optimized algorithms that run on the GPU and it is not dissimilar to the Accelerate framework, although the latter runs on the CPU’s vector processor.

MetalPerformanceShaders is tailored to provide best performance possible on Apple hardware by ensuring behind the scene that:

• the right algorithm for a given class is chosen, i.e., an algorithm that is correct and is the fastest available for the given input data, parameters, and device GPU. As an example, there are multiple ways of implementing a blur algorithm, each with its own startup costs and overheads, thus making one concrete implementation work well for a small kernel radius but poorly for a large kernel radius. This optimization has been determined experimentally.
• algorithms are fine tuned for such paramters as kernel radius, pixel format, underlying hardware architecture’s memory hierarchy, number of pixels per thread and thread group dimensions.
• the correct CPU optimizations are performed, such as optimizing program loading speed, reusing intermediate textures, etc.

Sample Code for MetalPerformanceShaders can be found on Apple developer site.

ReplayKit

ReplayKit enables recording audio and visuals of the running application. Furthermore, it allows to:

• add voice commentary to recordings
• play back, scrub, and trim the recordings and share them.
• safeguard user privacy by supporting the usual iOS mechanisms: user is asked for permission; ReplayKit can be disabled; no UI element is ever recorded; apps do no have access to recorded files.

According to Apple, ReplayKit is extremely efficient and has a low impact on both performance and power usage.

The main components that make ReplayKit are:

• RPScreenRecorder, which handles the recording tasks by controlling the replay daemon running in its own process. It allows to start, stop, and discard recording; check the ability to record; enabling the microphone.
• RPPreviewViewController, which is responsible for the preview UI, editing and trimming, and the share sheet. This controller is the only mechanism available to interact with the movie.
• RPScreenRecorderDelegate, which is a protocol that allows to customize handling of recordings.

Sample code can be found on Apple developer pages.

WatchConnectivity

The Watch Connectivity framework enables two-way communication between an iOS and watchOS app that allows to copy files back and forth. This can take place through background transfers, which are suited for cases where the transferred content is not needed immediately and will allow the OS to optimize communication; or, it can take place interactively through a live transfer mechanism. Two-way communication is required to support watchOS apps running locally on an Apple Watch and exchanging data with their companion app on the iPhone and is enabled by the WCSession class,

let session = WCSession.defaultSession()
session.delegate = self
session.activateSession()

session.paired == true
session.watchAppInstalled == true
session.watchDirectoryURL != nil

The watchDirectoryURL will be available after a watchOS has been paired to an iOS app and will remain active while the watchAppInstalled property is true. Afterwards, the contents of the watchDirectoryURL will be wiped out. So, this mechanism can only be used for data relevant to a specific instance of a Watch app.

Background transfers can be of three types:

• application context: e.g., an iPhone app could make available to its related Watch app information about a restaurant that the user selected, or the latest 10 entries in a RSS feed.

• info transfer: this mechanism allows to transfer user info dictionaries, such as to represent a user progress through levels in a game.

• file transfer.

Interactive messages can be sent only when the two devices are connected and the Watch app or the WatchKit extension is running, i.e. when WCSession reachable property is true. In this cases, sendMessage:replyHandler:errorHandler: or sendMessageData:replyHandler:errorHandler: can be used to transfer data immediately to the counterpart.

Next Article in Series

The next article in the series will focus on changes and enhancements to frameworks that were already present in previous versions of iOS.

About the Author

Sergio de Simone  is a software engineer. Sergio has been working as a software engineer for over fifteen years across a range of different projects and companies, including such different work environments as Siemens, HP, and small startups. For the last few years, his focus has been on development for mobile platforms and related technologies. He is currently working for BigML, Inc., where he leads iOS and OS X development.

At WWDC 2015, Apple introduced iOS 9. Although the new SDK does not introduce as many new or enhanced features as iOS 8, which included more than 4,000 new APIs, it does still provide a wealth of new functionality and enhancements. Along with the new SDK, iOS 9 is also marked by new developer tools to support some of its features, and new releases of Apple’s major programming languages, Swift and Objective-C.

This series aims at introducing all that is essential for developers to know about building apps for the latest release of Apple’s mobile OS. It comprises five articles that will cover what’s new in iOS 9 SDK, new features in Swift, Objective-C, and developer tools, and Apple’s new bitcode.

This InfoQ article is part of the series “IOS 9 For Developers ”. You can subscribe to receive notifications via RSS.