Our physical world is about to become more and more digitally enabled. According to various predictions by Gartner or Cisco, there will be many billions of IoT devices going online in the coming years. Those devices can be grouped into two major categories: devices that “do” something (actors) and devices that collect data (sensors). Billions of devices constantly gathering data will leverage the true potential behind IoT: building information contexts and deducing intelligent decisions from this information to make our daily lives more secure, efficient and convenient.
Since all this data needs to be analyzed and stored, IoT is also challenged in the field of big data. We got in touch with Wayne Carter and Ali LeClerc of Couchbase to discuss whether Couchbase Mobile - a NoSQL database specifically designed to address requirements of mobile devices - is also ready for the upcoming era of Internet of Things.
InfoQ: Couchbase is known as a NoSQL database engine. But besides offering data capabilities in ordinary (server-side) applications, Couchbase is also prepared for the increasing amount of devices out there. Could you please outline how Couchbase can be used beyond our servers?
Couchbase Mobile is our NoSQL database solution for mobile. It’s comprised of:
- Couchbase Lite - our embedded NoSQL database that resides on the device
- Sync Gateway - our internet-facing cloud component that securely syncs data from the device to the cloud
- Couchbase Server - our highly scalable and performant NoSQL database in the cloud
Couchbase Mobile enables mobile developers to build apps quickly that work both online and offline. With Couchbase Lite, data is stored locally on the device and with Sync Gateway, data is synced between the device and cloud. Without a network connection, mobile applications continue to work with Couchbase Lite, and data is synced via Sync Gateway when a network is available.
InfoQ: Lately I read some news that said there will be 20-50 billion IoT devices by 2020. No matter if this is really the correct number, but there will be lots of devices, that’s sure. Will there also be special support for IoT?
With the massive amounts of data being ingested within the IoT sphere, technologies to capture, clean, ingest, and process that data is increasingly important. Couchbase Mobile supports these types of cases.
In our Couchbase Mobile offering we are adding features specifically for IoT. One example of a feature that we are finding to be important for IoT is support for time-series data. Another is incremental data processing (e.g. incremental map-reduce) that allows data to be preprocessed on the device and the results of that processing to be incrementally reprocessed at another point in the system.
Take for example General Electric, a Couchbase Mobile customer. GE recently launched their Predix platform, a software platform for the industrial internet. Predix connects machines like wind turbines, railcars, power plants and more - machines that are counted on to work all day, everyday for the next 10, 20, 30 years. Companies like GE need a technology that can be relied on to work all the time, regardless of network availability. They also need a technology that can store, transfer, and ingest these massive amounts of data. Couchbase Mobile plays a key role in this platform to resolve these needs.
InfoQ: There is still a connectivity challenge when being mobile: low bandwidth, sometimes high communication costs or bad to none connectivity at all. As soon as we rely on data in our apps and devices, this can be a serious problem. How can we solve this?
An assumption that a lot of mobile developers make is around the ubiquity of high performance wireless networks. Specifically, it’s apps whose functionality only works with a good (i.e. available and fast) internet connection. But users are starting to expect mobile apps that work all the time, regardless of network availability.
There is a lot of evidence in the market that this assumption of ubiquitous networking will be proven wrong, which will require app developers to start developing functionality that works with or without a network connection. In fact, many of the most popular mobile apps have already started adding offline functionality (e.g. the Facebook mobile app recently added the ability to post status and pics without an internet connection. All the major airline mobile apps now have portions of functionality that work offline, etc).
It’s important that the system allows data to be managed and sync’d as the networking environment allows (e.g. only sync large images on a wifi network and don’t use a user’s data plan for that).
Couchbase Mobile allows apps to work offline and then will automatically sync data to and from the cloud when network connection is available.
InfoQ: When talking about distributed data storage, we also have to mention CAP - I suppose there’s a lot of P when being mobile. Which of C and A are available when using Couchbase mobile and what trade-offs do we have to expect?
With Couchbase Mobile, we have an AP system. We guarantee availability and partition tolerance. Every request receives a response because the database is local. The tradeoff is at the consistency level. Reconciling inconsistencies is the key concern. This is reconciled using multi-master replication and conflict resolution. Couchbase Mobile resolves replication internally, replicating data throughout the system resulting in eventual consistency. Conflict resolution is facilitated by Couchbase Mobile which allows reconciling conflicts on the client or on the server (e.g. n-way merge, most active branch wins, last in wins, etc.)
InfoQ: When working with distributed data, there is also a lot of security related discussion. I suppose that thinking about security upfront is even more important in the field of IoT than in mobile apps. Usually there are less options to update IoT devices compared to smartphones and tablets. In what way will Couchbase mobile protect our data and can those techniques also be leveraged for IoT?
Security is extremely important when you start talking about mobile devices and IoT. For one, your users have to trust that you are protecting their data. Second, data is being transferred over the internet which can lead to security vulnerabilities.
Couchbase Mobile handles security in 5 areas:
For User Authentication we support pluggable authentication. Out of the box we have support for popular public login providers like Facebook or you can write your own custom provider. You can also restrict access to the system to successfully authenticated users or optionally allow anonymous users.
For Data Read/Write Access there are fine-grained policy tools that allow controlling data access for individual users and roles. Read-side permissions are at the document level and write-side permissions are down to the field level.
Data Transport on the Wire, for data in motion, is over TLS.
Data Storage on Device, for data at rest on device, uses the device’s built in File System Encryption and additionally data-level encryption. For data-level encryption, Couchbase Lite encrypts and decrypts data as you read and write it from the database.
Data Storage in the Cloud, for data at rest in the cloud, you can configure Couchbase Server to use File System Encryption.
All of the above security features also apply to IoT.
InfoQ: A lot of distributed data solutions also offer distributed data processing. How important is this for mobile devices and is this possible at all for IoT devices? What kind of restrictions do we have to keep in mind when distributing our data logic?
Yes, it’s important for mobile devices and yes, it’s possible for IoT devices (we treat them the same). We have map-reduce built throughout the entire database stack - both in Couchbase Lite and Couchbase Server. This allows data processing on mobile devices, IoT devices, and in the cloud.
One of the key things to keep in mind is the consistency of data processing, meaning data processed at one point in the system (e.g. a mobile device) should be consistent with data processed at another point in the system (e.g. another mobile device or the server). Where you can’t guarantee consistency (e.g. with one mobile device using v1.0 of the data processing and another using v1.1), you must guarantee compatibility between the differing results. The system must allow for this.
InfoQ: What is the vision that Couchbase follows when we think of a connected world in the future? What will data storage, processing and communication look like in five years? How will this scenario impact our work as architects and developers and how will it feel like from a user’s point of few?
We are solving the data storage, transport, and security concerns at every layer of the application stack - all in a single integrated data platform.
More and more, users will be reliant on their devices for everything. With that in mind and to my point above, architects and developers must design applications that will always work. Networks are not and will never be ubiquitous, but it will be an expectation that applications (mobile, IoT, etc.) will always work.
We also think NoSQL will play a much larger role in mobile going forward. Today, mobile developers know about NoSQL, they know it exists, somewhat understand the value but they haven’t been exposed to it in a big way yet. (NoSQL has traditionally been a server-side technology). However, we’re beginning to see a trend in the market with all the newest mobile database technologies being some form of NoSQL (e.g. Couchbase Mobile, Google Firebase, Parse, Amazon Cognito, etc).
Synchronization technologies will also play a big role in resolving the transport concerns of the distributed system. With apps needing to provide more and more offline functionality, they’re going to need a technology that enables this. Synchronization is what allows for that. There’s no way to get around it - mobile developers will have to learn and get comfortable with this new technology to power application in our growing distributed world.
Couchbase and Couchbase Mobile - including Sync Gateway and Couchbase Lite for mobile devices - can be downloaded free of charge as a community edition for several platforms including Linux, Mac and Windows on servers and iOS, Android, Java and more for devices. There are also client libraries for .Net, Java, Node.js and others and the possibility to integrate Couchbase with Hadoop, Elasticsearch, Spring and Solr via plugins.
Pricing for enterprise versions with additional features and support can be requested via Couchbase’s website.
About the Interviewees
Wayne Carter, Chief Architect of Mobile, Couchbase Wayne is the Chief Architect of Mobile at Couchbase, where he is responsible for leading vision, strategy and development for the company’s mobile solutions. Before Couchbase, Wayne spent 7 years at Oracle as the Architect responsible for driving mobile innovation within the CRM and SAAS product lines. He has 10 patents and patents pending from his work there. Prior to Oracle, Wayne held technical leadership positions at Siebel working on their CRM product line.
Ali LeClerc, Product Marketing Manager, Mobile, Couchbase Ali is the Product Marketing Manager for Couchbase Mobile, where she manages and executes the worldwide marketing strategy for their mobile products. She joined Couchbase in 2011 to drive open source product adoption and awareness. Prior to Couchbase, Ali held marketing positions at Time Warner. She holds a degree from Yale University in Political Science.