Transcript
Lerner: Today I'm going to talk to you about machine-to-machine interfaces. Before I do that, I want to introduce who I am. My name is Ari Lerner, I live here in Manhattan now, I used to come from San Francisco. I work at AWS. However, what I'm going to be speaking on today does no
t represent the views of AWS. These are my own views and I'm not speaking on behalf of AWS. It's important that I say that and that you remember that, and you don't tweet that I'm speaking on behalf of AWS, because I like working there. If you want to follow me, I'm auser on Twitter. Also, I put a lot of photos of her, so if you find her adorable, follow me on Twitter.
Even though I am going to be talking to you today about a technical topic, I'm actually going to be mostly focusing on the UX and the CX side, and you'll understand why as we get through the talk. When we talk about interfaces, specifically when we talk about interfaces in software, we generally talk about buttons, button colors, where should something go? Does it make sense if I put it here? Do we need another menu? Things like that. But one thing that we forget about is that we don't operate in a disconnected world. You don't step away from your desktop machine at home and you're, "No one can reach me." You probably already know that as we grow in interconnectivity, we have an expectation that things are going to happen the way that we expect them to happen. I'll give you a few examples in just a minute.
Before I give you those, CX stands for customer experience and UX stands for user experience. These are very distinctly different terms from UI. This is specifically the experience that you have as a user, you have as a customer. The more interconnected in our world that we get, the less of these interfaces that we both expect that we need, and do we need actually to interact with the world. For instance, when you walked in here today - I would be willing to bet - that you expected to walk in, get your badge, and go to any session that you wanted. There was an expectation there. That expectation was fulfilled by machines, it's fulfilled by you registered online, you put your credit card information in or somebody did on your behalf, and that your name and your email and some UUID was generated in some database, Google form, still database. Maybe MySQL, maybe Mongo.
Before today, that database was tweaked and pruned, maybe there were duplicate entries. Maybe the person who was actually writing that code actually didn't need to do any of that tweaking, although it's usually pretty rare. That database was printed and used as a source of truth to generate your badges. No one did that. Somebody sent out your names off to some printer and that printer fed that into their printing machines and did all the printing and then got those back in big boxes. That's all machine interaction. That's an expectation that we have in our world today.
Another example is going to the movies, this is the classic example where you buy a ticket. If you're cool like me and you forget, you buy a ticket on your way to the movie on your phone, you get to the movie theater, you present your QR code, they scan your QR code and you walk in and you stuff your face full of popcorn as you watch Jar Jar Binks. What happens behind the scenes is when you purchase that ticket, your name, just the same as getting into this hotel, you purchased that ticket whether you have a user name in their system or not. That ticket was created with a UUID. A UUID is sorted in a database that's used to generate a QR code which is that ticket. Then when you get to the movie theater, you get that scanner and that's the person who gets paid minimum wage, scans your QR code and says, "You can actually go to see this movie." Those are machines talking to each other, there's no human in that process.
This is all expected, this is a user experience expectation that we have. When you get in your car, you expect that your car is going to turn on, you expect that the GPS is going to work, you expect that you're going to be able to get into that movie. These are two really minor examples of what I'm talking about today, and obviously what I'm talking about today is all about expectation. In this talk, I'm focusing specifically on the technical and non-technical side. How can I be specifically speaking about two different things? I'm going to take back the specifically. Today I'm going to be talking about the technical and the non-technical implementation of this.
Let's get ready to forget the UI for this talk, and just talk about how machines interact with each other. In order to support a lot of this interaction, I like to call this machine-to-machine communication, I think it has a nice ring to it. This machine-to-machine interaction - we need a protocol to talk in between different machines. Was anyone alive during the '80s and on a keyboard here? Do you remember that time when everyone was coming up with a new protocol? It was, "Things are talking back and forth." TCP got settled on and then the government stepped in and was, "HTTP is the best one ever," and yet we all still disagree. Those are the Wild Wild West days of protocols.
Today we generally use something that we're all really familiar with, and that's HTTPS APIs, some people call it black boxing, the APIs, that's cool too. We generally will use HTTPS with a JSON today. That is enough of a protocol, we all understand it specifically in this room. JSON stands for JavaScript Object Notation, it describes a JavaScript object, it's a competitor to XML. If you're a competitor, if you come from the Javaland, it's a competitor, but if you don't know then it's not. The demo that comes without a car was generated using open source components. I have an embedded controller that's Arduino I'm going to talk about, the RFID reader, a NodeJS back-end server which is running on my laptop, and I'm also interacting with the Ethereum blockchain, because why not?
Another couple of things that we're going to do, we're also going to talk about other possible implementations of what I'm actually doing. We're going to talk about little edge cases around why other possible implementations would make more sense or less sense. We're going to talk about updating and upgrading both your hardware and your software, managing security, both updating in transit and at rest, and then I'm also going to talk a little bit about scale, not too much. If you're really super interested in scale, come and see me or tweet at me. I might follow you back if you follow me on Twitter.
Demo
In this obvious demo, the idea that I wanted to present today is paying for parking. Paying for parking is a pain. Who drives? You've all had to pay for parking. In the ideal case, you have no interaction between the user and the payment back-end. You don't put in coins in a meter, you don't scan a thing and put your numbers in and then put your credit card in and pay for that. In the ideal case, you just park your car, your car knows how long it's there, the spot knows how long you're there. Ideally, the spot knows who you are and charges you, that's an expectation that I have. I don't have a car, but I also expect that my cars fly.
A cool example of this is paying for bridge tolls. If you live or have ever been to London, London is pretty famous for this - you go over a bridge and they take a picture of your license plate and they use your license plate to charge your bridge toll. You don't have to have a little fast pass, unlike you do here. That's a cool example. That is, by the way, another possible implementation of this demo that I'm presenting today. I also actually started out by doing some image recognition in self-driving cars, and I decided that that was a little bit too complex to present here, especially without a car. If you are interested in learning more about machine learning, specifically deep learning and convolutional neural nets, I wrote a book on it. Come see me after, I'll get you a coupon code.
Rather than having really complex operations running on a really lightweight device, specifically lightweight for cost reasons- and we'll talk about that when we talk about other considerations - a much cheaper, more inexpensive, very well understood technology is the RFID. RFID is a zero or super low power interface. You get in your hotel, if you're staying in this hotel room, oftentimes they use RFID to get in there. If you have a key to get in your office it uses RFID, I have an RFID reader on my Arduino. That sounds like a good idea, RFID is super cheap.
In order to actually handle understanding how long your car has been parked in a spot, we need something to pay attention to the fact that there's a car in a parking slot. We needed some embedded controls actually handle reading the RFID, and taking both the state of the parking spot, "Is there a car here or is there not a car here?" and also, communicating that back to some back-end server. I just mentioned we need a back-end server - I have a HTTP server but we can use TCP just fine here. You don't want to use something UDP because you'll lose a lot of packets and then you'll lose data, and then you may never get paid for that parking spot that the city really cares about. We need a backend server to handle loading this, handling that data.
The other thing is, everything I'm talking about right now is, I'm talking about something that happens in the real world. These are embedded circuits that would go in the ground underneath a parking spot, next to a parking spot, above a parking spot. We need to make sure that we're not building something, A, too complex, B, too expensive, and C, that can't be updated, can't be migrated, can't be changed in the future, because things are constantly changing.
Taking the approach that we talked about before, let's take the CX/UX storyline that we're talking about here. Let's say that you're lucky enough to get a car from someone who really loves you. Maybe you purchase a car, or you rent a car, and you get a license plate that is associated with that car. That already happens, I expect that if you're driving a car, either you have a temporary license plate, that little paper one that everyone seems to lose, or you have one that actually sits on the back and you've already forgotten about the fact that you have a license plate until you're trying to find your car in a parking lot mall and you're, "Wow, there are three thousand cars here, How am I going to find mine?"
The approach that I'm proposing is that you would take a photo of a QR code when you receive your license plate in the mail, and you register a payment method. This payment method is something you do once, it's a UI interaction that you need. There's some data that says how do I charge you for parking. Then you drive around the city and attend your favorite movies, and you cross bridges and you get in accidents.
Behind the scenes, how I propose that works, is the license plate has an RFID tag in there. When you register for a payment method, that associates your license plate number which is unique to your car and your state, so there's going to be no data collusion between you and someone else. You're not going to pay for somebody else's parking.
When you move into a parking spot, you have an RFID tag reader that that sits underneath that parking spot and says, "There is a car here." Maybe you also have a motion sensor that pings at the same time and says, "Is there actually a car here?" Then while there is a car there, while the state of that parking spot is full, you pull until that car has moved. I propose 15 seconds is long enough. That's one state update at any given time for every 15 seconds, super fast. Also, it doesn't take that much power.
As soon as you move your car, the RFID tag reader breaks, or the motion sensor says, "There's no car here anymore," the parking space takes notice, sends your UUID and the amount of time - the start time and the end time that you were in the parking spot - back to some server somewhere. At the end of the month, all those charges are racked up and throw that charge up on, say, a blockchain, if you want to be a cool kid, or just VISA if not. I was not bagging on VISA, I think they're really cool.
. The features of this approach are, there's really limited user interaction. One time you put your credit card information in there. There are automated payment mechanisms, where you don't have to have somebody sitting there swiping a card a whole bunch of times. It's super simple hardware, which is really super important in production use, and it's a well-known technology. We're not betting on anything new, except for maybe the blockchain if we're going to actually use the blockchain. Plus, the last feature of this is, it's low power, you don't want this to draw a ton of power. You want it to pay for itself and then some. For example, if you're mining for the blockchain, it's now a losing proposition because it takes too much power to actually mine for the blockchain, to actually win the award. Even if you do win the block mining award, it's more expensive to actually mine for that award. Anyway, I just went really far deep into the blockchain.
How the Systems Work Together
Let's talk about how these systems work together. The hardware - I mentioned this earlier, the reason why I chose RFID hardware is because they're really common, they're ubiquitous. You have one if you work in an office, even if you don't work in an office, you probably likely have an RFID tag. They're inexpensive, they're mass manufactured. If you work at WeWork, you have one. If you don't have one of these guys, you have one. They're ubiquitous, they're everywhere. RFID tag readers similarly are the same thing. One thing that's neat about RFID tags is that you can write to them. You can write to them and make sure that those are unique to the RFID tags. If you have a unique ID generator, for instance like a license plate, you can write that license plate tag to your RFID, or you can write your license plate unique identifier to your RFID tag.
In this demo, we're using an Arduino but we can get reduce the costs over time, because we can either design the actual interface that we want on our own circuitry, which will reduce the costs of mass manufacturing, or you can even use cheaper technology and not have to worry about that, something like the ESP820, 8255 which is a super cheap one. Then we need a way of sending this data back to our central control. In this case, I'm using Arduino with Wi-Fi connected to my laptop. A more secure method - you can sniff that, that's not super secure, in fact, I hope you're all using a VPN here because I'm sniffing for your e-mail passwords and I am reading all your emails. I'm just kidding, I'm not actually doing that.
This is a super simple example of the server I'm running. By the way, I should have mentioned, all of my talk will be open-source on GitHub, so I'm a user there too. Then we have some back-end server that says, "I have your I.D., I have the amount of time that you spent in that parking spot. I'll roll up those charges, at the end of the month or the end of the year, or at the end of the day, for that matter, and I'll charge you." Finally, the payment gateway, Ethereum sounded fun, so we're using Ethereum, Ethereum in JavaScript is super easy. That's the whole demo. Pie-in-the-sky, right? We're done.
There are a couple of other things I want to talk about. Is this the right implementation? “What if things fail?” is a question that I get all the time. How do we ensure that payment happens? Maybe I should list that at the top. That's probably the most important question. What about upgrading both hardware and software? What if the hardware breaks, or you take the Arduino's out and you want to put in your embedded code, how do you update that? Then we're going to talk about security, because security is really important.
Is This the Right Implementation?
Addressing each one of these one by one, is this the right implementation? There are other ways to handle this, you never really know until you start experimenting. This is a big core tenant at AWS and this is a core tenant of anybody who's ever run a company before. Experimentation is super important, if you don't agree with me, that is totally cool. I will duke it out with you later.
What are some other possible implementations? One I thought of when I was starting this, was a raspberry pi with image recognition, have a little camera that runs, that's one way to do it. Maybe on the parking meter, you have a little scanner, but then you have this human interaction that has to happen. Can anyone come up with another way of handling automated paying for parking without thinking about it? Maybe a GPS device on your phone that matches up to a parking space, although GPS is not super accurate. One of the things that I want to mention, one of the reasons that I'm asking this question, is because as we think of other ways of handling this, notice that there are machine-to-machine interactions in everything I just said. Even though there is a user scanning a QR code on a parking meter, there's still something that has to happen between a machine to another machine that says, "Someone just scanned, put their credit card information over the cell network and they're sitting in this parking spot." The machine has to actually take that QR, take that information, sort that new database, and then later some process comes through and runs charges for that.
Failure
What if things fail? I dislike this question. I think if you're not failing, you're not doing enough, and if you're not doing enough, you're probably not making any impact in the world. I think operating with failure in mind is a normal operation mode, so I don't think that question should actually exist. I think the question really should be when things fail. What do I do when things fail? If you're designing for failure, then you're designing a robust system and you're already pre-optimizing for the case in which the batteries run out of my Arduino. What happens then?
The first thing that I would say is tell your users. Second thing I would say is tell your users. The third thing I would say is tell your users, and I'm pretty sure I would say tell your users one more time. The reason why I say that is because when we're talking about UX and CX, that communication is really important, because trust is really hard to earn, but super easy to lose. So tell your users if something is failing, specifically when we're talking about hardware. If you're going to purchase movie tickets and you're walking into the movie theater and the charging gateway is down for whatever reason, maybe there's a maintenance upgrade and you're seeing a midnight movie, and nobody would assume that anyone would be buying a ticket at midnight, tell your users that. "I'm sorry. You have to go and go to the old-fashioned way and go see someone at the movie theater to purchase your ticket." That being said, did I make that point? Anybody want to tell me what the point that I just made was. I said it four times.
Participant 1: Tell your users.
Lerner: Yes. Tell your users.
Participant 2: Can I ask a question, please? For example, with RFID in your case, the program or what is called protocol expecting a [inaudible 00:28:36]. What would be the kind of used protocol in this case?
Lerner: What would be the protocol?
Participant 2: How you would signal use or aid the [inaudible 00:28:45]?
Lerner: I see what you're asking, that is a really good question. The question was, if there is no interaction between the user, how do you actually interact with the user? Super good question. I don't have the answer to that, actually. My first instinct would be when you register with your payments, if you have an email contact or phone contact, send a push notification that says, "I'm sorry. The parking spot is broken. You can leave your car here, we're not going to charge you for it," but definitely you'd want to actually have some interaction with your user for sure. Everyone likes free parking. Then all of a sudden if you live in, let's say Austin, Texas, all of a sudden you like Austin even more, because they said, "Our bad. You get free parking." Also, maybe free barbecue.
Participant 2: Where could we stop [inaudible 00:29:56].
Lerner: He said too much free stuff. I would argue that trust is the most important asset that you have. Whatever product that you're building, trust is the most important thing that you have. Getting back to failure, it's actually really important for us to define what failure actually is, because you can say, "What if it fails?" Failing is not charging a user because that's obviously why the city invested in this product.
The way that we determine what failure is, the way that I would suggest that you determine what failure is, is by using some metrics. Is it responding when we send network data, or is my Arduino know actually up? When was the last time it responded? That might actually be pretty important because you might be seeing significant latency. Despite the fact that it's actually still working, maybe a line got cut because there's some road work down the way and you still have some connectivity, but it was charted a little bit. That's still a failure, even though the whole system is actually still working.
Another question is, does the hardware actually work? In my case, is the Arduino actually up? I'll go back to that. These bring up other issues, things that we need to do. We need some way of keeping track of what hardware we actually have. We keep track of that response time. That's part of that other system that we need, and also hardware pings. Hardware pings actually are a difficult thing to do, specifically if you don't have a control-plane that you're operating off of. If you're not familiar with a control-plane, Kevin Mitnick, super-smart guy, he's the guy who found out that in the '80s if you blew into a 2600 hertz whistle which came in the Cracker Jack box into a payphone, you could get free phone calls. It's an interesting story.
As I said before, if you're building your system with the expectation that things are going to fail, you're already optimizing for the future. Things like cost - if my hardware is going to fail, I want to make sure that replacing that piece of hardware is going to be the least amount of money that is going to be. That includes both training and implementation, as well as hardware building.
Simplicity is really important. This is something that a lot of software people generally tend to overlook in general, is simplicity. It's really a design feature that your software doesn't need a manual to operate. You open your iPhone, you already know how to send a text message, and that's because there's a focus on simplicity there. I would also argue that when you're doing physical interactions, simplicity is really important both from the user perspective, as well as from a maintenance perspective. You want the person who is replacing your device in the parking spot to need as little training as possible in order to get that device in the parking spot.
Availability is also really important. RFID tags are available and ubiquitous everywhere. There are a ton of different creators. If you're using something unique - SONY, I'm looking at you - then your manufacturers are going to be the ones who are going to be your bottleneck. The same goes for accessibility. Then also, if all the power goes out or there is a line that's cut, you want some backup power, so batteries are a really easy way to do that. If you're embedding this in a parking space, I would argue that underground you actually have connectivity to power. Specifically for this demo, the idea is that you have some backup power.
Designs for Network & Server Software Failure
To design for network failures, backup networks. You might operate over a hardware line but most modern cities, especially the modern cities who invest in automated parking, generally tend to have Wi-Fi. Wi-Fi is not a bad option if your hardware network fails.
Onboard storage - this one I would argue is actually almost more important than anything else when talking about network storage, which is rather than when that car leaves sending out a single update, batch process that. The reason is because before you go and you send off a query to a backend server, you can detect whether or not you're connected to the Internet. If you're not connected to the Internet, no need in engaging that radio wasting valuable power resource, especially if you're operating on backup power.
Instead, you can send batch process. You can send a hundred parking spot notifications at a time. That also helps with preparing for network failures. Also, what happens if the node server goes down? "Oh no. We only have one and it's running on my laptop and I close my laptop, now no one pays for parking." One way that you can handle that is horizontal scaling. At AWS we do a lot of that, and every cloud provider on the planet provides you with the ability to do automated horizontal scale-up.
Let's talk about what happens if your server software fails. We want some way of easy upgrading. What happens if you throw an exception or a junior developer comes in and decides that they don't like try-catch, and they say, "It's ok if it fails. I have it tested." We want some way of easily upgrading your server. Centralized DNS control is really exciting. I'm being facetious when I say it's really exciting, it's not really exciting, but having some way of managing your DNS is really practical and important, especially if you're horizontally scaling, how do you connect to all those different nodes that you're scaling out?
Also, automating as much of your infrastructure as possible. You don't want to have to wake up at 3:00 in the morning and run a command and then go back to sleep, and then forgot that you didn't put a period at the end of that command. I don't know why you'd need a period at a command line, but maybe you actually were like, "Ah," and you put an exclamation point and that means something totally different in Bash.
How Do We Ensure Payment Happens?
How do we ensure that payment, in this case, how do we ensure that payment happens? I would ensure that having a backup method of doing payment makes sense. One way you can think about that is how does it actually operate now? The way that you pay for parking today is you put coins in a meter, you take a QR code image of the thing, you put your credit card in some device, some box in the middle of the street. But ultimately, the way that parking attendants ensure that you're paying for parking is by issuing tickets. There's no need to remove that process, especially when you're switching over to a new system. The proposed solution that I'm offering here adds data, and it makes them more empowered. It gives them more data, it allows them to take a longer lunch break because they know they can see who's in a parking spot and for how long.
Let's talk about upgrading hardware really quick. I think a really awesome way of handling upgrading hardware is treating it like software. Hardware is going to fail and we should expect it to, so the cheaper that we can design our hardware solutions, off-the-shelf commodity hardware, inexpensive hardware, that's one way that you can mitigate dealing with how you update and upgrade your hardware. Upgrading firmware - if you're not familiar with firmware, firmware is a software that runs on those little black chips that are on your computer. Upgrading firmware is actually another interesting piece of this puzzle. On this, I would argue if you're integrating any hardware in your system, this is really important, specifically for firmware updates.
The way that I would handle this is you can use something like - I think I mentioned it before - the ESP8266. It's like a little thin board that sits on top of an Arduino that allows you to update your Arduino by flashing or by sending a new firmware to that device. Those are super easy, they're also really super cheap. I would argue that is also really important.
Security
Finally, let's talk about security and skill. Two main things that I'm going to talk about today are encrypted data at rest, and encrypted data in transit. Also, it shouldn't matter if I'm talking about this demo that I'm talking about today, or we’re sitting down and looking at your source code that you're actually writing or your developers are writing, that approach is really important. You want encrypted data at rest and you want it encrypted in transit.
When you're deploying something in the field, if you have PII Information, Personally Identifiable Information, like someone's blood type, for example, and someone gets ahold of that device, you're in a whole big world of pain. Luckily for us, we're not dealing with any PII information, or a license plate which is effectively a UUID, or not even effectively, literally the is UUID that is mapped to a license plate and two timestamps, or one timestamp depending how you implement it.
You don't have to do this, you don't have to approach it in using the HMAC protocol, but this is a really well-understood way of sending verifiable data over the network. I would argue you'd still want to send it over SSL so it's encrypted, but you can also verify that on the backend server you're receiving what you expect from a client that you expect to be receiving it from. I always suggest SSL or TLS encryption at a bare minimum. These are super well-understood protocols. Because we have an embedded device, we can just embed that certificate in there and have no problem with it.
Then finally, the control-plane. A control-plane is not the layer that says, "How much time has this person spent in a parking spot?" but, "Is my hardware responding?" It's a different network that is segmented for security reasons. I don't want just anybody to be able to restart my Arduino because they have access to the data plane. Instead, I want only people that I trust to be able to restart it if they need to restart it, or send a ping to it or upgrade the firmware. For example, I would say the way that you can handle the security is run it over a VPN, which is encrypted and only allows the people on the VPN in that network to actually have any access to that control-plane layer.
Lastly, weird things happening in scale. Huge data growth skews numbers. If you have a dashboard, for example, say you're using D3 and you're not using the scale function, you might have a huge spike, but that really represents something really tiny just because you weren't expecting the amount of people trying to park to go watch the Lakers game. Addressing gets weird and kind of funky, and also, if something isn't responding, how do I know which Arduino to ping on my control-plane? Especially at scale, when you get 100,000 sitting devices, that's an actual full-time job, trying to figure out how to do that mapping. Naming becomes harder. Naming is always hard, but naming becomes harder.
Takeaways
Here are the takeaways from today's talk. Not all user interaction has to have a user interacting. I actually think that the majority of user interaction should not have a user pressing buttons and putting credit card information. Failure is a part of the process, embrace it, design for it. You'll be thankful a month from now when you don't remember what your code looks like or you looked at your code and you're like, "Did I write that?" Failure is part of the process. Design for it. If you don't, if you're afraid of failing, you can't have any successful experimentation, because experimenting literally means it fails or it doesn't, not it succeeds or it doesn't.
Also, hardware isn't scary. It just requires a bit of planning and a little bit more forward-thinking, and if you are doing something using hardware, don't give up. I know that it can be difficult and especially jumping out of software and into the hardware land, that can be scary. But if you're doing it, you're probably making a difference. Physical interaction between you and your users is only going to make things better. If you get this right, point set match.
Finally, UX is not just about users; it's about the interaction of the users. UX is about the users, it's not about the product. You're not integrating UX into your product; you're integrating your user experience in your product, which means UX is not UI.
Finally, I have a prediction. The next big thing that happens, you won't see it because it will be behind the scenes and it's already happening, like AI.
See more presentations with transcripts
