Transcript
Palmer: I'm Chris Palmer from the Chrome security team. I'm going to talk about getting the most out of sandboxing, which is a key defensive technique for software that faces the internet, like a browser or a server application, or any sorts of things. I'm going to talk about the limitations and benefits that we found while doing this. We've had about 10 years of experience. I'm going to talk about what else to do in addition to that once you've hit those limits, but you probably haven't yet.
I've been on the Chrome security team for about nine-and-a-half years. Before that I was on the Android team. I've done a bunch of different things. I've done usability, HTTPS, and authentication stuff. These days I'm on what we call the platform security team, where we focus on under the hood, low-level security defense mechanisms, like memory safety, and sandboxing, obviously, and exploit mitigations. We try to make sure we're making the best possible use of the operating system to get the maximum defensive value out of that.
What Is Sandboxing?
First, let me give you an idea of what sandboxing even is. I'll talk about how we do it, what it means, and how you can use it. Then, I'll talk about stuff that comes after that as well. Here's a picture from Chromium's website, chromium.org, where we have a simple picture of what sandboxing looks like for us. Chrome is a complex application, obviously. We break it up into multiple pieces, and each of those pieces runs in a separate process. We have like the browser process, here it's called the broker, and that runs at full privilege. It has all the power of your user account when you're logged into your machine. Then we fire off a bunch of other processes called renderer processes. Their job is to render the web for a given website. If you go to newyorktimes.com, there's one renderer for that. Then when you go to Gmail, or Office 365, or Twitter, that's three more renderer processes. They're each isolated from each other, and they're isolated from the browser process itself. They also have their privileges limited as much as we can manage in ways that are specific to each operating system. What that gives us is, if they crash, of course, the whole browser does not go down. If site A causes its renderer to crash, all your other sites are still up and running. You get reliability and stability. Also, if the process is compromised by malicious JavaScript, or HTML, or CSS that helps take over a website, usually it's JavaScript or WebAssembly, then the damage is contained, we hope, in that renderer process. If site A gets compromised, sites B and C and D, have some protection against that compromise. The browser process itself up here, is protected against that compromise. The attacker to take over your whole computer, for example, would need to find additional bugs, whether in the browser process, in the operating system kernel. If they want to extend the reach of their compromise, they have to do more work. It's a pretty darn good defensive mechanism, and we use it heavily.
Good Sandboxing is Table Stakes
I want to say, first of all, that sandboxing is table stakes for a browser, and every browser does some: Firefox, Safari, all the Chromium based browsers like Edge, Brave, Opera, Chrome itself, obviously. They all do sandboxing. It's not just for browsers. I think you can get a lot of benefit from sandboxing in a wide variety of different application types. For example, the problem is, we are reading in complicated inputs, like JavaScript, HTML, images, videos, from anywhere on the internet, and there's a risk there because we're processing them in C++ or C. There's all sorts of memory safety concerns. There's buffer overflows, type confusion, use-after-free, bugs like that, that can allow an attacker to compromise the code that's parsing and rendering these complicated formats. Any application that parses and renders and deserializes complicated stuff has the same problem, and it has to somehow defend against it. That includes your text messenger, a web server that takes in PDFs from users and processes them, for example. If you have a web application that converts images from one format to another, or takes videos from users or a font sharing website, or all sorts of things like that, you have the same basic problem. You don't want to let a bug in your image decoder or your JavaScript interpreter, take over your whole server. You don't want to let an attacker have that power.
In Chromium, we've been working on this for about 10 years. We think we've taken sandboxing just about as far as we can go with it. I'll talk about how we face certain special limitations that you might not, depending on your application. You may be able to go further with it than we can, or maybe not as far as we can. It all depends on a bunch of design factors you have to take into consideration. I'll talk about what those are. Again, in any case, sandboxing is like your step one of all the things you want to do to defend against bad inputs from the internet. I think it's table stakes. You got to start there.
How to Build a Sandbox
How do you build a sandbox? It depends what you can do. It varies with each operating system. Android is very different than macOS, and Linux is a whole different thing. Windows is a whole different thing. They all provide different options to us, the defender, they give us different kinds of mechanisms. We have separate user accounts in Windows that are called SIDs, security identifiers. There's various access tokens. You can take a token away from a process or give them a restricted token. That's a Windows thing. You can filter the system calls that a process can call. We do a lot of that on Linux based systems, Chrome OS, Android, Linux desktop. You can do it on macOS with a Seatbelt system. We bend the rules and define our own Seatbelt policy, but Apple gives you some baked in ones with Xcode. I think these days the default is not to have full privilege anymore. They make it easy for you to reduce your own privilege with Seatbelt. That is very powerful, very good defense. Another idea is that you could segment memory within the same process. What if we had a zone that code could not escape from and then another zone that other code could not escape from? I'll talk a bit more later about how we are looking at that thing, and Firefox people are also. It's a very cool idea. For the most part, our basic building block is the process. Then we apply controls to different types of processes.
OS Mechanisms (Android)
On Android, it's very different than Windows and Linux, the key mechanism is what is called the isolated process service. It's a technique that they invented for us. We asked them, could we have this on Android, and then we can make Chrome better? They said, sure. They built it for us. It does a couple things. It runs a new process. It runs an Android service, in its own special separate process with a random user ID separate from your main application, and then you can talk to it and get data out, and parse data in. That's our first line of defense on Android. Renderers run as isolated services. It works pretty good. There's a couple things that come with it. There's some system call filtering. It comes with a policy that Android platform people define for us. They also define a SELinux profile for us. SELinux stands for Security-Enhanced Linux. It's an additional set of policies you can use to say, don't let this process access these files, or, raise an alarm if it tries to do this or that. That's useful for us too. It comes with the isolated process service. That's number one on Android.
OS Mechanisms (Linux, Chrome OS)
On Linux and Chrome OS, we put together what we want by hand. We use Seccomp-BPF. Again, it's a system call filtering mechanism. We can define whatever policy we want, so we have all sorts of finer-grained policies for different types of processes. We also use Linux's user and PID and network namespaces feature, where you can create a little isolated world for a process, and it doesn't get to see the whole system. It doesn't get to see processes that it shouldn't know about. It doesn't get to see network stuff that it shouldn't know about, and so on. Where we don't have that, not all Linux systems enable that subsystem of the kernel. We have a set UID helper that does some of that stuff itself, and that it spawns children and takes away their privilege, and then those are the renderers. We consider that to be a bit of a legacy thing. We think the future and certainly our present is the namespaces idea. It's not risk-free. Namespaces come with bugs of their own because they change the assumptions of other parts of the kernel. That's why some distributions don't turn it on, but we do on Chrome OS, certainly. We think it's pretty useful.
Limitations and Costs
The key thing here is that you can't sandbox everything. If you think about the most extreme form of sandboxing, you could sandbox every function call or sandbox every class. Especially if you had that segmented memory idea that I was talking about, you could give each component of your code its own little zone to live in, and it couldn't escape, we hope. As it is now, for the most part, we have to pretty much create a new process for each thing that we want to sandbox. You'll see in the pictures that are coming up, we use fairly coarse-grained form of sandboxing, because processes are our main thing, and processes are expensive. On Windows and Android, processes are a big deal, and threads are cheap. They want you to do that. The way the systems are designed, you'd have one process for your application, and many threads used during different things. That's not enough of a boundary for us. We have to be careful in how we use them. Then starting up a new process on Android and Windows, a new process gives you a lot of stuff that you typically want, like all the Android framework stuff, and all the nice Windows libraries. It costs time and memory to create those things for us. They're not quite as free on those two platforms. On Linux and Chrome OS, they're very cheap, indeed. macOS also quite cheap to make a process. We face different headwinds on different platforms.
Site Isolation
A key thing I mentioned before, and this is going to be the introduction to how you should think about sandboxing for yourself, for your application, different sites are in their own renderers, and so we have to have a way of deciding when to create a new one. We would like to create a new process for each web origin, which is the principal in the web platform, the security principal. The origin is defined as the scheme, host, and port like HTTPS, google.com, Port 443, or HTTP, example.org, Port 80, things like that. Each of those should be separated. We can't always afford to make that many new processes. Instead, we group several origins together, if they belong to what we call the site, and that is just the scheme, like HTTPS or HTTP, and just the second level domain after the first register or bold domain, like google.com, everything under google.com counts as one site. Example.org, everything under that counts as one site. It's a bit of a tradeoff to save some time and memory, but if we had our way we'd isolate each origin in its own process.
Here's a simple view. This is a bit like what the status quo is, but it's a little trickier. We have the browser process with full privilege. It creates different renderers for different sites. We also have the graphics processing interface, the stuff that talks to the graphics system of your operating system. We put that in its own process to separate it. It might be crashy. We can reduce its privilege a little bit, so we do. It doesn't need the full power of the browser process. We have coming up on most platforms, a separate process to handle all the networking too, all the HTTP, all the TLS, all the DNS, all that complicated stuff, we're putting it in its own process. We're, on each platform, gradually reducing the privilege of that process on each platform. It's an ongoing adventure. It's done on macOS. It's starting to happen on Windows. We got a plan for Android.
Now, you might imagine that, if we had all the memory and time we wanted, we could make a separate networking process for each site, and then it would be linked to its renderer. That would be great. We would do that if we could, if we could afford it. Similarly, we're creating a new storage process to support all the web APIs that handle storage, like the local storage cookies. There's an IndexedDB database API for the web. That stuff is also complicated and makes sense to put in its own process, so that's happening. You could imagine, we could have a separate storage process for each site also. Again, it would be linked to its renderer. Again, they would all be isolated from each other. Security would go up, resilience and reliability would go up. It would be cool. For us on the platforms that are popular for Chrome, mainly Android and Windows, it's just too expensive, so we can't do it. That might not be true for you, on for example, a Linux server, where processes are cheap and fast to start up. You might be able to have many different sandbox processes supporting different aspects of your user interface, or doing the things for your users that you need to do, whether it's database processing, giving them their own frontend web server, maybe hosting some JavaScript for them, things like that. It depends.
Moving Forward: Memory Safety
That doesn't get us everything we need, though, but it does get us a lot. The key thing you may have gotten so far is that we're using sandboxes to contain the damage of the problems of memory unsafety, like C++, and C, are just hard to use. It's all sorts of, out-of-bounds Reads, out-of-bounds Writes, objects leaking and they never get cleaned up when you're not using them anymore. That can happen. Or you use them after they've been cleaned up, use-after-free. It's an exploitable vulnerability, a lot of the time. Or type confusion if you have an animal, and it's actually a dog, but you cast it to a cat, and then you call the Meow method on it. That's not going to work, and trouble may ensue. C++ will let you do that a lot of the time, whereas other languages wouldn't. Java, for example, would notice at runtime, this is a cat, and it would raise an exception. Not so necessarily in C++.
Containing memory unsafety is a key benefit of sandboxing. It also gives us some defense against stranger problems, like Spectre and Meltdown, the hardware bugs where, even with perfect memory safety, you could do strange things and the hardware would accidentally leak important facts about the memory that's in your process. You can get a free out-of-bounds Reads, even if the code were perfect, which is pretty wild. We get some defense against that from sandboxing. However, the real fix for that is at the hardware level. As an application, I and we can only do so much. It's a little tricky. There are variants of that problem that can in fact go across processes and even into the kernel. That's pretty exciting. We have to just wait for better hardware to get rid of that problem. There's a lot we can handle before we can get there, before that's our biggest problem. C++ is dangerous. C is dangerous. Sandboxing helps a lot. To get all the way to Wonderland in software, we really would like to have a language that defends against memory unsafety, baked in.
Investigating Safer Language Options
You could think, obviously Java has a lot of that, Kotlin. Swift on macOS and iOS has a lot of safety baked in. Rust, that's their selling point. WebAssembly has a chance to give us that isolated box inside a process so we could create little sandboxes without having to spawn new processes, and so we could have cheaper sandboxing and we could sandbox more stuff. We're actually experimenting on that now this quarter. Firefox also is. They have a thing called RLBox. We call ours WasmBox, for WebAssembly Box. The basic idea is the same. There's different efficiency tradeoffs and different technical ways of going about it. The basic idea is, you give a function or a component, a chunk of memory, and then you by various means enforce that it can never read or write outside that area. If it goes wrong, it's at least mostly stuck in there. It might still give you a bad result, and you'll have to evaluate the result for correctness. It can at least, hopefully, not destroy your whole process and compromise the process and take it over. We'll see how that goes. We're hoping to maybe ship something with it soon. It could be a very big deal for us, and perhaps for you.
Migrating to Memory-Safe Languages
We'd also like to migrate to a safer language to the extent that's possible. No one's saying we're going to rewrite everything in Rust or replace all of our C++. That's not possible. What we can do is find particular soft targets that we know attackers are attacking, and we can replace those particular things with a safer language. Like you could take your image processors, your JSON parser, your XML parser, for example, and get Java, or Kotlin, or Rust versions of those and put them in place. Then you could have a much smaller memory safety problem. That would be a complementary benefit to sandboxing. Neither one by itself gets you all the way there. I think they're both necessary. Neither is sufficient on its own. Together, I think you have a really solid defense story, at least as far as software can go. That's where we're heading. Of course, hardware continues to be a difficulty.
There's also the matter of the learning curve of a new language. To get everybody to learn C++ is hard. It's a complicated language. Any language that can do what it does is likely to be at least as complex. Rust does a lot. It takes a while to learn it. Swift does a lot. It takes a while to learn it. There's a way to use Java well. It takes a while to learn it. We're asking more of our developers, but we're thinking in the end that we're going to get better performance, better safety, for sure. That it will be beneficial to the people who use our application.
Improving Memory-Unsafe Languages
Again, you can think along the same lines, don't use C++ if you can avoid it. If you're starting something new, have a plan to migrate away to the extent you can. Getting the two languages to work together is a key aspect of the problem, and it's improving. In the last year alone, we made great headway with a thing called CXX and autocxx. It's a way to get Rust and C++ to talk to each other in a more easy to use way. It's very cool. We're also doing things with C++ as much as we can, garbage collection, new types of smart pointer that know if the thing they own still exists or not. Then it'll stop you from doing use-after-free, for example. Then there's new hardware features coming that can help us with the memory safety problem. There's memory tagging coming from ARM. Control-flow integrity, we're already shipping some of that in Chrome now. We're very happy about that, and there's more coming. Generally, we can replace some of the undefined behavior that's in C++'s libraries with our own. We've done a lot of that already too. For example, does operator brackets on a vector allow you to go out of bounds? In standard C, they don't guarantee that it's not, but we can define our own vector that does guarantee that it will not allow you to go out of bounds, and so we do.
These bugs are real and important, memory safety bugs, I should say. We have a wide variety. We have a lot of use-after-free, managing lifetimes is very hard. There's other bugs too.
The Future
The future is, sandboxing is giving us 10 good years. We're going to keep using it, of course. It's great. We need to move to our next stage of evolution, which is adding strong memory safety on top of that.
Questions and Answers
Schuster: How much variance is there for different architecture builds of Chrome?
Palmer: For different architecture builds, I assume you mean like ARM versus Intel, that hardware architecture? We don't yet have a huge variance, but we do expect to see a lot more coming in the future. For example, on Intel, there's a feature called CET, Control-Flow Enforcement Technology. It helps you make sure that when you're returning from a function, that you're returning to the same function you came from, which turns out to be pretty important. Attackers love to change that. The hardware is going to help us make sure we always return back to the right place. That doesn't exist in the same form on ARM. ARM has other mechanisms, for example, Pointer Authentication Codes, or PAC. It covers more and it covers things differently. It's along the same lines of the basic idea of control-flow integrity. We do different things there.
Then, similarly, there's a thing for ARM called Memory Tagging Extensions, where you can say, this area of memory is of type 12, and this other area is of type 9. Then it will raise a hardware fault if you try to take a pointer to type 9, and instead make it point to some memory of type 12. That'll explode. Similarly, you get a little bit of type safety in a super coarse-grained way, not in the same way you would get from Java, where it's class by class. You can get a pretty good protection against things like use-after-free, type confusion, and even can stop buffer overflows in certain cases. If you're about to overflow from one type into another, that can in certain cases become detectable. That doesn't exist on Intel. On ARM, we're hoping to make good use of it. Increasingly, we're seeing more protections of that kind from hardware vendors.
Someone also asked about memory encryption. Again, if we were to ever use that, it would be very different from one hardware vendor to another, and we have to do different things. As the future comes at us, I expect that our different hardware platforms may become, from a security perspective, as different operating systems. We can maybe have a roughly similar big picture, but the details are going to be totally different.
Schuster: You mentioned CET, which I think is being rolled out now. Has that been circumvented yet by the attacker community because it seems like every time some new hardware mitigation comes along, you think that's it, all buffer overflows are now fine. Then they come along, and say, "No, it broke ASLR and all that stuff."
Palmer: Yes. For example, there's already ways of working around PAC on ARM. There's three benefits to this. One is, they can make software defenses more efficient. Like for example, I mentioned earlier, a thing called MiraclePtr, where we're inventing a type of smart pointer that knows whether or not the thing it points to is still alive. There's another variant of the thing called StarScan, and that's like a garbage collection process where it looks around on the heap to see if there's any more references to the thing you're about to destroy. The trouble with that is it can be slow, because it has to search the whole heap. With memory tagging, it can speed up the scan by a factor of however many tags there are. If you have 16 different memory tags, it's weak type safety, but it speeds up scanning by a factor of 16, because you only have to even search for the particular one of 16 types on the heap. You don't have to scan the whole heap, you scan 1/16th of the heap. It speeds up software defenses.
Two is, the various hardware defenses work best together. If you combine control-flow integrity like CET and software control-flow integrity for forward jumps, like calls and jumps, and you combine them with Data Execution Prevention, where you can stop a data page from being used as a code page, for example, and you combine that with tagging, it starts to cover more of the attack techniques. You're probably never going to get all the way there, just by throwing hardware features at the problem. We can close the gap. We can speed up software based defenses. Then, performance, combining them to get a better benefit.
Then three, is, it really does make the attacker do more work, at least at first, but maybe even in continuing. It's like, yes, some very smart people can work around PAC or maybe even break it. Doing so, can be anywhere from, they had to invent a technique to do it once, and then ever after it's easy. Sometimes that happens, and that's terrible. Sometimes they invent a technique, and then they have to reapply it every time they develop a new exploit, and it's just a pain in the butt. It becomes an ongoing cost for attackers, and that's what we want. Like ASLR, address space layout randomization is a terrible technique. It annoys me. It's very silly. Attackers do have to have a way of bypassing it every time they write, or most of the time when they write an exploit. They often can, but they have to. Even though it annoys me, it's still valuable, on 64-bit. On 32-bit, it's maybe not.
Schuster: Yes, there's not enough room to do it.
Why does it annoy you? Does it make anything more difficult to bug or anything like that?
Palmer: No, not really. What annoys me about it is just that it's ugly. It's a hack. What I want is something like a language that just doesn't allow these problems in the first place. Why are we trying to hide things from the attacker? Like, "No, they might find where the stack is and then they'll know where to overwrite the return pointer." Why do we let them get to that point in the first place? That bothers me, because the software bug that let that happen, has no reason to exist. It's just, you didn't check the bounds on your buffer. Let's fix that. Let's fix that at scale by using a language that makes it easy rather than making every C programmer remember every single time. I want to fix the problem for real.
Schuster: There's lots of legacy software, lots of C, C++.
Do you have a lot of legacy backwards compatibility versus security tradeoffs?
Palmer: The main thing is, we support old versions of operating systems for some years. Then very gradually, we announce that we're going to not support Windows Vista anymore. I think Windows 7 is the oldest version we currently support. It might not be long before we don't support Windows 7 anymore. What we're able to do for the most part is we can do the best we can for you on each platform. If you have the latest version of Android, we can take advantage of what it offers. If you don't, we'll do the best we can with what we got. It doesn't really stop us from doing new things. It's just that we can't promise as much as we can on the newest version. As for hardware, it's a similar thing. If you've got some hardware that doesn't have CET yet, you don't have it, but we'll still do everything else for you.
Schuster: Since sandboxing can be expensive, is there a sense of maybe scaling it back on weaker clients?
Palmer: Absolutely. There's runtime configuration options in Chromium. I was talking about how I wish we could do site isolation on a per origin basis. Actually, there is an option in Chrome to turn that on. You can do that. It just uses up more processes, but if you have the memory, you can do it. On Android, we have a dynamic thing where we face a lot of memory pressure on Android, there's just less free memory available to make more processes. What we did was we said, if we notice that you've been logging in to a certain site, then we take that as a signal that it's important to you. Then we site isolate those. Then sites that you're using anonymously, like just the news or whatever, Reddit, if you're just reading Reddit, not logging in, then we don't need to spend a process on that. We can make several sites share the anonymous process, and then dedicate our resources to the ones that seem important to you. Where logging in is a signal, or if you use it heavily, that looks important to us, things like that. We do do some dynamic scaling. You could do a similar thing that makes sense for your application. If you've got a server application, you could say, this is for clients who have logged in, and then clients who haven't logged in, maybe they share some resources. It depends on how much resource pressure you face. I wouldn't adopt such mechanisms until you've measured that you have resource pressure. As long as you don't, you might as well sandbox everything.
Schuster: In what functional areas are your main challenges, so rendering, JavaScript, networking, stuff like that?
Palmer: It's mostly JavaScript. Networking is tricky. WebAssembly is tricky. When we give the attacker the ability to run code, like with JavaScript, or WebAssembly, and it used to be Flash before Flash was removed, you're giving the attacker a lot of power and a lot of chances to win. Those have always been tricky. The renderer, therefore, we sandbox it the most heavily because the most dangerous stuff is in there. Similarly, the network process has to parse and deserialize a ton of complicated stuff. QUIC and TLS and HTTP are quite complicated, actually, and there's a fair amount of risk there for attack. We have had some nasty bugs. It's not as dangerous as JavaScript, but it's not exactly easy. I really would like to break out the network process into one per site, because if you take over the network process now, you get access to networking for every site. That's not great. It's harder, but you get more power, and so I'd like to stop that.
See more presentations with transcripts
