Transcript
Moderator: We have an excellent group of people on our panel today to talk about the future of performance. I'm just getting everyone to introduce yourselves briefly.
Wakart: I'm Nitsan Wakart. I work for DataStax as a performance engineer and so on and so forth. You can come to my talk can see the whole bio thing.
Beckwith: I'm Monica Beckwith and I work for Arm as a performance architect for managed runtime.
Chu: I'm Howard Chu. I'm chief architect of OpenLDAP Project, founder of Symas Corporation. Also work on side projects like cryptocurrency.
Jaffer: Hello, my name is Sadiq Jaffer. I founded a startup, Opsian, that I work at and I'm also doing a talk later today at four something downstairs.
Thompson: I'm Martin Thompson, I'm independent and I spend most of my time sort of doing open source contributions to things like Aeron and SBE.
Upcoming Hardware Changes
Moderator: Fantastic. So this panel is an attendee-driven type panel. Really what we want is some questions from everyone in the audience to ask these experts. You have an opportunity to pick their brains. So try and think of some questions. I'm going to kick things off because I also want to pick everyone's brains on performance topics myself. I'm going to be very selfish and throw a question in to get us going. Since the theme of the panel is future changes in performance, what are the upcoming hardware changes that people feel are interesting or useful from a developer perspective? Does anyone want to have a go?
Wakart: Hardware manufacturer and all that.
Beckwith: I think, yes. Heterogeneous environment is where things are going. So, not just GPUs but even processing different languages as well. So multilanguage processing. You can think of it as a containerized environment, but more rich, more smarter.
Chu: One of the things I've seen especially if you look in the supercomputing HPC, is this combination of FPGAs with CPUs. So you're looking at being able to encode an algorithm basically in hardware, as an FPGA, to accelerate algorithms.
Jaffer: I think one of the interesting things that is happening is that the rate at which hardware companies, semiconductor companies that fabricate things, are able to shrink their process nodes is slowing, the costs of each extra node is getting exponentially higher. And what that means is that it changes the tradeoff between this algorithm we have right now, it's not fast enough if we just wait for a next process; no technology, we can do it on the general purpose processor rather than having to actually fabricate things ourselves, or get an ASIC made, or what have you. It kind of feeds into Howard's thing there, that I think what we're going to start seeing is a lot more specialized hardware, especially also that plays into the power envelopes with the modern processor. You have so many transistors, but you can't keep them all powered up at the same time because you blow your envelope.
So what you end up, with and you can see from what Intel's been doing recently, you end up with specialized units that do particular things that only those units need to be powered up, because you can't power up everything. We're also seeing it from another side. We see things like RISC-V, for example. There's a lot of interesting work being done in RISC-V on the extensions. So things like this, the scalable vector units and Arm's. Arm have a scalable vector unit. RISC-V also designed to be able to actually coexist with accelerators. So that's something else that's very interesting.
Thompson: Yes, I think hardware's been going away, in another direction for a while now, and things are starting to get pronounced, because I think it's been happening for a while, but it's now just starting to hit crunch. That's for me, fundamentally, the difference between, we’ve always been programming for decades where bandwidth has been the limitation. Most of our algorithms, most of our thinking in the industry is all around bandwidth limitation. We compress on the network, we do different things like that.
But what's happening these days is latency is the fundamental limitation, that has flipped. We're seeing advances in hardware. Generation to generation, we're getting advances, particularly in bandwidth. We're seeing extra execution units in CPUs, we’ve got wider buses and the wider buses are happening all layers, right down through a cache system, through the memory subsystems, through our networks and everywhere So we need to change so we fundamentally program with bandwidth as the thing we're going to utilize and we need a break away from being tied to latency where we've spent decades optimizing the other way. And that's a difficult shift for a lot of people. Specialization is one way around it, but it's not solving the general problem.
Moderator: Do we have any audience questions for the panel?
Heading into a Monoculture around OS
Participant 1: If we look at the last 20 years, there's this interesting thing happening where, where I think I talked to somebody yesterday and they said that Microsoft is internally giving up on Windows becoming a server operating system. They're moving stuff over to .NET Core and might have realized that battling Linux is becoming increasingly hard. If we go back 20 years, we will have this multitude of both hardware and software, and specifically software is what I find interesting. I'm talking to people working in performance-oriented services. So we have those HFT people who get slapped on the head with random 100-milliseconds penalty every blue moon.
Yesterday I talked to somebody from running a CDN and they found this interesting performance characteristic where they invalidate millions of inodes with the XFS, and it would block their applications for 100, 200 seconds, which is insane. The XFS developers didn't really consider that use case because they consider it too niche. So, the thing is that, that I believe that for Linux it's really hard to be everything to everyone. I'm wondering what you guys think about this sort of monoculture that we definitely seem to be heading into? Who is that going to affect and should we be deeply worried about that? That was a horribly long question. Sorry.
Wakart: If I understand the question correctly, you're worried about going into a monoculture around OSs. I think Linux is not as unified, apart from the brand name. There are a lot of flavors of Linux and you can run it with different file systems, which was part of your question. So I don't know that the monoculture is a real thing under the hood. Certainly in the JVM space, and I think actually now is a time of splitting different organizations working on the JVM from different areas and bringing new pressures to the ecosystem. So from the JVM side, I think things are more varied now than they have been in a long time. At least on that side we can be hopeful. but I'm not sure the impression that Linux has become a monoculture is accurate.
Beckwith: I wanted to understand what you first said about Windows. Did you say it was going away?
Wakart: Giving up on the server side.
Beckwith: Why do you think that?
Participant 1: No, Microsoft teams have come to the realization that most of the workload on Azure is going to be Linux, and transforming into services company, which basically means that it's going to be a lot more agnostic about its products. Right? Because that's not what it's about anymore. It's about services. Like, this was in the pub afterwards, so you should take this with a grain of salt.
Beckwith: They know their stack, Azure on Windows, they know, they understand everything. Linux is new to them and they see most of their customers may be already on the next … When they talk about migration or moving to Azure, maybe one of the things that comes in, "Oh, you know, we want to make sure that works on Linux." So I think that's why you see more effort there. But I'm not sure if I agree with Windows going away. You know? Yes.
Chu: God, I hope it does. In some ways I think it's kind of inevitable because Linux is so freely accessible. People are flocking to it just because they can get it. There are other free systems like BSD-based that you can get easily, but they are a minority. They don't have mind share and I think it's like in any system. In a lot of systems, you find that natural monopolies evolve over time just because popularity makes these things the place to go. And unless Oracle is more proactive about pushing Solaris, or the BSD guys can make a stronger case why they are better on a future basis or performance basis or anything, it is just going to go towards a monoculture. That is a natural evolution I think.
Thompson: I think the monoculture in Linux is really not the thing that's happening, particularly in the performance space. It's almost a side step from that. It's the particular hardware and the APIs that are used to access that hardware. For example, in the networking space, DPDK is where things is going to. You're actually taking the operating system out of it, even going to cloud. That's now the performance route to go, because the main providers are giving FPGA implementations or on ARM based implementations off to the side. It's not coming from the operating system. You go to file systems, yes, we've got choice.
But actually if you want to go really fast you tend to sidestep even the file system. So things like SPDK becomes API, some of that stuff is driving where things are going. You see all of the advancements happens around specialized hardware and in APIs that use that, and things go from there. Linux tends to be one of the first to adopt that because it's easy for people to go there. But if I want to work really fast on networking, whether I choose Windows or whether I choose Linux is almost irrelevant these days. That's how things have gone.
I think we're going to see more and more of that because an operating system is an abstraction over your hardware, and the cost of going into that is becoming more and more expensive. The system call side is a problem. BSD sockets is looking at being retired. There's a lot of research into where we go next because we just can't use 100 gig and beyond. We can't use the latest MVME drives. We move to having Optane memory directly connected in DIMMs. Forget file systems. It's not going to work the same anymore if we want to get the performance side of it. So the operating system's going to provide scheduling and containerization if anything, but very little on hardware access going forward at the performance end.
Chu: Just wanted to add to that. One of the recent experiences I had benchmarking LMDB on Optane, when we actually ran it on the raw DIMMs as a block device instead of through a file system, it was over two times faster. So you are paying a pretty high overhead cost for the operating system providing you services. If you don't actually need the OS to give you those services, yes, it becomes irrelevant.
Moderator: Do we have any other questions?
Participant 2: It's a direct follow-on from that, as you go to reducing your abstraction layers and talking directly to the hardware, the OS can't provide the containerization, the blocking for it, because it doesn't have the tools to do it without, for example, IO memory units from the CPU blocking you off from different areas. If you're just writing to ARAM memory, you own the box at that stage. Do you see it going that we simply accept that you don't have multiple people on the same box, that you don't have security on the box anymore and you skipped that for performance? Or do you see that hardware has to evolve to provide some sort of high level blocks between different areas of the system and that you carve the system up in a far more coarse-grained fashion?
Beckwith: I think it becomes the responsibility of the hardware to provide you those security fixes or security uptakes. I think that's where, if you see after all the other bugs that have come out, security bugs, and that the hardware is actually fixing them. So you no longer need Red Hat's patch or like those kind of patches for SVs, but you get those fixes in the hardware itself.
Chu: Oh, well one thing to remember is you're not really getting direct access to the hardware. If you're accessing a bank of nonvolatile DIMMs, you're still going through mmap, you're going through a memory map. So the OS is still giving you per user ownership and access controls on that. The memory mapping itself is still being controlled by an MMU. It's still has read-write protect bits on it. So that aspect doesn't change. You still have security there. Now, the fact that we have problems like Spectre, I think that says very clearly it's a bad idea to do multitenant hosting now. I just think that doesn't fly. But aside from that, the OS still has the protections that it can offer, but yes, obviously in a multi user environment there, they're becoming less effective.
Thompson: I think most end-users are not using the raw resources. Most people program to subsurface or API, there'll be using something for messaging, something for data storage, they'll be going to LNDB, they'll be going to Netty, Aeron, going to something higher level. They don't program to those low-level things. These systems and services that people are using can have the protections that are wrapping up everything else. In there, you can be totally abstracted like S3 on Amazon. You don't know what the underlying storage is. It can have the security and isolation that's part of that, and it also gives you the mobility to be anywhere else. So the people who are providing those, they're dealing at that lower layer and it's of interest to them. But most people are dealing at a higher level and sidestepping that they need different APIs and abstractions that they're dealing with to service their needs as we move to serverless in other things.
Heterogenous Architectures and Programming Languages
Participant 3: Since we're at QCon, a software question. We've talked about heterogeneous architectures before. So we have these special processing units, GPUs like neural processor and all of these coming to the scene right now. But to access them right now, you probably either require to know a c-level coding, special programming languages or this kind of thing. Do you think the high level language designers, like Java designers, would they need to do something with their program languages? Could it just be solved on the compiler space? Do they need DSL, or how do we better expose access to these specialized computing units and software engineers, I guess?
Beckwith: I think Java is also moving in that direction. If you look at the JABs that are upcoming, I don't remember - Panama, but then it's called something with “s”. I'll have to go back. But there's a nice talk by John Rose on how even Java is going to go into the heterogeneous environment and it's more welcoming. And that's what I was talking about. I wasn't just talking about the hardware. I was talking about how you will have these languages in a heterogeneous environment as well. So it's a hardware and software heterogeneous environment. I see that this trend already. Working in Arm, I already see partners and everybody looking at this and of course Java. I'm pretty sure the management, they're going to polyglot as well, which is a different flavor, but it's also like heterogeneous languages in a way. So I do see that.
Participant 3: I think it was project Sinatra you're thinking of.
Beckwith: Yes, that's the GPU integration.
Chu: I get the impression that a lot of these systems already have an escape hatch. They have unsafe APIs, right? If you wanted to get there right now today, you could do it in whatever source language you start from. Tailored languages down road, give them the time.
Jaffer: I think it also depends on where we're looking in the stack. So is it that the programming language itself should have support for these devices or, you mentioned the neural network processors. I spent too much money on crazy little dev boards that have cool things attached to them and a lot of these neural network processors support Tensorflow lite for example. So there you're seeing a higher level framework that is able to support that device and you can just say, "I've got a model in this format here you go, run it" kind of thing. So it may be the case that for some of these, a high level API being able to reuse your existing abstractions, but it just works on different devices is the best way of doing it. So it really just depends on the actual device.
Climate Change Concerns in Software Development
Moderator: Anyone else have a question? Right back to me then. One of the big things that we've got societally, as a concern, is things like climate change. That impacts us quite a lot from a software development point of view, from a hardware development point of view, because we do use power, we do produce CO2 emissions. What should we be thinking of moving forward to help save the planet in the most broadest way possible?
Wakart: I'm not sure everybody's aware of it, but if everyone could use more XML, if you moved away from XML, come back. I think most systems that I've seen have huge neglect for inefficiencies around the text processing, or just silly processing of conversions of data from one format to another to another, to another working on Cassandra. Now, Cassandra doesn't have the string problem, but they've redressed the problem as a ByteBuffer problem. So you don't have strings with substrings, you have ByteBuffers with lots and lots and lots of ByteBuffers that you slice and copy and move around.
I'm not saying people should have no abstractions to save on performance where they don't need. But there's an awful lot of people who have no idea where time is spent and could have huge wins in terms of system efficiency, if they just use a profile every once in a while. Unpopular view somehow. But if you haven't been using a profile in the last two weeks, there's probably a low hanging fruit you're missing. So have a go.
Beckwith: I work for Arm, so I think we have it solved already.
Chu: I love it. I just have to reinforce Nitsan's point, because I talk quite a bit about profilers in my talk later this afternoon. I think if you look over the past two decades of software development, there's been a conscious choice not to think about or talk about efficiency. And that's literally killing us. I tweeted a couple months ago, if you're not writing in C, you are murdering polar bears. If you're not using the most efficient tools available to you, you really are harming all of us. So I'll go into a lot of those tools later.
Jaffer: I'm just going to reiterate the profiling thing again. Instead of doing it every two weeks, you could do it all the time, and in fact we've got to talk about profiling all the time later on today.
Thompson: Yes. Just to complete profile, profile, profile. I think we've got a culture in the world now, where we don't want to know about what's really going on. It's really unfortunate. Software, we have gone past what the airline industry does in carbon emissions; just our data centers alone, we're beyond that. We have this culture of throw everything away, not looking at the implications of what we do. We use the excuses of, “Well, we don't have the time. That's not important. We can just sort of throw hardware at the problem.” And it becomes an issue. If you measure it, you can see it. We have destroyed oceans because we don't look into them, when things happen elsewhere. We look at cars and we say cars are a real problem. They're nothing compared to shipping. Shipping has six times the pollution carbon footprint of cars, yet because we don't see it, it happens out somewhere where we don't see.
I think it's the same in our software. We don't look, so then we just kind of forget about it. If you profile it becomes really obvious, and it becomes shocking. I gave one example from inside the core JDK. If you go to write a single character into a string, it turns it into a string and then writes out the character by iterating the string writing the first character. That's the shocking stuff that we've had, been around for a long, long time and just is not fixed. It's really shameful that we just don't pick up on these things. We go for new, shiny, whatever. It's the same thing. We've got fast fashion as a problem, so people sometimes wear something once. We're doing the same thing in our code and we just need to stop it.
Moderator: I was just going to ask a quick follow up question on that point. A lot of your answers are focused around efficiency. Is there anything beyond improving regular efficiency of software that we can think about it in terms of power emissions and power related efficiency?
Wakart: I think the power focus is kind of suggesting a bottoms up approach where you look at the power and then you think, "Oh, I could save a little bit here, a little bit there." But certainly you can take the power view from the top down. What parts of my system, what meaningful things that I offer people, waste a lot of power? And then go from there. Do they need it? Can I somehow unify these things? I was going to say at the end of the, once we went all the way around and everybody said profile, you can stop profiling because everybody said profile now. Once you have a profile, one of the things I find people don't do, they have a big profile, they're not sure where to start. When you look at the profile, one of the things it does, it kind of locks you into the way you are already doing things, because you see the process that you have and you're trying to micro-optimize it.
One of the things that helps gain huge efficiencies sometimes ,is take a step away from that once you've seen the profile, and just think a little bit about what should be the cost. Because if you don't think about what should be the cost, you might be buying the same Porsche but without the ashtray and thinking you're saving money, when actually you could just do with a pull along cart with the ashtray.
I'm working on a feature now where you've run out of low hanging fruit, but actually if you look at it from how much it should cost you, it should cost you probably a hundred times less. What you need to do at that point is probably rewrite the whole thing from scratch and stop just scratching at it. Just burn it to the ground without the CO2 emissions. Metaphorically burn it.
Beckwith: Just to add to that point, and one of the things we do at Arm is single core performance as well. We know multicores and multi-threads and the interactions and the barrier, overhead, and stuff like that. And this is again, everybody's plugging in their talks. I should too. So this is going to be in my talk as well. But the weak ordered model, memory model of Arm, we have to put barriers, even for single core performance, and trying to optimize that in hardware itself and trying to make sure that your barriers are not as expensive. Things like that, so your profiling leads to these kinds of findings, and then you feed it back to the hardware and kind of work together.
Chu: I think Nitsan should just give my talk this afternoon because I actually have slides that say burn it all down. The system that I'm talking about, we actually did get a 100x speed up on it over time, which involves, first of all, some very obvious hotspots in the profile. But after those were removed and the profile looked fairly flat with no obvious hotspots, that was time to take a step back and realize, “Hey, there are a lot of things here that we just shouldn't be doing at all.” And that involved a major rewrite. So, the moral of the story is you can get a long distance from incremental improvements, stepwise refinement, but sometimes you'll find you've gone down a dead end and you need to rethink from zero.
Jaffer: Yes. So I think when I'm building systems, I like to think about mental models. So what are the numbers? What are the ballpark numbers that we put together? I mean, my background was as an engineer and I like to say, "Okay, well, what volume do we need to deal with? Where do those requests go in the system? What kind of write volume does that result in? What kind of read volume does that result in?" And then check that those actual models map to reality.
Now, one thing that we've seen from profiling is that sometimes those mental models don't actually work, that developers actually work to different mental models of actually how their system actually functions. One thing that we've seen from profiling- not to knock profiling on the head over and over- but a good example actually, it was a customer of ours last year, they did some production profiling, and their system gets a request and then does a whole bunch of tests on it to decide on whether it actually wants to do further things with this request.
And what they actually realized is that the actual costs of those filters that they were doing, their mental model for those costs were completely wrong. Actually what they'd ended up with in production was doing all the expensive tests first which are actually not that selective, and then the cheap tests are at the end. And by simply reversing the order of that, to do the cheap tests first, you could just see a step change in your CPU usage basically. Boil the ocean a little slower.
Thompson: Yes. Just having the model of knowing what you're supposed to be doing. We see it a lot. People start into things with trying to roll out the toolbox of things that they want to do, or what I call resume-driven development. It's unfortunate of our industry that we just want to learn stuff. We want to play with stuff and we do it on someone else's job. We waste time, and we also get some weird things. This is kind of an interesting thing for me. Over the last year. I've been writing more C again than I've done in a long time, and I almost drank the kool-aid of like C is sort of evil, but now having got back into it again, it's actually not that bad. It's really, really fast and simple. Yes. We kind of go off into these crazy things with layers and layers of abstractions and, why? You write a lot of C++, the compile times are bloody painful, Scala's painful, Rust is painful. You say, well, why are we getting so slow on compilation? Feedback cycles is what really matters. Are we solving the actual problem that we're paid to solve, and not doing everything else? Because we seem to be creating stuff.
I mentioned about in my talk earlier, the profiling, I get to see loads of systems, where much more than 90% of the entire time is in all the layers that people have put in, and no one can give you a reason why those layers are in place. I had one client who was using spring and transaction management in their system and I said, "Why are you doing this?" He says, "Well, we need this for transaction management." So you've three queries to your database. Call commit yourself. It's like, really? This is the sort of stuff that happens and that is not unusual in our industry, unfortunately.
Culture Change around Performance Engineering
Participant 4: My question was reiterating on this topic, because there's a great talk from Joe Armstrong where he said the mess we are in and all those abstractions, layers, of course they are good in some sense, but maybe they hurt as well. So what I see is that you are talking about all those things that we are creating, and we are using all those abstraction layers to try to do something. In the end, we as industry are either throwing the ball to someone to fix the problem, or getting the ball and fixing someone’s other problem. I think this needs some culture change because only profile maybe doesn’t lead to that. We need to actually act on top of these issues.
How do you think this can change? Having more engineering background in universities, or maybe a possible boom in the cloud, where you cannot order VMs anymore? And then you need to stop to start caring about the money you are spending with that? How that can change from a practical perspective? Thank you.
Wakart: You're welcome. Manners. It's amazing. This is a panel about performance in a track about performance. I'm going to make a painful admission that not everybody should care about performance, and most people don't. So should everybody care about performance all the time and should everybody be optimizing all the time? Sometimes there are other market forces at play and maybe it doesn't matter that much. But I hope that people in this track have some career-related interest in the topic, and maybe we should all be here and be focused more on performance. Certainly, if you're running 100,000 machines running your software, give it a go, see if you can improve things. But, yes, it's a matter of finding the appropriate scope for improving performance and understanding that now's the appropriate time to do it.
Beckwith: I was going to add on to your question about do we need universities and whatever to cover performance engineering. What my mentor, Charlie Hunt and I, so many years ago, 10 years ago, maybe, we were talking about this, that I think we do. We learn software engineering in college, and then we don't really learn about performance engineering. You learn it on the job if your job requires. Lucky for us, I think most of us probably start on the performance engineering track. I give talks at various universities and I was at University of Waterloo and they have a keen passion, and you can tell that they have the performance engineering course. It's actually a track and then they work on optimizations, algorithm optimizations and other things in that performance engineering course track. And I think the kids are just more educated in a way, or come out understanding when you need to dive in and when you just want to stay up. If you're chasing that last 5%, then maybe that's when you need to dive in.
Chu: I think it's pretty clear you should have at least an awareness of computer architecture. If you're writing software, you have to understand the effects of everything that you write. The trend again, over the past couple of decades, has been to remove the programmer from the hardware, right? Java, the JVM is there to get in between you and real hardware. The most popular learning language today I think is Python, right? Which is again, completely interpreted, completely removed from the hardware. I think that's done a disservice to at least two generations of software programmers. Definitely you don't need to write assembly code every day, but you need to know what a computing machine actually does.
Jaffer: Sure. I agree with Howard there, but I'm going to be a bit contrarian, in that I think performance is very important, but as I say, speed kills, right? So you first and foremost need to make sure your code is correct. You can write some very fast incorrect code, right?
Man: Some of the best.
Jaffer: Yes. Just returns 42 and you could make C go very, very quickly until you shoot yourself in the face accidentally. So there is a line that we need to make sure our code is correct first and foremost, but then beyond that step, it's how do we make it correct and then performance. But I think definitely correctness needs to come first, because I've seen many systems where we need to just go faster. Is it actually producing the right results yet? And actually in the rewrite, it turns out that the original system wasn't even producing the correct results. So it could've gone even quicker. Yes.
Beckwith: I'm going to cover that.
Thompson: I'm going to challenge it and say it's actually not the right question. Most people don't care about performance and most people shouldn't care about performance. What most people should care about, and they don't care about, is waste. And those are two very distinct things. They shouldn't be trying to make the fastest program, but they should stop being wasteful, and that is not part of the culture. People don't profile anywhere regularly enough or at all. So just doing some profiling really helps, and it gets the waste down. But it also helps with correctness. I've seen lots of systems where exceptions had been thrown and silently caught and not handled, and profilers show that. There are allocations, allocation of exceptions? I want to know why. You find the reasons for that.
I think that's the performance side of engineering. We need to change that from an educational perspective and we can change that from an educational perspective. How we get better at the rest of it is really an approach thing. So yes, you can shoot yourself in the face with C or C++, but some of the best software I've ever seen that was the most robust that shot no one in the face, has been written in C. Most of our operating systems are written in C. That's not a language thing. It's the culture of the people who work on it, and how they work is the difference. Instilling the right culture- I've gone into banks and seen practices that are just shocking. People who have amazing academic grades have been cherry picked, gone in and destroyed with practices. And then I've seen smaller organizations with really talented individuals blossom into these just wonderful engineers with the right sort of guidance. It's craft, it's engineering, it's all sorts of different things, but it's actually learning the thing.
One of the things I've heard a lot, which really annoys me, is people say, “But people can't cope, they can't cope.” I've been told that my standards are too high that I expect of people, and I say, “Absolute bullshit.” Because if you have done anything where people can get hurt in, it's amazing what humans can do. Anybody's who has ever tried flying or climbing or sailing or anything where if you get it wrong, people have serious consequences, and yet people do amazing things. If you spend any time on a boat, see what people can do with a piece of rope and stuff. Basically learning to do things right.
And we can do the same with software. We just have a terrible culture and it's not really our fault in some ways. We've been pushed a certain way, but we need to grow up as a society and get better at it, because just like medicine and engineering and all in the past is, we will get regulated if we don't get better at it ourselves. So we've got to start getting better. The way stuff's easy at university, changing the bigger picture is a different thing. We need to change that somehow.
Chu: Just one more final note on that. Nitsan alluded to market forces, sometimes making it not possible to do as good a job on things and putting a political spin on this. Capitalism is really killing us in a lot of ways. When you let market forces control, dictate, the quality of your product, you've got problems.
Distributing Computing and Performance Changing the Industry
Participant 5: Thank you for asking the climate change question. I did the talk yesterday about that. So I appreciate the question. The other thing that I do putting my other hat on is, I do serverless consulting, I used to work at AWS. One of the things that I've seen a lot of is on-demand compute. So all of those new techniques that are coming out, tiny pieces of code being run at very, very small latencies and being run very, very quickly, very small binaries being created. It's a completely different way of doing code. So it's not massive monoliths. It's a very different way of thinking about distributed computing. It's a very different way of thinking about performance as well. And I think that's going to be the way we're going to be doing things in the next 5, 10 years. I just wanted a comment in terms of how you think that's going to change the industry?
Wakart: I'm going to say is, gut feeling, initially it's going to cause huge waste, and then later probably keep causing lots of waste and then later on from there, we might somehow cleverly find a way to let people do whichever granularity they like, while maintaining the efficiency of merging all of these operations into something that a computer can do efficiently. So there is a cost to granularity. Up until now, there was nothing. Oh, this is in my talk as well. So it was like my talk still had 100% value. So now I'm going to detract from that. But basically there are industry idioms and language idioms that encourage good things, like encapsulation, security, building microservice and all those good things. They come at a cost. And those abstractions that come at a cost, you can strip away the cost maybe later with a clever compiler or a clever environment, or maybe you don't, which is why I'm saying initially it's going to be the cause of tons of waste, and maybe later we'll fix it, but probably not.
Beckwtih: Yes. I mean coming from the background that I am, to me I look at as stack enablement and tuning the stack, and it could be anywhere. I think talking about serverless, I also represent Arm on OSG-Java, which is the SPEC benchmarking committee, and one of the things we're looking at is benchmarking serverless and a heterogeneous, again, runtime environment. So I'm looking at data from a different perspective and it's something very interesting. I think for us, from my perspective, enablement of the stack and fine-tuning the stack is more important, which is kind of what you mentioned as well.
Wakart: Eventually we'll make it cheap but right now it's expensive.
Beckwith: But we're looking at it, is my point.
Wakart: Any day now.
Thompson: Well, I'm old enough to have been through this before. CICS, KICKS, mainframes transactions, yes. I'm looking at the same thing when I see serverless. It's the same thing. Different packaging, new, shiny, slower. And that's what'll happen. We do this, we go round and round in circles. We do this, but fundamentally, how do you design an application that suits a user's needs that's a cohesive set of functionality? We don't focus on some of those skills. How do we do that as a process? Until we get a bit better at that, we'll keep going in these cycles, doing the same sorts of things and generating all that waste. Maybe that's what'll happen. It's evolution. We may get to the right thing eventually, but I don't see it as new and special right now.
Participant 6: Maybe we should stop tweeting and sending pictures and images and films over the internet, and all the crap going through our systems. Maybe that way we can actually [inaudible 00:47:02] all the systems without creating all this damage to the environment, maybe they'll save the environment. We don't actually [inaudible 00:47:11] inviting people to use all the [inaudible 00:47:19] I work in the industry, so I'm actually picking on myself. And we were talking about how to save energy, but we are actually coding more and more [inaudible 00:47:33]?
Wakart: But, cat videos?
Participant 5: You know what I mean.
The Future of Performance
Moderator: I think we're out of time pretty much. Yes. Maybe a minute or two left, but if everyone can perhaps have maybe a quick one or two sentences just to wrap up what your thoughts are on the future of performance.
Wakart: I hope it's good. I don't know what the future of performance is. I think there are certain improvements in the JVM world that are doing good stuff. But ultimately I think software is growing at a massive pace and as an industry, we're not that great at reducing waste or improving performance. I present to you a bleak future.
Moderator: A bleak future from Nitsan.
Beckwith: Not quite the future, but I think what performance teaches us is empathy, and I think I allude to- I don't know if Martin Thomson coined this term, mechanical sympathy- but it's always just something that comes to my mind when I think about performance and I think it's just something that's going to go on. It should be something that should be. For every performance-minded individual, I think that's something that's very important.
Chu: I think my main focus is, you should always have in mind the word efficiency, not performance, because that's really what we're talking about, is doing the most effective work with the least amount of energy. And then once you have an efficient system, you can scale that to whatever performance level you want just by throwing money at it. As far as how performance or efficiency evolves into the future, I'm hoping that we are rebounding from the past 20 years of isolation from hardware and becoming more hardware-aware, having mechanical sympathy as noted before.
Jaffer: I really like Martin's take on it. It's about waste, and I agree there that it's about waste, not necessarily performance at all cost. I think what we do need though is better tools for shining a light on that waste, because all too often people just are not aware of the waste they're actually creating and yes, those tools might only be able to tell you incrementally what you're doing wrong. But maybe in the future we can come up with better tools that can say, “Hey, look at a high level, this design sucks” kind of thing. And it's better coming from a tool than it is a person, because you're less likely to get fired. So maybe that's the future there, is shining a better light on waste.
Thompson: Yes. Mechanical sympathy was Jackie Stewart's originally, the racing driver. I just stole the term blatantly and applied it to computer science. I think reality is going to smack us in the face very hard, very soon and we're going to struggle to adapt. But those who adapt nature has shown are the ones that'll go forward. Those who don't adapt become dinosaurs and we have no choice.
Wakart: We don't blink, we're going to extinction?
Moderator: On that positive note, I think we're out of time. Thank you very much for everyone's who's on the panel, and thank you to the audience as well.
See more presentations with transcripts
