1. We are here at QCon London 2016. I am sitting here with Adrian Colyer. Adrian, who are you?
I am Adrian Colyer. I am a venture partner with Accel here in London. You may also know me through a few other things I have done. I am the author of a blog called “The Morning Paper” ( http://blog.acolyer.org/ ) and I previously used to be a CTO with SpringSource and in CTO roles with VMware and with Pivotal.
Yes, it is a fantastic question. So yes, the essence of the blog is that every weekday I read a computer science paper which I find interesting for some reason or other and write up a summary on the blog. The way I kind of stumbled into this actually began when I started working more in the center of London and with the commute on the train, it was one hour each way, and I figured a couple of things: one is that if I do not do something productive with this commute time I am really going to come to resent it. And I was kind of looking around the carriage and observing and, as you could imagine, there were many people there reading the morning paper, being The Times, The Guardian, The Telegraph or other papers that we have here in the UK.
I am not really that interested in reading that kind of paper, but I could read more useful papers, ie. research papers and on Hacker News there are always stories like “The top 10 papers that every programmer must read” and I am sucker for that stuff. Have I read all those ten papers? I do not know. Let me find out. So I started reading them on the train and then I thought “Oh, this is kind of fun. I am reading a morning paper and they are reading a morning paper. Let me just put the 'Morning Paper' in the paper title” and it kind of snowballed from there. That is how it all began.
It is a whole mix. It takes me quite a lot of time, as you can imagine. It is probably a minimum of two hours and if I decide to tackle more adventurous papers, sometimes 4 or 5 hours to really go through it. So I can do a limited number of those larger papers. Probably about 2 ½ hours a day on average. So that is a good chunk of time and I am pretty religious about using my commute time for it which is three days a week. Outside of that, my habit is to try to read first thing in the day, let it kind of distill around in the back of my mind through the day and write up my thoughts at the end of the day. As you know, days are random, stuff happens, you are traveling, you move around. It is never quite that perfect but I am normally about a week ahead of the blog. So that is something that some people wonder about – I did not read it that very morning when you see it posted. I normally read it about a week in advance and that just gives me that little bit of buffer to have sort of the five papers lined up and ready to go so that I can cope with the vagaries of everyday life. I bookend my days with it, basically.
This is something I get asked an awful lot. How do I find good papers? So it is one of those things that you can kind of get into over time. Here are some of the things that I do: firstly, I avidly collect any paper references I see – “This has come out. That has come out” I save them all away and I get a backlog of interesting things to read. You start out by kind of googling recommended paper lists in different areas. Then I stumbled upon something pretty obvious in hindsight which is that if you look at university courses in areas you are interested in and the professors lay out the key papers we have to cover in their course which normally gives you a great idea of the foundations and then you start to learn the research groups whose work I really like.
Let’s see what else they have done and I watch out for the new things that they do which leads to “Where do they publish those papers?” Because maybe there are other things that I like in those conferences. One of the things that I do now is I have like a set of conferences that I keep an eye on and when they come around, I dedicate a week of the blog to covering some of the papers I liked from that event. And I guess the last thing is that the more you read, every paper has references in it and that is a never-ending piece of string but you just start to find things that peak your interest. My only rule is that I find it interesting in some way or another and generally, also, I do not cover papers I do not like. I do not like to leave a negative – you know, write off anything. I have to like it, I have to find it interesting and outside of that, I am quite happy to look at quite a broad range of stuff.
Werner: I have been reading “The Morning Paper” and I confess I have not read every single one. I have a long queue.
That is something that so many people apologize to me – that they have not been able to keep up or they have not read every one. I certainly do not want you to feel that. It is not an expectation that you must read every one. One of the goals here is: once in a while, once a week, once in two weeks, whatever, hopefully, you see something that catches your interest, you get a quick digest of it. Maybe you go on read and the full paper. That is what I always imagined is happening. Maybe it is nice to think that there is someone who has this daily habit - they get their cup of coffee and they read the paper digest. But that is not what most people do. Certainly, I do not want anyone feeling guilty they cannot keep up.
Werner's full question: Well, I have definitely skimmed it. Obviously you are interested and I am interested in these things. What is the audience for the blog? Do you think everybody, every enterprise programmer should read them or not?
Again, this has sort of surprised me. Because I stumbled into it, it was not like I calculated the audience I wanted to reach. So I had a kind of impression that I wanted to share with practitioners some of the exciting stuff that was going on in research that they may not be aware of. That was definitely one of my motivations. One of the things that surprise me that was obvious with hindsight is a lot of researchers in research groups actually like “The Morning Paper”. They say “Surprise! People who do this for a living like papers!” Who would have thought? But really, in my mind, I guess I feel like I am writing more for a practitioner audience at least. I think that as long as you are curious and you like learning and I do feel like that if you have chosen to be in the IT profession, then I think that is one of those things that comes with this territory. The rate or pace of change is so vast. I think a lot of us have that bias. We like to learn about new cool stuff and smart algorithms, etcetera. So really, I guess it is for the curious and for the people who enjoy learning something smart.
I remember one morning over breakfast we were discussing something in the family. I cannot remember what the conversation was. I said “I like this thing!” and my son said “Daddy, you are weird. You read papers for fun!” And I thought actually, maybe it is weird not to read papers for fun if you are in this field. Why would you not want to be up to speed with all the cool stuff that is happening?
Werner: Yes. I mean all the research, particularly nowadays that Moore's law is slowly coming to a halt, - to use a cliché – you have to work smarter, not harder. For instance, we have to use probabilistic data structures and things like that rather than brute force. And as it turns out, research is useful.
Who would have thought?
Werner: Yes. They are not just sitting in their ivory towers.
That is the thing. I mean you can read an arbitrary blog post and I do it every day, obviously. But you can read a blog post and you do not really know how much time and thought went into this. Maybe this is somebody summing up years of work, but often they are spur-of-the-moment things. A research paper is generally months of carefully thought out work, setting context, evaluated, peer reviewed, presented at a conference and we all know there are some weaknesses and things in those systems but that is a good quality source of information. So it is a very good investment of time going in reading some of the stuff I find.
I guess one of the first things to say is – and I continually go through this – is “Where would I find more good papers?” And I ought to know better because almost every week I read something that makes me go “Wow! I wish more people knew about this. It is really cool!” So I did do these little announcement at the end of 2015 I went back and I looked back at the Twitter statistics and the Wordpress logs and just to see what were my top ten papers. If I can only read 10 papers of the ones I covered, these are the ones. And then I went to what the most popular were, in terms of the number of views and the number of mentions on Twitter. The reason I did the latter one is sometimes by random chance, a paper write-up will get on Hacker News and then just because it is a much bigger audience it spikes the stats.
So I think that Twitter is a more reliable indicator. Now I touched on some of these favorites in my keynote earlier this week. I love the writing of Frank McSherry. I think he makes you think very well about scalability at what cost, etcetera. A paper I did not talk about in the keynote that I really, really loved was from Peter Bailis and it is called “Feral Concurrency Control”. It is one of those ones where sort of academia meets practitioners and it is a group of like super smart seasoned database systems researchers looking at what we actually do in practice with ORMs etcetera and it is actually set in the context of Rails and looking at some of the things they do with validations etcetera and basically showing “Oh, these things are not sound. When you try to take them to the application level, it does not really work. You put a uniqueness constraint on and it is just not unique. Look!” And you read it, starting to feel slightly smug if you are not from the Rails community – “Of course, this could not happen in my language!” - and you get to the end of the paper and you see this happens in pretty much every kind of ORM framework. I really just enjoy that line “Oh, yes!”. There are these sort of gaps between theory and practice that pop up.
Let me just quickly check and see some of the other really popular ones that people cited. Let’s see!. Oh, the Unikernel stuff – that has been very popular. I am just looking at how you shrink that down. I loved a paper called IronFleet that Microsoft did that was a little bit kind of left field, but I have a bias and it creeps through when I am writing sometimes, that sort of maybe we can do better in the way we build our systems and sort of seeing things “Done right” as it were for some definition of “right” as in “correct”. I am a bit biased that way. There is this paper from a team at Microsoft called IronFleet and it is kind of one to annoy the Haskell programmers in your life. So the classic Haskell programmer is like “You know the type system is so great that by the time the code gets through the compiler, it is just works”. This Iron Fleet work, what they did was that they started with this very, very simple, high-level specification of a realistic key/value store and I think it was like a replicated state machine library from memory.
They traced it down through a series of refinement proofs, all the way down to an implementation of a pretty sophisticated distributed system with automated proof chains at every level. And they finally compile it, and they run it and it just works first time. I know you are familiar with some of the background, in formal methods, you should not be able to do this sort of problems of this kind of scale that are not toy problems and it's the sort of paper that opens your mind to what is possible and to continue pushing the barriers forward. I loved “Holistic Configuration Management”, the paper from Facebook which, again, I talked about. It is a lovely master class in how they do config management. Very, very practical, very easy to apply. So many things. I did a curious little series, by chance. I did a series on nature inspired optimization algorithms and you get to look at all the bats and the flocks of birds and all the other stuff and things that people have done to make algorithms off the back of it. Right now I am looking at some of recurrent neural network stuff and what is possible there. Amazing things everywhere you look!
Yes. I have a bias towards distributed systems and database systems. I guess I have grown up through my technical career sort of grounded in that field. So maybe it is the material I could initially feel the strongest connection to. It is very approachable. Also, within that sphere, if you look at what has come out of the other places, the gaps between the research and its application, it has really been narrowing. So I think that is quite exciting. I definitely have a draw to that. I think those are some of the most pressing issues of our time, but I do try to cover other topics as well, things in the mix, like the nature optimization algorithms, that came up. Interestingly – I never expected this – I started to get requests from people to cover more papers in area x. One that came up was “We would love to have more security stuff”, for example. And that is like, if you're not deep into security, this is an area you are slightly cautious to go into. I cannot really understand it, but I did what I normally do. I looked “These are the most recommended papers” and I started to work through them and I learned a ton doing that. I do try to keep it in the mix, but they are my favorite kind of conferences research groups topics, but I am always going to want to go back to, because I am really fascinated how that stuff evolves.
Werner's full question: There are some really interesting research papers mainly on all the ways buffer overflows can be exploited and evading ASLR and so on. There are some great titles. There is one with a grotesque title which I am going to add in the transcript [Editors note: the title is “The Geometry of Innocent Flesh on the Bone - Return-into-libc without Function Calls (on the x86)” ]. it was getting around buffer overflow protection. It is the one with the gadgets, I think.
Yes, I have covered this paper, but I can’t remember the title either.
Werner: “Flesh on bone”?
Yes. “Flesh on the bones” or something. That is right. It is just amazing how they do all this stuff. Another one which was a real “Wow” moment for me last year was a paper called “Minding your P’s and Q’s” and it is about how if you can get enough keys - so I got collective across the internet - you can start to find systemic weaknesses in the ways that keys are generated and you can use patterns that are mined across this vast collection to start cracking things. You just end up with this conclusion that nothing is secure. It is very hard not to draw that conclusion, the more you read.
Yes. One I am kind of just easing into now, the obvious thing that has been talked about a lot is anything around data science, machine learning, deep learning, etcetera. I am going through a series of papers in that right now. One of the reasons I have done a bit less there so far is that the heavily mathematical papers take a lot of effort to ingest, but they are hard to communicate succinctly. I have a certain time budget per paper. But there is so much happening, it is so interesting that I have started to go into that area. So I have just been looking and reading this week about some of the recurrent neural network things and what the Google Deep Mind team has done and the neural Turing machine work – I think that happens to be today’s post. So, I am starting to cover a bit more that and I am learning a ton from that.
Then, actually in a similar vein, this is actually a personal interest because I used to work on a language and a compiler. So, I used to lead the AspectJ project and I sort of think about the evolution of the language and the compiler etcetera behind it. I covered really very little around sort of type theory and beyond that, even some of the deeper, more theoretical parts of computer science. Again, because some of those papers are harder to be portrayed quickly. But that is an interest there of mine. I would like to do some of that. Also, on my list, there is actually that I would just like to work through some of the classic algorithms and just go present a series of those. The thing I found is that I am vastly outnumbered. There are so many people doing so much good work and there is only one of me kind of churning through, at the current rate, about a couple of hundred papers a year. So the list of the wonderful things you could cover is almost never-ending. Those are some of the things on my mind. I want to try to work some of these in. Maybe I will do some distributed algorithms, maybe I will do some sort of just classic algorithms. There is a ton of great material out there.
Werner: What is interesting about distributed algorithms, particular the classic ones are the ones that were created in the 70’s and the 80’s but are only now becoming relevant, like the Paxos stuff. It was built back then, but now, with NoSQL and all that stuff, it is relevant in the real world.
Yes, I was really lucky. In my track yesterday, I had a researcher from Cambridge called Heidi Howard who did a fabulous talk about this. You look at the dates on some of the papers and you think “It is very visionary at that time because they were sort of addressing a problem that really nobody had”. All this time later it is like the most pressing issue of our time. How do we make this work properly? How do we deal with these distributed systems and getting some ability to reason about them? That journey is quite amazing. When you go back and you read the very early papers, not the Paxos papers, but other very early papers, you have to find this remarkable simplicity and clarity of writing because there isn’t this decades of buildup of idea upon idea where you need to be quite sophisticated and nuanced to express something new. Going back and reading those early things can be very rewarding.
It takes me let’s say two and a half hours a day roughly and I do not do it during my core working hours or “paid-for” working hours, so I keep it outside of that. So it is early morning and it is evening and it is some weekend time, etcetera. Beyond the obvious andvery surprising benefits, one of the things that I talk about that I have to do a lot is to quickly assess. This new idea that the company is putting forward, that is coming from the left field – is this a good idea? Is this a bad idea? And of course, ultimately, is there a business behind it. The thinking skills that come from looking at all these problems and solutions and what you find in research is that the good papers expose the fundamental trade-offs so you can really see, what are the levers that really matter here. That is really great for quickly assessing new ideas. Likewise, seeing some of the frontiers and where things are going, helps to give you some forward-looking perspective.
Actually, the thing that I kind of did not expect is that I am always astonished about the corners of the world that “The Morning Paper” seems to have reached. So, quite often I will talk to a technical founder or a CTO and they have read “The Morning Paper”, they have heard about it. We can quickly establish a rapport. That helps tremendously because perhaps you would not expect a traditional venture capitalist to have the kind of technical depth. You know, I can go to an area I have never, there are many, pick any area in which somebody is way deeper than me, but for a venture capitalist, I am certainly much more technically oriented than many people in the industry and founders really love that somebody understands what they are really doing on this deeper level and we have some fabulous conversations and so it has proved to be very valuable in all sorts of ways I could not predict.
Werner: That's a great way to close and the audience will obviously also subscribe to “The Morning Paper”. Thank you, Adrian.
Wonderful. Thank you all!