This year was the tenth anniversary for QCon London, and was the first London event produced and run entirely by InfoQ and the C4Media events team after we acquired Trifork’s stake in the event last year. It was also our largest London event to date. Including our 140 speakers we had 1,400 team leads, architects, and project managers attending 112 technical sessions across 18 concurrent tracks and 12 in-depth workshops.
Adrian Colyer's opening keynote introduced one of the main themes for this year's event - the importance of computer engineers from industry reading academic papers. A practical example of industry and academia working together followed in Wednesday's keynote, when Peter Alvaro and Kolton Andrus described applying Failure Testing Research at Netflix.
Amongst a huge number of other engaging and thought provoking technical sessions were important conversations on topics such as unintended bias in machine learning systems, and the very real dangers of employee burnout.
Attendees had near-instant access to video from almost all of the sessions. We're making these available to everyone as quickly as we can, and have already started publishing them at the rate of five per week. The publishing schedule for presentations can be found on the QCon London web site. In addition, you can see numerous photos of QCon on Flickr.
This article summarizes the key takeaways and highlights from QCon London 2016 as blogged and tweeted by attendees. It is also available as a freely downloadable eMag along with all the rest of our coverage from the event.
- BLT: Babbage, Lovelace, Turing (So, Who Did Invent That Computer?)
- Monkeys in Lab Coats: Applying Failure Testing Research @Netflix
- Reflections on Software Architecture
- Unevenly Distributed
21st Century Culture from Geeks on the Ground
- Ending the Chain-of-blame: Continuous Consequence
- Far from the Mobbing Crowd
- How to Win Hearts and Minds
- @willhillbet: Love Failure & Embrace the Fall Out
- Cassandra at Apple Scale
- Staying in Sync: From Transactions to Streams
Architectures You've Always Wondered About
- #NetflixEverywhere Global Architecture
- Architecting Google Docs
- Cloud-based Microservices Powering BBC iPlayer
- ECS & Docker: Secure Async Execution @Coursera
- Hot Code Is Faster Code - Addressing JVM Warm-up
- Java 9 - the (G1) GC Awakens!
- Netty @Apple: Large Scale Deployment/Connectivity
- Project Jigsaw in JDK 9: Modularity Comes to Java
- Spring Framework 5 - Preview & Roadmap
- How Will Persistent Memory Change Software Design?
- The Quest for Low-latency with Concurrent Java
- Understanding Hardware Transactional Memory
- Build, Ship and Run Unikernels
- Observe, Enhance, & Control: VMs to Containers
- The Dark Art of Container Monitoring
Data Science & Machine Learning Methods
- Applied CI/CD: Enabling Creativity @Volvo Trucks
- Continuous Delivery: Benefits Explained
- Immutable Infrastructure: Rise of Machine Images
- Building Trust Machines using the Block Chain
- Fighting the #Fintech Wave with DevOps
- Hacking Bank Mobile Apps
Head-to-tail Functional Languages
Microservices for Mega-architectures
- DDD and Microservices: At Last, Some Boundaries!
- Microservice Anti-patterns
- Microservices Chaos Testing at Jet
- Test-Driven Microservices: System Confidence
- The Microservices and DevOps Journey
- Distributed Systems in Practice, in Theory
- Effortless Eventual Consistency with Weave Mesh
- Not Quite so Broken TLS Using Unikernels
- Rust: Systems Programming for Everyone
- Successful Go Program Design, 6 Years On
- The Case for Bringing Swift to the Server
- Using Pony for Fintech
- Burnout
- Engineering You
- Lead the Revolution by Being Ordinary
- Making a Sandwich: Effective Feedback Techniques
Security, Incident Response & Fraud Detection
- Microservices for a Streaming World
- Real-time Stream Computing & Analytics @Uber
- Stream Processing with Apache Flink
Training
Domain-driven Design
by Eric Evans
Twitter feedback on this workshop included:
@dthume: All too often, the first decent idea is the last idea - @ericevans0 at #qconlondon
@manupaisable: You don't get all the benefits of modeling just from having a model @ericevans0 #qconlondon
@dthume: There are no nice surprises in software integration - @ericevans0 at #qconlondon
@manupaisable: Duplication inside a context is bad, duplication across contexts can be good tradeoff to allow decoupled evolution - @ericevans0 #qconlondon
@manupaisable: Distinguish core domain from supporting and generic domains, you need to focus energies on the former. @ericevans0 #qconlondon
Scaling Technology and Organizations Together
by Randy Shoup
Twitter feedback on this workshop included:
@heikkih: What if we designed our organizations like we design our systems. Nice group input as start of workshop with @randyshoup at #qconlondon
@heikkih: Really effective organizations are not made up of a small number of large teams, but a large number of small teams @randyshoup #qconlondon
@heikkih: We can engineer the system we want by engineering the organization" @randyshoup follows up on Conway's Law at #qconlondon
Keynotes
BLT: Babbage, Lovelace, Turing (So, Who Did Invent That Computer?)
by John Graham-Cumming & Sydney Padua
Tim Anderson’s attended the keynote:
On Monday evening we got a light-hearted (virtual) look at Babbage’s Analytical Engine (1837) which was never built but was interesting as a mechanical computer, and Ada Lovelace’s attempts to write code for it, thanks to John Graham-Cumming and illustrator Sydney Padua (author of The Thrilling Adventures of Lovelace and Babbage).
Twitter feedback on this keynote included:
@alblue: “If it’s warm, there are going to be cats” — #QConLondon keynote being scientifically as well as historically accurate.
@timanderson: Sydney Padua explaining Babbage's analytical engine #qconlondon "multiplication took 3 mins" https://t.co/Nd4iYd8E1K
Monkeys in Lab Coats: Applying Failure Testing Research @Netflix
by Peter Alvaro & Kolton Andrus
Peter Liljenberg attended this keynote:
Last day started with a very entertaining and inspiring keynote delivered by Kolton Andrus (Netflix) and Peter Alvaro (University of California). Peter and Kolton shared their experience of a very successful collaboration between industry and academia. Peter had a “big idea”, Lineage-driven fault injection, and together with Kolton this evolved from a theoretical model into an automated failure testing system that leverages Netflix’s state-of-the-art fault injection and tracing infrastructures.
@palvaro “My code is now actually running live on Netflix…” @KoltonAndrus “…well, minus all of the println statements”
Twitter feedback on this keynote included:
@techiewatt: Collaboration is all about compromise - @KoltonAndrus @palvaro keynote at #qconlondon
@danielbryantuk: Great start to #qconlondon keynote with @palvaro & @KoltonAndrus, with a comparison of success in academia/industry https://t.co/vSmT6M3dOF
@emirc: Morning keynote, professor and practitioner. 3rd day at #qconlondon https://t.co/vKvC1h2VDJ
@kevj121: Keynote: Break things on purpose :-) to make it better #qconlondon #gemsretail
@danielbryantuk: Popularity contests are fun when you are popular @palvaro on the metrics of success within academia #qconlondon
@danielbryantuk: Yet again a shout out to the benefits of reading academic papers when working within industry by @KoltonAndrus at #qconlondon
@danielbryantuk: Freedom AND responsibility are vital for success within software development. @KoltonAndrus @palvaro #qconlondon https://t.co/YzLgOdJkE3
@jrubr: Loving how Academia and Industry get together and collaborated to build test failure systems at #netflix #qconlondon https://t.co/8cV6Sk0XRN
@danielbryantuk: Ask "why did a good thing happen?" rather than "could a bad thing happen?" @palvaro #qconlondon #faultinjection https://t.co/G7ia1eVEe8
@danielbryantuk: HTTP status code 200, but returns an error... Welcome to the real world (it sucks!) @KoltonAndrus #qconlondon https://t.co/kxe8xk86t6
@KevlinHenney: It's expensive to know everything up front.— @KoltonAndrus #QConLondon
@danielbryantuk: .@palvaro "My code is now actually running live on Netflix..."@KoltonAndrus "...well, minus all of the println statements" #qconlondon
@charleshumble: Adapt the theory to the reality @palvaro @KoltonAndrus #qconlondon
@robyoung26: “Sometimes it’s easier to fit the model to the immutable reality than the opposite.” - @palvaro #qconlondon
@oli_codes: The real world is not as "clean" as the purist may like. #qconlondon https://t.co/1BScbnczNF
@techiewatt: Lineage-driven fault injection: @KoltonAndrus @palvaro #qconlondon: Start with success and work backwards & adapt the theory to reality
@charleshumble: Starting from a successful outcome is a good way to bound a complex problem. @palvarov @KoltonAndrus #qconlondon
Reflections on Software Architecture
Twitter feedback on this keynote included:
@OpenCredo: The quality of software architecture is often the determining factor in the longevity of a system. It's all about tradeoffs #qconlondon
@techiewatt: architecture - you've got one whether you know it or not @LindaNorthrop #qconlondon
@mjbros: Who is the architect of the devops process in an organisation? @lindanorthrup #qconlondon
@charleshumble: What I thought at the time was that the good news we had UML. It was unified. The bad news was we had UML. @lindanorthrop #qconlondon
@ouertani: Must books for architects #qconlondon https://t.co/iJGjaHA9JV
@danielbryantuk: Advancements in software architecture since the 90s. And yes, SOA was an advancement :-) #qconlondon https://t.co/aMNaP4ErZu
@charleshumble: The internet of broken things. Lots of challenges here. @lindanorthrop #qconlondon
@mpaluchowski: #Microservices get us the kind of continuous deployment we want. #qconlondon @lindanorthrop
@danielbryantuk: Evolution of software architecture. The common theme is the ability to continuously deploy new features #qconlondon https://t.co/CzQjwKuIY4
@mpaluchowski: Being agile requires #architecture with flexibility designed into the system. #qconlondon @lindanorthrop
@sirgoatofboy: Agility is enabled by architecture, not stifled by it - love it! @lindanorthrop #qconlondon
@ejtweet: Linda Northrop (SEI) at #qconlondon: There is a big difference between being agile and doing agile.
@charleshumble: Bad architecture choices are the top contributor to technical debt. @lindanorthrop #qconlondon
@danielbryantuk: DevOps deployment tools and automation can make software architecture a commodity #qconlondon Slightly controversial, but very interesting
@fabianpiau: Software architecture and refactoring is invisible in the backlog but must be address to minimize the technical debt #qconlondon #fact
@juliotrigo: What colour is your backlog? #qconlondon https://t.co/h7GNyJRuQ8
@charleshumble: Evidence is not the plural of anecdote. @lindanorthrop #qconlondon
@KingPrawnBalls: 4 big architectural challenges (1/2): 1) accelerating capability. New features faster & faster. 2) scale. Planetary scale! #qconlondon
@KingPrawnBalls: 4 big arch challenges (2/2): 3) s/w assurance. Deliver on time in budget on quality. 4) evidence. rely on data not assumptions. #qconlondon
@ludovicianul: Architecture is the medium for doing tradeoff analysis #qconlondon
@charleshumble: Software is *the* building material for our world. #qconlondon @lindanorthrop
@roywasse: You can't manage what you can't monitor - quote from keynote Linda Northrop #qconlondon
@laarchy: 90's - the Golden Age of Software Architecture #qconlondon https://t.co/99qrxQ5sU8
@toughplacetogo: The quality and longevity of a software system is large dependent on the architecture #qconlondon #keynote
@nora_js: If you want scalability, you need to make an architectural decision that supports scalability. - @LindaNorthrop #qconlondon #keynote
Unevenly Distributed
Luke Bond attended this keynote:
Adrian is the author of The Morning Paper, a daily digest of research paper summaries. Adrian spoke about what he has learned by reading and summarising a paper a day for over a year. He posited that papers A. make us think, B. raise our expectations of what's possible, C. share applied lessons with us, D.allow us to participate in the "Great Conversation" through the various references cited in the papers we read and E. give us a glimpse into the future (which is already here, but not evenly distributed).
Jeroen Gordijn’s attended this keynote:
Adrian had two goals with his presentation. The first goal is to tell the attendees why he is reading so much papers and why the attendees (or you as reader) should read more papers. His second goal is to show some of the fun and amazing research that he has found. Over the course of approximately 1 year he has read an article every weekday and written a summary about every article. By doing this he has read over 350 papers! In his view there are 5 reasons to read articles:
- Thinking tools - By reading articles you sharpen your mind. …
- Raise Expectations - Learn that you don’t have to settle for the status quo. …
- Applied Lessons - Articles aren’t only written by researchers, but also people who are working in the field. …
- The Great Conversation - If you are really interested in a specific area and you read a lot of papers about the subject you start to see how history builds. There are a lot of references in articles and when you follow these you can see how things changed over time. This will help you place things in context.
- Uneven Distribution - A famous saying goes: “The future is here, it is just not evenly distributed”. According to Adrian the here is in research papers. …
Twitter feedback on this keynote included:
@danielbryantuk: I'm hoping to convince developers and architects to read more papers. There are thinking tools, applied lessons @adriancolyer #qconlondon
@eoinwoodz: Five great reasons to read research papers from @adriancolyer at #qconlondon https://t.co/SVhjGMARs8
@dthume: Any system can scale arbitrarily well - if you add enough easily parallelizable inefficiencies @adriancolyer on COST at #qconlondon
@danielbryantuk: The Scalable Communtativity Rule paper providing insight on good API design... with @adriancolyer #qconlondon https://t.co/wI74UpCgCH
@feroGarcia: Do less testing...but don't sacrifice quality...! Awesome paper from the @Microsoft team #qconlondon
@charleshumble: You can save millions of dollars if you do less testing. @adriancolyer #qconlondon
@mpaluchowski: Computer used to be a job title. People went to work and did computing. @adriancolyer #qconlondon
@alblue: Lots of stuff coming up in computing — @adriancolyer at #QConLondon https://t.co/uC9LOZHgnp
@alblue: The new latency numbers everyone should know — @adriancolyer at #QConLondon https://t.co/ROFjhxIj7Q
@jovianjake: Highly encouraged to read papers by @adriancolyer at #qconlondon . Now that's a surprise
@KingPrawnBalls: #qconlondon keynote: share something you've learned every day and it soon compounds into a great resource. @adriancolyer
21st Century Culture from Geeks on the Ground
Ending the Chain-of-blame: Continuous Consequence
Magnus Ljadas’ attended this session:
Katherine Kirk concluded, the 3 characteristics for human existence is also true for our business:
- Everything is in a constant state of change
- We need to collaborate
- We will always battle dissatisfaction
Twitter feedback on this session included:
@ocklund: @kkirk Be grateful if it's your fault because then you can change it. If you pass on blame you make yourself helpless #qconlondon
@merybere: You don't loose your intelligence because you said I appreciate you Katherine Kirk talking about gratitude #qconlondon
Far from the Mobbing Crowd
by Steve Tooke & Matt Wynne
Twitter feedback on this session included:
@kevj121: Issues and concerns resolved faster and remote working all together #qconlondon #gemsretail https://t.co/Xl4K6C5qnc
@roywasse: Learning about #Mobb programming. It's about the teaming, not about being in a team. #qconlondon
@KingPrawnBalls: Effective remote team: foster a sense of adventure, community, purpose and play. But allow independence. #qconlondon https://t.co/HZszs1PgXF
How to Win Hearts and Minds
by Kate Gray & Chris Young
Twitter feedback on this session included:
@paulacwalter: The only thing you know about people is that they all have different agendas. @grisgraygrau #qconlondon
@danielbryantuk: Fantastic insight into driving tech cultural change with methods from electoral politics by @grisgraygrau and @worldofchris #qconlondon
@danielbryantuk: Segmentation, vision and polling - vital for driving a successful campaign! #qconlondon @grisgraygrau and @worldofchris
@amertner: If you want people to do something different, persuade them through reason and convince them through emotion #emotion #qconlondon
@paulacwalter: To deliver effective business change, be very clear about the goal and how you articulate it. Relevant & meaningful. #qconlondon
@paulacwalter: Do the hard work to find out why people are not with you right now. Show respect for others. #qconlondon @grisgraygrau
Architecting for Failure
@willhillbet: Love Failure & Embrace the Fall Out
Tim Anderson’s attended this session:
William Hill's R&D Engineering Lead Gavin Stevenson told attendees that they should celebrate IT failures. "The best time to understand how your system works is when it is dying," he said. …
Stevenson's underlying point is that examining how an application fails when under stress is more illuminating than simply observing it working. Failure identifies the limitations of the system. …
Stevenson's team also relies on Docker containers for deployment. "Everything we do in R&D, it's Docker," he said; though they have struggled with container load-balancing and orchestration. "There isn't a brilliant solution," he said, though they are looking at Docker Swarm, a product for clustering Docker engines.
"It's a reactive microservice-based architecture," said Stevenson. "Probably. Nobody seems to agree what microservices is."
Peter Liljenberg attended this session:
Gavin Stevenson … talked about WilliamHills betting engine and how they are transitioning from a large database centric solution to a microservice-based architecture (this was a common theme during the conference). They were building a “production ready” betting engine in 2 week sprints, testing it with real production data. The most interesting takeaway was how important it is to really try to break your system. When the system fails, that’s when you learn. The old saying “If it ain’t broken, don’t fix it” just doesn’t apply anymore.
“If it ain’t broken, try harder!” – Gavin on testing
Twitter feedback on this session included:
@danielbryantuk: Data flows through the system. We try to avoid explicit pushing or pulling. This improves fault tolerance @WillHillBet #qconlondon
@stefanoric: William Hill new settlement engine is written with Erlang #qconlondon
@danielbryantuk: Erlang syntax may initially make your eyes bleed, but use it for a couple of weeks and you'll like it. #qconlondon
@OpenCredo: The supervision model and pattern matching of Erlang is very useful for building robust and scalable solutions @willhillbet #qconlondon
@IzidorMatusov: If it isn't broken, break it and learn from the failure. #qconlondon
@timanderson: Which programming language? is the wrong question says Gavin Stevenson #qconlondon. Find good people, recruitment is hard.
@paulacwalter: small applications, good people and operational support are what matters, not the language you choose #qconlondon
@pliljenberg: If it ain't broken, try harder - Gavin Stevenson @WillHillBet talking about embracing failure. #qconlondon
@kylethompson86: First talk of the day at #qconlondon by william hill emphasizing the importance of monitoring because your tests will NOT cover everything
Cassandra at Apple Scale
Alex Blewitt attended this session:
There was also a Cassandra at Apple Scale presentation, including some of the failure modes that they have seen. These have resulted in a number of fixes being applied to the codebase, including signatures for host membership and incremental rebuilding to avoid problems when incomplete data and group membership change at the same time.
Twitter feedback on this session included:
@andyhedges: Apple has 100,000 instances of cassandra in production #qconlondon @kohlisankalp
@v3rtti: We have a lot of data. And it's real data, not analytics data. @kohlisankalp #lol #qconlondon
Staying in Sync: From Transactions to Streams
Peter Liljenberg attended this session:
Martin Kleppmann held one of the best presentations of the whole conference where he talked about keeping data sources in sync, moving away from (distributed) transactions to streams. The content was not very in-depth, but Martin had deep knowledge of the subject, excellent slide and a lot of energy when presenting. This is a talk everyone should watch and learn from.
“Stupidly simple solutions are the best” – Martin Kleppman
Magnus Ljadas’ attended this session:
Martin Kleppmann … proposed a stupid simple solution, as he put it, as an alternative to distributed transactions by using a distributed commit log.
Twitter feedback on this session included:
@mpaluchowski: Synchronization is probably a 40-year-old problem and embarrasingly we still don't have a good way of handling it. #qconlondon @martinkl
@mpaluchowski: Keeping systems in sync is essy when everything goes well. Handling failure is hard. #qconlondon @martinkl
@mpaluchowski: It's not eventual consistency. Won't resolve itself. It's perpetual inconsistency. #qconlondon @martinkl
@mpaluchowski: Most synchronization problems are order problems. Fixing order wins half the battle. #qconlondon @martinkl
@Solisolitude: Stupidly simple solutions are the best #qconlondon
@ocklund: @martinkl makes a strong case using ordered log (Apache Kafka) instead of transactions to stay in sync #qconlondon https://t.co/VIZoEG6tdZ
@MrAndyButcher: Event streams + total ordering, to avoid "perpetual inconsistency" and deadlock. Excellent talk from @martinkl as always #qconlondon
Architectures You've Always Wondered About
#NetflixEverywhere Global Architecture
by Josh Evans
Peter Liljenberg attended this session:
Josh Evans from Netflix presented how Netflix have expanded there streaming services to almost the entire globe (Netflix#Everywhere). Netflix have had some major outages and failures, both in their own software and the underlying cloud AWS-platform. Josh concludes that “Failure is inevitable” and that one really have to embrace the failure and not fail in the same way twice. This had lead Netflix to embrace a “Failure-driven architecture” approach when building their platform.
Josh presented Netflix four architecture pillars: data, caching, traffic and microservices, and how they use (among other techs) EVCache, Cassandra and DNS to keep their services up and running in case of total failures of an AWS datacenter/region. He also showed how they test failure in different regions and route traffic to another region to minimize customer impact. If infrastructure and architecture at scale is of any interest, watch this talk when it comes online!
“Never fail in the same way twice” – Josh Evans
Twitter feedback on this session included:
@worldofchris: Failure is inevitable, finger pointing doesn't help. Learn and never fail the same way twice @Ops_Engineering @netflix #qconlondon
@robyoung26: Soft routing gave Netflix the tool they needed to evacuate a large-scale regional outage. #qconlondon
@robyoung26: Visualizing error rates at @Netflix. Soft routing and then DNS-based region isolation during incident #qconlondon https://t.co/hXcB8y2pQ4
@robyoung26: The time-series anatomy of a @Netflix regional failure exercise #qconlondon https://t.co/MF9vBcMNd6
@KingPrawnBalls: Think globally, act locally. Netflix arch solutions were always taking them a step further towards going global. #qconlondon /Josh Evans
@robyoung26: Availability at @Netflix increased once they got more rigorous with their failure rehearsal exercises #qconlondon https://t.co/Anc6vBOgg6
Architecting Google Docs
Tim Anderson’s attended the session:
Lemonik was involved in a small company called 2Web technologies which developed an Excel-like engine in 2003-4, and joined Google (which acquired 2Web) in 2005 to work on Google Sheets. The big story here was the how Google Sheets became collaborative, so more than one person could work on a spreadsheet simultaneously. “Google didn’t like it initially,” said Lemonik. “They thought it was too weird.” The team persisted though, thinking about the editing process as “messages being transferred between collaborators” rather than as file updates; and it worked.
You can actually use today’s version in your own projects, with Google’s Realtime API, provided that you are happy to have your stuff on Google Drive.
I particularly enjoyed Lemonik’s question to the audience. Two people are working on a sheet, and one types “6” into a cell. Then the same person overtypes this with “7”. Then the collaborator overtypes the same cell with “8”. Next, the first person presses Ctrl-z for undo. What should be the result?
The audience split neatly into “6”, “7”, and just a few “8” (the rationale for “8” is that undo should only undo your own changes and not touch those made by others).
Google, incidentally, settled on “6”, maintaining a separate undo stack for each user. But there is no right answer.
Lemonik also discussed the problem of consistency when there are large numbers of contributors. A hard problem. “There have to be bounds to the system in order for it to perform well,” he said. “The biggest takeaway for me in building the system is that you just can’t have it all. All of engineering is this trade-off.”
Alex Blewitt attended this session:
Apparently Google Docs started off as a web-based excel spreadsheet server; a small company called 2WebTechnologies had a product called XL2Web that provided a remote viewing platform for spreadsheet content, by interpreting an Excel spreadsheet on the server and then rendering a remote HTML view. Since the leading browser was IE SP1 at the time, the data had to be stored on the server in order to came through. The collaboration was an accident as two people edited a document at the same time and they both saw the results. Fortunately the model they built had no non-commutative operations which meant that operations could be replayed. It’s the same model in Google Docs SDK today; and the fact that the APIs existed helped mobile adoption later on. Scaling is through sharding and consistency trade-offs; for a popular document in read/write mode, users may be switched to a read-only version that may be delayed, thereby trading availability for consistency. It turns out that once the unit of consistency is too large for a single server, it’s no longer consistent, which means finer-grained control of document sharding often implies sharding at a greater level, like per chapter on a book.
Twitter feedback on this session included:
@alblue: “If Google asks you something as a startup, you say ‘yes’” - on moving calculation to browser #QConLondon https://t.co/QVgVcdHcPU
@alblue: “The leading browser in the time was IE6 SP1 — let that sink in for a minute” #QConLondon https://t.co/5kDmu3nYGZ
@alblue: “Storing data server side was an artifact of how bad the client (IE) was” #QConLondon https://t.co/ZH1tyQu3Bd
@alblue: “The collaboration came about by accident—the first time we saw two people editing documents at the same time we were intrigued” #QConLondon
@alblue: “A feature which came about by accident—snapshot function was used to reconnect clients but became undo” #QConLondon https://t.co/oEwCosr7RF
@alblue: “The spreadsheet model we happened to build had no non commutative operations” - describing a happy accident #QConLondon
@alblue: “The transformations for this kind of model are still used today in Google Docs SDK” #QConLondon https://t.co/ziyfBS0AsR
@alblue: “The users love it — the engineers hate it. That’s how you know you’ve arrived at the right solution.” #QConLondon
@alblue: “We were lucky when phones and API clients showed up—because shared objects could immediately deliver” #QConLondon https://t.co/YoNw7VM7CZ
@alblue: “Once your unit of consistency is too large for a single server it’s no longer a unit of consistency” #QConLondon https://t.co/bMFU1UfHDH
@alblue: “Let me just check my notes on what I can say about Google storage … ...Google storage has a lot of data” #QConLondon
Cloud-based Microservices Powering BBC iPlayer
Tim Anderson’s attended the session:
Stephen Godwin spoke on Microservices powering BBC iPlayer. This was a compelling talk for several reasons. The BBC is hooked on AWS (Amazon Web Services) apparently and stores 21TB daily into S3 (Simple Storage Service). This includes safety copies. iPlayer was rebuilt in 2013, Godwin told us, and the team of 25 developers achieves 34 live deployments per week on average; clearly the DevOps stuff is working here. Godwin advocates genuinely “micro” services. “How big should a microservice be? For us, about 600 Java statements,” he said.
Twitter feedback on this session included:
@KingPrawnBalls: 10,000 hours of media published in an average week on BBC iPlayer. 10m requests to play video each typical day. Video @ scale! #qconlondon
@timanderson: BBC iPlayer was rebuilt almost from scratch in 2013 apparently #qconlondon
@timanderson: BBC stores 21TB per day into Amazon's S3 storage service #qconlondon
@danielbryantuk: Learnings from the BBC's use of AWS S3 SDK were fed back to Amazon, and changes were made (TCP timeout option on conn re-use) #qconlondon
@danielbryantuk: I'm not going to say how big microservices should be, but at the BBC we have converged on about 600 lines of Java @SteveGodwin #qconlondon
@worldofchris: Cool. @BBCiPlayer run @netflix #chaosmonkey in production in @awscloud @SteveGodwin #qconlondon
@mpaluchowski: An empty queue is a happy queue - is our rule. #qconlondon @SteveGodwin
@Mollydogsdad: Bbc can deploy iplayer to live in 15 mins. Impressive. #qconlondon
@timanderson: BBC iPlayer team: 25 devs, 34 live deployments per week. "Devs should spend 60% of their time writing tests" #qconlondon
@danielbryantuk: We write outside-in tests with Ruby, even though we are a Java shop. No code sharing forces use of 'front door' @SteveGodwin #qconlondon
@mpaluchowski: You can tell which services are too big. That's the ones developers don't want to work with. #qconlondon @SteveGodwin
@danielbryantuk: Elastic scaling is good, but linear scaling is excellent. This helps capacity planning, resourcing, and costing @SteveGodwin #qconlondon
ECS & Docker: Secure Async Execution @Coursera
Twitter feedback on this session included:
@fabianpiau: Back in 2012, @Coursera was a PHP monolith app! #qconlondon
@parcelbizerba: @bsaeta from coursera talks about their journey from PHP monolith to scala microservices #qconlondon https://t.co/wVBV4gskyQ
@fabianpiau: @coursera has developed its own job scheduler called #iguazu, a layer in between Amazon EC2 to manage autoscaling #qconlondon
@fabianpiau: @coursera has huge security challenges! Which company can say it is running arbitrary code on its own server? #codingAssignement #qconlondon
@fabianpiau: To face this security challenge, @coursera is using containers (@Docker) #previoustweet #qconlondon
@WelshSeanSter: How would you like random people from the Internet to upload code and then you compile and run on your infra? @bsaeta #qconlondon @coursera
Back to Java
Hot Code Is Faster Code - Addressing JVM Warm-up
by Mark Price
Twitter feedback on this session included:
@charleshumble: We get about a 20x speed up on trivial methods between interpreted and compiled #java byte code. #qconlondon
@charleshumble: JVM warm-up strategies: Warm-up in-place - send data through the system. Must be production like. #qconlondon @epickrram
@charleshumble: jitwatch - Log analyser /visualiser for Java HotSpot JIT compiler. Inspect inlining decisions, hot methods, etc. @epickrram #qconlondon
@charleshumble: JMH profiler - can only use in the context of a JMH benchmark but provides deep inspection of code. @epickrram #qconlondon
Java 9 - the (G1) GC Awakens!
Twitter feedback on this session included:
@charleshumble: In JDK9 the Heap Occupancy threshold for G1 is adaptive. Currently defaulted to 45%. #qconlondon @mon_beck
@mjpt777: Monica @mon_beck Doing a great job of explaining G1 GC but wow is it ever complicated. #qconlondon
@charleshumble: A heavily tuned JVM command line may be restricting the G1 GC ergonomics and adaptability. @mon_beck #qconlondon
Netty @Apple: Large Scale Deployment/Connectivity
Alex Blewitt attended this session:
Norman Maurer talked about Apple’s use of large-scale Netty deployments (over ˝ million), and some of the contributions that he and others at Apple had made to the framework. One of them was a high-performance networking and SSL termination layer, originally ported from Twitter’s Finagle. Part of the problem is that Java’s NIO has too many synchronized locks and the Java SSL libraries generates too much garbage, which prevents maximizing core usage. By providing a different SSLProvider for Netty and binding to OpenSSL/LibreSSL/BoringSSL he was able to demonstrate an increase from 10Mb/s to 60Mb/s and from a 40% utilization of a multi-core box to 100% utilization for a load tester.
Twitter feedback on this session included:
@alblue: Problems with NIO:* Selector.selectedKeys() produces too much garbage* Synchronized everywhere* Internal copying of buffers #QConLondon
@dthume: ...Although they can't tell us *which* services are netty based :-) #qconlondon
@dthume: 550,000 netty instances in production at Apple. 'Nuff said. #qconlondon
@alblue: “JNI has an awesome API” – sarcasm is live and well at #QConLondon
@alblue: Using Epoll transport for Netty results in less GC pressure and uses advanced socket setup #QConLondon https://t.co/eeMmO9doYH
@charleshumble: Who likes using ByteBuffers? No hands go up. #qconlondon @normanmaurer
@charleshumble: JDKs SSL implementation is very slow. With a naive benchmark 16mb/second, unable to utilise all cores. @normanmaurer #qconlondon
@charleshumble: JNI based SSLEngine was part of @twitter Finagle, now part of Netty. 4x faster than the JDK implementation. #qconlondon
@andyhedges: ~100k TPS with JDK SSL, then ~500k TPS with netty equivalent on same box. Netty fully uses the server's CPU resources too. #qconlondon
@alblue: Asynchronous IO frameworks need to take account of back pressure to avoid unconstrained memory problems @normanmaurer at #QConLondon
@pliljenberg: Massive scale? 10s PetaBytes of data through #netty every day at Apple - @normanmaurer #qconlondon
@dthume: Atomic*fieldupdater saved 4gb of heap w/1billion connections in netty #qconlondon
Project Jigsaw in JDK 9: Modularity Comes to Java
by Simon Ritter
Twitter feedback on this session included:
@fabianpiau: To date 23 classes, 18 interfaces and 379 method have been deprecated, nothing removed #javafacts #qconlondon
@fabianpiau: #JDK9 has a new directory tree, bye bye the jre folder and its lib folder containing some duplicated files! #qconlondon
@fabianpiau: From JDK8, the command line tool jdeps has been introduced to understand the static dependencies of your apps and libraries #qconlondon
@stealthness: #qconlondon Simon Ritter - sun.misc.Unsafe such a hidden gem to be lost in modularity. https://t.co/AQIWkhp4Me
@charleshumble: “We should definitely, hopefully, get Jigsaw in JDK 9. No promises though.” @speakjava #qconlondon
@fabianpiau: With #jigsaw, the 'public' keyword now has different flavours depending on the module visibility and configuration #qconlondon
@charleshumble: No more classpath hell. Now we have -modulepath hell. It is better though. @speakjava #qconlondon
Spring Framework 5 - Preview & Roadmap
Twitter feedback on this session included:
@charleshumble: Source code readability is the heart of what we're doing with our programming model. @springjuergen #qconlondon
@fabianpiau: In #spring 5, the autowired annotation becomes optional #qconlondon
@charleshumble: Themes for Spring 5.0: JDK 9, HTTP/2 Reactive architecture. Java 8+ baseline, skipping Java 7, #JUnit 5. @springjuergen #qconlondon
@charleshumble: It looks like Jigsaw will have no concept of versioning, just structuring + visibility enforcement. @springjuergen #qconlondon
@charleshumble: Servlet 4.0 is an excuse for servlet vendors to not support HTTP/2. HTTP 1.1 is 20 years old. #qconlondon https://t.co/oaIN9q73Bz
@ludovicianul: Spring Boot 1.4 will make HTTP2 a first class citizen #qconlondon
@charleshumble: Spring reactive is a Spring MVC like endpoint model. It is being developed as a public R&D project - https://t.co/vmJT9fjzsC #qconlondon
Close to the Metal
How Will Persistent Memory Change Software Design?
Magnus Ljadas’ attended this session:
New innovations in hardware will impact how we build systems: persistent RAM will fundamentally change how we database stuff, and hosts being able to access RAM on other hosts over super low latency network channels without consuming CPU will impact what’s possible to build.
The Quest for Low-latency with Concurrent Java
Alex Blewitt attended this session:
His open-source Aeron message server uses an out-of-process message broker that runs on the client and delivers messages outside of a Java process, so that client-side Java garbage collections don’t introduce unnecessary jitter due to GC pauses. Messages are passed by appending into a rotating set of queues and then shared memory is used to allow the client to write directly into the memory space read by the server. He also compared the standard Queue objects in Java, and showed that they either generated garbage, had locks, or both, and that the latencies (particularly the 99th percentile) were significant once contention between multiple producers kicked in. He then introduced the ManyToOneConcurrentArrayQueue and showed benchmarks to demonstrate its effectiveness over other standard queue mechanisms.
Twitter feedback on this session included:
@alblue: “If a system does not respond in a timely manner then it is effectively unavailable” – @mjpt777 at #QConLondon
@alblue: The performance benchmarks mentioned by @mjpt777 at #QConLondon talk are at @github here: https://t.co/tKEBzBUytj
@silverSpoon: Logging is a messaging problem @mjpt777 #qconlondon
@santmatthew: @mjpt777 is so right: If you don't know Amdahl's law and queueing theory, they will hunt you down. #qconlondon
Understanding Hardware Transactional Memory
by Gil Tene
Alex Blewitt attended this session:
Although HTM was attempted in Haswell, it was disabled due to firmware flaws – but now the latest generation of Broadwell chips have HTM enabled through the TSX instruction. This allows a transaction begin to occur (with a call-out to a cleanup/retry location if it fails) and then all cache state modified during the subsequent instructions stays in the cache until the commit, at which point the modified changes are written back or the state is restored as at the beginning of the transaction and the retry logic is called. This allows for some specific improvements (particularly with lock elision using effectively free optimistic locking) and particularly if the locks in case are over a variety of different data structures the speculation can provide increased concurrent throughput.
Peter Liljenberg attended this session:
Gil Tene talked about Hardware Transactional Memory. Really low-level stuff about CPU pipeline and cache optimization. HTM in the JVM is not new, Azul has been delivering both hardware and a customized JVM with JVM for 10 years. What’s interesting is that it will become mainstream now when Intel is shipping CPUs with support for HTM. Gil succeeded in a very educational way describe the complexity of HTM and how it can be implemented in for example the standard JVM. In the end Gil talked about how the developers must reason about locking and synchronization to make the most of HTM in their code.
Containers (in Production)
Build, Ship and Run Unikernels
Twitter feedback on this session included:
@mpaluchowski: There's code you want to run and the code your OS includes, that comes along for the ride. #qconlondon @justincormack
@fintanr: Operating systems are just kind of technical debt -@justincormack #qconlondon << at times it's hard to disagree here
@mpaluchowski: That Unix code written in the 70s is still there. Lots of technical debt in system programming. #qconlondon @justincormack
Observe, Enhance, & Control: VMs to Containers
Peter Liljenberg attended this session:
Mitchell argues that the “state-of-the-art” tools from the age of VMs are not really suited to handle the tasks anymore. Even though the tools are extremely good, they do solve a completely different problem.
Twitter feedback on this session included:
@fintanr: Observe, enhance, control @mitchellh #qconlondon https://t.co/sv4mK7u0y1
@fabianpiau: There was the age of virtual machines back in 2006, now we are in the age of containers @mitchellh #qconlondon
@toughplacetogo: ”Nowadays banks are software companies that happen to do finance" #qconlondon #containers @mitchellh
@fabianpiau: @mitchellh "If your software is not API driven, then it is probably a second choice one..." Thanks god, I'm working on a API! #qconlondon
@fintanr: 2016 datacenter problems - infrastructure management, service discovery, configuration and scale: speed and size @mitchellh #qconlondon
@fintanr: Updating a server and waiting for an hour for dns to sync is just not an option - @mitchellh #qconlondon
@fabianpiau: Configuration is the main problem when working with containers and microservices. Couldn't agree more! #qconlondon
@fintanr: The world in 2016 from @mitchellh #qconlondon https://t.co/I808DhY64Q
@fintanr: key value stores as replacement for traditional config management tools for #Containers - @mitchellh #qconlondon
The Dark Art of Container Monitoring
Twitter feedback on this session included:
@laarchy: An attendee to the speaker about a container monitoring tool : Do you support Windows? The speaker : WHAT ? %-D #qconlondon
Data Science & Machine Learning Methods
Natural Language Processing (NLP): Here Be Dragons
by Emma Deraze
Twitter feedback on this session included:
@charleshumble: Any problem you possibly have with txt you can have with @twitter Emma Derate, #qconlondon - for e.g. Hashtag wars
@charleshumble: Sentiment analysis. How do people feel about this. I don't know and honestly I don't care. Emma Derate #qconlondon
@charleshumble: The only thing sentiment analysis is good for is reviews, but you have the star rating so you already know Emma Deraze #qconlondon
@charleshumble: Bosch sues Dyson over vacuum cleaner claims. Is that positive or negative. I don't know, it depends. Emma Derate #qconlondon
@megamda: NLP is all about high probability and speculation != accuracy #qconlondon #nlp #machinelearning
DevOps & CI/CD
Applied CI/CD: Enabling Creativity @Volvo Trucks
Twitter feedback on this session included:
@timanderson: Smartphones are a small change compared to automation of vehicles says Volvo's Peter Thorngren #qconlondon
@timanderson: Sounds like "My truck's crashed" will have a different meaning in 2025 #software #qconlondon
@justincormack: Real world tests are mainly to calibrate models at Volvo #qconlondon
Continuous Delivery: Benefits Explained
Twitter feedback on this session included:
@amertner: Cycle time improvements have much larger impact than productivity improvements. #cd #qconlondon
@fiddur: Continuous Deployment becomes so boring so we deploy at peak time to make it interresting. #qconlondon
@phuturespace: #qconlondon. LianPing Chen's Paddy Power success story for Continuous Delivery. More releases per month. No releases per weekend.
Immutable Infrastructure: Rise of Machine Images
Twitter feedback on this session included:
@chrismare: #qconlondon @axelfontaine treat servers like cattle and not pets
@andyhedges: CRUD for servers is dead. We are discarding the U @axelfontaine #qconlondon
@roywasse: Complexity is the enemy of security #qconlondon (quote from @axelfontaine immutable infrastructure talk)
Disrupting Finance
Building Trust Machines using the Block Chain
by Ken Kappler
Twitter feedback on this session included:
@eoinwoodz: BlockChain is about consensus across a distributed network, not just Bitcoin @KapplerKen #qconlondon
@eoinwoodz: BlockChain doing for financial transactions what watches did for time (ubiquitous visibility and consensus). @KapplerKen #qconlondon
Fighting the #Fintech Wave with DevOps
Twitter feedback on this session included:
@danielbryantuk: A shout-out to the DevOps periodic table by @benjaminwootton at #qconlondon https://t.co/jfzBhzJ1pA
@danielbryantuk: You're not going to turn around a 10,000 person org with beer and pizza meetups @benjaminwootton on enterprise DevOps at #qconlondon
@danielbryantuk: Is a DevOps team a good idea? In our experience, it can help DevOps scale through federation @benjaminwootton #qconlondon
@danielbryantuk: Surprise! Banks are surprisingly DevOps mature... @benjaminwootton #qconlondon https://t.co/Po6l2NlDUQ
Hacking Bank Mobile Apps
Tim Anderson’s attended the session:
… He set himself the task of analyzing mobile apps from banks like RBS and Barclays to discover how they communicated with their servers….
One idea is a man-in-the-middle proxy, where the app communicates with your server thinking that it is talking to the bank's servers, but this does not work with banking apps, he explained. They use a technique called SSL pinning, where the app has a copy of the bank's security certificate, and verify that the server is using that same certificate for encryption.
There are ways round this, Graham remarked, but it is not worth trying to circumvent it. Instead, he used a technique called method hooking, where the functions in the app itself are augmented by the hacker. Objective-C makes this easy, he said, because of its dynamic dispatch system which defers the decision about which function to call until runtime. "You can insert shims that decorate or completely replace implementations," he said.
It is not as simple as that though, thanks to steps taken by the banking app developers to make them harder to reverse-engineer.
"Some banks, like Barclays, take numerous steps to obfuscate," Graham said. Nor did he share all his secrets, but he was able to demonstrate his success; though he said the effort took him most of a year….
"I have possession of the device. I can see the data that's on the device. If the OS can execute the app, it's possible for a human theoretically to reason about how it works. Barclays maybe want to protect their users from malware. That's a legitimate reason for hardening the apps. I'll still crack them. It's going to be an arms race, a game of whack-a-mole," said Graham.
"I don't want to reverse-engineer apps. I did it as a conversation starter, here's what's possible," he adds. He has, perhaps naively, a belief in the democratizing power of technology to make life better for ordinary people.
Twitter feedback on this session included:
@timanderson: Oauth - "that's another talk on why it's not secure enough" #qconlondon
@timanderson: Question to speaker: "any risk of legal issues"? "We'll find out, stay tuned" #qconlondon
@timanderson: I'm on your side, but a lot of people above me are not says banking IT guy in audience #qconlondon
Full Stack JavaScript
Meet the Node.js Anti-patterns
by Pedro Teixeira & Igor Soarez
Luke Bond attended this session:
YLD's co-founder and CTO Pedro Teixeira was joined by fellow YLDer Igor Soarez in a Node.js double-act discussing anti-patterns and bad practices in Node.js development. Built around a narrative of Jane, a Java developer doing her first Node.js project, we learned about callback hell, code style, error handling and some architecture and scalability challenges. Packed full of lessons, this will be one of those videos you go back and watch many times to extract as much as possible from it.
Twitter feedback on this session included:
@matthewrevell: Callback hell is the first #nodejs antipattern Pedro and Igor cover at #qconlondon https://t.co/9dV90SFg0f
@lukeb0nd: Solutions to callback hell in #nodejs @pgte @igorsoarez @YLDio #qconlondon https://t.co/Rl073r0CGr
@stefanoric: #qconlondon nodejs anti-patterns: replace a long list of arguments with an options object.
@stefanoric: #qconlondon nodejs anti patterns: use the revealing pattern instead of using the 'new' operator.
@timanderson: Modules can be too big, but no module is too small. Tip from Node.js session #qconlondon
@lukeb0nd: Use NPM scripts instead of gulp/grunt. Know if/when to switch though #nodejs @pgte @igorsoarez @YLDio #qconlondon
@lukeb0nd: Use NPM scripts instead of gulp/grunt. Know if/when to switch though #nodejs @pgte @igorsoarez @YLDio #qconlondon
Head-to-tail Functional Languages
An Introduction to Property Based Testing
by Aaron Bedra
Twitter feedback on this session included:
@dthume: Property based testing helps ensure you don't break even the unintended / unknown features - @abedra at #qconlondon
@tomkelgrove: #qconlondon Property-Based Testing - in principle test over domains rather than discrete values. Reduce number of tests to smaller set?
Microservices for Mega-architectures
DDD and Microservices: At Last, Some Boundaries!
by Eric Evans
Twitter feedback on this session included:
@dumitrupascu: When something becomes so popular, it doesn't necessarily mean that is bad - Eric Evans about Microservices at #qconlondon
@mpaluchowski: Microservices mimic the chaotic nature of enterprises. #qconlondon @ericevans0
@mpaluchowski: If teams need to use the same domain language then they're certainly not autonomous in their design work. #qconlondon @ericevans0
@mpaluchowski: There are always multiple (data) models. Microservices embrace this. #qconlondon @ericevans0
@v3rtti: Models need to be clear, not big. @ericevans0 #qconlondon
@andrewdjkilgore: There are always multiple models - Eric Evans #qconlondon
@worldofchris: If you're conforming to a mess then you become a mess @ericevans0 #ddd #qconlondon
@fiddur: Small balls of mud cascading between microservices with @ericevans0 at #qconlondon https://t.co/Jb7HGg6Er4
Microservice Anti-patterns
by Tammer Saleh
Peter Liljenberg attended this keynote:
When are microservices appropriate as an alternative to the monolithic app? The problem with monolithic apps is not about the code, it’s about the teams! Large teams (or multiple teams) can’t work effectively in the same codebase.
Tammer stressed that “the most common mistake is to start with microservices”. Start monolithic and extract as needed, because microservices are complex and impose a constant tax to your development.
Tammer explores how to draw the lines between services. dealing with performance issues, testing and debugging techniques, managing a polyglot landscape and the explosion of platforms, managing failure and graceful degradation.
“Boring is beautiful” – Tammer Saleh
Microservices Chaos Testing at Jet
by Rachel Reese
Twitter feedback on this session included:
@mpaluchowski: Microservices provide a more even distribution of complexity. #qconlondon @rachelreese
@silverSpoon: The tests were too uniform, there was no variance @rachelreese At #qconlondon https://t.co/tQiUlE45kA
@ocklund: @rachelreese on chaos testing. We should plan for chaos, because world is chaotic. Design for failure then emerges. Test in prod #qconlondon
Test-Driven Microservices: System Confidence
by Russ Miles
Twitter feedback on this session included:
@mpaluchowski: Distributed monolith, worked reasonably badly in one place, now so in many places. #qconlondon @russmiles
@oli_codes: Microservices .... "a monolith but distributed" HAHAHA @russmiles #qconlondon
@mpaluchowski: In our industry all we do is create our own problems. #qconlondon @russmiles
@mpaluchowski: The complexity of microservice systems should scare us. Focus on testing. #qconlondon @russmiles
@moogster31: mantra: "focus on testing the interface, not the implementation" #qconlondon
@mpaluchowski: The way to communicate and comprehend microservice systems are narratives. #qconlondon @russmiles
@kamkorz: #qconlondon In #microservices cognitive overhead matters, size does not. Well said.
The Microservices and DevOps Journey
by Aviran Mordo
Twitter feedback on this session included:
@robyoung26: “Don’t add something to your architecture until you are feeling the pain of not having it” #qconlondon @aviranm
@mpaluchowski: Solve only problems you have. Don't just add technologies. #qconlondon @aviranm
@mpaluchowski: Arrows in system schemas are failure points. #qconlondon @aviranm
@fabianpiau: The size of the microservice is the size of the team who is building it @aviranm #qconlondon
Modern Agile Development
Business Mapping: Building an Agile Organization
by Chris Matts & Tony Grout
Twitter feedback on this session included:
@Joe0reilly: “We don’t always need new things, we need to do the old things really well” Best phrase of the week from @tonygrout #qconlondon
@paulacwalter: Engagement and explaining why is key to prevent agile transformation being another management fad @tonygrout @PapaChrisMatts #qconlondon
@paulacwalter: Role play on agile decision making makes a good point @tonygrout @PapaChrisMatts scaling #agile #qconlondon
@KingPrawnBalls: Make transparent across your org the EPICS everyone is working towards.Agile in solos isn’t an agile org. #qconlondon /Lloyds transformation
Culture Eats Principles for Breakfast
by Ian Dugmore & Jonathan Smart
Twitter feedback on this session included:
@techiewatt: The most dangerous phrase in the language is, ‘We’ve always done it this way.’ -Grace Hopper via #qconlondon talk by @iandugmore @jonsmart
@danielbryantuk: Don't scale Agile first - de-scale the work first @jonsmart #qconlondon
@timanderson: Descale to improve velocity, eg reducing size of a dev team from 40 to 10 improved productivity at Barclays #qconlondon
@robyoung26: “Your practices are a function of your principles, given your context.” @iandugmore @jonsmart #qconlondon
@robyoung26: “primary measure of success for interactions between control functions (e.g. change control) is happiness” @iandugmore @jonsmart #qconlondon
@robyoung26: “You can’t release anything until the cost of release is lower than the value you’re trying to release” @iandugmore @jonsmart #qconlondon
@timanderson: Barclays gave some form of Agile training to 26,000 people last year #qconlondon
@robyoung26: “To effect cultural change, learning anxiety needs to be lower than survival anxiety” @iandugmore @jonsmart #qconlondon
Modern CS in the Real World
Distributed Systems in Practice, in Theory
Alex Blewitt attended this session:
One useful nugget; if you have systems that may fail, building in a lease (pull request heartbeat) and then disconnecting a client whenever heartbeat failures occur is a way to main overall stability; especially if the client’s job is re-submittable. And of course, a multi staged pipeline can be easily scaled if you separate the stages with queues and then have multiple consumers; but that’s distributed scaling 101.
Effortless Eventual Consistency with Weave Mesh
by Peter Bourgon & Matthias Radestock
Luke Bond attended this session:
The "Modern CS in the Real World" track was the place to go for mind-bending, cutting-edge computer science talks, and this talk was no exception. Peter and Matthias from Weaveworks talked about Weave Mesh, a library they abstracted from the internals of Weave itself that does eventual consistency over unreliable networks. They gave an applied example of building a Raft implementation on top of Mesh, which is pretty impressive, and the ease of building it shows the strength of their code and the interface and abstractions they chose.
Not Quite so Broken TLS Using Unikernels
Luke Bond attended this session:
I returned to the CS track to feel overwhelmed all over again, yet Anil managed to make a very difficult subject enjoyable and more-or-less understandable. Anil showed us how much of a mess the C code for very important libraries such as SSL can be, and how remarkable it is that they work as well as they do, despite the constant stream of exploits being announced. He proposes unikernels as a technology to enable us to write minimal library operating systems for our apps, rewriting these low-level libraries in type-safe, memory-safe and testable high-level languages to make the internet more secure.
Twitter feedback on this session included:
@silverSpoon: Who uses TLS/SSL? -all hands up- Who understands it? - two ppl At @avsm talk #qconlondon
@robertharrop: @avsm extolling the virtues of ASN.1 #qconlondon https://t.co/jL5LTsN0Hn
Modern Native Languages
Rust: Systems Programming for Everyone
by Felix Klock
Luke Bond attended this session:
Returning to the CS track for more confusion, I was again pleased that this talk was very clear and understandable. Felix is an energetic and enthusiastic speaker and very engaging. Not being familiar with Rust some of the details were over my head but I was able to get the gist of most of it. Enough to tell me that I need to go and learn some Rust, it looks great.
Alex Blewitt attended this session:
Given that Rust uses extensively checked static compilation to verify that objects are not shared or mutated other than expected, and that the lifetimes of objects are tied to lifetimes in the code, this was a great overview (for me) to the subtleties in the way that the Rust applications work.
Twitter feedback on this session included:
@alblue: Comparing #RustLang ‘s crossbeam MPSC benchmark to Java and Scala’s implementations @pnkfelix at #QConLondon https://t.co/MYg9zdQf9S
@alblue: The parallelism in #RustLang article that @pnkfelix mentioned at #QConLondon is here: https://t.co/JUvqzsvcYK
@alblue: “If you remember nothing else from this talk, remember this: #RustLang cargo is really simple to use” @pnkfelix at #QConLondon
@alblue: “Semantic versioning is really important” — @pnkfelix on the importance of semver in #RustLang at #QConLondon https://t.co/KHRowAAdUi
Successful Go Program Design, 6 Years On
Twitter feedback on this session included:
@danielbryantuk: Advice on @golang repo structure from @peterbourgon at #qconlondon https://t.co/kRe0o7QmTE
@alblue: “If you remember no other slide, remember this one: make dependencies explicit” @peterbourgon at #QConLondon https://t.co/YQ1JV2VSsA
@danielbryantuk: Key message I'm taking away from @peterbourgon's #qconlondon @golang talk is don't cargo cult stuff from other langs https://t.co/RAO0etwB7F
@danielbryantuk: Dependency management recommendations for @golang by @peterbourgon #qconlondon https://t.co/s9DH0qVpES
@danielbryantuk: Use 'FROM Scratch' if deploying @golang via @docker @peterbourgon #qconlondon
The Case for Bringing Swift to the Server
by Chris Bailey & Patrick Bohrer
Alex Blewitt attended this session:
IBM have invested a lot into Swift, on the mobile side for the applications in which they are partnering with Apple, but also on the server side on Linux. Chris does a lot of work with open-source projects, such as Node.JS and Java already, and he and the team are working on porting libdispatch (also known as Grand Central Dispatch) to Linux, so that there can be portable multithreaded code running on OSX, iOS and Linux platforms. Patrick introduced IBM’s Swift language sandbox on Linux, where each time an application is run the code is packaged up and deployed as a Docker container before compiling, running, and showing the results. He also quickly demonstrated Open Whisk which allows Lambda-style computation (writing in languages including Swift) to quickly deploy and upload snippets to the cloud.
Using Pony for Fintech
Alex Blewitt attended this session:
Pony is a fascinating language that builds upon LLVM and uses a sound type system to achieve millions of messages and actors in a single process. Each actor has its own queue of messages but also its own heap; so when garbage collection runs, it only works with an individual actor’s heap, allowing the remainder of the actors to keep processing. As a result, there’s no stop-the-world garbage collection pauses that are seen in other garbage collected runtimes (like Go and Java). In addition, the code is written with no blocking; everything is asynchronous. This means there’s no delay when work is required and that the entire system is event driven. (In fact, when there are no more messages, there are no more work, and the system shuts down.) The message queues are unbounded (to avoid blocking) but of course the system may run out of resources if that happens; there is a form of backpressure that’s applied to throttle the rate of the sending actors if that happens.
Twitter feedback on this session included:
@alblue: Actors have their own heap and garbage collection in @ponylang which means no stop the world GC #QConLondon
@silverSpoon: A language is a tool for solving problems, not an ideology, not about syntax... Sylvan from @ponylang at #qconlondon
@alblue: “If I can write to it, nobody else can read from it” - the data-race freedom of @ponylang at #QConLondon
@alblue: The deny capabilities write up by @adriancolyer of @ponylang paper is here: https://t.co/9AyNa0qB3M#QConLondon
@alblue: “If it complies, it’s data race free” - no locks, concurrency problems @ponylang #QConLondon
@alblue: The queues in @ponylang are unbounded and the queue push is atomic with no allocation #QConLondon
@charleshumble: I'm not going to show any benchmarks today because benchmarks are snake oil. @ponylang #qconlondon
@charleshumble: GC in @ponylang is mark-and-don't sweep.No safepoints,write/ read barriers,card table marketing,compacting, or pointer fixups. #qconlondon
@alblue: The write up by @adriancolyer of @ponylang garbage collection is here:https://t.co/jeNQfiBJOL #QConLondon
@charleshumble: Moving the mark bit out of the object means that unreachable objects are never traced. @ponylang #qconlondon
Optimizing You
Burnout
by John Willis
Luke Bond attended this session:
John Willis from Docker shared a moving story about the suicide of a friend who worked in DevOps, and went on to share some statistics and studies about burnout and some resources for those who need help.
Magnus Ljadas’ attended this session:
John Willis talked about burnout, and it’s a thing not only in society but in our business. Turns out software people tend to be more receptible than the general population, especially the high achievers. A slippery slope that may go unnoticed until too late. Regular self assessment to monitor indicators, just as you would monitor any system, and to just be there for your friends. Important stuff. Be alert.
The take-away is: talk, listen, ask if people are okay and, above all, care. Also check out the Maslach Burnout Inventory: take the test and see if you are at risk.
Twitter feedback on this session included:
@charleshumble: Burnout is the canary in the coal mine. @botchagalupe #qconlondon
@rachelreese: Some research shows burnout can be similar to PTSD. Self-test: https://t.co/NEiNpBamy8 #qconlondon
@ellispritchard: Great talk on recognizing burn-out by @botchagalupe at #qconlondon: I think if we look properly, we'll find it's an epidemic in our industry
Engineering You
Jeroen Gordijn’s attended this keynote:
He pled to go back to the basics of computer science. Even though learning a new JavaScript framework might be fun, the knowledge you gain will be obsolete in a matter of days, or weeks. However, learning about algebra, algorithms and mathematics will give you knowledge that will serve you for years. In IT we sometimes lose ourselves in technology, … instead of solving the problem the business has. A solution doesn’t always need to be the best, most beautiful solution, but it needs to fulfill business requirement. We should not forget that our main focus should be our customer/business.
We should focus on architecture principles, like loose coupling and tight cohesion. …
He shows that there we in IT are quite slow on picking on the basics. In 1968 there was a conference where agile was already described with the following description:
The design process is an iterative one:
1. Flowchart until you think you understand the problem.
2. Write code until you realize that you don’t.
3. Go back and re-do the flowchart.
4. Write some more code and iterate to what you feel is the correct solution.
Another great quote was made by A. J. Perlis in 1968, stating “A software system can best be designed if the testing is interlaced with the design instead of being used after the design”, effectively describing TDD.
Twitter feedback on this session included:
@dosaki: Agile development process described back in '68 #qconlondon https://t.co/kQsnPB10o0
@andyhedges: Good engineers care about algorithms and data structures @mjpt777 at #qconlondon (AH: surprising how many people miss this)
@andyhedges: Some really strong points about abstraction from @mjpt777 at #qconlondon, the wrong abstracts REALLY hurt your software.
@timanderson: An ORM is the wrong abstraction for a database says Martin Thompson #qconlondon - think set theory as as OO
@andyhedges: The first rule of Abstration Club is don't abstract @mjpt777 #qconlondon
@laarchy: What should you learn as a Software Engineer ? The answer by @mjpt777 at #qconlondon https://t.co/OrDPMCdu77
@oli_codes: You have two ears and one mouth, use them in that proportion. Great quote about being a good software engineer. #qconlondon
@eyads: The book any developer could have written. #qconlondon @mjpt777 https://t.co/tyuqHpvhPy
Lead the Revolution by Being Ordinary
Twitter feedback on this session included:
@tastapod: Narcissism, Psychopathy, Machiavellianism: no the Dark Triad is not a Marvell comic but psychological phenomena, says @kkirk at #qconlondon
@tastapod: Loving @kkirk describing blame as an adrenalin-fuelled psychopath trip! #qconlondon
@jonsmart: Stress = Stupidity (cortisol) + Power (adrenaline). Not a great combination. Calm the * down. Change our reaction. @kkirk #qconlondon
@tastapod: .@kkirk just used the word “compassion” in a talk about people and coaching. I don’t hear this nearly enough. #qconlondon
@KevlinHenney: I have to change the word 'compassion' to 'de-risking the people problem' when dealing with upper management.— @kkirk #QConLondon
@ocklund: @kkirk In new world of continuousness: "Gratitude is fearlessness. What's here that I can work with" #qconlondon https://t.co/1zPHdVjWjc
Making a Sandwich: Effective Feedback Techniques
by Dan North
Peter Liljenberg attended this session:
One of my most anticipated talks during the week was Dan North‘s “Making a sandwich”. I hade very high expectations for this talk, and Dan managed to exceeded them (as usual). Dan talked about giving feedback, how feedback in itself is a system and why we should do it. Giving and receiving feedback (which is really just to say ‘Thank you’) is, in my opinion, one of the hardest skills to master and we should really practice a lot! Dan presented some useful techniques and tricks, but you should really watch this yourself!
merybere attended this session:
· Ask for feedback
-
- to improve
- for help
- for recognition, feel good
· We offer feedback
-
- to improve the system of work, as a team
- to model a culture, (when new people comes to a team it helps)
- to control others (let me give you some feedback...)
- to demonstrate our superior knowledge
· Example: go and sit with people, observe how they do things, ask what they are doing is, start a conversation and being interested
Twitter feedback on this session included:
@nora_js: Dan North relating traditional workplace feedback to Systems Theory, putting it in terms engineers can understand :) @tastapod #qconlondon
@nora_js: You as a human are a system who relies on feedback. @tastapod #qconlondon
@mylenereiners: Who has an annual review? Who goes eleven months back to improve? @tastapod #qconlondon
@aviranm: Timing is everything. Small and frequent feedback is better than large and infrequent. @tastapod #qconlondon https://t.co/GghdKoW5Wj
@charleshumble: Go and sit with the people who use your product. Each time they sigh write down what they were doing. @tastapod #qconlondon
@nora_js: And remember, when you receive feedback, always say: 'Thank You'. @tastapod #qconlondon
@ocklund: @tastapod Low-trust environments: Offer specific positive feedback, everything else will self-correct #qconlondon https://t.co/SCCaLYOz0a
Security, Incident Response & Fraud Detection
Bitcoin Security: 1/10th Cent to a Billion Dollars
Twitter feedback on this session included:
@paulacwalter: Olaf Carlson-Wee: if your PC is slow be aware you may be mining Bitcoin for someone in another country #qconlondon
@techiewatt: Olaf Carlson-Wee #qconlondon there are generally 2 threats to your Bitcoins - attackers and you!
Building a Modern Security Engineering Team
by Zane Lackey
Twitter feedback on this session included:
@techiewatt: #qconlondon @zanelackey - surface security info for everyone, not just security teams!
@KingPrawnBalls: Put the security dashboards right where engineering teams are so they’ll see it! Don't hide them away #qconlondon / @zanelackey
@KingPrawnBalls: Reward engineers’ communication with the security team: t-shirts, gift cards, etc. Bootstrap the interaction! #qconlondon / @zanelackey
@techiewatt: #qconlondon @zanelackey contrary to populate belief "deploying quicker makes you more secure"
@KingPrawnBalls: Security in DevOps culture: 3 keys (1/3): 1) make security monitoring & metrics available to all #qconlondon / @zanelackey
@KingPrawnBalls: Security in DevOps culture: 3 keys (2/3): 2) incentivize dialog with the security team (gifts, t-shirts, etc) #qconlondon / @zanelackey
@KingPrawnBalls: Security in DevOps culture: 3 keys (3/3): 3) your policies should not take away capabilities #qconlondon / @zanelackey
@MrAndyButcher: Security teams, don't be a blocker - you'll get worked around. Positive and insightful advice from @zanelackey just now #qconlondon
Stream Processing @ Scale
Microservices for a Streaming World
Philip Carrington attended this session:
This talk looked at an add-on for Apache Kafka called KStreams that allow you to persist the latest version of a key so that it could be use by a microservice in combination with a stream to create other services. We also need to embrace decentralization.
- KStreams can be used to make KTables that can be joined with data from a stream to enable querying for a microservice
- Kafka has compacted tables that allow you to store the latest value for a key if you so wish!
Real-time Stream Computing & Analytics @Uber
by Sudhir Tonse
Twitter feedback on this session included:
@danielbryantuk: Interesting to see that @UberEng store steam processing data in ElasticSearch, rather than a typical DB. @stonse at #qconlondon
@OpenCredo: Which stream processing tools should you use? That depends on your use case @stonse at #qconlondon #NoGoldenHammer https://t.co/Aj4oIoMauq
@danielbryantuk: Listening to @stonse at #qconlondon, and I'm hearing that many stream processing tools are operationally complex (listen up tool makers :-))
Stream Processing with Apache Flink
Philip Carrington attended this session:
New product in a way that will get its full release tomorrow (08/03/2016). It is promising to completely subsume batch by allowing windowing over “large” timescales by utilizing in memory and disk persisted aggregations as well as a host of other interesting features that other systems do not offer.
Google Dataflow is being made into an Apache incubator project called Apache Beam
Twitter feedback on this session included:
@OpenCredo: Using log streaming (via Kafka) allows easy horizontal scalability, as individual workers can pick events and process them #qconlondon
@PaulAnkho: Flink state management seems to resolve issues suffered using storm at production. How will it perform? Pretty cool! Need PoC #qconlondon
@mpaluchowski: You need to use #Kafka because there's no good alternative. #qconlondon @rmetzger_
Sponsored Track
Using Technology as a Blind Long Distance Runner
Luke Bond attended this session:
Simon taught himself to run, blind, using the RunKeeper running app for his phone. Starting on a football field, using the audio feedback, he moved to closed suburban roads and then open motorways. Astonishingly, he uses only the combination of route memorization through under-foot feel, the pain of running into things, and the audio feedback of RunKeeper. He has since run marathons and is about to participate in a desert race using custom technology he has produced with the help of IBM.
Magnus Ljadas’ attended this session:
What a heartwarming moment to watch Simon Wheatcroft, the blind ultra marathon runner, on stage with his guide dog slumbering at his feet. Being a mere marathoner myself I could not constrain my amazement over this man’s achievements, let alone he’s blind on top of that. Technically speaking, the solution that will enable him to run 126 km unassisted through the Namibian desert, is rather straight forward—a gps based beeper thingie that simply beeps when he strays off track. No drones, no real time room mapping, nothing fancy at all. I will start hallucinate anyway so I want it to be simple, he explained. Stupid simple solutions are the best.
Opinions about QCon
Tim Anderson’s attended the conference:
This is a software development conference focused on large-scale projects and with a tradition oriented towards Agile methodology. It is always one of the best events I get to attend, partly because it is vendor-neutral (it is organized by InfoQ), and partly because of the way it is structured. The schedule is divided into tracks, such as “Back to Java” or “Architecting for failure”, each of which has a track leader, and the track leader gets to choose who speaks on their track. This means you get a more diverse range of speakers than is typical; you also tend to hear from practitioners or academics rather than product managers or evangelists.
Impressions expressed on Twitter included:
@rvanbruggen: Hey @qconlondon - congrats on your 10y anniversary conference this week. Here's a #neo4j #graphdb as a present :) https://t.co/D2yUPVQgjR
@dthume: 5 minutes into #qconlondon and I'm as impressed as ever by how helpful and friendly the staff are.
@timanderson: Of all the events I attend, #qconlondon has the best food*. Way to a developer's heart? *possible exception for #monkigras
@russmiles: Glad to see the simple card voting system in effect! #qconlondon https://t.co/2OqjQ9Mo3f
@igatanasov: #qconlondon ten years later but better than ever
@fintanr: The breadth of talks at #qconlondon is really impressive, lots of videos to watch post conference
@danielbryantuk: Every time at #qconlondon I think I have my schedule set, but the track hosts convince me otherwise :-) // cc @wesreisz
@ellispritchard: Last thing I expected to see at #qconlondon - amazing veg, including oxalis tuberosa & Stachys affinis! https://t.co/d6BDoQaFhq
@danielbryantuk: However you do it, share your knowledge. I'm convinced this is how the industry progresses @charleshumble #qconlondon Amen to that!
Takeaways
Luke Bond’s takeaways were:
QCon strives to appeal to a broad range of technologies and levels of developer expertise. Most of the talks reaffirmed some of my previous knowledge, but I was challenged and intrigued by the more advanced computer science talks. Overall I found it really valuable to get the "lay of the land", to see what technologies people are interested in today and using in production. It is also a good opportunity to meet like-minded people and "network".
Jeroen Gordijn’s takeaways were:
My take away of this conference is that I should read more articles with a focus on software science, instead of all the blogs concerning a specific technique.
Peter Liljenberg’s takeaways were:
Most of the interesting sessions I attended during the week were about failure, and how to handle failures. Quotes like “Failure is inevitable”, “Failure is an opportunity to learn” and the importance of building an architecture that can manage failures were common topics. Migrating from a monolithic application to a more micro service oriented architecture were also popular.
Takeaways from QCon London included:
@paulmfarrar: Sum up #qconlondon in one word? Collaborate!
@urbanisierung: Thanks @qconlondon for a great conference again! For me it was so far the most inspiring of all I have participated! #qconlondon
@kriz_davison: Thankyou #qconlondon for yet another amazing conference. As always its got me energised and broadened my horizons. But now my brain hurts!!
@herder: So, home from #qconlondon.Time to look at the videos of all the great stuff I didn't see live!
@nginx: Thanks for a great event #qconlondon. It was awesome to meet so many people interested in modern web architecture and #microservices.
@kennethmac2000: Going to a conference for 5 days in a row was hard work, mentally tough, but rewarding! #qconlondon
Conclusion
The tenth annual QCon London brought together over 1,400 attendees - including 140 speakers – who are disseminating innovation in software development projects across the enterprise.
QCon’s focus on practitioner-driven content is reflected in the fact that the program committee that selects the talks and speakers is itself comprised of technical practitioners from the software development community.
QCon London is one of a global series of events run by InfoQ's parent company, C4Media. Hot on the heels of QCon London we held QCon São Paulo on March 28th-April 1st, with QCon Beijingjust around the corner starting on April 21st. The next English language QCon is New York starting on June 13th, followed by San Francisco on November 7th.
QCon London will return on 6th-10th March 2017.