In this podcast recorded at QCon London 2019, Shane Hastie, Lead Editor for Culture & Methods, spoke to Sarah Wells, Technical Director for Operations and Reliability at the Financial Times about their adoption of DevOps.
Key Takeaways
- Adopting DevOps is both a technology and a very significant culture change
- It’s a big change for most developers to be operating the software they build and if you haven’t done it before, it’s terrifying
- A safe culture means not looking for blame but focusing on how to fix and how to prevent things that do go wrong
- The need for a really good relationship between technical teams and product people in order to explain the benefits of investing in technology improvements vs new features
- Do everything you can to reduce the need for coordination with any external teams, because that slows you down
Subscribe on:
Show Notes
- 00:31 Introductions
- 01:45 An organisation full of smart people, not moving as fast as they could have done
- 02:00 Adopting a DevOps culture, starting with automation
- 02:10 The goal to provision in minutes rather than days
- 02:18 They stopped tracking the improvement once provisioning times were down to minutes
- 02:34 Looking for other opportunities to improve flow
- 02:39 Adopting microservices architecture
- 02:47 The focus on being able to release change regularly, moving from 12 releases per year to many thousands of releases every year
- 03:04 This is not just a technology change, it is a big culture change
- 03:07 To move fast people need to be able to make decisions themselves without waiting for others
- 03:44 It’s a big change for most developers to be operating the software they build and if you haven’t done it before, it’s terrifying
- 04:10 Examples of some of the production problems the teams have dealt with
- 04:47 Microservices are complex and distributed and can interact in ways you don’t predict
- 05:04 Safety has to come from the top – no blame
- 05:25 Reviews that focus on what can be improved, not who to blame
- 05:49 If you drop a database in production as a developer, that is not your fault. The fault is that it’s too easy to drop a database
- 06:24 The mindset of experimentation – it’s not an experiment of there’s no hypothesis and it can’t fail (quoting Linda Rising)
- 06:35 Most organisations say they are experimenting but they are really just trying
- 06:45 The sunk-cost fallacy
- 06:56 Experiments need to be quick and cheap
- 07:02 How experimentation is built into the FT website
- 07:32 Describing one experiment and how the data was used
- 08:25 Making the “blast radius” small
- 09:10 Microservices encourage decoupled design because the boundaries are very clear
- 09:49 Buy – don’t build
- 10:15 Using value chain mapping to identify the core business and focus on innovating in that area
- 10:33 Moving from home-build cluster orchestration to Kubernetes
- 11:25 The need for a really good relationship between technical teams and product people in order to explain the benefits of investing in technology improvements vs new features
- 11:38 The flip side of moving fast is that sometimes the decision you make is wrong, and that needs to be OK
- 11:54 Decisions fall into two categories – one-way door or two-way door
- 12:10 Most decisions are easily reversible, like a two-way door
- 12:18 Decide – commit – revisit
- 12:40 Advances in technology have made it easy to make reversable decisions
- 12:50 Comparing early hack-days with what is able to be achieved today
- 14:15 Advice for other organisations
- 14:23 Migration to microservices by carving off small pieces
- 14:33 You need an automated release pipeline
- 14:46 Architect for zero-downtime deployments
- 15:27 Do everything you can to reduce the need for coordination with any external teams, because that slows you down
- 14:42 Find a way to get architectural guidance, but don’t impose architectural signoff
- 15:49 Change approval boards slow you down but do not increase the likelihood of successful change
- 16:17 Architects need to become part of the delivery teams rather than working in a silo
- 16:25 At FT they have moved to Principal Engineer roles rather than architects –
- 16:47 The need for T-shaped skills across the whole team
- 17:20 The impact on career planning, moving away from a ladder towards a map that covers many areas at different levels of competency
- 18:28 The value of a learning culture
- 18:43 The value of a very diverse team
Mentioned:
