Engineering Leadership: Balancing Autonomy, Growth, and Culture with Michael Gray

Transcript

Shane Hastie: Good day folks. This is Shane Hastie for the InfoQ Engineering Culture podcast. Today I'm sitting down with Michael Gray. Michael, welcome. Thanks for taking the time to talk to us today.

Michael Gray: Thanks for having me, Shane.

Shane Hastie: My normal starting point with these conversations is, who's Michael?

Introductions [00:59]

Michael Gray: Who's Michael? That's a good question. I'm Michael. I'm a principal engineer at ClearBank. Been there for getting on for three years now, which makes me one of the veterans actually 'cause we're only seven years old, so I've nearly been there for 50% of its lifetime. Before that I worked at Vanquis Bank, also in the finance sector, and before that I was in a completely different industry, which is security surveillance. So it's quite a contrast and they're quite different industries with very different challenges.

Yes, so that's my work history. What do I enjoy? Software. I enjoy systems, thinking about the bigger picture, understanding the problems that systems have, helping to think about how we might solve some of those problems, hence why I'm in a principal engineer role. It's very much the bigger picture for ClearBank and how everything joins together. I'm pretty passionate about engineering culture, which is why we're here, I think, Shane, so we're here to discuss that a bit. Safe environment for everybody to work in, how we can create that great culture that people can excel in and I think that's a huge part of successful engineering teams and companies that we don't often talk about enough. We focus a lot on the technology and not the people and culture side.

Shane Hastie: As a principal engineer, what is your contribution to culture?

Contribution to culture [02:12]

Michael Gray: Quite a lot, I guess. So we're leaders. We're leaders in an organization which very much means that we have to lead by example. How we behave is seen by most of the engineering floor so making sure we're communicating clearly, safely, making sure people feel heard, how we behave and interact influences how others do, so very much holding ourselves to a high standard with that. We own processes, how we make decisions as an example that we've put in place and I believe that's the big contribution to culture, to make sure that the right people are making the right decisions to feel like they're empowered to make the decisions themselves when appropriate and that those kinds of things are really clear.

As a group, we do a lot of mentoring across ClearBank. If you've seen a previous one of my talks, I talk quite passionately about mentoring and I think it's something we don't do enough in technology at the moment. We've grown as a industry very rapidly over the last few years and that means we've had a lot more people coming in. They've been promoted very quickly, that maybe haven't had the support that they need to grow into the roles that maybe they are currently in. So mentoring, I think, is a huge part, especially the more senior folk in companies should be embracing more than they currently are. Yes, so a couple of flavors there I guess.

Shane Hastie: So let's dig into some of those things. The first one maybe we can talk about is decision-making. We want our engineering teams to be as autonomous as possible. We know from the motivation research that autonomy mastery purpose are things that drive us in the knowledge worker environment, but we also have to, in ClearBank's case, live within a regulated environment and many organizations do have strict constraints. How do we balance that?

Balancing Autonomy and Regulation [04:09]

Michael Gray: Yes, it is a tough problem. Maybe if I take ClearBank back in time to where it was to where we are now and how we originally dealt with it versus how we deal with it now. So when I joined nearly three years ago, we had a centralized function, which was called architecture review, and this was the senior folk in ClearBank that came together to discuss architectural decisions. ClearBank was smaller then, but even then when I joined, that wasn't scaling very well. How that was run was we had minutes and they were stored in a wiki. That was kind of the extent of where we were at that point in time. I thought this was a bit madness really because those people weren't close enough to some of the problems that we're discussing and making decisions on to really know the consequences of them. So we've shifted, in recent years, to kind of what Andrew Harmel-Law's described as the Architecture Advice Process.

I'm sure a lot of people have seen his article on ThoughtWorks and he's got his book out, which I have reviewed the first couple of chapters, but then I was a bit rubbish, so I don't think I've been asked to review the rest of them to be fair, but that's on me. But the first two chapters were great. So we transitioned to the Architecture Advice Process and that's heavily leveraging architecture decision records as the mechanism for storing decisions that are mutable, they're a point in time. We're quite strict about how well these are written, especially for, I'll come onto decision scopes, which is kind of a layer that we've put on top of this, but when you think about a regulator's perspective, yes, the architecture review led the purpose, we have the minutes and we could record the decisions, but they missed a lot of the nuance and the detail. I actually think ADR’s are serving that purpose a lot more clearly than the minutes from the meetings were. So that's great.

So another problem that we had was because we had this centralized function, people didn't feel like they could make decisions and that was when you're a growing company, that's a problem, your engineer, so I think since I joined two and a half years ago, technology and products were maybe 150 people and now we're 310, something like that. It didn't really scale very well at 150 nevermind 310. And we had a problem where people felt like they couldn't make decisions and they needed to go to the centralized, people were continuously looking upwards to somebody else to make decisions for them. So we introduced something called decision scopes, which were very simple and it's all about impact, team, domain and enterprise, so team-based decisions. What we ask everybody to ask themselves is, does this decision just impact my team? If it does, your decision to make, discuss it within your team and move forward with it. We don't need to know, you know better than we do, but please write it down in an ADR, because at least it's recorded and audited and that keeps everybody happy.

Domain. That's how we're organized at ClearBank. There may be a couple of people in a particular domain and they may be changing something to do with, I don't know, the way we process payments is slightly nuanced and may impact multiple teams, let's put it that way. Again, it's about impact, how a discussion with the people that it's going to impact in your domain and write it down. Third one is enterprise, and that's a bit more of a bigger deal. That's where we do have a centralized forum, which we call the Architecture Advisory Forum. That's where we create a quorum of everybody that we need to have to approve these decisions. That's principal engineers, that is security, data. Who else have we got in there? Basically everybody that needs to be, hey, we can kind of tick this box.

That forum serves two purposes really. One, means everybody can have an input into these enterprise decisions 'cause it's going to impact them, and I think that's really important. Before it was a closed forum. Anybody can attend Architecture Advisory Forum if they like, so they can come in, have an opinion and feel like they've made input. And I truly believe that that means that these decisions have a bigger impact because people feel like they've contributed. It was partly their decision, they've been consulted. The other side of the forum is the clue's in their name, architecture advisory. People can bring stuff where they would like opinions, should they want to. Sometimes that is an opinion from the forum, but sometimes it's just we've got that quorum of people and you can find the right people that you need to speak to, to move what you're trying to achieve forward. So it also creates that open culture.

The final piece, which I mentioned earlier is anybody can attend. What I was really pleased to see, maybe a couple of months ago as one of our directors of engineering encouraged one of our associate software engineers, so we sponsored Code First Girls. So we took on a cohort of Code First Girls in and we had, so an associate software engineer from Code First Girls who's been at ClearBank for 10 months maybe, pretty new to software in the industry, sharing and presenting in a safe environment. They felt safe to share and present a kind of enterprise decision that they wanted to be part of in that forum. And I thought that was brilliant and I thought that showed that that was safe, what people feel like they can contribute. So how was that shared? That was quite a lot, but that's a summary of the very topic.

Shane Hastie: Very clear decision scope, clarification, and you're talking about that Architectural Advice Forum. You made a point there of the safety, the ability for very junior people to step up and talk there. How do you avoid this becoming a bureaucratic rubber stamp?

Finding the balance between architectural advice and overburdensome bureaucracy [09:25]

Michael Gray: So we have had challenges with that. I spoke about it at QCon because somebody in the audience asked me this very awkward question because it's a tough problem. We had examples where we have a problem at the moment with too many services, if we're quite honest, for the number of people, that kind of microservices evolution where everybody said everything needs to be 100 lines. We suffer from a bit of that and we're trying to bring it back together. But that's quite a contentious point and that's something that we've not solved through the forum because we couldn't get everybody to agree. And I think this one depends because for that decision to make an impact, you have to get buy-in. Because it didn't get approved and agreed with everybody, we probably wouldn't have had enough buy-in anyway, so we've kind of taken that to take a different tact, more of an influence in one particular areas in teams where we think we're going to have the most impact. As a group of principal engineers, we've done that.

In other cases, there are decisions that we need to make that we're going to have to dictate, we're a bank so sometimes it's regulatory stuff that we need to do and therefore that some of the decisions that we do just have to make. That's maybe a little bit of a challenge we've got now, it's not super clear. Normally that ends up we have to go there and say, "We need to do this because of X. This is for information and not for debate". To be fair, when we have done that, people do still input and actually suggest improvements quite often when that has happened, but it's maybe a bit of a gray area and one of the challenges with this way of working, I still think the benefits outweigh that.

Shane Hastie: Yes, thank you for that. As you say, these are sometimes gray and difficult areas that are not easy to figure out, and I think our audience will appreciate the openness there that hey, we haven't really got it all working perfectly, but it's better than it could be. So thank you for that. Another thing that we were chatting about before we started recording was the pressure for efficiency. What's happening in the industry today? So stepping outside of ClearBank but looking at the bigger picture.

Industry pressure on efficiency compromising quality and continuous improvement [11:31]

Michael Gray: So there's a general push in the industry, it's all about do more with less. I think that's the phrase a lot of people are saying. And you see the big tech companies, they're making layoffs and their share prices, they're going up, okay? But I think they were in a different position to most people 'cause they all purposefully over hired because they wanted to stop their competitors having the talent in the market. That's why they also paid so much.

But the do more with less, generally, is feeling like a squeeze on teams and it's all about value, efficiency. There's a pressure that people are becoming feature factories, and from my perspective, what that's doing in the industry is squeezing teams, which means they don't have space to continuously improve. It's all about the next feature, the next feature, the next feature. And I think that's pretty negative to be honest with you, and I think we're going to feel the pain of that in maybe three or four years time from now where we've stopped continuously improving because maybe got investors that need to see that we're being super efficient, that kind of thing.

Yes, and we've removed the space from people to improve or just refactor a bit of code or update a little bit of documentation or put that call in about the process that feels a little bit broken that they want to do something. While we've got this pressure, I just think it's having a negative impact in the longer term.

Shane Hastie: So how do we resolve it? How do we want to say push back? How do we challenge this?

Michael Gray: Yes, it's interesting. I think it depends. If I talk about, in a regulated industry, there's plenty of frameworks that you can use to your advantage. There's risk frameworks, if there is risks and you're regulated and it could cause detriment or maybe something that's reportable to a regulator, that's definitely something you can use to your advantage. Make it really clear why certain things need to be done. Another side is you leverage the processes that you've got that maybe you felt like you didn't need to use in the past. The other thing is I think leadership has a big role to play in this and that's leading by example. If it looks like, we, as leaders are absolutely flat out and don't have time to do anything, then they're going to feel like they should be operating in the same kind of way. So maybe even if you are completely flat out, make sure it's not perceived like you are to the rest of the engineering floor. Make sure you're preaching that culture of continuous improvement and encouraging it.

There's certainly things as leadership we need to make sure we're doing, influencing the right stakeholders, influencing senior people, the C-suite potentially in the organization to make sure they understand the value of this kind of continuous improvement mindset. I think about ClearBank, ClearBank's got to where it's today because of the continuous improvement mindset and it's really important we don't lose it. We've created space to create a really solid technology platform. Why do customers come to us? Because we never go down and we have a really solid technology platform compared to our competitors. So that's because we've continuously improved it and not needed to ask for permission to do so. So those are the few things I think we can do, but we do also have to acknowledge the macroeconomic climate is what it is, that sometimes we're going to have to make compromises on idealisms and pick our battles and make sure we're improving the bits that we absolutely think we need to versus the bits that are going to have less impact.

Shane Hastie: You made the point about ClearBank growing from, you said about 150 people when you joined nearly three years ago to over 300 today. What's the impact of that sort of growth and how do you maintain the culture that you want through this growth?

Maintaining culture during rapid growth [15:16]

Michael Gray: Yes, in my QCon talk, I talked about power boats and alignment. That was my metaphor versus like an oil tanker ship. Really it was in reference to incumbent banks versus ClearBank and why we had an advantage against them and how we try and avoid becoming an oil tanker and moving slowly. For us, part of it's about clear boundaries and ownership, what people own and therefore can make decisions on and make changes to. So we try and make sure that's really clear. Interactions between those boundaries is something we continuously focus on. The more awkward interactions there are between those boundaries, the less autonomy teams can have to move quickly and make good decisions, so that's certainly an area principal engineers focus on 'cause we get that wider picture of the organization.

To maintain the culture, the answer is it's more effort. You have to put work into it. When everybody's in an office together, so we've been through COVID, a lot of us are actually remote now versus in an office together. We have to put more, yes, more effort into making sure we're continuously learning, creating the environment. So one of the things we have is (Sophie Western is doing it, among a few of the people at ClearBank is brilliant, to be honest) it's like we have our own, really, it's our own internal meetup that happens every Friday. It's called Tech It Easy and again, similar to Architecture Advisory Forum, anyone can come in, anyone can talk, whether it's for five minutes or 10 minutes or anything. And it doesn't have to be on tech, it doesn't have to be on ClearBank. That's like, again, a safe space that's been created that anybody can go to and spend some time to share and discuss something that they're interested in.

Maintaining that culture. We talked about the Architecture Advisory Process. I think that's about maintaining the decisions as close to the people that know the decisions and impacts best. And leading by example, I spoke about that earlier as well.

Shane Hastie: How do we grow our engineering talent within organizations, but as an industry?

Growing talent as an industry [17:16]

Michael Gray: Earlier we talked about the impact and the current economy whereby it's do more with less and previously it was grow, grow, grow. So I think we're in an interesting situation at the moment where we've grown, grown, grown, and we've got a lot of people that are fairly inexperienced in the industry if we're honest, because they've probably been in the industry five, eight years maybe. So I think there's a lot of missing skills, at the moment, yes. So what do I need to do to grow that? I think, I talked about mentoring earlier, 100% we need to be doing more of that, especially from the more experienced people. I've always found mentoring to actually be quite a high-impact thing to be doing because if you think about it while you're growing somebody, that somebody tends to be in another team. If you grow them, they're in a team with other people, they're likely going to be sharing their learnings with that team that they are in.

Selfishly, I've also found that quite a good way to influence and have impact in an organization, upscale that person, suddenly how they test and change the software becomes better as a team. It makes them ask the right questions, so mentoring, I think, is, while some people may perceive it is it's like a one-on-one thing, I think it has a much wider impact than just the individual. We've talked about creating safe spaces to grow and learn, definitely. Yes, so safe spaces to grow and learn. It's that theme of psychological safety, I think, has been a theme through what I've been saying in this podcast so far.

Shane Hastie: So you've been in the industry a while. You have got into the role of a principal engineer. What advice would you have for yourself, five, six, seven years ago?

Advice for aspiring principal engineers [18:59]

Michael Gray: I did a talk on this actually, which is roughly a theme of me with my head in my hands in frustration versus how I've learned to navigate people and organizations over time. I think my biggest piece of advice would be someone who has been through, and you said this to me, not everybody sees the world the same way as you. I was fairly set in my ways, and I think that's quite true of a lot of engineers. You have your perspective, it can be quite black and white. When you start realizing there's a bigger world out there, and it's not just about technology, it's about where the company want to invest, where the company want to grow, your customers. It's not just about technical perfection. There's so many different inputs into the decisions that we need to make. And trying to become aware of those earlier in your career and sooner, I think, makes you a much more rounded individual and much more valuable to your company.

And another thing I talked about in that was about timing and when to challenge things. Earlier in my career, I would challenge everything all the time. That's wrong. "I've been to a conference, we should be doing this". "Well, this context is important". "Why? Why are we doing it this way?" The other thing is if I challenge it now, are people maybe at 100%, are they likely to hear me? Is it going to make an impact if I challenge it at this point in time? Is there a pause in a project? Well, it means we're going to have a retrospective and you can ask a question rather than saying, "I think this is wrong, why don't we try ...?" So there's just different ways of introducing ideas a little more softly rather than being set in ways, telling people stuff's wrong, that kind of thing. It doesn't have an impact to believe it or not, and you lose a lot of your influence and just become a bit of a white noise. Definitely that.

Being more patient is another one. I think that what I've just described there is a symptom of not being very patient and not accepting that change takes time because it does. So you need to set your horizons a little bit longer than tomorrow. Yes, I think that's the first summary of the biggest piece of feedback I would've given myself.

Shane Hastie: Mike, some really good advice. Advice to your younger self, advice for our audience, a whole lot of stuff to think about here. If people do want to continue the conversation, where do they find you?

Michael Gray: So you can find me on LinkedIn, also Twitter or X, as it's now called, @mikegraycodes. And I think you can find me on ... well, you can find me on Mastodon as well, I'm just @mikegray. The @mikegraycodes, I don't actually write that much code anymore as a principal engineer, I do a bit but not that much, so I've dropped the codes part of the handle.

Shane Hastie: Thank you so much for taking the time to talk to us today, Mike.

Michael Gray: No problem. Thanks for having me, Shane.

Mentioned:

• Michael Gray on LinkedIn and X 
• Andrew Harmel-Law - Architecture Advice Process
• Code First Girls
• QCon Talk - The Journey of ClearBank From Start-Up To Scale-Up