Leveraging AI Platforms to Improve Developer Experience – From Personal Hackathon to AI at Scale

Transcript

Shane Hastie: Good day, folks. This is Shane Hastie for the InfoQ Engineering Culture Podcast. Today I'm sitting down across 12 times zones with Lekan Elesin. Lekan, welcome. Thanks for taking the time to talk to us today.

Olalekan Elesin: Thanks a lot, Shane. Thank you for having me here. To the audience, my name is Lekan Elesin. I think Shane has done a very good job to pronounce the words quite well, to be honest.

It's not an easy name to pronounce, I'm from the, let's say the western part of Nigeria, but I stay in Germany. Been in Germany for about seven years. I'm happy to be here.

Shane Hastie: Thank you. So you're from the western part of Nigeria, you're in Germany, that's quite a long journey in many different ways. Tell us about how you got there, what's your background?

Introductions [01:14]

Olalekan Elesin: So interesting one. So my background is in computer science. I studied computer science as bachelor. So what happened, I think in 2013, I was... In Nigeria, when you graduated from university, you have one year to serve the country, and before my time, I was listening to Bloomberg News, and there was this story called Big Data. This was 2012, 2013, and it was called the sexiest job on the planet back then. And I felt like doing statistics, the odds of being a very, very good software, let's say programmer was very slim. So I decided to journey into the path of what most people were not aware of back then, which is big data, and that's how it started.

So computer science graduates turn out to learn how to, back then, a lot of O'Reilly, a lot of InfoQ summits as well, a lot of watching a lot of tutorials on InfoQ as well. My first job was as a software engineer, PHP, but then I learned scalar, also with tutorials from InfoQ, to be honest, as far back as 2013. And from software engineering, I switched to data engineering, and then had an opportunity to work in telecoms, e-commerce, and then education technology.

In 2017, I had an opportunity to move to Germany in an online marketplace, and after three years on the job, I wanted to get a new opportunity, so I decided to opt for product management. So I was data and software engineer, data engineer, product manager, technical product manager. And when I was the technical product manager for an AI platform at the company back then, at some point, there was a decision to reshape the company, or restructure the company, and there was no need for my position anymore.

So I spoke to my then head of engineering, saying, "I created a lot of tutorials internally for the data scientists to use, so is it possible for me to, let's say, exclude the content that is really, really tailored for the company, and then publish this information externally?" Because it could be valuable for the developer community outside of the company back then. And he gave me the go ahead, so that was how I started writing about my experience using AWS services for machine learning.

And then at some point, someone reached out to me from AWS saying, "Wow, you've written a lot. People are appreciating what you're doing. Are you interested in the AWS Hero program?" So that was how I got into the AWS Hero program. At that point in time, it was also in the middle of the pandemic, and I was getting tired of product management, and I had another opportunity to move back into technical leadership. No offense to all product managers out there, it was just not my thing, I tried it for about nine months. I had an opportunity to lead a data platform in my current company, HRS Group, which is in business travel. And ever since then, I grew from technical lead, to head of, to director, and then leading an entire business unit along with my product management counterpart. So that's the journey so far.

Shane Hastie: An interesting pathway. Machine learning, product platforms. At the recent Dev Summit, you spoke about leveraging AI platforms for developer experience.

Olalekan Elesin: Yes.

Shane Hastie: Let dig into that.

Personal hackathon - leveraging AI [04:16]

Olalekan Elesin: Definitely. So the talk, it was a very interesting one, because even though I lead multiple teams, at least 5 or 6 teams, software engineering teams, including data and AI team, I still find myself writing code once in a while, on my weekends, trying to keep up-to-date with what is going on, the new technology, just making sure that I'm up-to-date, so I can have productive discussions with the team leads with me, and also with the engineers. And on a Saturday, while watching... For the New Zealanders, you might not like this, I was watching the football, football in the English Premier League, and I wanted to build something, but I wanted to build something that was new to me, and at the same time, I wanted to leverage, let's say generative AI tools available.

So I launched my VS Code, and then I remembered I had Amazon Q Developer. So from that, I asked it to create the application from scratch, and I was quite surprised. From that perspective, it made me realize what would've taken me maybe 4 or 5 hours, going into two days, because I also have the toddler to run after, took me about 30 minutes while watching a game of football at the same time.

But at that point, I realized this could really elevate developer experience, because as much as possible, we like to be creative as engineers, and also as engineering leaders, we want our team members to be creative, and at the same time, to make sure that we get value to the customers as quickly as possible. So this was where the idea behind the talk started from. So for me, it was quite an interesting experience to see that I could watch football on the weekend, at the same time, build an application that I wanted to try out. So this is where it started from.

Shane Hastie: I recall for myself the first time using some of these generative AI tools to do some "real work". There was almost a feeling of, "ooh - I feel like I cheated".

Olalekan Elesin: Exactly. Exactly. It happened to me as well. This one was quite funny, so I cannot remember the actual workflow itself that I was trying to build, or the actual application, but I remember it helped me to write at least 200 lines of cloud formation code to deploy the infrastructure. And I felt like, "I don't want to think about this". So I put it to Q Developer, and said, "Please write this. It should be able to connect to X, Y, Z, A, B, C".

And then it wrote it out, and I felt like, "Oh my, this is cool". And I'm lazy. And I remember smiling, and my partner is in the house, she was like, "Are you sure what you're doing is correct?" I'm like, "Yes, it's working". To your point, it felt like I'm lazy. I'm not sure who said that, that the best engineers are the lazy ones, because they find ways to make it work at scale, with minimal effort possible.

Shane Hastie: So from that personal hackathon almost, experiencing it from the first time, what's involved in bringing this into actually doing work?

Shifting from personal hackathon to scaling within teams [07:17]

Olalekan Elesin: Good point. So for me, it was a hackathon, but then to scale it within my teams itself, it's a different experience entirely. Because on one side, we hear people granting interviews, saying, "There will be no developers in the next five years". On the other hand, we see that this can really be a collaborative effort between engineers and artificial intelligence, and it has a change impact on people.

And one of the stories I heard when it comes to change was, a couple of years back, there was the compact disk. Even before that, there was the video cassette or whatever it's called. And I remember having this, and so many people built businesses around renting video cassettes, even where I come from in Nigeria. And all of a sudden, things moved from video cassettes... Even we had people manufacturing spinners or rewinders for these tapes. All of a sudden, we had compact disks, and now we had to have compact disks. That meant that there was a lot of change. A lot of things would've happened to people that built their businesses around that. Not necessarily thinking about the digital content, but really about the hardware.

And for me, that was a light bulb moment when I am speaking with my teams to make them understand, "You have a concern with regards to this particular technology, but think about it when it works in collaboration with you. Then you see that you're more productive, and look at it also..."

For us at HRS, look at it from taking the customer view first. "Look at it from how fast do you want to get features into the hands of the customer to validate every assumption you have based on the line of code you're writing. And if you have one assistant sitting next to you as a pair programmer, enabling you to do creative work, and also helping you to write the code, why not leverage that?" And for me, the hackathon is, for me, scaling it across multiple people, and working through that change mindset is one of the biggest concerns that I think a lot of people don't talk about.

Shane Hastie: So what's involved? How do we make that work?

Olalekan Elesin: So this is an interesting question, and it's difficult to answer. But my view, because we also started scaling it with my teams right now, and also across the engineering teams at the company, is to educate people about the change as much as possible, and get them to pilot it. So what we're doing right now is getting a few people that we believe are enthusiastic, so the scientific approach. Believe that if you have a few people that can use this to validate the assumptions, that it'll help them become more... It would elevate their developer experience, or their experience when doing development.

And after validation, doing a lot of trial and error, a lot of expectation... I don't want to use the word expectation management, but there's no better word coming to me right now, because there's this assumption that AI can write all the code. You have to be aware that when you use it, it will make mistakes like every person when doing things initially, and you have to correct it along the way. And that expectation management, and get people to experiment as much as possible, and as quickly as possible, this is, for me, what is important.

A handful of people, get them to try it, get them to set the expectations, and use those people based on their learnings as multipliers across the organization. Not trying to do the big bang, saying, "Everybody, now go use..." Whatever it is called. But we have a handful of people, we have learning experiences from them, and then those individuals are part of different teams where they can then share their knowledge and their experiences with the team members, and also scale that across the organization.

Shane Hastie: As a software engineer who's defined themselves by writing the code, you're shifting from coder to editor.

Olalekan Elesin: Yes.

Shane Hastie: That's quite a big shift.

Engineers shifting from coder to editor [11:07]

Olalekan Elesin: Definitely, it is. It is. So I remember, my... I think it was my second year in computer science in bachelor's, that we were introduced to assembly language, or something like that. And we got to the final year, someone was telling me I had to write C#, and I'm trying to bring the relationship between what I learned in second year to what I'm being taught in the final year. And even in the final year, I remember we had to write Java, and more and more, I realized that as computer science has evolved with the years, that people have been building abstractions, let's say low-level languages.

And this is what is happening even from my perspective also, that there are abstractions. And at this point, the abstraction we are getting into is where everyone, as much as possible, that has an understanding of how to build stuff, that has imaginative capabilities, can leverage these tools to become editors, and not necessarily trying to write the code from scratch. For me, this is it. It's, we build a craft, we like to write the code, but then we need to build another craft, which is becoming editors, and letting something that can do it better, to a certain extent, to write the code, where we guide it across how we wanted to write the code.

Shane Hastie: You did make the point that these tools make mistakes. They make them very confidently.

Olalekan Elesin: Because they don't know better.

Shane Hastie: Yes. This implies that that problem solving, mistake finding is another, not necessarily new, but a skill we have to build more strength in.

The tools will make mistakes [12:43]

Olalekan Elesin: Yes, of course. Yes, of course. And while you were asking the question, I remember one of the first production errors I made while being a junior engineer, this was when I was in the telecoms industry doing my first job, we had production on-premise data servers, and we wanted to analyze the log files. And I had done a lot of tutorials about Hadoop, and I didn't know that I needed to move the data from where it is, which was on the production server, into another server, or another virtual machine which was separated from the production environment. So what I did was to install Hadoop Hive on the production server that was serving millions of transactions, and we had a major incident.

It didn't crash the business, but it was quite an interesting one, but the learning there was that I could make a mistake as a junior engineer. And this is what our folks out there have to realize, that these tools are still learning. And another key learning I took here from that experience I had was one of the senior engineers explaining to me how not to repeat that mistake next time, and how to solve the problem next time, and guiding me through the understanding of, this is what happened, this is how to solve it, this is how to make sure that it doesn't happen all over again.

And this is what we see with these tools, where... There are some techniques called retrieval-augmented generation, where you can put your data somewhere so that it doesn't go off tangent with regards to the recommendations it comes with. And there are some mechanisms where you can also fine-tune an existing model, which is helping it to understand better the context. But until that time where it is really affordable for us, or everyone, to train or fine-tune their own model, these things will make mistakes, and also, we should expect that. I think, for me, this is the biggest part of the expectation management that I do with the engineers that I work with on a daily basis, that it's expected to make mistakes. You have to be comfortable to see that it can make mistake, and be willing enough to take the time to edit what it's generating for you.

Shane Hastie: When that mistake is subtle, so it's not going to cause a syntax error, it's not going to prevent compilation, but it's one of those, and we've had them for a long, long time, requirements bugs effectively, where, "Oh, it's solving the wrong problem". What's the skillset that we need to develop as humans to find those mistakes?

Building the skillset to ask the right questions [15:11]

Olalekan Elesin: Excellent question. I think the skillset we need to develop as humans is asking the right questions. So when I interview candidates as well, when hiring for my teams, we usually ask them, "How do you deal with requirements that are not clear, or problem statements that are not clear?" And this for me is one of the things that differentiate engineers from developers. Engineers always try to understand the problem, developers think about code.

And this is why, for me personally right now, I'm not seeing this coding assistance as engineers yet. They might evolve to that level, but they are more development assistance. And the reason why I take it in the direction to answer this question is that being able to ask the right question, and this is what people call prompt engineering, is one of the skills that we need to add to what we do. If you have engineers that know how to ask the right questions, they would get a lot more productive by using these tools, because they can then ask it to refine, to change a bit, to adjust what the context of how they're asking the question to get better responses, that they can then edit from these tools. At least this is what I do. So the skill is prompt engineering, but the underlying thinking behind it is being able to ask the right questions.

Shane Hastie: We hear horror stories of security concerns around these tools. How do we make them secure for our own organizations?

Tackling the potential security risks [16:42]

Olalekan Elesin: This is a very tricky one. First, I think at the QCon developer conference also someone asked a similar question. And my first answer is that security is not there to restrain our space, it's to enable us to do the right thing. And first and foremost is to make sure that if in any tool that anyone is using in an enterprise setting, please ensure that it is aligned with your security objectives or security policies within the company. This is the first thing.

The second is that we need colleagues to think in terms of the customer, and I think in terms of the business itself. Customers want to have more and more control with regards to their data, so if I, as an engineer, I am responsible towards my customer, and I as an engineering team leader, person leading multiple engineering teams, we have a responsibility towards our customer, and my customer says, "I want my data not to be shared with some AI tool". Simply don't do it. Also because it has implications on the reputation of the business if that is not done right. So that responsibility, we all need to live with it, and understand that it has implications on the customer, has implications on the business, also has implications on how we do our jobs by ourselves. For me, this is it.

And I remember when I was building the data platform, and then we said, "We want to be security first". After that, "No, but this is how to do data responsibly". It's the same for me, it's security first, everything else comes after. As long as it's satisfying the customer objective with regards to security, wake me up in the middle of the night... This is what I tell anybody, "Security first". And fortunately, I'm also in Germany, so we're super cautious about data security.

Shane Hastie: What are the, I want to say future risks, considering developer education, for instance? You learned assembler, are we going to teach new developers assembler, or are we abstracting so far away that we almost don't know what's happening underneath?

Developer education needs to evolve [18:47]

Olalekan Elesin: Education definitely needs to evolve to make sure it's relevant for the generation that we're educating. That said, anyone being educated needs to be curious with regards to, how did we get here? And that curiosity is another trait of really good engineers. So example, one of my own personal experiences was when I started using Apache Spark for big data processing. I remember going through the Spark GitHub repository just to understand how the people that contributed to that amazing project wrote the code. And this was just being curious.

I also remember going through some parts of Hadoop at some point, some parts of other libraries that I used at some point. And I think that irrespective of the fact that we need to educate people with what is needed right now, those engineers that will be chief engineering editor, or however... Editor engineer, however the title might look like, need to be curious to understand, how did we get here?

When I mentioned that I stumbled into product management, my responsibility back then was... Even though I was building an AI platform as a technical product manager, I needed to understand how this AI platform will enabled the company to have a competitive edge. And I remember something on the book by... I've forgotten his name, his last name is Porter, that wrote Competitive Strategy. And this was a book written, I'm not sure, 1990 something, just to study, to really now understand how people thought about competitive strategy a while back. And what I'm saying is, engineers need to be curious with regards to understanding, how did we get to where AI is enabling us to write code and solve problems faster than what people 20 years ago, five years ago, 10 years ago could do?

Shane Hastie: How do we train that curiosity?

How do we train for curiosity? [20:37]

Olalekan Elesin: Oh my God, that's a really... To be honest, this is a question I ask myself every day. The only way that I've seen not to lose it is, we need to find a way to not lose the baby part of our brains, the toddler part of our brains. Where I see my toddler, and he's asking me, "Daddy, why?" And sometimes I want to answer him, sometimes I'm like, "Maybe I need to help him understand the reason why". And that curiosity, it's a trait, difficult to train, but it can be coached.

So in some sessions when I'm having, let's say one-on-ones with the engineering team leads that work with me, or the skip levels, and they make a statement about something that... A problem that they're struggling with, the first thing I ask is why. And when I have that discussion, the next thing I do is, Iinform them, "I want you to always have this type of approach to problems". And so asking why, so that you get as close as possible to the first principles, using the first principle thinking. I'm not sure it's easy to train, but I know, for me, what has worked is being able to coach it with the people I have direct interactions with.

Shane Hastie: How do we help engineers be coachable?

Helping engineers become coachable [21:52]

Olalekan Elesin: Oh my. There is this book called Billion Dollar Coach written about Bill Campbell, and I remember... I think I stumbled on it two years ago, I think he passed on maybe about three years ago or so. I remember stumbling on the book, and I cannot remember in detail about his approach to coaching people and making people coachable, but what has worked for me, and one of the learnings that I've made mine is really explaining the why, and to help the engineers understand how I also think, and making them understand that there is a world before now. And the world before now to get into that means to think in terms of first principles. So what's the minimum basic element of making this work? And really getting them to really, as much as possible, go back to rethinking, because that's also the foundation of solving the problem.

Another way I differentiate between engineers and developers when I'm interviewing is, does this person really understand the business problem, or the customer problem they're solving? And this is how I look out for curiosity in engineers again. And if I don't see it, I'm happy to share feedback, "I need you to understand the business problem. I need you to understand the customer problem". And then if I'm talking to skip levels, it's the week from now, "Send me your detailed understanding of what you see as the problem itself that you're solving, not the task that you've been assigned". And as leaders, we need to constantly push our engineers in that direction to really realize, "What problem am I trying to solve? Not the lines of code that I need to write".

Shane Hastie: Lekan, a lot of deep thinking and good ideas here. If people want to continue the conversation, where can they find you?

Olalekan Elesin: They can find me on LinkedIn. It's only one, but my name is longer on LinkedIn, so it's Olalekan Fuad Elesin, and I respond to messages. I have put an SLA on it maybe in three days, but as quickly as possible, in less than three days I respond to the messages. I'm also not as active on X, but I'm very active on LinkedIn, and this is where... I use my LinkedIn to share my thoughts and also engage as much as possible.

Shane Hastie: Thank you so much for taking the time to talk to us today.

Olalekan Elesin: Thanks a lot, Shane. Thanks a lot for having me. Super interesting discussion.

Mentioned:

• Book: Competitive Strategy (Michael Porter)
• Book: Trillion Dollar Coach
• Olalekan Elesin on LinkedIn