Transcript
Aslanifar: I really love this quote from Alvin Toffler. He's saying that in our ever-changing world, the illiterate are not those who cannot read or write, but those who cannot learn, and unlearn, and relearn new things. We see it all around us, especially these days, everything is about AI. Where are we headed to? Every company wants to innovate. They want to bring new technology, but at the same time, a lot of us are stretched to mass at work with new features. The question then is, when is that time that I need to set aside to learn new technology, to be fast, to be Agile, so that I can explore and innovate when the time comes. Also, I hear quite a bit that people say Agile fails, and what is post-Agile? To me Agile is a category. It's like music with rock and jazz, and so on. Agile has a lot of flavors, it has Scrum, Kanban, and whatnot. The issue with that is that some of those practices might have failed. Some of those practices have not evolved with time. It doesn't mean that Agile itself failed. Agile is just being fast and be able to move in the direction that you need to be when you hear feedback or when you receive feedback from your users, from your systems. My hope is really that by the end of this session, you can design your own Agile practices, and not really following necessarily any certain practice that does not make you faster in your team, or doesn't build the team that you want to have to be able to go back and change direction and be responsive to those feedback that you get.
I am a Principal Consultant at Equal Experts. I've been working two decades, most of my life. I've been a consultant, so I've been in many teams, seen many projects, and developed a lot of patterns that really helped me throughout my career. My label has been Procrastination Terminator, given to me about a decade ago by my team, because I love getting teams unstuck and always find a way for them to move forward. Few of my accomplishments have been doing a lot of actually migration works with absolutely no downtime, or no data loss for big e-commerce companies. I also work on safety critical applications for an electrical utility company here in California. That was a fun project. We had to actually notify the users, and that goes from household people, who their life depends on hospital equipment at home, all the way to prisons, and hospitals, if they're expecting an outage in their area. One of my favorites has been working on development treatment plans for certain epilepsy projects. A lot of those, I would say that if I had stuck with just Scrum or Kanban, or a certain practice, we wouldn't be able to go as fast as we could or iterate based on the feedback.
Stages: Following, Detaching, and Fluency
As people learn new skills, they pass through three distinct different stages, those are following, detaching, and fluency. What is that called in aikido? There are three things. It is called Shu, Ha, Ri, that is following. When you're learning something new, a practice, a tool, or anything, in the beginning you want to pick up a book, choose a mentor, get into the teams to really understand what that is about. Once you get a little bit more comfortable, you find yourself breaking some of those rules because it just doesn't fit with what you want to do. Eventually, you might just leave some of those practices and create your own. That is Agile. What I'm going to introduce is this framework that I came up with not long ago. It works for me, especially with exploratory projects, so projects that are Greenfield, you're looking for early validation. The stages are imagine, decide, execute, and assess. Within that, going from minimum viable product to minimum lovable product. That doesn't mean that this works exactly the same on every project or necessarily works for you. The goal is for you to decide what works for you.
Imagine Framework
On imagine, this is when your stakeholder or the company has a really great idea that they want early validation on. The process of how to get to that one good test or good idea is a different story. Just imagine we went already through some filtering. At this point, the idea is there, the group is there, and we want to come up with one best test. The reason for that is to focus. To focus in any new feature request that helps us to filter those out and put them in a backlog and get to them hopefully a little later. For the team, background information is very necessary, things like, where are we headed to? What problem are we solving for? Are we looking to have some profit opportunity, or anything financially? Also, can we afford this idea? How are we going to explore this idea?
To make this a little bit easier to understand, I'm going to talk about the last project that I worked on for a company called tastytrade. They are an online brokerage firm for self-directed investors that want to do their own trading. The CTO is Linwood Ma. Tastytrade already leverages a third-party service that provides them with creating alerts for their internal users, internal traders, as well as their customers. However, the alert creation is actually quite complicated, not so much for the customers, but for the in-house traders. Because what they need, they want to have very fine-grained alerts, very specific alerts. It's actually quite hard for them to create because they're not developers, they don't want to write SQL queries. Also, just like every product out on the shelf, it's very hard to bring it in the house and have it work with all the tools they have. Linwood had an idea, and his idea is, can we make a better platform for our users to easily create rules and get alerted when a certain trend or an event happens? An example of that is a user wants to be notified, for example, if Wayfair stock is down by 5%, while Amazon is not, within the past 24 hours. That's really specific, which their existing system is not alerting them without a lot of bells and whistles to create that. Wayfair and Amazon are competitors. If I'm a trader, I want to be able to get notified and do some actions.
Creating that platform alone, that's not enough. This is a financial and trading company we're talking about, so the ability to process complicated queries with very low latency and high throughput is very important, because without it, the product is a complete failure. If an alert happens, you hear about it a minute later, you miss an opportunity. Ease of use was quite very high on our list, because not only do we want the users to be very comfortable and have a great UI that they can use, their in-house traders, they want to write very complicated alerts, we had to build a system for them to be able to use it quite easily. We also wanted to architect the system in a way that later on once a proof of concept is there, they can extend it to support options, features, or any other instrument that the company wish to bring in. Another critical point that's really good to keep in mind is what is your client or your stakeholder, your manager's North Star? The North Star for us was ability to package this later on, and bring it to the public. The details of how to get to that is called noise. You don't want to introduce so much of that future North Star early on in your project, but giving your team an idea of where you're headed, it really helps the team to shape their decisions. Also, as that wasn't enough, we wanted to do this as much as we could in cloud native SaaS/PaaS. Again, for financial trading companies, they're still quite new or a little bit hesitant when it comes to working with cloud.
Decide Framework
The next step for us was to decide. This is end of day one that we talked with the CTO. The next step is to decide, is for us to understand what is the scope? How do we want to define our scope? What is our minimum viable product, as well as minimum lovable product, which I'm going to introduce later? Is the goal realistic? What challenges are we expecting? How do we want to deal with those challenges when they arise? As well as, what the structure of the team is going to look like. Because, again, this is a very explorative project, we don't want to put so many people early on on this and invest so much money, just to find out that this is not going to work out. Minimum lovable product is the hook. It goes a little bit beyond the minimum viable functional product, to something that a user is hooked, and they love, and they cannot wait to come back to it. Because if you just have a minimum viable product, but you cannot have that early validation from the user, if they want to come back to it.
With that initial team in place, at this point, it is actually me and my colleague who's also a consultant, plus a CTO and a team lead from tastytrade. We wanted to make sure we're building a system that if it's successful, the tastytrade team can take it over completely, and they can run with it. For us, it was very important that from day one, we had to plan on seamless onboarding, and documenting why. Because a few weeks in when we want to bring more people on, we want them to make sure that they understand why we are passionate about that project. What was the discussions when we started this project, so they actually get that buy-in, by the time they come to the project. A few years back, I took this workshop, it's called The Passionate Product Leadership with Jeff Patton. It's a really nice workshop. It provides you with so many tools and techniques to do your day-to-day job. To do anything from brainstorming, to come up with that great idea that you want to build, all the way to your release strategy and how to present it to the client. Before that, I was actually using a lot of these tools and techniques, but with paper, pencils, and Excel. This workshop is really great.
A few of the tools that I'm going to talk about, one of the most important one for us was the journey map. We had to understand where the user is today. What are some pains? Where are they going to be by the time we want to build the MLP, minimum lovable product, and also the North Star? How this outside customer is dealing with what's going on with the inside traders, how their journey is going to look like, as well as how the data travels. From there, we came up with everything on this map. We took all of that and put it in a grid. On the top left, you see the features that are very important for the client. Basically, without them, the product does not exist. I put a dollar sign in front of it, like four dollar sign is the most important ones. Also, out of all of those, we took the ones that are least complex to build, so fastest to get to market and get some early feedback, we put all of those in one grid. That was our minimum lovable product that we wanted to focus on. Again, tools like these, it makes sure that every idea is captured. It provides really good filtering for everybody to understand, a new feature that comes in, where should it go, when should we focus on it? Some of the key takeaways. Agile is not an excuse for lack of planning, actually a good planning and just enough planning and really great documentation is crucial to pass that knowledge on and build a team that feels as excited and passionate as the few first people who already started on that project.
Execute Framework
For us, this is day three, for the client, so we're moving really fast on this, it was the execute time. Now we had enough planning in place. We knew enough to get started. At this time, it's time for task assignment. Everything that we decided that is good for us to execute on from that top left grid, we just took it off and put it in a backlog so that we could quickly start doing the work. Development gets started during this execute time. Communicating with stakeholders, or managers, whoever is sponsoring that project is really important, because every couple weeks, every new thing that you learn, they need to be notified if you're still in the right direction. Are we making the right choices? Are the expectations still realistic or not? If it's not, let them know so they can terminate the project or shift, maybe go to the next best test that they have.
One of the fun things that we had to deal with when we started the project was balancing that exploration and exploitation. We were working with the CTO who was very excited all the time. He was just like, "Go to Rust, go to Scala, go to Go. Don't use Java if you don't want to." That's all really good, because what he cared about and what he communicated with us, just like a great stakeholder, he was like, "I do care about the last result. I want to see that one test that passes. The way you get there is up to you and the team to come up with." Eric and I, being a consultant, we wanted to make sure obviously, in a few weeks, when we were bringing more people in, they are ready to take this job. We had to go back and actually learn what the team is using. Some of their frameworks and languages of choice, so that we don't burn them out when we're about to introduce these, maybe two, three new technologies that are crucial for this project. That is what I mean by exploration and exploitation. When you are working with a team, that you're encouraging them to innovate, to explore around. You have to make sure the team doesn't burn out. Bring one tool, one technique at a time. Allow them enough time for them to become an expert or become very comfortable with that tool. Give them enough help and support. If you introduce too many things to the team without giving them enough time to actually feel the win, it's like pulling the rug from underneath their feet every time they get comfortable. That leads to a burnout.
Once we started having actually some early validation, so we're almost on the right track, we know what is going on, and we have confidence that we want to build more and explore more to get our timing corrected, it was time for us to put some of the monitoring and that safety blanket in place. We wanted to be notified that every time we build a new feature or worked on a more complicated query, if they affect our time, we wanted to be alerted immediately. This is Grafana. This was very important for us because we sometimes would run this through different news coming up and throughout the whole day just to see how timing gets affected on some parts of the day. If we build a new feature, does that affect this time or not?
Another great tool that I love using is VSM. VSMs are visualizing the flow of material and information that is needed to bring a product or a service to a customer, keeping time in mind. We chose this tool because we care about time quite a bit. This is after three, four iterations for us. When we built our modularized system, we had to make sure that visually, it helps us to put them in a place so that we, number one, see how they connect to each other. When we did round one, round two, we actually realized that some of the microservices that we put in a place, we can actually take them out and almost see even a better performance. We did that. Also, this helped us to understand not only that end-to-end overall time, how long it takes, which is important for the client and for the customer, but for us, we needed to know what's the latency within each system. For example, if system A has 1 or 2 millisecond latency, and system C has 10 millisecond latency, if we need to make something faster, our bet was to go to system C, and don't waste time on the rest of it. VSMs are a really great tool, especially if you're working with products that the timing and that latency is quite important.
Other tools and techniques that we put so that we help our team to come on board and feel confident that they can learn independently or even with us, was a lot of architecture document, from day one, and make sure that we maintain those. ADRs are one of my favorite, because you want the team to know what was that computational thinking and the arguments that went to your meetings when you chose a certain technology or a tool for them that they now have to keep up with it. Another thing is, obviously, Eric and I had quite a bit to learn, we were also learning with the team. Every great resource that we found, we would put that in some kind of documentation and put them in a format that the next engineer who comes to the team, they can pick it up and learn it.
Some of the takeaways from here, the most important one is exploration and exploitation. Make sure the team always feels encouraged, and they have the time to set aside a few days or so, just go heads-down, do their learning if they need to. Another thing is, put your safety blankets in place. If you want to be notified about any failure in the system, any change in the system, choose your tools, choose your monitoring tools very carefully, so that you have that confidence that if I do something wrong, I get notified quickly and I can backtrack and change things right on time. Lean methodology incorporates a key concept that is stop and fix. That allows that every time somebody notices a problem, they immediately stop. That's an opportunity to learn and get some feedback. What is that called in Japanese? It is called Jidoka. It's a fun concept. That's exactly what we want here. We want to make sure that at any given point in time, if we find a problem, we always go back. Don't put those in that tech debt, in the backlog because they will come back and haunt you later on. One thing that I always do, I make sure that the most important bugs in the backlog "tech debt" are always cleared up, because the new features that you build, you never know the new bugs that come in. Are they a result of the new features that you just built or they're just bugs, or they're based on some of the stuff that you just postponed because you didn't have time?
Assess Framework
Here comes the assess time. This is the minimum lovable product. By this time, we actually have onboarded a few of the tastytrade team and made sure that they're comfortable, they're coming up early on with us because we had some early validation, so we had built a minimum viable product. This is the time that you want a lot of feedback from your users, from your stakeholder, managers, or anybody else to make sure that we are building actually the right product for that user, and they can't wait to use it. Sometimes it happened in the past for me that we had to terminate the project. The good thing about it, the sooner you do, the more money and time you save, so there is no shame for that. Other times, you just need to go back to the drawing board and maybe backtrack some of the decisions that you did. If you have enough monitoring, and you have built that trust with your team and your stakeholders, hopefully those conversations are a lot easier. This is the time that excitedly we come back to basically our users. Here's Linwood, and show him what we build. He would just come back and say, actually, only if I had this one thing, that would be perfect. For us to make sure that we narrowed that one thing that hooks him up and get him excited about using this product.
Now throughout building this, a couple rules that I follow is, always go for a two-way door and not a one-way door. Number one is when you're in doubt, either flip a coin and choose something to experiment, or do two things in parallel and see which way it actually solves your problems better. For us, one example is when we were working with Kafka, and we wanted to decide Kafka cloud versus platform. Obviously, we were encouraged to use Kafka cloud because we wanted to push things to cloud as much as possible. Most of you know that platform is faster. For us, the gut feeling was that's faster, but let's just push Kafka cloud. We had to do those two things in parallel, but just keeping in mind that we have to switch between the two, or eventually choose something, or maybe don't even go with Confluent. That made it really easy for us to architect the whole platform in a way that we made it like a puzzle. Switching between the two different things or completely changing the way that we were building, was just a few days, like two, three days of code changes to take one piece out and put another one in place, but have enough data so that we can tell a good story about why we changed from one technology to another. The other thing to keep in mind here, especially early on when you don't have enough validation about what tools or product you want to use, is stay away from the multi-year, multimillion-dollar contracts. You start using a lot of free trials, ahead in the project. When the free trials are about to expire, you start getting a lot of push that, if you sign up right now within the next couple days, we will give you a year of cloud discount. The problem with that is it makes it really hard for you and your team to change later on. Only go through those contracts, if you're absolutely sure or very confident that's going to work for you.
Have premortems. Premortems are different from post-mortems. You have those early on, before you go to production or before "the disaster happens." By having these kinds of meetings and actually inviting everybody, and go around and ask them, ok, we're going to production tomorrow, or today, or however, we're going to go with this big feature, what is going to go wrong? It makes the team really think and it makes them think about, where should I put the logs? What monitoring tools do I need in place so that if something goes wrong, I get notified? What accesses do I need? Who should be in the room? That helps with the confidence. It takes that finger pointing and blame away from your team, because they have hopefully talked about a lot of those things ahead of time. They're prepared, maybe they have some practice so that if something was going to happen, they can get together and quickly debug the system. This is not going to completely prevent post-mortems. Hopefully, it's going to reduce the number of the post-mortems that you have to deal with.
Some of the takeaways. Again, premortems. Have those discussions with your team so they get that learning, they get those practice way ahead of time so they're very well prepared and coherent when things happen. Iterative development, make sure at any given point in time, you are communicating with the team, with the stakeholders. Setting expectations, if you need to take a few days off, learn a new technology or anything else that you need to deal with. Turn detours into directions, at this point. When the team comes in, and as we learn new things, several times we actually had to change the directions. We go back to the drawing board, change the technology, completely change our algorithms. When you have those discussions ahead of time and the confidence that if something goes wrong, I know that I can survive it, those really boost the confidence in the team. They're more encouraged to be exploratory and do improvements that they need to happen.
Agile Key Takeaways
Some of the Agile key takeaways. Throughout this whole project, we did adhere to Agile software development values, but we had to define our own practices. Those meant when we were only two on the project, we did not have story writing sessions, we did not have grooming sessions, we didn't have estimation, because anything that we decided, it had to be built. There was no point of estimating things that had to go there anyway. However, we did our standups. We did our demos. We constantly communicated with the team, with tastytrade team and the leads and those who needed to take this project and run with.
Another really important thing for us was that our team was actually quite broad. We had people from India, Poland, London, New York, and Chicago. We did not let the time zone to come between us when we were helping each other and onboarding the new team members. For us, we always find some time in the morning that everybody was available to help each other do pair programming and answer any questions every time a new member joined. That really helped the team. A little bit of empathy, know that 1% really needs pair programming, another person might just want some time alone. Have enough tools, technique, resources, documents so that they can read it and go experiment with stuff on their own. That's really important. Right before, we were still very early on on the project expecting and hoping that we want to bring more people on the team, with every new tool or feature that we were building, Eric and I, just every couple days, we blew our code base off and start going through the documents, and go through the READMEs, different operating systems to make sure that everything is very well documented. If a new member comes to the team, they can actually install everything within a couple hours and be ready to have their PR by midday, or latest by the end of the day. Always plan and make sure that your documentations are very comprehensive. Renew tasks for every new engineer that comes to the team. By day one, have some PR ready, approved, and push it to production to make sure that access is corrected, they have everything up and running. They can be working however they want to, whether in pairs or independently going forward.
Nail it, then scale it. Within three months, just doing those Agile practices that made us faster, we had created a minimum lovable product. That is a big accomplishment to go from one page document and an idea in somebody's head to actually have a fully minimum lovable product that people could use. By the time I stepped out, which is at the end of five months, we delivered a complete cloud-based alerting system for the clients, that their in-house traders were able to use it. They were fully staffed with their internal engineers who were equipped to take everything that we built together and all the new technologies, and they were able to even onboard more people to the team. I believe that actually, the last I heard, they're about 14, 15 people on that team, and everybody is feeling very comfortable with all the technologies. The big compliment is this quote from Linwood, that by doing so, and building a practice that really incorporated on that fast feedback, also allowed people to explore and innovate, we helped him think, and get that early validation that he was looking for.
Principles
One size does not fit all. This is not going to be working exactly from team to team. Every one of those need to be changed. You have to look at the structure of the team, the background of your team, what are they comfortable with? How are they going to learn? How are they comfortable with delivering the system? Change those practices the way that makes you faster, that makes your stakeholders or managers. Give them that early feedback and validations that they need to have. At the same time, trust and empathy goes a very long way when you're trying to build a very strong team that everybody trusts each other.
See more presentations with transcripts
