In 2013, J.P. Morgan’s Global Core Processing Technology group embarked on a journey using the Large-Scale Scrum framework (LeSS) to change their organisational design and improve their agility.
Craig Larman and Matt Winn’s LeSS @ J.P. Morgan describes many of the transformations that have already occurred.
The Securities group in Singapore were the first to undergo the changes necessary to start using Scrum and the LeSS framework and at the Agile Singapore Conference 2013 , Matt Winn presented the topic “The Pursuit of Happiness; A Journey towards Large-Scale Scrum”.
InfoQ interviewed Matt Winn to discover his personal perspective of how the beginnings of the journey transpired as well as sharing some of the experiences and lessons learned in the Securities group, so far.
InfoQ: Hi, Matt. Could you give me some background information about yourself?
Matt: Sure. I started in software development working on SCADA systems for Scottish Hydro-Electric, which was a GE company called SCADA Systems. That was great because I got a chance to work with very skilled engineers and there was a very good feedback loop when making changes. I then worked in computer communications and computer-telephony integration. I joined the financial sector in 2004, which was not long after the IT bubble had burst and a lot of small software firms in the UK went under; they lost their funding, off the back of the NASDAQ collapse, globally. J.P. Morgan was hiring and they were building a European Technology Centre, quite aggressively, so they were a very big employer in Scotland. I started as a C++ developer working on dividend systems and kind of quickly moved through the ranks to be the team lead, running dev teams and ultimately looking after multiple platforms. I moved to Singapore in 2010 and there was an opportunity to build out a Singapore development centre in the Securities space, from scratch. That really interested me because I loved the aspect of the job of identifying top talents and building out teams and I’ve kind of thrived. I built the teams from just myself and a couple of others from our Glasgow office to over 40 people now. I’ve always found something to keep me interested. In the last 12 months the Agile transformation that we’ve undertaken, ever since Craig [Larman] came in, has been one of the most reinvigorating parts of my career.
InfoQ: You mentioned that happening last year. Before those changes started to happen, did your management style change when you came to Asia?
Matt: I think it has to because you have different cultures, different regions and different sites. So you have to adapt your management style accordingly but it’s probably not as big as the management style changes when you go from having ten people to having to manage a big organisation. So now I look after about a hundred people and once you go over a certain number of people you get to the point where you don’t know everybody by name. You have to change your style from knowing everybody’s strengths and development opportunities to more a kind of a discovery and working with people in a more fluid fashion.
InfoQ: You mentioned the transformation using Lean and Agile started last year. Did you have any concerns before that started?
Matt: Yes, speaking truthfully. Yes, because it felt like we were going to make quite a fundamental change and when there’s a fundamental change sometimes it might mean that certain roles are no longer relevant. I think the great thing about J.P. Morgan is that, if you’re talented and you’re doing a good job, you may not have role security but you’ll have job security. There will always be something where a person with energy and drive can make a difference, so I felt some comfort in that.
InfoQ: And did you sense or know about other’s similar concerns?
Matt: Yes, one of the things we found out quite quickly was we were going to move to a flatter organisation, which meant some people who were in functional roles or level one management roles were quite nervous about what was going to happen to their job. So there were definitely people who were nervous about the change and what was going to happen next.
InfoQ: Were there rumours that things were going to change fundamentally or was it explained to you from someone at a higher level?
Matt: The communication was pretty good and pretty quick, so I think Craig was in with Simon Cooper’s management team in March. Very shortly after, our senior manager, Mike Goldverg, did a tour of his global sites and held town halls and explained to everybody what was going to change and why. There were no threats to people’s jobs, it was more that there might be changes in people’s roles and the way they received work or communicated may be different as a result.
InfoQ: After those town halls, how did you get started with educating people and moving into action?
Matt: We were really lucky because we had Craig come to Singapore for two weeks shortly after Mike’s visit and that really kick-started the Singapore development centre. Craig essentially started out by running a CSM [Certified Scrum Master course], a special flavour of CSM - because it was Craig. After that we did a mini-sprint, where he demonstrated the ways of working with Scrum. Although we knew about Scrum and XP practices, we had some fundamental misunderstandings of how to get most value out of Scrum.
InfoQ: How did you form teams? Did you have to form new teams?
Matt: Yes, it was a self-organise approach and we literally got everybody in the Singapore group into a room, this was Mike and I, and we gave them some guidance. The guidance was pretty simple. It was that we wanted teams to be a certain size, so seven plus or minus two. We want the teams to have a blend of skills, so they will be able to carry out work end-to-end on a product. Then we left the room and let the teams self-organise. So, in the room, we had people that previously had been testers, BAs, developers, information architects and various other role types and within two hours we had five Feature Teams.
InfoQ: Do those teams essentially still exist or have there been significant changes since then?
Matt: They pretty much still exist and the teams have bonded very well as units in general. There were some storming issues with some of the teams, but eight months down the line, we still have the same teams and approximately the same size.
InfoQ: What about Scrum Masters; was it difficult to find Scrum Masters?
Matt: It was. We actually made some mistakes in this space that we came to learn from. We were advised to hire full-time Scrum Masters, but we actually thought we would get the most out of everybody by having Scrum Masters be part-time and do tasks for the team as well as be masters of Scrum. We found out quite quickly that the teams were not as effective as they could be without full -time Scrum Masters. I think we found out in two or three months.
InfoQ: Did you do anything about that?
Matt: We did, we brought in an Agile consultant to help us select full-time Scrum Masters and the teams selected them.
InfoQ: Was there anything interesting about that selection?
Matt: It was interesting because we had one particularly experienced and charismatic individual. I think three of the teams selected him and again we were a bit nervous about that because advice has been that an experienced Scrum Master, who’d been a Scrum Master for many years in true Scrum, could maybe look after three teams. Somebody who is new to being a Scrum Master ideally would look after one, maybe at a stretch two teams, so we did that over a probation period and we found out that it wasn’t working as well as it could because although actually all three teams wanted to keep the individual because they really liked working with him, he was a great guy and he was helping all of them develop, he was starting to reach a glass ceiling in terms of his own development because he wasn’t able to concentrate on improving himself. So that guy stepped down to running two teams recently and we are looking for another strong Scrum Master to join the Singapore site at the moment.
InfoQ: Can you explain your role, perhaps by giving some examples of the things that you do?
Matt: My role now is line coach. Prior to the Agile transformation I had direct responsibility for a number of platforms, so that included things like stakeholder management, financial planning, creation of roadmaps, over and above running development teams. Some of those responsibilities have transitioned to a product owner group, so the product owner is responsible for release planning and budgeting, for example. Some of those responsibilities are not relevant in the context of a managerial position because the teams make a lot of those decisions themselves. For example, the teams do the hiring, so a team without a Scrum Master will hire the Scrum Master, whereas in the past I would have taken a much more active role in recruitment. So my role is more about coaching and mentoring as opposed to directing and allocating tasks.
InfoQ: You mentioned some fundamental things were going to change and it would result in a flatter organisation. How or why do you think J.P. Morgan decided to change so fundamentally?
Matt: Well, it’s not holistically J.P. Morgan, it’s the Securities organisation that has changed fundamentally. I think the reason is that we believe that in development, if we empower the people who are actually doing the value work to make a lot of the decisions they’ll make them better than somebody who is two or three levels up the ladder could. We also believe that if teams are autonomous, they will create a more positive, happier work environment, rather than pushing decisions in. We'd go further to say that we think that happier, motivated people are more invested in the products they create and will be more innovative.
InfoQ: You mentioned you have a number of people reporting to you still, as a line coach. Does that include performance reviews?
Matt: Yes, we still do bi-annual performance reviews to dovetail with our global CIB practices. It’s not practical for a line coach to do performance reviews for 40 people, so at the moment we have appraisal mentors. Appraisal mentors cannot mentor people in the same team, because teams are flat structures where there isn’t an individual who runs the team, but somebody who is in another team who is more experienced will mentor and coach each individual. For the more senior members of the teams, in some cases, they have selected me to mentor and coach them. Through this kind of mentoring and coaching method we capture feedback and dovetail with the review process.
InfoQ: Who or what else has influenced the transformation? Who’s been a significant contributor?
Matt: Internally, we have a Chief Development Officer, which sounds very grand. He’s actually a very experienced and talented guy who partners very well with the groups. Both Simon Nurrish and Tim Born have spent significant amounts of time with global teams to help with the transformation.
InfoQ: In terms of existing management, is there anyone you’d mention as being instrumental in some of the successful changes you’ve seen?
Matt: I think there is Mike Goldverg, who runs the Securities development group globally. He’s been fundamental in our success. There is also Mike Carr, who is the Product Owner for the Securities processing platform. I think both of them have helped us significantly on the journey.
InfoQ: You’ve got several months of experience in this. Do you think Scrum and agile fit the finance industry and if you think so, perhaps give an explanation of why.
Matt: I think yes, because so many times in my career in the financial sector, the market has changed quite radically. Quite often when the market’s changed, we’ve been three months into a twelve-month project and in a traditional waterfall cycle that’s your analysis phase, so occassionally I’ve seen many, many man-months, often man-years, spent on analysis for projects that were then never delivered. In the last eight months, because we are running Scrum, we have the ability to change direction every two weeks and every two weeks we release a slice to production. We now realise the value of that slice, so that’s definitely one area where we are seeing some early success. The other one is the small, urgent work. We found out quite early on that small, urgent things that are only semi-related or not related don’t fit into a sprint very well, that trying to use Scrum can be damaging. So, for that kind of BAU/production support work, we have a small number of teams operating in Kanban mode, so if the business have something small and urgent we can respond appropriately without damaging the reputation of the Scrum teams with our partners.
InfoQ: You mentioned this is in the Securities group. How large is the transition you are currently working on?
Matt: We have hundreds of people in the Securities group across eight different sites globally, so a very significant investment and department size,. We have, in other areas of J.P. Morgan, various levels of adoption of what I’d call true Scrum. The asset management group have some very good practices and are continuing to evolve them and then some of the other groups in core processing are, maybe slightly behind the Securities group, also adopting aggressively.
InfoQ: Can you give examples of problems such a large transformation has encountered?
Matt: We are applying Large-Scale Scrum (LeSS) and in Large-Scale Scrum the product is so large that no single product owner can look after all of the teams. It wasn’t immediately obvious what the product areas should be, so we have had some situations where a team on one site is working on a similar or related feature to a team on another site, because we are still trying to work out which features have affinity and which belong in the same area. So we have had some situations where there’s been collisions on trunk, in the code base, but because we are using CI and trunk development, although we want to organise more efficiently too, we have realised at compile time, rather than integration time, that we were having those issues. So that’s been a positive as well, that some of the practices we use have protected us from scaling issues.
InfoQ: You mentioned teams in different locations there so I presume you still have distributed teams. What sort of impact does that have?
Matt: Thankfully, we don’t have dispersed teams anymore, that’s out of the question, but we used to have dispersed teams that didn’t work very well at all. Now, at least our feature teams are all in the same location, although we have some teams in Singapore, some teams in Mumbai and some teams in the U.S. As we scale it means there are some challenges around the different time-zones. For example, we have a team in Delaware working with a team in Singapore at the moment. There is a 13 hour time-zone gap, so the teams have worked out some tactics to minimise the geographical pain and to interact better. They generally share the time-zone pain, so one day one team will take the pain, the next day the other team will take the pain. When they do refinement activities, they might fork and join, so they might do an hour of refinement together, agree who’s going to take which features and then rejoin again later, to share. We also use fully immersive video conferencing because when using over-the-phone we’ve noticed it’s a lot more difficult to collaborate. Now people can actually see each other which means more effective and rich communication.
InfoQ: Banks are known to have a lot of legacy systems, mainframes and such, which, perhaps because of the turbulence of the markets, are often left for significant periods. How does that affect your use of modern development practices such as CI and test driven development, if that’s another of the practices you use?
Matt: Yes, they are. You are absolutely right, we have significant amounts of Cobol and DB2 and various other older technologies and the tooling isn’t as good for those technologies necessarily, but you can still use the same practices. For example, we encourage pairing still when we have teams working on a mainframe product. We found that the teams working on the product have taken the opportunity to innovate. So only last week, I was at a demo where one of the developers working on our U.S. domestic platform did a demo of FitNesse using the Slim framework integrated with a platform that was based on Cobol and DB2. It was pretty impressive that he was managing to write that kind of test. He was able to use all the practices around behaviour driven development, the spec becomes the test, but on older technology. So definitely possible. Harder, but possible.
InfoQ: I believe you have guardians of some areas of your code. Is that related to the legacy issue?
Matt: We do, and we have guardians for a few reasons. One of the reasons we have guardians is to act as SMEs [Subject Matter Experts]. Because we have internal open source across most of the product, when a team wants to operate on a component that they haven’t operated on before, a platform they haven’t operated on before, a person would look up the guardian and the guardian would either do some paired programming with them or some KT [Knowledge Transfer] and that would help them get up to speed. The guardian, as well, is someone who can hold up a mirror and feedback if something wrong is happening in the component, maybe an acceptance test or unit test is not looking well. He can hold up the mirror and say “you know, guys, you may move a bit faster this week if you do it this way, but in the middle-to-long-term we are going to move slower and slower if we don’t sort this out”. So the guardians can fulfil those responsibilities for us.
InfoQ: And is it one per section of code or do you have a few people?
Matt: We try to encourage there to be at least one kind of apprentice because we want the role to rotate so that we don’t have “special” people, and we don’t have bottlenecks. So we’re trying to rotate the guardians maybe every six months.
InfoQ: Coming back to the adoption side. Can you give us your view of the role of a Scrum Master and what you think that involves?
Matt: I can tell you what the Scrum Master should not be. The Scrum Master should not do task allocation and should not run the team. Very often, if you try to hire a Scrum Master, and I don’t have anything against project managers, but you get people who think it’s a project manager role. The Scrum master instead should be somebody who serves the team and helps the team move faster by acting, as I mentioned earlier, by holding a mirror up and showing the team where they may have dysfunctions that are not obvious to them because they are immersed in doing the job. Examples might be teams not time-boxing meetings or stuck in analysis paralysis. Other examples might be where a team devolves into two-sub teams, which is something I have seen with larger teams, helping bring the team back together. Beyond that, the Scrum Master will facilitate the retrospectives and help the team design improvement experiments for the next sprint and really try and make sure the team carry those improvement experiments out successfully enough so they can improve all the time.
InfoQ: You mentioned earlier that you are looking for another Scrum Master. If i was looking for a job in the Securities group, what would I need to possess?
Matt: It would be more about skills than knowledge. We are not looking for anyone necessarily with domain skills, we’ll even consider somebody who doesn’t have technology experience. We are looking for someone who partners very well with people, somebody who can influence and negotiate, somebody who understands Scrum and why we do it. I would say it’s more around the attributes of the person than the knowledge and the skills that they have.
InfoQ: How do people like this new way of working?
Matt: People are genuinely happier. Because we worked on extended development cycles with large integration phases before, there was a lot of last-minute pressure at the end of every project because a lot of the risks were deferred to the end of the project. Most of the risks in our kinds of programmes or projects are integration risks and production implementation risks. With Scrum we’ve been integrating continuously and we’ve been releasing into production every two weeks, so there is no kind of pent-up pressure that happens. The teams are working in a more sustainable fashion and, because we have broken down a lot of the silos, they feel more empowered to change things. So they are not constantly waiting and context-switching. They are not waiting for testers to test their work. They are not waiting for specs from a B.A. If they want to get a piece of work done they can just do it and because they are delivering value more frequently they feel more of a sense of reward.
InfoQ: How do you know this? Do you know from observation or from discussion or something else?
Matt: There is generally more buzz and laughter on the floor than there was before and the teams go for lunch together. They like the people they work with and they socialise with them. We’ve gathered some anonymous feedback as well and, almost without exception, the feedback we’ve received is that people would find it very difficult going back to the previous way of working, now they have worked in this model.
InfoQ: Can you share an example of a significant change for people and their work?
Matt: We do have an example of a hands-on team member who was promoted this cycle to Executive Director. Now, traditionally, an Executive Director might be somebody who runs a department of 40 or 50 people, so a level 2 managerial role. This person has been promoted for the value that they are delivering in their team contributor role and will continue as a team member as an Executive Director.
InfoQ: What are the most valuable things you are looking to do next?
Matt: Next for us is we’re working on our next generation Securities platform. Singapore is a major contributor to the development of that product and over the course of the next 12 months we have some very significant deliveries. Our operational users will start to use the platform and it will be a real competitive differentiator for us over the next five to seven years. So the next big milestone for us is going to be putting the first market live on this platform.
About the Interviewee
Matt Winn is Securities Location Coach for Singapore and Manila at J.P. Morgan. He is responsible for coaching and mentoring over 100 staff who are members of 14 cross functional and multi-skilled Feature Teams in the Corporate and Investment Bank (CIB). Matt and the teams are focused on the Securities Processing product which is integral for many Business Units within the CIB. Prior to his current role he was Chief Business Technologist responsible for several mission critical applications. He has 15 years + of software development experience predominantly in financial services but also in computer communications, computer telephony integration, and the high speed telecommunications network test domains.