Thomas Gieling, solution architecture team lead at Adidas, and Kristian Skoeld, performance consultant at SOASTA, co-presented at the last Velocity Amsterdam conference how the shoe and clothes giant manufacturer tamed an out-of-control proliferation of third party tools in their global websites which was killing performance. Furthermore, this had led to a blame culture setting in between business and IT. A new third party governance process focusing on performance data and user experience validation was key to stop the bleeding.
Initial lack of governance (and capacity in the IT team as the number of sites grew 20x over a short period of time) meant that business needs in terms of analytics, tracking and even (some) functionality was recurringly addressed by yet another third party tool being added, regardless of their technical quality. Performance started to drop and there was a divide between technical staff (blaming business for too many tools) and business staff (blaming IT for the poor performance).
Skoeld helped devise a strategy for fixing the situation that required inventorying the tools in use (more than 60 per site), assigning business owners to each of them, and defining its purpose, impact (does it add site functionality, or is it an UX enhancer, or a data analysis tool?) and criticality level. Third parties unclaimed by business were removed first.
A data-driven process for reducing third party dependencies was put in place, whereby each tool's value vs performance cost ratio was analysed collaboratively by business and IT. A low criticality tool with insignificant performance impact might stay while a high criticality tool with high performance cost might have to go (or an alternative has to be found).
This new governance process also took into consideration the actual impact in user experience. A/B testing (version A with the third party, B without) allowed comparing the net effect on user conversion and even financial impact. The shared goal of reducing technical debt (mostly in performance terms) while prioritizing business value was key to bridge the gap that had set in the organization.
One illustrative example was a third party tool for collecting feedback from site users. This tool made 20+ requests and increased page size by 300kb. While this seemed unreasonable at first glance, data from A/B tests showed user experience (namely session length) was not affected, and sale figures were equivalent (with or without the tool).
Skoeld also recommended taking control of direct third party dependencies first. Finding out all the indirect dependencies (for example using a Request Map), can be hard to make sense of (Skoeld found 2800 third party domains reached by users in just 2 weeks on adidas.de only). It's more important to analyze direct dependencies and their external requests. Over time, the organization should aim to build direct relationships with the high criticality 3rd party tool providers in order to set performance expectations and establish a constructive feedback look. In short, business critical 3rd party tools require active governance instead of passive consumption.
