SOA Strategy and Spline Tactics

First things first: ‘SOA Strategy’ in the title of the article stands for ‘Service-Oriented Agile Strategy',’ and we will talk about how to transform IT to make it agile to Market. ‘Spline’ is a term unusual to non-mathematicians. In simple words, spline is an interpolation function that balances between the smoothing and closeness to the actual values. The trick here may be explained using commonly known example: if you need to reach a point B from a point A, the smoothest and fastest way is a straight line between A and B, however, if you know that then you need to move to a point C, you will try to change your way and even allow a small deviation from the point B to minimise the way from the point A to the point C. Moreover, if you have to follow to a point D afterwards, you will try to smooth both points B and C when heading to the point D, the line you construct in your mind is the spline. In other words, spline optimises your local behaviour based on the knowledge of your future – you are still near the point B but you know your target point D and you will adjust your behaviour or resources appropriately.

Business Case

Not long ago, a friend of mine, who works for a pan-European financial organisation, asked me what I would do in his case. His task was about transforming the IT product portfolio and platforms after several merges of different financial groups covering the space of Central and East Europe and crossing two different economical zones. Yes, he had to ‘boil the ocean’ of extremely heterogeneous platforms, languages and local laws. All of this was needed… just to achieve business flexibility. The goal of this IT transformation was set as a shifting IT into position where it would be able to meet future business requirements in a more effective and efficient way.

Matter is that his organisation had united business concept and management but still didn’t have consolidated business structure capable of generating and, maybe, unifying business models in more than one zone. That is, IT was on its own to offer a solution capable of meeting whatever business needs anywhere from Frankfurt and Milan to Moscow.

Another special aspect of this business case was in that the organisation used to work on a local level, i.e. it was exposed to all local law changes in two dozen countries. This usually brings additional risk of over-sizing and over-prioritising an importance of local specifics and compromising the overall corporate needs for the sake of benefits for local branch.

Since the organisation we are talking about belongs to the Financial Service Industry, Technology is the revenue enabler and the task of making IT agile to Business is almost equal to the task of making IT agile to Business Market. As we know, IT adopts changes much slower than Business (due to infrequent and complex release processes). As a result, Business has to expose IT to the market changes at the next moment when it is exposed to them itself. Business and IT have to go head-to-head in the change adoption if Business wants a market advantage nowadays.

So, the question was how to start and what the plan for such global transformation might be? I do not think this is an original question and the case per se is also typical. So, experience gained here might be useful for readers.

Strategic Agility

The strategic transformation goal has been formulated clear – create an application and platform environment capable of easy adopting changes that Business might need in the future. What the changes are and when this future happens to be are unknown. However, all spectrums of different acquired and built applications and IT products have to be refined for all geographical locations and LOB with a desirably unified approach if not a solution.

Described task sounds similarly to the request to ‘eat an elephant’. We will do it piece-by-piece more accurately, layer by layer. At the start, we might not know what changes we need to address, but we know where changes may happen. This is the sphere of business functionality. Changes in other spheres will either affect corporate business functionality or will be influenced by the corporate business functionality. Unless the company changes its business corporate model, the variety of changes is finite and defined by the business functions, features, processes and products. Let define this as a top layer.

One layer below comprises business functions and features realized in the form of business products, business solutions, assortment of combinations of business functions and features and alike, which have particular content and scope. We will call the entities of this kind the business components. An enterprise offers business components to its customers/consumers/clients and supports components by internal operational structure. A change in the layer of business functionality likely reflects in the change(s) in some business components (may be in one component only).

In the next layer, we position two major elements that realise business components: business operational units or their structure and technical components. Not all Business Components include Technical Component. Besides, some operational units can use more Technical Components than others. Technical Components may implement some business functionality, or automation utilities or just access means to the business information sources.

Figure 1. Stack of functional business layers in an enterprise

The lowest layer consists of Technical Applications. The applications form the content of Technical Components. The latter may consist of one or several Technical Applications. Technical Components sometimes may appear as technical products to the business users.

A business solution or product consists of the combinations of elements in several layers horizontally and vertically. For example, a combination of Business Components in the Business Functional Components level may be coupled with a combination of Technical Components comprised the combination of Technical applications.

Figure 1 shows us the diagram where described layers compose a stack of functional business layers in an enterprise or, in short, a stack of layers. The important things about the layer stack are:

1. Each layer represents a snapshot of particular combinations of its elements in the given time;
2. Each upper layer is mapped to the one and only one layer below.

High dynamics of modern market is associated with multiple large-scale changes; the market moves fast, with big steps. Thus, an adoption of these changes in the enterprise likely requires a re-combination of existing business capabilities rather than creation of new ones. There is simply not enough time for the new development. That is, an adoption of a change in the layer may be viewed as a change in one or several combinations of its elements, though we may have a change in the content of the element as well.

A change in one layer propagates to one or two neighbor layers via mapping. For example, if a Business Component can benefit from an automation of some of its business processes, its business functionality does not change but underlying Technical Components or component combinations will change as the result of the automation. This also might lead to some changes in the layer of Technical Applications. In another example, a technical automation may replace the manual process implemented initially by one of Business Components. If this process is the only content of the Business Component, the latter disappears from the Business Component layer and becomes a new Technical Component. However, the sub-Business Components that previously were used in the now-automated process continue to reside in the Business Component layer. Overarch, different business changes result in different re-compositions throughout the layer stack.

The graph on Figure 2 illustrates an expected ratio between changes in different layers if the layers and components are made right. In the 5 layers of the layer stack, we have 5 combinations of the elements - one per layer - named respectively:

• CBFF - Combination in the Business Function/Feature layer
• CBFC - Combination in the Business Functional Component layer
• CTC - Combination in the Technical Component sub-layer
• CBOS - Combination in the Business Operational Unit sub-layer
• CTA - Combination in the Technical Application layer,

We interpret a change in the combination of elements for particular layer as a ∆ (delta) of combination. We can see that a relatively small change in the Business Function/Feature layer can lead to significant change in the Business Functional Component layer. This change is supported by a bit smaller change in the Technical Components and Business Operational Utility layers and cascades to even smaller change in the Technical Application layer. In the contrast with traditional style, one change in a Business Functional Component can lead to a cascade of changes in the dependent Technical Components and Applications, which takes a lot of time and additional investments into ‘new development’. That is, a small ∆CBFF can lead to a significant ∆CTC and even bigger ∆CTA.

Figure 2. Ratio of changes in the business functional layers.

I have put so much attention onto the business functional layers to setup a context for strategic planning and solutions. Business Strategic Plan operates with and affects the Business Function/Feature and Business Functional Component layers. The Business Operational Unit sub-layer may be affected as well although indirectly. If IT strategy in the area of automation of business functionality is out of sync with the enterprise’s Business strategy, the integrity of the layer stack suffers or even breaks, and we need to talk about restructuring of IT rather than about agility. So, we assume that IT strategy is in line with Business strategy that, as I mentioned in the Business Case section, has a goal to transform IT into a flexible structure capable of supporting upcoming frequent changes in the business needs. Also, the dynamics of the market are those that the most efficient mechanism in adopting the changes is the re-composing of existing business capabilities implanted into the Business Function/Feature and Business Functional Components layers.

New compositions do not necessary replace the old ones and have to offer a certain level of Quality of Service (QoS), cost reduction, change adoption and compliance with the business and technology strategy plans simultaneously. That is, we have a classical situation where ‘more’ is required to be provided with ‘less’. If one tries to address mentioned requirements separately, one-by-one there may be a set of serious problems because requirements of QoS and strategy compliance do not necessary fit well with the cost reduction. Also, it may be difficult to valuate the collective effect of such sequential implementation. Though, these problems may be resolved much easier if we use a methodology that incorporates mentioned requirements all together, at once.

Fortunately, such methodology exists nowadays; it is based on the concept of service orientation. The power of service-oriented (SO) solutions is in that the services, if designed and implemented properly, adhere to the Principle of Service Composability. This Principle states that services should be constructed in the way that allows them to participate at the same time in as many service compositions as needed, and that some services may be compositions of other services or other service compositions. Two other Principles – Service Loose Coupling and Service Abstraction – allow services to adopt changes that cannot be addressed via re-composition internally and transparently to the consumers (unless the changes lead to the alteration of the service’s business functionality and Real World Effect (RWE) or results). The latest draft of OASIS SOA RAF standard positions SO architecture harmoniously in Business and IT bridging the two worlds. That is, services as Business structural elements are not my imagination.

Thus, agility of service-oriented solutions to modern business and market needs is in its natural ability to support change adoption and overarch flexibility. This is not only a technical ability; it is also business ability; if the business activities are viewed as services to external and internal business consumers. Talking about effectiveness of the business and technical services, the metric here is not how many times a service has been reused (or used) but how many potential meaningful business combinations the set of business services can produce if needed. In other words, services stick on strategic tasks addressing tactical needs at the same time.

If we choose service orientation as the IT strategy and the IT transformation mechanism, this transformation converts the application portfolio into a collection of services with particular business capabilities that may be in demand now or in the future. The SO development counts in the future use of the services by unknown service consumers. If your company’s business culture allows IT to move only as a “horse with blinkers”, reacting only to what is today in front of its nose, be very accurate with a service-oriented transformation of IT; your business has to transform its culture first because otherwise it will not value your efforts. Even so, if you are after cost cutting, Meta Group from Stamford, Connecticut, says that IT can trim budget more effectively by “taking a long-term asset management point of view on application costs”. In other words, again, look into the future to act properly today.

Transformation of the application portfolio into composable services is the first step that IT can do toward the support of strategic business initiatives. While the spectrum of offered business functionality is very important, in the modern economy it is even more important having the business ability to change offered functionality in tempo of the market changes. Forrester Research believes that, “the reality of the digital age is that your business is embodied in your technology, and your business can change only as fast as your technology can”. Service-based composability, therefore, is the key strategic enabler of the adequate technical support for the business flexibility.

Spline Tactics

Methods Learnt in Splines

Spline-function has been used widely in many human-oriented applications and in many areas of knowledge due to its exceptional behavioral characteristics. Here I will summarise a few of them with regard to IT transformation tactics and explain their difference from Agile Methodology.

The ideal situation for the spline exists when 1) we know exactly the conditions at the beginning and at the end of the smoothing process, i.e. we know the ‘future’ at each intermediate position, and 2) we know all allowed deviations from every to-be-smoothed point. Unfortunately, this ideal world does not exist in the reality of IT. In the best case, we know what we have in IT today; we also know some stakeholders’ opinions, desires and objectives, and we have some ideas about the final state of IT product portfolios, services, applications and modules specified in the Strategic Plan. So, how can we learn the ‘future’ to adjust out tactical steps toward the strategic goal?

Many years ago I noticed two things about splines:

a) The effects of the final conditions at the end of the smoothing process gradually decrease backward to the beginning of the process. This means that with the finite ‘mistake’ the spline can be constructed iteratively, without the full knowledge of the conditions at the end.

b) If the spline started with certain behaviour pattern in the point X, it can proceed with this behaviour (like in extrapolation) until the errors of passing intermediary points become unacceptable.

Figure 3. Optimising position for the highest flexibility

We will take advantage of these notes. If we look at the image on Figure 3, we can see three motorcycle riders - #1, #2 and #3 – on the curve road. Now, imagine that there are high walls along the road edges, i.e. the riders move in a sort of a tunnel. The white lines in the image show how far each rider can see forward depending on its position in the road. The #1 rider has the shortest observation, but it moves along the shortest way – the internal edge of the curve road – for the given position; so it has a chance to be the first to the next few points of the ride. The #3 rider sees very far from where it stands, but it moves along the longest path – the external edge of the curve road. If the #3rider continues its way along the external edge, it, probably, will be the last in the competition, but it knows how the road goes long ahead and can change or adjust its path with a small risk. This gives it a competitive advantage over others. The #2 rider can see far enough to take a risk of adjusting its position on the road; this risk is higher than for the #3 rider because the #2 cannot see as far as the #3 can but this might be a still acceptable risk.

This examination tells us that in each Spline Tactics time-box, the decision-makers have to observe several steps ahead (as many as possible) to properly adjust the particular tactical solution. They can do this because they know their strategic target, and they are learning the best way to reach it. So, the question is how to do this.

How Can We Do It?

We can do it using the following scenario.

Step 1. We select a relatively isolated area in the application portfolio with defined business functionality; the area, which is important but not crucial for the company. We will use this area of IT for the first transformation (see step 2). When the transformation complete, we will take another area for the next transformation. This area may be more important for the company, and so on. We can start in several areas simultaneously and transform each of them via independent iterations.

Step 2. According to our strategic goal, selected application portfolio area has to be transformed into one or several business services under one mandatory condition: each business service has to be constructed end-to-end from the User Interface layer to the Data Access Layer. The design of the service may include more business functions and feature than implemented in the first iteration.

The design and construction of the service(s) has to be driven by the decision whether the application(s) being transformed will be used as the service resources in the target strategic architecture or the application(s) or its parts must be retired.

In a tactical solution, the business service may include business functionality that includes only a part of functionality provided by an existing application. If the rest of application’s functionality is not utilized, it may be eventually retired. During further tactical transformations in this area, more and more business functionality will be transformed from the application(s) into the service(s). If the strategic transformation considers reuse of the legacy application in the future, the transformation in this case implies creation of the service as an additional layer on the top of the legacy application. This service may be viewed as a proxy or façade but it has to have its own independent body capable of dealing with an arbitrary functionality resource, not with that legacy application only.

Step 3. Release this partially transformed area to real consumers.

Step 4. Collect feedback for a period of time from the users and the support, i.e. gather the ‘deviations’ in the service behaviour. If collected ‘deviations’ are insignificant or acceptable, proceed with the Step 5; otherwise, go with the Step 6.

Step 5. Since it seems that our service more-or-fewer satisfies our consumers, we can continue creating other services and/or transferring more functionality from the application(s) to the existing services in the same manner as we constructed the initial service. In the spline technique, this means that our beginning assumptions about the target and intermediary results are close to the reality, and we can move to the next IT area for the transformation, i.e. to the step 1.

Step 6. The condition of this step is that the feedback from the users and the support is not positive enough to preserve our initial assumptions about the service; thus, we have to step back. This is the key moment for the adjustment of our tactical solution. We ‘return’ back to an intermediary point of our solution where we think our solution worked well enough and rethink, re-design and re-implement our solution starting from there. The difference in this case is in that we know some future (what we should but did not provide) and can adjust our behaviour/solution accordingly. For example, we might need to reduce business functionality for some services, add new services or continue to use the old application for a while; we can rethink the chosen areas for transformation – we might cut off important dependencies when we limited the scope of the transformation area the first time. That is, there may be many different considerations how to start the second iteration or tactical solution. It is recommended to make the next solution not that rich with functionality as the previous one if you find that few functions worked well. For example, if initial set of functions included 10 items and only 6 of them worked, i.e. the step back is about 4 functions; the next solution might include only 8 functions, 4 of which should be re-implemented functions from the first attempt. When rethinking and adjustment is done, we can continue with the Step 1 for the next tactical transformation in this area.

Overarch, we are doing a big step forward, test ‘the water’, step back a little, change direction a little, and make another step forward [5]. Those who sew can recognise a method called a “needle back” as illustrated on Figure 4. It consumes more threads but significantly firms the seam, i.e. you can rely on it for much longer time than on the one-by-one stitch seam.

Figure 4. Turn back to move further forward.

Comparison with Agile Methodology

If you are familiar with Agile Methodology, you, probably, have noticed that the Spline Tactics has the same target – agile IT application portfolio and each separate piece of it to the strategic solution that fits with the business needs – but it uses a different technique. Particularly,

1. the Spline Tactics uses time-boxes but they are based on the time for collecting consumer feedback rather than on the development time;
2. the Spline Tactics allows to start several tactical transformations simultaneously and proceed with an individual pace for each one;
3. the time-boxes in the Spline Tactics are sequential similar to the Agile Methodology but they consider a step back for the solution adjustment as a mandatory condition of the next step forward within the boundaries of initial targets rather than to meet new requirements;
4. all intermediate results in the Spline Tactics are based on the lessons learnt and on the refined knowledge about actual behaviour of provided solutions. This means that the Spline Tactics suggests a lot of preliminary thinking and design in each tactical solution, which is not typical for Agile Methodology;
5. success of each tactical solution in the Spline Tactics depends on how the development team is well informed about business intentions, new trends and strategic tasks/assumptions because the work of IT product transformation has to sustain in the observable future. In comparison, Agile Methodology does not look too far forward and promotes frequent requirement refining and delivery to catch happened changes; it does not try to anticipate these changes and count them in during the current time-box development;
6. Spline Tactics is about learning real life conditions and adjusting to them while preserving the initial requirements. The Spline Tactics is about the solution optimization in not fully known conditions and constraints.

The Spline Tactics and Agile Methodology also have different roots. The Agile Methodology was formed during the Web-enabling boom when the clients – business users – did not have clear understanding of what they really needed, when and why. So, IT had to satisfy the flow of ‘new discoveries’ and ideas facilitated in Business. When business Web domain stabilised, one of the inherited problems that Agile Methodology has to deal with is in that some business people have constructed an opinion that IT can work with very brief and weak requirements, that Business is not, actually, responsible for its Business Requirements and IT analysts will find everything they need to build the software (yes, IT does this but this finding is not necessary the same as the business needs from the IT work).

The Spline Tactics is motivated not by individual business units only but by the corporate needs as well: organizations see that outdated IT products cannot provide market advantages to the business and have to be refined. Strategically, the only requirements Business can formulate to IT with regard to this matter are: a) technology support for the enterprise functional business model; b) flexibility in adopting business changes whatever they will be and whenever they come. The Spline Tactics’ iterative time-boxes are driven by complexity of the transformation task and business priorities for the use of technical solutions. For instance, if IT has transformed several applications into the services in the area of business where activities are seldom, IT will be forced to wait for the feedback from the service usage for quite a long time, i.e. it might not be able to move to the next tactical solution with appropriate confidence.

Finally, I would like to illustrate a project aiming the transformation of a legacy system into the SO solution using the Spline Tactics. As we have learnt already, the Spline Tactics requires each transformation to be conducted in three steps:

1. Creation of the service that replaces the legacy application at some point of time in the future. At this step, the service is not self-sufficient yet and uses the legacy system as its resource. The core difference between this step and attaching new interface to the legacy system (e.g. a Web Service interface) is in that the service is built as an application ‘on the top’ of existing, legacy application from a logical perspective. In physical reality, this service should be deployed on the strategic technical platform. If the platform where the legacy application resides is not a strategic one, we have a case of distribution and remote invocation. This means, the service design has to consider a performance compensating mechanism
2. A period of time when the service and the legacy application run in a bundle in production, in the real world. This is the time for gathering mismatches with the reality and the time when the service is gradually filled with functionality previously provided by the legacy application
3. When all planned or needed functionality gets provided by the service itself, the legacy application may be retired.

The mandatory condition for a project of described transformation is that it is one, united project rather than three separate ones (a project per step). That is, the people who worked in and managed the step 1 should proceed to step 2 and 3. Until the step 3 is defined, funded and scheduled, the step 1 may not start. Creating a layer of services on the top of legacy applications increases the complexity and the cost of the technical solutions. Thus, there must be a step that decreases this cost but on the new basis (services) already and opens the door for the efficiency delivered by the service-oriented economics.

What If the Strategy Changes?

Indeed, if service-oriented agile strategy changes, how this would affect the Spline Tactics technique? This question is incomplete without specifying whether the strategy stays service-oriented or not. Also, new strategy may, for example, become either ‘growing’, or ‘sustaining’, or ‘survival’.

The principle of the Spline Tactics remains the same if the strategy remains service-oriented. Otherwise, the tactics have to be changed appropriately. Why should tactics change? Well, the Spline Tactics are based on the view of the world through the principles of service orientation. Every spline tactical solution moves the IT transformation to this view, which is not necessarily true for other views of the world.

Nonetheless, if the strategy is of type ‘growing’, keeping the Spline Tactics is recommended. The reason for such a recommendation is derived from the fact that all businesses are service-oriented in nature. That is, sooner or later, the business will appreciate the service-oriented ‘spirit’ of the tactical solution.

If the strategy is of type ‘sustaining’ or ‘survival’, the enterprise is, probably, in the corresponding ‘sustaining’ or ‘survival’ situation. The case where internal IT faces a risk of being replaced by outsourcing and wants to survive is the subject of different theme because from the strategy perspective IT does not disappear, it just changes its form. If the enterprise sustains downtime or tries to survive, it needs a lot of flexibility in the business. Simple down-sizing may help but for quite a short time. During the last economical crisis, many companies failed because they did not recognise on time that the downtime came to stay for long; they continued the same business in the same manner just reducing the volumes. However, the market changes in many aspects simultaneously and to sustain or survive the enterprises had to change their product portfolios and keep changing them at the pace of the market changes.

We know that wise people do not ‘change horses in the middle of the stream. So, what IT transformation I am talking about if the situation is about sustaining or surviving? Obviously, in such cases the priority of some previously active business functionality may be reduced but instability of the market demands may require this functionality in the next moment. Cutting cost is not the solution when the market is not only down but quickly changes. Cutting IT personnel and resources reduces corporate flexibility at this point when it is needed the most; this hurts the business first.

Thus, IT transformation during the ‘sustaining’ or ‘survival’ time is real and even may be recommended in many cases. The transformation I mean here is a venturous service-oriented transformation. It is characterized by two factors:

1. Target IT components allow for relatively easy compositions and re-compositions but are not composable by themselves because they are quasi-services.
2. Target IT components encore a high risk of failure in several conditions because they are not necessary robust and scalable as the services should be.

In short words, mentioned quasi-services are legacy applications (an application becomes a legacy application in the moment it is deployed) that provide a mechanism to be engaged in the external compositions. A classical example of such quasi-services is regular applications wrapped by dedicated interfaces for ‘reachability’. The example of venturous service-oriented transformation is so-called Composite Applications, which are based on the straight integration between its pieces without a concern that they might be incapable of working properly under in the service execution constraints. By nature the venturous service-oriented transformation is tactical. In the spline view, it is equivalent to a short step forward that anticipates almost a full step back to adjust starting assumptions for the next step forward. Anyway, the step forward here wins some agility, but primarily it wins some time. This time is needed to prepare for the long step forward – to learn whether the next market change will be more significant than the current one and will require more serious adjustment.

Thus, if the strategy changes but still allows a degree of business and technical flexibility, there is a chance to go with the Spline Tactics.

It is not easy to fail application portfolio transformation using the Spline Tactics. Let’s look at the difference between the traditional declaration: “we will build you a solution; if it breaks, we will fix it” and spline tactical declaration: “we will build you a solution that we will adjust as soon as your situation changes”. For the former declaration, a consumer can respond: “what am I supposed to do if your solution breaks all the time? When will I do my work?” From the latter declaration, the consumer understands that he or she is in the command and can control how and when the solution is adjusted to changing needs; notice the shift in focus from a perennially broken solution to an adapting solution. In other words, the latter declaration invites to the cooperation and collaboration while the former one stays in the master-servant relationships. The Spline Tactics anticipates a re-work and movement forward to the preliminary set target; re-work may be planned and counted into the delivery schedule while being open to upcoming changes after the tactical solutions started to progress and deliver business results. If agile solutions just announce that the changes are welcome, this means that the changes will be taken into consideration, eventually, nobody knows when and how much work they require. Just welcoming changes is the reactive tactic while spline/smoothing them is the proactive tactic.

Take-away Thoughts

This article discusses agility to Market changes that IT can gain using a strategy oriented onto the services. The concept of service-orientation is the major construct of the technical product portfolio in the dynamic Market. The service-oriented agility is accompanied by a technique represented here as Spline Tactics. This technique encourages learning the business and market objectives in the given area of Business first and, then, building a tactical solution, using it in the practice until the real-life feedback requires an adjustment, stepping back from the achieved results to adjust the solution in accordance with the changed Market needs and only after that making more agile tactical step forward again; the entire process repeats from this point. Finally, the article observes possible scenarios of the strategy change and related influence on the Spline Tactics.