IBM’s Kyle Brown starts hisnew comment by noticing that a lot people: "dig themselves into a hole by using Web services and XML in inappropriate ways". In his comment, Kyle describes three common pain points, explaining why they occur and providing some alternatives.
My message ate my server! Kyle notes that often web services developers start experiencing "out-of-memory" errors or strange "performance problems" often finding that the server has extremely high processing loads, with CPU utilization hovering close to 100% coupled with low throughput and high network latency. Typical cause for these symptoms are very large - sometimes 50 MB or more - messages. Moreover, these large messages often contain very large embedded binary information encoded using base-64 encoding as part of the main body of the XML message. This typically happens when a
...developer does not understand the limitations of the technology: XML processing is very useful for a lot of problems, but you have to realize that messages are parsed - and in most ... products, that means that much or all of the message will be held in memory.
Kyle recommends the following options to improve the situation:
- Don’t send redundant information. When sending binary data, in many cases, you might find that the messages are highly repetitive. If this is true, you might want to consider using compression at the HTTP level to improve your network latency. While this won’t help with your processing load, it might help mitigate one of the problems.
- Don’t embed the binary information in the body of the XML message at all. This is the better solution, and there are a few different ways you can achieve this. For example, you could instead use SOAP with Attachments or the Message Transmission Optimization Mechanism (MTOM) to bypass the parsing overhead, although this doesn’t help with the issue of network latency.
- ...An even better option might be to not send the large binary blob using SOAP at all. Instead, use an "out-of-band" transmission through a managed file transfer system ... or a "claim Check" pattern to avoid sending large binary files over SOAP and HTTP altogether.
Pardon me, your data is showing. Another typical web service "performance problem", according to Kyle, is that the level of Web services that are being used is very, very low - often each Web service corresponding to a single SQL statement, which comes from
misunderstanding of SOA architectural principles. One of the key principles of a good SOA architecture is that your services should be at a high enough level to be reusable.
According to Kyle, these situations typically occur when:
...service emerge in designs where the services are derived "bottom up" from existing code; often, developers will look at their existing architectural diagrams and decide to turn each layer in the architecture - including the persistence layer - into a set of services.
Instead, it would be better to apply coarse-grained Web services in the right place in an SOA architecture - again, looking at a standard layered model of an architecture, there is often a well-defined place in the architecture that already encapsulates the business logic of a system. These services can be wrapped using the "Remote Facade Pattern" to expose model-based services in an appropriate manner.
Schema? We don’t need no stinking schema! Kyle notes that often developers try to reuse existing code that has already been implemented to generate and parse XML as a basis for web services implementations. These implementations typically use an XML parser to marshal/unmarshal information, and Java HTTP classes needed to send and receive the XML documents. When adopting web services, a common approach, in this case is to create a WSDL document that uses the
Symptoms of this problem are that the organization doesn’t see the benefits of SOA that are promised, and that maintenance of their solutions seems to be harder (not easier) than before Web services were adopted.
The simple solution to that is that, whenever you are writing Web services, either using the WS-* standards or using a REST approach, is to make sure that you create a complete and accurate XML schema to represent the structure of your document.
If you are building WS-* Web services, then this XML should be included as part of the WSDL that describes your Web services. Even if you are following a REST approach, having an easily accessible XML schema will encourage reuse of your services.
Avoiding the pitfalls described by Kyle seems like common sense. Unfortunately, our industry proves that unless SOA implementations are well understood and governed, we keep repeating the same mistakes over and over again