Great, thank you. I’m Dave Duggal, I’m the managing director of Consilience International, we’re an early stage, Lean startup, we have a Java based application platform, it supports metadata driven real time applications for dynamic business environments, so we are now actually implemented, we now have implementations on four continents, so we’re working in the Middle East, we’re working in Europe, United States, as well as Australia and New Zealand.
Yes, great, good question. So, I’ve been in the software industry for a little while and one thing I’ve noticed regardless of vendor, technology or sector is that business users, people in the business units really struggle with implemented software that typically doesn’t support their needs, doesn’t support them in the ability to adapt to customer requirements, so for those business users that are out there on the fringe creating value for the customers, they really need to be able to adapt and support variants so they can match customer needs.
Eric’s full question: So, this is a little bit like trying to find another next step or an improvement to bridge the gap between business and technology, I know as long as I’ve been in computer science and IT, there’s always been a problem of understanding the business requirements correctly and getting the technology to understand the business, getting the business to understand the technology. So is this kind of in that space trying to help bridge that gap and get better apps for business alignment?
That’s exactly right. So, actually I was just talking to Janelle Hill at Gartner, she’s an analyst at Gartner and we were talking about their latest research on intelligent business process management systems and Janelle pointed out that maturity model wise, if you look at most business they are very familiar with business process technologies and implementing them in the enterprise but at this point they are mostly achieving efficiencies, they are taking best practices and they are standardizing them and implementing them globally. The problem with that is that if you standardize something you are not always matching it to business need, right? And it’s not always creating the flexibility and agility for the business users out on the edge to actually adapt to actual real customer requirements or events.
Eric’s full question: I remember at Credit Suisse where I was working in the IT department, there was a big frustration on the business side with the lack of ability probably on the IT side, to respond quickly to market changes, business changes, business process changes and a lot of it is because of big investments in existing systems How do you get them to change? It’s not easy so it’s a big challenge.
Sure, it is a big challenge, I think there’s a whole bunch of challenges, you can’t chide the business for wanting to be responsive to the customer and you can’t chide IT for wanting to be efficient. The problem is we need to find some sort of balance here, because the demands have gone up, the business world is getting more dynamic, the business wants to be more responsive, they want to be more context aware, they want to be more Agile but IT budgets are being cut, so how do we balance all those things and deliver value? So, we looked at that as being a challenge and the fundamental challenge there we saw as sort of a tension between matching, which is really what the business units need to do, matching value to need to what the IT departments want to do, create efficiencies through standardization and reuse. That was really what we were trying to attack.
Sure. I went to a whole bunch of conferences, just like QCon and learnt a lot from online, researched thoroughly, what was going on in the latest in SOA, the latest in REST architecture, web-style architectures, and we felt there was a fundamental gap in technology, in these architecture styles, clearly we need to support composite, open, extensible technologies today, right? We need to be able to work with services and APIs from third parties, but the problem with the way things are configured today is they don’t allow for a shared understanding between the client and the server, everything is compiled and everything is statically typed on the server side and the business user is just the recipient of it, effectively business users work around the system, not the system working around the business user.
So, when we looked at SOA, we look at the idea of decomposing into logical units of work that’ll make great sense but data was a second class citizen in SOA, we saw that as a fundamental problem . Then we looked at REST, of course we found it very elegant, we loved the scalability, it’s stateless etc., but the document architecture doesn’t do anything in itself, it’s great for discovery but otherwise it doesn’t do anything actual. What we did we actually fused SOA thinking and REST thinking so we can actually work with services, we deliver a service in the end but we do it by actually building services out of resources. So typically what you hear is business processes are decomposed into services, we take that one step lower by decomposing services into resources, web resources.
Yes, it is like a business process oriented modeling environment, but instead of actually doing designing a process with static flow chart where you are actually defining, at design time, sequences, what we do is we decouple a process into loosely coupled tasks and then we allow them to be connected by rules, so we externalize the sequences into rules, and then we actually also allow the tasks themselves to be personalized, so what you have is a very loosely coupled dynamic process that comes together based on policies, what the business user is doing, what events, what’s happening, it’s really a fusion of complex event processing and business process, to give you the kind of a performance you want.
Well, I think the future is clearly more and more declarative, business users want to be more and more involved, that really requires two things. It means more user interface driven modeling, it also means real time modeling, where the business users, the process participants are actually co-creators of their own experience, they’re actually able to modify their experience, within permissions, within whatever authorities, within whatever approval processes, but those all happen in flight, instead of every time you run into an exception, throwing the exception, reporting the exception, if you actually do report it, waiting in a queue to be considered for the next committee meeting, then for it to be assigned to a developer who then goes off independently develops it, and hopefully the solution is ultimately delivered in a way that you want use it, but by that time the business logic might already have been lost or shadow systems were created in the gaps and neither works out well for the enterprise.
Eric’s full question: Right. One of the challenges with IT projects is, as you know, some of them can take a long time, a year or two years, especially the more significant ones and that causes some of the frustration and some of the approach that you are taking if I understand correctly with your tool is to help get things done a little more quickly, to maybe prototype things a little more quickly, to let the user see more quickly some concrete results and maybe even iterate a bit on the results, to improve them towards medium requirements.
That’s excellent, because once you have a fully reactive system, everything is loosely coupled, everything binds at runtime, what you can actually do is model to your level of need and then you can grow as you go, you can actually just say ”hey, we don’t know everything, the fact is at design time you never know everything, it's all about technical debt, right”, but instead of implementing something that’s static and just rolling with it, what you’re able to do is actually build something in a more experimental way which supports rapid iterations and rapid improvement and not only can you be improving it, you can be growing it at the same time so that is based on actual demands and real world experiences as opposed to something that was, you know, it took a year to write up and then a year to develop and by the time it was delivered it was no longer relevant.
Eric’s full question: Right. So you guys are really in support of the concept that business analysts and business users need to be more involved in the technology but just that the approaches that have been proposed or the prototyped or the products that exist in the space that try to meet that goal haven’t really worked and there needs to be a better way that’s more iterative, more light weight, more declarative.
Yes, because we’re talking about, it’s really two things, we really want to make a better division of labor overall but the IT department doesn’t want to dissatisfy the business, the business often feels that way, but that’s not really what the IT wants.
If we can externalize as much of the process into declarative rules and declarative business objects, declarative model business objects, we can let the business transform and adapt those things in their own time, at real time, without technical involvement, but also it’s important because IT budgets are being slashed and it’s also important to know that IT can now focus on delivering new capabilities and focus on delivering those capabilities as opposed to delivering tightly coupled applications where business logic and data are coupled and making it hard for business to make modifications in runtime.
Yes. So, we’ve been out for a couple of years, the company is now just about three years old, we now have, like I said, sites on four continents, we’ve done this under the radar, stealth, we’ve just been working with our network, finding customers that we knew had the painpoints, people who were on the edge, who wanted to innovate and had these problems about rigidity, inflexibility in their processes and they wanted to cure them. So we’ve targeted those sites with acute painpoints, successfully implemented software with them. We have around a half a dozen sites at this point implemented, with another half a dozen on the way.
We did just announce a partnership with a company in the UK, called Tribal. And Tribal actually is an interesting company, they dominate 75% of the higher end market for research and higher end software solutions. And they found us actually, because they wanted, if you can think of it, research is sort of the apex of knowledge work, so we’re talking about environments that are very event driven in, where you actually pay researchers for their discretion, and you want those processes to adapt around their research studies, so they actually identified us as the best potential technology for their customers and now we are working with them, so we’ve recently announced that, it’s actually on our website and their website.
Yes, so it’s applications for research institutions, so these are applications that manage their budgeting processes, their research proposals, their human ethics and animal ethics, researchers are not only highly dynamic environments, because it’s long running projects, people coming on and off of them, with the scope of work might be changing, so there’s all this kind of typical project management challenges, but it’s also highly regulated environment where anything you do with an animal, a human, anything that touches potential conflict of interests, export import controls, anything that touches money, it’s all regulated. In addition, the regulatory environment is constantly changing, so you actually have to have a software that can accommodate that change.
Eric’s full question: So that sounds like a very across the board, very general purpose technology that can be used for any type of business process or any type of business problem. What do you say to companies that already have implementations of software or applications that meet some of those business problems or business requirements, is there a place for you guys to come in and add value to an existing environment, is there a place to come in and help people modernize their environments, I think we talked a little bit about this the other day, work on the edges of things, to help make better abstractions of the interactions between the business applications and the existing systems? Do you see yourselves in that area?
So, we don’t see ourselves as just a greenfield solution, we actually understand the need to work with an enterprise in an existing environment, with their existing systems, in fact all of our customer sites are already involved integration with LDAP, with HR, finance etc. So you have the ability to build apps that you were never able to build before, apps that can adapt on the fly, you have the ability to extend existing applications by just integrating them RESTfully into our platform, you have the ability to improve governance, because we’re actually injecting governance at runtime, there’s also some opportunity, it’s a long term view, but for some organizations that are already starting to feel middleware bloat, we do actually present an opportunity to do some infrastructure consolidation as well.
We understand that’s more down the road, but it is an opportunity, just think about the massive infrastructures some people have and how much that costs them to license, to deploy on hardware, to the maintenance and running of their ESBs, their business prospects management systems, their business rules systems, their complex event systems, their master data management, indexing, we’re doing all of that in one unified environment, so there’s an opportunity also to consolidate. But we don’t lead with that, the idea is we’ll extend your existing apps, we’ll give you capabilities you didn’t have before and if over time you think it’s appropriate, you can grow into our framework, it’s an open extensible architecture.
Eric’s full question: Certainly we had a lot of those challenges at credit Suisse where I was working just until recently, and another thing we would’ve done would be to think about what would be our standard technology stack, what would be the technologies we’d want to use when we built applications and we were in a process of trying to consolidate our own applications into a common set of technologies because we had a lot of, you can say we had more technologies than we needed, more applications that we needed. And we were also thinking about things in the future for cloud computing, investing in a private cloud. How would you see your work and your product and your technology fitting into that kind of whole evolution of, let’s try to rationalize, we talked a little bit about that, the existing environment and get it ready for the future, meaning cloud computing, which I guess everybody is moving towards, big data, some if these trends that the IT departments are all considering?
That’s a great point again. Not to give them a plug, but I did have a conversation today with Janelle Hill at Gartner and literally she said that the biggest movement, obviously as you are describing there is a convergence going on, right? There’s is an explosion of targets, data targets and devices, there’s an explosion of data sources and then you have this huge middleware explosion from methods in between, for integration systems, process systems, etc. and what we are doing we are coming into the middle and we are offering the opportunity to consolidate around there, you need to, you it’s open and extensible by definition, but I think what Janelle pointed out today is that the future is actually unified architecture, unified object models, we take that one further because we decompose the object into resources, which gives us more flexibility at runtime, it’s a complete system, uncompiled system, extremely binding.
Yes, I mean people have been struggling working on this problem for a long time; we’re not the first ones to look at real-time interaction processing. Historically, there have been two problems: one, performance, because of all the indirection in their middleware systems etc. it’s been too slow because you are asking a lot of the system, you’re doing complex event processing, with rules, injecting into processing and you want it to be millisecond, if it’s longer than millisecond the business users just won’t use. So performance has been a big issue historically, the other issue has been governance and management, right? Because when you make these generalized architectures using like we are, fusing web architecture and functional programming, those are pretty heavy ideas and unless you surface them through a declarative interface, you are going to leave it to a reserve of high end developers, highly talented developers with specific expertises and then you are opening up a risk or liability that “what happens when this person leaves” or “will I be able to keep up this resource” and so what we are doing is we are actually using those kind of technologies, functional declarative technologies, we’re using in a way that delivers the value of metaprogramming to the frontend web developer and to the business user. And I think that’s the goal really and the direction of IT.
Sure. It’s all of the above but it’s actually nicely done because we have one abstraction, everything is in a resource, so that means all data is a resource, all business logic is a resource, all user interface information is a resource, and resources just means documents with URIs or tags or links on them. So everything just exists as a document that gets bound at runtime, so you think of that, that causes a virtual information repository, so we’re using that web style architecture to create a singular layer, it’s a virtual information repository, where all your business logic, UI and data coreside even if it’s external what we are doing, so you have external systems and devices, we’re storing resources that act as adaptors for those. So it creates a virtual environment that is extensible because everything is represented in that one space, business objects as well as development assets. What’s nice about that there is common methods now to do anything against that, so there’s enterprise-wide search, enterprise-wide search for developers against application resources, enterprise-wide search for business users against all of their records, with common methods to do that, so it’s very easy to learn, for developers and actually analysts, there’s a modeling environment and that modeling environment is innovative as a technology itself, we have to present that in a way that’s successful, right?
Yes, so you have to make it comprehensible. What’s nice about this system is we can break it up into a common metaphor for an IDE, which is those kind of panels and then we actually have some nice additional features because everything is just a resource and they’re tagged with their relationships, we can do also instant on the fly on the dependency mapping, also because everything is loosely coupled that’s a reactive architecture, there’s nothing compiled, you’re actually simulating as you go, there is no build time in the system. So in this modeling environment you are actually building applications from links and meta data references.
19. […] What would be the starting point typically?
Eric’s full question: So what you’re saying is the starting point would be getting some information from a system, trying to make a decision on an investment or decision on a type of risk that might be involved in an investment or maybe make a decision on a purchase based on some historical factors and then start with maybe getting some information around some analytics, some type of decision support, used to be called information work where people were trying to figure out where to put their money, what to do with their business plan and then maybe feed that into the development of the application that would start carrying out some of those things. What would be the starting point typically?
You can really start from anywhere, start from what we call adaptive case management, and adaptive case management is essentially think of it as a case that has no rules against it, essentially it’s a null set, and so you can actually now build data into that unilaterally, the business user can actually say “this is the data I want to collect” and then the connection will link out to resources in the system, sort of like object relations, and they can build up constraints by relating themselves to those resources so that’s all starting from the null set and you can actually manage it for very dynamic environments, they are hard to model that’s one way to just actually manage your business, so if you’re in emergency response, how do you model emergencies?
You can only model them loosely, you know the kinds of events that occur, you know the kinds of information you might need, you know the kind of application resources you might have to connect with, but you can’t actually say when you’ll need them, what sequence you’ll need them, how many you’ll need them, how often you’ll need them, so adaptive case management allows you to build that on the fly to address your problem. The other way you can do it is just start modeling things in the IDE, but you can do that more liberally, you can do that loosely, because you are not compiling, you are not building, you can actually build an application, have people test it, get feedback immediately, so now it’s actually an Agile IT architecture that actually lines up with the Agile IT movement. So you can actually truly be Agile, because otherwise if you are just doing Agile practices as it’s done by a lot of organizations that really means just down and dirty coding really quickly, throwing away some best practices. I know that is not what is intended by Agile.
Eric: Agile is unfortunately one of those things that has come to mean many things to many people, its practices are interpreted very broadly and variously, absolutely.
As it’s implemented. I don’t mean to, I’m not debunking Agile, in fact we’re firm believers in it, we just think it needs to be mated with an IT infrastructure that enables it. It can’t just be method.
Eric: It hasn’t really been the cases before, it’s been taking existing IT approaches and making them Agile, I guess, or existing IT tools and use them in an Agile methodology. Now your tools are implementing some of these things in an Agile way that would actually support these processes.
To actually allow you to iterate very quickly, to let you make the modifications, reducing the expense of change, because that’s ultimately what it’s about, you have this tension between matching needs and reuse, right? So, in our system by generalizing objects into resources we allow them to be dynamically configured, so we can reuse resources and we’re essentially virtualizing surface interface actually by taking away the interface, change the interface into rules, we allow those rules to be dynamically configured, we’re actually allowing those resources to be consumed in interesting ways and maybe serendipitous ways for different uses cases, you can still plug controls in it, you can through the IDE, you can always layer in rules that actually will restrict the use of resources. Now obviously, not everything is in flux all the time, but if you can’t adapt you fundamentally lose that Agility.
Eric’s full question: Right. I remember listening to a presentation, this is kind of back on the topic of how you get started, how do you view this in the context of your enterprise challenges; I remember seeing a presentation from a guy from Travelers once. He was talking about SOA and he made the point that when a new order arrived for a new policy it involved a state change to the business and they were trying to model all the activities after they received that state change to understand the effect on all the systems and try to see where they had to go to tie things together better. Would you see that as a type of entry point or type of application that you can output?
Yes, for sure, because the enterprise is littered with static schemas, so you have all these interfaces, flow charts, ERDs, and they’re all rigidly structured, right? When you want to make changes in one place as you just described in one rule, well changing that one rule doesn’t necessarily change your entire process.
Yes, because our way is completely schema less, the schemas are actually dynamic ontologies that are compiling, it’s generative so they’re being generated, we’re deriving the interface for that interaction, now we give you controls to the rules, if you tighten the rules you can make things more procedural, also importantly, remember we said there were two fall backs to dynamic environments, one has historically been performance and the other one has been governance. Not only can you apply governance, you have rules applying dynamically, you can also get a full trace, so we are actually providing traces, because we are actually storing the diffs every time after a resource changes, because it’s a stateless system , we’re actually updating these resource states continuously and we are indexing those continuously, so every time state changes we’re updating that, imagine those diffs, we can actually look at every single change in an application for its history and we can even roll it back, which actually gives you some advantages with BASE as well, right? So you have ACID and BASE.
Eric: Another big debate now in the industry, which way to go.
You really need to be both.
Eric: You have either option, sure.
In a sense there is a trend here, right? You have to be able to support governance and dynamics, you have to support both ACID and BASE. Well, ACID is great for transactions and data base transactions, but not all interactions are data base transactions, because business process is long running, collaborative, by its nature it is BASE, so we’re giving you one system where you can actually model the actual degree of reliability, if you want ACID you get ACID, if you want BASE you get BASE, and what you do is model BASE through process, through coordination and you’re actually say “ok, we’re allowing condition where two people have something at the same time, at this point it’s a running business process, how are we going to reconcile that”, well we can merge it if there is some logical way to merge it, that’s no problem, but in the absence of a logical way to merge it, we can actually issue that as a task to another person who brings that together and of course we can automate that merge for them and let them actually make the determination of what the final state of their record is going to be.
Eric: Often, it’s going to be just the visibility of the issue that needs to be raised to make sure it gets resolved rather than having the system do everything itself.
Yes, and isn’t that what people want?
Eric: I think it is, I think we get into a trap in the IT business, by figuring we have to do everything automatically all the time and it’s not the case.
Yes, and we don’t mean to be but it’s patronizing, it’s this idea, it’s this presumption that we can code everything and at some point if we just took enough time it would be perfect, but of course it will never be perfect no matter how much time you took, you’ve got all that technical debts accruing from the moment you received the requirements so what you want is to be able to adapt with control. Another nice aspect of the system is we see all those dependencies, so if there was an issue the system can actually call out issues, it can actually interpret what might be an issue, it can see what services and resources might not have been available, it can highlight those to you, it can show you all the resources that where consumed in an interaction so it’s always bringing everything in context to the persons whether that’s a developer who actually now has to fix something or enhance something or whether it’s to the business user who just wants to better understand the universe.
Eric: A lot of times more than half of the problem is just finding out what the problem is.
Yes. It’s maintaining things in context and as long as you have this separation between design time and run time, you have this gap in context, what we really need in systems, and this is a drawback of conventional SOA and conventional REST, what you really want is a shared understanding. In SOA the understanding is on the server side, on REST the understanding is on the client side. What you really want almost goes back to Turing and this notion of a choice machine which is actually the idea is that sometimes you have to kick that out to the human operator.
Eric: Right. Absolutely, and I think that’s actually the case.
And that’s what we are having, interruptive processing, agent mediant, bring the right information to the right person at the right time but that’s automation, we need to understand the limitation of automation and allow those business users, where they have permissions, to configure that.
Eric: Sure, absolutely, people sometimes want the right answer rather than what somebody to decide what to do in a given situation.
You pay people for their discretion. And what we are doing is we’re actually enabling what I think is the big shift of the 21st century is a move from proceduralism to goal orientation, static anything, static processes, proceduralistic, they’re actually driving you to do things, you’re just the recipient, you’re a human API, do this now. Companies are very conscientious of their labor pool and they pay people for their discretion, they pay them to achieve goals and objectives that align with their strategy, then why not allow them, where they have those rights, give them permission to actually achieve those goals, let them focus on the goal and deliver business values as opposed to gratuitously follow procedures, that’s almost the definition of bureaucracy. Every time you’ve ever interacted with a company, a phone system that is ticking you through prompts and you don’t get the answer or a clerk on the other side or a service representative on the other side who won’t listen to your grievance, doesn’t understand you, doesn’t try to understand your specific circumstances, you perceive that as bureaucracy. Of course now you can’t always please every single customer but if your systems are static then you don’t really have the opportunity to do it.
Yes, fantastic, please go to our website http://www.ideate.com, means to think, to imagine, and also call us to see the demo. This is very conceptual, we’re in a time where technology is getting away from a lot of people, people don’t understand how their iPhone works, people don’t understand how their GPS works, people don’t understand how Google processes their queries. So conceptually this is very high art, but when you actually see it, it’s very understandable, because to a business user they can really actually appreciate that, “hey, this is the information I need to make that decision and this is the flexibility I need to meet my goal”.
Eric: Sounds like something you really have to try, to try out to get a good idea about what it’s all about.
Yes. And at this stage we are looking for companies that have edge use cases and we are partnering with them, we’re giving them extended periods of time to trial the system. We don’t want anybody to feel like there’s risk involved with this, if someone actually has a sincere need and they’re struggling, they know the problems, and they know the current technologies aren’t solving the problems, we want to work with them, that’s actually our ideal customer. So want to take away technology risk, so I think that’s another thing that keeps people on the fence because the cost of change is too high for them. And I think once somebody see it, 30-60 minutes, they can understand it.
Eric: Dave, I want to thank you so very much for taking the time to talk with us today at QCon 2012 in New York. Thanks again and hope everybody found this as interesting as I did.
Great, thank you very much, Eric.