In a retrospective look at the Scala based web framework Lift, David Pollak has announced a new initiative for "the job of managing the piles of data that come into your application":
There needs to exist a unified model for building real-time interactive web applications from the persistence layer, through the messaging layer, through the business logic layer, out to the client. Lift is part of this equation. Goat Rodeo will become the other part of the equation.
"Goat Rodeo" will utilize a number of exotically named Apache projects and leading edge concepts:
- The Hadoop based ZooKeeper project for distributed transactions.
- The highly scalable, eventually consistent, distributed, structured key-value store Cassandra for persistence.
- Support for the Software Transactional Memory concept through use of Scala's for comprehension.
The fundamental data structure in Goat Rodeo will be known as Q's - quanta of information, which will be compiler checked and available for consumption by any language capable of understanding JSON.
David's vision for the system is to provide fluid data movement between layers and across processes:
- Scalable persistence
- Distributed actors
- Synchronization of data and data model from browser through long term storage (yes, I'll be working to make sure this works with Lee's jsync.)
- Unified interprocess communications, even across heterogeneous processes
- What I believe will be a very scalable system for social networks and other social, interactive web apps to build on top of
Goat Rodeo is still at an inception phase, but David hopes that it will be available for experimentation by the end of Summer.
