David Pollak, famous Scala advocate, wrote a blog post, "Yes, Virginia, Scala is hard" that has caused a bit of a brouhaha in the Scala community. The post claims that Scala tries to do too many things, has poor IDE support, has an intimidating type system, and more. Ultimately David's assertions question Scala's broad appeal and if Scala will ever be the next Java. The Scala community seems to be upset with what they perceive as attacks on Scala. David asserts that he is doing his best to preserve Scala success by addressing its issues publicly and level setting where Scala fits in the market. InfoQ caught up with David Pollak and another Scala advocate, Dick Wall, to discuss the adoption, success and future of Scala and to discuss this blog post, as well as related topics like Ceylon.
David's position actually seems a bit of a contradiction to some as Scala has really picked up steam since 2009 with several high profile Scala projects, including, Twitter, Foursquare, and guardian.co.uk. One can almost track the adoption growth after coverage of Twitter's high profile usage of Scala. Twitter continues to shift more of its code base to use Scala.
David's Scala credentials make his opinions on Scala important. In addition to creating Lift, David hosted one of the first Scala conference in 2008, the Scala LiftOff, and he continues to run that conference to this day (the London Scala LiftOff runs October 13th and 14th, this year). He is also the author of two books on Scala: Introducing Scala and Beginning Scala, and has taught hundreds of Java developers Scala. David is one of the top Scala developers, he has worked on many of the very projects that had such a propulsive impact on Scala's adoption.
David's goal was to make Scala a top tier language like Python. His post was about his frustration with the pace of this goal. "This Spring, I gave up that fantasy. I realized that Scala was never going to become as popular as Python." He holds hope that Scala has the staying power of Smalltalk or Lisp as a second-tier language. His worry seems to be that Scala will not hit that goal because Scala is being hyped as a Java replacement. David goes on to explain his position to InfoQ
While I agree that Scala-the-language has some tremendous technical merits and I agree that there are some excellent and innovative libraries in the Scala ecosystem, Scala (the language, the tool-chain, the ecosystem, nothing about Scala) is not mature enough to be a Java replacement and barring an order of magnitude or two more investment into Scala commercialization, I don't see Scala becoming a Java replacement.
So, if we take a Crossing the Chasm approach to Scala, Scala has not yet crossed the chasm. Right now is a particularly tricky phase in Scala's lifecycle. If Scala over-promised and under-delivers, there's a risk that Scala will become tarnished with a bad reputation. If there are too many tried and failed Scala projects, Scala will become tarnished. My goal is to slow down the approach to the chasm and help find one or maybe two beachheads that Scala can dominate in so that when it's time to cross the chasm, there will be a lot of success points.
I am committed to Scala's success, but I think that success will be limited, although it's entirely possible that Scala will fail completely (like Forth or Groovy... yeah yeah... Groovy's not dead yet).
He then gives examples of companies like Morgan Stanley in London adopting Scala heavily. This type of adoption could be a great asset for Scala's adoption or it can be a touchstone of regret that people point to when deciding to use Scala. He fears that Morgan Stanley might have a hard time finding the right Scala developers for their projects, and Scala skills are rare. If, for example, unguided recruiters recruit nascent Scala developers, Scala projects' success could become difficult. David laments,
"If they wind up with a few strong Scala developers who can do the right thing in terms of spreading good Scala design practices and good institutional process for learning, sharing and growing, they have a good chance of being successful. On the other hand, if Morgan Stanley fails with Scala, there'll be a big hurdle in the finance community to get another large Scala project off the ground."
David fears that since finance is a sweet-spot for Scala, the failure of a few key projects at a signature financial institution could spell the end of Scala adoption overall.
Dick Wall, of Java Posse podcast fame has been an outspoken Scala advocate. Dick discovered Scala three years ago and has been using it as his primary language for two years. Dick says he was overjoyed with strongly typed Scala's brevity and expressiveness, the level of which is usually associated with dynamically typed languages like Python, Groovy and Ruby. This seems to be a coveted feature of a whole new breed of strongly typed languages for the JVM. Dick has been busy in the Scala world. He wrote SubCut, an open source Scala dependency injection solution. He is also a contributor on ScalaTest. Dick also started a consulting firm focused on Scala training and consulting with famed Scala guru Bill Venners, which seems to be well received. Dick provides a different perspective on Scala than David.
InfoQ: Have you read David Pollak's "Yes Virginia" post?
Dick Wall
Yes, but I would choose to expand the title to "Software Development is hard" or perhaps "Vigorous software development is hard". When you set out to complete a project or write a system, you have a problem to solve. Chances are that if it is something good and new, it's going to be pretty hard. The complexity of the delivered item will be dictated to some degree by the problem to be solved, and that complexity bar will be about the same height no matter how you tackle it.
Choosing a language with more power is the first way you can get a boost on reaching that bar. Choice of libraries is the next, and the remainder you fill in yourself. In Java, the power is (by modern standards) fairly low, leaving a larger gap to reach the bar. Most people fill in with libraries, e.g. JPA, Wicket, Spring or perhaps full blown Java EE. These bring their own significant complexity to the project (not to mention their own learning curve). Then the work begins on the final part, the custom work necessary to reach the bar.
If you are writing something like a web application, the chances are that the libraries available (of which there are many in Java) will get you almost all the way there, albeit with a significant investment in learning the libraries the first time you do it. If the task is something a little less commonplace, perhaps a scientific or mathematical project, or just some totally new idea or approach, you have even more to do. At this point you want the most power, flexibility and expressiveness you can get, and that comes back to the language you choose.
As for Scala being hard, that's where I disagree with David. It can be hard, it can be as hard as the problem you have to solve, which is a benefit in my opinion. On the other hand, at recent BASE (Bay Area Scala Enthusiasts) we have seen demos of Kojo - a full Scala environment embedded into a learning environment for kids that looks a bit like Logo, the learning environment from years back) and last month a demo by an 11-year-old who had written his own version of Conway's Game of Life. Now the kid in question is pretty bright, no disputing that, but it still demonstrates the accessibility of the language. It comes down to the idea that complexity is the function of the problem you are trying to solve more than the language you are using to solve it.
InfoQ: What are the biggest conceptual hurdles that Java developers have to overcome to learn Scala?
David Pollak
Immutability. In Java, coders approach most problems in terms of changing some kind of state and querying some kind of state. Beans, getters, setters. This is conceptually similar to assembly language that we're pushing bytes all over the place. Scala embraces, but does not enforce, the approach of transformation (you don't change the thing, but given a thing, you return a new instance of the thing with the changes you want). A function transforms input to output with few or no side effects (no matter how and when you call the function, if the input is the same, the results will be the same.)
Developers often have lots of challenges wrapping their heads around immutability.
One of the problems with Scala is that it does not enforce immutability like Erlang and Haskell and Clojure. So, it's super-easy to fall back on Java conventions... it's easy to do things "The Java Way". Unfortunately, when you do things the Java way in Scala, you lose a lot of Scala's advantages and you also get caught up in Scala's disadvantages (e.g., documentation and poor IDE support.)
So, with Scala, unless you have a lot of discipline (either internal self-direction or a great team around you), it's easy not to realize the benefits of the language. Further, because there are so many paradigms floating around in Scala-land, it's really hard to figure out if you are doing things "the right way."
To be concise, the biggest hurdle a Java developer has to overcome learning Scala is not falling back on their Java mutable ways. But there are plenty of other hurdles (the type system, the documentation, lack of well defined patterns, etc.)
Dick Wall
There is a lot to unlearn from Java. Java developers are not aware of it, but Java has more than it's own fair share of lore and in-knowledge. Someone designing a language from scratch probably wouldn't have static methods and fields, for example, when class meta objects make more sense conceptually. Anonymous inner classes bring mostly the same features, and problems, as function literals and closures, but with a less convenient syntax that confuses beginners.
Of course there is the extra hurdle of learning the functional features Scala has to offer, but you don't have to do that right away - start sprinkling a few of those ideas in as you go, and see how you like it. To start off with, you can simply write Scala as you would write Java, and then the main hurdle is learning what you can leave out rather than what you have to put in.
InfoQ: What are some of the biggest complaints you have about Scala IDE support?
David Pollak
Slow. Broken. Memory Hog. Poor autocompletion. Weak code navigation support. Weak refactoring support. Inability to open large files or large projects. Just to name a few.
Dick Wall
Firstly, Scala IDE support has improved greatly in the past year. At present I use IntelliJ IDEA as I judge it has the best support right now, but both NetBeans and Eclipse have good support, and Eclipse is improving rapidly as a result of the involvement of the EPFL team on that project.
Looking at the state of the art in IDEA tooling for Scala, we already have support for code completion of all of the major features, including implicits. The syntax coloring is excellent and brings an extra dimension in an expressive language like Scala over what you would see in Java. There is refactoring support, some real time error highlighting (although this is not complete yet) and debugging support.
The last two are the weakest parts right now. The error highlighting works in some cases, but in others you will only get errors when you try and compile. Debugging also has some glitches, the primary one being that sometimes breakpoints get skipped when placed inside closures or function literals. Both of these are just annoyances though and can be worked around easily.
Lastly, everyone always complains about it, but the speed of compilation is perhaps the biggest bugbear. There are clever ways to work around it most of the time, like fsc (the fast scala compiler which works like a server and stays running) or sbt [build tool for Scala], but ultimately the scala compiler just has a ton of work to do. EPFL appears to be aware that this is a big issue and keeps working on it.
InfoQ: Is it hard to find developers to program in Scala? Why?
David Pollak
No. It's easy to find good Scala developers. It is hard, however, to find large swaths of Scala developers. I can put together a great team of 10 Scala and/or Lift developers next week to drop on a project. They won't be cheap (average cost $250/hr plus whatever markup I add.) They won't be local. But they will be excellent.
Foursquare and Twitter and others are hiring excellent developers of all stripes and generally making them excellent Scala developers.
What is difficult is to find an outsourcing team of 25 Scala developers at $40/hr fully loaded. You can find plenty of PHP and Ruby and Java developers at that price, but few Scala developers at that price.
It's a supply and demand issue. The supply is small because the demand is small. There's a positive virtuous cycle in terms of more Scala developers becoming available. But the quantity of available Scala developers is in the hundreds, not the tens of thousands. Unless Scala reaches the mainstream (like Python and Ruby and PHP and Java and C#), there's no economic incentives for the body shops to retain Scala talent or sell Scala projects.
Dick Wall
We have not found it that hard to find developers in Scala. There is an interesting effect with new languages, that energetic and dedicated engineers tend to seek them out and learn them, and then look for jobs in them. They tend to be self-starters, which is what most companies at least claim to want from developers. If you want to find people to work in Scala, you could do worse than check out your local Scala user group.
InfoQ: What are some advantages that Lift has over other web frameworks?
David Pollak
More secure. Faster. Better for writing highly interactive sites (lots of Ajax and Comet). Strongly typed (fewer run-time errors). Concise. Here are articles and some videos that discuss using Lift and how it changed developers view points on web development. (video)
Dick Wall
Lift is an interesting twist on web frameworks in that it treats the pages to display as node sequences of XML data that you pass through functions to get the results you want - it's a very functional way of looking at the world. The Comet support using Scala actors is also very clever.
Other libraries I like are: Play - for a web development approach that will be more familiar to many developers, Squeryl for Object Relational Mapping, ScalaTest for anything testing related, Akka as the nucleus for an enterprise framework solution, Borachio for test mocking, and SubCut (of course) for dependency injection and service location. All of these offer great expressiveness over Java equivalents by taking advantage of Scala features, particularly for construction DSLs (domain specific languages).
InfoQ: David, you mention in the article that you feel that Scala is really aimed at teams with 95 percentile skill levels. The top five percent of developers. How hard would it be for companies to attract such developers? Would it be worth it?
David Pollak
I was pretty flip. In my follow-up post, I tried to be more precise. I think Scala can be very successful for companies that recruit from the top of the talent pool. I quantified this as the top half of Java shops. It’s not a 5% of the pool thing. It’s a 50% of the pool.
Depending on your company, your pay, your incentives, your management, it may be easy or hard to attract top talent. Google attracts top talent every day as do SalesForce.com, Twitter and Facebook, etc. On the other hand, most companies, by definition, cannot attract the best talent. The pool is limited.
(Would it be worth it?) It depends. There are plenty of fill-in-the-blank, Spring ORM kind of apps that are not challenging or exciting. Recruiting folks who groove on a stable work environment, getting home in time to spend time with their families, and working in a low stress environment are great to have if you have a well defined, slow moving company and market. Writing back end systems for grocery chains or health care companies is oriented to project longevity, maintainability and low risk. In these environments, using tried and true languages and tools that will lead to predicable results is a good thing.
I think Scala is technically superior to Java, but it's not superior enough to overcome the predictability of using a known solution. Scala is not "better enough" as a language to overcome the institutional value of using Java in slow-moving projects. Yes, I could imagine a scenario that in 15 years, Scala has replaced Java, but that scenario is a low-likelihood scenario and selling that scenario today will do nothing to help Scala adoption.
But I think it’s more likely that Oracle wakes up and clones C# into Java or that one of the less technically aggressive Java replacement languages will become the mainstream while Scala will, hopefully, be technically excellent and push theoretical and practical boundaries.... It also took Steve Jobs' 18 years and force of will and a platform paradigm change to make NeXTSTEP a part of the mainstream. Will Scala last until its technical superiority is in the mainstream and that technical superiority is called Scala? It’s a long-shot bet.
InfoQ: Is Scala worth the effort for average developer teams to learn? Do the benefits outweigh the complexity and learning curve?
David Pollak
It depends on the project. If all of your concurrency is done in the database, then no.
If you're doing some form of event processing (trading floor, sports betting, near-real-time data analysis, social networking), Scala is a huge win over Java. If you've got complex, distributed systems, Scala and immutability is a huge win. In these scenarios, the costs of using Scala (learning curve, poor tooling, etc.) are small in comparison to the benefits of Scala (immutability, composability, good event processing, excellent libraries/frameworks that provide a starting point for these kinds of systems.)
The answer to the question depends on team inclinations and project type. The key thing, getting back to my top-level goal (more Scala success), is to use Scala in places that benefit from Scala's strengths.
Dick Wall
The problem is where the complexity is (or isn't) and the language is a tool along that path. If you have a very complex problem to solve, then Scala is worth every effort to use it, as is any other tool that you can bring to bear on the problem.
In addition, Scala represents a feature set that I would expect to see in any new language. Whether Scala ultimate "succeeds", whatever that means to whomever says it, learning the feature set will place you well for learning and using other languages, so what is there to lose?
InfoQ: Can a person be in the top 5 percent and still not like Scala?
David Pollak
Sure. There are tons of Smalltalk, Lisp, Clojure, Ruby developers who are not going to like Scala's static typing. Charles Nutter is not a Scala fan, yet he's one truly amazing developer.
Dick Wall
Of course. In this way languages are akin to wines, and people's tastes vary; there are no perfect wines, but there are wines that you like and those that you don't, and others tastes may vary. I suspect that if you poll many people working in Java it might not be their favorite language, or perhaps in some cases one that they particularly like, but it works and it is a de facto standard.
It also has little to do with being in the top 5%. Just like with wines, perhaps a wine critic can offer you some guidance, particularly if you establish that you have similar tastes and like some of their recommendations from the past, but ultimately it's what you like that matters. Just jump in and try it, you will know soon enough if you like it or not, and what do you care what other people think?
InfoQ: In theory if not practice, languages like Ceylon and Kotlin attempt to clean up some of the same problems that Scala does for Java without the complexity of Scala or, one could argue, the power. Do you feel Ceylon and/or Kotlin can provide value to teams who are not all at the 95 percentile level? What are they missing to target the mainstream while still providing value?
David Pollak
I'm going to wind up (upsetting) everybody off here...
Kotlin may have the best chance of being the next Java among existing JVM languages for one simple reason; it's built from the IDE out and the vast majority of the target audience (Java developers) are IDE users. While I'm not super-keen on the UI for IntelliJ, the JetBrains guys have an amazing ability to touch all the right places in Java IDE users' brains. And at the end of the day, I'm betting the JetBrains guys will be able to deliver a better Java than Java with a tremendous IDE backing it.
None of this is to say that Kotlin is technically excellent or theoretically sound, or will attract the broad range of library developers that Scala has attracted (Scalaz to Lift to Akka to specs). Scala has the broadest range of crazy-smart folks who approach problems in novel ways of any language or platform community I've ever seen, including the NeXTSTEP community.
Back to the question, if there's a bunch of Spring extensions for Kotlin and it's got solid documentation and excellent IDE support, I think it's got a very good chance of winning the "next Java" prize.
Oh... and I have nothing good to say about Ceylon.
Dick Wall
I am adopting a wait and see attitude to the languages. I learn constantly and look forward to giving these languages a try when I can. In the meantime, knowing more about Scala can only help me when I try them - since many of the features are similar.
I think the 5% and 95% division is arbitrary and I disagree with it strongly. The language either works for you and your team or it doesn't. If it makes you feel better to think, "I like Scala so I must be in the top 5% of developer" knock yourself out, but I would advise against believing your own press on that one. I have a bigger problem with people deciding that others are not smart enough to learn a particular language.
Again, the best advice I can offer is give it a try for yourself and see if you like it. If you don't you have your answer, but don't presume that everyone else can't or won't want to use it either. Likewise if you understand everything about functional programming and monadic programming, it doesn't mean you have to use that as a club to beat everyone else with until they learn it too.
InfoQ: One of the main advantages of Scala over Python, Groovy, or C# is that Scala fully embraces functional programming, yet with syntax that is closer to Java than most functional programming languages. Clojure also fully embraces functional programming with a syntax that is more like traditional functional programming languages. Are there any advantages to using Clojure versus Scala? What are the tradeoffs?
David Pollak
I'm not sure Scala fully embraces functional programming. I think Scala fully embraces OO and has some really nice functional features as well as immutability. Although inheritance and immutability are near impossible to reconcile. But I digress.
I think the differences between Scala and Clojure boil down to static vs. dynamic, syntax (oh, look, lots of parens), etc. I'm not a fan of dynamic languages. I'm not a fan of Lisps. But I think that Clojure's References are possibly the most under-appreciated language feature I've seen in the last 3 years. References unify a lot of forced immutability much better than Haskell's monads.
On the other hand, the kind of things that you can encode in the type system in Scala are vast and powerful. And that kind of power, in the right hands, can lead to far more compile-time errors and far fewer run-time errors. That's a GoodThing(tm) during initial development, but far, far more important when the project is in the maintenance phase. The more the type system and the compiler can guide a developer to doing the right thing, the lower the costs and the better the long-run outcome.
Dick Wall
Clojure is a fine language, and I like many of the principles of the Clojure space as well, like Rich Hickey, Stuart Halloway and Howard Lewis Ship. I have sat down and made sure I know Clojure well enough to include it in my toolbox too. I had a fair familiarity with it already from working with Lisp in the past (particularly when I used emacs a lot).
Two of the tradeoffs I see with Clojure are that you don't get as much support from the compiler (it really depends which side of the static fence you stand on if this is a pro or con) and that even more so than Scala, Clojure programmers can, and do, create an extension language in their own image, so that when you come in to someone else's library or codebase there can be a pretty steep learning curve. Some big pros are the STM model (which is very clever) and an extremely condensed syntax.
InfoQ: Any thoughts on Erlang? Is there any overlap in where Scala can be applied and Erlang can be applied? (Actors)
David Pollak
Erlang does distributed, fault tolerant really well. Scala (even with Akka) doesn't. Erlang has excellent VM support for code distribution as well as data distribution. The closest that the JVM has is GridGain. Erlang is a win where Erlang has succeeded to date. Erlang has some excellent ideas that can be borrowed (and have been in Scala and other languages), but I don't see Erlang as high value in the general purpose programming language space.
Dick Wall
Scala obviously borrowed off of Erlang heavily for the actors library and also for some other features like pattern matching. Erlang has become legendary for reliability in the software development space, and I know that one of the aims of the Akka project is to learn from Erlang and its reliability.
InfoQ: In what use cases (types of applications) does Scala make the most sense?
David Pollak
Financial trading floors and sports betting. Stuff that requires performance, stability, and event processing. I used to think that Scala made most sense for social networking apps (e.g., Twitter, Foursquare), but with Node.js, the value of Scala for the hip crowd is much lower.
Dick Wall
Personally I would consider Scala for any project, and the web applications support and data persistency libraries have rounded out nicely in the last year or two. I find it suits mathematical and scientific applications particularly well though.
InfoQ: In what use cases (types of applications) does Scala make the least sense?
David Pollak
Database front end, CRUD apps. Most of the boring apps that are built with Struts and Spring. A lot of J/EE/Spring works well enough not to mess with and there's such a huge ecosystem of books and tools and training and contractors, that trying to push Scala into this space has, in my opinion, low value and a low probability of broad-based success.
Dick Wall
Scala might be a tougher decision if you try and integrate it tightly with an existing large Java project though. By that I mean Scala calling in to Java and Java calling back in to Scala a lot. It's not impossible by any means, but there is an impedance mismatch that can be tricky, and it's important to stick to rules and keep things clean.
That said, one great way to try out Scala with a minimum of risk is to use it to write your tests for an existing Java project. Testing code doesn't go out to production, and libraries like ScalaTest, ScalaCheck and Borachio can greatly increase your productivity when writing tests.
InfoQ: Where do you see Scala adoption in five years? ten years?
David Pollak
It depends on how Scala does over the next year or two. If Scala builds a beachhead in one or two markets and really owns those markets, I see Scala as having the kind of longevity that Smalltalk and Lisp have. If Scala "flames out" with failures, it will once again be a research language in 5 years.
Dick Wall
I don't really have a strong opinion on this, although I am pretty sure we will still have plenty of opportunity for training and consulting in it in 5. As for 10 years, who knows what kind of clever languages we will be using by that stage? I believe Scala is already past the "survival point" as a language, and will obtain more popularity over the next few years. I don't believe it will become as big as Java, but then I don't think any single language will do that again for a long time, if ever.
InfoQ: There are some who argue that whilst Scala is an interesting language, the syntax is hard to look at and understand. They complain about Scala's type system being more complex than needed, and having several language features that harm understandability and readability, and a ugly syntax ("bit of a dog's breakfast"). How do you respond to this criticism?
David Pollak
I've seen Scala's type system evolve over the last 5 years. I think every step in its evolution is necessary and valuable from a library writer's perspective. Look at what the Rogue guys have done with a compiler-enforced query system against MongoDB. Rogue is jaw-droppingly cool and so easy to "get right."
So, Scala has a tremendous type system that allows encoding of more checks than any other language I know of that's used day to day. The syntax of Scala's types are not ideal. Haskell, in my opinion, has a much nicer general syntax and has a cleaner syntax for describing types in most cases. On the other hand, Scala is unabashedly a C-syntax derivative language and C-style languages are naturally ugly. Spend 3 months doing Ruby or Haskell and then look at some C++ and tell me it's not "dog’s breakfast." ...
I don't think non-Scala developers think much about Scala's type system or its syntax.
Dick Wall
Going back to wine and tastes. I disagree with the statements over the syntax, which I find to be expressive and, in well written code, quite beautiful. The type system seems like overkill until you really try and do something clever with it, like the Scala collections (which are fabulous and showcase higher kinded types extremely well). That's an important point though - most developers won't use the full power of the type system for writing their own libraries, but practically everyone uses it in the collection APIs during the course of their normal development, and it is clear and consistent and very obvious what is happening.
As I always say, the best way for people to get a resolution on differing opinions is to try it out for themselves and make up their own minds.
InfoQ: Why do you think developers who program in Scala are so critical of Ceylon and Kotlin? You used to see the same type of passion for Ruby, but not so much from Groovy developers.
David Pollak
I have my thoughts, but none of them will help move the discussion along. Plus, I'm sure I've managed to upset more people in this interview and walking down this path would be gratuitous.
Groovy is the worst of all possible worlds. It exists because JRuby wasn't quite up to snuff when Groovy was born. But Groovy is ugly (like Java), has inconsistent syntax, has weak meta-programming, weak performance, junk for a type system, etc. ... I think that Groovy is in the space of, "Well... there's nothing better. It's not something to get excited about, but it's better than nothing." Kind of like plastic garbage cans. They dent less than metal ones and have very little else to get excited about.
InfoQ: What makes Scala developers so passionate?
David Pollak
Part of it, at least for me, is the sense of freedom from bad design decisions. My eyes were opened to a lot of language excellence (and corresponding bad language choices in Java) when I started using Scala. For me, that feeling of liberation earned a lot of passion.
I think that there are people in the Scala community that do groove on intellectual bullying. But that's a significant minority, although they are pretty vocal.
Dick Wall
Some people like it, other people don't, again I think anyone curious should try it and decide for themselves.
I must say I haven't really noticed this outside of a few individuals. Many of the Scala developers I know or work with just get on with development and don't even blog, must less criticize other languages publicly. All communities have their enthusiasts, and sometimes the energy gets a little high. My own point of view is that I can't wait to give Ceylon and Kotlin a try, but in the meantime my bread and butter is coming from Scala and I couldn't be happier.
InfoQ: Will learning Scala make you a better software developer?
David Pollak
Exploring new ideas and concepts and new/different takes on old ideas will always make one better. A great chef can learn something from McDonalds. A McDonalds food designer (or whatever they're called) can learn from eating at a five-star restaurant. For those who want to broaden their horizons and improve their coding skills, Scala is a great place to learn. But so are Ruby and Clojure and Smalltalk and Haskell and even C#.
Dick Wall
I believe learning makes you a better developer, no matter what you are learning. Going through pilot ground school had a beneficial effect on my programming skills, particularly understanding the importance of testing and automation.
Learning Scala is a great way to future-proof your language skills in my opinion. At present, I am learning Haskell because that doesn't let you cheat on functional programming, and is helping me round out my functional approach to development.
InfoQ: Can you give a three paragraph elevator pitch for adopting Scala, or at least point us to the best one that you know of?
David Pollak
I'm going to pass on this question. I haven't crystalized my ideas for a target market for Scala into the pitch. Sorry.
Dick Wall
In my opinion, for the right team and project, Scala will make you more productive and provide a more readable and maintainable code base. It can also give you a competitive advantage in speed of development. For problems involving a mathematical and/or scientific component, it is a very strong fit, and it's a lot of fun.
At the very least, learning Scala will familiarize you with the features you will probably see in other modern statically typed languages that will be released in the next few years, so what have you got to lose?
InfoQ: What do you think of recent updates and proposed updates to the Java language? Specifically, what do you think of Java's proposed Lambda syntax?
David Pollak
It's better than a poke in the eye with a sharp stick. Too bad it took 7 or so years to get finished and it changes the semantics of the Java language (the "return" keyword has become "sometimes optional".) Saying much more doesn't do anyone any good.
Dick Wall
The aims and the syntax of Project Lambda make sense for Java, in that they are an improvement in convenience of the features that are already there. I don't think they will satisfy all the needs people will have for full blown closures and first class function literals, but other languages provide nice alternatives for those anyway for people who need more power.
Interface injection, if delivered, will also be a benefit, and could help solve some compatibility problems Scala has between version releases. Any and all careful improvements are a good thing for Java, but perhaps at this point they are as much a stepping stone to the more powerful features of other languages than they are just an incremental improvement to Java.
InfoQ: Is there room for a strongly typed language that is easier to learn than Scala, and fixes the Java warts adequately? Ceylon, Gosu, Kotlin, Groovy, Groovy++? Or something else?
David Pollak
Sure. Once types can be as non-intrusive as they are in untyped languages, the whole world will be better.
Dick Wall
I don't know on this one - I will play the wait and see game.
Personally I find Scala as simple as I want it to be, or as powerful as I want it to be, and largely it comes down to taste how easy the code is to read in it. That's OK for me, and for the teams I work in. If something much simpler but still offering lots of power comes out, I will assess it, but I think something will have to give in either the simplicity or the power.
Another language I don't see mentioned above is Fantom. Some say that already is a simpler language that offers improvements over Java without the overwhelming power of Scala, but it hasn't been widely adopted. There are many theories for this, my own is that it maybe doesn't offer enough new stuff for someone to make the effort to switch. Scala clearly does add a lot of value, so that should be a lesson for newcomers. Offer enough new stuff for people to be interested.
InfoQ: It looks as if Scala is actually catching on in a lot of places.
David Pollak
Having a hundred or so job listings is nice. It proves that Scala's not dead. Yay. But 2009-2010 should have been that sweet spot in the growth curve. Scala adoption is and has been linear, but it's the same linear it was 2 or 3 years ago. So, there's a huge difference between "not dead" and "on the trajectory to be Python-popular." The Scala ecosystem supports maybe a dozen consultancies and that number is growing linearly. This is materially different from Ruby/Rails circa 2006. In 2006, Ruby/Rails took off. Scala and Lift and Akka seemed to be about 5 years behind Ruby and Rails in terms of community and adoption. But instead of taking off last year, Scala continued to grow nicely, but nothing like what's needed for Scala to become a long-term part of the programming language landscape rather than a source of research that others commercialize.
My goal is to get Scala and Lift into a place where adopters self-select (not "everybody" or "all Java developers", but the kind of organizations that will benefit from Scala's strengths) and there are plenty of success stories with very few failures. Thanks for the interview so that hopefully I can clear up some of the discussions around my posts so that people broadly understand my goals and my perspective.
Dick Wall
Yep, it's doing well. I didn't learn it because I necessarily believed it would be a success, I learned it because I liked what I saw and it solved my problems nicely. Now, I am glad that I did.
David responds to some of Dick Wall's answers.
David Pollak
Scala is hard until you get it and then it's easier and better than Java. But do not underestimate the cost of learning it either for individuals or for teams. ... Simply arguing that Scala is ultimately better, but ignoring the institutional costs of adoption, will not help Scala adoption in the enterprise.
David went on to explain the ultimate benefits can be negated when planning organizational adoption, and the cost versus benefit needs to be balanced, as well as the opportunity costs associated with long transitions.
David's position is that the thoughts of Dick mirrored the comments and responses to his post about Scala being difficult. He argues the logic is faulty ("hypothetical syllogism"). He sums up the logic as follows
David Pollak
"If your organization invests the time, effort and money to teach its employees Scala and transition to Scala then your organization will be proficient in Scala. If Scala is a superior language to Java then an organization that uses Scala will produce superior programs."
David argues the logic "Scala is a superior computer language to Java and programs written in Scala are superior ... than those written in Java" is flawed if you are considering using Scala in a large organization. His focus has been on the point "the cost of institutional change to Scala is very high". He asserts ignoring the cost and focusing on the benefit is a logical flaw.
To summarize a bit.
Is Scala is hard?
Dick Wall's perspective is you either spend your time learning a great language which helps you solve hard problems or your time learning frameworks that have to compensate for what he sees as the deficiencies of a language such as Java. Thus, he argues, the amount of learning curve is about the same. He also indicates Scala is only as hard as the problem you are trying to solve. Even if you assert Scala has warts, Java has its own warts and problems as well as pointed out by Dick. It may be more a matter of known complexities (Java) versus unknown complexities (Scala). David brings out the biggest hurdle to learning Scala is to forget the bad habits that Java forced you into.
Is a lack of good IDE support and big corporation backing hurting Scala adoption?
The tool stack according to David is just not mature enough for mainstream adoption. Dick points out that the support is quite good, but perhaps not perfect. IDE support can be a big show stopper for many organizations. Also if Eclipse does not support Scala well, for example, and you work in a shop where Eclipse is the sanctioned standard, it can be difficult to use another IDE that supports Scala better. Depending on the organization this could be a small roadblock or a show stopper. Good IDE support seems to be a must for wide adoption at the first tier language scale (C#, Java, Python, Ruby). On the other hand, Ruby and Python did not get good IDE support until rather recently, and yet they had major success that predated good IDE support.
David's points Scala needs a big corporate backer (Oracle, Microsoft and Sun) to be a first tier language. One could argue that it took Python (created in late 1980s) and Ruby (created in 1993) years to really catch on (2003 and 2005), and they did not have big corporate backers and are considered tier one languages. Thus, Scala might be on the same longer but sustainable curve as them.
It is hard to find good Scala developers?
David states it is easy to find good Scala developers if you can afford $250.00 hour rates or you are willing to hire top notch developers and train them. It is hard to find good developers at $40.00 rates. The potential of making $250.00 per hour alone could encourage some on the fence about learning Scala in their spare time (or propose it for a project) to go forward. Those rates could convince developers it's worth the effort as a sheer economic catalyst.
Dick states that it is not that hard to find Scala developers. He asserts the developers you find will be the energetic, dedicated, self-starter kind. One could also imagine that the developers that are early adopters could be the same ones that adopt the next hot trend, which could imply that less cutting edge languages could attract long term stable developers who are going to stick around a while.
Will learning Scala make you a better developer?
One could argue that if you really wanted to focus on developing your functional programming muscles, Clojure or Haskell would be good languages to learn without the distractions of Scala's other features. This could perhaps make you a better Scala programmer, or at least one that could think more functionally (programming). Although, Scala seems like a great place to learn what a modern strongly typed OOP/functional language can do.
Competitors around the bend:
Kotlin and Ceylon are inspired by Scala strong types with modern language features. The pressure of Kotlin and Ceylon could be pushing folks like David to question Scala's broader appeal. Certainly, a vocal minority of Scala developers seem to pounce on every announcement of any JVM language that is strongly typed with modern language features. Scala could see some stiff competition for adoption. Does a less powerful language with a more mainstream syntax, and good tools support that fixes Java's warts have a place? If so can it eclipse Scala adoption? Time will tell. It is here where Scala's strength (its power) could become a weakness due to a longer learning curve. On the other hand, Kotlin and Ceylon could become gateway languages to even wider Scala adoption.
Whether Scala is hard to learn seems to depend on your perspective, background and tastes. If you need a few frameworks to make up the difference in power, then you have to factor in the complexity of the frameworks to the equation. It does seem like Scala is here to stay in some shape and form. Scala has impacted Java's Lambda syntax. Will Scala be the next Java? Unlikely, but it seems like it has a shot at being a tier one language like Ruby or Python. It just may take fifteen years. Scala, with its strongly typed, modern programming language features, and few legacy warts, is perhaps inspiring a bumper crop of new competitors (eg Ceylon, and Kotlin) and some retooling of olds ones (Groovy++ and Gosu). David's points on macro adoption should be considered when migrating a large organization to Scala as should Dick's points on Scala's power and utility when sharpening your own skills.
About the Author
Rick Hightower has worked as a CTO, Director of Development and a Developer for the last 20 years. He has been involved with J2EE since its inception. He worked at an EJB container company in 1999. He has been working with Java since 1996, and writing code professionally since 1990. Rick was an early Spring enthusiast. Rick enjoys bouncing back and forth between C, Python, Groovy and Java development. He has written several programming and software development books as well as many articles and editorials for journals and magazines over the years.
