I, Analyst: Groovy: The Sleeping Giant

Groovy: The Sleeping Giant
It‘s become apparent that dynamic programming languages such as Ruby, Python and even Java/ECMAScript are coming into vogue for developers on the cutting edge. Ruby on Rails has done much to promote the Ruby programming language and Ajax is focusing more interest on JavaScript. Python hasn‘t found its muse yet, but its also gaining more attention. The age of dynamic programming languages is imminent. What I mean is, a time when dynamic programming languages are the norm, rather than the exception, is just around the temporal corner.
Within the next five years most organizations will use dynamic programming languages in some enterprise development. Not all development: There will always be a need for conventional programming languages which allow you to detect bugs at compile time rather than run time. However, the times when dynamic languages are dismissed as "dangerous" or "toys" is coming to a close.
While cutting-edge developers are flocking to dynamic languages like Ruby, the mainstream of the development community has been less inclined to adopt dynamic languages. There are two reasons for this: FUD and Productivity. Fear, Uncertainty, and Doubt (FUD) continue to muddy the waters for developers who are trying to grok the significants and use of dynamic programming languages. It‘s not unusual to hear people say that dynamic languages are "toys" or are "dangerous". What I‘ve discovered, however, is that these same people usually have little or no experience with a dynamic language. The FUD was born and propagated out of ignorance as is usually the case with these things. There is, however, a far more pragmatic reason why dynamic languages have not been taken up by the masses: Productivity. Learning a new programming language platform is not a small endeavor. While syntax of a language can be fairly easy to learn, learning an entirely new set of libraries is a huge obstacle. For example, Java programmers spend proportionally much more time learning how to use standard, enterprise, proprietary, and open source APIs than they do the syntax. In fact, I would say the ratio is something like 20:1 the first year and 100:1 thereafter.
If you‘ve spent all of your mental effort learning how to properly use Java packages and APIs (rather than syntax) then the thought of throwing all that time away in order to learn a completely new platform may seem ... well ... stupid. Or more accurately, its not a pragmatic choice when the advantages of the new platform are not obvious. Essentially, everything a Java developer knows about accessing databases, manipulating strings, doing input and output, etc becomes obsolete. In addition, the new platform may not have as strong of an ecosystem as Java.
In the case of mainstream Java developers, its seems less than optimal, for example, to change platforms to something like Ruby. Don‘t get me wrong! I‘m a big fan of Ruby, but to be perfectly honest compared to the Java Platform the Ruby platform is positively anemic. The number of APIs (libraries) and the size of the Ruby ecosystem pales in comparison to the Java platform. There are probably around 4 million or more Java developers in the world. I would be surprised if there were more than 100 thousand Ruby developers and probably only 50 thousand that use Ruby on a daily basis in production code. Of course that‘s pure speculation, the real numbers for Ruby might be smaller or larger but few people would argue that the size of the Ruby development community is anywhere near the size of the Java development community.
Dynamic programming languages (e.g. Ruby, Python, JavaScript) are generally much more productive than conventional programming languages. In my research, which now spans about 3 years on the subject, I‘ve consistently found that developers are about five times more productive with a dynamic language than they were with a conventional language in the same context. This means that once you‘ve mastered a dynamic language, the benefits are enormous in terms of productivity and maintenance. Less code is much easier to maintain provided its clean code.
So this creates a Catch-22 of sorts: Dynamic languages are much more productive yet it would take time for developers to learn a new platform, so much time in fact that in the short run it would cost more to move to Ruby or Python than would be saved by the productivity of these languages. In addition, there really isn‘t a dynamic language with the same ecosystem and infrastructure as Java - so adopting a dynamic language necessitates accepting less in terms of resources, tooling, and skilled people.
The solution, therefor, would seem to be a dynamic language that is based on a platform with which the Java community is already familiar. There are many dynamic languages that have been created for the Java platform including Jython, JRuby, BeanShell, Pnut, and others but of these I believe that Groovy has the most potential. The primary reason is that Groovy is being standardized within the Java community. Although standardization is never a measure of quality, the horrible truth is technology does not win on merit alone. Companies will understandably be more comfortable adopting a language that has been standardized via the same process that standardizes J2SE and J2EE than a language that is not. Right or wrong, the powers that make decisions look to standardization as a litmus test for acceptance and usage.
Besidies being standardized, Groovy has the advantage of being a dynamic language that was built from the ground up with Java developers in mind. It‘s retains much of the same syntax and semanitics familure to Java developers while also offering dynamic language features like dynamic typing, mixins, and closures. These are the types of features that make dynamic languages much more productive than conventional languages. Finally, all the Java APIs that you use today can be used in exactly the same way in Groovy. Groovy compiles into byte code, so any Java API you can import and access in a Java program you can import and access in Groovy.
The Groovy standard (JSR 241) was started close to three years ago and is now coming to fruition. To some that seems like a really long time, but this is an opinion that is not based on a historical understanding of how long it takes a language to mature. Ruby, for example, which is just now getting a lot of attention has been in development in the open source community for 13 years, JavaScript for 14 years, and Python for 15 years. The fact that its taken only 3 years for the Groovy language standard to be developed is actually a phenomenal achievement.
The future seems obvious: Dynamic languages are growing in popularity and their productivity and broad applicability cannot be ignored. The Java platform is supported by a huge ecosystem of 4 million-plus developers and thousands of tools and APIs. If any dynamic language is to be successful it has to (a) be standardized (b) appeal to Java developers (c) be fairly easy to learn, and (d) leverage the existing Java ecosystem. There is only one dynamic language that meets those needs and that‘s Groovy.
The hype surrounding Groovy has died down and buzz about the project has been pretty quiet and in my opinion that’s the best thing that could have happened. The result is a language that has had time to be developed into something that is going to be truly useful. In many respects Groovy is the best combination of the Java platform and dynamic language design. Groovy has been quiet, but like a sleeping giant Groovy will awaken and effectively take over as the dominate dynamic programming language after it becomes a JCP standard.
February 28, 2006 |Permalink
TrackBack
TrackBack URL for this entry:
http://www.typepad.com/t/trackback/4331275
Listed below are links to weblogs that referenceGroovy: The Sleeping Giant:
»RMH on Groovy from Stefan Tilkov‘s Random Stuff
Richard Monson-Haefel has this to say about Groovy: The hype surrounding Groovy has died down and buzz about the project has been pretty quiet and in my opinion thats the best thing that could have happened. The result is a language that has had...[Read More]
Tracked on March 01, 2006 at 04:36 PM
»NFJS - Day 3 from ntschutta.com
WhewIm starting to get caught up (well, not with the Foundations of Ajax account!) By Sunday, I was plenty tuckered out - but I powered through. I had a good chat at breakfast with a few guys, talked about productivity and off-shoring ...[Read More]
Tracked on March 22, 2006 at 07:46 PM
Comments
Richard,
As you know from our work together, I‘m in sound agreement with the premise of the rise of dynamic languages. I will quibble with some of your assumptions, however.
Quibble 1: I wouldn‘t say that dynamic languages are library poor. This is true for Ruby, the dynamic language I‘m using today, but it won‘t be true for long, I‘d wager. The same thing can‘t be said for Perl (which I know) and Python (which I don‘t). Both of them are knee deep in libraries.
Quibble 2: Well, more than a quibble. While it makes intellectual sense for Java developers to migrate to Groovy, I wouldn‘t say that considerations such as yours factor into language choice. Its all about the buzz. And today the buzz is around Ruby and Python. Perl? Not so much. I think developers outside of the Fortune 1,000 would drop Java and their accumulated knowledge in a heartbeat
You‘re right, though, that developers inside the Fortune 1,000 can‘t abandon their accumulated code base, and that largely precludes the use of today‘s dynamic languages. There‘s a lot to be said for getting access to all those Java APIs and also to a high-performing, secure VM that‘s got real threading support. (Though there‘s no reason to bring forward all of the Java-based frameworks.) So, how to do this if not with Groovy, JRuby, etc? The answer is Sun. They have to do whatever it takes to realize this vision espoused by Tim Bray:http://www.tbray.org/ongoing/When/200x/2006/02/02/LAMP-Java-Sun .
Posted by:Pete Lacey |February 28, 2006 at 06:49 PM
Pete,
Your right about Perl, it is definitely not a library-poor language. It‘s been around for almost 20 years and its it an excellent language. The problem with Perl, and the reason I didn‘t mention it, is that it will probably never become a mainstream enterprise computing language because its very difficult to maintain. (Note: it probably is however the most popular Unix/Linux administration language). Personally, I love Perl, but I wouldn‘t trust a team of developers new to the language to create a maintainable application with it. I‘ve spoken to far too many developers who admit that in many cases their code is so obscure they can‘t maintain it after a couple of years. There is just too many ways to write functionality in Perl. Python on the other hand espouses this idea that there is "one best way to do everything" meaning it tries to focus on a single approach to language solutions. That‘s great! In addition, Python has a very healthy set of libraries (their motto is ‘batteries included‘). That said, the Python ecosystem and available libraries (including third party libraries not just core) is still quite small compared to the Java platform - I would bet by an order of magnitude. It‘s difficult to appreciate just how many third party APIs are available for Java until you start looking at source-forge and then combine in the thousands of commercial offerings not to mention the huge number of JCP APIs. I think Java APIs may even rival C/C++, but then again the ecosystem surrounding those language is also huge. Too bad C/C++ languages are not as productive as others.
As far as quibble 2: I think you missed the point - or more likely I didn‘t make it clear enough. The fact that Groovy will be hugely successful where other languages (Ruby and Python) will be moderately successful has nothing to do with technical merit and everything to do with perception. First and foremost, the perception that Groovy is a standard will play a huge role in its adoption. Imagine a CTO being convinced that adopting a dynamic language makes sense and then comparing Ruby, Python and Groovy. Groovy is a JCP standard. It already integrated seamlessly with Java APIs and you call Groovy classes from Java. The programmers may find Ruby or Python more attractive, the manager - assuming he‘s a pragmatist - will naturally choose Groovy. Actually the same is true of most Java developers. Given a choice between Ruby, Python, and Groovy most Java developers are going to choose Groovy. Why? It‘s easier to learn and does the same thing as Ruby and Python. I seriously doubt that Ruby or Python are significantly more productive than Groovy, if at all. Hype is a pretty strong motivator, but once people realize that as a "standardized" dynamic language Groovy provides the same benefits as Ruby or Python, only its easier to learn, Java developers will choose it over other most if not all of the time.
Tim Bray vision is not unique or revolutionary. Many before him have made similar observations about the three-legged stool that is Java and how the language is the least important of them. In fact I wrote a long entry about that here(http://rmh.blogs.com/weblog/2005/03/bytecode.html). What Tim Bray is missing is the that the syntax and even the semantics of Groovy are much closer to Java. As I said, the Groovy language was designed from the ground up so its easy to learn by Java developers. None of the other languages in Tim‘s pie chart can make that claim. The fact is Groovy is Java friendly. The largest development population right now is the Java community. If Java folks go anywhere, its going to be to Groovy.
Posted by:Richard Monson-Haefel |February 28, 2006 at 08:04 PM
Richard,
I‘ve just spent the last couple days trying to see if there would be any way to possibly use ClassLoader magic (compiling/reloading classloaders) to emulate some of the scripting languages. I made some decent progress by using Commons-JCI + the Eclipse compiler. The result: you could launch WebWork in a special mode called QuickStart that autocompiled your .java files on any change. At first glance it really looks like the best of both worlds: strong refactoring support plus super quick deployment/changes. But digging deeper, I began to get in to major classloading hell and I‘ve come to the conclusion that using classloading tricks won‘t ever get us there.
But there is another option: complete HotSwap support in Java. See this URL for more info, at the end of that thread I have linked to several blogs and discussions:
http://forum.java.sun.com/thread.jspa?threadID=572396&start=15
Sun is being very quiet on this, which is frustrating. I really believe that while your comments on Groovy are spot on, you are missing one of the other majors benefits of Java: the tools. I use IDEA and consider myself to be an expert user. I know almost all the shortcuts and I can pump out paragraphs of code in literally just a handful of keystrokes. Moving to Groovy would negate much of the tool support. Why not encourage Sun to introduce one of the main benefits of scripting languages (quick changes) to the Java language via better HotSwap support? I imagine this could make up for at least ~3X of the 5X productivity boost you mention.
Posted by:Patrick Lightbody |March 01, 2006 at 07:56 PM
An interesting post, I had to comment on it :)
Graeme
Posted by:Graeme Rocher |March 02, 2006 at 08:15 AM
Patrick,
Sounds like interesting work, but it doesn‘t sound like you are on the same topic. You are talking about ClassLoaders which are a technical aspect, rather than a more general aspect of Groovy and dynamic programming languages. With regards to IDEA, mastering an IDE is a sure way to gain productivity and it sounds like you‘ve done exactly that. Some people are better at these things than others, in general however the features that make dynamic languages such as Groovy more productive are tangental to those skills you developed using your IDE. That‘s just my opinion and maybe I‘m missing your points. Not sure.
Richard
Posted by:Richard Monson-Haefel |March 02, 2006 at 12:05 PM
Graeme,
You‘ve made some excellent points in your own posting which people can find here (http://graemerocher.blogspot.com/2006/03/groovy-sleeping-giant-elaboration.html ). I would like to say, however, that you mischaracterized my position a little bit. You said that you disagree on several counts starting with "Firstly, I believe in a blend, its not going to be dynamic everything. ". For the record: I never say that dynamic languages will do everything. What I say is that more development will be done with dynamic languages and they will become a first-class tool in enterprise development. General languages are not going away and I agree they make more sense in many cases, but the fact is that general languages (e.g. Java, C#, C++) are frequently used in scenarios where dynamic languages would be much more productive. Anyway, I think we are in agreement but just wanted to point out that you may have misunderstood my meaning. Dynamic languages will not replace general languages. It won‘t be "dynamic everything". Dynamic languages are a complement to general statically compiled languages. Good luck with Grails. Sounds pretty cool.
Richard
Posted by:Richard Monson-Haefel |March 02, 2006 at 12:13 PM
Richard,
I‘d tend to agree with Pete regarding Perl, and I think you put your finger on the right spot when you commented that "there are just too many ways to write functionality in Perl". But the problem may be partly resolved when you have a framework that sits on top of Perl and forces less experienced programmers to follow a standard process aka Ruby on Rails. We‘ve managed to follow that principle and keep Perl as our primary programming lanugage (along with Javascript) for developing web based enterprise applications for the last 10 odd years (cross my fingers). The only thing I think Perl lacks is a strong lobby of a big name to back it - ActiveState is the best we have for Perl, but it‘s not a Sun :-)
Posted by:Ashutosh Bijoor |March 02, 2006 at 11:24 PM
Richard,
I only hinted at what I should have said outright: I believe that dynamic languages, while having nicer syntaxes than Groovy, are mostly useful because of their ability to roll out changes quickly. Instead of the "code-compile-package-deploy-wait-refresh browser" process, with Perl/Ruby/PHP/Python you can simply do "code-refresh browser" when developing webapps.
My point is: what if we could still use Java (including the more verbose syntax) but get this major benefit of dynamic languages?
At this point, I‘ve come to the conclusion that you can get close to this with ClassLoaders and well-designed libraries (such as ones that reload when they detect configuration files and/or Class changes), but to really do it right Sun must support complete HotSwap support in the JVM. I truly believe that if they did that, Java could compete with dynamic languages in a good chunk (80%?) of the cases where people wish to use dynamic languages today.
Posted by:Patrick Lightbody |March 06, 2006 at 02:46 PM
"If any dynamic language is to be successful it has to (a) be standardized (b) appeal to Java developers (c) be fairly easy to learn, and (d) leverage the existing Java ecosystem. There is only one dynamic language that meets those needs and that‘s Groovy."
Actually, BeanShell also meets all of those qualifications, and it‘s even easier to learn than Groovy. But I agree with you that Groovy holds more promise--I think it‘s more "dynamic" than BeanShell from a syntax perspective, and it comes with a compiler.
Posted by:Joe Wolf |March 30, 2006 at 04:06 PM
This is a little tangential (I apologize for that) but Squeak/Croquet are showing serious promise. Although the problem domain they are concentrating on may be limited I believe it is mostly due to the number of researchers currently engaged – there is no reason to believe additional effort couldn’t develop enterprise level applications.
Friends of mine are working on the project, at the UofMn, and I’ve commented to them about some of the same issues Richard has raised. I’ve not heard a compelling reason why it would be better to abandon Java for a dynamic language instead of evolving Java into a dynamic language. It seems to me the amount of effort required to start from zero and get to where Java is today is far far greater than the effort required to evolve Java.
But if Java can’t evolve – that would be reason enough – I just haven’t heard anyone make a convincing argument that Java can’t evolve.
If you have some free time I would look at Squeak/Croquet – they have a really nice R&D project going on. They aren’t too far away from a killer-app.
http://en.wikipedia.org/wiki/Croquet_project#Squeak
http://en.wikipedia.org/wiki/Croquet_project