Wednesday, May 26, 2010

The horror, oh the horror

I thought I'd seen horrible code before...then I had to figure out what happened that caused this working monstrosity to stop working when the site it was on was ported from Drupal 5 to Drupal 6. I think I'm going to lose a few hairs on this one.

Sunday, May 16, 2010

Groovy: duck-typing not always the best choice?

Working through Grails In Action chapters 3 & 4, I noticed something. The integration tests written to test domain classes and controllers use Groovy defs, presumably for simplicity.

The test cases look a lot like this:

void testFirstSaveEver() {
def user = new User(userId:'joe', password:'secret')
assertNotNull user.save()
assertNotNull user.id
def foundUser = User.get(user.id)
assertEquals 'joe', foundUser.userId
}

I'm not going to go too deeply into the whole dynamic versus static typing debate, though I do have some verbose opinions on the matter. I was particularly intrigued with Groovy-- after learning Python and PHP and coming from a C++/Java/.NET background-- because it seems to be at the sweet spot of statically typed and dynamically typed. Being able to do both is a powerful thing.

For one, the type can serve as a form of documentation. Anyone looking at the code can tell what the original developer is expecting to receive when the type is declared, which may at times be easier/clearer/more telling than evaluating the expression on the right.

Additionally, I would expect that declaring the type at compile time would enable the Groovy interpreter to do some fancy things. Interpreted languages with dynamic typing get a bad wrap because they're slow. This is for a reason, as the duck typing resolution is not a free operation. I figure that if you can help the process along by telling the interpreter what the type is, when you know what the type should be, it should speed up the interpretation.

Curious, I decided to start writing the tests presented in the book, and tests that had statically typed variables instead of def. Like so:



void testFollowing(){
def before = System.currentTimeMillis()
def glen = new User(userId:'glen', password:'password').save()
def peter = new User(userId:'peter', password:'password').save()
def sven = new User(userId:'sven', password:'password').save()

glen.addToFollowing(peter)
glen.addToFollowing(sven)
assertEquals 2, glen.following.size()
sven.addToFollowing(peter)
assertEquals 1, sven.following.size()
def after = System.currentTimeMillis()
println "User following using duck typing: " + (after-before)
}

void testTypedFollowing(){
long before = System.currentTimeMillis()
User glen = new User(userId:'glen', password:'password').save()
User peter = new User(userId:'peter', password:'password').save()
User sven = new User(userId:'sven', password:'password').save()

glen.addToFollowing(peter)
glen.addToFollowing(sven)
assertEquals 2, glen.following.size()
sven.addToFollowing(peter)
assertEquals 1, sven.following.size()
long after = System.currentTimeMillis()
println "User following using static typing: " + (after-before)
}

Through the end of chapter 4, I've written quite a few of these tests. The output is shown below


Unit Test Results.


Designed for use with JUnit and Ant.



All Tests


NameTime(s)
PostIntegrationTeststestFirstPost0.188
PostIntegrationTeststestAccessingPosts0.078
PostIntegrationTeststestPostWithTags0.069
PostIntegrationTeststestTypedPostWithTags0.050
QueryIntegrationTeststestBasicDynamicFinders0.248
QueryIntegrationTeststestTypedBasicDynamicFinders0.051
QueryIntegrationTeststestQueryByExample0.052
QueryIntegrationTeststestTypedQueryByExample0.040
TagIntegrationTeststestSomething0.016
UserIntegrationTeststestBlankUserName1.518
UserIntegrationTeststestFollowing0.055
UserIntegrationTeststestTypedFollowing0.033
UserIntegrationTeststestFirstSaveEver0.022
UserIntegrationTeststestSaveAndUpdate0.022
UserIntegrationTeststestSaveThenDelete0.101
UserIntegrationTeststestEvilSave0.046
UserIntegrationTeststestSaveEvilCorrected0.029


In every comparison of typed versus def, the typed version has been faster. To verify, I also look at what I printed out to console:



--Output from testBasicDynamicFinders--
Duck typing testing dynamic finders: 230}
--Output from testTypedBasicDynamicFinders--
Typed testing dynamic finders: 41}
--Output from testQueryByExample--
Duck typing testing query by example: 42}
--Output from testTypedQueryByExample--
Static typing testing query by example: 30}

I guess that's kind of pointless. I just verify that JUnit tells time. :) The point being, however, that there seems to strong evidence that using the type when possible dramatically improves performance. The testBasicDynamicFinders method saw an almost 5x reduction in time of execution. This seems to completely contradict John Wilson's assertions that "Knowing the type of a parameter makes the call slower!" Of course, it was 4 years ago, but it looks like there was a good reason for blackdrag to investigate!


The question becomes: are there times when declaring the type is slower than def? Are these results valid, or is my system just nuts? If they are, should *hint hint* these portions of the book be rewritten in the next edition to spread best practice?

The Importance of Learning The Right Way

Related to my earlier thought on Test-Driven Development and why I struggle implementing it, I think it's important to learn how to code The Right Way.


What does that mean?


In the beginning, as young programming babies we do a lot of guessing and checking. We write loops that go out of bounds, don't really understand the difference between myClass->property and myClass.property in C++, etc etc As time progresses we discover tools and methodologies that make us more productive, like testing and design patterns. But, integrating these new techniques into our workflow is difficult. Oftentimes, for the sake of brevity, clarity, or lack of understanding, I fall back to my old approaches.


You can't teach an old dog new tricks

I'm not cynical enough to believe this is true, but as a life-long student and an academician this is an interesting topic.


I want to build high quality software. Software that not only meets requirements, but is comprehensible, maintainable, adaptable, reliable, secure, etc etc. But it's difficult to integrate best practices that one learns in a tutorial session with daily programming life without heavy repetition and rigorous use of those skills. Until using those skills becomes part of our pattern, we'll still find ourselves not writing tests or writing scriptlets.


The best way to avoid this integration challenge-- which one can analogize as changing the infrastructure of an application in the maintenance step in the software development life-cycle of the Software Engineering application of our brains-- is to start from a good foundation. Learn best practices as you're learning to program-- analogous to spending extra time in the Design phase, when refactoring costs are cheaper. Yet educating programmers in this way proves to be difficult for a variety of reasons.


First, for those programmers who are self-taught, it is difficult to know where to go for "best practice" information. Googling for code-snippets reveals a variety of tutorials written by developers of varying levels of expertise. Most of the code in these tutorials is not written at a production quality, high level. How many web-development tutorials properly sanitize their user inputs? How many tutorials use doubles for currency when it is better to use a more precise library? etc etc


Secondly, for programmers who learn to code in high school/college (like me), most introductory programming instructors are not well-versed in best practices. This is why, in many places, programming instruction remains pretty much the same today as it was in the 1970s. The programming languages have changed, the tools, etc etc...but essentially, the pedagogical methodology is the same. Learn about data types, variables, control flow statements, iteration, arrays, maybe do some OOP at the end.


I think these are mistakes. Best practices should be integrated throughout. The best time to pick up good habits is when you're learning.


One can argue, of course, that there is a sweet spot. Putting too much detail about bounds checking and security considerations in an example about arrays may obfuscate the clarity of the example. You don't want to digress into unrelated topics, too much code is confusing, etc. I would refute the argument by placing responsibility for good explanation of the example to the educator. Line by line, point by point.


A good programmer may be dumb and lazy. A good educator can't be.

Grails in Action -- Initial thoughts

I've been working through Manning's excellent Grails in Action book to strengthen my skills in Grails. I've been away from heavy duty programming for a while, maintaining poorly made sites in Drupal has made my development skills rusty. So far (partway through Chapter 2), I'm happy with the results.

I like this book so much for a few reasons.
  1. Good humor. The Quote Of The Day application in chapter 1 starts it off right, and sprinkled references to Chuck Norris keep the material fresh
  2. You build a really cool app -- I'm not all the way through yet, but it looks like Hubbub is essentially Twitter built on Grails. Not a Twitter client. Twitter itself. That's a pretty sweet app to build as a learning tool.
  3. Development the right way -- I've been learning about modern practices in web development like Test-Driven Development for a little while...yet when it comes to building new software, I often find myself falling back into my usual patterns of "just code and see what happens", sprinkling System.out.println() everywhere. I hypothesize that this is because this is the way I learned to code, in high school and in college. Old habits are hard to break. This book promises to devote proper time to testing in Chapter 7, which at first seems a little late. But then even in the chapter 2, domain modeling, it has you write integration tests for your domain classes right after you write them.
As I continue to work through the book, I'll reflect on my thoughts.