Panasonic Youth rob sanheim writes about software, business, ruby, music, stuff and things



Rails 2.3, JSON, and lamentations

There is a nasty gotcha in Rails 2.3 involving rendering JSON from models and a surprising change in behavior. If you are familiar with how Rails renders xml for models, you may expect json to be very similar. You would probably expect a top level root node followed by the attributes:

{"person" => { "city=>nil, "name"=> "Foo Bar" } }

It appears somewhere on the road to 2.3, the JSON encoding changed so there is no longer a top level node:

{"city"=>nil, "name"=>"Foo Bar" }

The end result of this change means that if you have consumers expecting JSON in a typical “Rails” format, with a top level element denoting the class, there will be pain. We just hit this in a project when trying to use ActiveResource to consume some models exposed via standard restful actions.

The following demonstrates how this breaks down exactly:

# in the backend web app
Person.create! :name => “foo”

# meanwhile, in an active resource consumer
person = Person.find(1) # finds a model okay, but…

# the below all raise NoMethodError
# they _should_ find the attribute from the attribute hash
# and quack like an ActiveRecord model
person.name
person.age
person.any_field_on_the_model

person.attributes # returns a hash in the format of { “person” => { “name” => “foo”…}, which is where the problem lies

It appears this issue is fixed or on the way to be fixed in the 2.3 stable branch, but for now be wary of JSON support in general on 2.3 until more of the below tickets get resolved.

References:

  • the bug in question - still open
  • xml vs json - fight!
  • 2.3 related (?) fix that caused other issues
  • an unrelated JSON issue, but very annoying and has hit us on multiple projects — require “json” considered harmful)

  • CapGun 0.2.0 - now with all Git commits since the last release

    Relevance has released CapGun 0.2.0 to GitHub and Rubyforge.

    We’ve added more Git knowledge to this release, courtesy some great work from Muness. Your deployment notifications will now contain the branch they were deployed from and a list of all revisions included in the deployment (since the last deploy, of course).

    There was also a overhaul to the internals. Muness refactored all the email logic to a presenter, so it should easy to change the notification email to your liking. CapGun now uses Jeweler and Micronaut, to match the rest of the standards for Relevance open source.

    Keep us honest by checking the build status on RunCodeRun, and file feature requests and bug reports on Github Issues.


    Pair programming? TDD? Where is the proof?

    Often while discussing agile practices with agile skeptics, the question of proof comes up. “Are there studies done showing TDD? How do you know pairing works and doesn’t just waste time?”

    Those are tough questions. Software is not a science, and there are no double-blind, randomized controlled projects. The study of software development barely approaches the level of a social science. Its easy to discuss the flow and intuitive sense that agile works with other people who have experienced it, but try expressing that to someone who has never stepped beyond the waterfall model.

    Things are getting better in the area of real, hard studies done on agile. One example is the work being done by Microsoft’s Empirical Software Engineering Group (ESM). The summary from the TDD study is worth citing:

    “The results of the case studies indicate that the pre-release defect density of the four products decreased between 40% and 90% relative to similar projects that did not use the TDD practice. Subjectively, the teams experienced a 15–35% increase in initial development time after adopting TDD.”

    I’ve included full links to some of the standout papers on my @to-read list below.


    Upgrading git via MacPorts

    Git 1.6.1 was released at Christmas, and if you are still on 1.6.0 or lower its well worth upgrading. MacPorts is the way to go for Git on the Mac - the ports team has been awesome in keeping up with the latest versions. Here’s how I upgrade:

    First, make sure you have the latest portfiles:

    sudo port sync

    Deactivate any preexisting versions - not sure if this is strictly necessary, but I’ve seen failures when I don’t.

    sudo port deactivate git-core

    Install the new hotness, with all optional add-ons.

    sudo port install git-core +svn+bash_completion+doc

    Verify it:

    git --version
    # => git version 1.6.1

    Note that there is a MacPorts ‘upgrade’ command, but I’ve seen it fail to actually upgrade things from time to time. Doing the install this way always works for me.


    Best Albums of 2008

    So, here are my top ten albums of 2008, just a bit late. All links go to the album on the Amazon MP3 store (if available) and they are affiliate links.

    This album was on everyone’s top albums list of the year for good reason. Whether you like folk, pop, or country, this album has no filler and is great through and through. Pick up their Sun Giant EP as well to round it out.

    I’m a Wisconsin native, so I may be a little biased in picking an album created in the Wisconsin north woods. Still, a superb album for cold winter nights and hot cocoa or coffee at your side.

    Before the New Year, there was a Texan band named Bedhead in the 90s. Bedhead was quiet, contemplative, and perfect for those lonely hours between midnight and dawn. Since then, Bubba and Matt Kadane (real names, and real brothers) formed the New Year, got a little more accessible and upbeat, and released three albums. This third self-titled album is where slow-core and pop come together seamlessly. The Company I Can Get is the closest thing the Kadanes have done to a radio friendly single, while still jaw-dropping beautifully sad songs like The Idea of You.

    Metalcore that doesn’t suck and tries new things. Not easy to find these days.

    Mark Kozalek released an insane amount of music in 2008, both under his own name and Sun Kil Moon. April is the best of the bunch and will grow on you.

    Catchy, quirky indie pop, producing a sound far bigger then any two humans should be capable of with only drums, guitar, and voice.

    Epic post-metal. Recorded on the site of a demolished mental institution, with lyrics from the diary of a madman who was once a resident there. If you are a fan of metal, what else do you need to know?

    Minimal techno is notoriously difficult to translate out of the club and into the living room. Beyer is a top techno producer and DJ, and this mix really captures the banging minimal techno that became so popular in 07/08. Avoid if your responses to techno include “wait, where are the lyrics?” or “is the CD skipping??”. If you can get into the Chemical Brothers, Underworld or even Paul Oakenfold, put this one on and give it some time. You may just find yourself nodding along and digging techno.

    Sigur Ros finally gets a little less serious and overbearing and make a wonderfully fun album with some actual pop songs alongside the epic twelve minute compositions.

    (This came out in Europe in 2007 and made it over here in 2008.) Two French post-hardcore/screamo bands in the vein of Page 99 or Envy (read: much heavier and harder then MTV emo) join for an album that is everything good about hardcore right now. Takes elements of post-rock and math-rock, yet still retain the passion and hooks that made screamo so captivating to begin with. You don’t need to understand French to rock out to this split.

    Honorable Mention


    MacPorts Ruby performance gotcha

    If you install Ruby from MacPorts (all the cool kids do), be aware that the first 1.8.7 port had some issues that basically broke performance, making it run three times slower than normal. Verify that you have the 1.8.7-p72_2 version active, and not 1.8.7-p72_1:

    ~ $ sudo port installed |grep ruby
    ruby @1.8.7-p72_1+thread_hooks
    ruby @1.8.7-p72_2+thread_hooks (active)
    

    The defect on MacPorts Trac has more details, along with links to a relevant ruby-talk thread.

    Its worth pointing out that this issue went from discovery to being fixed and released into the port stream in about a days time. MacPorts has really come a long way from just a year or two ago. Fairly major packages used to lag behind the latest stable releases by multiple versions, but now I find everything I care about stays up to date with ease from MacPorts. They are even able to keep up with git releases, an impressive task considering how fast and furious releases git releases come.

    Kudos to the MacPorts team for doing a fine job and making developers’ lives easier.


    Tarantula 0.0.5 Released - the “Naked Aardvark” release

    Announcing version 0.0.5 of Tarantula.

    Tarantula is a big fuzzy spider. It crawls your Rails application, fuzzing data to see what breaks. It can verify HTML validation across all your pages, ensure you don’t have 404s, and pretty much anything else you want via custom handlers.

    Don’t let the version number fool you, we’ve been using Tarantula across many projects at Relevance and its very stable. This release fixed a number of annoying bugs, including namespace conflicts with other classes due to Rails dependency loading, improved gem spec with correct dependencies, and clean up on the html reporter.

    Install it via the Github

    gem install relevance-tarantula --source http://gems.github.com

    or via Rails 2.1+ gem handing:

    config.gem "relevance-tarantula", :source => "http://gems.github.com"


    Scp or rsync failing with no error message? Check your startup scripts…

    The other day I was having issues trying to scp/rsync data, with no real error message to try and debug things. Turns out that any output produced by your startup scripts will break rsync/scp hard. I had some simple ‘echo’ statements print when different scripts were being loaded…turns out scp/rysync don’t like that.

    My capistrano task was a very simple call out to the ‘get’ helper, which just uses scp under the hood. The task ran and looked as if it completed, only nothing was ever transferred and the scp progress bar never came up. Sometimes it would block and do nothing, which was real fun, too.

    The solution was simple - change all the bash scripts we use to not output any echo anything when running. I deployed the new scripts to all servers I needed to scp with, and the issue was resolved.

    Since this is a known issue in the faq, it won’t be fixed or improved with a better error message. It’s just something you need to be aware of and work around, either via detecting if the session has an interactive terminal before sending output or removing your output statements altogether from you startup scripts.


    Git Clone vs cp -R –> WTF?

    I knew git was fast, and I even knew it was faster than a lot of plain linux local file operations. Still, this still blew me away:

    CODE:
    1. rsanheim@ares:~/src/personal/oss $ du -hd 0 insoshi/
    2.  26M    insoshi/
    3.  
    4. rsanheim@ares:~/src/personal/oss $ time git clone insoshi/ /tmp/insoshi
    5. Initialize /tmp/insoshi/.git
    6. Initialized empty Git repository in /private/tmp/insoshi/.git/
    7. Checking out files: 100% (2193/2193), done.
    8.  
    9. real    0m3.826s
    10. user    0m0.251s
    11. sys 0m0.658s
    12.  
    13. rsanheim@ares:~/src/personal/oss $ time cp -R insoshi/ /tmp/insoshi_cp
    14.  
    15. real    0m9.065s
    16. user    0m0.114s
    17. sys 0m1.442s

    Ok, so a 26 meg repo takes almost three times as long to copy via a recursive cp than a local git clone. Thats a fairly small repo, lets try something bigger:

    CODE:
    1. rsanheim@ares:~/src/relevance $ du -hd 0 rails
    2.  75M    rails
    3.  
    4. rsanheim@ares:~/src/relevance $ time git clone rails /tmp/rails2
    5. Initialize /tmp/rails2/.git
    6. Initialized empty Git repository in /private/tmp/rails2/.git/
    7.  
    8. real    0m2.321s
    9. user    0m0.151s
    10. sys 0m0.465s
    11.  
    12. rsanheim@ares:~/src/relevance $ time cp -R rails/ /tmp/rails
    13.  
    14. real    0m7.133s
    15. user    0m0.067s
    16. sys 0m1.505s

    The rails repo at 75 megs is still ~ 3 times faster.

    Obviously, this is not scientific at all, but the point is pretty clear. Git is doing some magic that lets it move files around locally 2 to 3 times faster than a plain copy. From looking at the man page, I would guess it has something to do with git using hardlinks for things in .git/objects when cloning locally. My linux fu falls down a bit here -- what are the ramifications of using hard links versus doing a "real" copy?

    (This also makes me want to try out gitbak even more...)


    Quick: Find the Bug or Gotcha with named_scope

    Think fast! Where's the bug?

    RUBY:
    1. named_scope :active, :conditions => ["activated_at <= ?", DateTime.now.utc.to_s(:db)]

    Looks fine, right? Maybe you've hit this already, and you see it immediately.

    The symptoms are that the DateTime.now always seems to be a bit off - maybe you just restarted your server and its a only a few minutes off.

    The bug is that DateTime.now gets evaluated at the time the class is loaded, not when the finder is run. What makes this easy to miss is that it will always work fine in tests and development, as everything is constantly getting reloaded there.

    The fix, obvious once you've spent a combined time of over an hour trying to figure out what is going on:

    RUBY:
    1. named_scope :active, lambda { { :conditions => ["activated_at <= ?", DateTime.now.utc.to_s(:db)] } }


    ← Before
    Flickr View All » IMG_6396IMG_6392IMG_6384IMG_6379IMG_6378IMG_6363IMG_6353IMG_6352IMG_6351