Matthew Bass

Musings on software and life…

June 25th, 2010

Drop seconds when querying MySQL timestamps

One of the Rails apps I’ve been working on formats times as HH:MM in the view. No seconds are displayed. This is a pretty common way to format things. When doing timestamp comparisons in SQL, however, the seconds are taken into account. This is bad since it can cause discrepancies in the view.

For example, say I have a table of records with created_at timestamps. My view displays all records with timestamps equal to or before the current time. Let’s assume the current time is 15:00:00 precisely and I happen to have a record with a timestamp of 15:00:00 in the database. The SQL comparison would work fine in this case.

SELECT * FROM records WHERE created_at <= "2010-06-25 15:00:00"
=> 1 row in set

What if the timestamp in the database is 15:00:03 though? Let’s run the query again.

SELECT * FROM records WHERE created_at <= "2010-06-25 15:00:00"
=> Empty set

Since 15:00:03 is greater than the current time of 15:00:00, the record doesn’t get returned. This would be fine if we were displaying seconds in the view, but we’re not. From the user’s perspective, the timestamp on the record is still 15:00 and should appear in the view since it’s equal to the current time. But it doesn’t.

One way to fix this would be to handle the time comparisons in Ruby. This is certainly a legitimate option. For this particular project, though, performance was a big issue. (And we all know that Rails can’t scale.) I needed a way to continue letting the database handle the comparisons while disregarding seconds.

The solution I ended up with isn’t ideal (it relies on a couple of functions built into MySQL) but it works fine and runs fast:

SELECT * FROM records WHERE (created_at - INTERVAL SECOND(created_at) SECOND) <= "2010-06-25 15:00:00"
=> 1 row in set

The number of seconds is extracted from the created_at timestamp and then subtracted from the timestamp. So if the timestamp was 15:00:03, MySQL subtracts 3 seconds to end up with 15:00:00.

This solved the comparison problem for me and made my client very happy. Double win.

June 18th, 2010

Ruby Hoedown 2010

This year’s Ruby Hoedown is happening in Nashville again on September 3rd and 4th. I’m really looking forward to attending. The quality of the talks combined with the smaller attendance size makes for some great hallway conversations. Last year’s Hoedown was at the Opryland hotel which was a stellar venue. I have seriously never seen such a large hotel. Unfortunately, it can’t be used this year due to the recent flooding. But the new venue, the Hilton Downtown, looks really nice as well. As before, the Hoedown is completely free (as in beer) and talk proposals are currently being accepted. Are you going?

The Ruby Hoedown MMX

June 11th, 2010

RubyConf in New Orleans

This year’s RubyConf is being held in New Orleans on November 11th – 13th.

Count me in.

I’ve only driven through the area once so it’ll be interesting to make a longer visit. Although I’m ultimately keeping my fingers crossed for a Raleigh RubyConf one of these days. Hey, I can dream.

December 3rd, 2009

test_spec_on_rails now runs on Rails 2.3

If anyone still happens to be using test_spec, you’ll be thrilled to know that the test_spec_on_rails plugin is now compatible with Rails 2.3. It has also been converted to a gem. Install with:

sudo gem install test_spec_on_rails

Add to your Rails app’s test.rb like so:

config.gem 'test_spec_on_rails'

Enjoy the goodness of test-spec helpers from inside your Rails tests. Fork and submit patches via the GitHub project. Tell your friends. Donate money. Vote for Pedro.

December 3rd, 2009

Processing malformed files with FasterCSV

On a recent project, I had to implement a CSV parser that would gracefully handle malformed files. I’m talking about files with unescaped quotes, wacky UTF-8 chars, and various other abominations of nature.

I originally assumed FasterCSV would handle this automagically, but it turns out that the library’s most commonly used methods are pretty strict when it comes to handling CSV files.

For example, parsing a malformed file one line at a time will result in an exception being thrown, even before any rows are yielded to the block:

FasterCSV.foreach("malformed.csv") do |row|
  # use row here...
end

Not cool! I managed to get around this by manually looping over each row and rescuing a malformed CSV exception if one gets thrown:

FasterCSV.open("malformed.csv", "rb") do |output|
  loop do
    begin
      break unless row = output.shift
      # use row here...
    rescue FasterCSV::MalformedCSVError => e
      # handle malformed row here...
    end
  end
end

Anyone have a better way to do this?

December 3rd, 2009

Time warping gem goodness

The time zone warp code I posted about last week is now a gem:

sudo gem install time-zone-warp

To configure in your Rails app, add this line to the bottom of test.rb:

config.gem 'time-zone-warp', :lib => 'time_zone_warp'

You can also fork the code from the project on GitHub.

November 18th, 2009

Time zone warp

One of my Rails projects makes heavy use of time zones. I’ve run into some issues writing good tests for this type of thing. In particular, I’ve needed my tests to run within a time zone outside my own. But I don’t want to permanently change the time zone within the scope of the entire test run. I ended up coding this handler:

module ZoneWarp
  def pretend_zone_is(zone)
    original_zone = Time.zone
    begin
      Time.zone = zone
      yield
    ensure
      Time.zone = original_zone
    end
  end
end

Test::Unit::TestCase.send(:include, ZoneWarp)

Simply stick this code in a file inside your config/initializers directory (or include it from test_helper.rb or spec_helper.rb if you insist on doing it the right way) and you’re all set to write tests like this:

test "code works in other time zones" do
  pretend_zone_is "Mountain Time (US & Canada)" do
    # assertions go here
  end
end
August 11th, 2009

Learn about Prawn at raleigh.rb on August 18th

I’ll be giving a presentation about Prawn at this month’s raleigh.rb meetup. Prawn is a Ruby gem that enables fast PDF generation. It is a dramatic improvement over previous libraries like PDF::Writer. It can be used standalone or inside your Rails applications. The markup is powerful and relatively painless to use. I hope you can join us for the fun on August 18th at Red Hat HQ.

June 29th, 2009

Audio interview for RubyRX 2009

RubyRXJared Richardson just posted a series of interviews in anticipation of the upcoming RubyRX/AgileRX conference taking place in Reston, Virginia in September. In my interview we discuss iPhone development, MacRuby, Git, and testing frameworks.

I’m really looking forward to presenting again at RubyRX. I’ll be giving two talks this year. Git with Ruby will explore the Git source control system and how Ruby can take advantage of it. In Which Ruby Testing Framework Should I Use? we’ll briefly examine several leading testing frameworks and study the pros and cons of each. You’ll leave fully prepared to pick the best framework for your next project.

Let me know if you’re coming to the conference this year and we can link up in Reston. If you haven’t registered yet, what are you waiting for? RubyRX is a chance to network with the best and brightest developers in the area, and hear from thought leaders like Andy Hunt, Rich Kilmer, Joe O’Brien, and Chad Fowler. It’s a great way to keep your skills sharp in a down year.

June 29th, 2009

Lindo testing helper gets some love

Lindo helps you write and verify Rails functional and integration tests by opening the HTTP response body in the default browser for inspection. This can be a real time-saver when you’re trying to figure out why your assert_select or have_tag calls aren’t passing.

In its initial version, Lindo assumed that your app was running at localhost:3000 (a fair assumption given the prevalence of Mongrel last year). Now that Passenger is on the scene, something better needed to be done. The reliance on a running app server was a disadvantage to begin with. Now Lindo doesn’t require anything to be running. It dumps the HTML to disk, fixes any relative asset URLs, and opens the file using your default browser.

Once you’ve written your first test with the assistance of Lindo, you won’t want to go back!

Lindo was developed by my company, Adeptware, and can be pulled from GitHub. I’ve also posted a brief introduction to Lindo and some basic installation instructions.