It’s blog theme refresh time again

Well, despite the fact that it’s been a year since I wrote anything here on my blog—mainly due to a shift in focus to posting on The Last Pixel instead—I decided this week it was time to freshen things up on my personal blog.

Looking through the site I found many stale, uninteresting, or just plain broken pages from years ago. I’ve fixed most of those issues, and invested some time in re-organizing content and updating the blog’s look-and-feel to something more modern and clean. There are still a few old pages to clean up, and some new content I want to add, which will be completed soon.

For posterity, here’s a look at the “outgoing” blog theme, live from September 12, 2007 through December 18, 2011.

mfischer.com theme from 2007-2011

Blog Bifurcation

Originally this blog started out as a fairly personal journal, but has recently adopted a more technology-oriented theme. I plan to focus even more on technology blogging in the future, but I also want to occasionally blog about personal happenings, so I’ve decided to split my blog in two.

This site, mfischer.com, will return to its roots as the personal blog about Mike Fischer. To publish my technology articles I’ve created a new blog under a new domain: thelastpixel.net. I’ve just gotten started with it, but plan to post regularly on software development and other technology issues. If you’re interested, check it out and sign up for the RSS subscription! If not, just keep watching here for more non-technical news from my life…

Essential Ruby on Rails plug-ins and gems

Now that I’ve been using Rails for a while, I’ve accumulated a few gems and plugins that I end up using over and over, so I thought I’d share a little about each.

haml and sass

I was sold on Sass from the first few minutes I used it. Sass lets you write CSS using a simplified abstraction format, and compiles your sass code into CSS. Best of all, it can be used without Rails (or Ruby), so I’ve been using it with static web pages as well. Here’s an excellent example from the sass web site:

Haml does the same for HTML. It took me a little longer to get used to, probably because it’s actually a replacement for HTML and Embedded Ruby (ERB). Haml really cuts down on the amount of typing needed to generate HTML, auto-closes tags (based on indentation), and makes it far easier to see the structure of your document. This snapshot from the haml page is a great demonstration:

friendly_id

This extends Rails’ RESTful resources by making URLs look “friendlier”, using a user-specified attribute of a model instead of the auto-generated numeric ID. In other words, by simply adding a statement like has_friendly_id :name to a users model, friendly_id converts URLs from http://my.site.com/users/17 into the much nicer http://my.site.com/users/mike.

Authlogic

This is the leading user authentication framework for Rails. It you need to allow users to register, sign in, change passwords, use OpenID, user LDAP, etc., this is the plug-in for you. It takes some time to learn how to integrate it, since each web site’s implementation is going to be a little different.

mislav-will_paginate

This makes it dead simple to take a model with many records, and display it in pages, where you can control how many are displayed per page, and the style of the page links:

thoughtbot-paperclip

I’ve researched and tried many file upload/attachment gems but Paperclip is the one I settled on. It seems to be the most actively maintained, and one of the most flexible. File uploading is still not as easy as it should be, especially if you want to provide feedback during upload (usually via a Flash plugin) or allow multiple attachments to a model (usually with an associated polymorphic upload model). Paperclip doesn’t do these things out of the box, but it’s possible to wrangle it into submission…

unobtrusive_date_picker

These days there’s no reason to make your users select a month, day, and year from three separate drop-down boxes, or type a date in some pre-determined format. This plug-in automagically adds a pop-up calendar to your date selection input fields.

jRails

When I started learning Javascript a few years ago, it was a huge pain in the neck. I quickly came across Prototype, which made my life much easier. Before I got too far along with that however, I found jQuery and fell instantly in love. This was clearly the way Javascript programming was meant to be. I used it for everything. Then last year when I started learning Rails, I was a little disappointed to find that the javascript library built-in to Rails was Prototype. Fortunately, others apparently had the same complaint, and created JRails, which integrates jQuery with Rails.

Others

There are a few other gems and plugins I use, but not as frequently. They include: Maruku, acts_as_audited, mechanize, and hpricot.

Am I missing out on any must-have gems or plug-ins? Let me know in the comments!

Multiple image upload and crop with Rails

For a project I’m currently working on, I needed a way to upload several photos to attach to a “dog” model, and allow easy cropping of the photos. In Ruby on Rails, this turned out to be harder than I expected.

The main problems I encountered were:

1. There are several attachment uploading plug-ins for Rails, like attachment_fu, Paperclip, or UploadColumn, with various features and complexities, but none had good references on handling multiple uploads associated with one model instance.
2. To add multiple attachments to an object requires a separate model for attachments, associated with the main model via a has_many/belongs_to relationship. This isn’t hard if the model views are kept separate and RESTful (create a new dog, then add a photo, then add another photo, etc). That’s not a good user experience however — I want to be able to add a new dog, and upload many associated photos at the same time.
3. It turns out that complex forms to handle multiple associated models is pretty tricky in Rails (but should get better with Rails 2.3, coming soon).
4. I also wanted to let the user manually crop the photos after upload, to just select the dog’s face for example.

After several weekends of experimenting, reading lots of blogs and tutorials, and playing with various plug-ins, I finally have a working solution I’m pretty happy with. Here’s what I ended up doing:

• I’m using the Paperclip gem to handle the actual uploads. The main reason I chose this over other options was that Paperclip lets me easily create additional “styles” of an uploaded photo, like a small thumbnail, a page-sized version, etc.
• I’m using the attribute_fu plug-in to handle the multi-model forms so I can upload any number of images at the same time as a new dog is created.
• I’m using the jsCropperUI JavaScript library to let the user select an area to crop.
• Since Paperclip requires ImageMagick to do its resizing, I installed RMagick and use that for my cropping.

Getting started

I have two models: “Dog” and “Upload”. I started by calling my second model “Attachment”, but after running into problems, I learned that Paperclip uses that name internally and will not work correctly if you name your model “Attachment”.

$ rails dogs
$ cd dogs
$ script/plugin install git://github.com/giraffesoft/attribute_fu.git
$ script/generate model dog name:string
$ script/generate model upload description:string dog_id:integer \
         photo_file_name:string photo_file_size:integer
$ script/generate controller dogs
$ script/generate controller uploads

Models

The Dog model just has a name for now. The Upload model will represent each uploaded photo. It has a description which the user can enter, a dog_id foreign key, and two attributes for Paperclip: a photo_file_name string and a photo_file_size integer. See the Paperclip documentation for more details on how these (and other) special attributes are used.

# models/dog.rb
 
class Dog < ActiveRecord::Base
  has_many :uploads,
           :attributes => true,
           :discard_if => proc { |upload| upload.photo_file_size.nil? }
end

I updated the Dog model to have many uploads. The “:attributes” parameter is part of attribute_fu, and lets the Dog controller create associated models at the same time a new Dog is created. The “:discard_if” tells attribute_fu not to create an associated model if certain conditions are met — in this case, if there’s no file that was uploaded. This is needed because we may put many file upload fields in the “new Dog” form, but a user may only select one or two photos. In that case, all the blank form elements would be submitted too, and created as empty Upload instances.

#models/upload.rb
 
class Upload < ActiveRecord::Base
  belongs_to :dog
  has_attached_file :photo,
                    :styles => {
                      :thumb => ["100x100", :jpg],
                      :pagesize => ["500x400", :jpg],
                    },
                    :default_style => :pagesize
end

This Upload model belongs_to a dog, to match the has_many association above. “has_attached_file” is part of Paperclip, and the “photo” attribute must match the prefix of the database fields above (e.g. photo_file_name). Paperclip can automatically re-size photos and keep multiple versions, based on the “styles” hash. Here we create a small thumbnail, and a version good for embedding in web pages. In both cases, the photo is converted to a JPEG if needed. The original photo is stored with a style of “original”.

Views

The main view we’re concerned about is the “new dog” view. This will give the user a form to add a new dog, and upload photos to associate with the dog.

# views/dogs/new.html.erb
 
<h1>New dogh1>
 
< % form_for @dog, :html => { :multipart => true } do |f| %>
  < %= f.error_messages %>
 
  <p>
    < %= f.label :name %><br />
    < %= f.text_field :name %>
  p>
 
  <div id='uploads' style='border: 1px solid silver'>
    < %= f.render_associated_form(@dog.uploads, :new => 5) %>
  div>
 
  < %= f.add_associated_link('Add another photo', @dog.uploads.build) %>
 
  <p>
    < %= f.submit "Create" %>
  p>
< % end %>

The first part of the above view gives us a form for a new dog instance. It also calls attribute_fu’s “render_associated_form” to create a nested form for upload instances. Here we create five blank instance forms. The instance forms are generated by a partial named after the associated model… in this case the partial is _upload.html.erb (below). We also add a link to create additional file upload forms on the fly via attribute_fu JavaScript with the “add_associated_link” call.

# views/dogs/_upload.html.erb
 
<p class='upload'>
  <label for="upload_description">Description:label>
  < %= f.text_field :description %>
 
  <label for="upload_photo">Photo:label>
  < %= f.file_field :photo %>
 
  < %= f.remove_link "remove" %>
p>

The partial is pretty standard, just a field for our photo description, and a file upload field. The only unusual part is attribute_fu’s “remove_link” JavaScript call, which lets the user remove one of the photo upload forms.

Here’s what the completed form looks like:

Controllers

The nice thing about attribute_fu is that no special controller logic is needed! The dog controller is just a standard RESTful controller! When the form is submitted, the dog instance is created, and all the associated upload instances are created simultaneously:

# controllers/dogs_controller.rb
 
class DogsController < ApplicationController
 
  def index
    @dogs = Dog.find :all
  end
 
  def show
    @dog = Dog.find params[:id]
  end
 
  def new
    @dog = Dog.new
  end
 
  def create
    @dog = Dog.new params[:dog]
 
    if @dog.save
      flash[:notice] = 'Dog was successfully created.'
      redirect_to @dog
    else
      render :action => "new"
    end
  end
 
end

Image cropping

The last feature to add is the image cropping functionality. We’re going to do this in the “edit upload” action. The edit form will show the image, and let the user drag a rectangle around the region to crop. The form will return the rectangle coordinates, and we’ll override the standard update_attributes method of the Upload model, to perform the crop.

In the main view template, I’m including the jsCropperUI JavaScript code and its dependencies. The scripts go in the public/javascripts directory when you install the jsCropperUI code.

# views/layouts/application.html.erb
 
< %= javascript_include_tag 'cropper/lib/prototype.js' %>
< %= javascript_include_tag 'cropper/lib/scriptaculous.js?load=builder,dragdrop' %>
< %= javascript_include_tag 'cropper/cropper.js' %>

The “edit upload” view is a little complex. Here’s how it’s put together:

# view/uploads/edit.html.erb
 
<h1>Editing uploadh1>
 
< % form_for(@upload) do |f| %>
  < %= f.error_messages %>
 
  <p>
    < %= f.label :description %><br />
    < %= f.text_field :description %>
  p>
 
  <!-- CROP FORM -->
  <div id='cropwrap'>
    < %= image_tag @upload.photo.url, :id => 'cropimage' %>
  div>
 
  <div id='cropresults'>
    < %= f.label 'x1' %>
    < %= f.text_field 'x1', :size => 6 %>
    <br />
    < %= f.label 'y1' %>
    < %= f.text_field 'y1', :size => 6 %>
    <br />
    < %= f.label 'width' %>
    < %= f.text_field 'width', :size => 6 %>
    <br />
    < %= f.label 'height' %>
    < %= f.text_field 'height', :size => 6 %>
    <br />
  div> <!-- cropresults -->
  <!-- END CROP FORM -->
 
  <p>
    < %= f.submit "Update" %>
  p>
< % end %>

In addition to the edit field for our model’s normal attributes (“description”, in this case), we also have fields for the geometry of our crop rectangle. These fields will be filled in automatically by the jsCropperUI. We also need to display the image we’re cropping, and identify it with a CSS ID so the jsCropperUI can attach to it. To attach the jsCropperUI, we need to add some JavaScript to the page:

# view/uploads/edit.html.erb
 
<script type="text/javascript" language="JavaScript">
function onEndCrop( coords, dimensions ) {
  $( 'upload_x1' ).value = coords.x1;
  $( 'upload_y1' ).value = coords.y1;
  $( 'upload_width' ).value = dimensions.width;
  $( 'upload_height' ).value = dimensions.height;
}
 
Event.observe( window, 'load', function() {
  new Cropper.Img('cropimage', {
    minWidth: 50,
    minHeight: 50,
    displayOnInit: true,
    onEndCrop: onEndCrop
  } );
} );
script>

This attaches the jsCropperUI to the photo we want to edit, identified by the “cropimage” CSS ID. It also sets up a callback function to set our form fields with values based on the crop coordinates. Check the jsCropperUI documentation for more details.

Here’s what our page looks like:

There are two other things to be aware of here:

First, the “image_tag” is displaying the “:default” style of the uploaded image. If you look back to the Upload model code above, that means it’s an image that’s been scaled down to at most 500×400 pixels. This is important, since we don’t want to display a full-sized image if someone’s uploaded a giant photo from a 10 megapixel digital camera!

Second, we’re telling the form helpers to create form fields for attributes like “x1” and “width”, which aren’t attributes of our Upload model. To make this work, we need to add some virtual attributes to the model (virtual, because they’re not associated with any fields in our database table):

# models/upload/upload.rb
 
attr_accessor :x1, :y1, :width, :height

Now, when the form is submitted, the standard “update” action in the uploads controller will call “update_attributes”, just like any other form and controller:

# controllers/uploads_controller.rb
 
def update
  @upload = Upload.find params[:id]
  if @upload.update_attributes params[:upload]
    flash[:notice] = 'Upload was successfully updated.'
    redirect_to @upload
  else
    render :action => "edit"
  end
end

To make this actually crop the image, we need to override the update_attributes method in the upload model:

# models/upload.rb
 
require 'RMagick'
 
def update_attributes(att)
 
  # Should we crop?
  scaled_img = Magick::ImageList.new(self.photo.path)
  orig_img = Magick::ImageList.new(self.photo.path(:original))
  scale = orig_img.columns.to_f / scaled_img.columns
 
  args = [ att[:x1], att[:y1], att[:width], att[:height] ]
  args = args.collect { |a| a.to_i * scale }
 
  orig_img.crop!(*args)
  orig_img.write(self.photo.path(:original))
 
  self.photo.reprocess!
  self.save
 
  super(att)
end

What’s going on here? We want to crop our original, high-resolution image, but we used a scaled down version in the jsCropperUI to select our crop rectangle. To adjust for that, we need to check the pixel width (“columns”) for the original and scaled images, and compute a scaling factor. Next we prepare the four arguments for the RMagick crop function, which expects x1, y2, width, and height. We create an array with the arguments, convert to integer (they’re still strings, since they came from a text field in the form), and scale them. We can pass multiple arguments to a method by prefacing an array with an asterisk. That’s something new I came across in this project, and I suspect may come in handy again some time. The image is cropped, and then written back to disk, replacing the original.

“reprocess!” is a Paperclip method to re-generate thumbnails and other scaled versions of an image from the original. And finally, we call the true “update_attributes” method to handle things like the “description” or other actual attributes of our model.

So there you have it… a framework to deal with multiple image uploads to an associated model, with user-selected cropping.

I’ve packaged up the example code into a GitHub repository. It’s a fully functional Rails 2.2.2 app, assuming you have the Paperclip gem and RMagick already installed.

Comments and improvements are welcomed.

Ruby memory leakage

I’ve been working on a Ruby on Rails web application to track and compare Xbox player scores and achievements—a direct ripoff of a similar Perl/Mason app that Ty wrote. I recently discovered that the web site started getting very slow to respond, and Ruby was using up lots of memory.

I did a fair bit of Google research into memory usage and garbage collection in Rails, but couldn’t find any smoking gun. I did come across a really tiny Memory Usage Logger for Rails, that just adds the process memory size to the server log after each request. I added this handy code, and used the Apache Benchmarking Tool to send several hundred web requests to my app, and graphed the results. Memory was definitely increasing linearly with the number of requests.

After doing some basic troubleshooting and code analysis, I was getting nowhere. Instead, I created a new, simple Rails app. Just an empty app, with one model and controller generated with Rails’s scaffolding and default views. This exhibited the same ever-growing memory problem. Weird. Rails is used widely for major web sites, so I knew this couldn’t be a widespread problem. I next installed Ruby Enterprise Edition, which was fortunately a fast and painless process. When I restarted my application and re-ran my stress test, memory was completely flat!

I notice now that although both versions of ruby (standard and Enterprise Edition) are version 1.8.6, they are different patchlevels, so I’m not sure if the memory fix is due to REE or due to some more recent ruby patches. Either way, I’ll probably stick with REE in the future.