Archive for the ‘Ruby on Rails’ Category

Rake Task for MySQL Backup to Amazon S3

Saturday, August 25th, 2007

There are a couple of these around, but I rolled my own because I wanted something quite simple.

# S3 Backup Task for MySQL
# Assumes InnoDB tables
# Database User needs the "reload" permission on the database (for --flush-logs in mysqldump)
#
# Stores files in Amazon S3 using the excellent AWS Gem: http://amazon.rubyforge.org/
# For information about Amazon S3: http://aws.amazon.com/s3
#
# Installation
# 1) Install AWS Gem
# 2) Enter your S3 Bucket, access_key_id and secret_access_key in this file
# 3) Run (rake db:backup)
#
# Inspired by code from:
#   http://blog.craigambrose.com/articles/2007/03/01/a-rake-task-for-database-backups
#   http://www.rubyinside.com/advent2006/15-s3rake.html

namespace :db do
  require "aws/s3"

  desc "Backup database to Amazon S3"

  task :backup => :environment do
    BUCKET = "bucket"

    begin
      AWS::S3::Base.establish_connection!(
        :access_key_id     => "access_key_id ",
        :secret_access_key => "secret_access_key "
      )

      db_config = ActiveRecord::Base.configurations[RAILS_ENV]

      backup_path = "db/backup"
      File.makedirs(backup_path)

      datestamp = Time.now.strftime("%Y%m%d%H%M%S")
      file_name = "#{RAILS_ENV}_#{db_config['database']}_#{datestamp}.sql.gz"

      backup_file = File.join(backup_path, file_name)

      sh "mysqldump -u #{db_config['username']} -p#{db_config['password']} --single-transaction --flush-logs --add-drop-table --add-locks --create-options --disable-keys --extended-insert --quick #{db_config['database']} | gzip -c > #{backup_file}"
      puts "Created backup: #{file_name}"

      puts "Storing file in S3: #{BUCKET}"
      AWS::S3::S3Object.store(file_name, open(backup_file), BUCKET)

      puts "Backup Complete"
    rescue AWS::S3::ResponseError => error
      puts error.response.code.to_s + ": " + error.message
    end

  end

end

Speaking at Melbourne Ruby User Group

Friday, August 24th, 2007

I’m speaking at the Melbourne Ruby User Group next week. Either on using Amazon Web Services. or hacking with Prototype (probably focussed on the new 1.6.0 features).

Lineup & Location

ThoughtWorks
Level 11, 155 Queen St

  • Pete Yandell – SpiffyAttributes
  • Clifford Heath – HAML
  • Mike Bailey – Jester
  • Ben Schwarz  – JS Hacking
  • Toby Hede – Amazon AWS and/or Prototype Hacking
  • Marty Andrews – Logging and/or the Rails Plugin Architecture

REST: I don’t quite get it

Friday, August 10th, 2007

I’ve been playing with RESTful Rails on one of my projects. I must admit to being a bit perplexed.

You have to bend your code to get REST working properly, which smells to me.

For example, when editing a model, you need to push a hidden element into your form to spoof a HTTP PUT method. Rails automates some of this, but … why?  What do you gain by forcing the system to only accept puts for particular actions, particuarly when browsers need to be tricked into playing nicely?  What is lost by having an update action accept POSTs?

Anyway, I will keep playing …

2 things I most loved about Ruby on Rails this week

Thursday, August 9th, 2007
  1. attachment_fu
    From install to an Image Upload with Amazon S3 storage in about 10 minutes. See the attachment_fu tutorial.

  2. RESTful rails
    An XML API for nothing? Why thanks, Rails.

The REST stuff is really fun to play with … I am finding it breaks a little in the real-world with non-trivial use-cases. Start adding auto-completes and other AJAX elements to your page and you start bending the RESTful model a little. I am figuring it’s best to have the core REST architecture and build on it, rather than bend my code to adhere mindlessly to some REST ideal.

I {heart} Rails.

Everything you know about scaling is wrong

Friday, August 3rd, 2007

Scalability of the favourite argument topics of the technically inclined. Having developed some fantastic, all-singing, all-dancing, user-generated social networking Web 2.0 platform*, someone will invariably ask:

“Oh, you used {LANGUAGE X}”
“Will it scale?”

You have to imagine the slightly derisive tone for yourself.

Trouble is, languages don’t scale, systems do.

Assessing scalability on the basis of a particular programming language is like saying to someone who is writing an Encyclopedia: “English won’t scale”.

Systems scale.

If you wrote your Encyclopedia the same way you wrote your product brochure, it wouldn’t scale – small page format, lots of pictures, minimal text with plenty of whitespace. If you write an Encyclopedia, you need to develop a system for handling the information – table of contents, indexing, cross referencing, multiple volumes, alphabetical organisation, thin paper, multiple columns of text, readable fonts.

The language used to write the Encyclopedia or Brochure is secondary to the system that delivers it.

The same is true of Web Applications.

Languages are irrelevant to scale and frameworks (being very close to languages and essentially an extension) only have a minimal impact.

I moved to Ruby on Rails from PHP. The standard argument against Rails from PHP developers is that “Rails won’t scale”**. However, this focus misses the point entirely. I have seen arguments from PHP developers who suggest using single quote for strings ‘ rather than double quotes “, because single quotes parse faster in PHP. This thinking is radically broken.

Scalability has nothing to do with processing performance at the language level.

Scalability is about the system as a whole.

At some point as an application scales, it will invariably require multiple servers and multiple databases, and there is very little any language or framework can do to mitigate this requirement. These requirements are system level, not language level – your only real option is to be ready to build your system out accordingly as it grows. The only trouble you find is when you have made decisions early on that limit the way your application can grow out.

As a corrolary to the scaling issue, my final point is this:

You aren’t going to scale

Call me cynical, call me pessimistic, but lots of people build for scale prematurely.

The focus should always be on creating great user experience.

Unless you have the load to warrant a particular system decision you should not be creating that system (but always within the framework of sensible architectural decisions). Scaling issues are often unpredictable (you don’t know your load profile until you hit it, or are hit by it) but worrying about them before you have to wastes valuable developer resources on infrastructure rather than the interface.

In the Web 2.0 world, scaling problems are a sign of success, but the focus should be on the user, not the system.

* We don’t develop applications or, god forbid, sites anymore, we develop platforms.

** I am aware the argument is a bit confused here, because PHP is compared to Rails, when Rail is a framework based on the Ruby Language and should be compared to a PHP framework like CakePHP.

Note to self

Friday, July 6th, 2007

A couple of things that have stumped me today.

A syntax error in database.yml caused the TextMate model generator to die silently. In case of doubt, check the config.

After installing rFacebook on Dreamhost, the application would not pick up the gem without unpacking it and requiring the explicit path in production.rb:

require "#{RAILS_ROOT}/vendor/rfacebook-0.6.2/
lib/facebook_rails_controller_extensions"

Useful resources for building Facebook Applications with Rails

Monday, June 25th, 2007

I’ve been looking at building a Facebook app using Ruby on Rails.

Some handy resources:

My favourite Ruby Gem is Magic

Tuesday, June 19th, 2007

On Ruby asks in June’s Blogging Contest:

“Other than Rails, what is your favorite Ruby Gem, and why?”

My favourite Ruby Gem is Dr Nics Magic Models.

Magic Models weaves a dark spell on your code that makes your ActiveRecord Models just disappear. It’s glorious for demonstrating to the skeptical the possibilites affored by Ruby, and it makes prototyping and development even more efficient. As Dr Nic says:

In a way, MM works like scaffolding does at the controller level – it lets you get your application “working” faster even though you’ll probably rewrite it in the end anyway.

Starting with Magic Models is a matter of installing the Gem and requiring it in your application:

gem install dr_nic_magic_models
require 'dr_nic_magic_models'

Now you have Models that appear based on your defined database tables, no other code required. So if you have a People Table, you automagically have a Person Model with attributes matching the columns defined in the table.

If you crack open the gem, you find the Magic isn’t so magic after all. The crucial action occurs in const_missing.

def const_missing(class_id)
  begin
    return normal_const_missing(class_id)
  rescue
  end
  @magic_schema ||= DrNicMagicModels::Schema.new self
  unless table_name = @magic_schema.models[class_id]
    raise NameError.new("uninitialized constant #{class_id}")
      if @magic_schema.models.enquired? class_id
  end
  superklass = @magic_schema.superklass || ActiveRecord::Base
  klass = create_class(class_id, superklass) do
    set_table_name table_name
    # include DrNicMagicModels::MagicModel
    # extend DrNicMagicModels::Validations
  end
  klass.generate_validations
  @magic_schema.inflector.post_class_creation klass
  klass
end

const_missing is called when the Ruby Interpreter cannot find a constant. In the example above, if we call the Person model (using Person.find, for example), const_missing is called and Magic Models goes into action – checking the database for a corresponding table and creating a class with the appropriate attributes and validations. The rest writes itself.

Magic Models looks like magic to those uninitiated in the wonders of a truly dynamic language.

I use them to get a leg-up on new development. I have also used them to demonstrate to some colleagues the things that are possible with Rails – scaffolding for free, testing baked-in, easy AJAX and code that isn’t there but works anyway.

It’s worth learning, just to see the expression on the PHP and Java guy’s faces.

If you’re looking for more: