Top Secret Project

But will it scale?

Wednesday, March 12th, 2008

I am very pleased to announce the availibility of the Hypothetical Megastructure “but will it scale” t-shirt.

The t-shirt provdes the wearer with the following powers:

Effortlessly win any technical discussion instantly!
Impress your friends!
Immediately dismiss any rival language or framework!

“BUT WILL IT SCALE“?

Available in LARGE, EXTRA-LARGE, GRID, and the apparently defunct N-TIER.

I like to wear mine in a cluster (simply buy two or more t-shirts and wear simultaneously).

Amazon SimpleDB: death of the database?

Saturday, December 15th, 2007

With the announcement of Amazon SimpleDB Scoble has asked if Amazon Web Services are going to kill MySQL and Oracle.

I think the short answer is no, but the game is changing rapidly, and Amazon is at the vanguard.

SimpleDB basically acts like a big structured bucket.
The Model is represented by Domains. A Domain can have Attributes. Attributes are key/value pairs.

There is no schema attached to Domains, you can PUT any combination of attribute-values in the domain:PUT (item, 123), (description, sweater), (color, blue), (color, red) PUT (item, 456), (description, dress shirt), (color, white), (color, blue) PUT (item, 789), (description, shoes), (color, black), (material, leather)

The real issue here is that developers are strongly tied to the RDBMS. Frameworks assume you are running on top of a relational database, so there is an instant barrier to entry. However, I suspect it won’t be long before the first set of libraries and drivers for SimpleDB are developed.

The much bigger issues are in the features that SimpleDB doesn’t have:

No ACID
No transactions (see ACID, above)
No relationships
No data integrity
No SQL
Latency

SQL is actually pretty incredible - it makes accessing structured data very simple. If you’ve ever played with other types of data store you miss SQL almost instantly (Prevayler, anyone?).

All of that said, Amazon continues to lead the way with Web Services. Consider SimpleDB a warning shot to the database incumbents.

Everything you know about scaling is wrong

Friday, August 3rd, 2007

Scalability of the favourite argument topics of the technically inclined. Having developed some fantastic, all-singing, all-dancing, user-generated social networking Web 2.0 platform*, someone will invariably ask:

“Oh, you used {LANGUAGE X}”
“Will it scale?”

You have to imagine the slightly derisive tone for yourself.

Trouble is, languages don’t scale, systems do.

Assessing scalability on the basis of a particular programming language is like saying to someone who is writing an Encyclopedia: “English won’t scale”.

Systems scale.

If you wrote your Encyclopedia the same way you wrote your product brochure, it wouldn’t scale - small page format, lots of pictures, minimal text with plenty of whitespace. If you write an Encyclopedia, you need to develop a system for handling the information - table of contents, indexing, cross referencing, multiple volumes, alphabetical organisation, thin paper, multiple columns of text, readable fonts.

The language used to write the Encyclopedia or Brochure is secondary to the system that delivers it.

The same is true of Web Applications.

Languages are irrelevant to scale and frameworks (being very close to languages and essentially an extension) only have a minimal impact.

I moved to Ruby on Rails from PHP. The standard argument against Rails from PHP developers is that “Rails won’t scale”**. However, this focus misses the point entirely. I have seen arguments from PHP developers who suggest using single quote for strings ‘ rather than double quotes “, because single quotes parse faster in PHP. This thinking is radically broken.

Scalability has nothing to do with processing performance at the language level.

Scalability is about the system as a whole.

At some point as an application scales, it will invariably require multiple servers and multiple databases, and there is very little any language or framework can do to mitigate this requirement. These requirements are system level, not language level - your only real option is to be ready to build your system out accordingly as it grows. The only trouble you find is when you have made decisions early on that limit the way your application can grow out.

As a corrolary to the scaling issue, my final point is this:

You aren’t going to scale

Call me cynical, call me pessimistic, but lots of people build for scale prematurely.

The focus should always be on creating great user experience.

Unless you have the load to warrant a particular system decision you should not be creating that system (but always within the framework of sensible architectural decisions). Scaling issues are often unpredictable (you don’t know your load profile until you hit it, or are hit by it) but worrying about them before you have to wastes valuable developer resources on infrastructure rather than the interface.

In the Web 2.0 world, scaling problems are a sign of success, but the focus should be on the user, not the system.

* We don’t develop applications or, god forbid, sites anymore, we develop platforms.

** I am aware the argument is a bit confused here, because PHP is compared to Rails, when Rail is a framework based on the Ruby Language and should be compared to a PHP framework like CakePHP.

Archive for the ‘Scalability’ Category

But will it scale?

Amazon SimpleDB: death of the database?

Everything you know about scaling is wrong

Author

Subscribe

Pages

Archives

Categories