Author Archive for matthew

The more I have to hack ActiveRecord’s guts

to make it do something right - the more I’m tempted to rewrite it from scratch.

That whole “I never took (or never understood) a database theory course at university, so I’m just going to pretend it doesn’t matter, and that a relational database may be treated as nothing more than a glorified filestore for my Objects” attitude just doesn’t cut any ice with me, but seems sadly prevalent within the Rails community. Yes, good object-oriented design is really important - but you need a sophisticated relational approach in order to get a handle on the data model behind any kind of non-trivial inheritance & mixin hierarchy, and to persist it in a logically sound and efficiently-indexable fashion.

What’s more, I contend that your ORM tool needs to understand something of the relational algebra in order to represent what is going on in a sufficiently elegant, flexible way - otherwise you’ll always be piling hack ontop of hack whenever you want to map the results of a moderately complex query over to the OO side. Joining SQL strings together is not the way forward - these things are syntax trees with structure!

Ahem. Sorry if I sound exasperated. SQLAlchemy on the Python side gets this kind of thing ABSOLUTELY SPOT ON, and dare I say it, so do some of the Java ORM frameworks (shame about the XML config files and Java’s tendency towards boilerplate code and bloated syntax, but don’t throw the baby out with the bathwater Rails-ers)

The problem with Rails’ ActiveRecord is that it’s neither here nor there - neither the kind of lightweight, simple ‘map objects to database rows and nothing much else’ approach originally implied by Fowler’s Active Record design pattern - nor the kind of powerful ORM tool which is capable of turning the kind of tricks that are increasingly demanded of it in anything like an elegant fashion.

It seems the Rails team’s solution to some of the endemic problems with ActiveRecord’s messy guts is to wrap them up in a huge plastic bag known as caching - an acceptable pragmatic approach, I accept, in many situations, but one which would not be nearly so necessary had a different approach been taken to ActiveRecord’s architecture.

I feel that superior approach needn’t have come at the cost of ActiveRecord’s ‘convention over configuration’ and ‘easy to get started with’ benefits either - it just would have required a little more forethought and a little humility in learning about the Relational Model before attempting a tool which maps complex data models to a Relational Database.

Crap, I’m starting to sound like Fabian Pascal now aren’t I.

We moved to Git

I’ll admit it took a while to convince me of the merits of decentralised version control. But after a really nasty couple of merges, enough was enough. We’ve dropped SVN in favour of Git - which seems to be the biggest contender.

The main selling point for me was that it keeps track of all the metadata surrounding merges - information which SVN forgets about, requiring you to document your own merge metadata in commit messages, and scour the SVN logs trying to figure out which changesets have already been merged, when, whence, whither, by whom.

Other wins:

  • It’s really fast
  • Easy creation and painless switching between local ‘topic branches’, which you can create for each feature you’re working on and merge into eachother easily
  • Easy to swap work-in-progress patches with other developers without having to commit to a centralised trunk
  • Easy to make lots of quick local / offline commits, which you can later crunch down into one whn merging, if you want
  • Have the whole history available locally, and lots of backups of the repository

Some minuses:

  • The git-svnimport tool appears slightly buggy. Don’t count on it to import your SVN branches properly, especially if you moved them around at any point in the SVN history. In our case the branches it created only contained the files which had changed since the branch was created in svn - rather than fiddle around sorting them out, I just deleted them and re-created them from the git master branch by applying an svn diff. Which is OK if the history of your branches isn’t super important, but less than ideal otherwise. I also found that a small number of files were missing from the trunk, and had to be re-added manually - I suspect git-svnimport gets a bit lost when files have been moved around in non-trivial ways in the SVN history.
  • There is something of a learning curve with Git, especially when it comes to more complex merging, branching, tagging, cherry-picking tasks which were the reasons I first wanted to move to Git. I found this set of lessons learned helped a lot, ontop of the ‘Git from SVN’ tutorial. Once you know what you’re doing though, it’s faster and a lot less fiddly at merging than SVN.

Baby Aborigine

A new Damascu Bite opened on Brick lane. They do rather tasty Syrian food. More amusingly though, their take-away menu features:

Mix Yalanjy (Staffed vine leafs, Baby corrugate, Baby aborigine)

Good news if you’ve been waiting years for the chance to tuck into minature native Australians…

Multiple inheritance in Javascript

Javascript’s prototype-based OO would be so much better if objects could have multiple prototypes. One could then build arbitrary Directed Acyclic Graphs of object delegation, and build pretty much any kind of desired object-oriented semantics, any kind of inheritance scheme or method resolution order in the world ontop of that. Raw Power.

Currently all the approaches to building class-based inheritance in javascript either rely on copying attributes (which rather defeats the point of prototype-based OO semantics in my mind), or are limited to a single superclass (no mixins or multiple superclasses).

I think this is the main weakness in the argument that prototype-based OO is better than class-based. Yes - but only if you can spare me multiple prototypes. In my opinion I think the ECMAScript standards should evolve more in this direction, and less in the direction of hacking class-based inheritance into the language. Just add multiple prototypes and provide class-based inheritance as a library.

ActiveSupport’s dependency auto-class-loading code is officially Too Damn Magical For Its Own Good.

Can I just say that, after spending the better part of a day over the last week having to hack around its mysterious and messy guts, ActiveSupport’s dependency auto-class-loading code is officially Too Damn Magical For Its Own Good. Whenever you have non-trivial dependencies between classes and modules in your project, mysterious class loading bugs will often cripple your code in hard-to-track-down ways. Judicious use of ‘require_dependency’ is your friend here, although too much use of it will lead to further problems. Latent cyclic dependencies are often the culprit, and can be fixed or hacked around, but discovering them from the rather opaque exception (’expected foo.rb to define Foo’) and stack trace can be hard.

The following Rails bug: http://dev.rubyonrails.org/ticket/6001 doesn’t exactly help matters - if any of your classes refer to ApplicationController and are themselves referred to (and hence auto-loaded) during the environment loading process (environment.rb etc), this will lead to problems, as Rails does a straight ruby ‘require’ on application.rb after environment.rb has run. As it’s not called application_controller.rb, the auto class loading doesn’t work normally for it, which can cause further problems if ApplicationController gets tied up in any other dynamic dependency chain reloading issues.

Now obviously these issues are necessarily complex - loading classes from files with complex dependencies is a tricky problem when programming with a dynamic language. Although I can’t bring myself to use Squeak, the ‘directly edit the live code image’ approach Smalltalk/Squeak takes seems to avoid this problem of dynamically reloading classes from files during development. It’s something that needs to be thought about at the level of the Ruby language, and while the ActiveSupport hacks are admirable in their cleverness, cleverness that only works cleanly in the simplest cases is a mixed blessing.

‘Hijax’ - aren’t buzzwords great

Turns out, unbeknown to me, someone had already invented a buzzword for my “optional ajax navigation” technique.

Hijax” - because it Hijacks clicks on anchor tags and form submissions, and takes them over with Ajax.

I wonder if his code deals with all the same tricky corner cases that mine does, though.

Turns out someone’s made a dynamic history library for jQuery too. It looks a lot more modern and less verbose than the dhtmlHistory code we’re using at the moment - any kindly souls fancy porting it to work with prototype.js ?

Of Nouns and Verbs

As the site develops we’ve been putting a lot of thought into how to architect things in a suitably general way - to build a playground for users with a set of objects they can play with, and a consistent set of actions they can perform. We found the following approach to be one rather nice way of thinking about it (cue monster diagram which Wordpress refuses to format nicely):

VERB \NOUN STASH (save / collect / organize) STALK (watch / observe / subscribe to) CHAT ABOUT WRITE ABOUT TAG BUY / DOWNLOAD UPLOAD / create
MUSIC / artist / label Save music for download queue; make playlists; organise music library Watch for new releases and events by artist or label; who’s listening to an artist/release/track? Bitch about music Review releases; Interview/profile artists; write music news Categorise music (genres?) Buy music! or download unlimited music if subscriber Upload your own tracks; create mixes and playlists
WORDS Bookmark content; compile your own editorial selections See who’s chatting about a piece Bitch about someone’s news/review/etc   Categorise writing; tell us what it’s about   Upload your own content
CHATS Organise your chat history; bookmark chats Get alerts about new posts in a chat     Categorise chats; tell us what they’re about   Start a chat! Participate!
PEOPLE Friends list; create user groups Get alerts on what friends are up to! Bitch about that guy/gal Profile / interview / write a testimonial about someone Categorize your friends (?)   Sign up! Invite friends!
EVENTS / venues / locations Bookmark events; add to your calendar Watch for new events at a venue / near to a location; Look for new attendees of an event Bitch about that band you saw the other night Review / preview events Categorise events Buy tickets; download flyers Add your own events
PICTURES Make photo galleries   Bitch about that pic your mate took Attach photos to content Tell us what photos are of; tag your friends Download photos and cover art Upload your photos and art

Obviously some of these things are a while off in terms of development - but this should give eager loudplayers an idea of what to expect!

From a software architecture standpoint, I found this an interesting exercise too. On a data modelling / OOD level, our Nouns correspond roughly with classes or data types, and our Verbs with interfaces / method signatures. Of course in practise it’s not quite as simple as the diagram’s layout suggests. But thinking about things in this way does help to identify a useful set of shared interfaces. Rather than keep tacking on additional, distinct modules of functionality (let’s throw in an events section!), we’d like to add nouns and verbs to our playground.

On the user interface level, too, this kind of analysis helped to clarify thinking on how to present a structured, consistent user interface, and how to structure navigation. We have a lot of work to do on this front too, but I feel we’re making great progress. John, our graphic designer and Tim our new recruit and go-to man for UI development, have been battling valiantly with photoshop, css, rails helpers, partials and view code. The task has been to build a suitably general and standardized way of presenting different types of object across the site - music, words, chats, people, artists and so on, all need representing consistently at different sizes and in different contexts, together with icons and links for the actions users can perform on them. It’s been frustrating at times but I think we’re beggining to nail it, and it’ll pay large dividends in the future in simplifying and providing structure for further UI development.

Log rotation with Rails and cronolog

Some problems doing log rotation on Rails’ production.log:

  • The built-in rotation feature of the Logger class doesn’t play nice with concurrent fastcgi processes
  • Rails doesn’t appear to support the more advanced Log4r library, despite claiming to in a few places. In particular benchmarking seems to break it
  • Using logrotate proves annoying as you need to kill the fastcgi processes after rotating the logs

The solution appears to be relatively simple though - just use the standard logger, but pass it a pipe to cronolog rather than a filename. Don’t know why this wasn’t documented anywhere! So in environment.rb, or environments/production.rb say, something like:

# Use cronolog for log rotation
cronolog_io = IO.popen(
    'cronolog /space/log/%Y/%m/%d/production.log', 'w')
config.logger = Logger.new(cronolog_io)

We no longer support IE6

As a web developer who’s been through a lot of pain in the last 5 years trying to support this backwards browser, I have to say this was rather a relief. We’ve previously attempted to get everything functional, if not feature-complete in IE6, but after the latest round of changes we just didn’t feel it was worth the considerable additional effort. IE6 users will now see a politely-worded message:

You appear to be using Internet Explorer 6. While Playlouder might work OK for you, it’s not supported by us, and there will probably be a few problems. For a better browsing experience, why not upgrade to Internet Explorer 7, or get Firefox. Either of these will ensure a great improvement in your experience of the Playlouder site, and should improve your general internet experience and computer security too!

We didn’t take the decision lightly - but we feel it’s the way forward. Google analytics tell us that around 60% of Playlouder users (loudplayers?) are accessing the site from Windows, and of those 60% using IE6. So that’s a sizable 36% chunk of our audience we’re asking to upgrade. Why?

  • IE6 is, to put it politely, extremely backwards and its lack of standards support is unrivaled by any other widely-deployed browser
  • With IE6 users now in the minority, the web needs compelling sites to start requiring an upgrade for IE6 users, to really spur upgrades and push that figure down even further
  • We’re already requiring invites for new sign-ups, and feel that the upgrade isn’t too much of an extra step for users who’ve already had the motivation to seek an invite
  • We’re not forcing any ideology on users - while I’d personally prefer they use Firefox, I’d be equally happy if they carry on using what appears to be their preferred browser (IE), provided they upgrade to the latest version. IE7 is easily available and people need to be told that the older version just won’t cut it on the web any more.
  • The additional development costs just aren’t worth it for us at this stage - especially bearing in mind that given another year or so IE6 should be consigned to the dustbin of browser history.

A subtle Safari 2 bug, and parallel Safari installs

In case anyone else runs into this:

Safari 2 will not return any innerText for an element which is display: none. Safari 3, however, has fixed this bug, which made it hard to track down after the Safari 3 beta install replaced my old copy of Safari 2.

I’ve now gotten the two running in parallel though - follow the instructions here and follow to the letter, including all the renaming. I found that unless I copied the old backed-up Safari to /Applications with the name Safari.app, there were some nasty issues with other apps loading the Webkit framework from the wrong place. In particular Drosera wouldn’t start, trying to use the old Webkit framework that was copied into the bastardized Webkit install, even when I overrode its DYLD_FRAMEWORK_PATH manually to use Safari 3’s. No Drosera rather defeats the whole point of having Safari 3 for me, so yeah. Watch out!