Archive for the 'Rails' Category

Meet us at LRUG

Published
by
matthew
9 months, 2 weeks ago
in Development, Geekery, Rails, Software architecture, The web and User interface
. 0 Comments

This monday I shall be giving a talk at the London Ruby Users Group. I’ll be giving a tour through our experiences building a modular composable Widget UI framework ontop of Ruby on Rails. Some of the steps we took along the way, problems we encountered, and a tour of the results.

There’s also a juicy debate to be had comparing the REST-driven web application architecture pushed by the Rails project, with our more modular widget-based approach, and in deciding what’s appropriate for your application. For those in the know, comparisons abound with the approach taken by Avi Bryant’s Seaside framework, and the Apotomo plugin already in development for Rails.

The framework comes with a client-side component too, and means of serializing Widgets to constructors for corresponding client-side javascript classes - so the talk may also interest those attempting to do Javascript in an unobtrusive, object-oriented way with Rails.

After that I believe (sincerely hope ;-) ) there are drinks.

The talk is open to all but the venue ask that you Register here; see LRUG for more details.

The more I have to hack ActiveRecord’s guts

Published
by
matthew
10 months, 1 week ago
in Development, Geekery, Rails and Software architecture
. 11 Comments

to make it do something right - the more I’m tempted to rewrite it from scratch.

That whole “I never took (or never understood) a database theory course at university, so I’m just going to pretend it doesn’t matter, and that a relational database may be treated as nothing more than a glorified filestore for my Objects” attitude just doesn’t cut any ice with me, but seems sadly prevalent within the Rails community. Yes, good object-oriented design is really important - but you need a sophisticated relational approach in order to get a handle on the data model behind any kind of non-trivial inheritance & mixin hierarchy, and to persist it in a logically sound and efficiently-indexable fashion.

What’s more, I contend that your ORM tool needs to understand something of the relational algebra in order to represent what is going on in a sufficiently elegant, flexible way - otherwise you’ll always be piling hack ontop of hack whenever you want to map the results of a moderately complex query over to the OO side. Joining SQL strings together is not the way forward - these things are syntax trees with structure!

Ahem. Sorry if I sound exasperated. SQLAlchemy on the Python side gets this kind of thing ABSOLUTELY SPOT ON, and dare I say it, so do some of the Java ORM frameworks (shame about the XML config files and Java’s tendency towards boilerplate code and bloated syntax, but don’t throw the baby out with the bathwater Rails-ers)

The problem with Rails’ ActiveRecord is that it’s neither here nor there - neither the kind of lightweight, simple ‘map objects to database rows and nothing much else’ approach originally implied by Fowler’s Active Record design pattern - nor the kind of powerful ORM tool which is capable of turning the kind of tricks that are increasingly demanded of it in anything like an elegant fashion.

It seems the Rails team’s solution to some of the endemic problems with ActiveRecord’s messy guts is to wrap them up in a huge plastic bag known as caching - an acceptable pragmatic approach, I accept, in many situations, but one which would not be nearly so necessary had a different approach been taken to ActiveRecord’s architecture.

I feel that superior approach needn’t have come at the cost of ActiveRecord’s ‘convention over configuration’ and ‘easy to get started with’ benefits either - it just would have required a little more forethought and a little humility in learning about the Relational Model before attempting a tool which maps complex data models to a Relational Database.

Crap, I’m starting to sound like Fabian Pascal now aren’t I.

ActiveSupport’s dependency auto-class-loading code is officially Too Damn Magical For Its Own Good.

Published
by
matthew
10 months, 4 weeks ago
in Geekery and Rails
. 0 Comments

Can I just say that, after spending the better part of a day over the last week having to hack around its mysterious and messy guts, ActiveSupport’s dependency auto-class-loading code is officially Too Damn Magical For Its Own Good. Whenever you have non-trivial dependencies between classes and modules in your project, mysterious class loading bugs will often cripple your code in hard-to-track-down ways. Judicious use of ‘require_dependency’ is your friend here, although too much use of it will lead to further problems. Latent cyclic dependencies are often the culprit, and can be fixed or hacked around, but discovering them from the rather opaque exception (’expected foo.rb to define Foo’) and stack trace can be hard.

The following Rails bug: http://dev.rubyonrails.org/ticket/6001 doesn’t exactly help matters - if any of your classes refer to ApplicationController and are themselves referred to (and hence auto-loaded) during the environment loading process (environment.rb etc), this will lead to problems, as Rails does a straight ruby ‘require’ on application.rb after environment.rb has run. As it’s not called application_controller.rb, the auto class loading doesn’t work normally for it, which can cause further problems if ApplicationController gets tied up in any other dynamic dependency chain reloading issues.

Now obviously these issues are necessarily complex - loading classes from files with complex dependencies is a tricky problem when programming with a dynamic language. Although I can’t bring myself to use Squeak, the ‘directly edit the live code image’ approach Smalltalk/Squeak takes seems to avoid this problem of dynamically reloading classes from files during development. It’s something that needs to be thought about at the level of the Ruby language, and while the ActiveSupport hacks are admirable in their cleverness, cleverness that only works cleanly in the simplest cases is a mixed blessing.

Log rotation with Rails and cronolog

Published
by
matthew
1 year ago
in Geekery and Rails
. 6 Comments

Some problems doing log rotation on Rails’ production.log:

  • The built-in rotation feature of the Logger class doesn’t play nice with concurrent fastcgi processes
  • Rails doesn’t appear to support the more advanced Log4r library, despite claiming to in a few places. In particular benchmarking seems to break it
  • Using logrotate proves annoying as you need to kill the fastcgi processes after rotating the logs

The solution appears to be relatively simple though - just use the standard logger, but pass it a pipe to cronolog rather than a filename. Don’t know why this wasn’t documented anywhere! So in environment.rb, or environments/production.rb say, something like:

# Use cronolog for log rotation
cronolog_io = IO.popen(
    'cronolog /space/log/%Y/%m/%d/production.log', 'w')
config.logger = Logger.new(cronolog_io)

The relational model, RDF, and data modelling with loose semantics in ActiveRecord

Published
by
matthew
1 year, 2 months ago
in Development, Geekery and Rails
. 0 Comments

Recently I’ve been attempting to model a variety of generic interfaces in our data layer: obvious ones like ‘Taggable’, but also ‘Discussable’, ‘Write-about-able’ (for want of a better word), ‘Nameable’ (useful for matching/splitting/merging records based on names and aliases), ‘Stashable’, ‘Buyable’ (currently ‘Product’). I’ve found some of these present tough challenges to good relational data modelling principles, and push the limits of ORM tools like ActiveRecord in a way that I felt like writing about.

Up until now I’ve resisted the temptation to use ActiveRecord’s ‘Polymorphic Associations’, which are the obvious candidate for implementing these sorts of generic relationships between objects. Why? from a mathematics background I can be a bit of a purist about the Relational Model, and these sorts of dirty ORM tricks violate it in all sorts of assorted horrible ways that would doubtless have Edgar Codd turning in his grave. My main objections are to the ideas of storing foreign keys to different tables’ primary keys in the same column (making proper foreign key constraints impossible), and storing schema metadata in the schema itself as strings (ActiveRecord’s ‘type’ columns).

It also brings up deep-seated philosophical unease about surrogate primary keys in general, but these are a necessary evil which I’ve grown to accept. That said, this plugin will help those who seek to minimize their use in ActiveRecord, which, unchecked, will sprinkle the things around like MSG in a chinese takeaway.

I’ve struggled to find good alternatives to polymorphic associations, though. The problem is this: you have a large-ish variety of tightly-modelled entities (Artist, User, Release, Track, Event, Content, Location, …) and you want the ability to form loose, generic, semantic associations between these objects. There’s a parallel here, on the OO side of the ORM chasm, with Java’s Interfaces, or Ruby’s Mixins, or C++’s multiple abstract superclasses. You want the ability to link to objects using some kind of common superclass/module/interface. Different approaches:

  • Have a common ‘base’ table, used by every object in the database, which is basically just holds a sequence of IDs which all the objects draw from for their primary keys. This is the approach Perl’s Tangram library takes, although is a little tricky in ActiveRecord. Codd-turning-in-grave factor: medium roast, due to the semantically-useless surrogate key, and the loss of static assurances from foreign keys. A foreign key pointing at a global ‘base’ table is rather pointless.
  • Have a table for each generic interface, with its own sequence of IDs as primary key. Objects which implement the interface have an extra column (’taggable_id’ say) with a unique key constraint and a foreign key pointing to the ID from (say) ‘taggable’. This has the advantage that the ‘taggable’ table can also store data common to all objects which implement the Taggable interface. Codd-turning-in-grave factor: slightly lower, since foreign keys can now point at specific interfaces. We’re still throwing surrogate keys around like confetti though. This approach parallels with the Class Table Inheritance ORM pattern, and reveals the way in which the ‘Has a’ / ‘Is a’ distinction beloved of OOP is a rather artificial one at the level of relational data.
  • Create separate join tables for each type of association. So ‘artists_tags’, ‘releases_tags’, ‘tracks_tags’, etc. This is best from a strict relational modelling point of view, but if you want associations that are polymorphic on both ends, the number of join tables can explode as n^2 in the number of object types, and this doesn’t lead to elegant queries or happy programmers. There is a possibility of building a View ontop of these separate tables which makes them all available in a combined table for ActiveRecord, but this I suspect would be highly inefficient. Codd-turning-in-grave factor: low.
  • ActiveRecord’s polymorphic associations. Codd-turning-in-grave factor: high, for the reasons I explained. But, it’s easy to implement with ActiveRecord and it works.

Why did I settle for the ugly solution (polymorhpic associations) in the end?

Firstly, while ‘table for each generic interface’ was my preferred approach, doing this in ActiveRecord, while possible, isn’t terribly pretty - it’s hard to make it look like an ‘implements’ or ‘is a’ relationship on the OO side, rather than ‘has a’. It also means adding a column for each interface to each table which needs to support that interface, and hence more migration work.

Secondly, I realised that the Relational Model isn’t all it’s cracked up to be when it comes to loose semantic modeling. There’s a parallel with the distinction between dynamic typing (Ruby) and static (Java) - sometimes you just need that extra flexibility in your model, and are willing to give up the static assurances to get it.

Taking this to extremes, of course, is the alternative philosophy to data modelling espoused by RDF triple-stores and query languages like SPARQL. Sadly robust support for these technologies is somewhat lacking in common web development frameworks, and the database back-ends don’t appear nearly as highly-developed, scalable and robust as SQL databases.

There are also parallels with single-kinded vs multi-kinded predicate logic, and logic databases with languages like Datalog (cf Prolog). Sadly these databases seem to have fallen by the wayside too in favour of SQL.

On structured Javascript in Rails - first some criticisms

Published
by
matthew
1 year, 4 months ago
in Development, Geekery and Rails
. 0 Comments

Much as I love Rails, and ubiquitous as its use is amongst the current crop of agile ‘Web 2.0′ start-ups, I feel its approach to Javascript and client-side code leaves a lot of room for improvement.

As the new Playlouder app does some quite clever things with Javascript, improve it is what we’ve done. So I thought I’d write a little about our approach and some of the issues we’ve had. Before this though, I’m going to outline some of my issues with the now-standard Rails approach, RJS templates. For the un-initiated, these consist of a domain-specific language embedded in Ruby which allows one to generate a range javascript expressions. Expressions which, when evalled in response to an AJAX request, are rather handy for updating the DOM and running some of the neat range of effects available through the Scriptaculous library.

At first glance, these are wonderfully magical and clever, but on a deeper inspection they leave rather a bad taste in my mouth. Why?

  • The magic makes it look, at a first glance, as though one can genuinely compile Ruby to Javascript in a similar way to, say, Google’s Web Toolkit and its compilation of Java, or HaXe (an interesting platform which I’m keeping an eye on). This is not infact the case, and the magic is limited to a fairly restrictive set of possibilities. I would prefer something a little more transparent, to a clever Ruby DSL whose magic boasts more than it can realistically deliver.
  • They’re geared very much towards a server-centric model where client-side code is kept to a minimum, DOM updates and effects are generally only pushed to the client in response to an AJAX request. If yours is just a matter of adding the now-requisite ’sprinkle of AJAX’ to a traditionally-structured web application then this is fine - ideal even - but if you need to push things even a little beyond this then you’ll start to find it a real pain:
  • When you’re maintaining a library of ‘real’ client-side javascript code alongside the magically-generated kind, it can lead to duplication in two different languages (RJS and Javascript), and to confusing spaghetti code. Some code will be found in standalone little snippets of RJS in your view folders and controllers, whereas other code that runs alongside it will be in proper javascript classes - often the two will get out of sync, and bugs will arise from updating the one without remembering about the other.
  • While it’s true that one can make one’s own RJS helpers, the little snippets of RJS can be rather brittle and don’t exactly lead effortlessly into good object-oriented design and code re-use practices for the client-side code.
  • RJS encourages requests to the server (what should I do next? give me more javascript to run!) when they’re not really needed, when more sophisticated client-side code could easily cache results or figure out what to do without additional data from the server. I suspect this may be part of the reason some AJAX-heavy Rails-based apps have gotten a bad rep for performance.
  • Whatever happened to the MVC design pattern so beloved to Rails on the client-side? It’s nice to have the client run its own controller code, and to have a little model/view/controller separation on the client side as well. It’s also nice to have structure in place for re-usable UI widgets - with their own controller code, views and data binding. RJS doesn’t really address this need or provide structure for it - although the helpers we have for some of the Scriptaculous controls are a step in the right direction, they too leave a lot of room for improvement.

In case this sounds overly critical, I have absolutely no beef with using RJS in applications which don’t require a significant client-side component, but which benefit from a little special AJAX sauce on some of their forms. Use the right tools for the right job and so on. But I would definitely warn against it to those setting out to write a moderately complex web application with significant client-side interactivity.

Next entry, find out which alternatives we chose, and some suggested best practises for structuring client-side code within a Rails application.