As a note, this isn’t about these specific flaws: It’s that these kinds of flaws are endemic in ruby and particularly rails.

Set Processing

devon@chronos ~/Projects/sets $ cat sets1.rb
#!/usr/bin/env ruby
require 'set'
require 'pp'
large = Set.new(Range.new(1,10000000))
small = Set.new(Range.new(1,10))
pp small.intersection(large)

real 0m35.421s
user 0m23.900s
sys 0m9.910s



devon@chronos ~/Projects/sets $ cat sets2.rb
#!/usr/bin/env ruby
require 'set'
require 'pp'
large = Set.new(Range.new(1,10000000))
small = Set.new(Range.new(1,10))
pp large.intersection(small)

real 0m16.917s
user 0m12.720s
sys 0m3.430s


So the order that I intersect the sets drastically impacts the speed of the algorithm (2x). Let’s take a look at python to see whether a language in the same class behaves similarly.


devon@chronos ~/Projects/sets $ cat sets1.py
#!/usr/bin/env python

real 0m2.583s
user 0m1.850s
sys 0m0.670s



devon@chronos ~/Projects/sets $ cat sets2.py
#!/usr/bin/env python

real 0m2.598s
user 0m1.760s
sys 0m0.710s


So no, python handles the choice of which set to intersect with the other gracefully. This is a good example of how the ruby set api is immature. I’m ignoring that it’s taking between 13.7 and 6.5 times as long to execute, because I think that it’s reasonable for people to do code in something that is easier for them to work in and then optimize. My problem here is that based on the direction you do the intersection, it can take 2x longer in ruby. This is shoddy engineering and is the kind of thing that gets solved in a language as it matures. The problem is that the language was immature when it exploded into widespread use, and no one seems to be doing anything to make it ready for prime time.

So what about 1.9? 1.9 is supposed to be where the energy is going to make the language faster and more mature.

devon@chronos ~/Projects/sets $ cat sets1.rb
#!/usr/bin/env ruby1.9
require 'set'
require 'pp'
large = Set.new(Range.new(1,10000000))
small = Set.new(Range.new(1,10))
pp small.intersection(large)

real 0m13.310s
user 0m11.240s
sys 0m0.680s



devon@chronos ~/Projects/sets $ cat sets2.rb
#!/usr/bin/env ruby1.9
require 'set'
require 'pp'
large = Set.new(Range.new(1,10000000))
small = Set.new(Range.new(1,10))
pp large.intersection(small)

real 0m9.181s
user 0m8.280s
sys 0m0.480s


While it’s faster (which wasn’t really my complaint anyway), it still is very different based on which direction you do your intersection from. I see evidence of speed work being done on ruby, but not on maturing the language so that it meets tried and true engineering principals like the rule of least surprise. (Yes, code is a user interface for coders, so the rule of least surprise applies) (1)

Nil as an object
How about this one that has actually done damage to our database:


devon@chronos ~/Projects/sets $ irb
irb(main):001:0> nil.id
(irb):1: warning: Object#id will be deprecated; use Object#object_id
=> 4


It certainly explains why we have a bizarrely large number of items in our database with foreign keys set to 4. This conflicts directly with the rails standard for naming a database primary key as ‘id’, which you assess on a model at obj.id. Yes, code should have lots of null checks, but real world code has flaws. Sure it throws a warning, but since it doesn’t break on nil when you call .id, you get data corruption. Gee, thanks, ruby.

Ruby in general
These are just a few examples, but ruby is rife with happy path coding. The language needed another 5 years of development before going into widespread use. Instead, I think it’s getting worse because a set of rails engineers who don’t know anything about languages are at the helm, adding lots of syntactic sugar but with a distinct lack of substance behind that candied shell.

_{1. Perhaps it’s not fair to compare ruby to the unix philosophy, because the ruby design philosophy violates the 17 rules in ways that are clearly intentional. Specifically, ruby and rails egregiously violate the rules of clarity and transparency, and rails in particular violates the rules of separation, composition and representation.}

Well, I’m picking my side, standing with free speech and joining Everyone Draw Mohammed Day. Pakistan, I’m calling your government out. Ban me.

One of the things that this game does is that it lets you create in game territory with real world boundaries. Since I’ve been playing it from the start, as soon as they offered the ability to claim territory, I jumped at the chance and started placing flags in lower Manhattan near my office, and some more at my apartment at the time in Brooklyn. Eventually I merged my two kingdoms, giving me a domain that starts in brooklyn, crosses the east river, covers nearly all of lower Manhattan as far north as 40th st.

Recently they just released a new version of the game that adds a ton of new features, and as a trailer for it, they released this video. Much to my surprise and pleasure, it shows the rampant growth of territory in the game with lower Manhattan the area in the video. The video centers on my territory not once, but four separate times. The coolest part for me is that in game, you can only see things that are very close to you, so this is the first time I have ever had a chance to actually see my whole kingdom all at once.

This is my parallel manhattan in the green:

This is a much broader view showing much more of the area: (I’m still the light green in lower manhattan, the top of the screen is south)

Now with the new release, the game lets you found cities, and you can only do so where there is a city in real life. If I can find 2 others who will join me, I fully intend to found Parallel Manhattan.

Using ssh as a proxy server (to avoid your company’s/country’s anal probeware)
SSH has a built in SOCKS proxy that you can use in any program that can run over a SOCKS proxy. This includes Firefox (really, all browsers), Thunderbird, Pidgin/Adium (as well as almost all other IM clients like Google Talk, AIM, etc) and a ton of other networking programs we all use. Using SSH with the -D flag lets you create a dynamic ssh tunnel that acts as a SOCKS proxy, and sends all your traffic from the endpoint of your ssh connection.

To run ssh as a proxy, you need to run it like this:
ssh -D 127.0.0.1:3128 someserver.com

This command says you want to run a dynamic port forward, and it should be available on your local computer on port 3128 (default for SOCKS). All traffic that runs through this will appear to be coming from someserver.com (where you need to have an ssh account). Further, all traffic that you send through this tunnel will enter the tunnel without ever leaving your box, travel over that tunnel using ssh’s crypto, then exit at it’s endpoint where it can connect to whatever you want it to connect to. Return traffic follows the same path but in reverse. This is a great trick not only for avoiding firewalls (so long as your firewall does allow port 22 traffic), but also for surfing the web on any network that you don’t trust. Instead you can browse the web/read your email from a server you control and trust by doing all your networking via an encrypted ssh proxy.

To configure Firefox to use this crypted tunnel, click on Edit->Preferences, and select Advanced. From there, under Connection, click on Settings. You will want to select Manual proxy configuraion. Leave everything blank except SOCKS Host: localhost and Port: 3128. Check SOCKS v5. If you want to be able to access any resources that are on your local network without using the proxy, enter the local addresses or networks into the No Proxy For: box.

Using a single connection (and only having to enter in the password once)
If you want to be able to make connections to a box, and are sick of typing in the password multiple times for multiple connections, you can use ControlMaster. This allows you to have multiple ssh terminal sessions shared over one connection. This means that you only establish one connection, and one consequence of that is you also only need to do auth once when you establish the connection. You can enable this by editing ~/.ssh/config and adding *:

Host *
ControlMaster auto
ControlPath ~/.ssh/master-%r@%h:%p

One nice side effect of this is that you can do an ls ~/.ssh and see exactly which boxes you are connected to presently.

Using multiple ssh keys transparently
When I started at my present job, everyone was using keychain to manage the large number of ssh keys they had to deal with from our various servers. You don’t need a key manager to transparently utilize multiple ssh keys. Once again pop open ~/.ssh/config, and now instead of workign with the Host * section, you can create sections for each class of server you access (or individual servers). If you have multiple ssh keys that need to be used on multiple servers, you can avoid using -i and instead add a section for a class of server, and declare the ssh key inside of that via IdentityFile. Example:

Host *.production.somedomain.com
IdentityFile ~/.ssh/id_rsa_production

Host *.staging.somedomain.com
IdentityFile ~/.ssh/id_rsa_staging

Now, whenever you ssh to any box in the staging subdomain, it will use the staging key, and when you ssh to any box in the production subdomain, it will use the production key.

Keepalive
Some firewalls regularly snip idle connections. If you regularly find ssh sessions that you have left idle for a while to be unresponsive when you come back to them you are suffering from this. Not only is it irritating to lose your connection, but you may even find the whole terminal unresponsive, forcing you to close the terminal as well. Well, ssh config has an answer for you for this as well.

Host *
ServerAliveInterval 15

ServerAliveInterval is normally off, but when you turn it on, ssh will start sending a packet every n seconds to try to keep the connection open. Even better, now that it’s testing, if the connection truly did go away without being closed, SSH will automatically close the session if 3 of these keep alive packets go unresponded to. (you can set that to a different number by setting ServerAliveCountMax). This means that even when connections get snipped, you won’t come back to find an unresponsive terminal. Either the connection will be alive, or the connection will have closed, none of these unresponsive zombie connections.

Defaulting your remote user
Few people are lucky enough to have the same username on all the servers they regularly visit. Just as you can set the identity file on a per domain basis, you can also set username:

Host somedomain.com
User foo

Host someotherdimain.com
User bar

Now when you ssh to somedomain.com, unless you specify the username at the prompt (baz@somedomain.com) it will use foo. This overrides the default of using your logged in username.

Server aliases
Do you have a lot of domains you ssh to that you wish you had shorter names for? Do you have some things that you ssh only to via IP address? Both of these can be solved with the HostName directive. For example, if your gateway accepts ssh traffic, and your internal domain doesn’t use dns, you can use ssh config to give in an alias for ssh.

Host router
HostName 192.168.1.1

Now it’s as simple as typing ssh router to get to the gateway. This trick works with ip addresses or dns names.

Executing remote commands with ssh
Sometimes you only need to get a small piece of information from a server, and don’t want to bother with a full ssh session. Sometimes you may want to have a process that goes out to a server and runs a command as a part of a bigger script. Either way, you can use ssh to run individual commands on other boxes, and get the results on stdout and stderr. Give this a try:

devon@chronos ~ $ ssh evilsoft.org "uptime"
14:00:23 up 7:38, 1 user, load average: 0.84, 0.34, 0.22


Others
SSH is a powerful toolkit, and I have only scratched the surface of it’s possibilities. Spend some time reading man ssh_config and you’ll be amazed at some of the incredibly cool stuff in there for you to find.
* note, you only need one Host * section in your ssh config. All the variables I am discussing can be consolidated into one Host * section.

1. Conservatives would be more comfortable with a browser delivered by a major corporation that has faced antitrust charges over that browser then liberals would.
2. Liberals anecdotally are more prone to counter-cultural choices, and thus would be more likely to seek out an alternative to the default browser.

The Google interviews made me realize one other thing. In testing for this kind of effect, you would need to eliminate people who didn’t know what a browser was. Clearly, a person who doesn’t even know what a browser is is highly unlikely to proactively switch from their system’s default browser. Further, if a person’s political views are correlated at all with their likelihood of understanding what a browser is, not eliminating people who don’t understand could hide real results.

The company I work for, Knewton Inc, is becoming known in certain circles of its clever usage for the vastly underused (IMHO) Amazon Mechanical Turk service. When I mentioned my contention to our guy who’s been pioneering our MTurk usage, Dahn Tamir, he suggested that we build an MTurk task and get some real data to find out whether my hypothesis had any basis in reality. The rest of this post represents my findings.

The Experiment
We posted the following poll to MTurk, allowing users the ability to do the task only once (making it a lot harder to stuff the ballot box, or, at the very least, require a hell of a lot more effort on the part of the users):

The options for political affiliation were:

• Republican
• Independant
• Democrat

The options for political philosophy were:

• Strongly Liberal
• Liberal
• Moderately Liberal
• Moderate
• Moderately Conservative
• Conservative
• Strongly Conservative
• Socialist
• Libertarian

For the web browser, we considered a person to have passed the test if he or she answered Internet Explorer or Firefox, but no others. We considered them to have failed if they answered Yahoo, Google or MSN. It was pointed out that AOL did have a browser that could be reasonably referred to as AOL, so we ignored this field. Finally, we recorded the workers’ User Agent, letting us see what browser/os they were using.

Raw Results
Observations:

• Clearly, MTurkers are going to be more likely to know what a browser is, and it shows in the data. Google found that 8% of people in Times Square knew what a browser was, but 86% of respondents here did.
• We appear to have more Democrats/Liberals than Republicans/Conservatives one would expect in the normal population.
• I only added Socialist on a lark, and I was pretty amazed to see that it still got 2.24% in a poll of Americans
• Firefox polled a lot higher here than one would expect in the wild. This should make it a lot easier to see any effects, along with the high pass rate of the test.

Party Affiliation
Total % Republican 244 23.71% Independent 397 38.58% Democrat 388 37.71%

Political Philosophy
Total % Strongly Liberal 92 8.94% Liberal 188 18.27% Moderately Liberal 169 16.42% Moderate 197 19.14% Moderately Conservative 157 15.26% Conservative 105 10.20% Strongly Conservative 34 3.30% Socialist 23 2.24% Libertarian 64 6.22%

Browsers
Total % Firefox 479 46.55% Internet Explorer 439 42.66% Safari 63 6.12% Chrome 41 3.98% Opera 7 0.68%

Browser Test Results
Total % Pass 886 86.10% Fail 143 13.90%

Test Results
Next, after pulling in the raw results, we compared browser, affiliation and philosophy to the test results.

Observations:

• Unsurprisingly, IE fared better amongst people who failed the test. Firefox did substantially better amongst people who passed the test. Safari also did better amongst people who passed the test, which was a little surprising, because we expected all default browsers to do better with people who failed. The sample, however, is small enough that the Safari factor could just be noise. It has been mentioned that this could have been caused by Safari for windows. That supposition can potentially be answered in the data.
• Democrats did worse on the test, while Republicans and Independents did basically the same. We see the likely reason why in the political philosophy section. The further left you go, the worse you tend to do on the browser test, until you hit strongly liberal. Strong Liberals reversed the trend entirely and tested better than any other group.

Test Results by Party Affiliation
Pass % Fail % Republican 87.70% 12.30% Independent 88.16% 11.84% Democrat 82.99% 17.01%

Test Results By Political Philosophy
Pass % Fail % Strongly Liberal 91.30% 8.70% Liberal 81.91% 18.09% Moderately Liberal 84.62% 15.38% Moderate 86.29% 13.71% Moderately Conservative 86.62% 13.38% Conservative 87.62% 12.38% Strongly Conservative 88.24% 11.76% Socialist 86.96% 13.04% Libertarian 89.06% 10.94%

Browser by Test Results
Pass % Fail % Internet Explorer 39.16% 64.34% Firefox 49.89% 25.87% Safari 6.21% 5.59% Chrome 4.06% 3.50% Opera 0.68% 0.70%

Browser choice by political opinion
Here we have the final analysis. Looking at how browser choice changes across the political spectrum.

Observations:

• My initial hypothesis appears to be supported by this data set. IE falls off sharply the further left you go on the spectrum from Moderate. Strong liberals use IE at half the rate of strong conservatives.
• Both Libertarians and Socialists appear to fit in on the left end of the spectrum at least where browser choice is concerned. Libertarians appear to loath IE and are nuts for Firefox. They have the highest Firefox rating of anyone with an astounding 70.18%
• Firefox’s fall as you go to the right is less pronounced as IE’s fall as you go to the left. This is probably caused by the generally higher likelyhood of picking Safari as you go leftward.
• Opera appears to have its ’stronghold’ with Socialists, who were as likely to pick it as Chrome or Safari.
• Libertarians also appear to have an exaggerated preference for Opera, Safari and Chrome. Basically, Libertarians really don’t like IE.

Browser by Party Affiliation
IE % Firefox % Safari % Chrome % Opera % Republican 45.33% 45.79% 5.14% 3.27% 0.47% Independent 36.00% 52.86% 6.57% 3.71% 0.86% Democrat 38.51% 49.38% 6.52% 4.97% 0.62%

Browser By Political Philosophy
IE % Firefox % Safari % Chrome % Opera % Strongly Liberal 25.00% 63.10% 8.33% 3.57% 0.00% Liberal 35.71% 50.65% 9.74% 2.60% 1.30% Moderately Liberal 41.96% 48.25% 4.20% 5.59% 0.00% Moderate 45.88% 44.12% 4.12% 5.88% 0.00% Moderately Conservative 47.06% 43.38% 7.35% 2.21% 0.00% Conservative 44.57% 46.74% 4.35% 3.26% 1.09% Strongly Conservative 50.00% 46.67% 0.00% 3.33% 0.00% Socalist 30.00% 55.00% 5.00% 5.00% 5.00% Libertarian 12.28% 70.18% 8.77% 5.26% 3.51%

Edit
Some have found the graph confusing, so here are two other versions. Both horizontal, showing the data in one by browser, and in the other by political philosophy. Click on them to enlarge

(enlarge)

(enlarge)

Conclusion
There appear to be a few strong correlations in data between a person’s political thinking and his or her browser choice. Still, this data is not representative of the population as a whole. It is, instead, a sample of people who are web savvy enough to use MTurk and who are motivated to take polls like this for small amounts of money. Also, the strong bias of the tech and open source communities toward libertarianism is probably causing the strong bias of libertarianism toward Firefox in this sample. Still, I thought these results were too interesting to keep to myself. Enjoy.

Notes
If you want to toy with the raw data and get access to the source I used for experimenting with the data, as well as my analysis spreadsheets, those are available here. We did this in 3 separate data gatherings, and those samples are still separated into 3 data files the raw data.

So here goes, my unvarnished opinion of the problems facing the rails community:

1. The community

The community is the single most ego centric community I have ever seen in a decade and a half hanging around the greater free/open source movement. Everyone remembers Zed Shaw ironically (but deservedly) ripping into the rails community for it’s overblown personalities. But numerous other examples abound. How can I possibly view the authors of a monitoring framework with the name God (and a tagline “like monit only awesome”) as anything other then a people with enormously expanded and wholly unjustified egos? How can I possibly take HAML seriously when the main dev’s bio starts off with “Hampton Catlin is one of the best people in the world. Flat out.” Seriously? One has to know that this guy is going to give any production problem you have very serious attention</wry>

2. Totally decentralized, egocentric documentation

No other way to say it, the rails community is obsessed with personal branding, not with being a helpful community. Is there any centralized site that has a good set of user commentable docs covering most extensions? Php has that, and it’s part of why it’s still a defensible choice for writing web apps despite its numerous other flaws. The end result is that when you google for just about anything having to do with rails code, the first five items are all personal blogs

With rails, the community doesn’t build any sort of centralized corpus of knowledge, what they all do is write a blog entry about such and such technique, or how they fixed a problem. The problem here is that nearly all of this knowledge is outdated, and impossible to tie to any one version of the application stack. So instead we get tons of blogs with disjointed suggestions that take you in the wrong direction whenever you google about an issue (sure, this solution worked in 2.1, but not in 2.0 or 2.3). Too bad it’s in a blog without with the applicable version number instead of in a comment at the end of the documentation like mysql or php.

3. Excessive reliance on ORM

I’m sure you have all heard of the N+1 problem. Sure, lots of devs will tell you that it’s simple to avoid, but based on all the teams I’ve evern worked on, all it takes is one lazy programmer and all of a sudden your templates are taking 7000+ queries to render. It’s no wonder the rails community seems to think sql is slow. Hint: the answer isn’t memcached, the reason your rails site is slow is because the sql it’s running is craptastic. 99.999% of rails sites could be vastly improved by just hiring one or two people who actually understand relational databases. The part that I really just don’t get is that sql is much easier then coding, and RDBMS is not rocket science, but it’s treated like this big hideous bear that your developers have to hide from.

4. Pure ruby is the answer to everything

The community is utterly obsessed with pure ruby, and looks down on impure extensions that are wrappers around solid C libraries. This results in a general race to the bottom for speed, stability and maturity. The unix world is packed with easy to integrate, solid libraries. Libxml-ruby is nearly two orders of magnitude faster then rexml, and has top notch compliance with the xml spec. Yet somehow rexml is the standard and libxml-ruby is the red headed step child.

5. Fragile ecosystem

Now, I’ll be the first to admit that the rails maintainers shout far and wide that rails isn’t enterprise ready, and it really truly is not. Over and over I have gotten bitten by the myriad of flaws in the rails ecosystem. Any one specific issue I’ve dealt with could have been fixed in version such and such of some gem (or it’s even worse with rails plugins.) This complaint is not about specific flaws so much as the whole ecosystem feels like it’s standing on a very fragile foundation. Some examples of the ways we have gotten bitten over the life of the multiple rails projects I have been involved with are:

• rexml violating the xml spec by totally ignoring the encoding of documents parsed by it.
• gem silently failing (not returning the proper unix return code) and thus allowing some of our instance launches to blow up without a reasonable way to catch it. (And as a note, when you have to automatically install large amounts of gems from rubyforge, it chokes a *lot*).
• Innumerable gems regularly break api compatibility even in minor releases (I’m looking at you Rack). When you add calls, you can’t just remove them between 0.9.2 and 0.9.3, it’s damn unprofessional.
• i18n has been a real problem right from the beginning.
• Passenger’s ApplicationPool choking and never coming back, forcing you to restart apache once a month
• Constant movement from one webserver to another. It’s damn frustrating to have to switch web servers every 3 months as the community hops from one the the next, as everything else gets filed in the community folder for utter crap. At that point it becomes damn hard to find support for problems. First it’s lighttpd, with mongrel, then thin, then nginx with thin and on and on. Somehow people need to support this mess. Apache works, is well supported, can be very lean, and basically every systems person on the planet knows how to work with it. Hint: as a programmer, it’s not all about you. Your applications need to be supportable; running away from standards for the sake of being different is dumb.

And just to reiterate, some of these could be actually fixed, or hell, they may have always had a *way* to work right, but the default was broken, and caused project problems that never should have existed if the rails community could ever get around to implementing a spec as it is written.

6. Generates bad, bad habits in devs
• Creates a view of the world that implicit is good. This goes all the way to the language itself and the silly fact that explicit returns use more processor power then implicit returns.
• Creates a view of the world that clever is good. Anyone who has maintained software for a few years should be able to tell you, clever is bad, very bad.
• My personal least favorite, the technological abortion known as Single Table Inheritance. Admittedly, even the community is now arguing for people to not use this. STI is a wretched way to bind a database with an object model, it encourages sparse tables as well as overloading of tables. Both of these things will bite you in the ass down the road.
7. Convention magic

Magic is the core of rails, and seems to be one of it’s biggest selling points. The problem is, there’s simply no place for magic in a world where people need to actually support your code. Explicit configuration and code is much more useful to operations folks who aren’t familiar with the codebase, framework, and language. Even if you use your developers to support your app, when they rely on a bunch of clever magic it can make for longer outages while people have to figure out what implicit thing they are failing to take into account.

8. Rails hype will kill ruby

Ruby could be a decent language. It has a lot of awesome things going for it. In a few years it really will be ready for real heavy lifting. At least that was ruby’s future. Rails has taken over the show. The Rails community is the ruby community, and it has not set ruby on a course to be a truly dependable language. Instead as rails project after rails project fails in all sorts of business environments, it is creating a permanent bias against the language in the management staff that oversees projects in american business. I’ve personally seen it in multiple companies, and expect to see it in most shops that are today fiddling with rails.

Here’s the original for comparison.

Darwin’s 200th

They just arrived fresh from the presses, and they look fantastic!. If you happen to want your own tribute to darwin, you can pick it up at cafe press.

You can see some more pics in my flickr photostream. The part I’m most proud of is turning the pipe into a bushing to create a pretty good bearing at the top.

Here’s some video of it in action: (and a second video here)

Materials:

• 2 pipe flanges (1/2″)
• 4 pipe elbows (1/2″)
• 3 pipe tees (1/2″)
• 2 pipe bushing2 (1/2″)
• 4 pipe nipples of various sizes (1/2″) from close to 6″
• 1 solarbotics gm3 motor
• 1 zip tie
• some shapelock (this stuff rocks)
• a couple laser cut parts (the big plastic wheel and the box that holds the watch)
• a power supply for the motor (I used what I had on hand, a uChobby breadboard power supply)

While I was out for the week in Rome over the new years, a New York Times article about the hackerspace I’m a member of was published. In the photo I’m the one on the right with the Tinct.

As a note, it was a Club Mate not a beer