Evilsoft.org

I’ve always been a fan of RPGs, so of course as soon as I got my G1 last year, I started scouring the market for anyone making a good RPG or MMORPG on the phone. Just a bit over a month after the phone came out, Parallel Kingdom was released. The game managed not just to fill the need of an RPG, but it also managed to hit on one of my other interests: Consensual Reality. See, the game places you in another reality smack dab on top of the one we are in. It uses the GPS to position you in the game world based on where you are in the real one. Personally I expect this to be the first of many realities we layer across the face of the planet, and so I follow the improvements in this game with much excitement, because in many ways they are the first ones to attack solving some of these layered reality problems.

One of the things that this game does is that it lets you create in game territory with real world boundaries. Since I’ve been playing it from the start, as soon as they offered the ability to claim territory, I jumped at the chance and started placing flags in lower Manhattan near my office, and some more at my apartment at the time in Brooklyn. Eventually I merged my two kingdoms, giving me a domain that starts in brooklyn, crosses the east river, covers nearly all of lower Manhattan as far north as 40th st.

Recently they just released a new version of the game that adds a ton of new features, and as a trailer for it, they released this video. Much to my surprise and pleasure, it shows the rampant growth of territory in the game with lower Manhattan the area in the video. The video centers on my territory not once, but four separate times. The coolest part for me is that in game, you can only see things that are very close to you, so this is the first time I have ever had a chance to actually see my whole kingdom all at once.

This is my parallel manhattan in the green:

This is a much broader view showing much more of the area: (I’m still the light green in lower manhattan, the top of the screen is south)

Now with the new release, the game lets you found cities, and you can only do so where there is a city in real life. If I can find 2 others who will join me, I fully intend to found Parallel Manhattan.

So I use ssh a lot. And this is one damn powerful little program. I decided to write up a log of some of my favorite ssh tricks.

(more…)

A few weeks ago, a video was passed around our office that was about Google interviewing people in Times Square about what a browser is. The results were fairly appalling. For the most part, people responded with various portal and search web sites instead of something like “IE” or “Firefox”. This coincided with a conversation I was having with someone in which I posited the hypothesis that IE as a browser would bias to the political right, and other browsers (specifically firefox) would bias to the political left. My reasoning was:

1. Conservatives would be more comfortable with a browser delivered by a major corporation that has faced antitrust charges over that browser then liberals would.
2. Liberals anecdotally are more prone to counter-cultural choices, and thus would be more likely to seek out an alternative to the default browser.

The Google interviews made me realize one other thing. In testing for this kind of effect, you would need to eliminate people who didn’t know what a browser was. Clearly, a person who doesn’t even know what a browser is is highly unlikely to proactively switch from their system’s default browser. Further, if a person’s political views are correlated at all with their likelihood of understanding what a browser is, not eliminating people who don’t understand could hide real results.

The company I work for, Knewton Inc, is becoming known in certain circles of its clever usage for the vastly underused (IMHO) Amazon Mechanical Turk service. When I mentioned my contention to our guy who’s been pioneering our MTurk usage, Dahn Tamir, he suggested that we build an MTurk task and get some real data to find out whether my hypothesis had any basis in reality. The rest of this post represents my findings.

(more…)

I got involved with a back and forth on twitter today about rails, and ultimately I realized that there was no way to give my full opinion of the problems facing the rails community in 140 byte snippets. I’m not saying my opinion is the end all be all, but I have lead probably half a dozen rails projects and I did speak at RailsConf 2007.

So here goes, my unvarnished opinion of the problems facing the rails community:

1. The community

The community is the single most ego centric community I have ever seen in a decade and a half hanging around the greater free/open source movement. Everyone remembers Zed Shaw ironically (but deservedly) ripping into the rails community for it’s overblown personalities. But numerous other examples abound. How can I possibly view the authors of a monitoring framework with the name God (and a tagline “like monit only awesome”) as anything other then a people with enormously expanded and wholly unjustified egos? How can I possibly take HAML seriously when the main dev’s bio starts off with “Hampton Catlin is one of the best people in the world. Flat out.” Seriously? One has to know that this guy is going to give any production problem you have very serious attention</wry>

2. Totally decentralized, egocentric documentation

No other way to say it, the rails community is obsessed with personal branding, not with being a helpful community. Is there any centralized site that has a good set of user commentable docs covering most extensions? Php has that, and it’s part of why it’s still a defensible choice for writing web apps despite its numerous other flaws. The end result is that when you google for just about anything having to do with rails code, the first five items are all personal blogs

With rails, the community doesn’t build any sort of centralized corpus of knowledge, what they all do is write a blog entry about such and such technique, or how they fixed a problem. The problem here is that nearly all of this knowledge is outdated, and impossible to tie to any one version of the application stack. So instead we get tons of blogs with disjointed suggestions that take you in the wrong direction whenever you google about an issue (sure, this solution worked in 2.1, but not in 2.0 or 2.3). Too bad it’s in a blog without with the applicable version number instead of in a comment at the end of the documentation like mysql or php.

3. Excessive reliance on ORM

I’m sure you have all heard of the N+1 problem. Sure, lots of devs will tell you that it’s simple to avoid, but based on all the teams I’ve evern worked on, all it takes is one lazy programmer and all of a sudden your templates are taking 7000+ queries to render. It’s no wonder the rails community seems to think sql is slow. Hint: the answer isn’t memcached, the reason your rails site is slow is because the sql it’s running is craptastic. 99.999% of rails sites could be vastly improved by just hiring one or two people who actually understand relational databases. The part that I really just don’t get is that sql is much easier then coding, and RDBMS is not rocket science, but it’s treated like this big hideous bear that your developers have to hide from.

4. Pure ruby is the answer to everything

The community is utterly obsessed with pure ruby, and looks down on impure extensions that are wrappers around solid C libraries. This results in a general race to the bottom for speed, stability and maturity. The unix world is packed with easy to integrate, solid libraries. Libxml-ruby is nearly two orders of magnitude faster then rexml, and has top notch compliance with the xml spec. Yet somehow rexml is the standard and libxml-ruby is the red headed step child.

5. Fragile ecosystem

Now, I’ll be the first to admit that the rails maintainers shout far and wide that rails isn’t enterprise ready, and it really truly is not. Over and over I have gotten bitten by the myriad of flaws in the rails ecosystem. Any one specific issue I’ve dealt with could have been fixed in version such and such of some gem (or it’s even worse with rails plugins.) This complaint is not about specific flaws so much as the whole ecosystem feels like it’s standing on a very fragile foundation. Some examples of the ways we have gotten bitten over the life of the multiple rails projects I have been involved with are:

• rexml violating the xml spec by totally ignoring the encoding of documents parsed by it.
• gem silently failing (not returning the proper unix return code) and thus allowing some of our instance launches to blow up without a reasonable way to catch it. (And as a note, when you have to automatically install large amounts of gems from rubyforge, it chokes a *lot*).
• Innumerable gems regularly break api compatibility even in minor releases (I’m looking at you Rack). When you add calls, you can’t just remove them between 0.9.2 and 0.9.3, it’s damn unprofessional.
• i18n has been a real problem right from the beginning.
• Passenger’s ApplicationPool choking and never coming back, forcing you to restart apache once a month
• Constant movement from one webserver to another. It’s damn frustrating to have to switch web servers every 3 months as the community hops from one the the next, as everything else gets filed in the community folder for utter crap. At that point it becomes damn hard to find support for problems. First it’s lighttpd, with mongrel, then thin, then nginx with thin and on and on. Somehow people need to support this mess. Apache works, is well supported, can be very lean, and basically every systems person on the planet knows how to work with it. Hint: as a programmer, it’s not all about you. Your applications need to be supportable; running away from standards for the sake of being different is dumb.

And just to reiterate, some of these could be actually fixed, or hell, they may have always had a *way* to work right, but the default was broken, and caused project problems that never should have existed if the rails community could ever get around to implementing a spec as it is written.

6. Generates bad, bad habits in devs
• Creates a view of the world that implicit is good. This goes all the way to the language itself and the silly fact that explicit returns use more processor power then implicit returns.
• Creates a view of the world that clever is good. Anyone who has maintained software for a few years should be able to tell you, clever is bad, very bad.
• My personal least favorite, the technological abortion known as Single Table Inheritance. Admittedly, even the community is now arguing for people to not use this. STI is a wretched way to bind a database with an object model, it encourages sparse tables as well as overloading of tables. Both of these things will bite you in the ass down the road.
7. Convention magic

Magic is the core of rails, and seems to be one of it’s biggest selling points. The problem is, there’s simply no place for magic in a world where people need to actually support your code. Explicit configuration and code is much more useful to operations folks who aren’t familiar with the codebase, framework, and language. Even if you use your developers to support your app, when they rely on a bunch of clever magic it can make for longer outages while people have to figure out what implicit thing they are failing to take into account.

8. Rails hype will kill ruby

Ruby could be a decent language. It has a lot of awesome things going for it. In a few years it really will be ready for real heavy lifting. At least that was ruby’s future. Rails has taken over the show. The Rails community is the ruby community, and it has not set ruby on a course to be a truly dependable language. Instead as rails project after rails project fails in all sorts of business environments, it is creating a permanent bias against the language in the management staff that oversees projects in american business. I’ve personally seen it in multiple companies, and expect to see it in most shops that are today fiddling with rails.

It’s becoming a Friday tradition. Judy He, the front end engineer at my office appears to have started a weekly tradition of photoshopping Knewton employees into movie posters. This week I got put in in place of Anthony Hopkins in the poster for Legends of the Fall (which I might add was a truly wretched film. In Brad Pitt’s place is Sven Northcott and for Adian Quinn it’s Dave Cascino.

Here’s the original for comparison.

My friend John as reading an article on Ars Technica about Darwin’s birthday that had a pic that was just screaming to be made into a magnet. So I present to you my second work of magnetic commentary:

Darwin’s 200th

They just arrived fresh from the presses, and they look fantastic!. If you happen to want your own tribute to darwin, you can pick it up at cafe press.

My parents recently visited my aunt and uncle in china, and while there they picked up some rather nice high end watches direct from the factories. Most of the time, my father prefers his digital, and only wants to wear the watch he picked up in china when he’s in more formal dress. Thing is, the watch he bought is autowinding, so it’s always either dead, or at the very least really wrong every time he puts it on. So he asked me to come up with a crazy contraption to wind his watch. This is what I came up with:

You can see some more pics in my flickr photostream. The part I’m most proud of is turning the pipe into a bushing to create a pretty good bearing at the top.

Here’s some video of it in action: (and a second video here)

Materials:

• 2 pipe flanges (1/2″)
• 4 pipe elbows (1/2″)
• 3 pipe tees (1/2″)
• 2 pipe bushing2 (1/2″)
• 4 pipe nipples of various sizes (1/2″) from close to 6″
• 1 solarbotics gm3 motor
• 1 zip tie
• some shapelock (this stuff rocks)
• a couple laser cut parts (the big plastic wheel and the box that holds the watch)
• a power supply for the motor (I used what I had on hand, a uChobby breadboard power supply)

While I was out for the week in Rome over the new years, a New York Times article about the hackerspace I’m a member of was published. In the photo I’m the one on the right with the Tinct.

As a note, it was a Club Mate not a beer

Last friday I was approached by Wazi to write an article about the release of Python 3. Go take a gander: Python takes a great step forward: Python 3

Some time ago I built an ambient orb to watch over the build process for the code at my office. Recently I used upgrading it to RGB leds as an excuse to get my first board cut by Advanced Circuits in Colorado (Why buy from China when you can buy from the U.S. at a lower cost?), and it’s turned out pretty well.

My first board, ain’t it pretty?

The new orb is substantially brighter, with colors that are visible (and even border on vivid) even in direct sunlight. I’m really quite happy with the results.

Here’s the board diagram and schematic:

You can grab the source and eagle files (All under the GPLv2 & CC Attribution/Share Alike licenses respectively). It’s got 3 sample apps now. The build warden for cruise control, a biff for email, and for tonight it’s got a electoral college meter that goes blue or red based on live results.

(Warning, everything is only tested under linux. The applications are written in python and ruby, and should work under windows and mac os, but I haven’t tested them)