Unix needs an undo command.

This morning, my roommate and I hauled out some of our “big iron” (a languishing Pentium 4 box) to use as a photo server. Because we had initially planned to use that box to host bitquabit.com and its sister sites—a plan since scrapped—it had a full clone of all the data on my Linode hub. Before my roommate got going, then, I thought I’d quickly clean the box and return it to a neutral state. First stop, hose the duplicates of the websites I host. Fire up SSH, sidle into /var/www, double-check with pwd I’m where I think I am, fire off an rm -rf *, and check that the directory’s clean. It was, so I decided to write a message to my roommate. Since I couldn’t remember what his Unix login was, and I knew he was logged on, I ran the w command.

benjamin pts/0    fcfwbeac.fogcree Fri18    2days  1.20s  0.08s citadel
benjamin pts/1    192-168-40-51.c3 21:26    1.00s  0.06s  0.01s w

I don’t remember even installing Citadel on this machine, I thought. And why am I the only one on the server? As very, very dark thoughts started to wander out of my amygdala and set up shop in my frontal lobes, I decided to check that w wasn’t malfunctioning.

benjamin@bitquabit ~> w
9:43:23 up 18 days,  4:29,  2 users,  load average: 0.00, 0.04, 0.01
USER     TTY      FROM              LOGIN@   IDLE   JCPU   PCPU WHAT
benjamin pts/0    fcfwbeac.fogcree Fri18    2days  1.20s  0.08s citadel
benjamin pts/1    192-168-40-51.c3 21:26    1.00s  0.21s  0.00s w
benjamin@bitquabit ~>

I wonder why my roommate’s not showing up, I wondered to myself. Maybe if I run it one more time…

And then it hit me. benjamin@bitquabit. benjamin@bitquabit. Oh frak, I hosed my production server!

But no, wait. I’d set up the spare box to be a bitquabit clone; it makes sense it’d think it was bitquabit. So I dropped out of ssh and read back .ssh/config:

benjamin@bitquabit ~> exit

Good bye
Connection to bitquabit.com closed.
Mungus:~ benjamin$ cat .ssh/config
Host vera
HostName 192.168.1.10

Host bqb
HostName bitquabit.com
Mungus:~ benjamin$

…frak. Sure enough, I hosed all the websites on my production server. Oh, what jolly day.

There were two good things: first, although backup on my server wasn’t automated, I had written a backup script, and it had everything except four images from a very recent article and my hit tracking system, Mint. Second, I got motivated to actually automate my backup system.

Now, every day, at three AM, the server makes a tarball of all relevant data and throws it into a special backup directory. A launchd-powered script on my Mac grabs the tarball daily and puts it in a place on my machine where Mozy can find it. The end result is a system I’m pretty happy with. I shouldn’t lose data that way again.

All that said, it seems to me like this situation shouldn’t even be possible nowadays. I understand that I should’ve been more careful; I won’t argue that. But…shouldn’t Unix have an undo command? I know for a fact that I’m hardly the only person to have hosed data by not paying enough attention while doing sysadmin tasks. Indeed, it’s regarded as a right-of-passage for system administrators, and focusing on the fact that this is the first time I’ve ever done something nearly so stupid on a production system makes me happy. But, still: on a Mac, or even on Windows, I have undo all over the place. I can’t think of any command on Unix that has undo. Isn’t it about time that started changing?

One of my friends adamantly refuses to carry a cell phone on him. Although I don’t have a lot of sympathy for that these days, I’ll be changing my opinion very quickly if blasting your friends starts to become common.

One of the Big New Features in WordPress 2.2 was a dynamic sidebar. The idea was that developers would write reusable Widgets that users could add to their sidebar through drag-and-drop—a huge improvement over the old method of modifying a bunch of PHP by hand. The good news is that building a sidebar from widgets works great. Unfortunately, the bundled widgets don’t. The archive widget has an invalid capitalization of its onchange event that keeps this site from validating, while the links widget serves up duplicate IDs if you have links in multiple categories. No software is perfect (sadly including my own), but there’s a difference between having a few random bugs, and failing profoundly at something integral to your product—in this case, generating valid XHTML. The second bug in particular is surprisingly severe, since having duplicate IDs breaks the DOM and violates not just XHTML, but also the generally more bug-tolerant HTML 4.

What I don’t understand in looking at these bugs is why they happened in the first place. HTML validation can be trivially automated on a build server; it’s equivalent to verifying that the code you check in compiles (something which indeed frequently is automated on build servers). Shipping validating XHTML is something that the developers should be able to get “for free.” Yet it’s apparently not part of their process. The bug for the onchange mistake, for example, didn’t get filed until July—several months after WordPress 2.2 shipped.

Even if HTML validation were part of their build process, I’m not sure it would have helped much: the duplicate ID bug was filed the instant WordPress 2.2 came out, but making a fix has been pushed to WordPress 2.4, due sometime in 2008. They had to do this because the underlying bug is actually in the core of WordPress, which means fixing it would be hard.

What good is knowing about bugs when you’re not willing to fix them?

Lotus Symphony is IBM’s rebranded version of OpenOffice, and ships with Lotus Notes. As of today, Symphony is free. Even if you have OpenOffice, Symphony may be worth checking out, as it sports what in my opinion is a superior interface.

I’ve talked before about the value of a good REPL (scroll down to “The REPL in .NET”). Unfortunately, the programming language I write the most code in, C, lacks one.

Or at least, it used to.

The aptly named C REPL provides a REPL for C. Their trick: compile a DLL for each line of code, then load it into a new process. Presto: instant, portable interactive C.

I’m generally quite paranoid when it comes to server security—doubly so because I’m no guru at it—so I tend to take a shotgun approach. The virtual server running bit qua bit has a restrictive firewall setup, has root disabled, only allows secure IMAP/POP/SMTP, disallows password login through SSH, and mails me daily security audits, among other things. I also monitor Debian’s security-announce list like a hawk. (If you’re the sysadmin for a Debian server and you’re not on that list, sign up. There’s no excuse for not keeping your box secure when you can get told about all the known vulnerabilities.)

One of the things that’s crossed my mind in the past month as I’ve been whipping bit qua bit into shape is that the login pages for WordPress and a handful of other utilities I keep on the server are served over HTTP, when they ought to be served over HTTPS. A couple weeks ago, I made available a secure version of the login page for those who wanted it. (Because I don’t want to bother paying a CA, I left the default insecure, since you’ll get a Scary Dialog Box telling you to panic and fear because the world is ending and you are going to die and incidentally the cert’s self-signed, but nobody reads that far and fewer people understand what it means, so they just leave the site instead of posting.) A week ago, I moved all the back-end pieces so that they were available HTTPS only, and a few days ago, I decided to move the WordPress admin interface to HTTPS as well.

I naïvely thought the process would be simple. I fired up Emacs, opened the configuration file for bitquabit.com’s insecure site, and added the following line:

Redirect /wp/wp-admin/ https://bitquabit.com/wp/wp-admin/

I restarted Apache, checked things out, and bingo! Everything seemed to be working spiffy.

Except…I gradually began to notice that something wasn’t quite right. WordPress no longer automatically saved my drafts, and I couldn’t modify the websites appearing in my blogroll. When I found I also couldn’t upload pictures for an upcoming article, I had to dive underneath the covers and figure out what was going on.

It turns out that a lot of URLs in WordPress—among them, all the AJAX ones—are hard-coded to be HTTP, not HTTPS. When they were trying to execute, they’d get back a 302 response code (redirect), and then fail. Not good.

There are two ways to secure WordPress: the first is to simply modify the blog’s URL to be https://—not really what I was aiming for. The second is to use the Admin-SSL plugin so that only the login and admin pages are secured. That seems to work perfectly, but it forces all users to login through the secure interface, which means that everyone gets the Scary Dialog Box when they try to post. I’m not sure what solution I’m going to end up adopting; part of me still just wants to rewrite the whole blog myself, in which case I can easily customize it to do what I want, but it’d probably be easier to learn just enough PHP to fork the Admin-SSL plugin. For the moment, I’ll just keep using an SSH tunnel to the box when I want to post, which is probably more secure anyhow.

As approximately 30% of my readers undoubtedly noticed, Internet Explorer had serious issues displaying the redesigned bit qua bit properly. Unfortunately, since IE lacks developer tools like the excellent Firebug and Web Developer toolbar, trying to figure out exactly what IE was choking on threatened to become a game of guess-and-test—the main reason that I’ve been so slow to get it fixed.

Today, by happenstance, I came across a bookmarklet called Xray. You simply throw Xray into your toolbar, click it on any web page, and an easy-to-use CSS inspector pops up. Click on any element of the page and its CSS properties will be displayed; navigating through the DOM involves simply using your cursor keys. I fired up Xray in IE, and in under a minute, found the causes of the layout glitches and added two lines of CSS to fix them. The result: bit qua bit now renders properly in all major browsers. If you have to debug CSS problems in IE, I’d definitely encourage you to give Xray a shot.

After reading about Michael’s attempt to turn his homepage into an aggregator for all his computer activities, I got inspired to try LiveJournal again. I last used LiveJournal when it was cool because it was running on Linux, and Linux was really cool because it had the singular ability to wipe out huge chunks of nominally backed-up data if you didn’t understand how UMSDOS worked. Since (as you “old-timers” have already figured out) I was thirteen at the time, I had of course forgotten my password, so I decided to make a new account. After a few moments reflection, I opted to name my new account “bitquabit.” I filled out the new user registration form, clicked a button, and started using my new account got a violent, red error:

Username already in use; please select a different one.

Now, I know that the name “bit qua bit” was new at the time I created this site, because a) I googled it thoroughly on all the major search engines, and b) most people I know who speak Latin can’t agree on whether “bit qua bit” is even grammatically legal, let alone what it means, so chances were low that someone just happened to come up with the name independently. Furious to find out what nefarious scoundrel would dare abscond with my originality, I rushed over to bitquabit’s LiveJournal and was greeted with…

…the headlines for my own blog, linked to my own blog.

It turns out that LiveJournal has a feature called syndicated feeds, whose sole purpose in life, as near as I can tell, is to let you friend arbitrary RSS and Atom feeds that aren’t on LiveJournal. I understand the underlying motivation, but I’m perplexed by their implementation. It wouldn’t be hard for LiveJournal to implement the same functionality without pseudousers; as-is, they’re conflating two radically different classes of accounts behind the same user interface. Even if, for some technical reason, they do need pseudousers, I don’t get what the point is of hosting an HTML version of the RSS feed on LiveJournal. There already is an HTML version of the RSS feed. It’s called bitquabit.com.

I’m not complaining, mind you; the way they’ve implemented the feature—just the headlines, with links to the full content—all it can do is boost my pagerank, and since I only wanted to log into LiveJournal to see what it’s like these days, I don’t really care that I had to go back and come up with a random user name. I just find the whole concept truly bizarre.

As I stand on the dawn of a new year, I find myself drenched in the past.

This is the first Rosh HaShanah in five years I won’t be spending at Duke, and the first in two years I won’t be spending with someone I love. Perhaps for that reason, I am completely unable to get the smells, the textures, and the sounds of college out of my head. The smells, in particular, carry with them an unreasonable power to move me through time. I don’t merely remember, I feel I can actually smell the luscious scent of the Duke gardens, the coldness of the sanctuary’s tile floor, the succulent aroma of Henry’s cooking, the seductive hint of a young lady’s perfume. With those smells come back vibrant images and emotions that have twice moved me to tears. My psyche for the moment is unwillingly, relentlessly dragged backwards, out of the present and into memories that have until now lain dormant.

Rosh HaShanah and the Days of Awe have always been a time of intense introspection for me. Whether it’s merely the deluge of religious practices, or whether it’s something more primal in my heart, I find myself always asking, What am I doing? Why am I doing these things? What lies ahead of me, and how much of that will be left to my control? As part of answering these questions, I must look backwards. But looking backwards, and living in the past, are different, and I have slid far too much into the latter.

Therefore, this year, during the Days of Awe, I am going to turn myself forward. I am blessed with a loving family whom I can see easily and frequently. I have a very close batch of kind and intelligent friends who are extremely supportive. I have a stimulating and fun job working on things I love. These are things that I want to carry with me into the future. To the extent that I look backward, I will be doing so with the intention of learning how to maintain and grow these blessings.

I barely ever visit blogs anymore. Instead, I tend to just read everything in the comfort of my news reader (Google Reader, if it matters). Nevertheless, I still think there’s a place for having a good website design, and bit qua bit’s stank like a disemboweled skunk in an outhouse. So, rather than sleep tonight, I’ve been working like mad trying bang out the last of my redesign before the holiday, and I’m happy to say that I’ve basically succeeded. There are a few things I don’t like, a lot that needs a bit of polish, and one or two bugs left, but, overall, I’m extremely pleased with the result.

Comments on what you like and what you don’t more than welcome; I’m quite willing to admit that web design isn’t exactly my strong suit.

Edit: Any IE experts want to help me figure out what magical incantation I need to say to make the design look the same in Explorer as it does in Opera, Firefox, Safari, and basically every other browser made except IE?

I was happy to read today that the VisualStudio team is working on significant performance improvements for VS.NET 2005. One of the things that frustrates me most about working on Fog Creek Copilot is simply that VS.NET can sometimes be so slow that I actually lose my train of thought. Hitting a breakpoint in some of our programs can lock VisualStudio for five to ten seconds. Building the small copilot.com website can take a full minute if the master Aardvark.dll assembly has been modified. It’s like trying to maintain flow when your work environment is a series of dams. This patch can’t come out a minute too soon.

This morning, I’m at the big NYC FogBugz demo, the only part of the FogBugz World Tour I’m actually attending. Hopefully, I’ll see some of you there.

It’s so nice when bad things happen to bad people.

Now if they could just do something about the other idiot in this affair…

A few days ago, I watched Who Killed the Electric Car?, a documentary covering the growth and decline of electric cars in the 90s. The movie focuses on the GM EV1 as its poster child, interviewing several EV1 drivers, sales personnel, and parts manufacturers. Because I had only a dim memory of the EV1, or even of the concept of electric cars being on the road, I found a lot of the documentary fascinating.

To be sure, the documentary has a clear message: the electric car was killed because it was too reliable (hurting dealer’s repair centers) and too damaging to the oil industry, and thus lost the support of an industry-kowtowing government. Yes, the “murder trial” at the end of the movie actually “convicts” consumers, big oil, car companies, government, the California Air Resources Board (which for some oddball reason isn’t part of the government, I suppose), and hydrogen fuel cells, but quite frankly, I don’t think there’s enough evidence in the movie to support that. Consumers in the movie practically want to have conjugal relations with their EV1s, and hydrogen fuel cells don’t even make an appearance until literally three minutes before they’re convicted. CARB and GM are presented throughout the entire movie as alternatively stupid and evil organizations. Despite the closing segment, the viewer knows who’s truly guilty. Not consumers; not the technology; just the car companies and big oil.

I don’t honestly think that’s fair. There were significant problems with electric cars as they existed at the time of an EV1. The biggest problem, and the reason that I can’t see myself buying an EV1 if they still existed, is that the 80-mile range really isn’t sufficient for a lot of Americans. The movie (and electric car proponents in general) like to point out that 80 miles is well above what the average American drives in a day. That’s true. What it’s missing, though, is the distribution curve. When I used to have a car, I drove on average about 20 miles per day during the week, but on weekends, when I would be out with friends or running errands, I could easily put on three times that number. A round-trip to downtown Indianapolis, excluding any other errands I might need to run, was 40 miles all by itself. Add in summer heat, idling in traffic, and a swing by a friend’s house, and you’re looking at trouble. Similarly, when my sister went to college, her average daily driving during the school week probably came out to only a few miles per day, but on weekends, when she came home, she could easily slap 160 miles on the odometer. Many of my friends over the years have had similar driving patterns. It’s the classic focus on mean without remembering to look at the standard deviation.

Of course, there’s no reason that anyone should have to drive such large amounts. The problem here is one of infrastructure. In Europe, certainly, you don’t have to drive nearly as much, nor do you in vast tracts of the Northeast. Instead, the trains take over, providing convenient mass transit for the medium haul. If the country had a decent rail system, I think that a purely electric car would have a much better shot. Since that’s realistically not going to happen in the near future, any successor to the gas-powered automobile is going to have to handle both short and medium distances.

Thankfully, there are green alternatives in the pipe that can do exactly that. The one I’m most enthusiastic about is the Chevy Volt, a re-envisioned hybrid. The Volt, currently just a prototype, has the ability to drive 40 miles off an electric charge, while gracefully falling over to gasoline, biodiesel, or (blech) E85.

Digression: E85 is a completely idiotic concept in the US until we eliminate the sugar tariff. I’m sick and tired of propping up corn farmers making horribly inefficient, environmentally damaging, and unhealthy substitutes for sugar products just because America’s addicted to growing corn. (End digression)

The result is a zero-emissions electric vehicle for daily chores that transparently becomes a traditional automobile for longer trips. I’m extremely enthusiastic about the future of the Chevy Volt concept, and very hopeful that it will enter production in some form very soon. This is what the EV1 should have been to begin with.

I should also point out that pure electric car technology itself has improved tremendously in the last ten years. The Tesla Roadster, for example, is a brand-new electric sports car that has outstanding performance characteristics and manages more than 200 miles on a charge—a distance that I’d argue is nearly sufficient for everyone. It’s only just starting to roll off the assembly line, so we’ll have to wait and see how efficient it is, but I’m hopeful it will do well enough in its first few markets to expand.

Overall, I have to recommend Who Killed the Electric Car?, if only because it does bring a lot of facts to light about electric cars that I doubt most people know. I wish it were slightly more coherent in its ending, but even with it, I think it’s a good watch.

Over the last two weeks, I’ve altered my schedule in a very simple way: on most days, I get up earlier and go to bed earlier.

At one point, I was a morning person. Sleeping ‘til 8 or later was a rare treat; most days, I got up at 6:05 AM sharp. For the first two years I was in college, I made the radical change of getting up at 7 instead of 6, but otherwise kept the same schedule. In my last two years of college, though, I fell apart. Most college students sleep from about 2 AM—10 AM, and if you want to socialize—or even worse, date—that’s got to be your schedule, too, so slid everything in my day back by three three hours. Because Fog Creek has flexible hours and I had a college-attending girlfriend through July, I kept my 2—10 schedule when I began work as well.

For the last couple of weeks, though, I changed that, sleeping most nights from 11 to 7. On the surface, there should be no difference. I’m awake the same total number of hours. New York stores are open plenty late enough that I can get done everything I want to get done regardless of my hours. Tyler runs on the same schedule I ran on, so there’s no reason my work should’ve been affected. Empirically, there should be no difference.

But…there is. I’m better about praying and meditating regularly. I get more exercise. For heaven knows what reason, I eat more healthily, get more done at work, and am generally in a better mood. I used to think that the old adage, “Early to bed and early to rise/Makes a man healthy, wealthy and wise” was nothing more than a saying, but I’m beginning to think that, at least my body, the statement holds more than a bit of truth.

For the last several months, I’ve been powering bit qua bit’s mail system with Citadel. In the yonder years, Citadel was a very powerful BBS for Unix systems. As the bulletin board days drew to a close, and its developers began searching for a way to keep Citadel relevant, they hit upon the idea of turning it into a groupware system. The current version of Citadel runs on most Unix platforms, supports secure IMAP, SMTP, and POP3 out of the box, provides the GroupDAV protocol for synchronizing with calendars and address books, and comes with a robust web client that lets you access everything when you’re away from your laptop.

Citadel’s been simply wonderful. Installing it under Debian Etch was trivial, involving little more than apt-geting from their repository and answering a few very short questions. Integration with SpamAssassin involved little more than a few clicks, as do most administration tasks. Setting up restorable backups took all of five minutes, and making Citadel receive mail from multiple domains only took a bit longer. And, when I get really nostalgic, I can still connect to the system as if it were a text BBS to chat with users, perform administration tasks, or (not recommended) read my mail. (Use mutt if you want to go that route, as the Citadel text mail interface is gleefully unchanged from 1995 or so.)

Citadel’s hardly perfect. It gets a bit confused if your mail client tries to grab several IMAP folders at once; its web interface, though rich and functional, feels primitive; its sendmail replacement is still under-featured; and setting up aliases is more confusing than it has to be. Still, I think it’s worth a look.

I was pleasantly surprised today to discover Squeak by Example, an open-source book on writing programs with Squeak Smalltalk. If you want the bleeding-edge version of the book right now, you’ll need Subversion and an up-to-date LaTeX installation, but a four-month-old PDF version is also available if you don’t want to muck with all that. Combined with Stéphane Ducasse’s compilation of free Smalltalk books, I don’t think any Smalltalk neophyte should be wanting for learning material.

My day job is working on Fog Creek Copilot, a powerful, cross-platform remote assistance solution. This week, Tyler and I were talking about how it’s too bad that Copilot didn’t really exist when we were in college, because we always ended up doing tech support for our families over the phone, which always went something like:

Me: What do you see now?

Family Member: A dialog box.

Me: What’s it say?

Family Member: It’s got a stop sign with an exclamation mark and says that the server can’t be found.

Me: Okay, click “Okay,” then read me back the line that says “SMTP Server.”

Family Member: Wait, I just clicked “Okay” twice. Now what do you want me to do again?

And so on. Not fun.

The good news is that Copilot exists now, and makes doing remote tech support really easy. Unfortunately, college students are basically perpetually broke. Tyler and I remember what that’s like. It sucks. You just got to school. There are eighty bajillion things going on and at least ten girls or guys that have caught your eye and (if you’re very lucky) your pants. Having to pick between wasting an hour helping someone, or using valuable booze money just so you only need five minutes, can be a painful choice.

Well, we’ve got a proposal: for the month of September, we’ll let anyone with a .edu address use Copilot for free, up to three times. These aren’t two-minute trials; they’re real, legit, 24-hour day passes. Plus, if that’s not enough, we’ve got a referral program: if you refer a friend with a .edu address to use Copilot, you get another three day passes. Ad nauseam. No limit. No catch.

So if you’re in college, the next time someone asks you for help, grab a free Fog Creek Copilot day pass. Then spend your leftover time grabbing a beer. Best of both worlds.