close this bookVolume 7: No. 83
View the documentJournal calls
View the documentCareer jobs (in our CCJ 7.42 digest this week)
View the documentDatabase-backed Web sites

Philip Greenspun has a nonlinear structure. I have a linear structure, the Computists' Communique. Whenever I come across anything worth remembering and sharing, I put it in the Communique. Philip Greenspun does the same with his website, but does it in hypertext. Whenever he has a fact or thought to share, he finds a place for it in his website.

is also a "node," or coordination point for other people's expertise. Rather than seek out specialized knowledge that might be of interest, he's found ways to draw contributions from other people. That has pluses and minuses. Philip has to write less, and no longer has to guess what might be important to his readers, but now must serve as his website's gardener. Someone has to prune pointless comments from the website, and he's stuck with the job. (I almost said "janitor," but janitors only fight decay. Gardeners channel growth.)

Someone also has to answer email queries and direct readers to the FAQ pages that answer their questions. Philip is handy with web tools, so he's automated most of that. Some of the web tools are spiffy, so now people ask about them as well. Instead of saving time, his automation has increased the range of material that he's responsible for.

His response was to write web pages and eventually a book about how to build interactive websites like his. "Database Backed Web Sites: The Thinking Person's Guide to Web Publishing" (Macmillan/Ziff-Davis Press, now Que). You can read an edifying history of the publication process on .

Philip asked if I'd like a copy, and I took him up on it. I thought he wanted me to review it so that he could sell more books. Now that I've read it, I sense an additional motive. He wants to change the world. All that he needs to do is get everyone to see reason.

It all started with some travel pictures. Philip had access to big hard disks at MIT, plus some networked computers and sysop friends. So he put up one of the first really good graphic websites, Travels with Samantha . (Samantha was his PowerBook 170, since stolen.) The pictures are high-quality scans direct from slides onto PhotoCD. People liked the travelogue, and used it to show their friends what the Internet could do. So Philip added more pictures (about Berlin, New Zealand, the Cayman Islands, and Costa Rica), some related travel books for which he licensed the online rights, and a lot of info about quality photography. He also added a bitterly humorous career guide for computer scientists, , with a clock to compute Bill Gates' current wealth, .

Website traffic grew, enormously, as the Web grew. (Like, 500K hits per day. Samantha now gets about 100K hits from 1K readers per day.) Fortunately, Greenspun had used a high-speed database called AOLserver (formerly NaviServer) to implement the interactive pages on his website. (CGI is too slow because it forks a process for each page hit. Large databases such as Oracle and Sybase are too slow unless you've got mainframes to run them.) That's one of the lessons of the book: stick with AOLserver and other fast/simple systems if you possibly can.

Philip uses his Website/book to air several of his pet peeves. (This was partly to please his publisher, who needed page count in order to sell the book. A mystery, since they did very little to get it displayed in stores.) Philip believes in Common Lisp and Emacs; can do what he needs to in Perl and Tcl; and hates C with bitter passion. ("There is no faith strong enough that it cannot be shaken by Unix or any Microsoft product. The devil is real. He lives inside C programs.") Java is an improvement over C, but is hosted on unreliable machines -- not like Lisp machines. Philip repeatedly implies that all of the good ideas date back to MIT Lisp work in the 1960s, with everyone since then just trying to fight the bugs and incompatibilities introduced by commerce-driven C implementations. He doesn't give Lisp examples in his book, though. He talks about the poor numeric capabilities of Tcl and how you can get around them, or the problems of HTML and how to get around them. Then he points readers to sources of Oraperl, Meta-HTML, and AOLserver. If they won't do your job, chances are you can find a guru to extend them.

Philip says he can make Unix work if he finds the right guru -- of which there are plenty at MIT -- even though "buying a Unix machine guarantees you a descent into Hell." Macintosh and PC/Windows boxes just don't have what it takes to run database-backed websites, or even stable static sites. ("A $50 chip will consume $10K in system administration time.") He likes HP's Unix systems best, but would choose Sun if he needed access to lots of existing source code. Maybe Apple's Unix-based Rhapsody will become an option. The choice of machine is often dictated by the software you need to run.

Several chapters of his book are about the pitfalls of complex relational database management system (RDBMS) approaches that he's had to implement for some of his clients. (He's done over 50 websites. "Even if your own site loses money, once you've absorbed the lessons of the next three chapters you can pull in $1,250 per day as a Web/RDBMS programmer.") He does warn that you don't want to implement your own database kernel; that's too big a risk. ("If fixing bugs and adding features to online systems handling 20 hits a second were easy, you would not be getting paid $1,250 a day.") SQL has been around long enough to be stable, so most of the RDBMSs are reliable -- once you figure out how to set them up. You may want the context-sensitive full-text information retrieval capabilities of Oracle with ConText or of Illustra/Informix Universal Server with PLS and Verity blades. (Expect to pay all of your website income, plus $50K/year for "support." You'll also need a fast machine with lots of big disks.)

(If that $1,250/day figure interests you, consider the $6,500/day charged by Philip's friend David. He's a SAP consultant. SAP is a German company that sells data models, which US companies buy in the hopes of getting their computers to talk to other companies' computers. Automated product ordering, for instance. David doesn't have to know SQL, or do programming, or do systems administration. "He just has to fly first class around the world and sit at conference tables with big executives and opine that perhaps Oracle Financials would be better for their company...")

If most of your action is retrieving records by name or index, you may want to consider an object database system such as Franz Inc.'s Lisp-based Allegro Store. Franz also offers a Lisp Web server and connectivity to any RDBMS, all mostly free if you're using the Linux version of Unix. See .

Whatever commercial or open-source RDBMS system you choose, try to keep complexity to a minimum. Many of Greenspun's solutions involve multiple servers running partial databases instead of one CPU trying to do it all. (He's still trying to perfect an intranet-based approach for real-time information publishing and access within a hospital environment. The design philosophy looks impressive.) For most applications, the occasional downtime of a simple Unix Web server with $500 uninteruptable power supply will be far less of a problem than the complexities of a full RDBMS real-time transaction processing system.

People do a lot of things for fun rather than profit, and there's no reason webmastering has to be different. Philip believes in sharing with others via hobby websites, without worrying about commercial payback. 'Trouble is, it takes professional skills (and time) to run fancy databases on a 24/7 Web host. Even Philip doesn't have that kind of time, so he's piggybacked most of his interactive Web services on the ArsDigita commercial site, , that he and some friends run for their clients. (As consultants, they can now do development work on their own multi-RDBMS system instead of upgrading client's computers with needed hardware, operating systems, applications, and Internet connections. Much easier. Besides, now they can charge $50K setup and $5K/month hosting fees to support the system maintenance that they would have had to do anyway for their own use.)

Now here's the great part: he's willing to do the same for you. All of his website structures and interactive forms, including his Loquacious 3.0 reader Q&A forums, can be duplicated for your own website. You just have to link to his server via URLs whenever you need a dynamic page -- your web visitors probably won't even notice. You can see demos and download code from Greenspun's , or read about the capabilities on .

The book? It's pretty good. Conversational, easy to read, inspiring. Has a few good phrases, such as "a metric buttload of information" and "Your Web site will suck if you see it as a pimple on the butt of something much larger." Points you to some powerful tools you can download, and warns about the dangers and seductive promises of commercial "middleware." Has lots of real-world how-to advice, plus introductory theory and chat to keep it all readable. The book comments on ISDN vs. T1 vs. cable modem vs. ADSL; offers quite a few tips on handling digital imagery; and entertains with lots of strong opinions. Greenspun's acknowledgments section is particularly fun, though reprinted from Olin Shivers' Scheme Shell Reference Manual. The book was worth my time. Purchase "Database Backed Web Sites" from Amazon.com, or read "How to be a Web Whore Just Like Me" -- the title he wanted to use -- on . (You can link to Amazon's order page from Greenspun's pages.)

Greenspun's final chapter pets his peeves again, and is somewhat hard to follow. His chief point is that we should abandon our commercial software model -- with binaries sold like tables and chairs -- and allow users to pay a flat rate for access to large collections of software. Producers would be paid from the income pool in proportion to use of their code, either used directly or called from other programs. Producers would then have an incentive to offer bug-free code, intuitive interfaces, and attractive documentation in order to spur more use (as opposed to fancy Web sites, big advertising campaigns, and lying press releases to spur more sales). The result would be "A Future So Bright You'll Need to Wear Sunglasses."

-- Ken