The Recombinant Web

This article appeared as my column for Connect Magazine in October 2005.

Lately, I've been seeing more and more examples of what could be termed the recombinant Web, but are more affectionately called "mash-ups." A mash-up happens when someone takes two or more different services on the Web and combines them in a new and interesting way. Usually the person who creates the mash-up has nothing to do with either of the services being combined. Mash-ups have been around for a while, but they're exploding now, partly because of Google maps.

CheapGas shows the location and driving directions for cheap gas in Salt Lake City. (click to enlarge)

CheapGas is one great example. Using CheapGas, you can see a map showing where cheap gas is in your neighborhood and even get driving directions (see figure). CheapGas doesn't have a list of gas prices or mapping software and it doesn't need them. Those are provided by GasBuddy and Google respectively. CheapGas remixes those services to add value in a way that neither of them did before.

Another great example is Paul Radermacher's HousingMaps that combines data on housing from craigslist with Google maps to show houses for sale or rent in a given area. Again, this is done with no support from the original sites and adds new value.

Lest you think this is all about Google Maps, consider GreaseMonkey. GreaseMonkey is a Firefox extension that allows you to add functionality to a Web site at the browser. For example, if you've used GMail, you'll know that it doesn't have a Delete button on the main page. Deleting email is a two-step process--archive then delete. Using GreaseMonkey, however, and a script you download from the Web, you can add a Delete button to the main Gmail page.

The motto of the recombinant Web is: "if you don't like how something works, change it." Most of these mash-ups and remixes are not complicated and people throw them together in a few hours, or at most a few days. I think we've only begun to see the power of the recombinant Web and what people will build, but there are a few flies in the ointment.

Here's one: www.mapsexoffenders.com, a product of Orem UT based Orbizon, use publicly available data about sex offenders and Google maps to display a map of your neighborhood and any known sex offenders who live there--unless you live in Utah. As of this writing, Utah's Department of Corrections won't let them use the data! Here's a company willing to build a great and useful service for free and we won't let them. For the recombinant Web to flourish, the data that makes it work has to be available. I'm especially concerned when data that's otherwise publicly available can't be recombined.

Another impediment to the recombinant Web is the many funny ways that people build Web sites. There are a few simple principles, that if followed, make remixing Web sites fairly easy. These include the right use of HTTP methods, using XML where possible, and not using frames. Remarkably, most of the interesting mash-ups are done by screen-scraping--literally grabbing the data out of the HTML itself. That works, but its not very robust. Changes to the HTML page break all the applications that depend on it. Microformats, which I discussed in last month's column are one answer to this problem.

Finally, there are identity issues. Many of the most interesting mash-ups aren't possible because the sites that you want to combine require user authentication and almost every Web site in existence uses its own, home-grown membership and login system. As long as there's only one site in the mash-up that requires authentication, you can usually manage, but after that, it can become messy. This one won't be solved cleanly for some time. There are bigger forces than mash-ups pushing the identity problem to a solution, however.

While XML and SOAP are finding great utility within the firewall, it's things like mash-ups, microformats, tagging, and RSS that are having the biggest effect on the Internet and forming what we might term Web 2.0. Web 2.0 isn't just a place of Web pages and single site applications, like Web 1.0. Rather it's a place where new value is created by recombining old applications. I think it's fun.

Phil Windley teaches Computer Science at Brigham Young University. Windley writes a blog on enterprise computing at http://www.windley.com and is the author of a book on digital identity published by O'Reilly Media. Contact him at phil@windley.com

Last Modified: Wednesday, 24-Aug-2005 16:10:29 UTC