Alexa from A to Z: The Rise and Falter of a Web Pioneer

Alexa has been around since the dawn of Web time, well pretty much. This organization has seen it all. Alexa was caught in the maelstrom of the greatest boom and bust since the Depression… and survived. The group enjoyed exceptional growth in a totally new business and technological environment, followed by a long and languid fall from eminence. It innovated like crazy, and its innovations reshaped the landscape of the Web. And it was subject to one of the Web’s first huge, headline-grabbing takeovers. Latterly, Alexa has faced stinging criticism over its traffic measurement model, and has responded by going back to the drawing-board with a new model and a fresh attempt to resurrect itself as a relevant Web player.The Alexa story is part cautionary tale, part glorious legend. This is that story in 26 letters.

A... is for A New Beginning

Until recently Alexa's website popularity rankings were based solely on data taken from users of its browser toolbar. The install base of the toolbar was seen as a representative sample and user habits were extrapolated to estimate the relative popularity of different websites.But that all changed in April 2008 with the news that the company was casting its net further to take into account other sources. The metrics data you now see when you profile a website on Alexa.com is still taken largely from toolbar users, but now also "aggregates data from multiple sources" (nobody knows), which includes a vague and misty "global panel". How exciting, we say.

B... is for Bot for Hire

By 2005 Alexa's crawler was one of the Web's biggest, hitting 4 to 5 billion pages a month and archiving 1 terabyte of data a day (it’s now up to 1.6 terabytes per day, and a total of 4.5 billion pages from over 16 million sites ). In a bold move designed to open up a whole new market, it announced that it would open up its crawler to requests from the floor to anyone willing to pay.The Alexa Web Search Platform, which enabled programmatic access to Alexa's web search engine, briefly threatened to spark a search revolution... and was then sucked into Amazon Web Services.

C... is for Competitors

...in the website rankings space like Compete and comScore. According to Compete's own measurements, they now have the upper hand on Alexa (below). More notable is the Quantcast graph, but the 'People Count' metric may be polluted somewhat by Quantcast's innovative model (feel free to fill us in below). Quantcast gets their figures for opted-in websites via direct measurement, and we covered the model in some depth earlier this month (and here). The rest use toolbars, panels, random irritating phone calls, data from ISPs and a variety of other methods to get their traffic figures.Compete sold for 'up to' US$ 150 mil. earlier this year. Quantcast, the new kid on the block, has raised US$ 20 mil. in funding. comScore floated in 2007, got into a big row with Google about paid clicks, and have a current market cap. of US$ 650 mil.

alexa vs compete vs quantcast, alexa, quantcast, compete

D... is for Alexa Directory

Yes, they've got a directory too, which is in fact a direct copy of DMOZ, the Open Directory Project. Alexa has added the ability to "sort categories by Popularity and Avg. Review Rating, and you can access all of Alexa's helpful site information just by clicking on the title of any site." Find the Alexa Directory here.

E... is for Egypt

...which is where the Library of Alexandria stood, from which Alexa gets its name. One of the Wonders of the ancient world, the Library was by far the greatest collection of recorded information in olden times, storing so many archives, it is said that it would take the length of three SteveNotes speeches toread them all. The Library was eventually destroyed (no one is entirely sure how), so the conjecture will likely remain untested.

F... is for Flat File

CMSWire.com Useless Geek Fact of the Day # 10009: Alexa uses a flat-file and not a SQL database for analysis and storage. Apparently for massive data sets flat-file is the only show in town. More on this fascinating topic on O'Reilly Radar. Flat-file is an emotive topic, try not to step in the puddles of blood over there.Meanwhile, if you really want to see what a Flat-File bloodbath is all about, have a look in the comments section of this, for what is surely the greatest comments-war Content Management has ever seen.

G… is for Bruce Gilliat

Bruce is the co-founder of Alexa along with Brewster Kahle. He studied at MIT, Golden Gate and Berkeley, and worked in fibre-optics before teaming up with Kahle at WAIS Inc. in 1994 and jumping onto the roller-coaster that became Alexa (see V). He remained with Alexa after the 1999 takeover (see J), served as CEO, and left the subsidiary in 2007. Gilliat has sat on various boards and seems to keep pretty quiet.

H… is for Hornbaker, Ron

Hornbaker got into a legal tangle with Amazon Corp. in 2006 after starting a website called Alexaholic.com, on which he displayed Alexa traffic graphs and ‘…misappropriated the Alexa name’. Hornbaker disagreed with this assessment. And Alexa vs Hornbaker duly went to the lawyers. Hornbaker was ultimately forced to re jig his service and rebrand it as Statsaholic.

I... is for Internet Archive

IA is a non-profit which was started by Alexa’s Bruce Kahle in 1996, which maintains an on-line library and archive of Web and multimedia resources.Its explicit mission is to "help preserve [Web] artifacts and create an Internet library for researchers, historians, and scholars." The Wayback Machine is a service created by the Internet Archive which allows public users to view snapshots of websites as they were on a given date. Wayback gets its data from Alexa’s crawler, and “As of 2006 … contained almost 2 petabytes of data and was growing at a rate of 20 terabytes per month, a two-thirds increase over the 12 terabytes/month growth rate reported in 2003.” (Wikipedia).Check out the very first recorded edition of CMSWire, which ran off Movable Type 2.64.Internet Archive is based just down the road from Alexa at the Presidio, SF (see P), in a strikingly beautiful old Colonial-era building.

Internet Archive HQ

J... is for Jeff Bezos

Bezos is the infamous head bottle-washer at Amazon. It was 1999 and the first, crazed Internet bubble showed no signs of bursting. Flush with equity (despite still being several years away from profit), Amazon decided that Alexa would be the perfect acquisition to round out its portfolio and try to invade the browser.Alexa’s hand was strong: the Alexa toolbar was running in millions of browsers, but the world was still stunned when Bezos ponied up US$ 250 mil. in Amazon stock. At the time it was a headline-maker -- two Web startups, of all things, locked in a multi-million dollar battle to the end. What Google ultimately became, that’s pretty much what people thought Amazon/Alexa might become: a multi-disciplinary, domineering monster. Things didn’t exactly work out that way, but as they’re both still in business and making money, the bottom line is that things worked out pretty okay. Bezos used to sell hedge funds until he got the big idea that there might be a way to make money out of the nascent Web, founding Amazon in 1995 and going live with Amazon.com in 1996. For some baffling reason, online bookstores emerged as the first killer commercial applications of the Web.The first Internet bookstore is thought to have been a San Francisco Sci-Fi outlet (tell us if you know), and the second is believed to have been Kenny's of Galway, Ireland. (The guy who built the latter taught this blogger eCommerce, which is how he knows such a morbidly obtuse and utterly useless fact.)

K... is for Brewster Kahle

Kahle was co-founder of Alexa with Bruce Gilliat. He graduated MIT in 1982, where he studied Artificial Intelligence under the godfather of that frustratingly elusive field, Marvin Minsky. He subsequently worked at Boston firm Thinking Machines, which made several breakthroughs which probably seemed more relevant to AI at the time (see V), but which ultimately proved more important to the very, very young World Wide Web. By nature a philanthropist, Kahle founded the Internet Archive (see I) and fronts a charitable foundation (The Kahle/Austin Foundation) with his wife. He is also a member of the American Council of Arts and Sciences

L... is for the Library of Congress

... to which, with much fanfare, Alexa began donating copies of its web crawl in 1998.

M... is for Microsoft

Microsoft contributed hugely to making Alexa an early Web giant by incorporating Alexa technology into Internet Explorer 5 in 1999. The move came in response to Netscape’s earlier bundling of the Alexa toolbar with Navigator. CMSWire Useless Geek Fact # 10010: IE 5 incorporated the first version of XMLHttpRequest, thus setting the scene for AJAX.

N... is for Nineteen Ninety-Six

One thousand nine hundred and ninety six is the year that Alexa came to be. It was also the year that Deep Blue beat Kasparov, the Bulls beat the Supersonics to land the NBA title, and Duke Nukem 3D and Resident Evil were released on Sony Playstation.

O... is for O'Driscoll, Niall

Nial is the current Alexa CEO and has been with the company since 1996. O’Driscoll studied Mechanical Engineering at MIT (again…), before studying at Columbia and University College, Dublin. Here's the rest of the Alexa management team.

P... is for the Presidio of San Francisco, Building 37

It's in San Francisco's Presidio that Alexa finds its home. The Presidio is on the site of an old Spanish fort which dates back to 1776. The Spanish were subsequently routed and in 1846 the Presidio was seized by the US military, which eventually based the 6th Army there. The Army eventually moved on in 1995, and doubtless looked on in horror as a procession of pale-faced, slack-jawed, millionaire whiz-kids started snapping up office space in the former military citadel.

Q… is for Quiet Please

...while we try to come up with something for R...

R… is for Rankings

Alexa started ranking websites in 1998, based on the browsing habits of its toolbar install base, but updated this methodology earlier this year (see A).Here’s where rankings come from: “A site's ranking is based on a combined measure of reach and pageviews. Reach is determined by the number of unique Alexa users who visit a site on a given day. Pageviews are the total number of Alexa user URL requests for a site.

S... is for Self Selection

Self selection is a key criticism of toolbar-based models for ranking websites. From Wikipedia: “Self-selection is a term used to indicate any situation in which individuals select themselves into a group. It is commonly used to describe situations where the characteristics of the people which cause them to select themselves in the group create abnormal or undesirable conditions in the group.”In other words only a certain type of person is likely to go to the bother of installing the toolbar. Mostly tech-savvy, beardy Web fiends back in the early days, when Men were Men and browser plugins were the eighth wonder of the world.

T… is for the Alexa Toolbar

The Alexa Toolbar has been downloaded at least 10 million times over the years (10 mil. downloads by 2005), and is reckoned to have been the first browser toolbar. Today it comes in two primary flavors: Toolbar for Internet Explorer and Sparky for Firefox.Toolbar for IE offers a search box, Site Info (Whois information), anti-phishing protection, the traffic rank of the page you’re visiting, related links and a couple more useful gadgets.Sparky for Firefox shows related websites for the site you’re visiting, traffic trend, Traffic Rank and Reach meter.

U... is for "ummmmm... are you sure?"

What everyone says when Alexa throws up statistics like Yahoo.com being more popular than Google.com.Here's the top 10 as of 5/21/08:# Yahoo.com# Google.com# YouTube.com# [Windows]Live.com# MSN.com# MySpace.com# Wikipedia.com# Facebook.com# Blogger.com# Yahoo.co.jp

V... is for Very Complicated

Which is what WAIS (Wide Area Information Server) is. WAIS was "the first internet publishing system" according to Brewster Kahle. Kahle was either one of the inventors of the technology or the sole inventor, depending on the version you prefer, while working at Thinking Machines of Massachusetts. He and Bruce Gilliat left Thinking Machines to start WAIS Inc., which subsequently was sold to AOL for US$ 15 mil. in 1995. The move funded the startup of Alexa and the Internet Archive.More on WAIS.

W... is for Alexa Widgets

And you you can get them here.

X... is for Xanax

This is what Niall O'Driscoll starts popping when heavyweights like Michael Arrington start venting about Alexa's metrics. (We're just kidding Niall, don't sue us.)

Y... is for "Yes Alexa is still important."

Because, like it or loathe it, advertisers like TextLinkAds compute ad rates partly on the basis of Alexa rank and a lot of people still consider it a semi-respectable measure of popularity.

Z... is for Zeep

Because 'tis said by many that the real source of Alexa's published traffic figures is the mysterious Captain Zeep, who is conjured by going to a dark crossroads in the San Fernando valley at midnight, putting a lit candle behind your Macbook and twittering certain secret Jakob Nielson quatrains.