Newsletter:

Survey: Most Popular Open Source CMS

By Irina Guseva

Jul 29. 2008

Filed Under: drupal joomla open source open source cms web cms wordpress

Water & Stone, a web development company specializing in Open Source technologies, released a popularity survey of open source content management systems. In the analysis of 19 of the most prominent open source content management systems, Water and Stone evaluated CMS’s on the basis of Rate of Adoption and Brand Strength, as well as a variety of other viability indicators and trends.

Leaving Alfresco Out

Aside the strange fact of not including the open source giant Alfresco in its evaluation, the discussion seems to be pretty interesting. Water & Stone described this decision as “although popular and suitable for web publishing,” Alfresco and others are “primarily intended for more narrow uses.” Not sure what that means, but we will go on.

SPONSORSHIP

CMSWire speaks to a specific audience of professionals and opinion makers focused on content management, publishing and collaboration.
Advertise here.

While WordPress, Joomla! and Drupal lead across a wide range of measures, the survey also identified Elgg and MODx as rising stars.

How the Judging Was Done

Whereas the methodology used by Water & Stone is far from the principles of either quantitative or qualitative analysis, the company is judging open source CMS by two categories:

Rate of Adoption
Brand Strength

Additionally, and unheard of before, Water & Stone uses Twitter prominence and Social Bookmarking statistics as part of their “scientific” approach to the analysis of open source CMS popularity. Must be the influences of the 21st century… However, what happened to hard facts, numbers, simple mathematics and customer interviews?

The Leaders, Movers and Laggards

In any case, let’s look at leaders, movers and laggards as they are identified in the paper.

Rate of Adoption

To measure the “Rate of Adoption,” the company used the following metrics: the number of downloads and installations; and third-party support by developers and publishers.

To gain further insight, Water & Stone looked into the publishing market and evaluated open source CMS’s based on the extent of publishing market penetration, i.e. books in print. Granted, they only looked at books published in English — not sure why — but Joomla! came in first, closely followed by Drupal and WordPress.

To further sabotage the quality of its own data, Water & Stone announce that the adoption rate analysis revealed “no dispositive statistics, due largely to the incomplete and unreliable nature of the data reviewed.” Hmmm, OK. Is it that hard to track downloads and installations?

Somehow — even though unable to review the “incomplete and unreliable nature of the data” — Water & Stone concludes that WordPress and Joomla! exhibit significant download volumes.

It continues with the following statement “Given the lack of direct evidence on the rate of adoption, we are forced to turn to indirect indicators,” and come to the conclusion that Joomla! is the winner in the third-party support group. Seems like Water & Stone folks just gathered around a table, shut their eyes and poked at a random name on a list of open source CMS names, fresh off the printer.

Brand Strength

Brand Strength was measured by the following metrics:

Search engine visibility
Popularity metrics
Evidence of mindshare
Evidence of reputation

The search engine optimization methodology has never been more popular. Which leads us to wonder that if an open source CMS vendor did a poor job in filling out all those title and meta tags does their open source CMS suck? Not to mention the fact that Alexa rankings data is not the most accurate source of web popularity out there.

News mentions, used as a metric, cannot be as far from reality as the fact that Mars revolves around Earth. Again, the fact that PR folks did a poor job publicizing a certain open source CMS shouldn’t reflect badly on the product. The top three WordPress, Joomla! and Drupal come close in this metric. Hooray to WordPress PR department for being in the news more consistently than any other open source solution.

Joomla! is leading the pack by the number of Google’s Blog Search entries — which can be nothing but a mere correlation to Joomla!’s summer release of new versions.

Judge the report for yourself. Once you’ve had a chance to digest it, come back and tell us what you think of the analysis and results.

Was this article useful?

Email a Friend Digg It!

Digg It

Comments

Wordpress!

Posted by: cenourinha on July 29, 2008 8:11 AM

LOL - I've just read it and it really as bad as you've made out. Since when did popularity equate to being the best? Does this mean that Internet Explorer is the best browser?

The products that were omitted seems a little strange. So we've got PHPNuke, Xoops, etc - all of these are basically portals with content publishing... ...err just like Liferay. they've stated that Interwoven isn't there because it's proprietary so thats fine and well but Alfresco isn't there because.. err.. why exactly?!?

I checked their homepage. It states "We began in 2003, as a specialty house focused exclusively on building web sites with Mambo, Joomla, Drupal and osCommerce". I mean, what are the chances that they're going to favour anything but the software that they base their business on?

Posted by: davo on July 29, 2008 8:29 AM

Thanks for the coverage of the report -- we appreciate it.

As for the criticisms, noted, but I do think saying we "sabotaged our data" is rather odd -- after all you are basically criticising us for being upfront and disclosing the shortcomings of a data set. One would think that would qualify as being transparent and even-handed.

Remember, no one has undertaken this survey before -- most likely for the very reasons we discuss, i.e., the poor quality of direct metrics. The approach employed (looking at a very wide sampling of data for trends) is really the only way to undertake this without massive a spend on surveys, etc.

As for Alfresco, it is a fine system but was excluded from the sample for the reasons we stated in the report, that is, it is primarily a document management solution. Liferay is not part of the set for the reasons stated in the report as well. We can't cover it all, guys -- some decisions had to be made.

As for the rather strained allegation of bias from "davo" above, I deny it. We did not slant the data or the analysis. The report is long because we disclose all the data -- we make it possible for you to draw your own conclusions or repeat the research even with URLs to the various sources.

Posted by: Ric Shreves on July 29, 2008 9:31 AM

Thanks for the coverage of the report -- we appreciate it.

And as for using popularity as a measure -- considering that we were covering market share, that seems not too unreasonable to me, especially since it was only one factor of several!

Posted by: Ric Shreves on July 29, 2008 9:34 AM

This report seems to be more an PR exercise than a serious piece of market research. They succeeded, though :-) My employer uses Drupal [you guessed that probably by the provided name] but, that doesn´t mean that I claim there are no other excellent CMS frameworks out there.

I would take that information too seriously - in the end its an eternal discussion anyway.

Posted by: Drupal-ista on July 29, 2008 2:21 PM

I wonder if some are reading more into this then the report suggests? The study offered verifiable, fact-based metrics from many sources with a central question of open source CMS "popularity." It might be a more interesting discussion if it related to the same topic.

Here is the list of metrics on which analysis, and ultimately conclusions, were shared.

- Average weekly download rate (Source: Project)
- 3rd party developer and publisher support (Source: Elance and Guru)
- Books in Print (Source: Amazon)
- Inbound Links (Source: Google July 2008)
- Search Engine Rankings specified phrases (Source: Google July 2008)
- Alexa Ranking (Source: Alexa February and July 2008);
- Google Trends (Source: Google 12 month aggregate July 2008)
- Twitter (Source: Twist July 2008)
- News Mentions (Source: 12 month Google Trends July 2008)
- Blog Mentions (Sources: Technorati, BlogPulse, IceRocket, - Google Blog Search July 2008)
- Neutral Demo Site (Sources: Open Source CMS)
- Social Networking Sites (Sources: Facebook, Google, MySpace)
- Awards (Sources: Packt, CNET Webware 100, LinuxWorld, SourceForge, BNP)
- User Ratings: (Source: OpenSource CMS)
- Social Bookmarking (Source: Delicious, Digg, and Reddit)
- Project Reach (Source: Quantcast, Alexa, Google Trends)

Which of that list would you refute as a metric of "popularity?" And, which metrics would you substitute instead?

If one believes another open source CMS should have been considered, the data is easy to access. Go get the data for that CMS and share those values.

Then, based upon that data, sharing one's well-reasoned analysis and set of conclusions would be of interest.

I mean, anyone can rant. :-P

Posted by: Amy Stephen on July 29, 2008 7:28 PM

PR exercise? Presumably yes. But still a valuable snapshot of the industry using an extensive and imaginative array of metrics. Good work imho. Apart from leaving Alfresco out, which makes no sense to me.

Posted by: john conroy on July 31, 2008 9:56 AM

It should be relatively easy to apply the same metric analysis to see how Alfresco compares with the study group since the report analysis is pretty much reproducible for any product that takes your fancy. Nevertheless, I would still agree with the report writer's assessment that it is a specialist application and I would argue it is probably better assessed with its peers rather than this study group.

I don't know of any publicly available (let alone reliable) source of installation data for any web based system - please share if you know of one.

I agree with your notes that popularity does not make for the best - but it should follow that WordPress, Joomla! and Drupal need to be very good to be where they are (not to mention Typo3 and the Nukes certainly deserve their mark on history). To think otherwise would really be taking a marginal position which is not supported by publicly available information. Though, to find the best (actually defining what that means will be a challenge in itself) would be a daunting task - but pretty exciting nonetheless. I'd be keen to read that report as well - if it's possible to write it at all.

The reality is though, if even one of those systems never existed, the internet would be a very different place. They have all shaped this world in one way or another (love them or hate them). That's pretty awesome well you sit back and think about it.