On Friday we announced the release of the second annual 2009 Open Source CMS Market Share Report. The project was a collaboration with water & stone, an APAC-based interactive agency. The initial announcement addressed what the report covers, its purpose and some key highlights.
For the sake of disclosure, while I did look over the preliminary survey and advise on some tweaks, I otherwise wasn't involved in this report and am looking at this data with fresh eyes and no particular bias from the side of CMSWire.
Now let's take a look more in depth.
The authors of the 2009 Open Source CMS Market Share Report made efforts this year refine the selected CMS's over last year's choices -- both in terms of including a broader range of programming languages (.NET and Java-based CMS's as well as PHP), and ensuring that those projects chosen are similar enough that they can be compared apples to apples (all Web CMS's) rather than spreading out to too many different types of tools (this time social networking systems and wikis were not included).
In particular, the report's authors point out that this report in no way represents which system is best, the most full-featured or the most powerful. Its focus is strictly on market share and brand strength. The content management systems considered were:
- CMS Made Simple
- eZ Publish
Changes in Methodology Generate Meaningful Numbers
For the 2008 report, data was gathered through Twitter mentions and social bookmarking statistics. This year, water & stone teamed up with CMSWire to present a survey to their readers. Doing so gave the authors targeted data to interpret from more than 600-1200 people (depending on the question asked). Don't expect to see comparative metrics in the 2009 report for these numbers, as there's nothing from 2008 to compare them against.
According to the survey results, the typical participant is a 35 to 44-year-old male in North America, with a graduate degree or higher. He's worked in IT for 10 to 15 years and still works in the computer, software, or technology fields. His annual household income is between US$51,000 and US$100,000, he works for a small organization of 1 to 5 people.
Areas considered were rate of adoption and brand strength.
Pros and Cons of the Methodology
While there were survey results to deal with, it's helpful to get data through other avenues as well, to both correct for any bias in your survey sample and in general get a broader picture of the world you're exploring.
The problem is that with survey results, you've designed the questions so that you can get some sensible measurements out of them. With real-world data, this desire doesn't come so easily. For example, in measuring the rate of adoption through looking at download metrics, the authors ran into a number of hairy issues. These issues are detailed in the report.
- You can't get data on the number of downloads for every project
- Even when you can get the data, the time scales the data covers differ wildly
For some projects the number of downloads are counted from the beginning of time. With others, they may be counted just for a particular major version or point release. Some don't give you any idea of what the time scale is at all.
Once you deal with these two issues, you also have to face:
- Mirrored download sites where statistics aren't automatically aggregated
- Skewed weekly averages since download rates are highest when a new version is released and taper off until the next release
- Excluding installation packages such as cPanel, Plesk, Fantastico, and Linux distribution package managers
- Having access only to the download numbers for the free, open source/community edition versions of those projects that also have commercial versions
In other cases, attempting to find hard and fast numbers can limit you to sources that themselves introduce bias. For example, consider needing to measure the number of third parties offering services and support around an open source project. The authors chose here to look at two classes of service providers: developers offering services around the projects, and publishers releasing books about the projects.
Here you have to make a choice on where you get your numbers. To determine developer support, the authors consulted two sites for finding freelance technology professionals: Elance and Guru. Choosing these two sites skews the results toward the types of small-scale and one-person shops that use them. It would have been interesting to see similar results through the survey as well, to see if they were the same or quite different.
Also, given that some of these projects are more customizable than others (though anything's customizable with the right programmers) it's hard in some ways to quantify market share with just these numbers. CMS Made Simple is one of the lowest for development services offering, but what percentage of its users heavily customize it?
When it comes to publishers, it's important to note that only books in English were included for this section. Given that some of the projects have their biggest fan bases in Europe, Asia, or other areas that aren't primarily English-speaking, this factor might highly skew the results. Also, this data comes exclusively from Amazon.com, so if the book isn't sold or listed there for some reason, it wouldn't be included.
Always consider the methods for each section when evaluating such reports and deciding how much weight you want to rely on each section.
The Report's Conclusions
Overwhelmingly, the results were dominated by WordPress, Joomla! and Drupal, though they swapped order here and there. On top of this, there was usually a large drop after these three, showing that the open source Web CMS space is heavily dominated by these key players.
Joomla! Wins the Popularity Prize
The authors point out that while last year the three had similar market share, this year Joomla! has taken the lead in a few key metrics, such as how many survey respondents said they're currently using it, and areas such as brand recognition. For this reason they've named Joomla! the most popular open source web CMS.
However, they also point out that Joomla! had a higher level of negative brand sentiments, and it lags in social media prominence. So the Joomla! camp shouldn't get too cocky.
WordPress' Dominates Brand Strength Category
When it came purely to brand names, WordPress is the winner, however. The report partially attributes this finding to the fact that there are two products with the same name: the WordPress hosted blog service and the WordPress CMS project. From there, the authors postulate that the WordPress hosted service is seeding the market for the installed software by getting people started with turnkey solutions and then inspiring them to run their own sites. Will the same happen for Drupal with Drupal Gardens? Only time will tell.
It's also hard to miss that PHP-based CMS's still dominated the open source CMS field. DotNetNuke is the clear leader in the open source .NET CMS space, while Jahia lagged badly behind the other Java-based offerings.
The authors close with a list of projects to watch and those that need to watch out. In particular, they identify Alfresco, Liferay, and MODx as gathering strength and market share. DotNetNuke, Plone, and Xoops are all identified as struggling to maintain their market share. Those they identify as being at risk of total irrelevance are phpWebSite, Textpattern and TikiWiki.
For more detailed information, download the report for free and read it for yourself. We're certainly interested in your thoughts.