This article is an update version of my previous article CY12-Q3 Community Analysis — OpenStack vs OpenNebula vs Eucalyptus vs CloudStack. Readers who are intested in further discussions please contact me via email at the above-mentioned address.
A Chinese version of this article is published at the same time, which can be found at CY12-Q4 OpenStack, OpenNebula, Eucalyptus, CloudStack社区活跃度比较.
It should be noted that this community analysis project was initiated in CY11-Q4, and this particular report is the 5th quarterly report being published since. Although the author became an employee of Eucalyptus Systems Inc in October 2012, the opinion presented in this report belongs strictly to the author rather than the employer of the author. It should also be noted that the employer of the author completely agreed that the author could continue this project with an independent perspective.
The objective of this article is to compare the OpenStack, OpenNebula, Eucalytpus and CloudStack user and developer communities, base on the communications between community members in the form of mailing lists or pubic forum discussions. The data being discussed include the total number of topics (threads), messages (posts), and participants (unique email addresses or registered members). To obtain the above-mentioned data, a Java program was written to retrieve all the forum posts and mailing list messages into a MySQL database for further processing. The analysis results were presented in the form of graphs generated by LibreOffice.
It should be noted that the CloudStack project moved completely from the original http://www.cloudstack.org/ site to http://incubator.apache.org/cloudstack/, and stop using the forums originally hosted on http://www.cloudstack.org/. In this report, we neglected the data published on the forums during CY12-Q4, which would produce some impact on the analysis results.
Also, when the CY12-Q2 report was published, some people questioned the inclusion of the incubator-cloudstack-dev mailing list. This particular mailing list contains a lot of messages that are automatically generated by JIRA. In CY12-Q3, we set up a filter to reject all messages with identifier “[jira]” in the subject.
Figure 1 and 2 represent the monthly number of topics (threads) and posts (messages). It can be seen that
(1) the volume of OpenStack and CloudStack related discussions is much higher than that of Eucalyptus and OpenNebula; and
(2) the Eucalyptus and OpenNebula clubs are exhibiting similar behaviors, with only minor differences.
Generally speaking, the number of replies to a specific topic represents the attention being received, and the depth of discussion for that particular topic. When the number of master posts (the original post that started a particular topic) is more than the number of replies, it is safe to conclude that the participation of the forum or mailing list is very low. Therefore, the ratio between “the number of posts” and “the number of topics” represents the participation rate of an online community. In this study we call this ratio the Participatin Ratio.
In the past the OpenStack project had a much higher participation ratio than the others. However, during the past 6 months, the participation ratio of CloudStack and Eucalyptus are growing steadily, while the participation ratio of OpenStack is decreasing gradually. Currently CloudStack and Eucalyptus have the highest participation ratios, which are close to 4. OpenStack and OpenNebula have relatively low participation ratios, which are close to 3.
Figure 4 shows the number of monthly participants of the four projects being discussed. It can be seen that the active participants of CloudStack and OpenStack are much higher than OpenNebula and Eucalyptus. However, during the past 3 months, the number of participants for both CloudStack and OpenStack have decreased slightly.
It should be noted that although the number of active participants of CloudStack is somewhat less than OpenStack, but the volume of discussion (in terms of monthly number of threads and messages) of the two projects are on the same level. This indicates that the active members in the CloudStack club are talking more than those in the OpenStack club (on average).
Accumulated Community Population refers to the total number of users and developers who have participated in forum or mailing list discussions. (This number does not include those who have registered into discussion forums or mailing lists but have never participated in any open discussions.) These are people who have tested or used a specific product for a while, but not necessary currently an active user.
Figure 5 shows the accumulated community populations of the four projects being discussed. It can be seen that OpenStack and Eucalyptus have far more accumulated population than CloudStack and OpenNebula. Also, in our CY12-Q3 report we predicted that the OpenStack population will exceed the Eucalyptus population in CY12-Q4. Actually, it only took OpenStack a month to accomplish that!
In our CY12-Q3 report, we raised the problem of possible duplicated population counting due to the membership overlap in forums and mailing lists, and carried some measurements to eliminate duplications. In our CY12-Q4 report, further measurements were taken to de-duplication. It is safe to say that some degree of duplication still exists, but does not have a significant impact on the analysis results.
Figure 6 shows the monthly population growth of the four projects being discussed. During the past 3 months, the populations of OpenStack and CloudStack are growing at the same pace. (The CloudStack data exhibit an abnormal point at 2012-10 because we excluded the forum data from this analysis.)
The populations of Eucalyptus and OpenNebula are growing at relatively slow paces, as compared to that of CloudStack and OpenStack.
Figure 7 is a combination of Figure 4 and Figure 6. The solid lines represent the monthly participants, while the dash lines represent the monthly new members.
For OpenStack and OpenNebula, around 30% of their monthly participants are new members. For CloudStack and Eucalyptus, around 50% of their monthly participants are new members. This indicates OpenStack and OpenNebula communities are more “sticky” than CloudStack and Eucalyptus communities.
For each of the projects being discussed, the monthly population growth is somwhat “synchronous” with its monthly participants. That’s to say, the populatoin growth of a community is somewhat related to the “activeness” of the community. This also suggests that both the population growth and the “activeness” of a community might be event-driven. A new software release, a technical conference, or a marketing event, might be the cause of the growth in population and “activeness” of the respective community.
Figure 8 shows the total community population, active participants of the past quarter, and active participants of the past month, of the four projects being discussed. It can be seen that
(1) OpenStack has the largest total population, followed by Eucalyptus, CloudStack, and OpenNebula;
(2) OpenStack has the largest active population during the past quarter, followed by CloudStack, Eucalyptus, and OpenNebula;
(3) OpenStack has the largest active population during the past month, followed by CloudStack, Eucalyptus, and OpenNebula.
In our CY12-Q3 report, we invented the concept of “Community Activeness Index”. This magic number should be the combination of the following parameters:
(1) monthly messages, which represents the volume of the discussions;
(2) participation ratio, which represents the average number of answers to a question;
(3) active population of the past quarter, which represents the possibility to get help from community in the long term; and
(4) active population of the past month, which represents the possibility to get help from the community in the short term.
In this analysis, we choose the average values of these parameters as the reference data set, and compare the corresponding parameters of each community with the reference data set. Then we call the sum of the relative values of a community the “community activeness index” of the community. Now we can say the project with the highest “community activeness index” is THE most active project in this area.
As can be seen from Figure 9, OpenStack is currently THE most active project (with obvious advantage), followed by CloudStack, Eucalyptus, and OpenNebula.
The above-mentioned concept of “community activeness index” is still very primitive, with a lot of space to optimize. However, it is an attempt to replace the old-fashion “I think”, “I believe” and “I guess” practices with quantative analysis. In our future community analysis, we will continue to use this concept to provide a quarterly ranking for OpenStack, OpenNebula, Eucalyptus, and CloudStack. Improvements to the algorithm (such as adding/removing parameters or changing the weight of different parameters) will be make when necessary.
From an end-user’s perspective, it is good to see the competition heating up because that means more choices with better quality. Cloud computing is still an evolving market that is highly inmature, and we expect more competition to come in the future.
For your convienience, a PDF version of this presentation can be downloaded from here. Please kindly keep the author information if you want to redistribute the content.
Apart from my quarterly report on community analysis, Balman A. Baset from IBM is also doing some work to compare the different IaaS projects. In the recent ACM Symposium on Cloud Computing (SoCC-2012) he published a tutorial on Open Source Cloud Technologies, which is a good reference for all cloud computing practitioners.