CY13-Q2 Community Analysis — OpenStack vs OpenNebula vs Eucalyptus vs CloudStack
This article is an update version of my previous article CY13-Q1 Community Analysis — OpenStack vs OpenNebula vs Eucalyptus vs CloudStack. Readers who are intested in further discussions please contact me via email at the above-mentioned address.
A Chinese version of this article is published at the same time, which can be found at CY13-Q2 OpenStack, OpenNebula, Eucalyptus, CloudStack社区活跃度比较.
It should be noted that this community analysis project was initiated in CY11-Q4, and this particular report is the 7th quarterly report being published since. Although the author became an employee of Eucalyptus Systems Inc in October 2012, the opinion presented in this report belongs strictly to the author rather than the employer of the author. It should also be noted that the employer of the author completely agreed that the author could continue this project with an independent perspective.
The objective of this article is to compare the OpenStack, OpenNebula, Eucalytpus and CloudStack user and developer communities, base on the communications between community members in the form of mailing lists or pubic forum discussions. The data being discussed include the total number of topics (threads), messages (posts), and participants (unique email addresses or registered members). To obtain the above-mentioned data, a Java program was written to retrieve all the forum posts and mailing list messages into a MySQL database for further processing. The analysis results were presented in the form of graphs generated by LibreOffice.
Starting from CY13-Q2, the OpenStack project kicked off the Ask.OpenStack, which is now included as a data source for this analysis. It should be noted that there exists significant membership overlap between Ask.OpenStack and other OpenStack mailing lists and forums. Due to time limits we were not able to carry out membership de-duplication for this data source. We will try to deal with that in our CY13-Q3 report.
The Apache CloudStack graduated from the Apache incubator in CY13-Q2. The names of their mailing lists changed a little bit, but the membership and discussion scope remained the same. It should be noted that before this change there were a significant amount of message in the incubator-cloudstack-dev mailing list that were auto-generated by JIRA. Such messages became a lot less in the new cloudstack-dev mailing list. However, we still keep our previous filtering settings, which excludes any messages with “[jira]” in their topic.
Figure 1 and 2 represent the monthly number of topics (threads) and posts (messages). It can be seen that
(1) During the past 12 months, OpenStack-related discussions and CloudStack-related discussions were approximately on the same level, while Eucalyptus-related discussions and OpenNebula-related discussions were approximately on the same level.
(2) During the past 12 months, the volume of OpenStack and CloudStack related discussions were much higher than that of Eucalyptus and OpenNebula; and
Generally speaking, the number of replies to a specific topic represents the attention being received, and the depth of discussion for that particular topic. When the number of master posts (the original post that started a particular topic) is more than the number of replies, it is safe to conclude that the participation of the forum or mailing list is very low. Therefore, the ratio between “the number of posts” and “the number of topics” represents the participation rate of an online community. In this study we call this ratio the Participation Ratio.
As can be seen from Figure 3, during the past 12 months the participation ratios of CloudStack and Eucalyptus were relatively higher, which were close to 4; the participation ratios of OpenStack and OpenNebula were relatively higher, which were close to 4.
We do notice that the concept of “participation ratio” generated some disagreements. Some people think that a lower “post-to-thread ratio” represents the ability to resolve problems in a very short time, therefore only a very limited number of discussions are needed. Some people think that a higher “post-to-thread ratio” might be an indicator that the community are on flame, and during a flame a large portion of the posts might be off-topic. Anyway, we agree that when we call this parameter “participation ratio” it somewhat represents our own opinion and it undermines the objective of this report. However, because we do not find a better name to represent this parameter, we will just use it for the time being. (Dear readers, you are more than welcome to contribute a better name for this parameter.)
Figure 4 shows the number of monthly participants of the four projects being discussed. It can be seen that the active participants of CloudStack and OpenStack are much higher than OpenNebula and Eucalyptus. During the past 6 months, the number of active participants for CloudStack, Eucalyptus and OpenStack are growing at various degrees.
It should be noted that although the number of active participants of the CloudStack project is somewhat smaller than that of the OpenStack project, both projects have approximately the same amount of discussions.
Accumulated Community Population refers to the total number of users and developers who have participated in forum or mailing list discussions. (This number does not include those who have registered into discussion forums or mailing lists but have never participated in any open discussions.) These are people who have tested or used a specific product for a while, but not necessary currently an active user.
Figure 5 shows the growth of the accumulated community populations of these 4 projects. Currently OpenStack has the larges accumulated community population, followed by Eucalyptus, CloudStack, and OpenNebule.
The problem is, after years of changes, a long-term (such as 4 to 5 years) accumulated community population might not be a good reference for community activeness. Some of the early members of one community might have switched to other communities (and probably more than once), some of the early community medium (such as mailing lists and forums) might have become EOL’ed. From a community analysis point of view, it might be better to count the accumulated community population of the past 6 to 12 months, while extending the range to the dinosaurs age will make this parameter meaningless. Therefore, starting from CY13-Q3 report we only count the accumulated community population of the past 12 months
Figure 6 shows the monthly population growth of the four projects being discussed. During the past 3 months, the populations of OpenStack and CloudStack are growing at about the same pace.
The populations of Eucalyptus and OpenNebula are growing at relatively slow paces, as compared to that of CloudStack and OpenStack.
Figure 7 is a combination of Figure 4 and Figure 6. The solid lines represent the monthly participants, while the dash lines represent the monthly new members.
For OpenStack and OpenNebula, around 30% of their monthly participants are new members. For CloudStack and Eucalyptus, around 50% of their monthly participants are new members. This indicates OpenStack and OpenNebula communities are more “sticky” than CloudStack and Eucalyptus communities.
Figure 8 shows the total community population, active participants of the past quarter, and active participants of the past month, of the four projects being discussed. It can be seen that
(1) OpenStack has the largest total population, followed by Eucalyptus, CloudStack, and OpenNebula;
(2) OpenStack has the largest active population during the past quarter, followed by CloudStack, Eucalyptus, and OpenNebula;
(3) OpenStack has the largest active population during the past month, followed by CloudStack, Eucalyptus, and OpenNebula.
In our CY12-Q3 report, we invented the concept of “Community Activeness Index”. This magic number should be the combination of the following parameters:
(1) monthly messages, which represents the volume of the discussions;
(2) participation ratio, which represents the average number of answers to a question;
(3) active population of the past quarter, which represents the possibility to get help from community in the long term; and
(4) active population of the past month, which represents the possibility to get help from the community in the short term.
In this analysis, we choose the average values of these parameters as the reference data set, and compare the corresponding parameters of each community with the reference data set. Then we call the sum of the relative values of a community the “community activeness index” of the community. Now we can say the project with the highest “community activeness index” is THE most active project in this area.
As can be seen from Figure 9, OpenStack is currently THE most active project (with obvious advantage), followed by CloudStack, Eucalyptus, and OpenNebula.
The above-mentioned calculation of “Community Activeness Index” uses both the active population of the past quarter and the active population of the past month, which are quite similar in nature. In our CY13-Q3 report, we will use only the active population of the past quarter. Also we will use quarterly messages to replace monthly messages, and use quarterly participation ration (post-to-thread ratio) to replace monthly participation ratio.
We are seeing increasing number of suggestions to analyze the git activities of these open source IaaS projects. We also noticed that all of these four projects use git as the SVM for their source code. Starting from our CY13-Q1 report, we tried to do some basic analysis base on the git log data. It should be noted that for the OpenStack project, the data source includes the Cinder, Glance, Horizon, Keystone, Nova, Quantum and Swift sub-projects hosted on github.com.
In our CY13-Q1 report, we used “git log” to obtain log information. Starting from CY13-Q2, we will use “git log –no-merges” to obtain log information.
Figure 10 shows the monthly number of commit operations for these four projects. Generally speaking, the commit frequencies of the OpenStack and CloudStack are relatively higher. During the past 12 months, both projects maintained a commit frequency of about 600 commits per month. Eucalyptus was also committing frequently, with significant fluctuations from month to month, which seems to be a typical batch-commit behavior. The commit frequency of the OpenNebula project is relatively small, with an average of 200 commits per month.
Figure 11 shows the monthly number of commit operations for the sub-projects of OpenStack. Generally speaking, the commit frequency of the Nova sub-project is about 3 times as high as the other sub-projects. It should be noted that although the commit frequency of these sub-projects are different, but they exhibit similar time-series curves, and their highs and lows occur at the same period of time. This indicates that although these sub-projects are relatively independent, but they work around the same development plan and the same release schedule. This is an indicator that the OpenStack project is well organized in terms of sub-project management.
Figure 12 shows the monthly number of contributors (identified by unique github.com accounts) for these projects. Generally speaking, the number of OpenStack contributors is much higher than the other three projects, and is growing rapidly. The number of CloudStack contributors also exhibits some growth, but the growth is relatively slow. The number of Eucalyptus and OpenNebula contributors is relatively small, and does not exhibit significant growth during the past 12 months.
Figure 13 shows the monthly number of contributors (identified by unique github.com accounts) for the sub-projects of OpenStack. It can be seen that the number of Nova contributors is about 3 times as big as the other sub-projects.
People usually try to identify the institute to which a contributor belongs to by his/her email address. It is true that such method is defect in nature (different institutes have different policies regarding contributing to open source projects, some institutes even encourage their employees to contribute to open source projects with their personal account), but still this parameter can be used to show the contributions of certain institutes to certain open source projects. Figure 14 shows the monthly number of unique institutes (identified by the domain name of the contributor’s email address) contributing to these projects. We can see that the number of contributing institutes for OpenStack is much larger than the other three projects, and is growing rapidly. The number of contributing institutes for CloudStack is also growing, but at a relatively slow pace. The number of contributing institutes to Eucalyptus and OpenStack is relatively small, and does not exhibit any growth during the past 12 months.
Figure 15 shows the monthly number of contributing institutes to the sub-projects of OpenStack. It can be seen that the number of contributing institutes for Nova is about 3 times as big as the other sub-projects.
The following table lists those institutes that make the most contributions to these projects during CY13-Q2, according to the number of commit operations, along with the percentage of their commit operations. It can be seen that both Eucalyptus and OpenNebula are open source projects dominated by single institutes, while CloudStack and OpenStack are open source projects contributed by multiple institutes. For the CloudStack projects, influence from Citrix is quite obvious, over 51% of the commits come from accounts belonging to citrix.com and cloud.com. For the OpenStack project, ivm.com contributed 14 of the commits, redhat.com contributed 12% of the commits, mirantis.com contributed 9% of the commits, and hp.com contributed 5% of the commits.
| CloudStack | Eucalyptus | OpenNebula | OpenStack | ||||
| Domain | % | Domain | % | Domain | % | Domain | % |
| citrix.com | 46 | eucalyptus.com | 50 | opennebula.org | 100 | gmail | 17 |
| apache.org | 23 | gmail.com | 35 | ibm.com | 14 | ||
| gmail.com | 12 | brainfood.com | 13 | redhat.com | 12 | ||
| cloud.com | 5 | fedoraproject.org | 1 | openstack.org | 9 | ||
| schubergphilis.com | 2 | mac.com | 1 | mirantis.com | 8 | ||
| leaseweb.com | 2 | hp.com | 5 | ||||
| tcloudcomputing.com | 2 | nicira.com | 2 | ||||
| zonker.net | 1 | rackspace.com | 2 | ||||
| stratosphere.co.jp | 1 | dmllr.de | 2 | ||||
| betterservers.com | 1 | solidfire.com | 2 | ||||
The following table lists those institutes that make the most contributions to the sub-projects of OpenStack during CY13-Q2, along with the percentage of their commit operations.
| Cinder | Glance | Horizon | Keystone | ||||
| Domain | % | Domain | % | Domain | % | Domain | % |
| solidfire.com | 13 | gmail.com | 22 | redhat.com | 28 | gmail.com | 28 |
| hp.com | 10 | hp.com | 16 | gmail.com | 23 | ibm.com | 25 |
| redhat.com | 9 | redhat.com | 13 | hp.com | 13 | redhat.com | 18 |
| mirantis.com | 8 | rackspace.com | 12 | mirantis.com | 10 | openstack.org | 4 |
| ibm.com | 8 | openstack.org | 5 | metacloud.com | 4 | 99cloud.net | 3 |
| gmail.com | 7 | ibm.com | 5 | dmllr.de | 2 | inaugust.com | 2 |
| openstack.org | 7 | dmllr.de | 3 | inaugust.com | 2 | anarkystic.com | 2 |
| dmllr.de | 7 | markwash.net | 3 | me.com | 2 | hp.com | 1 |
| canonical.com | 2 | inktank.com | 3 | canonical.com | 2 | stillhq.com | 1 |
| unitedstack.com | 2 | inaugust.com | 2 | griddynamics.com | 1 | enovance.com | 1 |
| Nova | Quantum | Swift | |||||
| Domain | % | Domain | % | Domain | % | ||
| ibm.com | 22 | openstack.org | 27 | gmail.com | 25 | ||
| gmail.com | 17 | mirantis.com | 12 | swiftstack.com | 16 | ||
| redhat.com | 8 | gmail.com | 12 | brim.net | 12 | ||
| mirantis.com | 7 | redhat.com | 10 | redhat.com | 10 | ||
| openstack.org | 4 | nicira.com | 9 | ibm.com | 7 | ||
| hp.com | 4 | ibm.com | 4 | mirantis.com | 6 | ||
| pavlovic.me | 3 | cisco.com | 4 | not.mn | 3 | ||
| rackspace.com | 3 | da.jp.nec.com | 2 | kotori.zaitcev.us | 3 | ||
| codestud.com | 3 | bigswitch.com | 2 | enovance.com | 3 | ||
| stillhq.com | 3 | dmllr.de | 2 | hp.com | 1 | ————————- | — |
For your convenience, a PDF version of this presentation can be downloaded from here. Please kindly keep the author information if you want to redistribute the content.
Themocracy
[…] 本文同时发布了一个英文版本，可以参见CY13-Q2 Community Analysis — OpenStack vs OpenNebula vs Eucalyptus vs CloudStack这个帖子。 […]
最后一个代码机构提交很不准，应该是没把.mailmap算进去
[…] health and activity is compiled by an employee of Eucalyptus, named Qy John, out of China. His graphs also show the scales have tipped to […]
[…] article is an update version of my previous article CY13-Q2 Community Analysis — OpenStack vs OpenNebula vs Eucalyptus vs CloudStack. Readers who are intested in further discussions please contact me via email at the above-mentioned […]
[…] article is an update version of my previous article CY13-Q2 Community Analysis — OpenStack vs OpenNebula vs Eucalyptus vs CloudStack. Readers who are intested in further discussions please contact me via email at the above-mentioned […]
[…] and activity is compiled by an employee of Eucalyptus, named Qy John, out of China. His graphs also show the scales have tipped to […]
Do you have contents in the home that require insurance coverage?