Evaluating modelling tools for Software Ecosystems

Introduction

As described by recent research, software organizations can gain a lot from being aware of their Software Ecosystem (SECO) and their position within the ecosystem. To gain that knowledge a thorough understanding of its SECO should be obtained. Especially in large organizations this can be difficult, because of the many actors the organization is involved with. Therefore using a good modelling tool to get an overview of the ecosystem is necessary.

There are currently several network modelling tools available, but one specially designed for modelling SECOs does not exist. Although using general network modelling tools is a possibility, SECO modelling requires a different functionality. For example the emphasis within SECOs is with flows between actors. Therefore, being able to model these flows is necessary whereas in, for example, social network analysis this is not that relevant.

The aim of this research was to identify the specific functionalities required for SECO modelling and find the modelling tool that has implemented these functionalities in the best way possible.

Method

The research consisted of three phases. First the tools to be analyzed were selected. A first list was created by doing literature research and talking to experts. That list was then further narrowed down to a second list. That process is shown in the figure below. The tools are shown that have been eliminated together with the reasons why.

Figure 1

Figure 1: Tool selection

The second phase consisted of doing literature research in as well software ecosystems as network analysis. A thorough understanding of these subjects resulted in a clear list of the important SECO characteristics and the proper network analysis techniques to be used to model these characteristics. That resulted in a clear list of functionalities that the modelling tool should possess in order to be suitable for modelling SECOs.

The feature list:

  • General
    • SECO size calculation
    • Role count calculation
    • Multiple views
  • Nodes
    • Color
    • Size
    • Visibility
    • Labeling
    • Shape
    • Range
    • Customized properties
    • Connecting nodes based on their properties
  • Edges
    • Weight
    • Color
    • Label
    • Multiple labels
    • Different kinds of labels
    • Shape
    • Visibility
    • Direction
    • Customized properties
  • Clustering
    • Clusters/Containers
    • Overlapping clusters
    • Subgroups
  • Scope
    • Zooming
    • Flow paths
    • Flowpaths: Choice
  • Statistics
    • Centrality
    • Density
    • Distance between nodes
    • Connectedness
    • Hierarchy
    • Egocentric networks
    • Structural equivalence
    • Degree

The third phase was the actual tool analysis. A data set consisting of 398 dutch software vendors and their connections to their partners and to each other was used to analyze the tool. For each tool individually the data set was used to test all the desired functionalities. While doing that quality measures such as reliability, efficiency and usability were judged.

Results

As explained before the three tools that were thoroughly analyzed were Pajek, Gephi and Cytoscape. Before showing the results a short explanation of each tool will be provided.

Pajek is a freeware tool created in Slovenia and it is designed for general network analysis. According to the developers Pajek’s strenghts are abstraction, visualization and efficient data analysis. Gephi is an open source tool which also focusses on general network analysis. Usage analysis showed that Gephi is currently the most popular tool for network analysis. Cytoscape was originally designed for analyzing molecular interaction networks, but it has evolved to a more general open source tool.

Pajek

For visualization Pajek distinguishes itsself by using several different data types for their analysis instead of using one network file that stores all the information. By using different files the tool is flexible, because multiple different files for every aspect can be loaded. On the other hand it decreases efficiency, because the different files have to be loaded and saved separately.

Functionality of Pajek is quite good, 27 out of the 35 features were implemented in a sufficient way. However, Pajek lacks some quality. The usability is old fashioned with ugly and unclear visualizations and efficiency could also be better whereas Pajek uses a separate visualization window which forces users to continuously switch between windows in order to make changes to the visualization. The figures below show the main window of Pajek with the general user interface and the different data types as well as a screenshot from the data set viualized in Pajek.

Figure 2

Figure 2: Main window Pajek     

Figure 3

Figure 3: Screenshot Pajek

The biggest advantages advantages and disadvantages for using Pajek are summarised in the table below.

Table 1

Table 1: Results Pajek Summarised

Gephi

Gephi has a pleasant appearance with a modern graphical user interface which is shown in figure 3.9. It is clearly noticeable that effort has been invested in giving the tool a good look. Furthermore, they focused on the navigation by making sure most features are reachable in one click ensuring options to be at hand without having to go through many menus.

The visualized graphs in Gephi have a modern, smooth appearance. Differences in weight and node sizes are subtle, but easy noticeable and the colors used are carefully chosen. With the toolbar at the bottom of the graph window most common settings like enabling labels and changing sizes of components can easily be modified. The graphs are automatically visualized when files are being opened. Furthermore, the visualization automatically adapts when changing settings or running algorithms. These algorithms are executed live on screen showing every step taken in the algorithm. Besides showing what the algorithm is doing it is also possible to pause or stop the algorithm at the point the user is satisfied with the result. One of the visualizations of the data set is shown in the figure below.

Figure 4

Figure 4: Visualization Gephi

Gephi has implemented 28 out of the 35 features of the feature list. Most important feature that is missing is that it is not possible to use node attributes in calculations. The results are summarised in the table below.

Table 2

Table 2: Summarised results Gephi

Cytoscape

It is clear Cytoscape has put effort in optimizing the visualization. The networks are automatically visualized when a file is loaded. Several layout algorithms are available to nicely position the nodes, which together with the many style settings that can be altered ensures a nice representation of the network.

Cytoscape does a remarkable job in visualizing edges. Even when the layout is far from ordered it is often still good visible what nodes have most connections and where they are going. That is shown in the figure below which shows a visuzalization in which the nodes are very close together and the edges all go through one another, still it is very clear what nodes have most connections and most edges can easily be followed to its destination.

Figure 5

Figure 5: Screenshot Cytoscape

Furthermore, Cytoscape can visualize multiple networks at the same time. Cytoscape allows users to select nodes and create new networks with one press of a button. These networks can then be visualized right next to the original network instead of replacing the original network. That makes it for example possible to keep parts of a layout as a second network before using different layouts. In that way it is easier to combine layouts and, hence, get more out of the visualization. In SECO that could, for example, be used to compare two actors and its connections or even to compare two or more different SECOs.

Unfortunately Cytoscape lacks some functionality, only 24 of the desired features are implemented. Most important features that are missing are good edge labels and the possibility to alter/use single nodes. Some other pros and cons are shown in the table below, which summarises the results for Cytoscape.

Table 3

Table 3: Summarised results Cytoscape

Conclusion

Overall the conclusion is clear: Pajek lacks quality, Cytoscape lacks functionality and, therefore, Gephi is the tool best suitable for analyzing SECOs. On the other hand the conclusion can also be drawn that Gephi is not yet totally fit for the job and more research has to be done, especially on usability and expandability, to ensure Gephi will be totally suitable for modelling SECOs.

When that is done Gephi can be used in general SECO research to help in data analysis, SECO governance and comparing SECOs. Doing that, hopefully, new insights are gained in how to manage a SECO and how software organizations can create their ideal SECO around them.

Advertisements

The Reality of an Associate Model – Comparing Partner Activity in the Eclipse Ecosystem

This post consist of the main findings of the paper; ‘The reality of an associate model – comparing partner activity in the Eclipse Ecosystem’ by Aarnoutse, Renes and Snijders (2014). We have researched the partnership model of Eclipse and the activity of the different types of partners. This is done on platform as well as Ecosystem level.

Eclipse is an open source community in which individuals and organizations can collaborate. The projects of Eclipse “are focused on building an open development platform comprised of extensible frameworks, tools and runtimes for building, deploying and managing software across the lifecycle”.

We identified that Eclipse has 205 formal partners, of which 6 are sponsors and 202 are members of Eclipse. Google, IBM and Oracle are sponsor as well as member of Eclipse. For the exact distribution see the table below.

These partners do have differences in how they contribute to the Eclipse platform and Ecosystem. We have analyzed several items that represent activity. For example, the average amount of projects a partner contributes to, average amount of active committers of a company, the average amounts of commits and average amount of commits per committer. In the table below, there is a part of this comparison of activity between the members of Eclipse in the platform and in the Ecosystem.

EclipseActivity

The pie charts below show the total amount of commits and committers for the Eclipse platform as well as the ecosystem. Based on the availability of the data, we give an overview of the distribution of the different membership types as well as the people who code for the Eclipse Foundation of the platform. In addition, for the ecosystem we show the distribution of the membership types accompanied by the people who did code for a company which isn’t member of Eclipse.

Eclipse Distribution

Based on this research we conclude that a higher level of membership is related to more activity. The strategic and associate members both significantly relate to a higher amount of activity, while this is not the case with the solution members. Companies without a membership are more active than the associate members, which suggests that Eclipse should improve their partnership model by motivating associate members and incorporating active non-member companies.

If you are interested in reading more of the analysis, additional conclusions or implications of our research, please contact one of the authors; {f.Aarnoutse, C.Renes, R.Snijders, S.Jansen}@uu.nl.

ICSOB 2013: Can’t Wait

The 4th International Conference on Software Business will be held on June 11-14, 2013 at the University of Potsdam. ICSOB 2013 addresses researchers and practitioners, who are concerned with software business in different ways as well as the start-up community, which is increasingly focusing on mobile and social software. Check out their site here: www.icsob.org. See you all there!

header_1

ICSOB 2012: MIT in Boston, MA, June 18-20

Hey guys, please check out the International Conference on Software Business (ICSOB). This year it’ll be organized in Boston. Deadline is 21st of January. Please check out the Call for Papers or just go to www.icsob.org.

What do open source business models of software companies tell us about their ecosystem?

The aim of this research will be to analyze the business models attached to open source software in different companies and communities. This includes software produced as an open-source project by a company that has a clear view of what needs should the product serve (an open source ERP system as opposed to the java platform), a one that people can freely manipulate, use, modify etc., and has a specific business model made up by the company in order to make profit. Furthermore an investigation of open-source software platforms (e.g. android) will be conducted. More specifically the attention will be towards finding how companies or people use these platforms to make profit, in other words how they create different business models to the open source software so a profit can be made. The research will be oriented towards finding common characteristics of the business models of open source software companies. Another important pursue of the research will be finding what are the internal and external factors of the ecosystem that drive companies towards choosing the models they have.

The research will be qualitative in its essence as most of the data will be collected from application stores companies, and official websites of open source software companies and projects. We use triangulation for the gathering and analyzing of the data. This includes a combination of structured, human and observational techniques for recording key factors when browsing through the data online. Furthermore the observational technique will be direct as in the analysis stage e-mails will be sent to and interviews will be attempted with authors of open source applications where information online is not enough to determine how the business model works or is constructed. The research will also include a couple of case studies that show interesting business models relations between different open source applications.

The analysis spans over companies of different sizes, that satisfy the criteria of producing open source software, have a structured business model and have available information online, from which it can be determined the type and characteristics of the business model.

The assessment criteria of identifying the different business models are taken from Rajala et al.’s framework for analyzing software business models. The four dimensions (categories) of analysis are product strategy, revenue logic, distribution strategy and services and implementation. This research evaluates how components in each of the four categories shape relationships in the ecosystem of open source software.

References:

  1. Rajala, R., Rossi, M., Tuunainen, V.K., (2003) A framework for analyzing software business models, Proc. 11th European Conf. Information Systems (ECIS 03)

Revitalizing communities of practice through software release management: a survey on the Android community

Communities of practice in software ecosystems are like any other community of practice which we define as follows:

A group of people who share a concern, a set of problems, or a passion about a topic, and who enlarge their knowledge and expertise in this area by interacting on an ongoing basis.

Within the Android Ecosystem large communities of developers play an important role in the health of the Ecosystem. We propose to research the effects of new platform/SDK releases on the developer community. We expect that every major Android release has a revitalizing impact on the health of the community and thus on the ecosystem.

The goal of this study is to analyze the dynamics of an online software community in relationship with software releases from a software ecosystems perspective.

The main research question in this survey is: “How do software releases impact developers in an open source software ecosystem?”

In order to answer this question there needs to be a relationship between a software release (seen as an event) and the activity in the community. . A case study with statistical analysis should give insight in the strength of this relationship. This community should offer information about the members and their activities in a significant time range so that we have several releases and release-intervals to measure.

Data about software releases and community members and activity should be available for research purposes. At the time of this research the open source Android system seems to have a community capable of delivering the information needed to perform this research. The Android system is an open source mobile operating systems with eight major releases and dozens of small releases in the past four years. A notably large community is the “Android Community” at androidcommunity.com.  Some numbers on the community: Threads 37,234 Posts 333,370 Members 117,155 Active Members 8,501.

We use a custom-built tool to mine data from this community.

After the data is stored in a database an adjacency matrix is constructed that is required for the social network analysis tool UCINET. A relationship between actors is established when users post a message in the same thread.

Using social network analysis on a date interval on the dataset it is possible to create a representation of a before and after scenario of the community. Mathematical analysis, social network heuristic analysis, and visual analysis of social networks is used to analyze the effect of a software release.

Ideally we would graph the activity in the community and see peeks which pinpoint the Android platform releases.

Research proposal Dataminers

Research questions & Research method Data miners

 Problem statement

Leading web shops (e.g. Bol.com, Pixmania and Amazon) are increasingly integrating the products of third-party sellers in to their own web shop [1] [2]. This requires the transfer of information between two or more web shops. This way, web shops can present competitive offers on their website. By doing this, they are opening up the potential to serve the customer better by selling items outside their own product portfolio. By opening up their website the  leading web shops are transforming from a standard webshop into an open sellers platform. It is not clear if the collaborative webshops are ready for this trend or are able to act on it.

Research questions
Main question:

  • What is the readiness of collaborative webshops towards open selling platforms?

Sub questions:

  1. What information needs to be shared between an open sellers platform and the collaborative webshop?
  2. In what way were the required IS/IT  and business processes of the collaborative webshop altered?
  3. How do the capabilities of webshops compare to the requirements of a collaborative webshop?

Research method

two research methods were used. A case study is performed with a CW to determine the requirements and impact. And a survey is performed by sending questionnaires to web shops that haven’t engaged in this trend yet, in order to determine if they are ready for this trend.

Based on these two researches and by answering the three sub questions, we are able to determine the readiness of collaborative webshops towards open sellers platforms.

Sub question Research approach
What information needs to be shared between an Open Sellers Platform and the collabotative webshop?
  1. Arrange an interview with a collaborative webshop;
  2. Perform a literature research.
In what way were the IS/IT requirements and business processes of the collabotative webshop altered?
  1. Arrange an interview with a collaborative webshop;
  2. Perform a literature research.
How do the capabilities of webshop compare to the requirements of a collabotative webshop?
  1. Process the interview results;
  2. Process the literature study;
  3. Send questionnaire to webshops;
  4. Store and aggregate data;
  5. Perform statistical analysis on acquired data.
  6. Describe conclusion based on acquired insights.

References

  1. (2011, May). Erwin Boogert: Specialisten e-commerce reageren op Bol.com Plaza [Online]. Available: http://www.emerce.nl/nieuws/specialisten-ecommerce-reagere-op-bol-com-plaza
  2. (2011, May). Erwin Boogert: Bol.com opent site voor de concurrentie  [Online]. Available: http://www.emerce.nl/nieuws/bol-com-opent-site-voor-de-concurrentie