Eliminating waste in software projects: Effective knowledge management by using web based collaboration technology

Contents

1 Introduction: Inefficient Knowledge Management in Corporate Software Projects vs. Efficient Knowledge Management in the Internet
1.1 Motivation
1.2 Problem Definition
1.3 Aim of this Book
1.4 Research Methods
1.5 Definitions

2 Analysis: Lean Software Development, Lean Knowledge Management, and Enterprise 2.0
2.1 Lean Software Development
2.1.1. Lean Software Development Principles
2.1.2. A Lean Software Development Method: Scrum
2.2 Lean Knowledge Management
2.2.1. Empirical Process Control Required for Successful Knowledge Management
2.2.2. Lean Knowledge Management Principles
2.3 Web 2.0 Based Knowledge Management for Companies: The Enterprise 2.0 Concept
2.3.1. Web 2.0: The Race for Collective Knowledge
2.3.2. Web 2.0: Dealing with Fast Changing Information
2.3.3. Web 2.0: Keeping Users Attracted
2.3.4. Enterprise 2.0: Web 2.0 in the Intranet

3 Results: Lean Knowledge Management in a Lean Software Project
3.1 The Benefit of Enterprise 2.0 and Lean Knowledge Management
3.2 Putting it Together: An Architecture for Web Based Knowledge Management

4 Conclusion

5 Outlook

List of Abrreviations

illustration not visible in this excerpt

List of tables

Table 1: Lean Knowledge Management Principles

Table 2: Comparison of Enterprise 2.0 with Traditional Knowledge Management in Software Projects

List of figures

Figure 1: The Waterfall Model

Figure 2: A Lean Software Development Model: Scrum

Figure 3: Scrum Process Overview

Figure 4: Scrum Burndown Chart

Figure 5: StumbleUpon Toolbar

Figure 6: A Family Tree Created with Geni

Figure 7: A Tag Cloud

Figure 8: The Memeorandum News Aggregation Service

Figure 9: The News Reader Google Reader

Figure 10: A Web Based Pull System

Figure 11: Components of a Web Based Knowledge Management System in a Software Project For an executive summary, please read section 4. Conclusion starting on page

‘In colloquial usage, a fractal is a shape that is recursively constructed or self-similar, that is, a shape that appears similar at all scales of magnification and is therefore often referred to as "infinitely complex."’^[1]

1 Introduction: Inefficient Knowledge Management in Corporate Software Projects vs. Efficient Knowledge Management in the Internet

1.1 Motivation

Today, in a typical software project, people do not document at all or invest a large amount of time to create documentation, which at the end proves to be not useful or undiscoverable for others.^[2]

The goal of documentation is to conserve knowledge and make this knowledge accessible for others, thereby easing collaboration. However, this goal is rarely achieved. The whole process of producing documentation and searching for valuable information is highly ineffective and means a lot of wasted time.

Often, in a shorthand approach, Word, Exel and Powerpoint files are used for documentation. Among these documents, information is not interconnected. As a consequence relevant information is hard to find. Full text searches take a long time and deliver poor results. Imagine searching all documents of a file server of a medium cooperation using the Windows Explorer search functionality. The search will probably run several hours and hardly deliver any useful results. If any information is found, it is hard to tell if the document is up to date, who changed the document last and what was changed. There is no automated mechanism to inform interested people about updates on any document or information.

Another approach is using proprietary knowledge management technology and collaboration software, like knowledge repositories. In the late nineties companies started to implement knowledge management strategies with heavy use of technology.^[3] The problem herein is that people have to be trained to use the knowledge management technology, learn and understand the underlying principles and additionally the handling of specific software. If the knowledge management technology is complex, it imposes an entry barrier. Using knowledge management technology will then be burden for a software development team. Since the pressure in a software development is high, it is unlikely that a team will be willing to invest much time for learning the technology and hence the pay off will never be realized. Furthermore if the basic principles and processes of a chosen technological approach turn out to be inadequate for a specific situation there is often no chance to adapt the knowledge management technology quickly. This finally can lead to a situation that the technology is not used at all, because at the end it is easier for workers to simply ask someone else they know.^[4]

1.2 Problem Definition

A lean management principle is to eliminate waste.^[5] Stopping to produce documentation in an ineffective way is desirable, in order to be competitive. Agile or lean approaches stress to focus on the value creating task, which in case of software development is writing source code, not documentation.^[6] Knowledge management in agile projects is mostly handled through face to face communication. Because of this it is said that agile software projects and documentation or knowledge management technology are incompatible or contradicting. Agile approaches see documentation as a non value creating burden and state that most documentation will never be read. There is an agile principle called TAGRI: They aren’t gonna read it.^[7]

On the other hand, the knowledge of a company is one of its most important intangible assets. And there are various situations where documentation is valuable in order to transfer, share, or conserve knowledge and information. For example, imagine global projects, where developers work in different time zones. Face to face contacts are not easily possible, so there has to be another mechanism for the effective sharing of knowledge and information. Some software applications tend to run for centuries. Just think of the millions of lines of code written in COBOL which were written sometimes twenty years ago and are still operating today in core backend systems of major financial services companies.^[8] Without backup documentation it is hard to conserve the knowledge of those who initially wrote the core components of such systems, not mentioning the generations of developers who modified the system during the years.

And sometimes the customer simply demands that the software is documented. So if there are situations, where documentation is needed in order to transfer knowledge, which tools should be used in order to be more – and not less – effective? What should be documented, what not? Which knowledge should be transferred without documentation? In which cases does documentation deliver no value?

Surprisingly enough, the Internet, which deals with far more information than any company, – without any strategic initiative or predefined processes – does organize information and even knowledge very effectively. Compared to corporate networks users assert that it is easier to find something on the Internet^[9]: users access their favorite search engine and enter some keywords. In a lot of cases they will at least get some good starting points for further research. When searching for common topics, they will probably find a gem within the first ten or twenty search results. Sometimes of course, when they are looking for something very specific, they might “feel like the SETI researchers at the University of California, Berkeley, searching through an unstoppable flood of meaningless information from outer space for signs of intelligent life”^[10]. However the situation in the Internet is far better than the situation in most companies.

In spring 2006 Prof. Andrew P. McAfee coined the term Enterprise 2.0 in his article “Enterprise 2.0: The Dawn of Emergent Collaboration”.^[11] McAfee points out that the typical knowledge management problems companies face can be addressed by employing the Web 2.0 technology and concepts widely and successfully used in the Internet. He described the SLATES principles, including search engines, links, authoring (wikis, blogs), tags (labeling instead of categorizing information), extensions (considering information as useful based on the behavior of other users, like amazon.com's recommendations) and signals (being informed if something changes instead of having to check all information resources).^[12] McAfee successfully started a lively discussion about this topic.

However, the question if and how these principles can be applied to a software development project is not answered.

1.3 Aim of this Book

This book will show if and how web technology can be used in order to support knowledge management, documentation and collaboration when applying the lean software development method Scrum^[13] and applying Mary and Tom Poppendieck’s^[14] lean software development principles.

Understanding the key concepts of lean software development is a prerequisite for defining workable knowledge management approach in such an environment. Hence, the lean software development principles and the lean software development method Scrum are analyzed and presented in section 2.1.

Second, considering the principles of lean management in the area of software development, an approach for lean knowledge management is derived. Section 2.2 presents eight lean knowledge management principles. These principles point out what can be done by a manager to control availability, distribution and transfer of knowledge in the most efficient way.

Third, section 2.3 explains the Web 2.0 principles and technology. These principles are the foundation for successful knowledge management technology in the Internet. It is shown what the prerequisites are to take Web 2.0 to the Intranet and create Enterprise 2.0 applications.

A traditional approach of documentation and knowledge management will be compared with the suggested lean, web based approach in section 3.1. The impact of both approaches on the fulfillment of the lean knowledge management principles is illustrated, emphasizing the benefit of lean knowledge management and Enterprise 2.0. Section 3.2 presents the basic setup of web based tools for effective knowledge management in software projects.

Section 4 will draw a conclusion on the subject and section 5 will suggest further development and research in the field of web based knowledge management and collaboration technology.

1.4 Research Methods

- Theoretical analysis of lean software development and the method Scrum.
- Theoretical analysis of the key collaboration and knowledge management needs in a lean or agile software project.
- Theoretical analysis of lean principles in order to define lean knowledge management principles.
- Theoretical analysis of the Web 2.0 principles and cooperation forms in the Internet.
- Comparison of the traditional approach and the web technology based approach to knowledge management.

1.5 Definitions

According to Gilbert Ryle there are two kinds of knowledge: knowing how and knowing that.^[15] Knowing how means that someone has the knowledge to do certain things, like someone knows how to play an instrument. Knowing that includes knowledge about something. Someone might know a lot about music and successful operas. However, this knowledge about music does not enable him necessarily to compose a successful opera.

The primary goal of knowledge management is to transfer knowledge from one human being to another, in order to enable the knowledge requester (the one who needs the knowledge) to solve a specific problem or handle a specific task. Of course, knowledge transfer is done most easily when two people communicate directly, ideally face to face, and the knowledge owner (somebody how has the required knowledge) teaches or tells the knowledge requester whatever is needed. But there are various situations when such an interaction is not possible or not efficient, e.g. when these two people live in different locations, when there are far more knowledge requesters a knowledge owner can handle, or when the knowledge owner is not able to teach somebody else.

Furthermore, the knowledge transfer ideally happens whenever the knowledge requester requires it. This imposes another hurdle, because the knowledge owner might not be available at the time or place where the knowledge is needed. Therefore, in situations where a direct interaction is not possible, knowledge needs to be transferred through some kind of media, for example other humans, documents, videos, expert systems, web sites, or books.

Knowledge about something can be typically gathered easily through consuming information stored on media. Learning how to do something always requires practice and feedback. Hence this kind of knowledge can not be easily transferred trough media.

Since easing (or in some cases preventing) a successful transfer of knowledge is the goal of knowledge management, the term knowledge management will be used according to the following definition:

Knowledge management is controlling the transfer, distribution, and availability of knowledge.

Knowledge management technology then is any technology (or organizational approach) that supports knowledge management, including books, libraries, the Internet, videos, audio CDs, repositories, newspapers, content management systems and many things more.

2 Analysis: Lean Software Development, Lean Knowledge Management, and Enterprise 2.0

2.1 Lean Software Development

2.1.1. Lean Software Development Principles

Lean software development is based on the principles of lean production, which where developed by the Toyota production engineer Taiichi Ohno in the 1950s.^[16] Ohno’s lean production principles had a significant impact on the automobile industry. Compared to mass production, lean production requires only about half of the engineering hours, manufacturing space, human effort in the factory, investment tools, and time to develop new products.^[17] But even though the outstanding results of lean production enabled Japanese auto manufactures to successfully gain world market share, it took European and American auto manufactures decades to catch up and adapt the principles of lean production successfully.^[18] History seems to repeat itself in the area of software development.

Software development approaches that are still en vogue today are based on the waterfall approach, first depicted and at the same time criticized in an article written by Winston Royce in 1970.^[19] Winston Royce suggested an iterative approach of software development but his depiction of the sequential model of software development, looking like a waterfall (Figure 1) had already made its way into the management theory of software development. Today, over thirty years later, lean software development approaches are still considered as “high-risk […] endeavor”, even though they deliver considerably better results.^[20]

illustration not visible in this excerpt

Figure 1: The Waterfall Model

Source: Royce, W (1970), p.329

Mass production like approaches like the waterfall model, Rational Unified Process (RUP)^[21], PMBOK^[22] and ITIL^[23] aim at creating reliable, describable, and repeatable processes for software development. Like Henry Ford did for his factories and workers, these concepts define detailed procedures for software engineers and project managers. It is assumed that adherence to these processes will eventually lead to high quality software. Managers have the responsibility to assure that everyone follows these predefined processes.

In a software project, following these procedures requires a lot of complex planning, work and often documentation, distracting developers from their value creating duties, which is writing source code. In such a complex framework any change occurring after the planning has been done, leads to extensive change control procedures and high costs.

But modifications in a software project are inevitable, let it be that technical problems are encountered or customer needs vary during the development project.

However the most inefficient fact is that the developers with their detailed knowledge have no chance to change the process even if they know that following these processes is decreasing their productivity..

Because of their bureaucracy and their inability to respond to changes adequately these approaches are not competitive from a lean perspective.^[24]

The lean and agile approach to software development is different. Lean software development focuses on the customer, writing source code, and the developers, not on processes.

Software development is not seen as a sequential process, but as a highly iterative proceeding (Figure 2), meaning that one single project has various development cycles and delivers numerous prototypes.

Prototypes and regular face to face interaction with and feedback from the customer is valued, in contrast to specifications and sign-off documents. The customer has a continuous influence on the priorities of the features to be developed during the whole process, not only during the requirement analysis phase. The developers, and not management, commit on a defined set of features which they can deliver during the next development cycle.

Furthermore, lean software development accepts that software development is too complex to plan and define every single step needed in advance. This concept concentrates on optimizing the output, not the processes and process adherence. It relies on the self organizing abilities of a software development team.^[25]

illustration not visible in this excerpt

Figure 2: A Lean Software Development Model: Scrum

Source: Schwaber, K. p.10

Mary and Tom Poppendieck developed seven lean software development principles which are presented here and will illustrate the lean approach.^[26]

1. Eliminate waste. Everything that does not create value for the customer is regarded as waste. Identifying what is waste can be done by imagining the situation of a troubled project which has only very limited time left until delivery. All that is not done in such a situation but which would be done in a regular project is likely to be waste. Below, examples for waste are listed:

Partially done work is regarded as waste. It ties up resources and is a source for complications (e.g. untested code, unfinished functionality leads to disappointed customers and service calls).

Extra processes, like sign off procedures and written change approval procedures are regarded as waste. From a lean perspective a sign off procedure does not add any value to the product. Rather the feedback of the customer on the product or prototype leads to improvements.

Task switching is regarded as waste, because extra time is needed to think about what has to be done every time a task is switched.

Moving is regarded as waste, because time wasted when people have to walk or travel long distances to meet a person they regularly need to work with. When artifacts are moved waste is also likely to be generated. For example when a design document moves from a systems architect to a developer. It is likely that there is something the developer needs to know which the architect did not think about. And some tacit knowledge of the architect will not be transferable through a document.

Defects are regarded as waste. Any failure in a system is will generate additional work. Finally, unwanted extra features and waiting are regarded as waste, too.^[27]

2. Amplify learning. Learning is crucial to successful software development. Software development is not a repetitive task.

Every customer requirement is unique and so is the software needed to address the requirement. Solutions for common problems are quickly encapsulated in software libraries and can be reused. Therefore a software engineer rarely writes code for solving a specific problem twice. So a software developer constantly needs to learn about the customer needs as well as the capabilities of his tools, because he has to use them for different jobs all the time. Learning, though, is not a sequential task either. Effective learning needs feedback, so an iterative approach is needed.

This means that a software developer, who is able to communicate with the customer and present prototypes of the work done often, will deliver considerably better software. Additionally, the developer needs feedback of other developers on the code he has written, which requires interaction and collaboration. Easy access to solutions for similar problems will have a positive effect on productivity and efficiency.^[28]

3. Decide as late as possible. Deciding as late as possible is helpful, because the later something is decided the more informed the decision will be. On the on hand there will be more information about future developments, like which technologies will be available. On the other hand it is easier to deal with changes when the final decision has not been made.

Delaying changes as far as possible allows concurrent development. For example, the developer will start coding, even if customer requirements are still incomplete. For concurrent development to be successful, strong interaction and collaboration among developers and developers and customers is necessary. All information (e.g. design documents, requirement specifications), even if incomplete should be shared immediately. This way experienced workers will be able to anticipate what solutions are needed, if not in every detail. But the basic building blocks will be clear and development can start quickly.^[29]

4. Deliver as fast as possible. Fast delivery will satisfy the customer. It will enable him to generate the expected business value quickly which means more business flexibility.

Long development cycles have the risk that business requirements change during the development and the software has to be changed even before it actually went into production and delivered any value.

Fast delivery also amplifies feedback. The faster the software is delivered the sooner feedback is available. This enables the customer and the development team to optimize the software solution quickly, so quality will be enhanced.

One way to speed up delivery are pull systems or signaling systems, which are used with just-in-time delivery. Instead of trying to plan (sometimes months or even years) in advance, what developer will be assigned to which task, developers will assign themselves to tasks as soon as they finished previously done works. This way, development resources are used in an optimal way, leading to higher efficiency.^[30]

5. Empower the team. Lean software development empowers those to make the decisions who know most about the details.

It is not possible to make quick decisions, if central authorities would have to be informed about the problem and asked for a decision.

In order to benefit from the learning of the team members, the team has to be able to make its own decisions. Enabling the team and pushing the responsibility down the hierarchy as far as possible leads to increased motivation, quality and performance.^[31]

6. Build integrity in. Perceived integrity is when the customer feels that all his needs are addressed correctly by the system and the components of the system are working smoothly together. No unnecessary hurdles are contained; no extra clicks are required; everything is focused on getting the job done effectively.

Conceptual integrity addresses the parts of the system from the developer’s perspective. Everything is useful, all parts of the system interact seamless, simple, and straight forward.

Both kinds of integrity are needed. Perceived integrity satisfies the customer and lets him focus on his business. Conceptual integrity is needed to react quickly when new features have to be added to the system. To keep conceptual as well as perceived integrity requires discipline, effective communication, expertise and wise leadership.^[32]

7. See the whole. The last principle for lean software development demands, that with every change and optimization the effect on the whole system is considered. It is not useful to spend time on the optimization of specific parts of a system, when they have no or little effect on the whole system. Optimization and changes should be done where they have the biggest effect.^[33]

[...]

^[1] Unknown Author (2006), http://en.wikipedia.org/wiki/Fractal, accessed on 06.11.2006

^[2] Cf. Davenport, T.H. (2005), p. 109

^[3] Cf. ibid., p. 90-91

^[4] Cf. Downing, J. R. (2004)

^[5] Cf. Poppendieck M. and Poppendieck T. (2003), p. xxv

^[6] Cf. ibid., p. 1

^[7] Cf. Ambler, S.W. (2006)

^[8] Cf. Mitchell, R. (2006)

^[9] Cf. McAfee, A.P. (2006), p. 27

^[10] Brown, J.S and Duguid, P. (2000), p. 12

^[11] Cf. McAfee, A.P. (2006)

^[12] Cf. ibid., p. 21-28

^[13] Schwaber, K. (2004)

^[14] Poppendieck M. (2000)

^[15] Cf. Ryle, G. (1949), p. 27-32

^[16] Cf. Womack, J.P, Jones, D.T. and Roos, D. (1990), p. 48-49

^[17] Cf. ibid., inner cover page

^[18] Cf. ibid., p. 47

^[19] Royce, W. (1970)

^[20] Cf. Cook, J., & Semouchtchak, V. (2004)

^[21] Kruchten, P. (1998)

^[22] Institute of Electrical and Electronics Engineers, Inc. (2004)

^[23] Office of Governement Commerce (2000)

^[24] Cf. Poppendieck M. and Poppendieck T. (2003), p. xvii-xxiv

^[25] Cf. Schwaber, K. (2005), p. 1-15

^[26] Cf. Poppendieck M. and Poppendieck T. (2003)

^[27] Cf. ibid., p. 1-8

^[28] Cf. ibid., p. 15-47

^[29] Cf. ibid., p.47-69

^[30] Cf. ibid., p.69-95

^[31] Cf. ibid., p.96-125

^[32] Cf. ibid., p. 125-153

^[33] Cf. ibid., p. 153-178