Frequently Forgotten Fundamental Facts about Software Engineering

Frequently Forgotten Fundamental Facts about Software Engineering

Robert L. Glass

This month's column is simply a collection of what I consider to befacts—truths, if you will—about software engineering. I'm presentingthis software engineering laundry list because far too many people whocall themselves software engineers, or computer scientists, orprogrammers, or whatever nom du jour you prefer, either aren't familiarwith these facts or have forgotten them.

I don't expect you to agree with all these facts; some of them mighteven upset you. Great! Then we can begin a dialog about which factsreally are facts and which are merely figments of my vivid loyalopposition imagination! Enough preliminaries. Here are the mostfrequently forgotten fundamental facts about software engineering. Someare of vital importance—we forget them at considerable risk.

Complexity

C1. For every 10-percent increase in problem complexity, there is a100-percent increase in the software solution?s complexity. That's not acondition to try to change (even though reducing complexity is alwaysdesirable); that's just the way it is. (For one explanation of why thisis so, see RD2 in the section "Requirements and design.")

People

P1. The most important factor in attacking complexity is not thetools and techniques that programmers use but rather the quality of theprogrammers themselves.

P2. Good programmers are up to 30 times better than mediocreprogrammers, according to "individual differences" research. Given thattheir pay is never commensurate, they are the biggest bargains in thesoftware field.

Tools and techniques

T1. Most software tool and technique improvements account for about a5- to 30-percent increase in productivity and quality. But at one timeor another, most of these improvements have been claimed by someone tohave"order of magnitude" (factor of 10) benefits. Hype is the plague on thehouse of software.

T2. Learning a new tool or technique actually lowers programmerproductivity and product quality initially. You achieve the eventualbenefit only after overcoming this learning curve.

T3. Therefore, adopting new tools and techniques is worthwhile, butonly if you (a) realistically view their value and (b) use patience inmeasuring their benefits.

Quality

Q1. Quality is a collection of attributes. Various people definethose attributes differently, but a commonly accepted collection isportability, reliability, efficiency, human engineering, testability,understandability, and modifiability.

Q2. Quality is not the same as satisfying users, meetingrequirements,or meeting cost and schedule targets. However, all these things have aninteresting relationship: User satisfaction = quality product + meetsrequirements + delivered when needed + appropriate cost.

Q3. Because quality is not simply reliability, it is about much more than software defects.

Q4. Trying to improve one quality attribute often degrades another. Forexample, attempts to improve efficiency often degrade modifiability.

Reliability

RE1. Error detection and removal accounts for roughly 40 percent ofdevelopment costs. Thus it is the most important phase of thedevelopmentlife cycle.

RE2. There are certain kinds of software errors that most programmersmake frequently. These include off-by-one indexing, definition orreference inconsistency, and omitting deep design details. That is why,for example, N-version programming, which attempts to createmultiple diverse solutions through multiple programmers, can nevercompletely achieve its promise.

RE3. Software that a typical programmer believes to be thoroughlytested has often had only about 55 to 60 percent of its logic pathsexecuted. Automated support, such as coverage analyzers, can raise thatto roughly 85 to 90 percent. Testing at the 100-percent level is nearlyimpossible.

RE4. Even if 100-percent test coverage (see RE3) were possible, thatcriteria would be insufficient for testing. Roughly 35 percent ofsoftwaredefects emerge from missing logic paths, and another 40 percent are fromthe execution of a unique combination of logic paths. They will not becaught by 100-percent coverage (100-percent coverage can, therefore,potentially detect only about 25 percent of the errors!).

RE5. There is no single best approach to software error removal. Acombination of several approaches, such as inspections and several kindsof testing and fault tolerance, is necessary.

RE6. (corollary to RE5) Software will always contain residualdefects, after even the most rigorous error removal. The goal is tominimize thenumber and especially the severity of those defects.

Efficiency

EF1. Efficiency is more often a matter of good design than of goodcoding. So, if a project requires efficiency, efficiency must be considered early in the life cycle.

EF2. High-order language (HOL) code, with appropriate compileroptimizations, can be made about 90 percent as efficient as thecomparableassembler code. But that statement is highly task dependent; some tasksare much harder than others to code efficiently in HOL.

EF3. There are trade-offs between size and time optimization. Often,improving one degrades the other.

Maintenance

M1. Quality and maintenance have an interesting relationship (see Q3 and Q4).

M2. Maintenance typically consumes about 40 to 80 percent (60 percentaverage) of software costs. Therefore, it is probably the mostimportantlife cycle phase.

M3. Enhancement is responsible for roughly 60 percent of softwaremaintenance costs. Error correction is roughly 17 percent. So, softwaremaintenance is largely about adding new capability to old software, notabout fixing it.

M4. The previous two facts constitute what you could call the "60/60" rule of software.

M5. Most software development tasks and software maintenance tasksare the same—except for the additional maintenance task of"understanding the existing product." This task is the dominantmaintenance activity, consuming roughly 30 percent of maintenance time.So, you could claim that maintenance is more difficult than development.

Requirements and design

RD1. One of the two most common causes of runaway projects is unstable requirements. (For the other, see ES1.)

RD2. When a project moves from requirements to design, the solutionprocess's complexity causes an explosion of "derived requirements." Thelist of requirements for the design phase is often 50 times longer thanthe list of original requirements.

RD3. This requirements explosion is partly why it is difficult toimplement requirements traceability (tracing the original requirementsthrough the artifacts of the succeeding lifecycle phases), even thougheveryone agrees this is desirable.

RD4. A software problem seldom has one best design solution. (BillCurtis has said that in a room full of expert software designers, if anytwo agree, that's a majority!) That's why, for example, trying toprovide reusable design solutions has so long resisted significantprogress.

Reviews and inspections

RI1. Rigorous reviews commonly remove up to 90 percent of errors froma software product before the first test case is run. (Many researchfindings support this; of course, it's extremely difficult to know whenyou've found 100 percent of a software product's errors!)

RI2. Rigorous reviews are more effective, and more cost effective,than any other error-removal strategy, including testing. But theycannotand should not replace testing (see RE5).

RI3. Rigorous reviews are extremely challenging to do well, andmost organizations do not do them, at least not for 100 percent of theirsoftware artifacts.

RI4. Post-delivery reviews are generally acknowledged to be important,both for determining customer satisfaction and for process improvement,but most organizations do not perform them. By the time such reviewsshould be held (three to 12 months after delivery), potential review participants have generally scattered to other projects.

Reuse

REU1. Reuse-in-the-small (libraries of subroutines) began nearly 50 years ago and is a well-solved problem.

REU2. Reuse-in-the-large (components) remains largely unsolved, eventhough everyone agrees it is important and desirable.

REU3. Disagreement exists about why reuse-in-the-large is unsolved,although most agree that it is a management, not technology, problem(will, not skill). (Others say that finding sufficiently commonsubproblems across programming tasks is difficult. This would makereuse-in-the-large a problem inherent in the nature of software and theproblems it solves, and thus relatively unsolvable).

REU4. Reuse-in-the-large works best in families of related systems,and thus is domain dependent. This narrows its potential applicability.

REU5. Pattern reuse is one solution to the problems inherent in code reuse.

Estimation

ES1. One of the two most common causes of runaway projects is optimistic estimation. (For the other, see RD1.)

ES2. Most software estimates are performed at the beginning of thelifecycle. This makes sense until we realize that this occurs before therequirements phase and thus before the problem is understood. Estimationtherefore usually occurs at the wrong time.

ES3. Most software estimates are made, according to severalresearchers, by either upper management or marketing, not by the peoplewho will build the software or by their managers. Therefore, the wrongpeople are doing estimation.

ES4. Software estimates are rarely adjusted as the project proceeds.So, those estimates done at the wrong time by the wrong people areusuallynot corrected.

ES5. Because estimates are so faulty, there is little reason to beconcerned when software projects do not meet cost or schedule targets.Buteveryone is concerned anyway!

ES6. In one study of a project that failed to meet its estimates, themanagement saw the project as a failure, but the technical participantssaw it as the most successful project they had ever worked on! Thisillustrates the disconnect regarding the role of estimation, and projectsuccess, between management and technologists. Given the previousfacts, that is hardly surprising.

ES7. Pressure to achieve estimation targets is common and tends tocause programmers to skip good software process. This constitutes an absurd result done for an absurd reason.

Research

RES1. Many software researchers advocate rather than investigate. Asa result, (a) some advocated concepts are worth less than their advocatesbelieve and (b) there is a shortage of evaluative research to help determine the actual value of new tools and techniques.

There, that's my two cents' worth of software engineering fundamentalfacts. What are yours? I expect, if we can get a dialog going here,that there are a lot of similar facts that I have forgotten—or am notaware of. I'm especially eager to hear what additional facts you cancontribute.

And, of course, I realize that some will disagree (perhaps evenviolently!) with some of the facts I've presented. I want to hear aboutthat as well.

Robert L. Glass is the editor of Elsevier's Journal of Systems and Software and the publisher and editor of The Software Practitioner newsletter. Contact him at rglass@indiana.edu; he'd be pleased to hear from you.