On Software Quality...

One way to radically improve the quality of the software is to create libraries.

Libraries reduce the duplication of code and make the same piece of code to be reused extensively, increasing the amount of testing that is done over the same piece of code.

Having the same code spread all over the system is maintainance hazard, since if a bug is found in that code, it for sure is apread all over the system. Fixing a line of code, statiscally, introduces a new software defect, so if you touch 20 lines of code to fix a defect, statistically you are introducing 20 new defects.

Some projects detect a defect that needs to touch 2 thousand lines of code to be properly fixed, so they decide not to fix the bug, because the cost of finding and fixing all those 2,000 potential new bugs is a burden that very few projects could take.

Libraries can usually be tested aside of the rest of the system, because they are reusable. This is a very important characteristic of libraries, since they can be reused in other projects, so they are an investment, but also it is important because defects and dependencies can be contained within the library, and although it can affect systems built over that library, the library can be easily fixed and tested independently, so once it has been fixed, it can be incorporated again into the system built on top of it.

Case Analysis at Microsoft

Microsoft is a well known firm in the PC software market and has some very well known good practices like the informal code review before check-in, written code conventions, the daily build, the smoke test and the doog food processes. Nevertheless, ex Microsoft employees critizise Microsoft as having too low abstractions. Well, having too low abstractions is having no abstraction at all.

There are some coding metrics used at Microsoft that measure performance by the number of lines of code each developer writes. Many Microsofties complain that this practice encourages sloppy coding and doesn't measure the actual effort nor the actual benefit of the performed job. This creates a perverse incentive of any developer to use copy and paste, rather than to create reusable functions, since at the performance review, not only themselves, but their whole team will appear underperforming.

There have been some debate inside Microsoft as if measuring performance by counting the lines of code is a good practice, and several key executives within Microsoft have publicly declared that using function points would be more appropiate, but since there is a known correlation between the two, anyone can easily use one or the other interchangeably, with the added benefit that counting source lines is cheaper.

Companies that insist on use function points have the problem that there is no automatic way of telling how many function points a program has, some function points are harder than others (the same can be said of lines) and usually companies forced to use function points simply reduce it to how many reads and writes does a program do, be it to a file, to a database, or using getters and setters. When management is measuring like that, any competent developer can make his program use more reads and more writes (ie: make it innefficient) if that means having a better evaluation.

Let us suppose we have two developers names A and B. Developer A is very good reducing duplicated code and using very fast data structures, minimizing the amount of data moved in or out of classes, and in general making programs that perform faster than the rest. Developer B is all the contrary, he writes very long programs with very simmilar code copied and pasted all over the place. Then he adds lots and lots of optimizations which improve all the non-important parts of the code and make very little improvements to the speed, but make the program move a lot of data.

Developer A will get a very bad review while developer B will get a very good review, probably an increase in salary and a promotion. Eventually either developer A will leave for greener pastures or he will work under B's supervision, which means that all developers in the same company will do as B does, no matter what they do.

The point I'm trying to make is that both counting lines of code and function points are aberrations. We will see why this logic does stand the reality test using a gedanken experiment (a thought experiment).

Let us suppose that developer A needs to create a new reusable function Fa of 10 lines and use it in 3 different places, but since he has already been told not to create small functions, he rather decides to copy and paste them, instead of the 10 lines of code called from 3 places (13 lines total), he ends up with 30 lines of code.

Now comes developer B that needs to grab the function Fa and call it inside his new function Fb that contains 5 lines of code (inclusing the call to Fa), so that Fb will be called 20 times. But developer B has the same motivation as developer A not to create that function, specially since he has also been told not to create small functions and a five line function doesn't seem big at all.

In the scenario where both developers were encouraged to create new functions, developer B would already be done writing his 5 lines function and calling it 20 times (25 lines total). But under the scenario where both developers are encouraged not to create new functions, developer B has to look at 30 lines of code written by developer A, understand those lines, see which of them could be copied and pasted easierly, and then copy that code 20 times, plus 4 additional lines each time. That's exactly 14 lines copied 20 times, or 280 lines total. If statistics are real, that means that developer B will take almost 10 times longer without using functions that using them, and will introduce 10 times more defects.

Also, since the program is now less modular and less structured, it will take 10 more times to test it throughly, the number of defect present will be higher, but also the amount of defects detected will be higher, while the percentage of defect detected will be lower, because there will be less time to find the defects, less time testing the system, and removing defects that were reproduced by copy and paste.

Let us suppose that the average number of defects is 1 defect per 10 lines of code. In the case that both developers used functions, A wrote 13 lines (1.3 defects) and B wrote 25 lines (2.5 defects), so the total number of defects would be 3.8 defects (let us suppose it is 4, since 3.8 is statistically speaking).

In the case that no developer used functions, A wrote 30 lines of code (3 defects) and B wrote 280 lines (28 defects), so the total number of defects would be 31 defects, so it would be 8 times higher.

Statiscally, each developer can fix 3 defects per day while introducing a new one each day, so the rate would be 2 defects a day. The time it takes to fix all 4 defects for a developer would be 2 to 3 days, depending on if he introduces a new defect the last day or not.

In the case of 31 defects, to fix them, if we use one developer, he would take 15.5 days, or 15 to 16 days depending on if he introduces one defect on the last day or not.

Let us consider the total cost of development, asuming each developer writes 100 lines of code per day and that the testers take the same amount of time testing the code as the time necessary to write it:

Strategy	Write the code.	Test to find defects	Fix the detected defects
No functions	3 days	1 day	16 days
Use functions	3 days	1 day	3 days

This example shows what happens with a very small task and very few developers, now let us supposse that we have many developers and very large tasks, what would happen?

First, let us suppose that only developer A does his task and developer B is not required to do so. Do we see a huge impact?

Strategy	Write the code.	Test to find defects	Fix the detected defects
Write the code.	0.3 days	0.13 days	2 days
Test to find defects	0.3 days	0.13 days	1 days

We now see that the numbers are not so different for smaller tasks and smaller teams, but when teams grow larger, the number of copy-pastes increase exponentially, since every line that was copied and pasted is now a potential candidate for a new copy and paste.

I think the real reason that Microsoft prefers to copy and paste instead of creating new functions is the added cost of creating a new name. I mean what can you expect from a company that names their windowing operating system "Windows", their document processor "Word", their web browser "internet explorer", their planning program "Project". They have problems inventing meaningful names, since Bob, PowerPoint and Excel are not really names one could consider remotely related to the functions they perform. (Well maybe making a power point does have some sense, but that is an exception).