An investigation of the psychological processes underlying the debugging of computer programs

Abstract

The primary objective of this thesis is to contribute to a theory of programmer expertise with respect to debugging computer programs. It is important to develop a theory of programmer expertise for the following reasons. First, understanding the knowledge structures that expert and novice programmers possess will aid in such tasks as recruitment and training of programmers, evaluation of programmers, the development of programmer rating instruments, and the design and choice of debugging aids. Second, researchers must control for the knowledge structures programmers possess if they wish to measure the effects of other factors that influence programming performance, namely, factors such as language design, program layout, and programming mode. Furthermore, since debugging is a diagnostic task, this research will also contribute to a general theory of expertise in diagnosis. Since the aim of the research was to investigate problem solving processes, a protocol analysis methodology was employed, i.e., a methodology designed to investigate what happens during problem solving rather than to measure the outcome of problem solving. The task materials were deliberately kept simple so that processes could be investigated free from the effects of differential knowledge of the application domain. Two levels of task difficulty were incorporated into the study by introducing a bug at different levels in the program. It was hypothesized that differences beteen expert and novice programmers would become more apparent as the task became more difficult. Professional programmers were used as subjects to increase the external validity of the research. The major results from the research stem from the difficulties perceived in defining groups of expert and novice programmers. Two classifications were originally used to distinguish experts from novices. One was based on ratings provided by the programmers' superiors, while the other was based on a programmer's ability to recall a computer program. Neither of these two classifications appeared to differentiate the performance of programmers. Consequently, a third method of classification was derived the protocol data itself. This ex post classification was founded on the assumption that expert programmers display smooth-flowing rather than erratic approaches to problem solving. Based on this classification, the statistical analyses did differentiate the performance of expert and novice programmers. In addition, the program bug discriminated between the two groups of programmers. Analysis of the strategies adopted by programmers showed that those classified as experts by the ex post classification are characterized by their top-down, breadth-first approach to problem solving and by their ability to conceive of the program as a system, whereby they could deduce a causal model of the error. Novices, on the other hand, are characterized by their depth-first search for the error and/or by their lack of system thinking. In summary, the research has contributed to the formulation of a theory of programmer expertise and of a general theory of expertise in diagnosis by identifying the following strategic propositions for further investigation: 1.  (a) Experts use a top-down, breadth-first approach to problem solving and, at the  same time, conceive of the problem area as a system. (b) Experts display smooth-flowing approaches to problem solving. 2.  (a) Novices use a depth-first approach to problem solving, or they are unable to use a systems approach, or both. (b) Novices display erratic approaches to problem solving.

Keywords

Related Publications