Comparison of Expert and Novice Solution Approaches to An Industrially Situated Process Development Project

Engineering educators must be mindful of the diverse range of careers their students will pursue and the rapid integration of new technologies into those careers. Such a perspective naturally leads to a shift of focus from teaching students “what to think” to teaching them “how to think” and, even more specifically, “how to approach novel problems.”  Within this context, a greater emphasis is being placed on developing students' abilities to transfer knowledge and understanding from the classroom to professional practice. The research reported in this paper seeks to identify and compare methods that expert and novice engineers use to solve the same novel process development project. Through this study, we seek to contribute to the understanding of transfer in ill-structured engineering problems.

Participants in the study were tasked with developing an optimal “recipe” for a Low Pressure Chemical Vapor Deposition (LPCVD) reactor that deposits thin films of silicon nitride on polished silicon wafers, an initial step in the manufacture of transistors. The optimization problem is completed using a virtual laboratory and involves iteratively developing, testing, and refining solutions based on adjusting input parameters and measuring outputs.  The computer simulation generates data representative of an industrial reactor and includes both random and systematic process and measurement error.  Users are also encouraged to apply sound engineering methods since they are charged virtual money for each reactor run and each measurement.  This problem is complex and a typical team spends approximately 15-25 hours to complete it.

For many years, the virtual LPCVD project has been delivered in a capstone chemical engineering course. The solution process of one student cohort is used to represent different possible novice solution approaches. Additionally, three industrially accomplished experts led teams complete the project.  Two of the experts had no direct experience with CVD reactors.  This choice was deliberate as we seek to characterize how experts will solve the problem by transferring their core engineering knowledge and skills to a novel problem. One expert was a mechanical engineer who lacked domain specific knowledge, but is an internationally recognized leader in engineering design.   

The virtual LPCVD project contains the features of an ill-structured problem. It is authentic, ambiguous, and has many possible solutions and solution paths.  However, it has more constraints than design problems found “in the wild.” This aspect reduces the degrees of freedom and facilitates comparison of different solution processes.  Additionally, the virtual environment enables a more thorough assessment of a team's proposed “recipe.” Since the error can be removed and film thickness “measured everywhere,” an absolute metric of performance can be obtained and used for comparison.

Solution processes were documented using talk-aloud protocol analysis, collection of laboratory notebooks and written reports, and data acquired by the computer interface. Previously we have reported a graphical method, termed model representation and usage maps, which allows us to characterize participants' model development as they solve the problem.  These maps show the nature of each model component (quantitative, qualitative, graphical, empirical, statistical), its correctness, and its use in the solution process (did it direct the values of input parameters for a future run, was a run used to quantify model parameters, was the model qualitatively verified, etc.). The model maps are the primary artifact of analysis and form the basis to compare participants' solution processes. Additionally, semi-structured interviews are used to validate the results of the study and provide further insight.

Solution characteristics and approaches of the expert and the student teams are identified. Compared to the student teams, the expert engineers devote a higher proportion of time to information gathering and problem scoping. They also tend to access large “chunks” of information during the problem solving process.  Once activated, these chunks tend to remain relatively constant in form and are used throughout the solution. However, regarding performance, the final recipes of the experts were no better than a large fraction of the student teams. Specific ways that the participants' prior knowledge and previous experiences were applied to this new situation are identified and will be discussed.

---