Key Note Oscar Dieste
¿Is It Possible to Build General, Empirically-Backed Knowledge in Software Engineering?
Abstract. The application of empiricism to Software Engineering (IS) brings the possibility to obtain knowledge comparable to the knowledge produced in other engineering disciplines. A large share of the research community has embraced this methodology. We have obtained a great amount of results, primarily research papers, so far. Nevertheless, we have been unable to generalize particular findings into pieces of knowledge, or theories having predictive values in more or less generic contexts.
What could be the reasons that restrict our ability to build general, empirically-backed knowledge in IS? Some accidental difficulties are well known: the (relative) youth of empirical IS, the scarcity of replications, and the limited external validity of studies conducted using students as experimental subjects. Should we have time enough, create an adequate reward structure so that researchers are persuaded to replicate, and get practitioners and companies involved in the research, then we will be able to build general knowledge.
However, right now, I am not that sure that this is completely true. Far from my intention the commencement of an epistemological debate in IS. Simply put, I have been observing that many other accidental difficulties exist, which jointly interact to give origin to impressive obstacles that deserve the name of essential difficulties. The core problem is our inability to assess in a reasonable precise manner the context effects. Having unlimited time/resources at our disposal, it is theoretically possible to ascertain the influence of context; in practice, resources are limited and the context effects remain uncertain.
I wonder whether the right approach is the other way around, i.e.: start the research with some promising technology (/phenomenon) of apparently general character. For instance, we could start the research using students, and later try to establish its usefulness (/truth) in particular contexts, keeping away from pretensions of generality. This reflection is not totally my original. A.M. Davis & A. Hickey y R. Wieringa have expressed similar opinions, although starting from different premises (theirs theoretical, mine from the trenches).
Keywords. Empiricism, theory, replication, students, practitioners, industry, experiment, case study, synthesis, meta-analysis, context