TP

What should a Software Engineering course look like

When I joined the School of Computing at the University of Kent, I was asked what subjects I wanted to teach. One of the topics I chose was Software Engineering. I spent quite a lot of time reading about the history of software engineering when working on my paper on programming errors and I go to a fair number of professional programming conferences, so I thought I can come up with a good way of teaching it! Yet, I was not quite sure how to go about it or even what software engineering actually means.

In this blog post, I share my thought process on deciding what to cover in my Software Engineering module and also a rough list of topics. The introduction explaining why I chose these and how I structure them is perhaps more important than the list itself, but it is fairly long, so if you just want to see a list you can skip ahead to Section 2 (but please read the introduction if you want to comment on the list!) I also add a brief reflection on why I think this is a good approach, referencing a couple of ideas from philosophy of science in Section 3.

I will be doing a talk based on some of the ideas in this blog post at the NewCrafts conference on 16-17 May in Paris, a software development conference for professional developers who care about quality code and bettering their practices. If you want to chat about these ideas, NewCrafts will be a perfect opportunity!

1. The problem with Software Engineering

Many universities treat software engineering as a course where they should give students some directly practical useful skills that future employers will ask for. Although we believe that topics typically covered by the computer science curriculum (such as algorithms, complexity analysis, logic, but also compilers or theories of human computer interaction) are important, we also recognize that most students will go on to work as software developers and companies hiring them will be very happy if students know some of the practices, tools and methodologies that they are using.

Consequently, software engineering becomes just a course trying to cover whatever methods and tools the industry needs. This is something that universities cannot, in principle, teach very well. The tools of choice and even popular methodologies keep changing quite quickly. The trend of the day is no longer object-oriented modelling, but event sourcing; three-tier architecture gets replaced by microservices with tools like Kubernetes; heavyweight processes get replaced by Agile methods, and even (some aspects of) these are frequently being questioned. Even if a university designs an up-to-date course it will be obsolete in 5 years.

Should we try keeping up with industry? Should we just give up and let students learn current methods and tools on the job? Or is there something else we can do with the dreaded software engineering course?

1.1 Why teach Software Engineering at all

If we see software engineering just as whatever methods and tools the industry needs, then we can just say that this is not something the university needs to be teaching at all. We teach algorithms, complexity theory and other computer science subjects because they will largely remain valid in 100 years. They might not be directly applicable in many programming jobs, but at least they capture a valid body of knowledge that we have collectively accumulated. Programmers need to learn new methods and tools all the time, so teaching them the first one they'll need after they finish university is not adding a huge amount of value anyway. (We might still do this because some universities confuse themselves with vocational training centres, but that's another topic.)

The question is, is there something about software engineering that is capturing knowledge that will (likely) still be relevant in 100 years?

There certainly should be something. Most computer science topics are rooted in more scientific applications of computers, and later ones in problems arising within computer science itself. However, computers are only ubiquitous because they stopped being a tool used by scientists and the military and became tools for business data processing throughout the 1950s and 1960s. This introduced completely new kinds of problems - around addressing business needs, team work, processes and reliability - that professional software developers still face today.

The problem is that (academic) computer scientists found the nice fundamental concept of an algorithm and focused their efforts around this concept. In contrast, the data processing industry never quite found good foundations for its use of computers. In a 1960 letter to the editor, an ACM member complained (quoted in Computer Boys Take Over):

All of us, I am sure, have read non-ACM articles on business data processing and found them lacking. They suffer, I believe, from one basic fault: They fail to report fundamental research in the data processing field. The question of 'fundamentalness’ is all-important (...) [It provides] a technique for getting the field of business data processing on a firm theoretical footing.

I think that a good Software Engineering curriculum needs to address this issue. What are fundamental insights about software development? Computer science, with its focus on algorithms and mathematization, found one possible answer to this question, but definitely not the only one. Part of the reason why I'm interested in teaching Software Engineering is that I think the course provides a nice opportunity to find other possible interpretations of what fundamental might mean...

1.2 Where Software Engineering comes from

In the 1960s programming appeared to be a black art. Some programmers were good at it, but there was no clear way of recognising who had the necessary skills, training new people, or scaling the production process to a large system. The NATO Conference on Software Engineering in 1968 had a goal of "turning the black art of programming into a proper engineering discipline" and it also introduced the term Software Engineering.

The attendees at the conference were a diverse group including computer scientists, managers and military officials. Everybody agreed on what the problem was, but there was little agreement on how it should be addressed. The next NATO Conference a year later is widely regarded as a complete failure for this reason. Computer scientists advocated use of mathematics and proofs, managers advocated a factory model of organization, while the military was more used to achieving reliability via over-engineering. In the following years, the term Software Engineering largely became the name for a managerial approach to the challenge and came to cover things such as requirements gathering, producing specifications, cost and time estimation, and detailed up-front modelling of software architecture.

The challenge of turning the black art of programming into a proper engineering discipline still remains (although we might phrase it a bit differently these days). We still have not solved the problem and new ideas and methods appear and disappear over time. Many of the ideas that are traditionally associated with Software Engineering are products of the 1970s and 1980s (or 1990s, if we're lucky). Some of those are, no doubt, interesting, but they are not very relevant in modern software development. More recent methods that address the problem (such as Agile) do not favour the "software engineering" term, but they certainly fall into the same general space.

The problem is, if we just replace 1970s and 1980s content with 2000s content, we are not developing any more fundamental knowledge. We're just replacing one, already obsolete, idea with another, soon-to-be-obsolete idea.

2. Fundamental Software Engineering knowledge

Universities need to teach Software Engineering knowledge that is fundamental in the sense that it does not become obsolete in the next 5 years and in the sense that it captures some unchanging principles of Software Engineering. Heavyweight software engineering methods from the 1970s and 1980s are not this, but more modern methods like Scrum and Agile programming are not this kind of knowledge either.

What I believe is unchanging is the motivation behind software engineering and the reasoning that led to those particular methods and tools in the face of particular problems that the industry was attempting to address. In other words, we should not teach Waterfall, UML, Scrum or TDD. We should teach how different circumstances, problems and goals motivate these, why the various methods are a reasonable response to the challenge (even for Waterfall!), and circumstances where they do not work.

This also gives us a good answer when we ask what practical value does a Software Engineering course like this have. We might not cover the favourite tools and methods of the day, but we can help students think about the motivations for and limitations of the tools and methods. When they work for a company that follows a particular methodology, they should be able to recognize whether the methodology is actually suitable given the problems, motivation, and other circumstances.

I believe such historically situated analysis of Software Engineering is fundamental knowledge. It is also a valuable thing that universities can teach, because it is hard to learn on the job. It requires a historical perspective and an additional level of reflection. It is worth noting that what I'm advocating here is quite different from, say, fundamental knowledge that computer science collects about algorithms. It is historical, technical and sociological rather than mathematical, but that does not mean it is any less fundamental. It's just a different kind of knowledge.

2.1 Historically situated Software Engineering

If you came here for a list of topics that I think we should teach in a software engineering course, you finally reached the right section! You'll notice that many of the technical topics I want to cover are, actually, topics that are covered by typical software engineering courses. The key difference is that I think we should spend about third of the time on the actual technical details and two thirds on the history and critical analysis.

Methodologies and approaches in context

Today, outdated software engineering courses teach a variant of the waterfall development methodology while more up-to-date courses teach agile or scrum. Both of these are wrong. Even with agile, the way it is practiced differs significantly across companies and so the particular practices an academic course might cover will not really be directly relevant. Instead, we should teach:

Modelling in context

Another topic that is often featured in software engineering courses is UML. I do not often meet people who actually use UML in their work, but the general idea of somehow capturing key ideas about a system in a more readable form is definitely worth discussing. Again, I think the particular methods need to be discussed with an appropriate context that explains when they work.

Software architectures in context

Many software engineering courses also include the Gang of Four Design Patterns. I suppose those are often presented as an answer to some problems arising in software architecture, but really, most of them are very technical and address limitations of C++ and later Java-like languages. I think there is a useful lesson to be learned from GoF patterns if we add context, but we can also consider software architecture issues more broadly.

Finally, one reference that I have not yet read, but which could be an other useful source of material for discussing software architecture in context is The Architecture of Open Source Applications book.

Current hot topics

I think the core of a software engineering course should cover fundamental principles that do not go out of date and I think that critical historical reflection on different methodologies and architectures achieves that. However, I recognise that we also need to cover some practically useful hot topics and I like teaching some of those myself! Fortunately, the historically situated approach to software engineering allows that:

2.2 Unchanging principles of Software Engineering

Aside from historically situated perspectives on different contexts and software engineering approaches responding to them, there are a few more things we should teach. In computer science, the halting problem is a fundamental piece of knowledge, because it captures the limits of what can theoretically be done. Are there similar fundamental limitations in software engineering? Given that software engineering is an essentially human discipline, the answer will not be mathematical. There are not many similar principles, but there are some:

2.3 Where Software Engineering ideas come from

Finally, I think that another topic worth our attention in a Software Engineering course is the diversity of sources that are useful for understanding software engineering. Good software engineering relies on broad interdisciplinary understanding and I think it's not a surprise that interesting ideas about how software should be built borrow ideas not just from engineering, but also from urban planning, architecture, or even gardening:

3. Is this still Software Engineering?

You might be wondering if what I'm advocating as Software Engineering in this blog post is still computer science, or whether I'm saying that we should replace "normal" Software Engineering courses with something more akin to history and philosophy of software engineering.

Historically, software engineering is following the tradition of the data processing industry which has never quite become the same thing as computer science. This is also apparent in the present-day debates about whether programming job interviews should involve computer science questions (say, the \(O\) notation). If we are teaching Software Engineering, we need to expand our focus and also our methods, because a good Software Engineering course will need to be different.

If learning software engineering at a university taught you the same things as learning software engineering on the job, then I think universities should not bother teaching it. (Universities are not, or at least should not be, training centres.) However, I believe that teaching what I called historically situated software engineering does give students something very valuable that they will not get through work experience. It provides a framework for critical thinking about software engineering, and past examples to guide this thinking.

The approach that I'm advocating is somewhat akin to Bruno Latour's position presented in the We Have Never Been Modern book. To quote from the summary on Wikipedia:

[Latour] claims [that] we must rework our thinking to conceive of a "Parliament of Things" wherein natural phenomena, social phenomena and the discourse about them are not seen as separate objects to be studied by specialists, but as hybrids made and scrutinized by the public interaction of people, things and concepts.

I think this is a perfect summary of what I think is a good way of thinking about and also teaching software engineering. The technological aspects of software engineering such as source control or design patterns need to be linked with the social phenomena surrounding them, including the business context that motivates them, but also with critical reflection on those entities and their contexts.

Acknowledgements. Thanks to Richard Jones for feedback on earlier version of the post and to Roger Turner for catching and correcting some of my typos.

References

  1. Nathan Ensmenger (2010). The Computer Boys Take Over: Computers, Programmers, and the Politics of Technical Expertise
  2. The NATO Software Engineering Conferences
  3. Christiane Floyd (1993). Outline of a Paradigm Change in Software Engineering
  4. Manifesto for Agile Software Development
  5. William Scherlis, Dana Scott (1993). First Steps Towards Inferential Programming
  6. John Vlissides, Ralph Johnson, Richard Helm, Erich Gamma (1994). Design Patterns: Elements of Reusable Object-Oriented Software
  7. Amy Brown (ed.) (2011). The Architecture Of Open Source Applications
  8. Tools for collaborative development (CO886)
  9. Fred Brooks (1987). No Silver Bullet Essence and Accidents of Software Engineering
  10. David Parnas (1985). Software Aspects of Strategic Defense Systems
  11. Rebecca Slayton (2013). Arguments that Count: Physics, Computing, and Missile Defense, 1949-2012
  12. Jeffrey Liker (2011). The Toyota Way to Lean Leadership: Achieving and Sustaining Excellence through Leadership Development
  13. Jane Jacobs (1961). The Death and Life of Great American Cities
  14. Bruno Latour (1993). We Have Never Been Modern

Published: Friday, 8 February 2019, 12:22 PM
Author: Tomas Petricek
Typos: Send me a pull request!
Tags: academic, teaching, philosophy