In the title I’m kindof punning on the word “a” (it’s my blog, and I get to do what I want). Is there a single thing, software engineering, that all people making software should (or could, or would find to be beneficial) do?
It’s a question that’s sat with me all through my doctoral research, and has sat with the field for much longer. In 2007, Diane Kelly argued that there’s a “chasm” between scientific computing and commercial software, and that leaders should get together and identify their differences to come up with discipline-specific approaches to writing software. In other words, that there isn’t a universal “software engineering” that’s the best approach to writing software.
A decade later, Tim Storer described the chasm as a skills gap that needed to be closed. In this view, software engineering is the application of software knowledge to the production of software, and computational scientists don’t have enough of that knowledge to do it correctly.
There’s a whole community of research devoted to uncovering a grand unified theory of software engineering, in analogy to the Grand Unified Theories of physics that unite the electromagnetic and weak and strong nuclear forces. Members of this community (which goes by the name SEMAT: Software Engineering Methods and Theory) start not by constructing their theory but by deconstructing others.
They argue (convincingly) that any particular software engineering methodology is flawed, because it recommends a whole suite of practices but we don’t know which are relevant and useful, or how they interact, we just know that the methodologists managed to trademark their particular grab bag of practices and argue that if you’re not making enough software, you’re not doing it the way they propose. While there might be something to daily stand-ups, or to organising work by sprints, or to holding retrospectives, there’s nothing to Scrum because selecting all of these practices together is entirely arbitrary.
What the SEMAT folks argue for instead is more of a systems approach to software (in a Dana Meadows sense rather than a Jerry Weinberg sense): the team use their way of working to do work to generate a software system that realises requirements to exploit an opportunity identified by stakeholders; which part of that process is the most broken and what can you do to make it less broken than some other part?
I think that’s a great way to think about it, and I also don’t think that a GUT of software engineering will arise from it. To me, software is the reification of thought in a reusable and somewhat abstracted structure: we understand something about a context and try to capture our (or, commonly, someone else’s) understanding of that context in a way that can be automatically calculated using digital electronic equipment. To say that a universal theory of making software exists is to say that a universal theory of understanding thought exists, and we aren’t there.
Many of the open problems in software engineering boil down to not being able to capture thoughts precisely. Software engineering ethics is the inability to define universal rights and wrongs (which may not exist anyway). Software quality management is the inability to agree what the understanding was and whether we’ve captured it correctly. The fact that we don’t agree on whether object-oriented, structured, functional, or some other approach to analysis and design is the best choice is a sign that we don’t agree on how to encode thought in a way that we can think about.
In other words, software construction is thinking about thought, it is meta-thought. And we don’t agree enough on how thought works to be able to get consensus on the best way to think about thought, let alone the best way to encapsulate that thinking about thought in a saleable product.
Based on this mumbo jumbo, I take it you have applied for an academic job and need to publicly prostrate before those whose colorless ideas are sleeping furiously.
Or perhaps you are still hung over.
Do I infer that you disagree?
Like an academic paper (and many industrial ones), an important problem is identified, then a methodology is proposed (which of course has a book and consultants promoting it; let’s not talk about any evidence), and then things go meta.
Is there “… a whole community of research devoted to uncovering a grand unified theory of software engineering…”? There are many communities polishing dots, but almost nobody trying to connect them. There are certainly communities of consultants promoting their services as supporting a unified theory of software engineering.
> To say that a universal theory of making software exists is to say that a universal theory of understanding thought exists, and we aren’t there.
It’s not clear how you draw that conclusion. It’s true that no universal theory of making software exists which is to say we have yet to agree on one. I don’t think we need to reach to a universal theory of understanding (and capturing) thoughts, we just need to agree on an “encoding”.
It seems to me that as software engineers we keep treating programming languages and their paradigm as that encoding and refuse/resist to agree on software engineering principles.
I am pretty sure if our hand was forced by regulation, we would come to a set of principles we can all agree on.
Go educate yourself :
https://www.youtube.com/watch?v=CmIGPGPdxTI
I agree with everything you say here, and have already planned a part two in which I say that yes while there’s no universal _theory_ of making software, there are universal _considerations_ that apply no matter the context. I think the “we” in “as software engineers we keep treating programming languages…” is probably programmers who use the title “software engineers”, but software engineering is bigger than just programming. I doubt that people who focus on testing, or localization, or project management, are so focused on the programming language as a silver bullet.
Yes, there are, for example https://dl.acm.org/doi/10.1016/j.jss.2015.08.009 and https://users.ece.utexas.edu/~perry/work/papers/DP-sede11.pdf and https://ieeexplore.ieee.org/document/4299884 and https://dl.acm.org/doi/10.1109/GTSE.2015.18 (the GTSE workshop series is the same folks who did SEMAT, who are all in that academic/industry crossover space). As I say, I disagree that there will prove to be such a theory on principle, however I think that searching for it is a good way to find out what to do instead, in the same way that searching for the luminiferous ether helped us find out that it doesn’t exist.
> I doubt that people who focus on testing, or localization, or project management, are so focused on the programming language as a silver bullet.
Agree that people do exist who care about software engineering. Still, the tide has not turned in decades of computer science.
It appears to me that people treat *programming* languages more like a “language” and less as a tool to write software. Which in turn makes people lean more heavily on individual expression on how to design software and less on following software engineering principles.
Maybe taking away the “language” from programming will force people to focus on building great software instead. Following on that train of thought, maybe an AI model should be trained on software engineering principles and we use natural language to build software.
> maybe an AI model should be trained on software engineering principles and we use natural language to build software.
This is the promise of Cobol^W4GL^WUML^WBDD!
The question of whether software engineering is real engineering is different from whether there’s a single one-size-fits-all software engineering that everyone should use. Additionally, I think it’s pretty clear that it isn’t the same as a traditional engineering discipline. Conflating the two has been used before by people who don’t believe in accreditation for software engineers: “if a software engineer is an engineer then they’d have to pass an engineering exam and learn a load of useless facts about fluid dynamics, therefore let’s not license software engineers!”
Thanks for the links. These papers contain observations about software engineering practices, along with the obligatory review of papers, I assume, the authors agree with. This paper https://www.researchgate.net/publication/278743539_The_Tarpit_-_A_General_Theory_of_Software_Engineering gives its theory a name, and spends 65 pages discussing everything but a concrete theory.
Most research in software engineering is like this: https://shape-of-code.com/2021/01/17/software-effort-estimation-is-mostly-fake-research/
That is, it is fake research.