Structure and Interpretation of Computer Programmers

This episode continues a review of the 1969 conference on software engineering techniques looking at software adaptability, large software projects, and software engineering education. The episode is supported by members of the Chiron Codex Patreon (use this gift link for your first month free), so please do join the community or hit the Ko-Fi button to make a one-off donation. If you enjoy the episode, share it with at least one friend, colleague, or stranger.

Links

• Software Engineering Techniques: Report on a conference sponsored by the NATO Science Committee
• The NATO Software Engineering conferences
• Two Pizza Teams
• Net Negative Producing Programmers
• Software Engineering at Oxford | Subjects (by theme)

Transcript

Hello, and welcome to episode 62 of the Structure and Interpretation of Computer
Programmers podcast. I’m Graham Lee, and this episode is the sixth part in a
mini-series exploring the NATO Software Engineering Conferences. The episode is
sponsored by you, the software engineering community. If you enjoy this podcast,
please share the link with your friends and colleagues.

In episode 61, the conference was dominated by two issues. First, an attempt to
bootstrap a NATO Software Engineering Institute that failed, and it isn’t
covered in the conference report. And second, a gulf between theoreticians and
practitioners, between academia and industry, between computer scientists and
software engineers, and between Edsger Dijkstra and anybody else. That last
ravine appears again at the outset of section 4 on software flexibility.

Alan Perlis starts off trying to make an analogy between a hierarchical design
and an onion. The design has multiple layers, each of which has an inner
interface, the way that it uses things from the lower layers, and an outer
interface, the things that it provides to the upper layers. Dijkstra jumps in to
say that the onion is a bad analogy because it’s three-dimensional, and a
hierarchical design is one-dimensional. He’d prefer if we kept those higher
dimensions free to represent other attributes of the design.

The report editors split flexibility into two categories. Portability, the
ability to run software in multiple systems, and adaptability, the ability to
keep up with changing requirements. Although it’s clear from the report that
some attendees think of migrating software between systems as an adaptability
problem, the two categories aren’t actually disjoint. Perlis was trying to make
a point about portability, which is a valid but incomplete definition; that
portability is concerned with “creating support on new machines sufficient to
accept systems transported from old machines”. In other words, if you can
emulate the facilities or interfaces of the old computer on the new computer,
you can run your old software on your new computer. Think of the way that
Solaris on sun4v hardware, at least until version 10, and I have never used
version 11, still supports system calls defined in BSD, so that Sun 4 programs
that were written for sun4m hardware still work.

Much of the discussion in the section, though, takes different approaches, the
first being a high-level language. As Stephen Warshall of Applied Data Research
puts it, a program is “descriptions by people of computational processes”, and
you can communicate that description without having to describe a computer.
Describe your solution at sufficient abstraction from the hardware capabilities
that it doesn’t rely on specific hardware provision, and then provide the
support facilities, in other words, the language runtime, that allows it to
execute on both the original system and on possible future systems. This fits
Perlis’s Onion model. The program in its high-level language is the outer
interface of the onion layer, and the language runtime support is the inner
interface. The trap is that the more abstract the computational model of the
language, the more capable the runtime needs to be to make the program
executable. Warshall’s solution is to add another layer to the Onion. As Kevlin
Henney says, any problem in computing is solvable with another layer of
indirection, except the problem of having too many layers of indirection. The
layer he adds is the virtual machine—”Consider the use of software as a device
for eliminating representational commitments”. That layer provides a simulated
computer that the high-level program runs on well, that expresses its needs in
terms of the facilities the target hardware provides. You lose the ability to
fine-tune performance by directly expressing the software in terms of the target
system, but you gain a lot in terms of portability.

Two problems with this approach are, one, that you can’t actually predict the
future very well, so an upcoming target system may have facilities you can’t use
from the virtual machine, or a design that makes the VM highly inefficient. As
an example, imagine virtualizing the tape interfaces on some computers, then
receiving hardware that has random access storage. Secondly, that the
abstractions provided by the high-level language become unrealistic. Here, the
example Warshall gives is, you might write all programs as though you had
infinite memory, and then let some automatic device worry entirely about storage
strategy and resource strategy. That’s actually what we do these days. If you
look at the memory allocator in a modern operating system, you’ll usually find
that the malloc function can’t fail, and always returns a non-null pointer to
virtual memory. Or at least with a 64-bit address space, it takes a long time
before it runs out of pointers to give you, which is the only reason that it
actually does fail. It’s only when the program actually tries to use that memory
that the runtime tries to get the kernel to commit some memory to support it.
And it’s only when the kernel runs out of memory to offer that things start to
go sideways.

Of course, by relying on high-level languages, you really just kick the problem
down the road. Yes, your software is defined in a portable way, but until
someone ports the virtual machine or runtime library, it isn’t actually
portable. You can “bootstrap” porting something like a compiler by creating a
compiler that runs on the computer you already support and generates code for
the computer you want to support, and then compiling the compiler for that new
computer using the old computer, and then the compiler for the new computer
using the new computer. Strachey talks about porting Martin Richards’s BCPL
compiler, which is written in BCPL with some assembler language, by rewriting
the code generation part in ALGOL, providing a portable means of compiling the
compiler. Of interest to retro-computing fans is that Martin Richard’s also
wrote the Tripos operating system in BCPL, which became the basis of Commodore’s
AmigaDOS after their own in-house project ran into trouble.

An alternative approach to abstracting the target of your high-level language is
to use macros so that machine-dependent statements in your program have
different values on different target machines. Jerome Feldman and Dijkstra both
explore this idea, which evolved into its ultimate form with the GNU AutoTools—a
very complicated macro facility that generates programs to test what values to
give the macros, and that many people use without understanding it. They both
make the point that what you end up with isn’t a portable program, but a
software product line (though they don’t call it that because it’s a 1990s
phrase). What you end up with is a family of related programs and a process that
selects and creates one instance of that family.

Perlis wishes for tools that stop you doing machine-specific things, and warn
you if you somehow still manage to do it. But John Buxton of Warwick University
comes to the Gordian knot of portability in 1969, which is that the support
tools, the compilers, assemblers, runtime libraries and so on, that people
relied on in the 60s were, for the most part, supplied by the hardware vendors.
These are the people least invested in making it easy to port your software to a
different computer because you’ve already bought their computer if you’re using
their tools.

I just want to add a couple of other points from the discussion on flexibility,
because while they’re small, they’re very important. One is that General
Electric’s Bob Bemer and conference co-chair Professor Friedrich Bauer both
point to the needs to make data portable as well as programs to successfully
port a system, with Bemer lauding the COBOL separation of a program into
divisions as a facility that enables portability. Now, he would say that as he
was one of the creators of COBOL. The algorithms in a COBOL program are defined
in a procedure division. The variables, structures and data interfaces are in a
data division and the target machine and configuration are in an environment
division. You can use the same data and procedures on different machines or in
different configurations on the same machine just by changing the environment
division; the same procedures with different files by switching the data
division, and so on. Bauer’s idea is that you need to record the data schema
somewhere either in the program as COBOL does with its data division or in the
database itself so that the database becomes self-describing.

Another point is that Butler Lampson takes the point on flexible runtimes a
little further by saying that if you have modules with well-defined interfaces
you can wrap those modules in an “envelope” and reuse them in different contexts
without needing to change the module’s program. That idea crops up again in the
Gang of Four book where it’s called the Adapter Pattern.

This is the advert break. It starts now.

This episode is brought to you by me, Graham Lee. But really, by you. Chiron
Codex is a community of people who are learning how to become better software
engineers by adopting AI augmentation in a thoughtful way. We aren’t outsourcing
our understanding to coding assistants like Claude or Codex but becoming
software engineering centaurs by using AI tools to improve our knowledge and the
quality of our work. Join the community over on Patreon to find out about
interaction patterns that improve your work with AI coding tools, running LLMs
for software development locally, discussions of recent research in the field
and more. If you’re a software engineer who’s interested in the promise of AI
tools but sceptical about handing your skills over to the computer, this is the
community for you. Go to patreon.com slash Chiron Codex that’s
C-H-I-R-O-N-C-O-D-E-X now for more information and to join. Use the gift link in
the show notes to get your first month of insider access completely free.
Alternatively, you can show your appreciation by donating at Ko-fi. That’s
ko-fi.com slash Chiron Codex K-O-F-I dot com. Direct support by my audience is
the only revenue I get for my work as a software engineer and communicator. So
your support really means a lot to me and makes it possible for me to produce
this podcast. Thank you so much.

That was the advert break. It’s over now.

Section 5 is on large systems and we’re launched straight in with what has to be
the money quote for this episode from J.I. Schwartz of King Systems. “I must
admit that I have frequently asked myself whether these systems require many
people because they are large or whether they are large because they have many
people.”

Maybe both are true. We saw in the peak eras of Silicon Valley hiring
excess—which I define as the dot com boom that ended in 2001 and the web scale
boom that ended in 2023—that companies would hire software engineers just to try
to stop their competitors from hiring them, and would then need to find projects
for them, and would then grow systems in weird ways with weird personnel needs
that they could then cut when they needed to reduce headcount without materially
affecting their outcome. So maybe this is a kind of spatial variant of
Parkinson’s law, where the work expands to fit the headcount available.

After giving an example of a large project (the NASA Apollo mission which
featured at least 600 people from IBM with 300 programmers), J.D. Aron describes
their experience with “super-programmer” projects. Fred Brooks also writes about
this in 1971’s Mythical Man Month under the name “Chief Programmer Team”. In
both cases the super or chief programmer is Harlan B. Mills a man who with 36
PL/1 manuals open on his desk tried to replicate in six months a 30 person year
“army of ants” project. He failed at that, but still completed the project in a
small number of years. However, IBM were doing what they could to shield Mills
from their customer meaning that his more efficient project turned out to be
worse at doing what the customer actually wanted.

As Brooks presents it, the chief programmer is a highly capable expert in total
charge of the project with a few support staff to delegate some of the work to,
including: a full-time project wrangler; a support tool smith; and a programming
language expert. These teams can indeed get significant work done with fewer
resources than large teams but there are multiple factors at play. Is it the
Brooks’s law issue that there are fewer people so less time spent on
communication? This is the bet behind the two pizza team which doesn’t mention
chief programmers, just small teams. Is it that these chief programmers are
genuine 10x programmers and that the 300 person projects are full of NNPPs: net
negative producing programmers? Or is it that when someone comes along and says
“I bet I could do that in less time than your large team did”, they’ve
pre-selected the project as one that they think they can succeed at?

This is something that happened over and over again with the various Twitter
clones that have come and gone with one of the loudest being App.net. The team
behind App.net proudly announced how quickly they’d got their product off the
ground when it’s taken years for Twitter to build the same features. The two
differences of course were: one, that Twitter had to design and validate Twitter
whereas App.net just had to build an implementation of the existing design; and
two, Twitter still exists.

It’s in section 5.2, “the sources of problems in large systems,” that we find
another of the irreconcilable differences between industry and academia. The
conference has decided, just as its forerunner did, that large projects are the
source of most problems in software and there are a lot of regular software
initiatives that work just fine. So why did the large projects go wrong? On the
one hand Aron from IBM says “virtually all [reasons for failure are] essentially
management problems”. Yes there are technical problems but choosing the correct
techniques is itself a managerial problem. As Alan Perlis says, putting in his
runner-up entry for this episode’s money quote, “the managers wouldn’t know a
good technique if it hit them in the face”.

On the other hand Tony Hoare says that “basically all problems are technical”.
The reason that large projects fail is that they represent problems humanity
doesn’t know how to solve so it was a mistake for the manager to try to solve
them without first doing some basic research.

There’s a bit of presentation from working papers and discussion of the support
systems needed for large projects and the answer is basically an integrated
development and project management environment: version control, programming,
filing system, debugger, linter, job submission, time accounting, dependency
tracking, and more all in one database. It’s interesting to note that even
though IBM developed the environment they describe in this report (called
CLEAR-CASTER) the first commercial IDE was probably the rational R1000
workstation for ADA in 1985. Unless, of course, you consider the online
programming environment in Dartmouth Basic to count as an IDE.

The last section
to cover in today’s episode is section 6 on software engineering education. Alan
Perlis sets up the discussion with three questions:

1. Is there a real distinction between software engineering and computer
   science?
2. Given that the answer to question 1 is yes is there a need for education in
   software engineering as a separate discipline?
3. What form should university courses in software engineering take?

It seems like question 3 assumes the answer to question 2 is also yes so in fact
there’s only one question here. I think the subsequent discussion mostly focuses
on the perceived prestige of different degrees in the US and the broken
incentive structure of US academia, with only a little diversion into the
question of how to teach software engineering. It turns out that computer
science at the doctoral level results in a PhD which is considered a, and I’m
using my scare quote fingers here, “proper degree”. You do a lot of teaching and
a bit of original research to get a PhD in computer science. Now, is it
necessarily the case that someone who wants to become qualified in software
engineering needs to advance the state of the art? Can’t they just learn the
practice? That question doesn’t quite go unanswered, but the answers do really
circle around the issue quite a lot.

The problem American academics, at least
the ones at the 1969 conference, have with software engineering is that if you
graduate an engineering program with a doctorate in engineering in the US, these
aren’t, using my quote fingers again, “proper degrees” in the way that a PhD is a
“proper degree”. Feel welcome to email me about this point if you wish, but you
shouldn’t get angry with me, I’m paraphrasing what’s in the report and I don’t
have an opinion on the status of an American doctorate in engineering.

Bob McClure summarises the state thus. Most computer science departments output
lone workers with computer science degrees whose career capability is best
described as “can become faculty at another computer science department and
train PhDs in computer science”. Universities thrive on their reputation and
their reputation is determined by whether academics think they’re any good, so
aspirational universities will try to emulate good universities by outputting CS
PhDs who can become faculty at CS departments, ideally in good universities.

Meanwhile, industry wants a lot of people with bachelors in engineering who can
do quite good work, some people with masters in engineering who can do good
work, some people with doctorates in engineering who can do very good work, or
direct the work of those with the bachelors and masters, and no people with PhDs
in computer science who can extend the state of the art but can’t work with
other people. However, due to the aforementioned degree snobbery, students want
PhDs, which universities are happy to help them with to boost their reputation
for creating PhDs because those are the, fingers again, “proper degrees”.

The four things we learn about software engineering education in this section
are: one, practitioners don’t read any research; two, a practitioner who did
read the research wouldn’t be much of a theoretician, and a theoretician who
wrote a program wouldn’t be much of a practitioner. That division again. Three,
the textbooks aren’t worth anything, assuming that they exist at all. Butler
Lampson only has time for Knuth’s The Art of Computer Programming, and that as
an encyclopedia, not as a textbook to teach from. Four, the body of knowledge
isn’t well defined enough in 1969 for Strachey to believe there’s a whole
degree’s worth of material to teach, which probably also explains why there
aren’t any good textbooks. That last part probably was true. The IEEE Computer
Society didn’t even put out a prototype guide to the software engineering body
of knowledge until 1998.

A question that remained open was whether software engineering in itself was a
valid discipline, or whether people also needed to learn something of hardware
engineering, in which case they would be more like informaticians or
cyberneticians than software engineers. Clearly, the people in favour of the
combined discipline lost. I have an MSc in software engineering. I’ve also
taught on that degree subject. I’ve linked to the syllabus in the show notes. No
hardware in there. Now, the question is, what drove that collective decision
that society made? Did software people not want to bother with the details of
hardware? Did universities not want to add hardware modules to bloated software
courses? Did employers not care whether software people knew about hardware? Did
hardware get too complex to teach in a software course?

I did a little bit of hardware engineering in my undergraduate degree, which was
physics, not computer science or software engineering. We had a little 8-bit
processor made out of TTL chips, that’s transistor-to-transistor logic, and we
had to extend it by adding a subtract instruction. Interesting, yes, but did it
make me a better software engineer? Or, for that matter, did it make me a better
physicist?

All that’s
left in the 1969 conference report, and therefore in this mini-series on the two
NATO software engineering conferences, is the collection of working papers. This
is more than half of the report’s body, but I won’t cover the whole thing in
detail. In the next episode, I’ll share some highlights from those papers. In
the meantime, you can leave your thoughts about this episode at the post on
sicpers.info, or email me, grahamlee at acm.org. Thanks so much for listening.
Please consider supporting the podcast on Patreon or on Ko-fi. Take care, and
we’ll talk soon.

This episode is the first to discuss the 1969 NATO conference on Software Engineering Techniques. We find out that there isn’t a NATO-sponsored international institute of software engineering, though only a little about why; that computer scientists and software engineers have been ignoring each other for almost 60 years now, if not longer; and that if you want to propose a way to ensure software is correct, you can rely on Edsger Dijkstra for a snarky rejoinder.

The podcast is now available on YouTube!

The episode is supported by members of the Chiron Codex Patreon (use this gift link for your first month free), so please do join the community or hit the Ko-Fi button to make a one-off donation.

Links

• Software Engineering Techniques: Report on a conference sponsored by the NATO Science Committee
• The NATO Software Engineering conferences
• Computer History Museum — Iconic Voices of the Digital Age
• Curriculum 68: Recommendations for academic programs in computer science: a report of the ACM curriculum committee on computer science
• BCS Academic Accreditation Guidelines
• Computing Curricula 2020: Paradigms for Global Computing Education
• Research ethics committees: values and power in higher education
• The Humble Programmer
• Infer
• Issue 95: Fortran

Transcript

Hello, and welcome to episode 61 of the Structure and Interpretation of Computer
Programmers podcast. I’m Graham Lee, and this episode, which introduces the 1969
NATO Science Committee Conference on Software Engineering Techniques, is the
fifth part in a mini-series exploring the NATO Software Engineering Conferences.
The episode is sponsored by you, the software engineering community. By the way,
this episode is the first I’ve made since adding the podcast feed to YouTube.
That addition was long overdue, and if you’d rather listen over there or you
have friends who would listen on YouTube, check out the playlist that’s linked
in the show notes.

The 1969 conference, which was held in Rome, is very different from the 1968
conference in Garmisch, which has been the topic of the last four episodes of
the podcast. The 1968 conference was convened because people believed that
software development could be improved, that they didn’t know how it could be
improved, and that engineering practices might provide the answer. Briefly
recapping those episodes, the conference discovered that most software works
fine, thank you very much. The problem comes in projects that are both cutting
edge and large, and that the big ideas for fixing things are to get more and
earlier feedback about software, including interleaving testing and
implementation.

The 1969 conference had two aims. To dig deeper into the technical issues from
the 1968 conference, leaving behind the personnel and management problems that
were discussed in depth at that conference, and which continue to be some of the
biggest blockers to software engineering to this very day. That was one of the
goals, and that was the one that the report editors were most interested in. The
second aim that many attendees had was to advocate for a proposal to create a
NATO-funded International Software Engineering Institute. We know that there
isn’t a NATO-funded International Software Engineering Institute. We know from
Brian Randell’s recollections that the discussions were pretty fractious, and we
also don’t have any record, because he and John Buxton, who were the editors of
the conference report, decided, probably for political reasons, not to cover
those points in the report.

It’s this sort of thing that makes oral histories of a profession, such as those
in software collected and maintained by the Computer History Museum in Mountain
View, critical. The documented record of software engineering leaves out the
inconvenient parts, but those rough edges are where some of the pretty impactful
decisions get made.

What Buxton and Randell did record is the technical basis of the disagreement,
which forms the whole of the first section of the conference report. That is,
the distinction between theory and practice. There are actually three camps in
evidence. Theoreticians, who feel like they aren’t allowed to speak at the
conference because they’re not saying anything practical. Practitioners, who
feel like they aren’t allowed to speak at the conference because they don’t have
any theoretical basis for their claims. And Niklaus Wirth and Edsger Dijkstra,
who seem to be very happy to speak at the conference, who feel like their work
is both theoretically sound and practically useful, and that people need to stop
making the distinction between theory and practice.

The summary of the shape of this disagreement between theory and practice was
given by Christopher Strachey, who was, at the time, directing the programming
research group at Oxford. His address in the conference report is given as 45
Banbury Road, an address which has since been consigned to history. Actually, in
two senses. Firstly, the Computer Science Department is now slightly closer to
town, as it’s in two sites on Parks Road, and the building at 45 Banbury Road is
now used by the History faculty.

Strachey frames the debate with the two questions, can computing science be of
any assistance to software engineering, and what can computing science get out
of software engineering? Before we move on to how that landed with the
conference delegates, let’s play that forward and see how the industry has
responded in the subsequent decades.

Basically, my abbreviated and highly flippant take is that at the time of the
NATO conferences, the Association of Computing Machinery were not only
encouraging university computing departments to do computer science instead of
informatics, cybernetics, information theory, or related topics, but were also
defining computer science in their “Curriculum 68” to be the mathematics of
algorithms, a theoretical discipline that is so separate from practice that when
I was working in Oxford’s computer science department in 2019 as a research
software engineer, they were hiring for a senior prof, and some of the
candidates didn’t even need a computer to do their research.

For most of history, this was broadly the state of affairs. Software people
learned on the job. Some of them started with computer science degrees, but that
mostly didn’t set them apart. Software engineers were hired either through
nonsensical aptitude tests that were mostly geared towards discovering
introverted middle-class men, or programming challenges that determined
something similar to whether or not they could program because there was no
other signal to work with. This situation probably hit its peak with my
generation, the Xennials, who grew up with microcomputers and so entered the
workforce with 20 years of practical experience, and a uniform of combat
trousers and band or conference t-shirts that they adapted from cyberpunk novels.

Approximately a decade later, Generation Y had grown up with computers that were
less accessible, Salow’s paradox finally resolved itself and everybody wanted to
use computers because of the internet. Some applications suddenly became web
scale, and so the theoretical study of algorithms became the in thing. The
situation reversed, and people rushed into the computer science schools, and
rushed out into internships with the technology giants.

All of these approaches are pretty bad at generating a representative workforce,
at the time when computing finally lives up to its promise of entering all
aspects of society, or eating the world as Marc Andreessen put it. So then we
get the coding boot camp generation, where the pendulum swings back towards
practical experience over computer science theory.

Meanwhile, there have been a small number of academic institutions offering
“software engineering” courses (I have one of those degrees) and a few
researchers in computer science department doing “software engineering” research
(my doctorate is one of those). But much of the research isn’t read or acted on
by practitioners, for the straw man reason that Strakey gave in the conference
in 1969: “Well, there’s nothing we can get out of computer science, look at all
this rubbish they’re talking”. People eat up research that supports techniques
they want to use, or that repudiates techniques that they don’t want to use, but
without necessarily reading or otherwise engaging with it beyond the titles and
abstracts. Software engineering research mostly exists to get published in
software engineering journals, and cited by other software engineering
researchers, because that’s how you make a career in software engineering
research.

Computer science degrees do need to have a practical component these days to get
accredited in many places. For example, here in the UK, the BCS—full name BCS,
the Chartered Institute for IT. It used to be called the British Computer
Society—The BCS accredits CS curricula, and requires that degrees have both
theoretical and practical learning outcomes. A lot of the practical contribution
is farmed out to industry internships, or co-created projects that are in
practice led by industrial partners. Similarly, in the US, the ACM curriculum
2020 report now includes software development fundamentals in the CS degree
competencies, and two criteria for software engineering learning outcomes. Those
are reviewing a set of software requirements and designing a system in the UML.
That is two competencies out of the 56 that they list for a full software
engineering degree.

Software engineering is still taught on the job, with consultants doing a lot of
what is called coercive mimesis. That is the business of making teams all do it
one way, because that’s the way that other teams do it. Said consultants get
angry when people want to do it another way, even though the only evidence
supplied by adherence of both of these approaches, either doing it the way the
consultant wants, or doing it the way the team is currently doing it, is
survivor bias.

Okay, tangent aside, back to 1969. Strachey’s problem with the discussion of how
much software engineering can help CS, or vice versa, is that, and I’ve already
quoted this, “it’s rubbish”. Recapitulating an important point from the 1968
conference, he notes that some projects “have been quite astonishingly
successful”. The problem CS has in convincing software engineers to adopt their
techniques, and he uses recursive programming as an example of such a technique,
is that they, the practicing software engineers, “haven’t thought seriously of
doing it. They’ve been told to do it, and they brush it away, apparently because
it hasn’t got the right sort of software support, or because their machine
doesn’t do it easily, or because they don’t know about it.” This reminds me of
almost every single argument both for and against adopting TDD, and probably
applies to every other novel software engineering practice. Some people say you
should do it. Others explain why it doesn’t make sense in their specific
context. Nobody is correct.

Or, as Strachey says, a computer scientist can’t expect a huge software project
to change direction just because they say so. But the project leader can’t
expect the CS department to spin up a 500-person-year project to demonstrate the
benefits of their technique just because they say so either.

The discussion by the rest of the conference attendees of Strachey’s soliloquy
on the theory-practice divide quickly gets lost in the weeds. J.D. Aron of IBM,
sometime president of the IBM Federal Systems Centre, sometime contributor to
the SAGE automated defence platform, sometime editor-in-chief of the IBM series
on System Programming, notes that many projects go wrong because of a lack of
appropriate selection of scientifically recommended methods, when the point that
Strachey was making is that science doesn’t have a mechanism for recommending
the appropriate methods. Roger Needham at Cambridge University asks how big a
project needs to be before it can be considered an appropriate pilot project,
when the point is that even when faced with a pilot project that successfully
demonstrates a technique, people feel empowered to ignore that and make software
the way they like to do it.

Indeed, Dijkstra summarises this last point well. People welcome new software
engineering techniques and tools, as long as they don’t have to change their
thinking habits, programming tools, hardware, tasks or organisational structure.
Those listeners who have seen companies that claim to have undergone agile
transformations and are using SAFe, or who remember the gap between
object-oriented programming as promised in Smalltalk-80 and as delivered in
Java, probably need to reduce the amount you’re nodding your heads along at this
point to avoid sustaining neck injuries.

Ed David from Bell Labs suggests replacing the pilot project with research
investigations of production software projects, which is what IBM research,
Microsoft research and others did and continue to do exceptionally well. It’s
the source of almost all of today’s research into LLMs, as LLM companies tend to
have enough hardware to develop and research LLMs and universities tend not to.

And of course today, researchers can do the same using open source software
projects. Projects that represent the output of huge amounts of effort put into
making software, because people need to use the software, and which allow the
freedom for academics to demonstrate their practices in realistic, because real,
situations. Of course there are real ethical issues related to using a live
project with actual human contributors as a test bed for any old novel idea that
comes out of academia. And it’s not entirely clear that the way academics
navigate those issues is entirely great, but it is important to navigate them.

This is the advert break. It starts now.

This episode is brought to you by me, Graham Lee. But really, by you. Chiron
Codex is a community of people who are learning how to become better software
engineers by adopting AI augmentation in a thoughtful way. We aren’t outsourcing
our understanding to coding assistants like Claude or Codex, but becoming
software engineering centaurs by using AI tools to improve our knowledge and the
quality of our work. Join the community over on Patreon to find out about
interaction patterns that improve your work with AI coding tools, running LLMs
for software development locally, discussions of recent research in the field,
and more. If you’re a software engineer who’s interested in the promise of AI
tools, but sceptical about handing your skills over to the computer, this is the
community for you. Go to patreon.com slash Chiron Codex, that’s
C-H-I-R-O-N-C-O-D-E-X, now for more information and to join. Use the gift link
in the show notes to get your first month of insider access completely free.
Alternatively, you can show your appreciation by donating at Ko-fi, that’s
ko-fi.com slash ko-fi.com. Direct support by my audience is the only revenue I
get for my work as a software engineer and communicator, so your support really
means a lot to me and makes it possible for me to produce this podcast. Thank
you so much.

That was the advert break. It’s over now.

The second section of the report focuses on software specification. Right out of
the gate, the editors ignore one of the more commonly repeated points of
discussion in the 1968 conference, that the specification needs to be iterative
and rely on feedback received during development. They describe a specification
as serving as “the foundation of the subsequent system implementation”. Luckily,
the actual conference attendees were more on the ball. Discussing Seegmüller’s
IBM-derived checklist of system specification, multiple people criticised it for
not offering enough opportunities for a team to restart the specification
process from scratch as they discover new information. Scalzi, who is also from
IBM but a different office in a different country, points out that OS/360 had
been going through evolutionary and incremental development since it was
released.

Subsequent specification discussion, at least as presented in the report, is
split into two sections, specification languages and implementation languages.
I’d wager that many programmers have only really thought about implementation
languages, but it was a really big deal in the 1960s that there were two
different types of language that software teams used. It’s still in fact a
useful distinction, so it’s worth going into more detail.

I’ll start by expressing the limitations of implementation languages, so that
you and I have a shared understanding of the gap into which a specification
language could fit. An implementation language, at the time say FORTRAN or COBOL
or PL/1, but these days something like Swift or Kotlin or Rust, tells you how
the software does what it does, what instructions the computer performs in what
order (albeit in a somewhat abstracted way, at least in the modern case, a point
we’ll get back to when we catch up to the conference reports discussion of
implementation languages), and what memory locations the computer uses to store
variables. From that you can derive an understanding of what the software does,
but that information isn’t front and centre, and it’s a description that’s
focused on what transforms the computer goes through, not what the people and
systems surrounding the software go through.

A specification language then is a language in which you can express a
description of what a software system does in reaction to what the people and
systems surrounding it do, without cluttering it up with information about how
it does it. That means not only eliding information about what variables the
computer stores and operates on, but what algorithms entirely. To paraphrase an
example that Jerome Feldman, who was then at Stanford, gives in the report, you
might just say the software calculates the square root, without mentioning
whether it uses the Newton-Raphson method, or logarithm tables, or some other
approach.

The specification technique that gets most coverage in this report is the Vienna
Development Method, created by people at IBM’s office in Vienna, Austria. That
deserves its whole own episode (and if anyone has literature on the Vienna
Method that they are able to share with me please get in touch using the details
at the end of the episode). The Vienna Method is similar in scope to something
like the Z notation. You describe the data structures in the system and the
operations in the system in terms of the effects on the data structures, and you
design the system’s behaviour as a composition of those operations, or you
design the operations as refinements of a description of the system’s behaviour,
or you do both. Recall the top-down versus bottom-up arguments in the previous
conference that we encountered in episodes 58 and 60. When you have sufficient
detail to understand the design, you use an implementation language to provide
executable examples of the operations that you listed in your specification
language, the operations required to fulfil the specification. Once you have
that, in theory at least, everything works.

Programming teams in the 1960s commonly used different languages for
specification and implementation, even in situations where the specification
language they use could in principle be compiled into executable code, like APL,
which was mentioned in the 1968 report under the name of “the Iverson notation”,
or ALGOL. Part of the reason might have been not trusting or not paying for the
compiler on a particular computer system, or that compiler not even being
available, with programmers hand-compiling the specification and implementing
operations in assembler. That’s an approach that the conference attendees
bemoan, expressing what was then a forlorn hope for a machine-independent
implementation language that nonetheless produces efficient code on every
hardware configuration it supports. Well, eventually Fortran became that, and
Adrian and I discussed Fortran in the August 2026 issue of De Programmatica
Ipsum, for which there’s a link in the show notes. But the sine qua non example
is of course C, which was still a few years away from the Rome attendees, even
those of them who worked at the Bell Labs. And the way that we made sure C
compilers could produce efficient code on every hardware configuration was we
ended up redesigning all of the hardware so that it could produce efficient C.

Actually, going back to APL, Alan Perlis noted that APL and LISP are both very
good as specification languages. Because their adherents, “much like both ardent
Catholics and Communists”, understand their frameworks and provisions so well
that they can think about a system design in terms of those provisions. In other
words, if you describe a desired system to an APL programmer, they immediately
start thinking about the APL they would use to implement that system.

Of the attendees at the conference, apparently only Adin Folkoff, who’s also
from IBM and who worked a lot on APL, thought that English was appropriate as a
specification language. His argument is presented in one of the included
position papers in Section 7 of the report, so we’ll leave him alone for now and
come back to that in a later episode.

For now, let’s take a look at the next section of the report, Section 3 and
Software Quality, then we’ll call it a day for this episode. Section 3 opens
with a fascinating glimpse into the sociology of software releases. ME Hopkins,
yet another IBMer—IBM had 11 delegates at a conference of 63 people—reports that
OS/360 consistently has about 1,000 problem reports in every release. Now why is
that? He suggests some alternatives. That problem report frequency is connected
to the release cycle. That problem report frequency scales with number of field
engineers. Or that engineering management don’t feel comfortable adding new
features to the system until there are fewer than 1,000 problem reports in the
existing scope. Sadly, I haven’t seen evidence that these hypotheses were ever
tested, but that would make for a really important contribution to the
psychology of release schedules and of problem reporting in software.

Within software quality, there are two subsections on “correctness”; the first
being “formal correctness”, by which the editors mean the use of mathematical
proofs to demonstrate that software satisfies its requirements, the second being
“debugging”, by which they mean handling the fact that you were wrong about the
correctness. Niklaus Wirth observes that he designed Pascal with some of the
ideas from formal verification in mind, another of those maddeningly laconic
quotes from a working paper that we don’t get to read in full. This one, which
is called “The programming language Pascal and its design criteria”, does appear
to have been published elsewhere and people have cited it, but my university
library doesn’t have a copy. Again, if you can send it, please get in contact.

The common approach to formal correctness is the one that Tony Hoare originally
described, and that forms the basis of Eiffel’s design by contract. At every
point, list the preconditions that must be true for the next statement to work
correctly, and the postconditions that that statement guarantees to be true
after it works correctly. Your design of an operation is correct if the
preconditions for the first statement in the operation include the situation
that your customer finds themself in, and the postconditions after the last
statement has executed include the results that your customer wants, and that
the outcome of each statement is compatible with successfully executing the next
statement.

At the conference, Hoare suggests another approach to correctness in this
section of the conference report, and it’s one that includes testing. Construct
a proof by induction that if the software is correct in case k, it’s also
correct in case k plus 1, k plus 2, etc. Write a test that shows that the
software is correct in case k, and then you know the software to be correct.
This is somewhat related to what we now call “equivalence partitioning”, though
that’s more showing that the software is correct in one situation, and that
there are a whole class of situations where it behaves the same way. Of course,
mentioning tests at all brings Dijkstra out in hives, so he says “testing shows
the presence, not the absence, of bugs”. In context of responding to Hoare’s
statement (which he might not have been of course, we really have no information
on how the editors compiled the conference report), but in that context, this is
a really hollow statement. Nonetheless, it’s one that Dijkstra was particularly
proud of, so he recycled it in his 1972 Turing Award acceptance speech, “The
Humble Programmer”. And appearing in that speech has led to the quote, gaining a
life as a pithy aphorism for the ages, regardless of its veracity, or indeed its
relevance in its original context.

Onto debugging then. And Aron distinguishes between debugging, as discovering
mistakes a programmer made in their own work, and testing, as discovering
problems that occur on integrating components into a complete system. That means
within Erin’s framework that unit testing would be considered a debugging
technique, and I wonder whether, for some developers, it hasn’t entirely
replaced “traditional” debugging. It almost has for me. Most of my use of what
we call debugging tools is restricted to setting exception breakpoints, and
trying to work backwards to discover how the software got to a point where it
raised an exception, or setting watchpoints and trying to follow how the program
logic affects a particular variable. I make much less use of the live
state-changing behavior of debugger tools than I used to, instead writing tests
that explore interesting conditions, and investigating how the outcome deviates
from my expectations.

Wirth and Perlis fall out over whether “online debugging”—i.e. the thing that I
was just talking about, where you interact with a running program—encourages
sloppy thinking by making it too easy to create an almost working program, then
debug it into correctness. Wirth thinks that it does encourage sloppy thinking.
Perlis thinks that it’s an improvement over previous tools. If you have any
other examples of software engineers gatekeeping that the previous way to do
things was the One True Way, and that the new tools enable dangerous slop
because they’re too easy to use, just leave them in a pile by the door.

Debugging ends with a discussion of automatic analysis tools. This is a topic
that’s similar to the discussion of specification-slash-implementation
languages. One reason that we don’t just program in specification languages and
let the computer work out what to do (any prolog programmers who are listening
can note my email address is given at the end of the episode); it’s difficult to
write efficient solvers for turning the specification into implementation. In
other words, programming is a useful mental skill. While the same has proven to
be true of automated analysis tools, you can easily show that particular
programs are undesirable in that they crash, or run out of memory, or handle
untrusted input, or whatever. It’s proven much harder to write a tool that
demonstrates that a particular program actually does what you or your customer
desires.

There’s still, of course, loads of value in a type of analysis tool that shows a
program definitely isn’t what you want. Back when I worked at Facebook in around
2014, I went to a very enjoyable internal symposium on static analysis tools,
which was held in Sicily. The problem of static Objective-C analysis came up,
and people said that they could only demonstrate correctness within a method,
because things like method swizzling and forwarding mean you can’t trace a
program across message boundaries. I pointed out that while that’s entirely true
in theory, it mostly isn’t true in practice. Approximately 100% of Objective-C
programs don’t use those fancy runtime techniques. And you can treat the
program’s control flow as fully transparent unless you encounter one of those
constructs. The team considered that assessment, and now the Infer analysis tool
supports Objective-C. You’re welcome.

I’m skipping the one page in the report on performance measurements and
improvement, because it mostly discusses old techniques where the program has
access to various clocks, and the programmer has access to hardware monitors.
When the clock is running at multiple gigahertz, watching the blinkenlichter
flicker isn’t a feasible way to understand your software’s performance. We have
similar things these days where processors internally track events in counters
that programs can interrogate. But it’s worth leaving that to episodes
specifically on that topic, and not trying to connect it to this brief and
largely these days outdated segment of the 1969 conference.

So, that’s three out of the seven sections in the conference report. And, I
think, quite a good flavour of how the whole thing pans out. It’s still a shame
that we don’t have the content on the proposal to build Software Engineering
Institute, of course. Stay tuned for discussions on software flexibility, large
systems, education, and what actually proves to be a very interesting collection
of working papers submitted by attendees. In the meantime, you can leave your
thoughts about this episode at the post on sicpers.info, S-I-C-P-E-R-S.info, or
email me, grahamlee at acm.org. Thanks so much for listening. If you enjoyed
this podcast, please share the link with your friends or colleagues, and
consider supporting the podcast on Patreon or Ko-fi. Take care, and we’ll talk
soon.

This episode reaches the end of the 1968 conference, discussing Alan Perlis’s keynote address and submitted papers by Doug McIlroy, Edgser Dijkstra, and more. I summarise the impact of the conference on software engineering, and get ready to investigate the 1969 conference in episode 61.

The episode is supported by members of the Chiron Codex Patreon (use this gift link for your first month free), so please do join the community or hit the Ko-Fi button to make a one-off donation.

Links

• The NATO Software Engineering conferences
• The Structure of the THE Multiprogramming System
• Byte Volume 6 Issue 8: Smalltalk
• Object-Oriented Software Construction
• Object-Oriented Programming: An Evolutionary Approach
• A History of CLU
• Masterpiece Engineering
• “Crisis, What Crisis?” Reconsidering the Software Crisis of the 1960s and the Origins of Software Engineering

Transcript

Hello, and welcome to Episode 60 of the Structure and Interpretation of Computer
Programmers podcast. I’m Graham Lee, and this episode is the fourth part in a
mini-series exploring the NATO Science Committee conferences on software
engineering. The episode is sponsored by you, the software engineering community.

In this episode, I’ll take us to the end of the report on the first conference
held in Garmisch, Bavaria in 1968, as there are just two sections left to cover.
The episode ends with a reflection on the whole conference so that you and I can
leave 1968 behind, ready to emerge blinking into the bright future of 1969 and
the second NATO Software Engineering Conference, which starts in the next
episode.

Section 8 of the report includes the keynote from Alan Perlis and another
invited talk. It might seem a bit weird to bury the keynote near the end of the
report when its purpose is to set the tone for the whole conference. You’d have
to take that up with Peter Naur and Brian Randell, the report editors. Not with
me. Randell, at least, is somewhere in the same country as me, but I wouldn’t be
able to point him out.

Perlis’s point can be summarised thus. Making complex software is hard, but we
will be asked to make more software of increasing complexity before we’re given
time to understand how to simplify it. What’s the cause? Software is an
imperfect embodiment of our ideas of what it should do. Or, to use his words,
“Such shortcomings in design are probably inevitable, even in the very best
systems, and are simply consequences of the inevitable disparity between the
degree of connectivity of human thought processes and those of a programmed
system””. There are some in the world who say that Jerry Weinberg was the first
to consider human factors in software engineering in his book “The Psychology of
Computer Programming”. I’d argue that Alan Perlis got there first.

In Perlis’s view, software is a linguistic exercise. We describe our problems
and our innovative approaches to solving them, and we describe them in a way
that instructs the computer to carry out the solution. Therefore, our problems
in software are caused by the ease with which we can explain complex things
linguistically, and our solutions need to be to create software tools that make
it easier to express complex things in the language of the computer.
Specifically, they need to make it easier to demonstrate equivalence. The
description of the solution as written and the solution as executed by the
computer need to be the same, and software engineers need the ability to
convince themselves that they are the same.

How to do that? Hierarchies of virtual machines. Perlis again. “The
establishment of relevant states, their transformations, the design of
communication channels, the nature and magnitude of storages, the natural sets
of operations, the I/O problem, etc.”

Those people who have followed my writing through my deep dive into the world of
small talk in about 2013, and subsequent publication of OOP The Easy Way, link
in show notes of course, are probably screaming, “this is object-oriented
programming!” at their Overcast app right now. Indeed it is, but only because
it’s structured programming, and OOP is also structured programming. Yes,
object-oriented programming is all about separate virtual computers running
isolated programs, and the ma, the interstices between these programs, realised
as messages that the computers send to each other. But go back to Dijkstra’s
description of hierarchical layers from the THE multiprocessing system that he
presents in section 9, which we’ll look at later in this podcast, you see the
same thing: independent programs that use the primitive operations available to
them to provide a different set of higher level operations, which higher level
programs use as their own collection of primitives.

Perlis concludes by summarising the idea that we’ve seen throughout this
conference, that the biggest problems come from the mismatch between the
ambitions of programmers, or customers, and the top end of the programmer’s
capability. He was vocal in the discussion of software engineering education
that we covered in the last episode, and he lays part of the blame for the
capability gap at the feet of “the unevenly trained personnel with which we
work”.

The other invited talk is from Doug McIlroy, and it’s about mass-produced
software components. This is a position and argument I know best from Brad Cox’s
1980s and 1990s writing on the “software industrial revolution” and “software
integrated circuits”, and it’s related to ideas that Joe Armstrong of Erlang
fame discussed too. But McIlroy makes it just as forcefully and coherently here,
and does so before the other two back in 1968.

The problem, as all three of those authors state it, is that software is a
cottage craft. We create systems from whole cloth every time, each being
idiosyncratic, wrong in different ways, and difficult to reuse, even where other
people are solving the same or similar problems. Much better would be to create
standardised catalogues of software components that have compatible interfaces.
You read the datasheets looking for the component category with the behaviour
you want—McIlroy uses the example of a sine function—and then you compare
specifications, I/O expectations, time memory trade-offs, data storage
requirements, and so on, to find the component that best suits your needs,
purchase one of those, and plug it in. As Nauer points out in the discussion,
this is a whole social shift as well as a technical one, and software creators
would need training to be component pluggers who make rational buy-build
decisions more than they’d need to know how to write low-level components.

My long-held belief is that the two innovations that made the most headway into
creating a software industrial revolution weren’t anything to do with design or
modularity techniques, or programming systems that provide reliable interfaces
like Eiffel, or generic interfaces like Ada. They were the web, and the
permissive open-source software licence. We still don’t have a
component-oriented marketplace in things like the Node Package Manager, Python
Package Index, or even Comprehensive Perl Archive Network. We have brand
loyalty, and components with unstable interfaces (recall the discussion on
semantic versioning in episode 59). But making discovery free and reuse free has
done more to shift the buy-build decision point than any other intervention.

Those then are the two invited talks. The remaining section of the report is a
subset of the submitted working papers at the conference. Most of these are
quite dry reading in 2026, even if they do provide a lens into what computer
programming looked like back in 1968. In episode 58, I covered the
“Classification of Subject Matter” paper, while I was trying to work out the
difference between software design and software production. What I didn’t cover
there and do here is a collection of asterisks used to indicate which activities
the working group considered the most important. This list is mostly interesting
for the things it skips rather than the things it covers. Estimation, capacity
planning and workforce allocation all aren’t given asterisks and nor is
configuration management. Nor is feedback to design, which we saw in episode 57,
was considered both critical and fatally lacking in the conference discussion.
The procedure for generating, maintaining and modifying the system does get an
asterisk, but the set of test cases and results did not. Even though an upcoming
working paper from Llewellyn and Wickens tells us how important and flawed were
contemporary approaches to customer acceptance testing.

Bemer submitted a “Checklist for Planning Software System Production”, produced
in August 1966. Even then, he considered certain computerised management systems
table stakes that aren’t even universally used today. Field reporting, by which
he means issue tracking, production control, automated software production,
customer roster, i.e. customer relationship management, file maintenance of
source programmes, by which he means version control.

He also suggests rotating programmers into field support or operations teams,
and promoting operations staff and other personnel to programming. This is one
way to get that feedback to design that so many software initiatives lack. I’ve
seen it work well at a number of places that I’ve worked, either through an
on-call rotation or fully seconding a programmer to the support team for a month
at a time. I’ve also seen it go very badly when the on-call has far too many
fires to fight, and that’s useful feedback for your design in itself. Another
interesting point on his checklist is the question, are hardware manuals
forbidden to exist separately for users, so that the system is described in
terms of the software system? These days that’s almost universally true unless
you’re in a hobbyist community like Arduino. Your manual for your phone tells
you where the power button is, and then how to use the operating system. It
doesn’t tell you which data lines the camera detector readout gets streamed to.

Next up is Dijkstra’s hierarchical approach to design, which we trailed earlier,
with the THE multi-processing system. This paper got critiqued in the discussion
on software production, covered in episode 58, for being more about proving the
design correct than actually about coming up with a design. It introduces the
hierarchical approach of designing systems that expose facilities for other
programs to talk to. Dijkstra defines the height of a subroutine as one more
than the highest other subroutine it talks to. But there’s no stricter layering
than that. A layer 2 routine can use operations from both layers 0 and 1. It
isn’t forced to only communicate with layer 1 programs. Dykstra wrote a longer
paper on the design itself, called “The Structure of the THE Multiprogramming
System”, link in show notes, and I have that on the backlog for a future episode.

Then we have Stanley Gill’s “Thoughts on the Sequence of Writing Software”,
which introduces the top-down and bottom-up approaches to design; the
distinction being discussed in episode 58. He considers that each module has two
views. The implementation view, which describes the facilities it uses from
lower layers, and the interface view, which describes the facilities it provides
to higher layers. In this, his approach parallels Dykstra’s, though without the
explicit layer cake division. Gill doesn’t make it clear that layer versions
aren’t allowed, and actually, Dijkstra only does that implicitly, through his
definition of how to count layers. Gill suggests that the interface between each
layer should be a programming language, which exposes the operations from the
below layers in ways that are easy to consume in the next layer. Imagine a
system in which the kernel is implemented in one language, operating system
services in another, foundational libraries in another, applications in another,
and graphical user interfaces in another. Actually, I suppose that is how web
applications really work, so clearly this design is workable.

A paper by report editor Brian Randell summarises these views on software design
and makes an important point that I hadn’t quite empathised with because it’s
not how software works to me in 2026. In 1968, you might be designing a
hardware/software system together, so you might choose whether certain
facilities are implemented in hardware or in software. Dijkstra’s approach to
design is bottom-up, insofar as the hardware system was already complete, so he
had to start with the hardware capabilities fully defined and work upwards to
the software he wanted.

In other cases, you might decide that you get better performance but higher cost
by implementing a feature in hardware, and then make a decision based on the
specific needs of the integrated system. For most modern software applications,
the design is so far abstracted from the hardware that you only design the
software system. Indeed, hardware manufacturers regularly, though perhaps
infrequently, change, processor architectures, component interfaces, and so on,
and the software carries on working. Software and hardware development are, in
many cases, so isolated from each other, and hardware capabilities make up for
so many of the shortfalls in software design, that many software project teams
accept the entire stack of processor, hypervisor, operating system, container
host, programming language, and networking framework, then designers start to
think about how to build the software from there.

A little thought experiment to test that assertion. If your project is a web
application, or what we used to call a “Rich Internet Application”, that is, a
native app that acts as a client to web services, have you ever considered
whether HTTP is actually the most efficient networking abstraction for your
data? Have you evaluated any alternatives?

Okay, I promised a paper on the shortfalls of acceptance testing, and here it
is. The authors come from the UK government, and their problem is one that still
broadly exists today. As a customer, you don’t get to decide whether the
software is correct or not until you get the software, by which time it’s very
expensive for everyone involved to make any changes. The solution we have in
AD2026 to address this problem is “continuous delivery of valuable software”.
And so now you see the opposite problem. Customers who complain they don’t have
time to continually accept and test early buggy versions of software, they just
want the working thing when it’s complete, thank you very much. There’s no
pleasing some people, and apparently those people use computers.

The solution as proposed then was a detailed collection of tests of different
artefacts as they were ready. So you can test the documentation to check whether
you’re going to get a system that does what you need, and that people can use
when you have the documentation, and without needing to have the software too.
When you get the software, you can test its behaviour against the accepted
documentation, rather than going all the way back to the spec. Performance
testing does need to wait until you have the actual software running on the
actual hardware. Although another working paper on software testing does point
out the value of simulation, at least for internal testing.

This is the advert break. It starts now.

This episode is brought to you by me, Graham Lee. But really, by you. Chiron
Codex is a community of people who are learning how to become better software
engineers by adopting AI augmentation in a thoughtful way. We aren’t outsourcing
our understanding to coding assistants like Claude or Codex, but becoming
software engineering centaurs by using AI tools to improve our knowledge and the
quality of our work. Join the community over on Patreon to find out about
interaction patterns that improve your work with AI coding tools, running LLMs
for software development locally, discussions of recent research in the field,
and more. If you’re a software engineer who’s interested in the promise of AI
tools, but sceptical about handing your skills over to the computer, this is the
community for you. Go to patreon.com slash Chiron Codex, that’s
C-H-I-R-O-N-C-O-D-E-X, now for more information and to join. Use the gift link
in the show notes to get your first month of insider access completely free.
Alternatively, you can show your appreciation by donating at Ko-fi, that’s
ko-fi.com slash Chiron Codex, K-O-F-I dot com. Direct support by my audience is
the only revenue I get for my work as a software engineer and communicator, so
your support really means a lot to me, and makes it possible for me to produce
this podcast. Thank you so much.

That was the advert break. It’s over now.

There you are, we made it. Okay, I didn’t read through the appendices, but we’ve
been through the whole of the body of the report into the 1968 NATO Science
Committee Conference on Software Engineering. We discovered that a lot of ideas
that later became mainstream parts of software engineering including continuous
feedback, testing early, and using testing as input into design were already
being talked about by the attendees at the conference.

For me, this is evidence that a conference wasn’t hugely influential. People
adopted the words software engineering and software engineer, but these are now
defined in a mutually tautological loop like the name of the herd operating
system. A software engineer is someone who does software engineering. Software
engineering is the thing that software engineers do. The calls for education
curricula and university-level standards of software education basically went
unheeded, and schools went down the route led by the ACM of defining computer
science curricula that are abstract mathematical studies, and only recently paid
more than token lip service to the construction of working software.

There definitely isn’t a NATO software engineering programme that can trace its
formation back to Garmisch 1968, or count any of the conference’s attendees
amongst its faculty. The only sizeable academic software engineering centre is
Carnegie Mellon’s Software Engineering Institute, which is indeed funded by the
federal United States Government’s Department of Defence, but which wasn’t
instituted until 1984. Hardly a continuation of the 1968 initiative, and more a
sign that NATO still hadn’t sorted out its software issues 16 years later. The
second conference, which is the topic of the upcoming episodes of the podcast
starting with episode 61, shows why there wasn’t an SEI any earlier.

The low rate at which the report is cited, and the low quality of those
citations, is further evidence for its limited impact. As Haigh argues, people
will typically invoke the conference as the source of the “software crisis”
narrative, even though the phrase “software crisis” doesn’t appear at all, and
the use of the word crisis is in the sense “most of the industry isn’t in
crisis”. As Haigh argues, this seems to have been mostly a one-man effort driven
by Dijkstra to formalise the mathematical basis of software construction, which
other people found beneficial to sell their cure-all solutions to the software
crisis.

That said, while the conference and its report might not have changed how most
people made software, I think there’s a range of different lifespans for the
ideas in the conference. The ones I mentioned earlier in this summary—test-first
development, continuous feedback, close collaboration with customers—these seem
to have fallen out of fashion and got rediscovered later, mostly by the people
who were working with Smalltalk in the 1990s.

Some of the others—hierarchical modules with clear interfaces, virtual machines
and bytecode interpreters, closed interfaces that permit extensibility in other
modules—these seem to have been evergreen ideas that were already accepted by
some of the attendees at the conference, and that had to break out of their
ivory towers to get broader adoption in the software industry.

Simula is mentioned in the report as an example of the object or Plex approach
to designing software, and its ideas were perpetually reused and rediscovered
until Smalltalk broke out in the famous Byte magazine issue, link in show notes.
The Eiffel approach to interface design, the first place where the open-closed
principle got formally written down as a design principle in its own right,
despite appearing a number of times in the conference report. While coincident
with the OO revolution, and definitely object-oriented in style, owes its
heritage not to Smalltalk, but to existing ideas in modularity, correctness and
fault isolation, following from work like Liskov’s on abstract data types in the
1970s, and Eiffel got a popularity fillip from the association with Objects in
the 1980s.

Episode 61 of the podcast will continue the NATO conference miniseries by
looking at the first sections of the report from the second conference, held in
1969 in Rome, Italy. As Brian Randell recalls, “In Rome, there was already a
slight tendency to talk as if the subject already existed, and it became clear
during the conference that the organisers had a hidden agenda, namely that of
persuading NATO to fund the setting up of an international software engineering
institute. However, things did not go according to their plan. The discussion
sections, which were meant to provide evidence of strong and extensive support
for this proposal, were instead marked by considerable scepticism, and led one
of the participants, Tom Simpson of IBM, to write a splendid short satire on
masterpiece engineering.” I’ve linked the masterpiece engineering piece in the
show notes, as it wasn’t included in the final conference report.

Join me again next time, and we’ll explore what went wrong. In the meantime, you
can leave your thoughts about this episode at the post on sicpers.info, or email
me, grahamlee at acm.org. Thanks so much for listening. If you enjoyed this
podcast, please share the link with your friends and colleagues, and consider
supporting the podcast on Patreon or Ko-fi. Take care, and we’ll talk soon.

We’re closing in on the end of the 1968 conference report, in this section discussion software service, maintenance, and other “special topics” including educating software engineers, and whether it’s reasonable to pay for software at all. Along the way, we discover that there’s no silver bullet 18 years before Fred Brooks told us; decide whether 1968 software needed more blockchain; and find the horrific truth behind beta testing.

The episode is supported by members of the Chiron Codex Patreon (use this gift link for your first month free), so please do join the community or hit the Ko-Fi button to make a one-off donation.

Links

• The NATO Software Engineering conferences
• What’s Better than Semver?
• Software release life-cycle
• Alpha, Beta, and Sometimes Gamma
• Dilbert’s Team Writes a Minivan
• The Defect Black Market
• Shift-Left Testing
• Did Grace Hopper Create the First Compiler?
• Principles Behind the Agile Manifesto
• Superdistribution: Objects as Property on the Economic Frontier
• Crisis? What Crisis? Reconsidering the Software Crisis of the 1960s and the Origins of Software Engineering
• The Lean Startup
• No Silver Bullet: Essence and Accident in Software Engineering
• Should Software Engineers be Licensed?

Transcript

Hello, and welcome to episode 59 of the Structure and Interpretation of Computer Programmers podcast. I’m Graham Lee, and this episode is the third part of a mini-series discussing the 1968 and 1969 NATO Conferences on Software Engineering. It’s sponsored by the members of my Patreon, which can include you.

We’re up to sections 6 and 7 of the report, which cover software service, that is, the business of satisfying customers by delivering software, particularly maintenance, and special topics, which don’t fit into the subjects of the three
workgroups. Service was one of the workgroups, with the others being design and production, the topics of the previous episode of this podcast.

We join the service section with a subsection that has the provocative title, The Virtue of Realistic Goals. Essentially, nobody at the conference blames the programmers for failed projects. Either the customers or the users had unrealistic expectations, or the manufacturers made unrealistic claims about their system’s capabilities.

Klaus Samuelsson is particularly harsh in blaming the users, saying it’s their fault for accepting a system before they’ve satisfied themselves of its correctness. Brian Randall, who edited the conference report, agrees, saying, “the users are as much to blame for premature acceptance of systems as the manufacturers for premature release”. But what choice do customers have?

Indeed, even in this day and age, we only have a partial solution to this problem. There’s free software, where you can inspect the code, or get someone else to do it, and know it’s correct, then choose whether you pay the creators, and of course many people don’t. Or there’s shareware, where you can use the software and check it roughly works the way you want, or at least the trial features that you have access to do, and then pay to unlock the rest, which you haven’t been able to try.

Every other distribution mechanism or purchase model for software is either buy now or regret later, or buy now and hope that every back-end update keeps the bits you need working the way that you need them to work.

This, by the way, also came up in the report, where d’Agapeyeff says it’s generally a problem that we haven’t worked out how to make sure that any release of software is a strict subset of later releases. In other words, that newer versions can do more than previous versions, and do all of the existing things in the same way that the previous versions did.

I wrote a blog post back in 2018, which is linked in the show notes, about the way in which semantic version encodes the intention to introduce breaking changes between versions. I proposed meaningful versioning in which the smallest increment number, the z in x.y.z, applies to additional features, the middle increment, the y, applies to behaviour-preserving refactorings, and the largest increment, the x, to bug fixes. There’s no room for backwards incompatible changes in this scheme, unlike in semantic versioning. If you want to do that, you should release a different product.

This idea is related to the discussion of the open-closed principle that we had in episode 58. The whole software system should, according to d’Agapeyeff, be open to extension, you add capabilities in subsequent issues, the word he uses for releases of the software, and it should be closed to modification. The bit you’ve already released ought to work well enough
that you don’t need to change it. I should insert a placeholder call-out here to a subsequent episode of this podcast that I haven’t planned or recorded yet on Bertrand Meyer’s book, Object-Oriented Software Construction, because the idea of open and closed is all over this 1968 conference.

Next up are discussions on the state of the initial release of software and the frequency and nature of subsequent releases. The general sentiment is that the initial release should work well, even if it doesn’t have all of the planned capabilities. Growing outward from a high-quality core, preferably by adapting a modular design, is better than releasing a poor-quality version of everything.

However, François Genuys points out that people need to access pre-release versions of the software for training. On the one hand, I thought that this could refer to the interim systems that the conference discussed in the design and production sections, which we talked about in episode 58 of the podcast, where some of the components are real and the others are simulations. In this way, customers or support staff train with working systems, just not completely working. In the same way that the subsequent initial release is a high-quality subset of the total system behavior, so the pre-release training versions would be high-quality subsets of the initial system behavior.

Then I remembered that Genuys was at IBM, and that the idea of alpha and beta tests supposedly come from IBM, so I wondered about the timing of that. Wikipedia suggests that the terminology came from the 1950s, but it offers two citations, neither of which actually back up that claim. Jeff Atwood, at his coding horror blog, corroborates the IBM origin of these terms, but he does so by citing the Wikipedia article. Everybody else either cites the Wikipedia article, or plagiarizes it, or plagiarizes Jeff’s post, or plagiarizes both of them. So I think we’re stuck here. Unless anyone who’s listening has a contemporary source, in which case please do send it in and let me know in the comments, we don’t actually know where the terms alpha and beta testing come from.

Not that it’s important. The idea that you have pre-release alpha or beta tests doesn’t mean that you let people use the versions that fail those tests, nor necessarily that your testing criteria at those times are any less stringent than those for the initial release or for subsequent system releases. Speaking of the subsequent system releases, there’s a bit of tension over how frequently these should appear. Ashiroplar sums up the tension well. More releases means more churn, but it also means getting corrections into the hands of customers sooner.

Generally, people at the conference are in favour of fast corrections and infrequent major upheaval updates. This puts me in mind of my experience managing Debian systems, where it’s easy to accept the in-release updates, that is apt-get update and apt-get upgrade, without worrying that anything will break. And infrequently, you have to cross your fingers and do a dist-upgrade.

Or even the times when I’ve managed Solaris systems, you take the stream of minor updates forever and never reinstall the major version of the operating environment. It also calls to mind Microsoft’s Patch Tuesday approach, releasing interim updates at predictable times, so that administrators can prepare to deal with their installation.

This section on release frequency ends with an extract from Control Data Corporation’s H.R. Gillette on defining metrics for release quality. Here’s the quote.

“Below, I have written a copy of one of the paragraphs which has been put into a product objectives document. We struggled a great deal to define measurable objectives in the document, and this is an example. The numbers used do have relevance historically, and that is all that need be said about them. Finally, our objectives may not have been high enough in this particular area. We tried to push our luck while at the same time being realistic.

The total number of unique bugs reported for all releases in one year on ECS SCOPE will not be greater than the number given by the following formula. Number of bugs is less than or equal to 500 minus 45 divided by in brackets I plus 10 close brackets, where I is the number of installations using ECS SCOPE. 85% of the reported PSRs will be corrected within 30 days, and 50% of these will be corrected within 15 days. All PSRs will be corrected within 60 days.”

In that quote, a PSR is a bug report. What this is trying to get at is laudable. There won’t be many bugs, even if we have lots of different customers, and we’ll fix the bugs quickly. Unfortunately, measuring bugs reported is one of those meaningless metrics that’s easy to parody, and indeed Scott Adams covered this one in a Dilbert comic back in the 1990s, when the software team write themselves a minivan.

You game this metric by choosing quiescent customers who don’t report bugs, or by including obvious defects like typos that are quick to fix, so that you achieve your turnaround time goals, and customers don’t have time to fill out their PSR forms with more meaningful bug reports.

More recently, software engineers have finally read a 2001 article from Dr. Dobbs’ journal, and are shifting tests left, incorporating test design and implementation throughout the development process, and particularly from the beginning. This brings me on to the money quote from this section of the conference report, from Alick Glennie, who’s the inventor of AutoCode, an early family of programming languages and compilers, which may even have included the world’s first compiler, though that is a contested claim.

The quote is, “Software manufacturers should desist from using customers as their means of testing systems.”

Okay, me speaking again now. It’s better to see customer bug reports as a feedback mechanism than as a goal. You’ll always get them, as long as you have customers, and you have a reporting channel. But you don’t need to optimise for or control them. If you get a lot of bug reports for some module, that might indicate quality issues, or that it’s really popular. If you get not many bug reports for some module, that might indicate high quality, or that nobody uses it. And perhaps the reason that nobody uses it is because it’s too buggy.

As with the rest of this section of the report, the important thing to worry about is how you get working software delivered to your customers. Back in 2001, some people even said that this should be our highest priority.

I originally skim read the next sections, which were on replication and distribution of software, because they’re problems that don’t exist anymore. You put your software on the internet and distribute it for somewhere between zero and near zero cost, or you don’t distribute it at all and let people access it on your computer via their browsers. Gone are the days of the hologrammatic Windows XP CD, with its licence key that’s long enough to identify each atom in the universe uniquely.

But even back in 1968, duplicating software was so much cheaper than duplicating hardware, that Brian Randell recognised that economics is a reason that software quality was given less consideration by manufacturers than hardware quality. It’s that much more expensive to fix a hardware problem in the field.

And I have a vague recollection that at some point, Sun Microsystems swapped the SCSI IDs for recognising which drive is addressed by which number zero and three between machine architectures. And that might have been between the Motorola 68000
and the SPARC. And while it was possible for customers to switch some jumpers around to get back to the original behaviour, they did offer to send field engineers to customers to make the changes for them. But I can’t find documentary evidence for them doing so, so maybe it was just a scary story told at sysadmin camp when I was a young initiate.

A particular problem that distributing software on physical media had was ensuring correctness. If a tape even had one bit flipped or removed, it was useless and potentially dangerous if the software still ran, even though it was incorrect. In principle, most software that’s distributed electronically now has all sorts of digital signatures and checksums. And in practice, that’s wisely hidden from the customer. So you kind of have to trust the vendor that everything checks out.

Moving on to maintenance. And again, the blame for getting it wrong falls squarely with the customer. Each maintenance depends upon the proper recording of programming errors by the user and upon the quality of such records, says Mr. H. Köhler of AEG Telefunken. We know this not to be reliable, and so now we use telemetry and automated diagnostics to get the information we need in the form we need. It turns out that each maintenance depends upon the proper recording of programming errors by the programmer, and upon the quality of such records, but also upon the programmers choosing to act upon the records.

And also to a large extent, it depends upon the error reports actually going to the right place. In the 1960s, this would have been a simple problem. The customer blamed IBM for any bug. IBM blamed the customer’s local applications or modifications. Eventually, one or other of them either fixed a problem or worked around it. IBM didn’t unbundle their software from their hardware until 1969. We’ll see this discussion play out in real time later in the episode.

And there were only a handful of ISVs in the 1960s, mostly concentrating on filling in gaps in hardware vendor offerings. For example, Ken Kolence, one of the attendees at the conference, founded a company called Boole & Babbage, which created
profiling software.

Once you got more integrated software from more vendors on a computer, it became harder to decide who to blame for a problem. Is it Valve’s fault that Steam crashes on Windows or Microsoft’s? Or is it the fault of some third-party vendor because the customer installed a haxie that loads code into the app? In the 2000s, I worked for an antivirus software company, and we all got all of the bug reports from all of the software. Either the customer blamed the antivirus company because they didn’t like our software, or the software vendor blamed the antivirus company because they saw our kernel extension was loaded and used that as an excuse not to investigate their own customers’ problems.

I therefore spent a lot of the time as I was on support demonstrating that other people’s software crashed without the antivirus software installed, in the same way that it crashed with the antivirus software installed, so that I could send the crash report back to the company whose software crashed. In one instance, we had a report that Excel on Mac OS X crashed with our antivirus installed. Eventually, I, along with two people from Apple, a file systems engineer and a technical support professional, showed that Excel crashed on Mac OS X without antivirus installed because a rarely used code path in the spreadsheet application caused it to try to use an unimplemented feature in the HFS+ file system. Now, is it Microsoft’s problem that Excel does something that doesn’t work, or is it Apple’s problem that they exposed an unimplemented API? Thankfully, answering that question became somebody else’s problem about 20 years ago.

Just before we move on to the section on special topics, there are a couple of interesting talking points in part of the report that’s on acceptance testing. One is a suggestion from James Babcock, who ran a time-sharing services company, that we need software meters analogous to the present hardware meters so that our rental costs can be adjusted to allow for time lost through software errors as well as hardware errors. I would certainly welcome the rebates I’d get if some of the cloud computing services I use multiplied their subscription fee by their uptime ratio.

I’m going to mention Brad Cox’s 1996 book, Super Distribution, here as well. He invented a pay per use model for software pricing, and it’s a transitive pricing model. I pay for the application software I use as I use it. The application vendors pay for the library calls their software makes whenever it makes them, and so on. This model required special hardware to do the digital rights management in the 1990s, but I think now it could actually be a reasonable application
of a smart contracts blockchain like Ethereum.

In the Super Distribution model, I would automatically get money off during an outage, because I wouldn’t be able to use the software at all, so I wouldn’t be able to get charged for anything.

The other point on acceptance testing is a difference of opinion between Edsger Dijkstra, “testing is a very inefficient way of convincing oneself of the correctness of a program”, and Mr A. I. Llewellyn from the British Government’s Ministry of Technology: “Testing is one of the foundations of all scientific enterprise. In fact, it would be good to have independent tests of system function and performance published.”

This is the advert break. It starts now.

This episode is brought to you by me, Graham Lee. But really, by you.
Chiron Codex is a community of people who are learning how to become better software engineers
by adopting AI augmentation in a thoughtful way. We aren’t outsourcing our understanding to coding
assistants like Claude or Codex, but becoming software engineering centaurs by using AI tools
to improve our knowledge and the quality of our work. Join the community over on Patreon to find out
about interaction patterns that improve your work with AI coding tools. Running LLMs for software
development locally, discussions of recent research in the field, and more. If you’re a software
engineer who’s interested in the promise of AI tools, but sceptical about handing your skills over
to the computer, this is the community for you. Go to patreon.com slash Chiron Codex, that’s C-H-I-R-O-N-C-O-D-E-X,
now for more information and to join. Use the gift link in the show notes to get your first
month of insider access completely free. Alternatively, you can show your appreciation
by donating at Ko-fi, that’s ko-fi.com slash Chiron Codex, K-O-F-I dot com. Direct support
by my audience is the only revenue I get for my work as a software engineer and communicator,
so your support really means a lot to me and makes it possible for me to produce this podcast.
Thank you so much.

That was the advert break. It’s over now.

OK, we’re on to the section on special topics, which opens with software, the state of the art. We’ve actually already encountered most of the discussion points here, particularly the idea that most of software works very well, that people are doing what they need to at a much lower cost than ever before, and it’s only the edges of the field’s capability, both in terms of scale and novelty, for example, time sharing, where the problems arise. These were all in the executive overview at the start of the report that we discussed in episode 57.

This idea that people are doing what they need to do at a lower cost than ever before, though, is hard to square with Robert McClure’s assertion that “it seems almost automatic that software is never produced on time, never meets specification, and always exceeds its estimated cost.” He describes the causes as coming from “the refusal of industry to re-engineer last year’s model, from the inability of industry to allow personnel to accumulate applicable experience, and from emotional management.”

And that certainly all aligns with my own experience, having seen the phrase “rewritten from the ground up” used as if it’s a good thing, having experienced layoffs and limited career development prospects that limit retention, and management fads that come and go like high street clothing collections. However, this position is out of alignment with the rest of the conference report.

I think we see here the division that Thomas Haigh identified between the industry software engineers who think things are going well and would like them to go a bit better, and the academics who think that all of industry software is on a hiding to nothing until it adopts current academic practices.

Whatever the magnitude of the problem, somewhere between 1 and 100% of software projects running into difficulties, possible solutions were discussed. Ascher Opler suggested two approaches, either stealth mode, where the manufacturer doesn’t say anything about the capabilities of the system until they finish developing it, or loose promises mode, where they say what they’re doing but give a really long lead time and be honest about the uncertainty involved.

The subsequent third way that modern software engineers use is the lean startup approach, where the manufacturer says what it’s doing and then gets early feedback before it even starts building anything. It avoids both the risks of stealth mode, which are building something that nobody wants, and loose promises mode, where the risk is getting resumpt by someone who implements your plans faster than you do.

Going back to the theme of things that were subsequently rediscovered by somebody else but that already existed at the time of the NATO conference, Doug Ross is the only person in the conference report who actually refers to the contemporary state of affairs as a crisis. He warns against people who promise a breakthrough, a mere 18 years before Fred Brooks agreed that
there is no silver bullet in software engineering.

A second special topic in section 7 of the report is education, and Alan Perlis sets out criteria to define a curriculum in software engineering education. It’s useful to note his point that this is distinct from computer science education, as, according to him, “most of the computer science programs are producing faculty for other computer science departments”. This is actually a deliberate choice that the curriculum committee at the ACM made, choosing to focus on computer science as an academic and mathematical pursuit, rather than on software as a practical industry. But Perlis is damning in his assessment. “You have to look hard in a computer science department to find anything that is dedicated to utility as a goal”. Ouch.

Dijkstra produces the money quote for section 7 in this discussion on education. “You are right in saying a lot of systems really work, that is our glimmer of hope. But there is a profound difference between observing that apparently some people are able to do something, and being able to teach that ability”.

We could imagine this as being his way of digging in when the crisis narrative got debunked. Yes, everybody can make working software, but maybe they’re doing it wrong anyway because they don’t do it the way that I like.

One of the questions that managed to keep op-ed writers employed for decades after this conference was the extent to which software engineering and computer engineering share commonalities with, well, with engineering. This is a topic that I read deeply for my PhD thesis background, so I could go into way too much depth here. But suffice it to say that people are still discussing whether software engineers should be licensed engineers. And in fact, there are some places where they do need to be, and so in those places, people who write software just don’t use the word engineer.

One argument that was made in 2002, and seems particularly weak, is that engineering licensing would cover non-software disciplines, and it would be unfair to stop someone practicing software just because they don’t understand fluid dynamics.

The final topic we’re going to consider in this episode is the question of software pricing, i.e. whether software should be unbundled from hardware and sold as a separate product. We all know how this played out. Software was unbundled from hardware, became a huge economic engine in its own right, and even ended up eating its own tail when cloud computing changed the economic calculus so that hardware needs are factored into the software costs.

It seems like most of the attendees at the conference were in favour of software pricing, but the section in the report is presented neutrally with equal weight given to both sides. Tellingly, this is also the only section in the report that uses the Chatham House rule, where no quotes are attributed to named speakers. So, while we do know that most people at the conference were in favour of separate software pricing, we don’t know who or how many people were making the argument against.

If I had to guess, I would say that IBM representatives were against unbundling software, and everybody else was in favour of it, and that IBM lost the argument very shortly after. This was partly the work of a company called ADR, who have the distinction of being the first company ever to file a software patent. ADR brought an anti-monopoly case against IBM, saying that providing their software for free was stifling the market. This is, of course, an argument that came up again in the 1990s, with Microsoft bundling their browser and media player with Windows, and again very recently with the European Union’s Digital Markets Act and its definition of some services provided by large, typically American companies as gatekeepers. But let me know what you think. And also, your perspectives as people who rely on software being a commercial commodity, what do you think of the way that software is priced? You can email me at grahamlee at acm.org, or you can comment on this post, the post for this episode, over at sicpers.info slash podcast. That’s s-i-c-p-e-r-s dot info slash podcast.

The next episode will conclude the reading of the 1968 conference report by covering the keynote address and the working papers that are included in the report. Only a fraction of the submitted papers actually appear in the report. There’s no full proceedings, so a lot of the information that went into the conference is sadly lost forever, unless some attendee happened to file away their copies of the papers that they received. Until the next time, take care, and I’ll talk to you soon.

This episode digs into the problems of software design and software production as perceived in the 1968 conference, most urgently: just what are software design and software production? The episode is supported by members of the Chiron Codex Patreon (use this gift link for your first month free), so please do join the community or hit the Ko-Fi button to make a one-off donation.

Links

• The NATO Software Engineering conferences
• Choose Boring Technology
• Notes on the Synthesis of Form
• Notation as a Tool of Thought
• Nasal Demons
• From Newspeak to Cyberspeak: a History of Soviet Cybernetics

Transcript

Hello and welcome to episode 58 of the Structure and Interpretation of Computer Programmers podcast. I’m Graham Lee and this episode is the second part of a mini-series discussing the 1968 and 1969 NATO conferences. It’s sponsored by the members of my Patreon, which could include you.

In part one I reviewed the context of the first NATO software engineering conference in Garmisch which is in Bavaria in Germany, and approached the end of section three of the conference report with no clear idea—because the people in the room hadn’t agreed on one—which activities comprise design of software and which comprise the production of software.

Well, as this episode focuses on sections four and five which are about design and production it’s time for me to confidently tell you that I still don’t know what those terms mean. There’s a large extract in section 3.2 from a Mr. J. Harr of Bell Labs, which is the place in which a year later Unix would be invented. In his paper “the design and production of real-time software for Electronic Switching Systems, for which application a year later Unix would be invented.

In this paper the design process covers everything from specifying the overall hardware software system through division of the software into precisely defined blocks with defined interfaces and data structures, the compilation, simulation and testing of those blocks, integration into a software product and final load testing.

Of interest is that about 13% of the effort on the ESS project is on assemblers, compilers and translation. What they call translation is now what we would call a compiler for a high-level language. The 1968 then-compiler being a combination of a translator from a lower level language like Fortran to the machine language maybe with some helpful macros and also link editing or patching abilities that allow different program blocks to reference each other. So here was a project noteworthy for inclusion in the report (but hopefully that noteworthiness came from the fact that it was an ordinary project) where a significant number of staff and amount of effort were focused on creating the tools that create the product.

So because every activity in that report is a design activity I’ve tried to skip through to one of the working papers in section 9, the Classification of Subject Matter from the software product working group or production working group, because that paper categorizes production activities. These include training, indoctrination in conventions, determining and imposing productivity metrics as improvement, acquiring support staff and facilities, setting a budget, hiring staff, negotiating with customers, and design activities like specification, designing software units, creating test plans.

An amusing point from the Classification of Subject Matter is the inclusion of the entry “control of innovation and reinvention” in the list which, on the one hand, makes me think of the choose boring technology article, but on the other leads me to picture a manager who’s incensed that their staff has blasphemed by using their noodles in making the software.

If you were to press me to produce definitions of software design and software production (which implicitly you are by listening to a podcast in which I claim to discuss those topics) software design as a phrase used in the 1968 conference is the activity of understanding the system requirements and producing a collection of computer instructions that satisfy those requirements. Software production is doing that in a way that the customer wants to pay for, that you can afford, that the customer wants to use the output of, and is capable of using the output of, and preferably that the customer is happy with.

Okay, so pretending now that we know what software design is let’s look at the section of report on software design. It’s here that we find what I currently believe to be the earliest reference to the architect of real world buildings Christopher Alexander, he of the pattern language fame, in the software field, as Peter Naur describes software designers analogous to civil engineering or architecture in large heterogeneous environments.

Alexander d’Agapeyeff, who we met last time wishing we could do more to teach the design and testing of testable software, argued for designing a machine that was capable of running a high-level intermediate code translatable from high-level languages which is something that we might now recognise as Pascal p-code, the Lisp machine, JVM bytecode and so on.

A historian would probably find fault with my applying such modern ideas to these statements, the retroactive claim that conference attendees were prescient in defining the future with intermediate languages, OOP (as we shall see shortly), and TDD (as I expect to encounter multiple times in this series), and then the implication that the rest of the industry was too ignorant or too stubborn to notice for a number of decades. Certainly, certainly the NATO conferences have achieved a near mythical status now that they probably just didn’t get during the 1970s, and the reports are both incomplete and focused on points the editors considered interesting, whether because they were representative or provocative, but without telling us which. So the observations might not have landed with contemporary readers, and in fact people who were in the room at the conference may not have noticed some of these statements at all until they were typed up into the report.

Nonetheless, hardware that runs an intermediate language is a natural extension of the contemporary goals of closing the gap between large system design and implementation, so I feel like I’m on fairly stable ground making the association here. Similarly, two quotes seem to presage Bertrand Mayer’s open-closed principle with some precision. Letellier says a software package must be thought of as open-ended, and Gillette says generality is essential to satisfy the requirement for extensibility, and that the key to production success of any module construct is the rigid specification of the interfaces. In other words, you’re not allowed to modify the interfaces, they’re closed, but you do need to design the modules to be extensible, they’re open.

Anyway, back to the spooky foreshadowing of bytecode, d’Agapeyeff gives four reasons for intermediate languages to be executed on the computer. They are to increase the runtime checks a computer can make, thereby increasing program safety, provide more development facilities, increase portability, and to allow all communication with the programmer to be in source language. This last point is now also achieved for compiled languages using debugging data, supported by formats like STABS, COFF, and DWARF, which were all invented and introduced in the 1980s.

As projects, and I’m using my scare quote fingers here, as projects “scale”, which didn’t just mean the size of the software went up, it also referred to the expectations of the customers, or the situations in which they used the software, which might grow beyond those foreseen by the designers. As projects scale, application software might grow beyond the expressiveness of the design language used to describe it. Kolence blamed this on a lack of universal notation for software, which would do for programming what George Boole’s notation of logic does for electronic hardware design.

He suggested that Ken Iverson’s notation is the solution. Ken Iverson’s notation is the APL programming language, which actually grew out of a specification language used for a formal specification of parts of IBM’s System 360, among other things. APL was very popular through the 70s and 80s, and still has a hardcore following, but it never displaced the Algol-derived languages as a universal lingua franca for expressing computation. And it’s the context of a specification language in which to view the suggestion here. Not necessarily as an implementation language, not everyone who used APL even had an interpreter that would run on their computer, but as a specification language, in a continuity that includes Z, TLA, and the Unified Modelling Language, as other examples.

Dijkstra submitted a paper that proposes a hierarchical, or at least a layered design approach, which led to the discussion over the extent to which a specification should be complete. Willem van der Poel says that a complete specification is a working solution, i.e. if you describe your problem in enough detail you end up solving the problem. But Dijkstra says that an incomplete specification allows for useful flexibility. There’s an analogy here with the concept of undefined behaviour in a C programming language, which allows for a portable specification of the language that behaves in whatever way is most efficient on the host hardware. And, despite what detractors claim, a C compiler has never led to demons flying out of a programmer’s nostrils.

But, what does completeness mean? If a design can be complete, we need a definition of a complete design or an incomplete design. And, the idea of a logical closure was suggested by analogy to group theory, where a group is complete if it has a certain collection of operations. So, for example, a system that lets you write files and doesn’t have a facility to read them is clearly incomplete, because you have an operation that doesn’t have a corresponding logical extension operation.
But then, what about one that can read and write files but can’t delete them? Is that complete? So, while the idea of closure was introduced, it wasn’t very deeply pursued, or at least not in the conference report.

A particular problem with software designs is the issue of detecting and handling errors. Indeed, there is sentiment in the report that if you aren’t considering resilience and fault tolerance in your design, then you aren’t actually doing design. What makes errors difficult to design is that they tend to cut across all of your nice layers and modules, so that a failure of the storage drum to be ready means that you can’t complete a tax calculation.

To consider a more modern example, think about the Java null pointer exception. Java doesn’t even have pointers, and yet here we are, dealing with an error that is caused by one.

So, a lot of discussion took place on the directionality of design, whether that be top-down, meaning to start with the interface and requirements and work towards implementation on the computer, or bottom-up, meaning to start with reusable modules and combine them until you satisfy the requirements, or whether to do something else, because both top-down and bottom-up design have risks.

Your top-down design might paint you into a corner where you need to implement a module that you can’t actually build. Your bottom-up design might create a lot of reusable modules that don’t actually have any use at all in your system.

Naur describes a concept called design trees, where you build dependency graphs of the decisions that influence other decisions, so that you know which problems you need to solve first. Ed David, another employee at Bell Labs, suggested a skeletal coding approach, in which you actually build the whole system first, admittedly using stubs, simulations, and other doubles for modules that aren’t yet complete. Then you explore the aptness of that skeleton to your needs, tweak it, and progressively fill in the details.

Going back to our retroactive futurology, this is a process that eventually became popular as Boehm’s spiral model, which we mentioned in the previous episode, and the Rapid Application Development movement of the 1980s and 1990s. This iterative approach also addresses one of the big design drawbacks discussed in the report, which is that users and customers can’t clearly express what they want, but they can tell you when you’ve got it wrong.

Speaking of communication, Conway’s Law, which was brand new, having been published in April 68, makes several special guest appearances, as does the obvious corollary. If your organisation is going to make software that models this org chart, set up your org chart so that it models the software that you want to build.

This is the advert break. It starts now.

This episode is brought to you by me, Graham Lee.
But really, by you.

Chiron Codex is a community of people who are learning how to become better software engineers by adopting AI augmentation in a thoughtful way. We aren’t outsourcing our understanding to coding assistants like Claude or Codex, but becoming software engineering centaurs by using AI tools to improve our knowledge and the quality of our work.

Join the community over on Patreon to find out about interaction patterns that improve your work with AI coding tools, running LLMs for software development locally, discussions of recent research in the field, and more.

If you’re a software engineer who’s interested in the promise of AI tools, but sceptical about handing your skills over to the computer, this is the community for you. Go to patreon.com slash chironcodex, that’s C-H-I-R-O-N-C-O-D-E-X, now for more information and to join. Use the gift link in the show notes to get your first month of insider access completely free.

Alternatively, you can show your appreciation by donating at Ko-fi, that’s ko-fi.com slash chironcodex, K-O-F-I dot com. Direct support by my audience is the only revenue I get for my work as a software engineer and communicator, so your support really means a lot to me and makes it possible for me to produce this podcast.
Thank you so much.

That was the advert break. It’s over now.

From design then to production, and the big problem facing 1968 software people was being able to deliver large systems, both in terms of the amount of software and the amount of novelty introduced. The need to always chase the latest advances made, or should I say still makes, every project into part research, part development, and part implementation, even though it’s costed, presented to the customer, and charged for as a pure implementation project.

A Fortran compiler team will, by the time it writes its third Fortran compiler, be pretty good at writing Fortran compilers and at estimating how long it takes and how many resources they need to write a Fortran compiler. But most teams aren’t doing the same thing three times, they’re doing whatever it is for the first time, or for their first time anyway.

Your second Fortran compiler isn’t a Fortran compiler. It’s a Fortran compiler that works at an online terminal on a time-sharing computer, or in the cloud, or with blockchain, or AI assistance, or whatever’s new this week in the Datamation magazine.

The problem of scaling software production is so acute that there’s an argument over whether to just use a small team of people who know each other well for all software projects, or whether that limit would actually be the end of the software game altogether. Given the current Bot Farm amplified memes about a real-world Butlerian jihad, the event in Frank Herbert’s
Dune chronology where humankind turned against artificial intelligence, it’s kind of fun to imagine an alternate reality where the greatest computer scientists and electronic engineers in the world came together in 1968 and went, “no, this doesn’t actually work. Let’s just shut it all down.”

Digression. I said greatest in the world there, even though this podcast episode is about a NATO conference. Much as I’m not convinced the Western hegemony is the best way to organise society that one could invent, the truth is that communist bloc computing was on the back foot in 1968.

Under Stalin, cybernetics have been declared unsocialist as a tool for managerial control of the workers, so research into computing wasn’t easy to undertake, promote or secure resources for. This changed after Khrushchev’s Thaw, but it wasn’t until
the beginning of the 1960s that the Soviet government started sponsoring computing factories.

Competing interests and misaligned incentives meant that the dream of a centralised computer-controlled economy, a dream that Salvador Allende rediscovered for Chile in the 1970s, never came to fruition. At the beginning of the 1970s, Soviet computing policy turned to duplicating successful Western designs to the extent that the most popular microcomputer in the Eastern Bloc was actually a PDP-11 compatible.

The USSR undoubtedly had some very capable computing experts. Think of Ekaterina Shkabara or Lev Dashevskii, Viktor Glushkov
or Sergei Lebedev. And 1968 saw the release of the BESM-6, a machine with comparable capabilities to common American hardware. But the fact is that the Soviet Union was late to seeing value in computers and was relegated to copying Western innovations in both software (Algol, Fortran and Pascal were all popular compilers on BESM series computers) and in hardware. They typically designed integrated circuits by duplicating old designs from Texas Instruments.

Anyway, back to scaling software projects. And the conference perceived one of the biggest problems to be estimation in terms of both time and costs. If you could tell someone what they’d spend and how long they’d wait to get a working system and actually be correct about it, then you’d immediately make your endeavour more professional-seeming. Getting faster or cheaper at doing it or getting better at doing it could take a backseat to being reliable about doing the things that you claim you’re capable of doing.

The problem was nobody knew what they should be measuring which meant that they all ended up measuring the one thing that was actually countable: the number of instruction words produced. Everyone agreed that this was wrong but everyone agreed that there was no other game in town.

A couple of speakers suggested what would eventually become a decade later the “function point”: a measure of the amount of software requirements that you delivered. This was even presented in the context of measuring burndown in terms of test coverage. The amount of system you have done is the amount of software that actually does what was requested. This still suffers from the problem that we described in the previous episode where the requirements describe what the customers thought they wanted not what they actually need.

Harr listed 10 reasons that projects fail. Eight of these are the inability to estimate. They’re just the inability to estimate different things. One is a change management issue, which is not keeping the project documentation in sync with the reality, and the tenth is the one that Fred Brooks would seven years later name the mythical man month problem: trying to bring a project under control by throwing more people at it. The “human wave” approach was broadly derided at the conference even among those who didn’t think it realistic to keep software teams small.

Notice that modern software methodologies “solve” (again I’ve used my scare quote fingers) they “solve” these problems by backing away from them. We advocate for two pizza teams so that we don’t need to deal with solving communications problems. We heap scorn on people who try to solve those problems with ideas like SAFe or scrum of scrums. We advocate for short iterations so that we don’t have to do any estimation, beyond answering the question “do you think this will be ready within the next fortnight?”

We advocate for cross-functional on-site teams so that we can let gossip take the place of formal communication, or we drown remote workers under slack messages, emails, and wiki updates. In this sense, modern software engineering is more of a coping strategy than an answer to the challenges identified in 1968.

A section on performance monitoring in software production is mostly about testing—both of performance and of logical correctness. The section mentions automated suites of tests at both the unit and integration level, written in the same
language as the implementation, and checked in to the same configuration management system.

This brings me to what I consider to be the money quote from the report for this episode, and it’s from Alan Perlis:

A software system can be best designed if the testing is interlaced with the designing instead of being used after the design.

It turns out that there have been people advocating for test driven development in software longer than there have been people walking on the moon.

I want to end by coming back to this question of what constitutes design and what production, because there’s a section in the part of the report on production called Concepts which is about software paradigms.

Doug Ross advocates for “plexes”. Those are modules that combine data structure and algorithm very much like objects, in fact he references Simula as a good system for modelling these plexes. And Perlis observes that all of those abstractions exist in the Lisp programming language and they each have the name “function”.

It might seem like creating objects or functions is a design issue, but it influences so much of how you talk about and make software that it’s correct to consider it a management thing, a budgetary thing, and generally a production issue.

I’d love to hear your thoughts on this episode, or your reflections on the NATO conference report. You can comment on the blog post for this episode or you can email me at grahamlee at acm.org. Next time we’ll take a look at software support and see what the luminaries of 1968 made of helping their customers use their software.

Thank you very much for listening and we’ll talk later.

This episode contextualises the 1968 NATO Science Committee conference on Software Engineering, and explains what we learn through the executive summary, preface, and first three sections of the conference report. Upcoming episodes will cover the rest of the 1968 conference, the change in attitude shortly thereafter, and the entirely different report from the 1969 conference.

The episode is supported by members of the Chiron Codex Patreon(use this gift link for your first month free), so please do join the community or hit the Ko-Fi button to make a one-off donation.

Links

• The NATO Software Engineering conferences
• The role of the University in computers, data processing, and related fields
• An undergraduate program in computer science—preliminary recommendations
• From Airline Reservations to Sonic the Hedgehog: A History of the Software Industry
• “Crisis, What Crisis?”: Reconsidering the Software Crisis of the 1960s and the Origins of Software Engineering

Transcript

Welcome to episode 57 of the Structure and Interpretation of Computer
Programmers podcast. I’m Graham Lee, and this episode is the first
part of a mini-series discussing the 1968 and 1969 NATO conferences on
software engineering. It’s sponsored by the members of my Patreon,
which can include you.

Software engineering is commonly thought to have had its genesis at
the NATO Science Committee Conference on Software Engineering, held in
Garmisch, Germany, in the week of October 7th to 11th, 1968.

Certainly, the phrase software engineering was coined for the
title of that conference, and NATO didn’t already do software
engineering. The conference was initiated by a Science Committee
working group on computer science.

Computer science itself was a new idea, having been named by an
independent consultant, Louis Fein, in 1959. The ACM first put
together a preliminary CS curriculum in 1962 to 1965, and eventually
ratified it as Curriculum 68, the same year as the first of the NATO
conferences on software engineering.

This conference report has mostly gone down in history as a broadly
cited starting point for the so-called software crisis. But what does
it actually say? Before answering that, we need to contextualise the
conference, beginning, I suppose, by addressing the elephant in the
room. Why NATO? The answer is simply that NATO represented the largest
customer and a good chunk of the supply chain of computers and their
applications at the time. Electronic computers had been invented a
little more than 20 years earlier, and had found their first
applications in the military. In Britain, the Colossus system provided
brute force cryptanalysis to the government code and cipher school,
and in the United States, the ENIAC was funded to calculate artillery
tables, and applied by John von Neumann to thermonuclear reaction
calculations used to design the hydrogen bomb.

By the end of the 1950s, the United States’ semi-automated ground
environment defence system employed 800 to 900 programmers, more than
half of the total workforce in the country. The project would grow to
about 2,000 programmers over its lifetime. Many of the ideas of
division of labour between hardware and software people, and between
different software people, came from military projects. Computers are
one of the ideal examples of military technology that becomes dual use
through serendipity. NATO had the most to gain if people found better
ways to make software more efficiently and quickly.

Software engineering, the phrase, was according to the conference
report’s preface, and to the reminiscences of one of its editors,
Brian Randell, a name chosen provocatively to suggest that software
needed to be constructed with the same rigour as found in established
engineering disciplines. This conference brought together people from
academia and industry, about half were academics, nearly half from
computing companies or consultancies, and a few government employees
from computing using departments, and people from North America and
Europe, but mostly Europe. I count 37 European attendees or observers,
and 24 from the United States and Canada.

And the conference was organised into three work groups, software
design, software production, and software service, in which they would
discuss this notion of software engineering. Now software engineering
as a field almost implies the absence of hardware, at least the
absence of hardware is an important constraint on the design of
software. This move, certainly a political one in a field of
professional boundaries, in which programmers and analysts try to
assert their importance in the computing world as peers or even
superiors to the electrical and electronics engineers by describing
their own work as an independent engineering discipline in its own
right.

This move mirrors the slightly earlier development of academic
computer science by minimising the contribution of the computer. The
argument goes that as hardware gets more capable and flexible, the
specific limitations of any one device become unimportant, and
software designers can concentrate wholly on the problem domain. At
the outset of the integrated circuit era, this might have seemed a
reasonable bet, but in practice, there are a few domains where it’s
true even now.

Bob Barton made the opposite argument. He said, In design, we should
start by designing hardware and software together. This will require a
kind of general purpose person, a computer engineer. It’s unclear to
what extent the Software Engineering Conference, at which Barton made
that comment, actually served to widen the professional gap between
hardware and software, or whether the existing Taylorist fad for
subdividing knowledge work in the mid-20th century had already made
that split absolute. What we do know is that other than some
hobbyists and brief flurries at the beginning of the microcomputing
and Internet of Things eras, computer engineers haven’t existed, and
most organisations have separated their hardware and their software
divisions, assuming they even designed both at all.

From the very start of the report, the highlights section, that serves
as an executive summary, Randell’s recollection, and the conference as
presented in the report that he edited, and presumably the executive
summary that as editor he would have co-written, diverge immensely.

Randell recalls the conference as being the place where the software
crisis was named and acknowledged, and a field of software engineering
bent towards its resolution. In fact, it seems that the word crisis
hardly appears in the report at all, that conference attendee Edsger
Dijkstra popularised the software crisis myth in the 1970s, and that
the editors of the report were aware that it, quote, did not attempt
to provide a balanced review of the total state of software, and tends
to under-stress the achievements of the field, end quote.

Indeed, in another direct quote in the report from John Buxton, we
find that 99% of computers work tolerably satisfactorily, and Ken
Kolence says, “there are many areas where there is no such thing as a
crisis”, although the wording here implies that the idea of a crisis
was being discussed at the conference, at least.

So what are the problems that the conference addressed?
Interestingly, the highlights describe the problem crucial to the use
of computers as being the “so-called software or programs developed to
control their action”.

I wonder what this means. I initially interpreted it as suggesting
that the idea of software as a distinct entity was not yet settled.
Perhaps some people thought of a computer as a general-purpose device
that you add software to for a particular application, while others
thought of a computer as a component of a system that needs to be
programmed to fulfil its role in that system. Subsequently, I changed
my mind, and I think the editors might just mean to say that software
is a technical term that the broader reaches of their audience won’t
know the meaning of in 1968. But I’m interested to hear how you
interpret the idea of so-called software.

The specific problems they describe as being relevant to their broader
audience, that’s academics, policy makers, civil servants, people who
market computers, beyond the realm of people who directly work on
software engineering. And these are direct quotes from the highlights
section of the report.

Firstly, the problems of achieving sufficient reliability in the data
systems which are becoming increasingly integrated into the central
activities of modern society. I interpret this problem as one of the
earliest examples of the idea that software is eating the world.
Second, the difficulties of meeting schedules and specifications on
large software projects. Third, the education of software or data
systems engineers. And lastly, the highly controversial question of
whether software should be priced separately from hardware.

This is the advert break. It starts now.

This episode is brought to you by me, Graham Lee. But really, by you.
Chiron Codex is a community of people who are learning how to become
better software engineers by adopting AI augmentation in a thoughtful
way. We aren’t outsourcing our understanding to coding assistants
like Claude or Codex, but becoming software engineering centaurs by
using AI tools to improve our knowledge and the quality of our work.
Join the community over on Patreon to find out about interaction
patterns that improve your work with AI coding tools, running LLMs for
software development locally, discussions of recent research in the
field, and more.

If you’re a software engineer who’s interested in the promise of AI
tools, but sceptical about handing your skills over to the computer,
this is the community for you. Go to https://patreon.com/chironcodex,
that’s C-H-I-R-O-N-C-O-D-E-X, now for more information and to join.
Use the gift link in the show notes to get your first month of insider
access completely free. Alternatively, you can show your appreciation
by donating at ko-fi, that’s https://ko-fi.com/chironcodex, K-O-F-I
dot com.

Direct support by my audience is the only revenue I get for my work as
a software engineer and communicator, so your support really means a
lot to me, and makes it possible for me to produce this podcast.
Thank you so much.

That was the advert break. It’s over now.

Remember that in 1968, a lot of software programs were batch jobs that
ran on a whole machine, with no timesharing. There were already a
total of two computers at MIT that ran the CTSS timesharing system.
Development of Multics, the predecessor of Unix, was underway, and
Dijkstra’s team had been working on the THE multiprocessing system for
a while.

But, for the most part, while a computer ran your program, it did
nothing else. That also meant that it wasn’t running your compiler or
your assembler. Programmers had to wait in line for computer time,
just like everybody else. So, programs were written by hand, often
with flowcharts as design aids, and a lot of debugging incurred in
vivo, with programmers emulating the computer state in their head, and
checking that algorithms yielded the expected results. As we’ll see
in later parts of the conference, automated testing did exist, both at
the unit and system level.

Computer hardware had already adopted transistors, and even some early
integrated circuits. But, in 1968, there wasn’t the aggressive
upgrade cycle that we see today, and it’s likely that almost every
computer that had ever been built by the time of the conference was
either still in use, or had had its parts cannibalised for another
computer that was still in use. This includes computers based on
thermionic valves, and including those valve-based computers that use
non-binary storage, including valves that store octal and decimal
digits.

Many early computers were one-offs, designed to support the
applications they were commissioned for, but there were some standard
designs, and even one example of a family of compatible computers that
could all, well, almost all, run the same software, while offering
different specifications or capabilities. This was the IBM System 360.

Its operating system, OS 360, was released in 1965, and it
required 44 kilobytes of memory, when the System 360 family offered
between 8 kilobytes and 4 megabytes. The conference report makes a
note of this as a massive, expensive, staff-heavy project, as
expensive to IBM as a project to develop the System 360 hardware that
it ran on. But the world would have to wait until 1975 for Fred
Brooks’ detailed post-mortem in the Mythical Man Month.

To give some idea of the scale of software production at the time of
the conference, co-chair Dr. H.J. Helms estimates that there were
10,000 installed computers in Europe, a number that grew by 25% to 50%
per year, with more than a quarter of a million analysts and
programmers affected by the quality of software that manufacturers
distributed for those computers.

Alexander d’Agapeyeff reports that a decade earlier, in 1958, a
European general-purpose computer manufacturer often had less than 50
software programmers. Now, 1968, they probably number 1,000 to 2,000
people. What would be needed in 1978? he asked.

Well, fast-forwarding further than that, there are now big tech
companies with tens of thousands of software programmers who don’t
manufacture any computers at all.

As noted in the highlights, it’s large systems, where ambition
outstrips capability, in which the attendees saw a problem. With two
attendees, Asher Oppler and Stanley Gill, the latter being one of the
co-inventors of the subroutine, questioning whether customers should
even be allowed to request computer systems whose complexity outstrips
the capabilities of software creators.

As the complexity of system grows, the number of errors introduced
grows even faster. Doug McElroy and Collins both noted that the
process by which software is created uses backward techniques and has
a deservedly poor reputation. But why?

The report proposes two underlying causes in the section on Software
Engineering and Society, which was written for a more general
policy-making audience than the technical sections later. The first
cause is, according to Cambridge University’s Sandy Fraser, that
software production isn’t a linear path in which every activity takes
a step towards working software, and that managers don’t know what to
measure or how to measure it.

This is still a problem in 2026, as we saw with managers leaping on
the tokens-consumed metric without connecting that to working software
produced by their organisations.

The second cause, expressed by Robert Graham of MIT’s Project Mac,
which spawned the MIT AI lab, is that projects go on for years using
their initial poor understanding of the system, then deliver something
that doesn’t work as needed. Then they have to go back and start
again.

So even in 1968, it was seen that software construction needed more
feedback than projects were accepting from customers. And indeed,
that’s a core topic in Section 3 of the report, a discussion on the
nature of software engineering.

Two papers, one by a Mr. J. Nash of IBM UK and the other by
Dr. F. Selig of oil company Mobile, give schematic outlines of the
software engineering process, moving linearly from analysis to design
to implementation to deployment to maintenance. Both show activities
occurring in parallel, unlike the phased approach that became popular
among people who misread the Royce paper, with Nash’s diagram in
particular showing that technical support, documentation, test
development and control and administration, i.e. project management,
occur throughout the project lifetime.

Multiple attendees noted the lack of feedback in both diagrams and the
necessity to get feedback throughout the project. Bernard Galler,
then president of the ACM, recounted stories of projects delivering
poor quality results because of the lack of user feedback into the
designs and asked the question, why do these things happen? Why
indeed?

Selig himself points to feedback within the project with external
requirements informing software design and internal design constraints
informing the requirements. Sandy Fraser’s own description of the
progress of a software activity presages iterative and incremental
approaches like Barry Boehm’s 1988 Spiral model, in which, to quote
Sandy Fraser, each stage produced a usable product and the period
between the end of one stage and the start of the next provided the
operational experience upon which the next design was based.

With the benefit of hindsight, this sounds a lot like proceeding in
short iterations with time for retrospection in between them. In
practice, without access to the whole paper—the conference report is
comprised of working papers that were discussed in the conference but
never published as a proceedings as such—without access to the whole
paper, we don’t know if these iterations were weeks or months long or
who found the products to be usable. It could be that the output of
an early iteration was a system requirements specification that was
usable by a software designer, for example.

d’Agapeyeff described an inverted pyramid model in which a large
number of application programs depend on a smaller number of service
routines that sit on an even smaller base of control programs
buttressed by compilers and assemblers. Due to the lack of feedback
between applications programmers and hardware vendors who wrote the
control programs and the service routines, there was a necessary
middleware layer that adapted the service routines onto the
application’s needs but which couldn’t do anything to address
performance issues.

He described programming as still too much of an artistic endeavour
and suggested that more teaching was needed in structuring programs,
designing and testing modules and simulating runtime conditions. In
other words, in designing testable software and in testing it.

Assuming you listened from the start of the podcast and didn’t just
skip to here on the basis that I tend to take a long time getting
warmed up to a topic, you will remember that there were three
workgroups at the conference, design, production and service. At the
actual conference, attendees disagreed that design and production of
software were distinct activities.

Report editor Peter Naur says that the distinction is arbitrary and
only exists to support the division of labour in software projects.
Dijkstra says that we can’t separate the two if we are going to do a
decent job. And a consultant by the name of Kinslow says that design
is necessarily iterative. He describes the failure on large projects
as rushing to get the specification done, so skipping bits which you
expect to be able to fill in later, but which are then incorrectly
coded by 200 people. And then it’s too late to correct the damage
that’s been done to the project.

I’ve seen that failure mode on software projects in my career, which
started in 2004, but probably, at least hopefully, much less
frequently than the people in 1968, saw it.

The money quote from the first part of the 1968 report is, to my mind,
this from Doug Ross of MIT, who went on to invent the structured
analysis and design technique.

“The most deadly thing in software is the concept, which almost
universally seems to be followed, that you are going to specify what
you are going to do and then do it. And that is where most of our
troubles come from. The projects that are called successful have met
their specifications, but those specifications were based upon the
designer’s ignorance before they started the job.”

Think about this quote the next time you read a LinkedIn post on the
benefits of spec-driven development.

In this episode, we’ve covered the first 33 pages of a 226-page
report, one of two reports from the NATO conferences on software
engineering, and found that even then, software design was understood
to need iterative feedback from users, integrators, and producers, and
that everybody involved in the project had to share their knowledge
and build the software based on the latest knowledge integrated from
everybody, not on the designer’s initial feels.

Good news about the rest of this series is that the next page, page
34, is blank. But next time, we’ll start to look at the output of
some of the working groups and dig into the state of the design and
production of software in 1968.

Until then, remember that you can contact me with your feedback on
this episode. You can go to the page on the Structure and
Interpretation of Computer Programmers podcast where the post for this
episode is hosted. That’s at https://sicpers.info/podcast.

You can email me grahamlee at acm.org or you can join the Patreon to
support my work and join in the chat there. That’s at
https://patreon.com/chironcodex. I’ll talk to you again soon.