Is spec-driven development the end of Agile software development?

Posted on 2025-12-12 by Graham

A claim that I’ve seen on software social media is that spec-driven development is evidence that agile was a dark path, poorly chosen. The argument goes that Agile software development is about eschewing detailed designs and specifications, in favour of experimentation and feedback. Spec-driven development shows that the way to unlock maximal productivity in augmented development (and therefore in development overall, as LLM agents can type faster than programmers can) is by writing detailed specifications for the code generator to follow. Therefore detailed specifications are valuable after all, so Agile was wrong to do away with them.

Let’s take a look at what’s going on in—and behind—this argument.

Agile is alleged to eschew detailed specification.

It certainly looks that way, on a quick glance. As Brian Marick noted, one of the Agile Alliance didn’t care what was in the manifesto so long as there was no way IBM or Rational agreed with it. The people who came together for that skiing holiday in Utah were coaches and practitioners of ‘lightweight methodologies’, that favoured early delivery of working software to the customer over detailed product requirements documents, and design specifications, that the customer reviews and signs off before they ever see any code—indeed before any developer is allowed to begin coding.

The reason for this isn’t to create unspecified software. It’s to discover the specification through iterative feedback. If you compare the output of a team that follows Agile software development with one that follows a then-prevailing methodology, such as the Rational Unified Process (RUP) that irked one of the alliance members, you’d probably learn that the agile team actually has a much more detailed specification.

In addition, that spec is machine-readable, executable, and generates errors whenever the software falls out of conformance. The specification these teams produce through their emergent process is the test suite. If every software change follows the creation of a failing test, then all of the software—every feature, every bug fix—is specified in a detailed document.

Three evident differences between the ‘specs’ in something like RUP, and something agile:

  1. The agile spec’s format is closer to working software, which is what the customers ultimately value.
  2. The agile spec has no ambiguity: the software meets the spec and the test passes; or it doesn’t, and the test fails.
  3. The agile spec evolves throughout the life of the software endeavour, capturing elements of a dialectic between creators, customers, and software. Meanwhile, according to figure 1.5 in The Unified Software Development Process, ‘rational’ developers finish elaboration at an early point in the project and move on to filling in the details.

And one big difference between the purpose of the tests: RUP creates tests for verification: did we build it right in the implementation phase? Agile teams create tests that also supply validation: do we understand what we’re being asked to build?

Aside: the “specs” in spec-driven development occupy the same conceptual space as tests created to drive out design.

A developer listens to a description of behaviour that their software should embody, and writes a test to drive out the details and confirm they understand the situation. When happy that the test represents a description of the code they need to create, they write the code.

A developer listens to a description of behaviour that their software should embody, and writes a document to drive out the details and confirm they understand the situation. When happy that the document represents a description of the code they need to create, they create the code.

These are the same statement, made at different levels of abstraction with respect to the tools the developer uses. In other words, the people are doing the same thing, using different tools. If you “have come to value individuals and interactions over processes and tools”, then you will probably think that there is some value in the tools; but not as much as there is value in their application. Speaking of which…

The agile manifesto says that the alliance members value specifications.

The template statement in the manifesto is “[blah blah blah] we have come to value x over y. That is, while there is value in ys:[y], we value xs:[x] more.”

The instance that’s immediately relevant to the spec-driven development argument is x = working software; y = comprehensive documentation, where detailed specification is an example of comprehensive documentation (particularly in the prolix style adopted by many of the models). Performing the substitution, detailed specification is indeed valuable, but not as an end in itself: it is (or can be) useful in pursuit of delivering valuable software.

Agile software development isn’t about not doing things; it’s about understanding why you do anything, and being ready to try something else in pursuit of your goal. With the constraint that your goal needs to be “early and continuous delivery of valuable software”.

Returning to the aside, above, x = individuals and interactions and y = processes and tools. Spec-driven development is a process, generative AI is a tool; the point isn’t to use or avoid either, they can be valuable in the pursuit of working software.

Agile software development was about who makes software and how.

The comparison between RUP and lightweight methodologies made above was particularly apposite at the moment the manifesto for Agile software development was created; a single moment that highlighted a tension in a dichotomy. It isn’t the opposition of documentation and software, or change and plans. It’s the opposition of practitioner-led versus managerial software development.

The summary of the principles behind the manifesto is approximately ‘get people who know what they’re doing, let them talk to the customer and give the customer software, then get out of their way’. The list has an emphasis on technical excellence, simplicity, customer collaboration, and—crucially—self-organisation, all seconded to the ‘highest priority’ of customer satisfaction through valuable software, with ‘working software’ as the primary measure of progress.

In other words, we promise to make software, if you promise to let us.

The prevailing, heavyweight processes were predicated on a breakdown of this compact. Managers don’t trust developers not to gold-plate software for its own sake, continually polishing but never shipping. Therefore, it’s up to managers to work out what needs to be done and to keep developers’ noses pressed to the code face until they do it.

Bring that up to date, and every software organisation now pays lip service to agile software development, and yet the opposition between practitioner-led and managerial software development still exists. When developers of the 2020s complain about agile, they typically complain about facets of managerial agile: valueless ceremonial meetings; OKRs as the primary measure of progress; and perpetual sprinting with no warm-ups, cool-downs, or rests.

All of which is to say that the story of practitioner-driven software development remains partially told, and that whether spec-driven development contributes to its continuation is only a small part of the question; a question that remains open-ended.

Posted in agile, AI | Leave a comment

On the value of old principles

Posted on 2025-12-03 by Graham

People using AI coding assistants typically wrestle with three problems (assuming they know what they’re trying to get the model to do, and that that’s the correct thing to try to get it to do):

  • Prompt. How to word the instructions in a way that yields the desired outcome, especially considering the butterfly effect that small changes in wording can lead to large changes in result.
  • Context. The models deal in a tokenised representation of information, and have capacity to deal with a finite list of tokens.
  • Attention. The more things a model is instructed to attend to, the less important is each thing’s contribution to the generated output stream. This tends to follow a U-shaped distribution, with the beginning and end of the input stream being more important than the middle.

(It’s important to bear in mind during this discussion that all of the above, and most of the below, is a huge mess of analogies, mostly introduced by AI researchers to make their research sound like intelligence, and tools vendors to make their models sound like they do things. A prompt isn’t really “instructions”, models don’t really “pay attention” to anything, and you can’t get a model to “do” anything other than generate tokens.)

Considering particularly the context and attention problems, a large part of the challenge people face is dividing large amounts of information available about their problem into small amounts that are relevant to the immediate task, such that the model generates a useful response that neither fails because relevant information was left out, nor fails because too much irrelevant information was left in.

Well, it turns out human software developers suffer from three analogous problems too: failing to interpret guidance correctly; not being able to keep lots of details in working memory at once; and not applying all of the different rules that are relevant at one time. As such, software design is full of principles that are designed to limit the spread of information, and that provide value whether applied for the benefit of a human developer or a code-generation model.

Almost the entire point of the main software-design paradigms is information hiding or encapsulation. If you’re working on one module, or one object, or one function, you should only need to know the internal details of that module, object or function. You should only need to know the external interface of collaborating modules, objects, or functions.

Consider the Law of Demeter, which says approximately “don’t talk to your collaborators’ collaborators”. That means your context never needs to grow past immediate collaborators.

Consider the Interface Segregation Principle, which says approximately “ask not what a type can do, ask what a type can do for you”. That means you never need to attend to all the unrelated facilities a type offers.

Consider the Open-Closed Principle, which says approximately “when it’s done, it’s done”. That means you never need concern yourself with whether you need to change that other type.

Consider the Pipes and Adapters architecture, which says approximately “you’re either looking at a domain object or a technology integration, never both”. That means you either need to know how your implementation technology works or you need to know how your business problem works, but you don’t need details of both at the same time.

All of these principles help limit context and attention, which is beneficial when code-generating models have limited context and attention. Following the principles means that however large your system gets, it never gets “too big for the model” because the model doesn’t need to read the whole project.

Even were the models to scale to the point where a whole, ridiculously large software project fits in context, and even were they to pay attention to every scrap of information in that context, these principles would still help. Because they also help limit the context and attention that us humans need to spend, meaning we can still understand what’s going on.

And for the foreseeable, we still need to understand what’s going on.

Posted in AI, design, OOP | Leave a comment

Vibe coding and BASIC

Posted on 2025-11-28 by Graham

In Vibe Coding: What is it Good For? Absolutely Nothing (Sorry, Linus), The Register makes a comparison between vibe coding today and the BASIC programming of the first generation of home microcomputers:

In one respect, though, vibe coding does have an attractive attribute that is difficult to find elsewhere. For the first time since instant-on home computers that fell straight into a BASIC interpreter, it’s possible to make things happen with a bit of typing as a very naive user.

A photograph of the back cover of issue 7 of Input magazine, showing that PEEK and POKE are coming in issue 8.

A big difference between the two scenarios is that on an early micro, you had to use BASIC to get anything else done, in many situations. The computer I used was a Dragon 32 and unless you had a (very expensive) game or package that came on a ROM cartridge, even loading software from cassette required a BASIC command.

Actually it was one of two BASIC commands: you typed CLOAD to load a basic program, or CLOADM to load a machine-language program. Either way, the default behaviour of the computer was to load its BASIC interpreter from ROM and wait for input, that input being in the form of lines of BASIC.

The context of those two commands indicates a big similarity between BASIC and vibe coding: to get particularly far, you needed some knowledge of the workings of the rest of the computer. In this case, you needed to know whether your cassette tape contained a BASIC program or a machine-language program, but that isn’t the most egregious example.

As I said, the computer I used at the time was a Dragon 32, which was a kind of not-clone of the Tandy Color Computer. Let’s say that I wanted to write a game, which is entirely plausible because I both wanted to, and did, write many games. How do I read the joystick direction and fire button status in BASIC, so I can add joystick control to my games?

Direction is relatively easy. There are two potentiometer-based joysticks, right and left, and each has an x and a y axis. The JOYSTK function takes an index which identifies the axis to read, and returns a value from 0 to 63 indicating the extent of travel. For example, JOYSTK(0) is the x axis of the right joystick.

To get the fire button status, I use this operation: the Peripheral Interface Adapter’s side A data register is mapped to location &HFF00 in memory. The right joystick’s fire button unsets bit 0 when it’s pressed, and sets it when it isn’t. The left joystick’s fire button unsets bit 1 when it’s pressed, and sets it when it isn’t. For example, P = PEEK(&HFF00) : F = NOT (P AND 1) sets F to 1if the right fire button is pressed.

Yes, I’m still technically BASIC coding, but I’ve learned a lot about the architecture of the machine and applied advanced mathematical concepts (I was single-digit years old) to improve my BASIC-coded software.

Much more importantly, I’ve become excited to understand this stuff and apply it to the programs I write, and I’m enthusiastic to share those programs and find other people who want to share their programs. There’s an inert plastic box plugged into an old TV in the otherwise-unused middle room of the house, and I can make it do what I want.

The BASIC-haters pooh-poohed that notion: OK you can make your little toys, but real software is written in machine code. Citation needed I’m sure, but I suspect that the advent of Visual BASIC meant that far more real software was being written in BASIC than in machine code even by the 1990s, a decade in which that first generation of micros was redundant but not quite obsolete (the last Commodore 64 was sold in April 1994, because Commodore went bankrupt).

However, the people who learned BASIC just picked up other tools and continued the journey they had been inspired to set out on. I live in the UK and many of the professional programmers I meet, when they’re between about ten years older than me and three years younger than me, are in that BASIC generation. We typed programs on our micros, we learned how the computers worked, and then we transferred that understanding and that exploration to other systems and other programming languages. We watched TRON, and learned to fight for the users: being scared by computers because they refuse to open the pod bay doors was for our parents.

If vibe coding gives this generation that same sense of wonder and empowerment, of being able to control a device that has hitherto only done what other people charge you to do, if it starts that same journey of learning and applying skills, and of sharing that knowledge, then it really doesn’t matter whether you think it’s OK for real software. It really doesn’t matter whether it ends up being a great tool or not.

Posted in AI, history | Leave a comment

Essence and accident in language model-assisted coding

Posted on 2025-11-24 by Graham

In 1986, Fred Brooks posited that there was “no silver bullet” in software engineering—no tool or process that would yield an order-of-magnitude improvement in productivity. He based this assertion on the division of complexity into that which is essential to the problem being solved, and that which is an accident of the way in which we solve the problem.

In fact, he considered artificial intelligence of two types: AI-1 is “the use of computers to solve problems that previously could only be solved by applying human intelligence” (here Brooks quotes David Parnas), to Brooks that is things like speech and image recognition; and AI-2 is “The use of a specific set of programming techniques [known] as heuristic or rule-based programming” (Parnas again), which to Brooks means expert systems.

He considers that AI-1 isn’t a useful definition and isn’t a source of tackling complexity because results typically don’t transfer between domains. AI-2 contains some of the features we would recognize from today’s programming assistants—finding patterns in large databases of how software has been made, and drawing inferences about how software should be made. The specific implementation technology is very different, but while Brooks sees that such a system can empower an inexperienced programmer with the experience of multiple expert programmers—“no small contribution”—it doesn’t itself tackle the complexity in the programming problem.

He also writes about “automatic programming” systems, which he defines as “the generation of a program for solving a problem from a statement of the problem specifications” and which sounds very much like the vibe coding application of language model-based coding tools. He (writing in 1986, remember) couldn’t see how a generalization of automatic programming could occur, but now we can! So how do they fare?

Accidental complexity

Coding assistants generate the same code that programmers generate, and from that perspective they don’t reduce accidental complexity in the solution. In fact, a cynical take would be to say that they increase accidental complexity, by adding prompt/context engineering to the collection of challenges in specifying a program. That perspective assumes that the prompt is part of the program source, but the generated output is still inspectable and modifiable, so it’s not clearly a valid argument. However, these tools do supply the “no small contribution” of letting any one of us lean on the expertise of all of us.

In general, a programming assistant won’t address accidental complexity until it doesn’t generate source code and just generates an output binary instead. Then someone can fairly compare the complexity of generating a solution by prompting with generating a solution by coding; but they also have to ask whether they have validation tools that are up to the task of evaluating a program using only the executable.

Or the tools can skip the program altogether, and just get the model to do whatever tasks people were previously specifying programs for. Then the accidental complexity has nothing to do with programming at all, and everything to do with language models.

Essential complexity

Considering any problem that we might want to write software for, unless the problem statement itself involves a language model then the language model is entirely unrelated to the problem’s essential complexity. For example, “predict the weather for the next week” hides a lot of assumptions and questions, none of which include language models.

That said, these tools do make it very easy and fast to uncover essential complexity, and typically in the cursed-monkey-paw “that’s not what I meant” way that’s been the bane of software engineering since its inception. This is a good thing.

You type in your prompt, the machine tells you how absolutely right you are, generates some code, you run it—and it does entirely the wrong thing. You realize that you needed to explain that things work in this way, not that way, write some instructions, generate other code…and it does mostly the wrong thing. Progress!

Faster progress than the old thing of specifying all the requirements, designing to the spec, implementing to the design, then discovering that the requirements were ambiguous and going back to the drawing board. Faster, probably, even than getting the first idea of the requirements from the customer, building a prototype, and coming back in two weeks to find out what they think. Whether it’s writing one to throw away, or iteratively collaborating on a design[*], that at least can be much faster now.

[*] Though note that the Spec-Driven Development school is following the path that Brooks did predict for automatic programming (via Parnas again): “a euphemism for programming with a higher-level language than was presently available to the programmer”.

Posted in AI, software-engineering, tool-support | 2 Comments

Tony Hoare and negative space

Posted on 2025-11-12 by Graham

The Primeagen calls it Negative-Space Programming: using assertions to cut off the space of possible programs, leaving only the ones you believe are possible given your knowledge of a program’s state at a point.

Tony Hoare just called it “logic”, which is why we now call it Hoare Logic. OK, no he didn’t. He called it An axiomatic basis for computer programming, but that doesn’t make the joke work. The point is, this is how people were already thinking about program design in 1969 (and, honestly, for a long time before). I’m glad the Primeagen has discovered it, I just don’t think it needs a new name.

Imagine you know the program’s state (or, the relevant attributes of that state) at a point in your program, and you can express that as a predicate—a function of the state that returns a Boolean value—P. In other words, you can assert that P is true at this point. The next thing the computer does is to run the next instruction, C. You know the effect that C has on the environment, so you can assert that it leaves the environment in a new state, expressed by a new predicate, Q.

Tony Hoare writes this as a Hoare Triple, {P}C{Q}. We can write it out as a formulaic sentence:

Given P, When C, Then Q

Given-when-then looks exactly like a description of a test, because it is; although our predicates might not be specific states for the program to be in as in a specific unit test, they might describe sets of states for the program to be in, as in a property-based test.

We can also use the fact that the computer runs program statements in sequence to compose these triples. Given P, when C, then Q; given Q, when D, then R; given R and so on. Now, if we arrange our program so that it only runs C when P is true, and it always runs D next, and so on, we can use the composition rule to say something about the compound statement C;D;etc. If we wrap that compound statement up in a function f(), then we get to say this:

If the caller of f() makes sure that the precondition for calling the function is true before making the call, then f() guarantees that the postcondition is true after the call returns.

This combination of responsibilities and guarantees is a contract. If you start with the contract and work to design an implementation that satisfies the contract, then you are designing by contract. You’re following a discipline of programming, as Dijkstra put it.

Oh, speaking of Dijkstra. We said that you need to know that the computer always runs this part of our program in the sequential way, with the stated precondition, that means it isn’t allowed to jump partway into it with some other, different initial state. If we allowed that then we’d have a much harder time constructing our P at a point in the program, because it would be whatever we think it is if the program follows its normal path, or whatever it is if the program goes to this point from over here, or whatever it is if the program goes to this point from over there, and so on. And our Q becomes whatever we think it is if the computer runs C after the normal path, or whatever it is if the computer goes to this point from over here then runs C, and so on.

Thinking about all of that additional complexity, and persevering with designing a correct program, could be considered harmful. To avoid the harm, we design programs so that the computer can’t go to arbitrary points from wherever the programmer wants. All to support logically-designed programming, or “negative-space programming” as I suppose we’re meant to call it now.

Posted in history, software-engineering | Tagged | Leave a comment

When did people favor composition over inheritance?

Posted on 2025-11-06 by Graham

The phrase “favor composition over inheritance” has become one of those thought-terminating cliches in software design, and I always like to take a deeper look at those to understand where they come from and what ideas we’re missing if we just take the phrase at face value without engaging with the discussion that led to it.

This is one of those aphorisms with a definite origin story (compare with Aphorism Considered Harmful for an example where the parts have clear origins but the whole does not): it’s the second object-oriented design principle in the Design Patterns book from the “Gang of Four” Gamma, Helm, Johnson, and Vlissides. Well, sort of:

Favor object composition over class inheritance.

It comes at the end of a page and a half of justification, and actually just before another page and a half on delegation (an extreme example of object composition), so really in the middle of a three-page discussion. This contrasts inheritance as a “white box” form of reuse, because the inheriting class has full visibility over the implementation details of the inherited class; with composition as a “black box” form of reuse, because the composing object only has access to the interface of the constituent object.

That’s certainly true of Smalltalk objects, and Smalltalk is one of the Gang’s example implementation languages. But even a modestly more recent language like Java has visibility attributes that let a class control what its subtypes can view or change, meaning that any modification in a subclass can be designed before we even know that a subtype is needed. Looking at this the other way, we could also say that languages like Smalltalk and Python that have advanced runtime introspection let a composing object access internal state of its constituent. This part of the argument then is contextually and historically situated, and depends on designers playing nicely within the intent of object designs, even where those aren’t enforced by a language.

Another part is more compelling: inheritance is defined statically at compile time and comes with language support, which makes it easier to use than composition and harder to change. Composition is manually arranged by a programmer assigning the constituent object to a member field and calling its methods in the composing object’s implementation; which is more work but easier to change at runtime. Assign a different object to the field, and get new behavior.

Further, the classes are (assuming the polite agreement above is honored) related only by the public interface of the constituent object, so there’s no implementation dependency. The system’s design depends on the relationships between objects at runtime, rather than the inheritance tree defined at compile time. This is presented as an advantage, which we need to consider in the context of the modern preference to rely more on the compiler as a correctness checker and static analysis tool, and to avoid deferring decisions to runtime that might turn out to be buggy.

Bear in mind that a year earlier, Barbara Liskov and Jeanette Wing proposed this principle:

What does it mean for one type to be a subtype of another? We argue that this is a semantic question having to do with the behavior of the objects of the two types: the objects of the subtype ought to behave the same as those of the supertype as far as anyone or any program using supertype objects can tell.

Within this context, the preference for composition is liberating to the designer: their type isn’t morally a subtype of the thing they’re extending, so they don’t need to restrict themselves to a compatible interface. And indeed, Liskov had already made this point in 1987 in Data Abstraction and Hierarchy, with reference to polymorphic types:

Using hierarchy to support polymorphism means that a polymorphic module is conceived of as using a supertype, and every type that is intended to be used by that module is made a subtype of the supertype. When supertypes are introduced before subtypes, hierarchy is a good way to capture the relationships. The supertype is added to the type universe when it is invented, and subtypes are added below it later.

If the types exist before the relationship, hierarchy does not work as well[…] An alternative approach is to simply allow the polymorphic module to use any type that supplies the needed operations. In this case no attempt is made to relate the types. Instead, an object belonging to any of the related types can be passed as an argument to the polymorphic module. Thus we get the same effect, but without the need to complicate the type universe. We will refer to this approach as the grouping approach.

Liskov goes on to say that when the relationship is identified early in design, “hierarchy is a good way to express the relationship. Otherwise, either the grouping approach…or procedures as arguments may be better”.

That points to a deficiency in the “composition over inheritance” aphorism: those aren’t the only two games in town. If you have procedures as first-class types (like blocks in Smalltalk, or lambdas in many languages), then you might prefer those over composition or inheritance.

Posted in history, ooa/d, OOP | 3 Comments

LLMs and reinforcement learning

Posted on 2025-10-10 by Graham

My reflection on the Richard Sutton interview with Dwarkesh Patel was that it was interesting how much the two participants talk past each other, and fail to find common ground. Particularly that they couldn’t agree on the power of reinforcement learning, when it’s such a large part of the LLM workflow.

To be clear, it isn’t the large language model that engages in reinforcement learning, it’s the person who’s applying the LLM to their task. That’s all that prompt engineering is. Here’s the workflow:

  1. Identify a goal.
  2. Hypothesize a prompt that leads the LLM to satisfy the goal.
  3. Try out the prompt and generate an outcome.
  4. Observe the gap between the outcome and the intended goal.
  5. Repeat steps 1-4 (yes, include step 1, as you might refine the goal or the prompt) until the gap becomes acceptably small.

This process—commonly known in its isolated, bureaucratized form as “science”—is fundamental to the learning experience. It’s the same as John Dewey’s description of reflective thought, as Honey and Mumford’s learning cycle, as the Shewhart cycle (Plan-Do-Check-Act). Start at step four, and it’s the same as Boyd’s Observe-Orient-Decide-Act loop. And it’s fundamental to how LLM-assisted work unfolds. But because the LLM is incapable of manifesting the cycle, it’s left to the person to do all the learning.

Posted in AI | Leave a comment

Prompting software or supporting engineering

Posted on 2025-10-07 by Graham

As we learn to operate these new generative predictive transformers, those of us in the world of software need to work out what we’re doing it for. The way in which we use them, the results we get—and the direction the tools develop—changes dramatically depending on this worldview.

One option is that we’re augmenting or supporting software engineering. Perhaps asking a language model to explain how code works, or getting it to investigate whether there are test cases we haven’t covered, or identifying ambiguities in a user story, or getting it to fill in some boilerplate code that would bore us if we wrote it ourselves.

Another option is that we’re generating software using natural language prompts. Perhaps asking a language model to create an app, or generate an integration between two services, or create a website to achieve some goal.

These are (at least for the moment) very different things.

Posted in AI | Leave a comment

Unintended consequences

Posted on 2025-08-28 by Graham

As the shift in content of this blog has made abundantly clear, for the last five years I’ve been doing a PhD. I’ve also been working full-time, so that research and study has basically taken up all of my spare time. I would wake up early, sit at my computer until it was time to start work, then change computer, then at the end of the work day switch back to the university-issued computer. I knew this was having an adverse impact on my health. Particularly, I could feel that I had far less stamina. Then I got the wake-up call I needed.

My fitness tracker sent me a notification at the beginning of August that my VO2 max—the measure it uses to gauge cardiorespiratory fitness—had dropped to 28.6. The average value for a man of my age is 38, and 28 is the average value for a man over 60. Low cardio fitness, according to the app, is a predictor of various heart diseases, colon cancer, type 2 diabetes, and even dementia and Alzheimer’s.

I’ve been on holiday for about half of the time since then, but my holidays tend to be active as they involve lots of walking and dancing (I go to festivals). On workdays, I’ve added a ten-minute HIIT workout to my morning, and a half-hour walk at lunchtime. Just those changes has seen the (estimated) cardio value in my tracker go up to 30.2 in three weeks. This is still in the low category—31.0 is the boundary to the “below average” category—but it’s a trend in the right direction.

I’ve always defined my “success” by my intellectual contribution to society. But I need to stay healthy to continue making that contribution, and it got past time to take control of my health.

Posted in advancement of the self | Leave a comment

Is Foundation a Utopian vision?

Posted on 2025-07-11 by Graham

Important: while I’m only talking about the Foundation books in vague details here, I will end up summarising a number of key points through the whole series. If you haven’t read them, and intend to, I recommend not reading this post, yet.

Here, at the start of the third TV series of Foundation, I’m actually talking about the series of short stories and novels on which the TV show is based. The TV series takes its leave of the source material in many ways—most of them necessary to produce something watchable in 2025. For example, you don’t get a female character in the novels for a very long time, and you have to wait until the second book to encounter one that Asimov names. Also, atomic ashtrays no longer represent the pinnacle of technological advancement.

The Foundation series of books starts with a collection of short stories about the titular Foundation; indeed the first volume consists of stories that had already been published elsewhere, and which are intended to be read separately but describe events in the same timeline.

Hari Seldon comes up with the field of psychohistory, a branch of mathematics that explains human activity collectively without explaining how any individual acts, in a similar way to how statistical thermodynamics tells you the behaviour of heat transfer in a system without describing the story of a single molecule.

This psychohistory predicts that the great Galactic Empire will crumble, causing millennia of human suffering—or a single millennium, if the Foundation intervene and guide the creation of a new society, using the calculations of psychohistory as a blueprint. Seldon runs the maths, and identifies a number of crisis points that this transition will go through, recording talks for the Foundation to play at the times when these crises occur.

So far, this sounds incredibly Marxist. The Empire takes the place of, well, imperialism, grand in its reach and advances but ultimately destined to destruction through its own contradictions. Psychohistory represents historical materialism, a scientific examination of the currents of history that identifies how these contradictions lead to the collapse of imperialism and set out the blueprint for a transition to a just society. As in Marx’s XVIII Brumaire of Louis Napoleon, there are no great men of history, “they do not make [history] under self-selected circumstances, but under circumstances existing already, given and transmitted from the past.” Though as Napoleon was seen by some (other than Marx, obviously) as the Great Man of European history, so the Mule disrupts the statistical flows of psychohistory.

Then things take a turn, presumably because Marxism fell out of favour (to the extent it ever was in favour) in the America in which Asimov spent most of his life. From Foundation’s Edge onwards, even the Foundation’s psychohistory isn’t the end goal of society, but a Gaia-esque shared consciousness in which all of humanity participates. In its presentation, we learn of this as a sort of harmony-of-nature, in-tune-with-the-universe gestalt existence that sounds very much like a utopian way out of the struggles and conflicts of human existence, and the climatic destruction of a species that doesn’t take its home planet seriously.

Utopian, that is, until we learn the final revelation that Asimov wrote to tie all of his great science fiction series together, in the fifth (and chronologically final, though two prequels follow on) novel, Foundation and Earth. That is, in modern parlance, that the whole lot—the creation of psychohistory by Seldon, the Foundations, and Gaia—were put in motion by what we would now call an Artificial Intelligence, the robot R. Daneel Olivaw. It was Olivaw’s drive to create a better galaxy for humanity that led to the events of the preceding novels, and his vision of how that world should work that culminates in Gaia and the shared consciousness.

The interpretation of this point has probably shifted a lot in the intervening decades, and we even see that shift in other science fiction works. Through Asimov’s Robots series we see AIs as generally benevolent servants of humanity, programmed to do the right thing even if sometimes the programming leads to some surprising events. The main enemy of progress in that series isn’t the robots themselves, it’s the inability of most people to accept that they can exist and work alongside humans. If the spongy, platinum-iridium positronic brain of a robot comes up with the idea of Gaia, then it must be a good idea.

However, fast-forward to the present day—pausing to watch the horrors of the Cybermen on Doctor Who, the Borg on Star Trek, the Matrix, and most episodes of Black Mirror—and we’d have a very different take on “computer decides that the best outcome for humanity is to remove individuality and for everyone to be connected in a shared consciousness.” How often is individuality seen not as a weakness, but as the very point of humanity? Would Asimov add yet another denouement to the Foundation series today? Would he perhaps go back to the original outcome of psychohistory, and the triumph of intellectualism and careful thinking over demagoguery; or would that conclusion be too technocratic for today’s readers?

Posted in Fiction | Leave a comment