On Inheritance

I recently had the chance to give my OOP-in-FP-in-Swift talk again in NSLondon, and was asked how to build inheritance in that object system. It’s a great question, I gave what I hope was a good answer, and it’s worth some more thought and a more coherent response.

Firstly, let’s look at the type signature for an object in this system:

typealias Object = (Selector) -> IMP

A Selector is the name of a method, and an IMP is a function1 implementing that method. But an Object is nothing more or less than a function that maps names of methods to implementations of methods. And that’s incredibly powerful, for two reasons.

Reason one: Inheritance is whatever you want it to be.

You are responsible for writing the code to look up methods, which means you get to choose how it works. If you don’t like inheritance at all, then you’re golden: each object knows its own methods and nothing else.

If you like Javascript or Self or Io, then if your object doesn’t have a method then it can send itself a proto message, and ask that object what method to use.

If you like Smalltalk or ObjC or Ruby, then you can create an object called a Class that creates objects that look up methods by asking the class what methods to use. If the class doesn’t know, then it can ask its superclass.

If you like multiple inheritance, then give an object a list of classes/prototypes instead of a single one.

If you’ve got some other idea, build it! Maybe you always thought classification should be based on higher-order logic; well here’s your chance.

(By the way, if you want to do this, you would be well-off defining a convention where methods all take a parameter that can be bound to the receiver: call it this or self for example. Then when you’re deep in inheritance-land, but you want to send a message to self, you still have a reference to it.)

Reason two: Inheritance is whatever you want it to be at each point.

A failing common to all of the object systems named above (as a reminder, that’s Javascript, Self, Io, Smalltalk, ObjC, Ruby) is that they force you to work with a single object paradigm. If these things naturally follow a singly-inherited classification scheme, but those things can be better described as deviations from a common prototype, well, sorry, but you’ve got to pick one and contort it to fit both situations.

If an object is any arbitrary code that finds a method, then you can build whichever model is most appropriate at the point of use. You can mix and match. The core philosophy at a code level is that objects are just loosely-coupled functions. From a conceptual level that’s incredibly powerful: such loose coupling means that you aren’t forced to make assumptions about how objects are constructed, or glued together. You just use them.


  1. actually a closure, which is even more useful, but that’s not important right now. 

Posted in OOP, Talk | 1 Comment

How retrospectives ban shoes

At the end of each sprint, we hold a retrospective. The book “Agile Coaching” by Rachel Davies and Liz Sedley says:

An iteration retrospective should help the team explore the following:

  • What insights do they have from the last iteration?
  • What areas do they want to focus on improving?
  • What ideas can they act on in the next iteration?

Take this too literally, and you end up adding the thing next time that stops you getting wrong the mistake from this time. “That time, the explosive was hidden in a shoe, so let’s add shoes to the list of banned items.”

What if it was just a mistake? Do you need to change the way you do everything to fix a problem you encountered once?

Retrospectives need to take a longer-term view. How are we doing, and how specifically did that change this sprint? Do we need to change how we do all of our tasks because it didn’t work for that one task? How many hundreds of tasks has our process worked for?

I discussed this with Steven Baker who described a team where the retrospective facilitator kept a risk register across multiple sprints. Yes, this thing happened and it was bad, but how bad? How often has it happened before? How likely is it to happen again? How bad would it be if it does?

Posted in process | Tagged | Leave a comment

No True Humpty-Dumpty

Words change meaning.

Technical words change meaning.

Sometimes, you need to check out a specific commit of a word’s meaning from the version control, to add context to a statement.

“I’m talking about Open Source in its early meaning of Free Software without the confusion over Free, not its later meaning as an ethically empty publication of source code.”

“I mean Object-Oriented Programming as the loosely-defined bucket in which I can put all the ills of software that I’m claiming are solved by Haskell, not the earlier sense of modelling business processes in software with loosely-coupled active programs communicating by sending messages.”

“The word Agile here refers to the later sense of Agile where I run a waterfall with frequent checkpoints and get a certification from a project management institute.”

The problem is that doing so acts as a thought-terminating cliche to people who are not open to hearing a potentially valuable statement about Open Source, Object-Oriented Programming, or Agile development.

If your commit is too early in history, then you can easily be dismissed as etymologically fallacious, or as somebody who won’t accept progress and the glorious devaluation of the word you’re trying to use.

If your commit is too recent in history, then you can easily be dismissed as a Humpty-Dumptyist who’s trying to hide behind a highfalutin term that you have no right to use.

What I’ve come to realise is that my technique for dealing with people who use these rhetorical devices can be as simple as this: ignore them. If they do not want to hear, then I do not need to speak.

Posted in advancement of the self, edjercashun | Tagged | Leave a comment

It’s about the thinking

At some point in the past, programmers used to recommend drawing flowcharts before you start coding. Then they recommended creating CRC cards, or acting through how the turtle will behave, or writing failing tests, or getting the types to match up, or designing contracts, or writing proofs, but the point is that in each case they’re there for eliciting thought before the code gets laid down.

None of these things is mutually exclusive, none of these things is the one true way, but the fact that they all isolate some part of solving the problem from some part of coding the solution is the telling point. The problem is not having the correct type system or test coverage or diagram format, the problem is trying to work in two (or more) levels of abstraction – the problem domain and the computer – at the same time.

Posted in architecture of sorts | Tagged | 1 Comment

There is no browser, only Zuul

My short-lived first plan for a career was in Physics. That’s what my first degree was in, but I graduated with the career goal “do something that isn’t a D.Phil. in Physics” in mind. I’d got on quite well with computers as a hobbyist, and the computing and electronics practicals in my course labs. A job as systems administrator for those very systems (a hotchpotch of NeXTSTEP, Solaris, OpenBSD, and Mac OS X) came up at the time that I graduated so I applied, got it, and became a computerer.

Along the way, I met people who thought that some people should not be computerers because their backgrounds were not exactly identical. The Google manager who could not believe that as an applicant for the lowest-grade QA role, I had not encountered the travelling salesman problem. The software engineer at Facebook who was incensed that someone applying to build a web application in PHP and Javascript did not know that there are eight bits in a byte (never mind that there aren’t, necessarily, eight bits in a byte). And now the random on Twitter who insists that people who don’t fully know browsers aren’t allowed to write web applications.

It’s easy to forget two things: the first is that at some point in the past, you didn’t know what you know now, but you learnt it because you were allowed to participate and were taught. Even if you think you were a self-learner, you had access to playground materials, books, tutorials, online documentation…and someone made those for you and allowed you to use them.

The second is what it was like not to know those things. I remember not understanding how OOP was anything more than putting dots in the name of your function, but now I understand it differently, and don’t know what the thing was that changed.

The second of these things shows how easy it is to be the gatekeeper. If you don’t remember what not understanding something was like, then maybe it came naturally, and if it isn’t coming naturally to someone else well maybe they just don’t get it. But the first shows that you didn’t get it, and yet here you are.

When faced with a gatekeeper, I usually react flippantly. Because of my Physics background, I explain, I know how quantum physics works, and how that enables semiconductors, and how to build a semiconductor transistor, then a NAND gate, then a processor, and basically what I’m saying is I don’t see how anyone who doesn’t know that stuff can claim to know computers at all.

But what I say to everyone else is “this stuff is really interesting, let me show it to you“.

Be the keymaster, not the gatekeeper.

Posted in edjercashun | Leave a comment

To become a beginner, first become an expert

We have a whole load of practices in programming that only really work well if you’re already good at whatever the process is supposed to help with.

Scrum is a process improvement framework, but only if you already know how to do process improvement. If you don’t, then Scrum is just the baseline mini-waterfall process with a chance to air your dirty laundry every fortnight.

Agile is good at helping you embrace change, but only if you’re already good enough at managing change to understand which changes should be embraced.

#NoEstimates helps you avoid the overhead of estimates, but only if you’re already good enough at estimates to know that you always write user stories that take 0.5-2 days to implement.

TDD helps you design your APIs, but only if you’re already good enough at API design to understand things like dependency injection and loose coupling.

Microservices help you isolate modules, but only if you’re already good enough at modularity not to get swamped in HTTP calls.

This is all very well for selling consultancy (“if your [agile] isn’t working, then you aren’t [agiling] hard enough, let me [agile] you some more”) but where’s the on-ramp?

Posted in advancement of the self, software-engineering, TDD, tool-support | 2 Comments

machoo – Object-Oriented Programming in Object-Oriented Programming in the GNU HURD

For the last few weeks, my when-I-get-to-it project has been machoo, which is sort of an object-oriented system on the HURD but mostly an excuse to learn about how Mach messaging works.

I decided to build a Smalltalk-style “give me any old selector, and I’ll do something with it” messaging system on Mach, which is already a messaging system. I don’t yet know whether this is a good idea, but I like the design principle of highly loosely-coupled objects telling each other the names of the things to do.

The substrate for this system is MIG, the Mach Interface Generator. It takes descriptions of routines in an IDL that include the names of the supported routines and the argument names, types and return types. It’d certainly be possible to build each message I want to be able to send as a MIG routine, but then the client would have to import interfaces for every type it wanted to use: no bad thing, and many OO systems work that way, but not what I want.

MIG generates a ‘demuxer’ function that examines a message sent to a port, and dispatches it to a handler to the named routine. As an aside, it is highly likely that I’ve got one to many layers of indirection; that if I wrote my own function to take the place of the demuxer I could handle dispatch there without receiving arguments from a dispatcher to do more dispatch. OK, I could, but I don’t currently know how.

So what I ended up with is a two-level system, something like Smalltalk or ObjC. A “class” is a HURD translator registered on the filesystem, you send the zero-arguments message machoo_create_object to the class and get a send right to the new object. This is two-level in that if you need to initialise the object, you would then send an init message to the created object. Objects are messaged using the machoo_msg_send routine, which takes a string selector like Smalltalk.

There’s currently exactly one implementation of a class, a do-nothing null object class. It does the thing that I would extract out into a pattern for other classes: its response to a machoo_create_object message is to spawn a new thread to back the new object. So it’s like an Erlang-style system where each object has its own execution context, except that task switching is handled by the kernel rather than some virtual machine. All of the objects of a class are in a single task, and they are each on their own thread.

I still need to solve some problems, like how to return a reference to ‘this object’ from a message. I need to build more than one class of object, and extract out the common parts of class-ness so that a new class is only defined by its response to messages. So far this is proving to be an interesting and educational project.

Posted in code-level, OOP, techradar | Leave a comment

Discovery

I think the last technical conference I attended was FOSDEM last year, and now I’m sat in the lobby of the Royal Library of Brussels working on a project that I want to take to some folks at this year’s FOSDEM, and checking the mailing lists of some projects I’m interested in to find meetups and relevant talks.

It’s been even longer since I spoke at a conference, I believe it would have been App Builders in 2016. When I left Facebook in 2015 with the view of taking a year out of software, I also ended up taking myself out of community membership and interaction. I felt like I didn’t know who I was, and wasn’t about to find out by continually defining myself in terms of other people.

I would have spoken at dotSwift in 2016, but personal life issues probably also related to not knowing who I was stopped me from doing that. In April, I went to Zurich and gave what would be my last talk as a programmer to anyone except immediate colleagues for over 18 months.

In the meantime, I explored bits of the software world outside of my immediate experience. I worked on a Qt app in high-performance computing, and I’m working on “full-stack” Javascript now.

[Full-stack Javascript is, of course, a lie, but I’d rather it keeps its current meaning as “the application bit of the stack in Javascript” than took a more correct sense.]

During this time, I found that I don’t mind so much what I’m working on: I have positive and negative opinions of all of it. I have lots of strong opinions, and lots of syntheses of other ideas, and need to be a member of a community that can share and critique these opinions, and extend and develop these syntheses.

Which is why, after a bit of social wilderness, I’m reflecting on this week’s first conference and planning my approach to the second.

Posted in advancement of the self, Talk | Leave a comment

An update on the HURD project

Last time, on Structure and Interpretation of Computer Programmers, I was building an object-oriented programming system on top of the HURD, and had realised that I needed to use its trivfs library for a sender to be able to discover an object to send messages to.

I got it working very quickly, but ended up shaving a yak due to my poor understanding of the HURD translator lifecycle which meant that I didn’t think I had got it working.

My goal was to build a translator that works like the Objective-C nil object: it accepts any message and responds by returning itself. Given that I’m building on the Mach ports abstraction, “itself” is defined as a port over which you can send messages to the same object.

If I returned an integer from the message, everything worked on both sides. However, just because a port name is an integer doesn’t mean that sending it as an int will work, just as opening a file in one process then sending the file descriptor as a number to another process wouldn’t let the receiving process access the file. I tried sending it as a mach_port_t, but got a lifecycle error: the client was told that the server had died.

On doing some reading, I discovered that the port had to be sent as a mach_port_send_t for send rights to be transferred to the client. Making that change, the message now fails with a type error.

An aside, here, on getting help with this problem. There is good documentation: the HURD source is easy to read and with helpful comments, they have good documentation including examples, the OSF documentation is very helpful, there are books from “back in the day” and videos with useful insights.

On the other hand, “help” is hard to come by. I eventually answered my own stack overflow question on the topic, having not received a reply on there, the HURD mailing list or their IRC channel. The videos described above come from FOSDEM and I’m heading out there next week, I’ll try to make some contacts in person and join their community that way.

OK, so back to the main issue, I now have a fix for my problem. Well, sort of, because now that I’m correctly sending a port with a send right I’m back to getting the lifecycle error.

My current plan is not to “fix” that, but to take it as a hint that I’m doing it wrong, and to design my system differently. Using the filesystem as a namespace to look up objects is good, but using the thing I receive as the object to message is a separate responsibility. I’m changing my approach so that the filesystem contains constructors, and they return not their own port but a port to something else that represents the instance they created.

Posted in GNU, OOP, techradar | Leave a comment

Gently HURDing the side projects

I find it problematic that even at times when I’m avoiding computing outside of work, I still have ideas about things I would like to try out or improve in computing “if I had the time”. I tend to capture these somehow – usually written notes in paper or Evernote, and my personal technology radar.

Why might this be a problem? Isn’t having ideas good, and fun? Well it is, but with each comes guilt that I could be making progress on it but am not. Even when that’s my choice, when I deliberately put more effort into relationships with friends or musical projects or whatever, there’s still that nagging feeling that I’m leaving behind chances to make positive changes to computing.

My approach to addressing that started by building the radar. Now I don’t have a lot of different projects I could be working on but am not; I have a single related web of issues, and progress on any one thing counts as progress toward the whole.

The second change is to note that any progress is progress; sometimes I spend some time reading and make a sentence or two of notes on dealing with a problem. Sometimes I try a solution, find difficulties with it and write down that I discount that solution. If I’ve made some move forward from where I was before I’ve started, I can be satisfied and don’t need to burn the midnight oil to get a complete solution to a complex problem done before putting it down.

All of that goes toward describing the limited progress I’ve made on my current research topic, which is distributed message-passing. I like the idea from Erlang that objects run in separate contexts, completely decoupled except for passing messages between one another. This seems to be the best implementation of an object-oriented runtime environment, except that it is all done on the Erlang VM and in the couple of relatively esoteric languages that target it.

On the other hand, while Objective-C doesn’t make it easy to do that decoupling, it does have a very simple message-passing interface that can be implemented in any language with a C FFI. If you can wrap objc_msgSend or objc_msgLookup and expose it to your language runtime, you too can pass messages.

Why can’t we have both of these things? Why can’t we have the simple-to-integrate message interface that can work anywhere, along with the distributed and decoupled objects?

My theory is that Mach makes this possible so I’ve been investigating it using GNU Mach and the GNU HURD. Much of the documentation of Mach messaging uses name servers that register named ports for clients to find; this is how macOS, NeXTSTEP, OSF/1 and related systems work. HURD does not use a name server, it uses the filesystem: you attach a server to a file system node as a translator and clients find the ports by looking up their paths on the filesystem.

I found examples of filesystem translators in the HURD documentation, but they typically were examples that implemented the filesystem messages: seek, read, write, and so on. One could build message-sending on top of filesystem operations but it would not be pleasant:

  • marshall the message selector and arguments into some stream format
  • write() your message to the node
  • verify that you wrote as many bytes as you expect
  • read() the length of the reply
  • verify that you read as many bytes as you expect
  • read() the reply
  • verify that you read as many bytes as you expect
  • unmarshall the reply into an object of the correct type

Let’s be clear, all of this needs to be done, but it’s all already being done at the Mach message layer and hidden behind the MIG abstraction, so why should our clients and servers do it again in another abstraction built on top? I wanted to find a way to register a port that accepts non-filesystem messages using the HURD’s filesystem-as-name-server approach.

This morning I decided to look at how login was implemented on the HURD and discovered that the password server does exactly what I need. It is configured as a translator on the filesystem, and uses the trivfs library to check in with the bootstrap server and get its ports, but then it handles its own messages for checking passwords rather than the standard filesystem messages. Discovering that gives me enough new information to feel I’ve made progress, and a clear next step (pardon the pun).

Posted in GNU, learning, OOP, techradar | Leave a comment