If you drive a car, you might know that most of the complexity of driving is with the traffic, i.e. the decisions you make in response to your environment. Controlling the car itself is very simple. You have a slow-down pedal, a speed-up pedal, a turning wheel and a drive mode (PRND) selector.
Yet, beneath this simplicity, a car is a very, very complex system. There are tens of thousands of mechanical parts in a car, several thousand of them in the internal combustion engine itself, that easily do a few hundred million revolutions of the crankshaft over the lifetime of a car. You don’t need to know about it because collectively the automobile industry has figured out how to build a car so easy to use and so reliable that it would work faultlessly with nobody having to maintain it beyond a couple of hours every year.
The automobile industry has pared down, distilled and improved the driver operations to the minimal 3-control model over centuries.
The Missing Model
As with the automobile, the key to managing something complex is by having the right model for it. We arrive at the right model through a long period of doing things the hard way. We observe, understand and recover from failures. We identify patterns in failure and in complexity. Over time, we identify what keeps changing and what stays relevant throughout.
I’ve said this before but it’s worth repeating. We’ve been using a programming model that was designed for programming single, integrated computing units and stretching it to wrap around very difficult problems that are mainly the result of mixing parallel execution, persistent state, networks and physical distances. I’ve been dealing with it for 25 years myself.
Over the past few decades, we as an industry have learned a lot about how to build and operate reliable distributed applications. It’s just that there’s a lot to learn. Right now, we require every developer to learn all of it. Thus we have labels like “full stack engineer” and “devops”.
If you step back a bit and squint, you might realise that these are just the result of the lack of a model for distributed applications. We are with distributed systems right now where we were with automobiles in the 1960s. Where every Dad was a part-time grease monkey or they had a garage nearby where they spent a lot of time and money.
We’ve actually had several false starts – CORBA, EJB and Object-Oriented Databases from the 90s come to mind. CORBA famously tried to hide the network. Its whole bet was that a remote call could be made to behave like a local one, if we just built stubs that were clever enough. Eventually, we accepted that latency and network unreachability can’t be wished away. Then we built the boring, everyday infrastructure we have today, including things like single-ownership data models, durable logs and event processing systems, idempotency keys, circuit breakers, and of course virtualised disposable compute that makes “independent failure domain” a cost-effective reality.
So instead of hiding them, why not build a model around them? That’s what I have done with a development model that I call RITE — Reliable Interaction with Typed Endpoints, and my wager is that it’s the right model. It’s an attempt to take the constraints we’ve already, collectively, been forced to learn — single ownership, bounded calls, idempotent retries, invariants that don’t cross a boundary — and derive a small, minimal model from them, the same way three pedals and a wheel let a driver ignore a few thousand moving engine parts.
I’ve also built a programming language, Ritchie, based on this model.
Hello World
Let’s say Hello World with Go:
package main
import "fmt"
func sayHello(toWhom string) string {
return "Hello " + toWhom
}
func main() {
fmt.Println(sayHello("World"))
}
That was easy! How about remembering whom we’ve already greeted? That’s not so easy but not too tough either:
package main
import "fmt"
var Users = map[string]bool{}
func sayHello(toWhom string) string {
if _, ok := Users[toWhom]; ok {
return "Hello registered user " + toWhom
}
Users[toWhom] = true
return "Hello new user " + toWhom
}
func main() {
fmt.Println(sayHello("World")) // Hello new user World
fmt.Println(sayHello("World")) // Hello registered user World
}
Let’s take things up a notch… or two… no wait, three notches! We want:
- Persistent user registry, so we don’t forget whom we’ve greeted even if the program crashes and restarts
- We make the greeter a network API, like a REST endpoint
- We allow concurrent greetings and registrations, naturally, since we’re a network service
That’s what I’d call a minimal teeny tiny distributed system. I won’t paste the Go version here but you can take a look if you wish. I wouldn’t, because I didn’t even write it myself (LLM, yay!). Even after 70 lines, that version has severe correctness issues. The program ceases to be correct when run as multiple instances. Each instance has its own in-memory registry and all of them append to a file that was never designed to be shared this way.
A real version now needs a storage service, a schema and client, connection management, consistency model, transaction rules, service discovery, authentication, deployment configuration, health checks, scaling policy, observability and a failure model. The domain operation remains “remember a user and greet them.” The implementation has become mostly machinery required to make that operation survive distribution. You could try to ease the pain using a framework, but there’s still a problem. A framework is open. It can only assume that things are a certain way. As long as you’re programming in a lower model, using a framework just means that you promise to follow the instructions on the label. It’s not too hard to inadvertently go against the instructions. Therein lie production outages.
Distributed Hello World
Let’s see how we do a cloud-native hello world with Ritchie.
$User { unique String name } // Define persistent user registry
$User.name() -> String {
return self.name
}
@Greeter {} // Define a greeting network-API service
// An API endpoint, says hello
@Greeter.sayHello(String toWhom) -> String {
// Atomic get-or-create
$User u = $User[name: toWhom] else create $User{} then {
return f"Hello new user {toWhom}"
}
return f"Hello registered user {u.name()}"
}That’s the whole thing. Persistent, network-addressable, safe under concurrent requests, and — unlike the Go version – safe under multiple instances. You don’t need code to start any database servers or wire up clients or set up replication, or secure the endpoints with mTLS, or allow higher traffic through auto-scaling or write explicit tests to check whether you’re reading the right thing from the right place. You’ll even notice the lack of any error handling because there is’s built in semantics around error handling and propagation.
This example is same in size and upfront complexity as the second Go example – the one without persistence, concurrency and network. Ritchie effectively retained the domain logic while subsuming the distributed computing semantics. It does what it takes to make a good persistence layer a good API service and, (soon) a good data platform.
That’s not all, though. Ritchie is built to scale not only in terms of infrastructure, but also development processes. Whether you’re grappling with Conway’s Law, fighting Modular Monoliths vs. Microservices debates or trying to enforce Domain Driven Design principles in your architecture, Ritchie will have your back.
Not a Faster Horse
You run the Go Hello World program simply as go run hello.go today and, in a moment, that one command finds your source file, resolves every package it depends on, type-checks the code, generates machine instructions for your CPU, allocates registers, optimises the result, links it all together, creates a binary and executes it. It’s so ubiquitous that most developers under thirty have never had a reason to think about what it’s doing.
In the 1950s, that same journey — code to running program — was the work of a department. A programmer wrote on paper coding sheets, handed them to a room of keypunch operators, who punched the program onto a deck of cards. The deck went to a computer operator, who queued it for the assembler, a batch job that ran hours later on whatever shift had the machine. A separate program, the linking loader, combined the result with library subroutines in a later pass still. Register allocation — which of the CPU’s handful of fast working slots should hold which value, at which moment — was, in the 60s, a research career where specialists argued heuristics for a living.
None of that complexity was fake. It was real work, done by real specialists, and every stage of it was, at the time, considered the obvious and permanent shape of the job. Over time, slowly at first and then all of a sudden, it just stopped being the programmer’s problem to think too much about.
According to a fable, Henry Ford once said that if he’d asked customers what theey wanted, they’d have said “a faster horse.” Nobody working the keypunch room could have described go build in advance – they’d have asked for a faster card sorter.
So while everyone else is trying to find the programming equivalent of a faster card sorter, I’d rather be wrong trying to distill the next programming model – one that the machines writing our code will need as much as we ever did.
Here We Are Now
Over the past 25 years, I’ve built a global UGC platform, a multi-player game engine, multiple e-Commerce components, private cloud PaaS, and even led the development of a multi-cloud developer platform supporting a few thousand developers in eCommerce. Ritchie and its underlying model, RITE, are a distillation of everything I’ve learned as a developer, architect and leader.
Ritchie is a product of the MetaComputer series, if you want the backstory. Over the past few months, this project has moved on from thought experiments and concepts to concrete software. I built one throwaway prototype earlier this year to convince myself that the problem was tractable. The past few months, I worked through formalising the language semantics and building a toolchain. I’ve got Ritchie programs deployed as multiple daemons on my Mac, as kind and k3s clusters on my Linux mini-PC. There’s even an EKS + Aurora cluster on AWS where the toolchain configures secrets, mTLS,… the whole shebang. Both deployed out of the same build.
Having said that, the developer experience is still pretty awful and the functionality is only about 1/3rd done. So there’s still a ways to go until the first public release of any kind.
There’s a lot to do, so it’ll be great to have your support. Please visit the Ritchie website to know more. You can write to me on LinkedIn, or drop an email to queries@ritchie.dev.