Instant-feedback, instant-debugging Python coding
Robert Lechte / djrobstep.com / @djrobstep
Hi folks, I'm Rob. My current obsession is more humanistic software development. Here we're going to be looking at instant-feedback python coding. And to get straight into it, here's how it works:
The general idea is that every time you change the code, the code gets run. And by the time your fingers leave the keyboard, the state of the code at each line is right next to the code. Let me be clear about this, this isn't a product pitch, or a new project, or anything like that. This talk is about ideas. And to that end, let's step back for a moment and talk about the history of media.
Old media in new clothes.
The story of technology is the story of humans struggling to adjust. When new forms of technology enable new forms of media, it takes some time for us to collectively figure out how to use them.
Gutenberg changed the world forever with his printing press. But his first major project was simply to make more copies of the bible, not much different to the handwritten ones from before. Things like newspapers, magazines, comic strips, and the modern novel, would come later, and Gutenberg himself would be long dead.
Next slide please
Before computers there were overhead projectors and slide projectors. Most of the presentations at this conference or any other, have slides not much different to the slides of yesterday.
And back in 1978, before today's macbook pros, there was the VT100. It had an 80x24 character-based terminal. In spite of today's millionfold increases in processing power, fire up a terminal on your laptop and it probably still has 80x24 characters by default.
Faster horses, or...
But the true promise of computing is not just more and faster replication of existing tools and concepts, but the enablement of whole new ways of thinking.
Steve Jobs famously described a vision of computing as "a bicycle for the mind". But programming seems to fall short of that.
It's not just like riding a bike
The constant corrective feedback of a bicycle's lean helps you learn you ride easily, and quickly develop an intuitive sense for how the bike moves and responds. Programming you no feedback as you write a program and unfriendly error messages when you run it.
Blank by default
Unless output is specifically incorporated into the program itself, a computer program itself is a black box producing no output at all.
In almost every other creative pursuit, continuous sensory feedback is a given. When cooking you touch the ingredients, see the food change visually as the ingredients combine, smell the food as it cooks, and taste it regularly to tweak the flavours. When hammering a nail you feel the bounce of the hammer and observe the straightness or crookedness of the nail.
The unreasonable effectiveness of spreadsheets
In fact, even other areas of computing have this worked out too. Take spreadsheets for instance. Developers have a pretty universal disdain for spreadsheets, but you gotta hand it to em: You open it up, you type a formula, and you're programming. Every cell is right there for you to see.
Get a new bike and it’s familiar enough that you can just hop on and go. Compare that to the experience of setting up a new computer as a half-decent development environment for any "real" programming language. Now, some programmers might handwave away spreadsheets as "not real" programming.
But I think this attitude is mostly a product of a bad culture in software: That real programming has to be hard, it has to be unfriendly, and it has to be inaccessible to outsiders.
Stockholm syndrome: a condition which causes hostages to develop a psychological alliance with their captors during captivity.
It's part snobbery, and part Stockholm Syndrome caused by unfriendly tools. This attitude is probably why half the world still runs on spreadsheets rather than "real" programming.
Pacman is live
What about gaming? Open up a game of pacman, and you just start playing, and you see stuff happen instantly. There's no tedious upfront configuration, and you don't need to write the instructions for pacman in a text file and then run it in a separate window to see Pacman actually move.
"... software doesn’t break. [It] is not like the faulty rivet that leads to the crash of an airliner. The software did exactly what it was told to do. In fact it did it perfectly. The reason it failed is that it was told to do the wrong thing. Software failures are failures of understanding, and of imagination." -The Coming Software Apocalypse by James Somers in The Atlantic, 2017
But this is not about merely making programming a bit faster or less annoying. It is literally a matter of life and death.
A 2017 article in the Atlantic discussed the nature of software failure. Software failure isn’t like other forms of failure. When software fails it usually keeps on running: It fails because it has been told to do the wrong thing.
Software failures, then, are failures of understanding, and of imagination.
Understanding the code instead of understanding the problem
Developers are spending so much cognitive effort and mental space on just understanding the code itself, that thinking about the requirements, developing an intuitive feel for the problem space, and understanding the system in context takes a much lower priority.
I've been a programmer for a good few years, and it seems wrong on some level to admit this, but I've never felt fully comfortable with the tools.
I want better visualization than print statements, debuggers, and grepping log files. I'm sick of endless context switching between code files, the terminal, documentation and googling stack overflow. To me, programming shouldn't be about text entry. It should be about intuition, visualization, exploration, and creation. For a while I blamed myself for my own bad attitude.
Bret Victor and "Inventing on Principle"
But I was heartened to discover that there were like-minded folks out there, pointing out these same problems. Gary Bernhardt, in his 2012 talk "A Whole New World", reimagines a terminal with graphics, code structure, and profiling visualizations built in. But the truly life-changing, eye-opening experience for me was watching Bret Victor's "Inventing on Principle" talk. It showed a vision of what building software could be like: Immersively visual and intuitive, with instant feedback for every single change. In the most famous demonstration from this presentation, Bret adjusts the jumping power of Mario in real time, instead of tediously adjusting values in a text editor.
But most interesting to me was a different demo, where Bret writes code with visualization immediately alongside, much like what you saw at the start. After every single keystroke the code runs, and the state of the program visualized. An interface like this seemed familiar enough to my day-to-day programming experience in a text editor to be immediately useful. But different enough to profoundly change the experience of creating and writing software.
Instant-feedback coding for python
I wondered how well this would work in Python, and built the little prototype you saw before.
Conspicuous by its absence
What's different about it is what's NOT involved. There's no print statements, no log statements, no debugger commands, no terminal, no window switching, no context switching.
Internals: Overriding the debugger
Conceptually the way this works is pretty simple. Usually the debugger stops at each line and gives you an interactive prompt. To create an interface like this, you simply override the debugger to record the changes and continue rather than stopping.
Example: Binary search
So we’re going to take a look now at writing some real code. In Inventing on Principle, Bret implements the classic algorithm of binary search. Let's look , to see what the python experience is like.
Handling external dependencies
But, I hear you all say: That’s all very well for simple, self-contained functions. But in the real world there are external dependencies: Files, databases, web servers.
Luckily we can borrow some of the techniques we’re used to from testing in order to fake resources.
Example: Web scraping
Let’s use basic web scraping as an example. Let’s say I want to download a picture of my dog off my website.
Example: Segway controller
One more example. Let’s take a look at instant feedback coding with a real dynamic system. We’ve all seen segways, obviously to make them stand up straight they need some code to make the wheels do the right things. Let’s see if we can write the code that could make a segway do that.
The nice thing about this is how it feels when you’re working with. It doesn’t feel like work or drudgery, it kinda feels like just goofing around. But on the other hand, it’s quite engaging and really builds intuition for how the tuning process works.
What would disappoint me most is if people went away from this talk thinking that this is some kind of training interface, to be used by children and learners before graduating to real programming. What I hope to convey is that real programming must be like this: Instant and visual.
Computers aren't mainframes for batch processing any more. They're dynamic systems with powerful interactive and visual capabilities. A system truly designed for the spirit of modern computing must take advantage of these capabilities.
My hope is that what you've seen here can offer a small glimpse of what programming might look like in an optimistic future.
Not faster, but different
Away from tedious, responseless text editing towards instant feedback. Away from invisible processes and towards pervasive visualization. Away from black box coding and towards deep insight and understanding. Away from batch processing code files, and towards direct, intuitive manipulation.
"The best way to predict the future is to invent it." - Alan Kay
Do you ever stop and think what programming will be like in 100 years time? Alan Kay famously said, "the best way to predict the future is to invent it". Thing is though, it's not even about invention, because none of the ideas here are in any way new.
Back to the future
It's about taking time to reflect on what we'd like the future of our craft, and our profession, to actually look like. To stop hammering in nails for a moment and consider the hammer, and realize it's actually a carrot, not a hammer, and maybe we should be using a real hammer.
If we want to build a great building, let's maybe stop chucking bricks together in a big pile, and remember that the idea of an arch is been around for a while, maybe we should get around to trying it out?
segway bicycle for the mind
Because the key message of all this is: before you can start to build a real bicycle for the mind, first you’ve got to decide to stop stumbling around in the dark, and open your eyes, and see.
Inspiration: Bret Victor, Alan Kay, Gary Bernhardt
Photos: Kevin Eng (Gutenberg Bible), Jason Scott (VT100), Anthony (Astro)
Feedback: Nick, Jason, Jen, Rob, Katherine for feedback
- Read my related post: Programs are a prison
- Get in touch: If you have thoughts about the contents of this talk and how to make more humanistic interfaces happen, I'd love to hear from you. Tweet at me.
Thanks for listening!