I recently learned that in 1843, Ada Lovelace translated an Italian mathematician’s paper from French into English. As if that’s not impressive enough, she added her own notes, which ran about three times longer than the paper itself. One of them, Note G, held a table often described as the first computer program. It also held an argument now known as Lovelace’s objection
A machine, she wrote, “has no pretensions whatever to originate anything. It can do whatever we know how to order it to perform.” i.e., machines cannot create anything beyond what they have been programmed by a person to do.
With generative AI booming at the moment, I am drawn to Lovelace’s objection in the modern age. Several recent articles from SEP engineers and designers have got me thinking…
How much of what a machine produces comes from the people directing it?
What is Lovelace’s objection?
Lovelace’s objection is the argument that a machine can only do what people know how to tell it to do, so it can’t originate anything on its own. It comes from her 1843 notes on the Analytical Engine. She argued that the Analytical Engine could only do what people knew how to tell it to do. More than a century later, Alan Turing brought her claim into a bigger debate about whether machines can think. He took on “Lady Lovelace’s Objection” in his 1950 paper Computing Machinery and Intelligence. Turing described machines surprising him often and he also argued that a machine built to learn could do things its programmers never expected. Some historians think the label misreads her, since the same notes imagine the Engine composing music.
The argument itself may be old, but the implications feel perhaps more relevant than ever. With a few modern tweaks, we can explore what Lovelace’s objection might sound like today.
| Question | What SEP’s people say, in short | Read |
|---|---|---|
| 1. Can AI come up with original ideas? | Owen explores how to keep generated stories consistent. Brian examines the context an agent needs to produce useful work. | Owen Dossett, Brian Hanford |
| 2. Can AI think for itself? | The posts describe agents adapting to problems, alongside limits caused by missing context. | Jason LaJeunesse, Brian Hanford, Robert Herbig, Brian Ball |
| 3. Will AI replace software developers? | These accounts show how AI is changing the balance of coding, review, testing and design. | Robert Herbig, Nicole Selig |
| 4. Is AI making engineers worse at their jobs? | An internal debate explored the difference between working faster and keeping the skills to judge the results. | Vish Patwari, Robert Herbig, Nicole Selig |
| 5. What should humans still own? | Teams describe owning the problem, the plan, and the check on whether the result is what people meant. | Robert Herbig, Brian Hanford, Jason LaJeunesse, Nicole Selig, Keith Hanson |
1. Can AI come up with original ideas?
This is the question Lovelace answered most directly. Her word was “originate,” and she said the Engine had no pretensions to it. Anything it produced would trace back to the instructions someone gave it.
But Lovelace also imagined the Engine working with more than numbers, including music. If musical harmony could be written in its notation, she wrote, “the engine might compose elaborate and scientific pieces of music of any degree of complexity or extent.” The person behind Lovelace’s objection was the one imagining a machine that could compose.
Owen Dossett ran into this while building an AI Dungeon Master for a text-based Dungeons & Dragons game. His storyteller made up plenty. It “often figured the best thing to do in the moment was to create new narratives (even if they conflicted with old ones).” So he gave it a memory in three layers. The first is a log of everything that happens in the game. A vector database lets the system search past dialogue and scenes, and a knowledge graph maps how characters, factions, places, secrets and quests connect. Before the AI tells each scene, the system pulls in the pieces that matter.
Brian Hanford sees the limit from the engineering side. He looked at why AI agents produce what people call slop, and he asks how much of the problem we’ve explained to the agent. He concludes that what a model is “genuinely, remarkably good at is translation.” It turns a problem people have already solved into code.
So, can AI come up with original ideas? And can it do it without the original instructions from a person? I really don’t know! Owen’s example seems to show AI going off script from his provided instructions, which seems pretty creative. So creative that Owen had to provide even more guidance. But Brian’s post would say the agent was doing what it had been programmed to do, so those missing instructions were part of how it was programmed.
My Verdict: Undecided 🤷♀️
2. Can AI think for itself?
Turing’s 1950 paper opens with the question “Can machines think?” Lovelace’s objection is one of the arguments he answers. A machine that only follows orders never decides anything for itself. Today’s agents choose their own next steps, so the objection gets a fresh test.
Jason LaJeunesse, SEP’s Director of Data, says an agent can handle more as the model improves. He spent a month building a complex agent framework. Then he replaced it over a weekend with a simple loop (reason, act, observe, repeat) that worked better. “The architecture didn’t change. The intelligence inside it did.” Among the gains he lists is better judgment. “The model knows when to stop, when to try a different approach, and when to ask for help.”
Brian H. again focuses on the context an agent never received. An agent “never took that journey” through the problem the way the person asking did. “All it has is what made it into the request.” It can ask clarifying questions, he adds, but usually only when the prompts and tools around it are built to invite them.
Robert Herbig, SEP’s AI Practice Lead, describes a company rebuilding its payroll product by having AI write all the specs (detailed written plans) first. Five months in, they had about a million lines of specs and no code. One spec told the system to pull payroll data from the browser’s cache, which is temporary storage on someone’s computer. Nothing in the process caught it. When a team member asked how they knew the specs were right, the first answer he got was, “Claude says it’s correct.” As Robert puts it, “Doubt exists in this system, but it has to be summoned by name.”
In Brian Ball’s experience report, an agent worked through eight bugs the client’s software development product manager found in a beta build. It fixed seven, each with a test and its reasoning. The eighth needed more information and may have involved a misunderstood coincidence with the crash-reporting system.
This question of whether AI can think for itself feels particularly existential to me. I’ve often heard it argued that our ability to think is central to what makes us human. So if AI can think? I am actually now regretting asking such a heavy, philosophical question…
My Verdict: Probably Not (yet) 😅
3. Will AI replace software developers?
Lovelace’s sentence has a second half that gets less attention. The Engine “can do whatever we know how to order it to perform.” In 1843, the people who knew how to give those orders were the first programmers, Lovelace among them.
Robert noticed his own job change first. At some point in the last year, he was spending more time reviewing code than writing it. He explains it with the Theory of Constraints, an idea from manufacturing. Making one step faster doesn’t help the whole team deliver faster when work is already backing up somewhere else. For most experienced teams, he writes, “writing code was never the slow part,” so the pressure landed on review, testing and acceptance.
In another post, Robert describes a company whose CTO’s stated goal was to not need developers at all. A team member told Robert that five months of work had produced about a million lines of Markdown specs. There was still no code for the product rebuild.
Nicole Selig questions how much AI-assisted review can make up for the volume of AI-generated code. In her post on what AI did to code review, she writes that “asking another AI to inspect the mountain of code our first AI generated can only take us so far.”
With AI now writing so much code, it makes sense to ask what happens to the people who used to write it. What I’ve learned since joining SEP is that the plug-and-chug of writing lines of code is a small piece of a very big job that engineers do here.
My Verdict: No? 🧑💻
4. Is AI making engineers worse at their jobs?
In Note G, Lovelace showed step-by-step how the Engine would calculate Bernoulli numbers, a sequence used in math. No machine existed yet to run it. Her objection assumes the people giving orders understand the work.
Software engineer Vish Patwari brought this question to an internal debate at SEP. “Is AI making us incompetent under the guise of speed?” The side arguing yes “leaned hardest on code review, that people were approving generated code they couldn’t have written.” The side arguing no “pointed out that the same complaint followed Stack Overflow and IDEs.” In Vish’s words, “Nobody landed a knockout. That was the point.”
Nicole’s post fits this debate, too, because she looks at how the volume of generated code changes the experience of reviewing it. Big AI-generated changes turn a reviewer “from a thoughtful collaborator to a skeptical auditor.” She says that “leaves us feeling cognitively exhausted and socially empty.”
In Robert’s post, Joe Manley described noticing a change in his own approach to review. “I probably have taken a lighter approach with these PRs because, in the back of my mind, I know that AI has helped.”
I attended the first debate Vish hosted and walked away completely convinced that both sides were right. We’re also looking at the first generation of engineers using AI in their work, so they had experience with long-form coding that future makers simply won’t have access to.
I see a version of it in my own work. A lot of my coaching is helping writers who use AI make sure their own voice comes through, because the ideas are theirs and the post should sound like it.
My Verdict: Probably Not (yet) 😅
5. Should humans still own the code when AI writes it?
The machine “can do whatever we know how to order it to perform,” Lovelace explained. She went further in the same note. The Engine, she wrote, “has no power of anticipating any analytical relations or truths. Its province is to assist us in making available what we are already acquainted with.” In her Notes, the anticipating was hers. She took someone else’s description of a machine and added the context, the examples and the meaning it didn’t have.
Robert’s post looks at teams that make their expectations clear before code is generated. Rui Liu’s team writes a design document before any code. Reviewers can then check the code against a plan everyone already agreed on. “The humans are creating structure and enabling constraints so the AI can operate within them.” Brian Hanford puts it more briefly. “The hard thinking happens up front, before any translation starts.”
Jason adds domain knowledge, data and security. The teams getting the best results “deeply understand their domain and have invested in their data and security ecosystem.” That lets the agent work inside limits they trust.
Nicole says code review also spread ownership across a team and taught junior engineers. “AI has dramatically changed how quickly we can produce code, but it hasn’t changed the fact that software development is a team sport.”
Keith Hanson’s post about Cucumber gave me another way to think about Lovelace’s objection. He examines how hard it is to make the same description useful to technical and non-technical readers. “A single jump from technical to non-technical doesn’t really work.” Writing instructions for an AI agent works a lot like writing a Cucumber scenario. Someone describes what they want in plain language, and a machine turns it into working software.
This is perhaps the clearest answer of the five questions, IMO. The humans own the knowing and the ordering. They determine the business, its rules, and the design work that happens before any code exists.
My Verdict: Yes 😊
So, does Lovelace’s objection still hold?
Tallying up my verdicts, I have one undecided, two probably-not-yets, one no (with a question mark) and one yes. So it seems Lovelace’s objection mostly still holds, at least for now.
AI has definitely given me things I wasn’t expecting. Usually, though, it was working from something someone else had already shared with it. People learn from each other too, but we also draw on our own experience. Maybe experience is the piece AI is missing.
Lovelace had that piece. Her Notes ran three times longer than the paper because she brought her own examples, questions and imagination to someone else’s machine.
So I’ll keep letting AI take a good chunk of the way, and save the polishing, the improving, and the creative parts for the humans. 💜
✨ AI Post Recap
In 1843, Ada Lovelace wrote that Charles Babbage’s Analytical Engine could not create anything new. It could only do what people knew how to order it to do. Alan Turing took up Lady Lovelace’s Objection in 1950. Recent posts by SEP’s engineers on AI agents and code review show where her question still applies. Agents adapt to problems, and the people directing them still define the problem, supply the context and judge the results.
What is Lady Lovelace’s Objection? It is the idea that a computer can only do what people know how to tell it to do. Ada Lovelace wrote it in her 1843 notes on the Analytical Engine. Alan Turing named it and responded to it in his 1950 paper Computing Machinery and Intelligence.
How is AI changing software developers’ work? In accounts from SEP’s engineers, AI produces code faster, and more of their time goes to reviewing and testing it. One company tried to work without developers. In five months, it produced about a million lines of written specs and no code.
Can AI come up with original ideas? AI invents new material easily. Keeping it consistent takes structure. For example, Owen Dossett gave his AI Dungeon Master a three-layer memory so new scenes wouldn’t contradict old ones.
