I sit at a really weird saddle point currently. Professionally, I’m obliged to engage with generative AI technologies. I’m an AI technologist. I’ve been doing this for nearly twenty years at this point, and over ten past the doctorate. I can’t just ignore a new technology because I personally find it ethically dubious.

Conversely, I’m a creative person by my nature. I write both professionally and as a hobby. I take immense pride in bringing my thoughts and imagination to paper as an author or to the table as a game master. And let me tell you, both of those communities are incensed by how companies that train LLMs have benefited from their labor without giving back. I try not to mention what I do professionally in those communities because I’d like to stay in their good graces.

So, like I said, weird saddle point. Professionally I have to engage with the tools, because the industry, not any particular industry mind you, just industry as some sort of monolithic entity has decided these tools are useful and this is what we’re doing.

As a hobbyist, and a consultant, I don’t want to use them to do the work that literally brings me joy. I’m not going to be using LLMs to do creative writing on my behalf. I don’t want their help building out encounters, plotting out game arcs, or building a product roadmap. I have found a place where they seem to excel at the table though.

AI backed transcription has gotten really, really good in the last couple of years. It’s so good that I regularly use it when running tabletop roleplaying games. I’ve also taken to using it for a similar purpose in my professional life, in case you’re wholly disinterested in TTRPGs.

Transcribing at the Table

So, as I said, I often play the role of game master at my table. For those unfamiliar, that means it’s my job to referee the game that we’re all playing together: I describe the scene, keep the players on track, help adjudicate the rules. It’s effectively an evening of straddling the line between meat-computer and improv artist. And, with any luck, my players enjoy themselves and decide to return for another game another evening.

That means that, on top of everything else that’s going on at the table, I’m the note taker. I need to remember who contributed how, what story elements were decided on and try to participate in the ongoing game all at once. It’s a lot of plates to keep spinning.  Honestly, it’s a bit like running a meeting or a discovery session.

Offload Session Note Taking to AI

I have found that modern AI transcription techniques, with or without the aid of further processing by LLMs, are extremely useful tools in offloading some of the burden of taking notes during my aspirationally-weekly D&D sessions. Here’s how it works:

1. I record us talking and playing at the table
2. I use a local version of whisper to produce a transcript of that audio file
3. I supply the transcript along with some instructions to an LLM-as-a-service like ChatGPT, asking for a distillation of the events that occurred in the session
4. I proof read the factual digest, correct any mistakes or add any notes I think should have made it in, then hand this back to ChatGPT or similar asking for a session summary.

That’s the whole pipeline. In a physical setting, I use a fairly nice physical microphone that’s suitable for podcasting. In a virtual tabletop setting like discord, I use a tool like audacity to capture the audio as it comes across the wire, and the process resolves in the same was as an in person game.  Using a meeting or chat service here is honestly nicer because it provides insight into who was speaking when, which can improve the overall transcript quality.

But Don’t Offload Everything

I’d like to reiterate that there’s still a human in the loop here. Step 3 requires that I make sure the fact sheet is correct and complete. Similarly, I’m not just handing the session notes back to the table sight-unseen after step 4. No, I read them and edit them for clarity and correctness.

Even though this process requires some of my attention to produce output I’m happy with, critically it does so at a time which is not during the game when I should be paying attention to my players and not my notebook. By taking a recording and using tools to manipulate the format, I’ve made what would be a multiple-hour effort of re-playing the whole session (even at 2x speed) into something that takes about fifteen minutes of my time.

Also, It’s Useful at Work

So, the thing is, this isn’t just something that I use in my personal life to make my hobbies better. It’s a routine part of my work as a consultant. I tend to do a lot of up-front early work with my clients, figuring out what sorts of problems software might solve for them, discussing how hard it would be to build that kind of software, and so on. This work, capital D Discovery work, tends to look a lot like really long meetings, where we all sit around a table, refining a story collaboratively.

The Work Pipeline Isn’t That Different

In the work context, our heroes aren’t swashbuckling rogues or over confident investigators of the eldritch, they’re the users of our systems. Similarly, I would much prefer to spend my time in these discovery sessions engaged in the work of understanding the problem and seeking clarity rather than the tedious, but extremely necessary work of taking notes. AI transcriptioning tools let me do that, and the pipeline is nearly identical to the one I use at my tabletop sessions:

1. We record the discovery meeting
2. We use a tool like whisper to convert the audio recording into a transcript.
3. We pass that transcript, along with a prompt to extract facts, questions, and action items through an LLM in our own private cloud, rather than an LLM offered as a service, to maintain privacy on client data.
4. Attendees (typically my coworkers, but sometimes also the clients) validate the outputs
5. We update the outputs, and pass them through a subsequent step to generate a report of the discovery session, including findings, action items, and things intentionally left unresolved.
6. We review the report and add it to the record of the day

The stakes are a lot higher in the business setting, but fundamentally, the task isn’t all that different. We need the players at the meeting to bring their whole selves to the event, and they really can’t do that if they’re spending a significant portion of their time taking notes. While we could always field a dedicated scribe in every meeting, that’s a lot of overhead, and the tools produce reasonably accurate transcripts compared to even our best internal note-takers. We know, we looked.

 Riding Off into the Sunset

So here I am, perched at that awkward saddle point I mentioned earlier: technologist by trade, creator by calling. I don’t believe generative AI is going to save the world, and I’m not interested in pretending it will. I also can’t pretend it doesn’t exist, or that it doesn’t have some utility when used with care and constraint.

The same tools that threaten to commodify creative work beyond recognition can, paradoxically, help preserve the things I value most in both my creative and professional life. They help me be more present at the table, whether that table is surrounded by elves and warlocks or product owners and engineers. They help me do the work I want to do by taking a burden off my shoulders that was never central to the value I bring in the first place.

This, to me, is where generative AI actually shines.  It handles some of the tedious but necessary work that supports the outcomes we actually care about. It doesn’t make me a better GM or a better consultant. I still have to be good at those things on my own. However, it does let me focus on the events at the table by offloading the important, mechanical work until later, when my players or clients have gone home.