As a software architect, I do what I can to stay abreast of the world around me. As new technologies, design patterns, and so on emerge, I try to learn enough about them to make informed decisions on where they might prove useful or if I can safely ignore them. I think about their relevance and ways I can responsibly apply them on projects with which I am involved. I have to. It’s critical to look at which technologies best suit a specific problem so that the right solution can be implemented for SEP’s customers. That is to say, that which satisfies the needs of the project with an outlook on what might be needed in the future while trying to minimize both time spent and budget used to complete it. I say this as a disclaimer because…
As a hobbyist, I think it’s fair to label me as a Software Curmudgeon™. I don’t care about the shiny new things (Get off my lawn!) that other people in the industry have done. Those bore me. I want to innovate. I want to do terrible things. Terrible. Things. I want to redesign the wheel because, well maybe, just maybe there’s a better way to do it that nobody’s ever thought of before. And do you know what? Sometimes, I’m right. When I am, I apply those lessons to my professional projects and avoid those things that didn’t work.
Besides, if I’m going to fail, I’ll go all in and fail spectacularly. I feel that if I don’t do this, I’m doing myself – and potentially future projects I might work with a client on – a huge disservice. Personal projects are where failure is safe. I’m the only person affected by it.
This all leads me to a discussion about my latest personal project.
My Latest Terrible Thing
It all started with a dream a bit over three years ago. “Self,” I thought, “I have the land-loving mother pigeon of all terrible ideas.” Needless to say, this piqued my curiosity and I asked myself to elaborate… It turns out that when I dream, I dream big. I wanted to make a game. Without getting into the minutiae, what I needed at a high level was the following:
- An engine for the client that could give me a nice 3D world.
- A clean means of having a massive number of objects interact with one another without having to deal with threading, network, and resource contention issues that always arise when doing something of this nature.
- At the end of the day, I had a story to tell and wanted to get the hundreds of characters in the game properly voice acted. I also would need artists to create 3D models for me. I am bad at art. As for the music, I decided to try doing that myself.
Before I brought anyone else on the project, I set up my tool chain. While I was the only person writing software, I had about a hundred voice actors, a couple artists, and another person who created some musical compositions. I needed a good revision control system. More importantly, many of the voice actors were not comfortable using technologies more sophisticated than e-mail for submitting their work. I needed to be flexible.
Right now, 95% of the server side is done – only adding actual story content and a couple “nice to have” features are left. The client is about 50% complete. I’ve innovated. I have done terrible things. I’ve also done some cool things. I think it’s time to start sharing.
You chose WHAT???
It’s time now to make a horrible confession. The bulk of the “server side” technology I chose is from 1989 and really hadn’t seen much active development this century until I dove in. However, apart from a few admittedly significant holes, this technology was the perfect solution. I’ll save going into the details of what this technology is for later.
At a high level, it is a virtual machine (VM)/environment. Each object controlled by it is treated as a fully-separate, completely autonomous/asynchronous program. All programs get an equal slice of processing time from the scheduler. Everything is decoupled – interactions between objects are as simple as one object calling a method on another. The VM also has a full network stack and interactivity is all abstracted behind a few simple library calls.
The programs/objects are written in a language surprisingly similar to C# with some very heavy Scheme influence. It has pretty comprehensive support for closures. The programs are “compiled interpreted” – meaning that the first time they are accessed, they are purely interpreted but the result is then cached as a compiled object such that subsequent instantiations/cloning do not need to interpret the code again. However, as I said before, there were some rather significant holes that I had to resolve:
- There was no automated testing framework
- No true compiler/interpreter that could be used outside the VM
- Event handling was lacking
- No framework libraries. Having worked with modern development environments, this last point was particularly jarring.
None of this dissuaded me.
Looking ahead
In subsequent entries, I will be discussing the overall architecture of the project. This includes its external dependencies and how the applications and component services were broken down. I will also discuss what I did to fill the above-mentioned gaps and how I did it. What was it like doing a large software project as the only developer with a lot of non-developers? I’ll cover my experience with that as well. Some of the highlights include:
- The overall architecture for the project
- Modifying the driver / VM to support git, the file system’s (zfs) advanced features, and external service support.
- OK, this is kinda embarrassing, but it’s time for another confession: I decided to go complete overkill and add Oracle database support to the driver as well.
- Creating an automated test environment (creating a compiler for this language, building a test framework, and setting up Jenkins/VSTS to handle it)
- Adding what were really useful framework components for this project. These include: event handling, state machines, a fluent LINQ to SQL-like ORM, role-based authentication, data transfer using something somewhat like ASN.1, a service framework, and so on)
- “Soft skills” – what it’s like running a large project with many contributors when you’re the only one writing software and the only one familiar with software development practices.
