Purpose
TimberWolf is a not-yet-production-ready game engine written in pure Rust. It was previously being developed using C++, but in anticipation of the direction I expect game development to go in the future, and to assist in meeting the objectives that I had for the game engine, I made the decision to rewrite the whole thing in Rust.
TimberWolf currently has five ranked priorities, each one being higher priority than the next:
- to enable developing seamlessly cross-platform games
- to empower the communities behind the games to create content
- to make game code triply safe (type-safe, memory-safe, thread-safe)
- to maximize performance
- to offer extensibility and flexibility over ease-of-use
1. Seamless Cross-Platform Development
I love gaming, but I hate the near-monopoly that Microsoft has held on the gaming market for so long. I’m a huge proponent of Linux gaming. While Linux gaming has become vastly easier in recent years, especially with Valve’s Proton pushing Windows emulation to great heights, there’s still a lot to be desired. Frankly, if games aren’t developed with cross-platform in mind from the beginning, it tends to create a massive amount of tech debt to achieving it in the future, which in turn creates more barriers that platforms like Proton/Wine have to overcome. Simply put, Linux gaming needs developers to consider Linux. The same is true for MacOS, or any other operating system for that matter.
Plainly, project managers at game studios are hard to sell on the idea that supporting games on other platforms is a worthwhile venture. First because Windows has the vast majority of market share in gaming (which, I’d argue, is a self-fulfilling prophecy because of project managers like this). This in turn makes it hard for them to justify the potential additional effort, however small it may be, to build with cross-platform in mind from the start. It gets especially hard to get support as the development progresses, because the amount of work only increases the longer it is delayed.
What I intend is nothing new in game engines. For example, Unity has great cross-platform support that requires no additional effort on the developer’s part. With TimberWolf, I’d like to create a set of completely platform-agnostic APIs. Developers don’t need to get permission to support other operating systems because it’ll already be done for them. The only sensible course of action would be to release the builds for all platforms, which in turn, should help to bolster the Linux and MacOS gaming communities.
2. Empowering Community Content Creation
Many of the most successful, replayable, and memorable games are those that sustain themselves by allowing the creativity and passion of their player bases to help drive the development of the game.
One way that I would like TimberWolf to encourage this community-driven game development is to provide a easy, safe, and flexible modding API system. A good modding API allows mod developers to modify the game, and then release the code in a user-friendly format, which can then be run alongside other mods in an end-user’s game.
Easy
It’s often the case that game engines don’t provide for running third-party code out of the box, and instead, the implementation of such a system is left to the game developer to figure out. I would like to provide a library that allows developers to start integrating external mod APIs right from the beginning, using standardized patterns and systems.
Safe
Another common problem with mod APIs is that they’re an extremely simple attack vector for running malicious code. Most mod APIs run the code with the same privileges and context that the game itself has, without any safeguards to limit access. This means that mods can usually access and even modify any resource the game can, which is probably a lot.
I would like to solve this problem by providing a permissions-based access control system, which will help to reduce the attack surface of the mod API. Obviously this wouldn’t completely mitigate the possibility of malware in mods, but as an example: if a sufficiently cautious user is installing a mod that adds a set of new weapons for the player character sees that the mod is requesting unlimited access to the device storage, they might reconsider. As an added bonus, if a mod has to ask for permissions to communicate over the network, this will greatly discourage mod developers from snooping on users or backdooring their mod.
Being able to limit the access of third-party code running on a system requires that code to run in an untrusted context. This eliminates the possibility of running the code natively. Instead, the code has to be run inside of some kind of virtual machine. See the next section for more on how I propose to make this work.
Flexible
Most mod APIs (especially those attempting to have decent security) require the mod to be written using a specific programming language. This often results in the discouragement some developers who otherwise might be willing and able to contribute mods simply because they aren’t comfortable with the requisite language.
In tandem with the safety requirement from above, being flexible with the programming languages usable for mod development would require some kind of virtual machine. The JVM and CLR are both great VMs, but they are both unsuitable for this, because they are both targeted toward, and in fact, limited to, a small subset of programming languages. Fortunately, recent years have seen the creation of a new virtual machine spec called WebAssembly.
Don’t let the name fool you. Although WebAssembly was originally intended to provide high performance computing on the web, it is actually a very general-purpose virtual machine specification that can be used for anything. And better yet, it was designed to be a compilation target for any programming language that can compile to machine code. Thus, with WebAssembly as our VM of choice, we would get reasonably high-performance execution, the ability to run code in an isolated, untrusted context, and the ability to empower developers to use any language that can compile to WebAssembly (which is a growing list, and will likely continue because of the ubiquity of the web).
3. Triple Safety
Rust, enough said. You want more? Okay.
In any application, there are a number of recurring issues that result in bugs, or even worse, security vulnerabilities. Most of these fall into failures of one of three kinds of safety.
Type Safety
Code that fails to be type-safe is code that may mistake one type or format of data for another. For example, a floating point number being interpreted as an integer will probably not behave how you want, because the format of a floating point number is completely different. Instead, a type-safe language will cast the data to the correct type before attempting to use it. Or, in other cases, it may just warn you that the types are incompatible before letting you do something dumb with it.
Any strongly-typed language is type-safe by default, Rust included. Nonetheless, Rust, C++, C, most lower-level compiled languages in fact, allow you to do very unsafe things such as reinterpreting data or accessing a void pointer. In those cases, even these “strongly typed” languages can lose type safety.
In the case of TimberWolf, I hope to minimize the amount of unsafe code. Likely, some will be required, and type-safety could be at risk in those areas. However, Rust is uniquely suited to making “unsafe” code easier to audit, and more maintainable, because all unsafe code has to be marked “unsafe”.
Memory Safety
Code that fails memory safety is able to access memory that it isn’t supposed to have access to. This may be a pointer to a deleted object, or it could be a malformed pointer of some kind. These can result in some of the worst kinds of security bugs, they can create undefined behavior in the application, and in the best case scenario, they will crash the application with a segmentation fault.
Once again, Rust is uniquely equipped to keep memory safety sacrosanct. Rust uses a compile-time borrow checker to ensure references will be valid wherever they are used. When compile-time borrow checking cannot be done, Rust includes standard library features to enable runtime borrow-checking through a 100% safe API (100% safe meaning: not requiring you to write unsafe code; the library does all of the checks, and if you screw up, it’ll stop you).
Once again, using “unsafe” code in Rust would allow bypassing these protections, but minimizing unsafe code, and the nature of all unsafe code being grouped together and treated differently by the compiler should help to mitigate memory safety issues.
Thread Safety
Code failing thread safety usually results in undefined behavior such as race conditions, or the application freezing due to a deadlock on a shared resource.
Do you see a pattern yet? Once again, Rust is here to save the day (as much as you’ll allow it). Rust’s standard library includes many resources for what they call “fearless concurrency”. This basically means lots of utilities for lock-free multithreading and atomic data types. It also has easy-to-use mutexes for when lock-free programming isn’t an option (but lock-free probably is an option, you might just not have figured out how it’s supposed to work yet).
Rust definitely has made thread safety one of its focuses, and when using the standard library correctly, there are lots of protections in place. In order to mitigate thread safety issues in TimberWolf, I’ll be using lock-free concurrency wherever I can. Poorly-written threading code is always a risk, but I do hope to abstract thread creation and management into a game (or app)-specific API, so I can try to discourage developers from spinning up their own threads willy nilly.
4. Maximizing Performance
Once again, Rust. Enough said.
Okay, fine. Rust is a strongly-typed low-level language that compiles to native machine code on any platform you compile it to. The compiler, like most C and C++ compilers, includes extensive optimizations to greatly improve performance. If you look up any standard algorithm benchmarks of Rust code, you’ll see that in the worst case, it is barely slower than C, and in the best case, it is faster than C.
Of course, the language itself isn’t the only concern. I would like TimberWolf to include highly efficient state management system (ECS, or some other patterns, it hasn’t been decided yet). If the passing of events between subsystems can be optimized appropriately, this has the potential to cut down massively on calculations and memory operations that are unnecessary, which would drastically boost performance.
Concurrency and parallelism are also a major focus for performance. Currently, CPUs are scaling up in terms of core counts rather than clock speeds. This means that in order to scale with new tech, games need to follow suit and scale across cores.
And lastly, in games, graphics are one of the most performance-critical parts of the engine. Performance can be optimized by utilizing the latest graphics APIs. In the case of TimberWolf, I will primarily be focused on implementing and optimizing a Vulkan graphics backend with a high-level graphics API in the engine itself. This means that the game developer need not know anything about Vulkan or how to optimize it. They will simply be able to use standard modeling workflows, standard PBR shader workflows, and standard skeletal animation workflows to develop their game, and let the game engine worry about talking to the graphics card. By completely abstracting away the graphics API, TimberWolf also gains the flexibility of being able to make major changes to the graphics implementations without creating breaking changes in games, and if needed, in the future a different graphics API could be used (or likely, developers would be able to drop in whatever graphics API they want to use at compile time as a configuration option).
5. Extensibility and Flexibililty (Over Ease of Use)
This engine will not be targeted at beginners in game development. Ultimately, this will be intended to be a production-grade engine aimed at advanced Rust developers. While ease-of-use will not be entirely thrown out, whenever we can make the engine more robust or offer more capabilities, but it will incur the penalty of making the engine more difficult to use, it will be decided in favor of adding the capabilities.
Summing Up
To achieve these priorities most effectively, I also want to provide the most robust abstractions for game development that I can. Nothing is 100% set in stone quite yet, but the goal is to build the engine in a data-oriented way, and make the API as declarative as possible, with a more specific focus on organizing the entire engine around push-pull reactive state. A declarative/functional API will aid in the extensibility and flexibility, while also providing more intuitive behavior, which should make games written with TimberWolf less “buggy” than others (of course, the responsibility for ensuring this still rests with the end developer). Using the push-pull reactive state is quite a bit less certain and more experimental at the moment, but the general idea is that it will make the code governing the propagation of state more declarative, succinct, and intuitive, and at the same time minimize the number of times state needs to propagate through potentially expensive functions.
A high level of abstraction will be less of a priority than throughput performance and responsiveness, however. Where needed, the API will be lower-level in order to facilitate maximizing performance to an extreme degree. Rust is a language that is compiled to machine code, and the compiler includes high quality machine code optimization. Additionally, it is a statically-typed language, and uses zero-cost compile-time abstractions to provide most of its safety features. Thus, Rust is an exceptional language to achieve performance objectives. Additionally, TimberWolf will implement the latest graphics APIs, with emphasis on Vulkan, to provide the opportunity to fine-tune the graphics pipeline. TimberWolf will also include support for native multithreading and GPGPU for games with heavy computation needs.
Status
TimberWolf is still in active development, and the API is still in the alpha state. As such, it is not recommended for production work, however, if you’d like to get your hands dirty and help out with the development, that would be great!
TimberWolf is an open source project licensed under the MIT license. It goes great with any open or closed-source project, and can be used commercially (per the terms of the MIT license, read it before using it).