What is the PropellerEngine?
PropellerEngine is a cross-platform game engine written in C++. It has an experimental integrated editor written in C#. I started the project in January of 2014, over the next years I developed the engine by myself.
The project has always been educational, so instead of aiming to certain set of features I kept implementing and testing features that I would like to know how to implement.
Source code for the PropellerEngine has now been released under BSD license. You can find the code on my Github repository.
Please note that this code does not contain the full Git history, or any code that has been licensed by Sony, so do not expect to find any PS4 related code in there.
The release is as-is, without extra documentation added for possible future developers. I might add some if there is interest for using this engine.
The original plan was not to make an engine, but to make a 2D game. The game was supposed to be top down shooter. (Inspiration was from “Hotline: Miami” and “Tapan Kaikki”) I had created similar types of prototypes for this kind of games before, so technically the original scope should have been really easy.
Because I had made the same kind of games multiple times, I was sick of the basic things I had to do every time, this time I decided I could make a “framework”, so that making the next game would be faster. Little did I know that this would open the endless hole of engine development…
Before this project I have tried to make game engines couple of times, but all of them crashed due to too large scope or not enough experience.
As the project was an educational one, I was learning while I was implementing the features. Because of this I had to do some refactoring to keep things under control. Fortunately I was able to finish up almost all features without refactoring them.
During this project I ended up with almost 2000 commits. I went through the commit log to see the full history of the project.
Start of the project
January 2014 I started the project. During the first month of the development was fast and I was able to test out a lot of small features.
After the first month I already had a lot of base systems up and running for the engine.
- Basic scene system
- Input system using signals
- CVar/Console system reused form previous project
- Entity component system
There were some signs already of me starting to go away from the “small framework” idea, to a full game engine. There was simple implementation of the C#-editor already in place. At this point it was mostly planned to be a level-editor, not a fully integrated editor.
Graphics were done with SFML, as I was not planning to implement a custom renderer at all during this point. If I knew that I would not have used SFML: I would not have included it at all at this point of the project. All features that I had implemented using SFML had to be remade. (Input, 2D-graphics, etc)
There were some plans to have a custom text format for creating entities from them, later in the process the entities were saved in binary format.
Base system implementation
During the second month I heavily improved the ResourceManager and implemented a cross-platform filesystem support. This was mostly the final version, as no more major changes were made to the ResourceManager/Filesystem.
First base system for the serialization started to take shape. There were some factories to create objects and custom serialization data implemented into the objects.
C# to C++ interoperability was heavily improved, mostly complete. The editor was able to receive a list of game-entities on the scene.
(Note that the plan was for future weeks, not containing the first two weeks)
I had planned the following 12 weeks of work, during this time I would have finished up the framework and created a little prototype for a game.
Completely losing the original focus
Third month was the point where I seemed to have completely given up with the original idea of making a game and small framework along with it. I was aiming for a full engine. The editor was getting a lot of nice features, the entity-component system got a lot of attention and the week planning shown on the above screenshot was removed from the repository.
I was doomed to finish the original plan, but this new direction was much more exciting and interesting. In the end this was the right path for this project.
Interestingly I still had the GameInstance running under the game, not under the engine. It was one of the few remains when I was still working on a framework, not on a full game-engine. The game project still had more control over the runtime than the engine project.
During the following months I had great progress with the integrated editor. I implemented RCC++, full undo support, multiple engine instances and many more features to the editor. dynamic_cast usage was also changed in most places to use my own RTTI system.
Most of the features can be seen on a video I published around the time.
By June of 2014 I had started my first cross-platform experiments. The engine was successfully compiled for Linux.
SFML removed & Android added
By September of 2014, shortly after tagging the version “0.9” I started two big changes. Remove SFML and add Android support. Removal of SFML was going to happen anyway, but for Android-support it was necessary right away.
PropellerEngine was added under Travis at this time. It compiled the Linux versions, so when I coded on Windows I could easily see when I accidently broke some cross-platform related stuff. I should have added the project into CI earlier to the project. Later to the project I also used Jenkins.
CI is really helpful when working on project with multiple different platforms. The CI was running on Linux, so I was able to see when the Linux builds were failing without manually building them. When Linux builds fail, almost always most of the platforms failed. Now, I would try to extend the CI to compile all the platforms that were targets, not only one of them.
The engine was developed for over 1 month without a renderer at all. By this time I ported a lot of stuff to Android, implemented a new input support on Windows, etc. (As I no longer had input from SFML.)
About one month into the D3D11 support, I changed the scope of the project again. I decided to also support 3D on top of the 2D. Part of this decision was me wanting to do 3D-graphics, part of it was that I had already solved problems with 2D-graphics and wanted more challenge.
Conclusion for the year 2014
November of the 2014 was most likely the most productive month for the whole project. Even if I didn’t have as many commits as during the start of the project, I had more features implemented.
During this time I implemented my own 3D-math classes. I got crossplatform 3D-rendering working on both Android and Windows, using GLES2 and D3D11.
Lots of small features were also implemented. First version of a scoped profiler was introduced, it was later improved and released as a ProfilerPropeller. HTML5 version was also released using the Emscripten. (Works best on Firefox)
For 3D-graphics I implemented my own simple 3D-mesh format. (.PropellerModel) User would import a model using the editor, the engine itself would use the internal format only. This simplified the model loading on the engine side, as the more time consuming converting process was happening in the editor.
All thirdparty dependencies were moved into the repository itself, so now I was able to compile the project without any issues.
Here is a screenshot from a version that was 1 year old.
Note the integrated profiler in the editor. This was later replaced with the PropellerProfiler.
Second year of development
By 2015 I had started this blog so most of the development process can be seen on my previous blog posts, some of the most interesting ones are listed here:
- Multiplatform status of the PropellerEngine
- Drag&Drop for the editor
- Closing to the engine 1.0 release
- Rendering fixes and more platforms
- New renderersystem and multithreading
- PropellerEngine 1.0 and some automation
- Rendering refactor finished
To summarize the year of 2015. I started the year by working a lot on the graphics features, as well as implementing physics system using Box2D and Bullet physics. (Also some experimenting with PhysX).
Closer to the summer I started to port the engine to PS4. At this point I decided to completely rewrite the rendering backend in order to make the PS4 version working. During summer I took a break from the project again. After the summer I started the work on the new rendering backend. The D3D11 support was implemented in a couple of months.
Once I was done porting the engine to PS4 and finished the D3D11 rendering the least interesting 3 months of the project started. Fixing all the small bugs across all the features.
What could have been improved
For a project of this scale and size, it is not unusual to have some issues. Fortunately most of these issues were minor and under control. With some more development time most of these could be fixed.
There were some features in the project that could have been improved. Usually these features had to be cut because of lack of time or to make room for other features to be implemented and bug fixes.
While runtime compiled C++ worked really well, keeping it working had some issues. Big changes to the low level engine interfaces usually broke this feature. Any changes to the build configuration could also break the system. Also, the serialization had to have special care to make sure the state is not messed up during the reload.
RCC++ was heavily tied to the editor interface. This meant that if the editor wasn’t working, or there were big changes to editor, RCC++ would break.
If I would now go and implement RCC++, I would probably move it away from the editor into the base game itself. Instead, making the engine more modular would make this feature even more effective.
While the editor itself was a great success, it was also one of the most “work-in-progress” features.
Having an integrated editor is one of those parts of an engine you have to almost completely finish before it starts to be usable. For example I could add couple of small features into the rendering part of the engine and they would not feel “incomplete”, but with the editor even with multiple big features, it was far from a production level editor.
In the end this was not a technical failure, but rather lack of time to finish it up to the state I would like it to be.
While the automated build system worked really nicely for building and releasing the engine, the build configuration lacked a lot. I targeted PS4, Android and Windows from VS, while other platforms were targeted using premake4. For Emscripten the VS tools were old and using Emscripten with premake4 turned out to be inefficient.
When targeting multiple platforms with multiple different toolsets, a custom build system is usually going to be really useful in the long run.
During the end of the development cycle, deadlines and lack of time caused some drop on the quality of the code, some hacks were introduced to the engine. Fortunately these hacks were mostly minor and could be fixed in couple of a days. The scope of the project was a perhaps a little bit too large for one person to handle perfectly.
Features that worked well
Cross platform support is one of the best features in the PropellerEngine. The engine has been run successfully on the following platforms. (Some have had full rendering support, but headless support)
- Windows (DX11, full support)
- Linux (Headless server, GL was planned)
- Mac OS (Headless)
- Android (GL, full support)
- PS4 (GNM, graphics in progress)
- HTML5 (GL, full support, some optimization issues)
Windows has been the primary development platform, so naturally that platform had the most support. Android was the second platform that had fully working support and feature parity with the Windows version.
The Linux version is really solid, but it got never got full graphics. This was not a problem because I used Linux on my continuous integration systems. However, implementing graphics for Linux would be trivial, as there was already a working OpenGL backend written for Android.
Mac version is mostly identical to Linux, it has mostly the same systems running, just some headers changed. OpenGL would also be trivial to implement for that platform.
PS4 is the most interesting platform to port the engine into. It was also the most pleasant platform to develop for. The first rendering backend had some design issues that made it difficult to get the rendering to work on this platform. Unfortunately after the rendering refactor, I haven’t had time to get the rendering back to working on PS4.
The biggest issue with the HTML5 version was not the coding/porting, but the tools. When using Emscripten, the code behaves like it would be running on Linux, with GLES rendering. This made the task easier for me, as I already had working Linux version and rendering running on GLES.
Visual studio of Emscripten required VS2010, that I stopped using a while ago. Building stuff from commandline with Emscripten is easy, but my command line builds were done with premake4, a system which was not easy to build Emscripten support for.
For a project of this scale I would really consider writing my own build systems. If you want to target multiple different platforms, it is rare that you can find a tool which supports building to all of them.
Second version of the rendering backend was really successful. Big change from the first version was to separate the low-level rendering and the high-level rendering. This allowed me to work on the rendering features without touching the API specific code most of the time.
Shader cross-compiling is also something that worked out really well. Instead of rewriting all the shaders, I just automatically compile the .hlsl shaders into .glsl shaders.
Looking back, I should have perhaps cut Android out from the project completely and focused more on the 3D-graphics.
While the integrated editor was never fully finished, it was still one of the big achievements in the PropellerEngine. It had a lot of features that would be expected from a full integrated editor.
Multiple instances of engine running under the same process for example is one of those features that is not so easy to implement, but is not that visible for the user.
Undo-system is something that people expect from an editor. The implementation that I used for undo-system creates a list of all operations made for the active scene. All operations always have an invert-operation. This way, if you add an entity to the scene, the undo-list is going to get operation of removing that said entity.
Play-in-editor is a highlighted feature of the modern game engines like Unity and UE4. I was able to also get this feature to work nicely.
Serialization was enabling me to create a lot of other features in the system. All of the following features used serialization one way or another:
- Save / Load game (quick saving!)
- Save / Load scene in editor
- Entity list
- Component lists
- Experimental networking
- Experimental LUA support
During the final parts of the development, automating the release builds saved a lot of time and helped me focus on the code and not releasing things.
I wrote about the system more in depth on my previous post.
In the end, one full game was successfully created with the PropellerEngine by a team of other students. I was providing engine support, and did not work directly on the game.
Screenshot of the Foam Raider: Arena, running under the PropellerEngine
The PropellerEngine has been a wonderful project for me, I learned a lot during the development. If I could now decide other use for the time I spend on the project, I would not change it.