Hackero Next - Technical Decisions

Hackero Next Aug 1, 2024

In this blog post, I’d like to explain some of the decisions that led to the development of Hackero Next. If you’re not a techie, feel free to skip this.

When I started Hackero in 2019, I based it on an existing backend framework called Feathers.js. Feathers.js is a Node.js-based framework that simplifies communication between the frontend and backend, making it great for kickstarting development. It’s originally designed as a thin layer between distributed backend systems and the frontend to simplify communication. It follows a service-based architecture where every service has methods for adding (POST), listing (GET), reading (GET), updating (PATCH/UPDATE), and deleting (DELETE) entities, following a REST schema.

These were the first two mistakes. This framework is not designed for the complexity encountered in game development, and I totally underestimated this. This led to numerous performance issues. Given Hackero’s highly stateful nature, scaling became impossible.

To solve this I decided to reimplement the interfaces of the Node.js framework in Golang, which offers much better performance. Indeed, it did, reducing the most problematic actions, such as writing to the filesystem, from 500ms to around 50ms per request, drastically improving the experience. The story could have ended here, with everyone happy, but more problems arose.

As the game’s code grew, Feathers.js kept all the logic in hooks and services, which became increasingly messy. Since all the game logic was implemented in these services and the mechanics of Hackero are tightly coupled, this led to hard-to-read spaghetti code that was difficult to maintain. For example, sending an in-game mail would call a service that would also call filesystem services, and both services contained all behavior for when being called from the frontend and the backend.

This led to another crucial mistake: I put on more ducktape and did not consider the requirements for a project like Hackero. Instead, I tried to keep the frontend interfaces the same while improving the backend, without addressing where the problems arose. With the performance issues solved, the challenge of building around the service based architecture of feathers.js framework grew, resulting in increasingly difficult-to-maintain software with every update.

For example, changing something in the filesystem caused sending mails to fail, but only when they had an attachment. When the frontend sent the mail, everything was fine, but when the backend did, things broke. This led to missions not working anymore. Fixing this resulted in incorrect assumptions that broke sending emails from the UI, but only with specific attachments.

In the following section, I have another design example of how the filesystems were working:

Service filesystem:

POST,GET,PATCH,UPDATE and DELETE for a filesystem

Service filesystem-methods:

DISABLED: POST,GET,UPDATE and DELETE
PATCH offering the following actions:
- $add Add a file/folder
- $rm Remove a file/foler
- $mv/$cp Move or copy a file
- $log Write to a log file

This was really messy since logic is distributed.

(I will explain what this looks like in Hackero Next further down.)

This was somewhat manageable but became unworkable when I wanted to implement additional accounts other than root on a system. This task required touching multiple services and ensuring backend calls wouldn’t break, as many game mechanics depended on these services (for logging or configuration files). At the same time, I had to ensure the frontend logic remained intact. All in all, it was a total nightmare.

I began reimplementing and abstracting the logic but soon realized that part of the problem was the non-extendable REST-based API, which was far too limiting.

This is where hackero next comes into play

Hackero next follows a event based domain driven approach on a 3 layer architecture:

API Layer

The API layer connects the frontend to the backend. However, it now follows a custom, strategy-based architecture that allows structuring the frontend-related logic clearly. Strategies can be implemented for every feature of the game, allowing for separation and better organization.

So, what does this change mean for the service architecture mentioned above? Previously, all the filesystem logic was bundled into two services. Now, there are strategies for them, and the frontend can access the following:

filesystems:get - returns a filesystem
filesystems:set - creates a directory or creates or writes a file
filesystems:remove - removes a file or directory
filesystems:copy - copies or moves a file or directory
... (And 4 more I will talk about later ;))

As you can see this gets much cleaner and all the permissions checks for frontend calls live in there and the game logic is not affected by it.

Domain Layer

The Domain Layer contains all game-related logic in a decoupled, event-driven architecture. All business logic is enforced here, making the code more maintainable.

Using the filesystem example, strategy calls are translated to mutations applied in this layer. Here, all checks related to the filesystem, such as disk size limitations, are performed. The Domain Layer is event-based, and each domain is responsible for its own logic, reacting to each other based on triggered events.

In the old architecture, creating a new shop account depended on the filesystem and mail services for sending out the welcome mail with an attached pass-safe file. In Hackero Next, the account domain is standalone. Whenever an account is created, an event is sent, and the mail domain creates the corresponding email, reducing dependencies between them. If the newly created account belongs to a banking system, the same event that triggers the welcome mail also creates the bank account in the banking domain.

This approach ensures that each domain handles its responsibilities independently while communicating effectively through events, leading to a more modular and maintainable system.

Persistence Layer

The Persistence Layer is responsible for handling all updates made by the Domain Layer. It consists of the database and the mechanisms for storing information. This layer also includes optimizations for parts of the game that are frequently modified and accessed.

A good example of this is the handling of servers and filesystems. Servers are often used for various checks, such as hardware specifications or user permissions. Filesystems are accessed nearly all the time, making it crucial for them to be as fast as possible.

However, entities like credentials for web accounts, which are not accessed as frequently, can be retrieved from the database whenever needed.

To optimize performance, servers and filesystems are stored and modified in memory, making them faster to access. This reduces the load on the database, especially when multiple players interact with the same filesystem simultaneously.

By storing frequently accessed data in memory, we ensure a smoother and more responsive gaming experience.

Advantages of this approach

I've already explained a few advantages of this approach, but I'd like to highlight what makes this design better than the old one:

Clear Separation of Frontend and Backend Logic

In the old logic, the call logic for frontend and backend was mixed in the hooks of a service. In the new domain-driven, three-layer approach, each layer has a clearly defined responsibility.

Clearer Domain Borders and Separation of Concerns

In the old approach, breaking one service had vast effects on others, disrupting unrelated game logic. In the new approach, domains are only coupled by events, making the code much more resilient.

Testing is easier and more meaningful

Writing tests for code with many dependencies is never easy. In the tightly coupled approach of the old services, this was essentially an unsolvable task. By separating the code clearly and using dependency injection, writing tests for a part of the code is now much simpler and helps keep that part of the code stable.

Additionally, the information gained from a failing test increases significantly. In the service-based architecture, a failing test provided little insight into whether it was actually a problem. In the new approach, the clearer separation makes it easier to package expected behavior in a test.

Extensible API to the frontend

Without the limitation on the number of methods the backend can offer, the strategy approach scales much better compared to the old one. As shown in the filesystem example above, extending the logic and adding new strategies do not interfere with the existing logic as it did in the old approach.

I hope this provides a better understanding of the architecture behind the game and why it makes development easier. If you have any questions, feel free to reach out on Discord, and I can look into any aspect of this in more detail later on.