So you want to build an MMO(RPG)...

Disclaimer: this entire blog is written without any experience of ACTUALLY implementing a working game.

I consider myself a weekend-warrior game developer at best. It is a big interest of mine, I've spent a lot of time on reading/googling/YouTube, but I have yet to build an actual working game.

This blog post is based on my readings and current understandings, and experimentations so far, while working on development of a top-down, 2D game over years during my free time, between work and gaming.

Update: see links at the bottom, as I was reviewing I realized many of my notes are hijacked from various slides (e.g. Albion).

Question of Scale

While on my two-week stay-cation, I've come back to working on coding for an indie "MMO". I've tried to read every article I can find and watch every YouTube I can find on server architecture. There are many posts/questions from aspiring game developers on "how can I make my own MMO(RPG)", and many times are greeted with "Don't. It's the hardest thing to build".  Now and then you get the motivators saying the opposite, absolutely try.

I came across a great article a few months back that coined a new term where the context was something along the lines of "small-scale MMO architecture". An indie developer could make and target a 100-1,000 concurrent users MMO, and not necessarily the "millions of concurrent players within the same world" scale. This becomes more of a discussion about Medium Multiplayer Online Role-Playing Game, medium being subjective, but not massive.

Examples of indie games which I believe fit into this smaller scale are 8BitMMO and Project Gorgon. One-person or small team indie developers who have built an MMORPG, but perhaps not to the same scale as WoW, EQ, etc. And this can be successful.

Common Architecture Designs

I've come across some designs which appear to be somewhat common, at least at a high level. I often read/hear the same types of components referred to when researching MMORPG server architecture design, as listed below.

  • Proxy / Connection ¬†Server (all client connections go through these load-balanced servers)
  • Authentication / Login Server
  • Patch Server
  • Chat Server
  • Marketplace / Auction Server
  • World Server
  • Realm Server
  • Area Server
  • Game Server
  • Zone Server
  • Instance Server

Some of these are/can be synonyms of each other (e.g. World/Realm, or Area/Game/Zone).

Proxy / Connection Server

The role of this server is to handle all client connections, most likely running with multiple instances behind a load-balancer (TCP/UDP).

The idea here is that these servers would front all of the connection and message serialization/deserialization for all clients, and then proxy messages to other backend systems such as the world, game, or instance server(s).

Authentication / Login Server

When a client first tries to connect to your game, they probably need to authenticate/login to their account so that they can select a character to play. In the case where you are running multiple clusters/availability zones/realms, the authentication server would let the client know which game servers are available to connect to and coordinate the hand-off / connection to the selected server.

As an example, a user would launch your game and be presented with a login screen. After entering their username and password, this login request would be sent to your authentication/login server, which would probably talk to a single backend database containing all servers. Upon successful authentication, the server would return a list of all worlds/realms available for the player to connect. The user would then select a world/realm and then be presented with a list of their characters stored on that particular server (or create a new one).

In the background, the authentication server would probably generate a session token for the backend server to allow the client to connect the selected world server, such as a token that is sent on connection when establishing the new connection. Or in the case of having a proxy server design, the authentication server might let the proxy servers know about the new token before passing the client to them.

Patch Server

Many MMORPGs implement a game launcher, which is responsible for launching the actual game client. In addition to launching the game, the launcher is also typically responsible for connecting to a patch server to check for updates to the game client.

Having a patch server allows the user to stay up-to-date on bugs and content patches. A specific example might be if using GodotEngine and having the patch server send a new *.pak file with some updates.

Chat Server

I haven't dug into this one a ton yet, but it seems another common pattern is to separate chat from main game server, to avoid overloading your game server with chat operations.

A chat server (obviously) would be a dedicated system to handle messages between users, in channels, parties/groups, combat/system. This might mean that game servers connect to the chat server as a client themselves, perhaps sending messages in a separate thread asynchronously?

In my initial research, I was heading towards the direction of implementing a standard like Murmur-based (e.g. Mumble) or whatever open source solution exists for chat systems. I haven't come across much information on whether indie MMORPG developers end up implementing their own chat systems from the ground up.

Marketplace / Auction Server

Potentially this functionality could be part of your world server, but perhaps you abstract for performance/separation of concerns.

Assuming you implement a dedicated server, this is probably responsible for things like the auction house in WoW, or trading of currency, items, or crafting.

Your system design may implement a relational SQL server (PostgreSQL, MySQL, MS-SQL) as this data is very transactional and may require more strict ACID compliance.

The Marketplace Server may need to coordinate actions with the World Server for things like a player buying or selling an item. This server might send an event message to the world server with "Player Bought Used Sword", and when processing this message, the world server would add the item to the players inventory and decrement the users currency by the specified amount.

No idea if that is at all correct, and probably some race conditions to solve around this. I don't have much more content to write about here as I haven't spent any time on this aspect myself.

World / Realm Server

From what I have been able to infer from googling, "World Server" and "Realm Server" can sometimes mean the same thing.

In WoW parlance, an example of a realm would be joining Lightbringer, which is presented as a self-contained version/instance of the entire game world. Under the covers however, this is actually partitioned into multiple servers.

This seems to be the first layer where things can get really tricky and complex, depending on your game design. Do you want to present your players with a seamless world where there is never any "zoning" (loading a new area) like World of Warcraft (note: there is still zoning when crossing continents, etc), or are you okay with implementing zoning with loading screens when players cross certain boundaries?

For my use-case, I am totally okay with zoning and this seems to be the easiest to implement as you don't need to worry about potentially connecting to multiple area/game/zone servers if you happen to be in the middle of some boundaries.

Well back to what are world/realm servers responsible for. From what I can tell, there are a few common things.

  • Everything global
  • Guilds
  • Parties
  • Players, their positions, stats, inventory
  • Marketplace / economy (if you don't have them separated)
  • Rankings
  • Coordinating the creation of instances/dungeons
  • Coordinating the creation/zoning of area/game/zone servers (if they are dynamically created)

Area / Game / Zone Server

This seems to be the meat-and-potatoes of your system design. If the World/Realm server represents a copy of your entire game, the Area/Game/Zone server represents some chunk/distribution of the world for scaling purposes.

When your game first starts out, you might be able to serve the entire game on a single world server. Maybe you don't have many players or mobs/NPCs, and don't have a scaling problem to solve yet.

Fast forward a bit to now you have a bigger world with thousands of NPCs, and perhaps 500-1,000 concurrent users, all doing stuff within your game. At this point you probably need to start partitioning your game world into separate processes/servers.

In WoW parlance, you might have a Realm server (Lightbringer), that has one or more servers responsible for coordinating a continent, each which might have one or more server responsible for a zone (Elwynn Forest), and perhaps partitioned further by a sub-area within a zone (the lumber mill, bandit camps, various mines).

The game server is where things like updating NPCs/monsters, pathfinding, combat, and processing player actions are going to happen, possibly in coordination with one or more of the other servers (keep player position in sync with world server).

Instance Server

Depending on your game design, you might implement instances as a way to provide private copies of a part of your game world, e.g. a dungeon, player housing. Not all games implement this, which comes with pros and cons. A con could be fighting for a rare mob/item, which may reduce player satisfaction if they have to spent 10-15 minutes fighting other players for the chance to get credit for killing a named mob.

In addition to rare/named NPCs, there is also the matter of scalability. With instancing, you can run these smaller processes around multiple servers easier and scale up/down as needed.

The design and implementation of instancing is an interesting one, as some games combine this with other server roles (world, game) to provide scalability of general/shared areas via instances. Personally, I don't like this implementation, it just always feels less immersive to me.

An example of how this could work, again using WoW as an example, would be Stormwind. Instead of being on a realm (Lightbringer), you might join an instance of Stormwind, where you may or may not be with your friends. You might have the ability to switch to other instances (e.g. Stormwind #32). The pro of this solution is you might be able to handle scaling easier for common quest hub areas. In the case of WoW, they can somehow handle tons of players all standing in Stormwind, seemingly with little performance impacts.

Kubernetes-Based Design

Given a Kubernetes cluster and reading up on various designs/patterns, I've tried to diagram what a smaller scale MMO server design might look like. With this design, we would have an authentication server that sits outside of the cluster, because a Kubernetes cluster represents a World/Realm (maybe?). It could be possible to host multiple realms within the same datacenter/cluster, so it depends on multiple factors.

Physical Architecture
Logical Architecture

TODO

  • Find link to YouTube discussing WoW architecture, message passing (old school)
  • Try to find link to blob/article talking about small scale mmo design