Software

Server Software

Minecraft addons

MinecraftOnline is using:

• Sponge
• A custom multiplexer designed by  Flippeh
• Custom commands provided by a variety of plugins, some developed in-house
• Custom server scripts for ease of administration
• LionsShop for player shops that use existing player materials and chests
• Craftbook Extra for feature enhancements that are compatible with the survival feel of the server
• LWC for chest locking

Monitoring

For server monitoring MRTG is used to graph performance data, with data display primarily through Cacti.

Backups

Disk snapshots are taken which cause a small inescapable freeze, in the order of 5 seconds in the game world.

Every storage component in the server itself is mirrored for 100% redundancy.

Offsite backups are kept remote from the data centre, which is also on multiple-redundant storage hardware.

What if you were to go bust?

Concern has previously been expressed regarding contingencies in the case of server failure, and what would happen to Freedonia in such an event.

"[talking about the backup mechanism] Even a comet wouldn't stop us.. only running out of money would, and if that happened, we'd just rapidshare the last snapshot anyway ;)" - SlowRiot April 2011

To guard against any data loss contingency, all server hardware is mirrored in duplicate. The server uses ECC RAM, and two drive arrays, each in mirrored RAID-1. Any drive failure would not result in data loss, or even server downtime. Moreover, the minecraft server and world data is backed up across drives within the machine twice daily, so even one entire array failure would be survivable; up to three drives could fail simultaneously without resulting in more than twelve hours' data loss.

Further to that, off-site backups are currently maintained in at least three locations - Data centres in Scotland, Germany, France, as well as two private fileserver locations not connected to central infrastructure. It's no exaggeration to say Freedonia would survive even a nuclear war.

Up until recently, server backup snapshots have always been made publically available. The collaborative assistance from providing this service has been described as significant by SlowRiot. At this point, these have been removed to prevent archiving and unethical use by crawlers which disregard robots.txt. However, two weeks of backup snapshots can be made available for download upon request.

The server crashed and I lost hours of work...

The apparent duration of rollback isn't due to the frequency of disk snapshots but due to minecraft losing data in presently loaded chunks when it crashes. It saves chunk data as soon as all players exit them - for instance if 5 hours work was lost by a user, the chances are you or a group of people stayed within view of the same chunk for 5 hours straight. Increasing the frequency of disk snapshots doesn't solve this issue.

Each snapshot is roughly (as of April 2011) 2GB, compressed.

The current main concern is that it's possible for corrupt chunks to sneak into backups for several days before being detected, as recently happened with a chunk issue that caused crashes in early April 2011 when players moved within sight of one spurious bad chunk. For this reason, increasing the frequency of backups is not considered an optimal path to take.

Admins are currently focusing on eliminating the causes of server crashes, instead. The majority of these are bugs present in the vanilla server and caused by  notch, which only become obvious on high-availability servers such as this. Many have already been pinned down and obviated, such as the limited file handles java crash. Others have active mitigation efforts in place, such as the mislocated chunk issue.

At the time of writing, no known crash bugs remain that have not been dealt with. In future versions of  notch's Minecraft server, however, new bugs may manifest.

Bug feedback

SlowRiot works fairly closely with the Canary developers, and worked with developers hey0 and meaglin before them. This server has also been responsible for multiple bug reports and consequent fixes to CraftBook Extra, the Overviewer suite, CommandOn plugin, and various other public Minecraft components.

Several plugins and components to the server have been developed in-house by some of the admin team, most notably  Flippeh and  bastetFurry. Therefore priority maintenance, and efficient bug reporting and fixing is possible.

The core Minecraft developers do not have interaction with public server administrators as a general rule. Bug reports do not receive replies or acknowledgements and attempts to get into closer dialogue have not been fruitful. Therefore, this sever chooses not to rely on the core team for bug fixes, instead taking a direct approach of addressing and fixing bugs as possible with custom modifications.

Website

The website and donation system is developed entirely from scratch conforming to XHTML, using a CGI backend composed of Bash, Perl and Python scripts.

The map uses the Leaflet API for rendering.

The tiles on the map are generated using the Minecraft Overviewer, with a few tweaks and functionality additions by  SlowRiot.

This wiki is powered by MediaWiki with some minor functionality modifications.