Simple Serialization With Archivist
Posted by Patrick on Monday, January 24, 2011Mon, Jan 24, 2011

I'm thrilled to be a part of iDevBlogADay for which this is my first post. I hope I will be able to contribute something of worth to the community every Tuesday. I have several bits of code to release as I go along too, so hopefully you will find it helpful!

Yesterday, I outlined my motivations for a creating library called Archivist. You may want to check that out.

In short: Serialization in C++ can get ugly. Archivist attempts to solve this problem by being extremely simple and non-intrusive. It does not esteem optimization or efficiency over simplicity. I am, you see, a minimalist of sorts. That said, I think you will find it quite powerful, or at least interesting.

I think the best way to introduce Archivist is to show you.

Let us consider a game. In this game you have multiple players who battle it out in an arena against monsters using various weapons.

The code may look something like this:

class Point2
{
public:
    float x, y;

public:
    //...
};


namespace WeaponType
{
    enum Enum
    {
        WaterPistol,
        NerfGun,
        Uzi9mm,
        Bazooka
    };
};


class Player
{
private:
    string name;
    Point2 position;
    uint32_t health;
    uint32_t ammo;
    WeaponType::Enum weaponType;

public:
    //...
};


class Monster
{
private:
    Point2 position;
    uint32_t health;

public:
    //...
};


class Arena
{
private:
    map< string, Player > playerList;
    list< Monster > monsterList;

public:
    //...
};

We could argue about exact implementation I'm sure, but let's keep it like this for the moment. There is already a bit of complexity here: you've got a few containers, an enum and a few non-primitive types.

Okay, so let's say you're going to implement your save system, how would you serialize this to disk?

Well, chances are you'd break out the old fread() and fwrite() calls. Probably you'd start adding Save() and Load() methods everywhere and call them as needed. It's not that hard, but it does create a lot of extra code and introduces a lot of potential for bugs.

Let's start with the Point2 class since it is the simplest. Archivist lets you do this:

#include "Archivist"
using namespace Archivist;

class Point2
{
public:
    float x, y;

public:
    ArchiveAttributes( x, y );

    //...
};

And then you can do this:

Point2 point( 4, 5 );
Archive::Save( "point.plist", point.Encode() );

Which lets you open it in Property List Editor:

Open point.plist in Property List Editor

Does this get your attention? No?

Okay, let's fill out the rest of the code:

class Player
{
private:
    //...

    Enum< WeaponType::Enum > weaponType;

public:
    ArchiveAttributes(
        name,
        position,
        health,
        ammo,
        weaponType
    );

    //...
};

class Monster
{
private:
    //...

public:
    ArchiveAttributes(
        position,
        health
    );

    //...
}

class Arena
{
private:
    //...

public:
    ArchiveAttributes(
        playerList,
        monsterList
    );

    //...
}

The only tricky bit here is needing to wrap weaponType in Archivist::Enum because enums are somewhat retarded and get treated like second-class types. Don't worry, the wrapped enum will still behave correctly. The rest is cake. And it lets you do this:

Arena arena;

arena.AddPlayer( Player( "Mario", 10, 0 ) );
arena.AddPlayer( Player( "Luigi", 40, 0 ) );

for (int i = 0; i < 5; i++)
{
    arena.SpawnMonster();
}

Archive::Save( "arena.plist", arena.Encode() );

Which, in turn, lets you do this:

Open arena.plist in Property List Editor

Do I have your attention now? Wow… tough crowd.

Even the STL container members are correctly encoded. Nifty. Okay, let me show you how to decode things before get too far along:

Arena arena2;
arena2.Decode( Archive::Load( "arena.plist" ) );

And that's pretty much the basic use of Archivist. The rabbit hole goes a whole lot deeper of course and we'll get into the more complex scenarios like inheritance, polymorphism and wrapping structs and classes you don't have code for in future posts.

But before I sign off, let me add a few more points about how Archivist handles things internally.

Everything in Archivist encodes into a variant type called Unknown. Unknown can currently be one of the following types under the hood: Object, Array, Signed, Unsigned, Float, Boolean, String and Null. Unknown essentially wraps these. These types are enough to handle almost anything, including most of the STL containers.

The variants all know how to properly get encoded into and decoded out of any std::stream, which means along with saving to files, you can throw in things like:

cout << monster.Encode();

That can be quite useful for debugging. Here's another example:

float volume = 85.5f;
Unknown u = Encode( volume );
assert( u.Type() == Type::Float );
cout << u;

Which would output:

<real>85.5</real>

Or, to go back to our game example:

Arena arena;
Unknown u = Encode( arena ); // which just calls arena.Encode();
assert( u.Type() == Type::Object );
cout << u;

Which would output a whole lot more, so I won't post it in here, but you get the idea.

Object is a std::map-like container that stores Unknowns mapped to a std::string.

Array behaves pretty much like std::vector (std::deque actually), again storing Unknowns.

Signed and Unsigned wrap 64-bit signed and unsigned integers respectively. Similarly, Float stores a long double. String wraps a std::string internally. Boolean wraps a bool and Null is not really used to store any meaningful value and represents the default value of an Unknown.

I had considered making internal types for every possible primitive type, but that would result in 10 integer types and 3 floating point types currently. That kind of goes against my simplicity over optimization principle, but I may change my mind about that. :)

As it is, any possible value you can hold in a primitive type can be stored by Archivist.

The beauty of the Unknown variant type is that encoded data can be decoded before you know what it is. Then you can check its type and act appropriately. Most often Archivist avoids requiring you get your hands dirty that way, but the flexibility is there when you need it — and in the next installment we may just.

Well, that's it for now! There is more you need to know to use Archivist properly, but you can get the code from my GitHub account and play with it. Everything is in the Archivist namespace so you shouldn't have any conflicts with your own code.

There is more to show-and-tell, and a few features I want to add to the library. I'll get to those in upcoming iDevBlogADay posts. I also have some exciting ideas to experiment with.

You can get Archivist here: https://github.com/pbhogan/Archivist

I would love to hear your thoughts on what you've seen so far!

This post is part of iDevBlogADay, a group of blogs by indie iPhone developers featuring two posts per day. You can subscribe to iDevBlogADay through RSS or follow the #iDevBlogADay hash tag or @idevblogaday on Twitter.