|
|
On Reflection and Serialization
By Kyle Wilson
What I'd really like to see in C++ is a reflection mechanism which allows access to type information from template functions, so I could iterate through a class's member variables like template <class T> and have the iteration unroll at template instantiation time to a series of stream.Write calls to the appropriate functions based on the type of T's member variables. This is certainly information that's available to the compiler. It's just not exposed to programmers. Serialization Through Reflection What I'd like to do with this is automatically serialize objects. Serialization is the ability to write objects to and read objects from byte streams. In game terms, serialization encompasses export/import of static state, saving/loading of dynamic state, and transmitting/receiving of state information over a network. A reflection-based serialization scheme would allow me to write classes that look like class Example private: Where CoreSerial is simply defined as a templatized thin wrapper around any type template <typename T, int F> private: and kExport, kSave and kNetwork are defined as bitmasks. On iteration through the members of Example, I could process each member variable appropriately based on the flags associated with it. If an object was flagged for saving, I could save it. If not, I could ignore it. Unfortunately, I can't iterate through the member variables of a class. I can still make my CoreSerial template work, but it's going to be much more expensive. Self-Registering Serialized Variables I can make CoreSerial work by doing per object at runtime what really only ought to be done per class at compile time. I'd like to access at compile time the offsets for each member variable of class Example. Instead, my only option is to access those variables directly. I can create a singleton to manage all serialized state in the scene and have CoreSerial's constructor call CoreSerialManager::Instance()->Register(this, sizeof(T), F) and have CoreSerial's destructor call the equivalent Unregister function. (This approach owes a great deal to Adrian Stone, who first proposed something like it -- hsSaveable<int> -- when we were working at HeadSpin.) The cost of this is an extra function call for every serialized variable in your game, paid up front as objects are constructed, and again every time an object is deleted. CoreSerialManager will keep a vector of pointers and byte sizes that needs to be grown for each new serialized variable and, worse, searched and settled for each member of each deleted object. CoreSerialManager will need the space to store that vector, which holds pointer and size information to each variable. If most variables are 32-bit ints and floats, storage requirements for your game objects will triple. Storing pointers and sizes in a list instead of a vector might make object deletion faster, but at a cost of even more pointers for each link in the list, doubling storage requirements for serialized variables yet again (if you use std::list). Despite the space requirements and initialization overhead, this approach has a couple advantages that the other approaches I'm about to describe lack. First, CoreSerialManager can detect continuous blocks of serializable data and (on platforms where internal byte-order is the same as byte-order on disk) group them in single read/write calls for more efficient I/O. Two, all you need to do to serialize a variable is declare it properly. Everything else is done for you automatically. Explicit Serialization If we don't want to pay for all those list insertions and deletions at object construction and destruction time, then we're stuck writing an actual serialization function for every class. You should probably derive every serialized class from something like class CoreSerializable where CoreSerializer is defined as class CoreSerializer and serializers are derived from CoreSerializer to read from a file, write to a file, read from a network stream, write to a network stream and so forth. Separate Read and Write calls would let us be const correct, but double the amount of typing to be done as well as making it much more likely that Read and Write would get out of synch. I'm leaving out the flags fields for each variable, but it would be just as easy to flag variables for different types of serialization in this scheme as in those I discussed before -- probably even easier. The coding and maintenance cost of all this is that every serializable class now has a Serialize function that needs to be kept up to date as new variables are added to the class. The runtime cost is, at least, only paid at serialization time now. At that point, a virtual function call is made for each serialized variable. That's probably less costly than the fact that the serializer no longer has sufficient context to aggregate variables for bulk reading/writing. This is probably the serialization approach I'd use in most cases, but there's one more worthy of mention. Full Metaclass Implementation The heavyweight solution to serialization, of course, is to fully encapsulate each variable in a hold-anything structure and create "classes" that are really trees of value structs assembled at runtime. Reflection in this case reflects not C++ classes, but external "class" descriptions read from data files or script. Detlef Vollmann's article, "Metaclasses and Reflection in C++", describes one implementation of such a scheme.
I'm Kyle Wilson. I've worked in the game industry since I got out of grad school in 1997. Any opinions expressed herein are in no way representative of those of my employers. | |
|
|
||
