JSON Conversion

Since Conquest! was converted to the C language circa 1998, it has used an XML format to exchange information. This worked fine for years but I recently became concerned with the payload sizes, as the amount of data the server needed to send the client has increased. Additionally, I was not using any advanced XML capabilities (e.g., meta data support) that would warrant its continued use. I started looking at JSON and became convinced after seeing payload sizes reduced by 10-30%. For example, here are examples of common commands and sizes (in bytes).

Command	XML	JSON	Difference
Person	1043	916	-13%
Market	1471	1134	-23%
Map	562	421	-25%
Weather	1352	959	-30%
Review	4491	2988	-33%

None of the player or world data is stored client-side, so each action a player takes requires updates from the server. Over time, these will add up to huge savings and should improve the end user experience. Additionally, I wanted to ensure backwards compatibility with existing XML based clients, so the server had to support both formats until I force a client update. I'm using the client version number to determine which to try first.

Part of the challenge was finding existing libraries for both C (server) and Unity3D (client) to handle JSON parsing.
For the server, I decided on json-c. The library itself is great but I found the documentation on the GitHub repository lacking. It wasn't until I found this tutorial that it all finally clicked. This tutorial covers everything from installation to memory management. Another good reference was the header file documentation.

Now that the server was covered, I turned to the client. I installed the Newtonsoft Json package (available for free) and followed the guide here. Serializing and Deserializing the response back from the server took some time to get working. I ended up having to break down the JSON object by object and add them back in one at a time to locate my errors. I found this site very helpful when creating my initial classes.

As part of this conversion, I also moved from using Unity's PlayerPrefs to a JSON configuration file. Over time, the amount of data I was storing in PlayerPrefs had gotten large and I needed to represent more complex structures. I was able to use Newtonsoft for both the responses from the server and management of the configuration file. I even wrote a conversion function to silently upgrade client from PlayerPrefs to the new settings file.

It was a lot of work over several days, but I'm pleased with the results.

UPDATE: After some consideration, I ended up witching from Newtonsoft to the standard JSON utility included in Unity. I didn't need the advanced capabilities of Newtonsoft and the default utility works fine. One item which did cause me problems is that all classes and subclasses need to be identified with [System.Serializable] for the default utility to work properly.