c# - Strategies for migrating serialized Json.NET document between versions/formats

Question

I'm using Json.Net to serialize some application data. Of course, the application specs have slightly changed and we need to refactor some of the business object data. What are some viable strategies to migrate previously serialized data to our new data format?

For example, say we have orignally had a business object like:

public class Owner
{
    public string Name {get;set;} 
}
public class LeaseInstrument
{
    public ObservableCollection<Owner> OriginalLessees {get;set;}
}

We serialize an instance of a LeaseInstrument to a file with Json.Net. Now, we change our business objects to look like:

public class Owner
{
   public string Name {get;set;}
}
public class LeaseOwner
{
  public Owner Owner { get;set;}
  public string DocumentName {get;set;}
}
public class LeaseInstrument
{
    public ObservableCollection<LeaseOwner> OriginalLessees {get;set;}
}

I have looked into writing a custom JsonConverter for LeaseInstrument, but the ReadJson method is not ever hit...instead an exception is thrown before the deserializer reaches that point:

Additional information: Type specified in JSON
'System.Collections.ObjectModel.ObservableCollection`1[[BreakoutLib.BO.Owner,
BreakoutLib, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]],
System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'
is not compatible with 'System.Collections.ObjectModel.ObservableCollection`1[[BreakoutLib.BO.LeaseOwner, BreakoutLib, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]], System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'. Path 'Is.$values[8].OriginalLessors.$type', line 3142, position 120.

I mean, no joke, Json.Net, that's why I'm trying to run a JsonConverter when deserializing these objects, so I can manually handle the fact that the serialized type doesn't match the compiled type!!

For what it's worth, here are the JsonSerializerSettings we are using:

var settings = new JsonSerializerSettings
    {
      PreserveReferencesHandling = PreserveReferencesHandling.Objects,
      ContractResolver = new WritablePropertiesOnlyResolver(),
      TypeNameHandling = TypeNameHandling.All,
      ObjectCreationHandling = ObjectCreationHandling.Reuse
    };

Answer 1

You have the following issues:

1. You serialized using TypeNameHandling.All. This setting serializes type information for collections as well as objects. I don't recommend doing this. Instead I suggest using TypeNameHandling.Objects and then letting the deserializing system choose the collection type.

   That being said, to deal with your existing JSON, you can adapt the IgnoreArrayTypeConverter from make Json.NET ignore $type if it's incompatible to use with a resizable collection:

   public class IgnoreCollectionTypeConverter : JsonConverter
   {
       public IgnoreCollectionTypeConverter() { }
   
       public IgnoreCollectionTypeConverter(Type ItemConverterType) 
       { 
           this.ItemConverterType = ItemConverterType; 
       }
   
       public Type ItemConverterType { get; set; }
   
       public override bool CanConvert(Type objectType)
       {
           // TODO: test with read-only collections.
           return objectType.GetCollectItemTypes().Count() == 1 && !objectType.IsDictionary() && !objectType.IsArray;
       }
   
       public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
       {
           if (!CanConvert(objectType))
               throw new JsonSerializationException(string.Format("Invalid type "{0}"", objectType));
           if (reader.TokenType == JsonToken.Null)
               return null;
           var token = JToken.Load(reader);
           var itemConverter = (ItemConverterType == null ? null : (JsonConverter)Activator.CreateInstance(ItemConverterType, true));
           if (itemConverter != null)
               serializer.Converters.Add(itemConverter);
   
           try
           {
               return ToCollection(token, objectType, existingValue, serializer);
           }
           finally
           {
               if (itemConverter != null)
                   serializer.Converters.RemoveLast(itemConverter);
           }
       }
   
       private static object ToCollection(JToken token, Type collectionType, object existingValue, JsonSerializer serializer)
       {
           if (token == null || token.Type == JTokenType.Null)
               return null;
           else if (token.Type == JTokenType.Array)
           {
               // Here we assume that existingValue already is of the correct type, if non-null.
               existingValue = serializer.DefaultCreate<object>(collectionType, existingValue);
               token.PopulateObject(existingValue, serializer);
               return existingValue;
           }
           else if (token.Type == JTokenType.Object)
           {
               var values = token["$values"];
               if (values == null)
                   return null;
               return ToCollection(values, collectionType, existingValue, serializer);
           }
           else
           {
               throw new JsonSerializationException("Unknown token type: " + token.ToString());
           }
       }
   
       public override bool CanWrite { get { return false; } }
   
       public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
       {
           throw new NotImplementedException();
       }
   }
   

2. You need to upgrade your Owner to a LeaseOwner.

   You can write a JsonConverter for this purpose that loads the relevant portion of JSON into a JObject, then checks to see whether the object looks like one from the old data model, or the new. If the JSON looks old, map fields as necessary using Linq to JSON. If the JSON object looks new, you can just populate your LeaseOwner with it.

   Since you are setting PreserveReferencesHandling = PreserveReferencesHandling.Objects the converter will need to handle the "$ref" properties manually:

   public class OwnerToLeaseOwnerConverter : JsonConverter
   {
       public override bool CanConvert(Type objectType)
       {
           return typeof(LeaseOwner).IsAssignableFrom(objectType);
       }
   
       public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
       {
           if (reader.TokenType == JsonToken.Null)
               return null;
           var item = JObject.Load(reader);
           if (item["$ref"] != null)
           {
               var previous = serializer.ReferenceResolver.ResolveReference(serializer, (string)item["$ref"]);
               if (previous is LeaseOwner)
                   return previous;
               else if (previous is Owner)
               {
                   var leaseOwner = serializer.DefaultCreate<LeaseOwner>(objectType, existingValue);
                   leaseOwner.Owner = (Owner)previous;
                   return leaseOwner;
               }
               else
               {
                   throw new JsonSerializationException("Invalid type of previous object: " + previous);
               }
           }
           else
           {
               var leaseOwner = serializer.DefaultCreate<LeaseOwner>(objectType, existingValue);
               if (item["Name"] != null)
               {
                   // Convert from Owner to LeaseOwner.  If $id is present, this stores the reference mapping in the reference table for us.
                   leaseOwner.Owner = item.ToObject<Owner>(serializer);
               }
               else
               {
                   // PopulateObject.  If $id is present, this stores the reference mapping in the reference table for us.
                   item.PopulateObject(leaseOwner, serializer);
               }
               return leaseOwner;
           }
       }
   
       public override bool CanWrite { get { return false; } }
   
       public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
       {
           throw new NotImplementedException();
       }
   }
   

These use the extensions:

public static class JsonExtensions
{
    public static T DefaultCreate<T>(this JsonSerializer serializer, Type objectType, object existingValue)
    {
        if (serializer == null)
            throw new ArgumentNullException();
        if (existingValue is T)
            return (T)existingValue;
        return (T)serializer.ContractResolver.ResolveContract(objectType).DefaultCreator();
    }

    public static void PopulateObject(this JToken obj, object target, JsonSerializer serializer)
    {
        if (target == null)
            throw new NullReferenceException();
        if (obj == null)
            return;
        using (var reader = obj.CreateReader())
            serializer.Populate(reader, target);
    }
}

public static class TypeExtensions
{
    /// <summary>
    /// Return all interfaces implemented by the incoming type as well as the type itself if it is an interface.
    /// </summary>
    /// <param name="type"></param>
    /// <returns></returns>
    public static IEnumerable<Type> GetInterfacesAndSelf(this Type type)
    {
        if (type == null)
            throw new ArgumentNullException();
        if (type.IsInterface)
            return new[] { type }.Concat(type.GetInterfaces());
        else
            return type.GetInterfaces();
    }

    public static IEnumerable<Type> GetCollectItemTypes(this Type type)
    {
        foreach (Type intType in type.GetInterfacesAndSelf())
        {
            if (intType.IsGenericType
                && intType.GetGenericTypeDefinition() == typeof(ICollection<>))
            {
                yield return intType.GetGenericArguments()[0];
            }
        }
    }

    public static bool IsDictionary(this Type type)
    {
        if (typeof(IDictionary).IsAssignableFrom(type))
            return true;

        foreach (Type intType in type.GetInterfacesAndSelf())
        {
            if (intType.IsGenericType
                && intType.GetGenericTypeDefinition() == typeof(IDictionary<,>))
            {
                return true;
            }
        }
        return false;
    }
}

public static class ListExtensions
{
    public static bool RemoveLast<T>(this IList<T> list, T item)
    {
        if (list == null)
            throw new ArgumentNullException();
        var comparer = EqualityComparer<T>.Default;
        for (int i = list.Count - 1; i >= 0; i--)
        {
            if (comparer.Equals(list[i], item))
            {
                list.RemoveAt(i);
                return true;
            }
        }
        return false;
    }
}

You can apply the converters directly to your data model using JsonConverterAttribute, like so:

public class LeaseInstrument
{
    [JsonConverter(typeof(IgnoreCollectionTypeConverter), typeof(OwnerToLeaseOwnerConverter))]
    public ObservableCollection<LeaseOwner> OriginalLessees { get; set; }
}

If you don't want to have a dependency on Json.NET in your data model, you can do this in your custom contract resolver:

public class WritablePropertiesOnlyResolver : DefaultContractResolver
{
    protected override JsonProperty CreateProperty(MemberInfo member, MemberSerialization memberSerialization)
    {
        var result = base.CreateProperty(member, memberSerialization);
        if (typeof(LeaseInstrument).IsAssignableFrom(result.DeclaringType) && typeof(ICollection<LeaseOwner>).IsAssignableFrom(result.PropertyType))
        {
            var converter = new IgnoreCollectionTypeConverter { ItemConverterType = typeof(OwnerToLeaseOwnerConverter) };
            result.Converter = result.Converter ?? converter;
            result.MemberConverter = result.MemberConverter ?? converter;
        }
        return result;
    }
}

Incidentally, you might want to cache your custom contract resolver for best performance.