multithreading - Performance in multithreaded Json.NET serialization

Question

We're deserializing large JSON payloads from an external API, each response being roughly 30MB.

On my developer machine it takes 0.3s to call JsonConvert.DeserializObject(jsonString) with those 30MB in a single thread. But if I deserialize in multiple threads the time spent per deserialization drastically increases as the number of threads increases, up to 5-10s sometimes. My test code looks like this:

var json = System.IO.File.ReadAllText("c:/largejson.json");

var count = 100;
var ops = 0;
var results = new ConcurrentBag<TimeSpan>();

while (ops < count)
{
    var tasks = new List<Task>();
    for (var i = 0; i < Environment.ProcessorCount; i++)
    {
        tasks.Add(Task.Run(() =>
            {
                var stopwatch = new Stopwatch();
                stopwatch.Start();

                JsonConvert.DeserializeObject(json);
                ops++;

                stopwatch.Stop();
                results.Add(stopwatch.Elapsed);
            }
        ));
    }

    await Task.WhenAll(tasks);
}

var totalSeconds = results.Sum(r => r.TotalSeconds);
var secondsPerOp = totalSeconds / ops;

And here's the file I'm using: https://github.com/andersekdahl/files/blob/master/largejson.json

I've tested both with and without passing in a new JsonSerializerSettings() as a second argument but it doesn't make any difference.

My first instinct was that there was some locking inside Json.NET but when I profiled it using dotTrace nothing really sticks out. A lot of time is spent in JTokenWriter.WritePropertyName but that doesn't look like it's shared between threads/deserializations.

Anyone has any clues what's causing the slowdown?

question from:https://stackoverflow.com/questions/65917658/performance-in-multithreaded-json-net-serialization

Answer 1

Waitting for answers