c# - Await and SynchronizationContext in a managed component hosted by an unmanaged app

Question

[EDITED] This appears to be a bug in the Framework's implementation of Application.DoEvents, which I've reported here. Restoring a wrong synchronization context on a UI thread may seriously affect component developers like me. The goal of the bounty is to draw more attention to this problem and to reward @MattSmith whose answer helped tracking it down.

I'm responsible for a .NET WinForms UserControl-based component exposed as ActiveX to a legacy unmanaged app, via COM interop. The runtime requirement is .NET 4.0 + Microsoft.Bcl.Async.

The component gets instantiated and used on the app's main STA UI thread. Its implementation utilizes async/await, so it expects that an instance of a serializing synchronization context has been installed on the current thread (i. e.,WindowsFormsSynchronizationContext).

Usually, WindowsFormsSynchronizationContext gets set up by Application.Run, which is where the message loop of a managed app runs. Naturally, this is not the case for the unmanaged host app, and I have no control over this. Of course, the host app still has its own classic Windows message loop, so it should not be a problem to serialize await continuation callbacks.

However, none of the solutions I've come up with so far is perfect, or even works properly. Here's an artificial example, where Test method is invoked by the host app:

Task testTask;

public void Test()
{
    this.testTask = TestAsync();
}

async Task TestAsync()
{
    Debug.Print("thread before await: {0}", Thread.CurrentThread.ManagedThreadId);

    var ctx1 = SynchronizationContext.Current;
    Debug.Print("ctx1: {0}", ctx1 != null? ctx1.GetType().Name: null);

    if (!(ctx1 is WindowsFormsSynchronizationContext))
        SynchronizationContext.SetSynchronizationContext(new WindowsFormsSynchronizationContext());

    var ctx2 = SynchronizationContext.Current;
    Debug.Print("ctx2: {0}", ctx2.GetType().Name);

    await TaskEx.Delay(1000);

    Debug.WriteLine("thread after await: {0}", Thread.CurrentThread.ManagedThreadId);

    var ctx3 = SynchronizationContext.Current;
    Debug.Print("ctx3: {0}", ctx3 != null? ctx3.GetType().Name: null);

    Debug.Print("ctx3 == ctx1: {0}, ctx3 == ctx2: {1}", ctx3 == ctx1, ctx3 == ctx2);
}

Debug output:

thread before await: 1
ctx1: SynchronizationContext
ctx2: WindowsFormsSynchronizationContext
thread after await: 1
ctx3: SynchronizationContext
ctx3 == ctx1: True, ctx3 == ctx2: False

Although it continues on the same thread, the WindowsFormsSynchronizationContext context I'm installing on the current thread before await gets reset to the default SynchronizationContext after it, for some reason.

Why does it get reset? I've verified my component is the only .NET component being used by that app. The app itself does call CoInitialize/OleInitialize properly.

I've also tried setting up WindowsFormsSynchronizationContext in the constructor of a static singleton object, so it gets installed on the thread when my managed assembly gets loaded. That didn't help: when Test is later invoked on the same thread, the context has been already reset to the default one.

I'm considering using a custom awaiter to schedule await callbacks via control.BeginInvoke of my control, so the above would look like await TaskEx.Delay().WithContext(control). That should work for my own awaits, as long as the host app keeps pumping messages, but not for awaits inside any of the 3rd party assemblies my assembly may be referencing.

I'm still researching this. Any ideas on how to keep the correct thread affinity for await in this scenario would be appreciated.

Answer 1

This is going to be a bit long. First of all, thanks Matt Smith and Hans Passant for your ideas, they have been very helpful.

The problem was caused by a good old friend, Application.DoEvents, although in a novelty way. Hans has an excellent post about why DoEvents is an evil. Unfortunately, I'm unable to avoid using DoEvents in this control, because of the synchronous API restrictions posed by the legacy unmanaged host app (more about it at the end). I'm well aware of the existing implications of DoEvents, but here I believe we have a new one:

On a thread without explicit WinForms message loop (i.e., any thread which hasn't entered Application.Run or Form.ShowDialog), calling Application.DoEvents will replace the current synchronization context with the default SynchronizationContext, provided WindowsFormsSynchronizationContext.AutoInstall is true (which is so by default).

If it is not a bug, then it's a very unpleasant undocumented behavior which may seriously affect some component developers.

Here is a simple console STA app reproducing the problem. Note how WindowsFormsSynchronizationContext gets (incorrectly) replaced with SynchronizationContext in the first pass of Test and does not in the second pass.

using System;
using System.Diagnostics;
using System.Threading;
using System.Threading.Tasks;
using System.Windows.Forms;

namespace ConsoleApplication
{
    class Program
    {
        [STAThreadAttribute]
        static void Main(string[] args)
        {
            Debug.Print("ApartmentState: {0}", Thread.CurrentThread.ApartmentState.ToString());
            Debug.Print("*** Test 1 ***");
            Test();
            SynchronizationContext.SetSynchronizationContext(null);
            WindowsFormsSynchronizationContext.AutoInstall = false;
            Debug.Print("*** Test 2 ***");
            Test();
        }

        static void DumpSyncContext(string id, string message, object ctx)
        {
            Debug.Print("{0}: {1} ({2})", id, ctx != null ? ctx.GetType().Name : "null", message);
        }

        static void Test()
        {
            Debug.Print("WindowsFormsSynchronizationContext.AutoInstall: {0}", WindowsFormsSynchronizationContext.AutoInstall);
            var ctx1 = SynchronizationContext.Current;
            DumpSyncContext("ctx1", "before setting up the context", ctx1);

            if (!(ctx1 is WindowsFormsSynchronizationContext))
                SynchronizationContext.SetSynchronizationContext(new WindowsFormsSynchronizationContext());

            var ctx2 = SynchronizationContext.Current;
            DumpSyncContext("ctx2", "before Application.DoEvents", ctx2);

            Application.DoEvents();

            var ctx3 = SynchronizationContext.Current;
            DumpSyncContext("ctx3", "after Application.DoEvents", ctx3);

            Debug.Print("ctx3 == ctx1: {0}, ctx3 == ctx2: {1}", ctx3 == ctx1, ctx3 == ctx2);
        }
    }
}

Debug output:

ApartmentState: STA
*** Test 1 ***
WindowsFormsSynchronizationContext.AutoInstall: True
ctx1: null (before setting up the context)
ctx2: WindowsFormsSynchronizationContext (before Application.DoEvents)
ctx3: SynchronizationContext (after Application.DoEvents)
ctx3 == ctx1: False, ctx3 == ctx2: False
*** Test 2 ***
WindowsFormsSynchronizationContext.AutoInstall: False
ctx1: null (before setting up the context)
ctx2: WindowsFormsSynchronizationContext (before Application.DoEvents)
ctx3: WindowsFormsSynchronizationContext (after Application.DoEvents)
ctx3 == ctx1: False, ctx3 == ctx2: True

It took some investigation of the Framework's implementation of Application.ThreadContext.RunMessageLoopInner and WindowsFormsSynchronizationContext.InstalIifNeeded/Uninstall to understand why exactly it happens. The condition is that the thread doesn't currently execute an Application message loop, as mentioned above. The relevant piece from RunMessageLoopInner:

if (this.messageLoopCount == 1)
{
    WindowsFormsSynchronizationContext.InstallIfNeeded();
}

Then the code inside WindowsFormsSynchronizationContext.InstallIfNeeded/Uninstall pair of methods doesn't save/restore the thread's existing synchronization context correctly. At this point, I'm not sure if it's a bug or a design feature.

The solution is to disable WindowsFormsSynchronizationContext.AutoInstall, as simple as this:

struct SyncContextSetup
{
    public SyncContextSetup(bool autoInstall)
    {
        WindowsFormsSynchronizationContext.AutoInstall = autoInstall;
        SynchronizationContext.SetSynchronizationContext(new WindowsFormsSynchronizationContext());
    }
}

static readonly SyncContextSetup _syncContextSetup =
    new SyncContextSetup(autoInstall: false);

A few words about why I use Application.DoEvents in the first place here. It's a typical asynchronous-to-synchronous bridge code running on the UI thread, using a nested message loop. This is a bad practice, but the legacy host app expects all APIs to complete synchronously. The original problem is described here. At some later point, I replaced CoWaitForMultipleHandles with a combination of Application.DoEvents/MsgWaitForMultipleObjects, which now looks like this:

[EDITED] The most recent version of WaitWithDoEvents is here. [/EDITED]

The idea was to dispatch messages using .NET standard mechanism, rather than relying upon CoWaitForMultipleHandles to do so. That's when I implicitly introduced the problem with the synchronization context, due to the described behavior of DoEvents.

The legacy app is currently being rewritten using modern technologies, and so is the control. The current implementation is aimed for existing customers with Windows XP who cannot upgrade for reasons beyond our control.

Finally, here's the implementation of the custom awaiter which I mentioned in my question as an option to mitigate the problem. It was an interesting experience and it works, but it cannot be considered a proper solution.

/// <summary>
/// AwaitHelpers - custom awaiters
/// WithContext continues on the control's thread after await
/// E.g.: await TaskEx.Delay(1000).WithContext(this)
/// </summary>
public static class AwaitHelpers
{
    public static ContextAwaiter<T> WithContext<T>(this Task<T> task, Control control, bool alwaysAsync = false)
    {
        return new ContextAwaiter<T>(task, control, alwaysAsync);
    }

    // ContextAwaiter<T>
    public class ContextAwaiter<T> : INotifyCompletion
    {
        readonly Control _control;
        readonly TaskAwaiter<T> _awaiter;
        readonly bool _alwaysAsync;

        public ContextAwaiter(Task<T> task, Control control, bool alwaysAsync)
        {
            _awaiter = task.GetAwaiter();
            _control = control;
            _alwaysAsync = alwaysAsync;
        }

        public ContextAwaiter<T> GetAwaiter() { return this; }

        public bool IsCompleted { get { return !_alwaysAsync && _awaiter.IsCompleted; } }

        public void OnCompleted(Action continuation)
        {
            if (_alwaysAsync || _control.InvokeRequired)
            {
                Action<Action> callback = (c) => _awaiter.OnCompleted(c);
                _control.BeginInvoke(callback, continuation);
            }
            else
                _awaiter.OnCompleted(continuation);
        }

        public T GetResult()
        {
            return _awaiter.GetResult();
        }
    }
}