Mainly, it has to be this complicated today because it was this complicated yesterday and no one has come up with anything simpler.
I can't come up with a linear narrative here, so please endure the weaving back and forth that's required.
What is a PFX/PKCS#12 file?
While I can't fully say what the origins of the PFX are, there's a clue in the names of the Windows functions to read and write them: PFXImportCertStore and PFXExportCertStore. They contain a lot of separate entities (certs, private keys, and other stuff) that can use property identifiers to interrelate. They're seemingly designed to be an export/import mechanism for an entire cert store, like all of CurrentUserMy. But since one kind of store is the "memory store" (an arbitrary collection), .NET import/export make sense, but some complication comes in (from before .NET).
Windows Private Keys
Windows supports lots of different places for private keys, but for the legacy crypto API they come down to a 4-part addressing scheme:
- The name of the cryptographic provider
- The name of the key container
- An identifier of if this is a machine-relative key or a user-relative key
- An identifier of if this is a "signature" key or an "exchange" key.
This got simplified to a 3-part scheme for CNG:
- The name of the storage engine
- The name of the key
- An identifier of if this is a machine-relative key or a user-relative key.
Why is the machine-or-not identifier needed?
CAPI and CNG both support directly interacting with named keys. So you create a key named "EmailDecryption". Another user on the system creates a key of the same name. Should that work? Well, probably. So, huzzah, it does! Separate keys, because they're held under contexts tied to the user who made them.
But now you want a key that can be used by multiple users. It's not the thing you normally want, so it's not the default. It's an option. The CRYPT_MACHINE_KEYSET flag is born.
I'll go ahead and say here that I've heard that the direct usage of named keys is now discouraged; the CAPI/CNG team much prefers GUID-named keys and that you interact with them via the cert stores. But it's part of the evolution.
What does importing a PFX do?
PFXImportCertStore copies all of the certificates from the PFX into the provided store. It also imports (CryptImportKey or BCryptImportKey, depending on what it thinks it needs). Then, for each of the keys that it imported it finds (via property values in the PFX) the matching certificate, and sets a property on the cert store representation for "this is my 4-part identifier" (CNG keys just set the 4th part to 0); which is really all that the cert knows about its private key.
(PFX is a very complicated file format, this description is true provided none of the "weird parts" get utilized)
Key Lifetimes
Windows Private Keys live forever, or until someone deletes them.
So when the PFX imports them, they live forever. This makes sense if you were importing to CurrentUserMy. It makes less sense if you were doing something transitory.
.NET Inverts the Relationship / Makes it "Too Easy"
The Windows design is (mostly) that you interact with cert stores, and from cert stores you get certificates. .NET came later, and (one presumes, based on seeing what applications really were doing) made certificates the top-level object, and stores sort of a secondary thing.
Because Windows certificates (which are really "store certificate elements") "know" what their private key is, .NET certificates "know" what their private key is.
Oh, but the MMC Certificate Manager says it can export a certificate with its private key (into a PFX), why can't the cert constructor accept those bytes in addition to the "just a cert" format? Okay, so now it can.
Reconciling Lifetime
You open some bytes as an X509Certificate/X509Certificate2. It's a PFX with "no password" (via any of the various ways that can be true). You see it's the wrong one, and you let the cert go off to the garbage collector. That private key lives forever, so your hard drive slowly fills up, and key storage access gets slower and slower. Then you get angry, and reformat your computer.
That seems bad, so what .NET does is when (a field of) a cert is getting garbage collected (actually, finalized) it tells CAPI (or CNG) to delete the key. Now things work as expected, right? Well, so long as the program doesn't abnormally terminate.
Oh, you added it to a persisted store? But I'm going to delete the private key after the new certificate store entity "knows" how to find the private key. That seems bad.
Enter X509KeyStorageFlags.PersistKeySet
PersistKeySet says "don't do that deleting thing". It's for when you intend to add the cert to an X509Store.
If you want the same behavior without specifying the flag, call Environment.FailFast, or unplug the computer, after doing the import.
About that machine-or-user bit
In .NET you can easily have a grab bag of certs in a collection and call Export on it. What if some have machine keys, and others have user keys? PFXExportCertStore to the rescue. When a machine key is exported it writes down an identifier that says it was a machine key, so import puts it back to the same place.
Well, usually. Maybe you exported a machine key off of one machine, and you want to just inspect it as a non-admin on another machine. Okay, you can specify CRYPT_USER_KEYSET aka X509KeyStorageFlags.UserKeySet.
Oh, you created this as a user on one machine, but want it as a machine key on another? Fine. CRYPT_MACHINE_KEYSET / X509KeyStorageFlags.MachineKeySet.
Do I Really Need "Temporary" Files?
If you're just inspecting PFX files, or otherwise wanting to work with them on a temporary basis, why bother writing the key to disk at all? Okay, says Windows Vista, we can just load the private key directly into a crypto key object, and we'll tell you the pointer.
PKCS12_NO_PERSIST_KEY / X509KeyStorageFlags.EphemeralKeySet
I'd like to think that if Windows had this feature in NT4 that this would have been the default for .NET. It can't be the default now, because too many things depend on the internals of how the "normal" import works to detect if a private key is usable.
What about the last two?
PFXImportCertStore's default mode is that the private keys should not be re-exportable. To tell it that it's wrong you can specify CRYPT_EXPORTABLE / X509KeyStorageFlags.Exportable.
CAPI and CNG both support a mechanism where the software keys can require consent or a password before the private key can be used (like a PIN prompt for a smart card), but you have to declare that when first creating (or importing) the key. So PFXImportCertStore allows you to specify CRYPT_USER_PROTECTED (and .NET exposes it as X509KeyStorageFlags.UserProtected).
These last two really only make sense for the "one private key" PFXes, because they apply to all the keys. They also don't encompass the full range of options that the origin keys could have had... both CNG and CAPI support "archivable" keys, which means "exportable once". Custom ACLs on machine keys also don't get any support in PFX.
Summary
For a certificate (or a collection of certificates), everything's easy. Once the private keys are involved things get messy, and the abstraction over Windows certificate (stores) gets a little thin and you need to be aware of the persistence model and the storage model.
