I had your same issue and researched it fairly extensively back in April 2011. As of then, the only way to have a top-level document in "standards mode" and a document in a child iframe in "quirks mode" in IE9 was to use a meta tag to have the browser behave as if it were IE8. (As far as I know, this is still the case and Microsoft has no intention to change it.) There are a variety of meta tags you can use to change browser mode, but the one I have used that has worked was:
<meta http-equiv="X-UA-Compatible" content="IE=8" />
If you include this meta tag, all of the documents should be properly rendered (per IE8 rules) according to their doctype.
Note, however, that this precludes you from using any of the newly supported css features in IE9, even in the top-level document. You won't be able to use border-radius, box-shadow, opacity, etc..
There's some more info on this at Will an iframe render in quirks mode?, which asks a more general question about iframes and doctypes in ie.
与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…