Unicode Regex; Invalid XML characters

Question

The list of valid XML characters is well known, as defined by the spec it's:

#x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | [#x10000-#x10FFFF]

My question is whether or not it's possible to make a PCRE regular expression for this (or its inverse) without actually hard-coding the codepoints, by using Unicode general categories. An inverse might be something like [p{Cc}p{Cs}p{Cn}], except that improperly covers linefeeds and tabs and misses some other invalid characters.

Answer 1

I know this isn't exactly an answer to your question, but it's helpful to have it here:

Regular Expression to match valid XML Characters:

[u0009u000au000du0020-uD7FFuE000-uFFFD]

So to remove invalid chars from XML, you'd do something like

// filters control characters but allows only properly-formed surrogate sequences
private static Regex _invalidXMLChars = new Regex(
    @"(?<![uD800-uDBFF])[uDC00-uDFFF]|[uD800-uDBFF](?![uDC00-uDFFF])|[x00-x08x0Bx0Cx0E-x1Fx7F-x9FuFEFFuFFFEuFFFF]",
    RegexOptions.Compiled);

/// <summary>
/// removes any unusual unicode characters that can't be encoded into XML
/// </summary>
public static string RemoveInvalidXMLChars(string text)
{
    if (string.IsNullOrEmpty(text)) return "";
    return _invalidXMLChars.Replace(text, "");
}

I had our resident regex / XML genius, he of the 4,400+ upvoted post, check this, and he signed off on it.