c# - Loading XML to an XDocument with a URL containing an ampersand

Question

XDocument xd = XDocument.Load("http://www.google.com/ig/api?weather=vilnius&hl=lt");

The ampersand & isn't a supported character in a string containing a URL when calling the Load() method. This error occurs:

XmlException was unhandled: Invalid character in the given encoding

How can you load XML from a URL into an XDocument where the URL has an ampersand in the querystring?

Answer 1

You need to URL-encode it as &:

XDocument xd = XDocument.Load(
    "http://www.google.com/ig/api?weather=vilnius&hl=lt");

You might be able to get away with using WebUtility.HtmlEncode to perform this conversion automatically; however, be careful that this is not the intended use of that method.

Edit: The real issue here has nothing to do with the ampersand, but with the way Google is encoding the XML document using a custom encoding and failing to declare it. (Ampersands only need to be encoded when they occur within special contexts, such as the <a href="…" /> element of (X)HTML. Read Ampersands (&'s) in URLs for a quick explanation.)

Since the XML declaration does not specify the encoding, XDocument.Load is internally falling back to default UTF-8 encoding as required by XML specification, which is incompatible with the actual data.

To circumvent this issue, you can fetch the raw data and decode it manually using the sample below. I don’t know whether the encoding really is Windows-1252, so you might need to experiment a bit with other encodings.

string url = "http://www.google.com/ig/api?weather=vilnius&hl=lt";
byte[] data;
using (WebClient webClient = new WebClient())
    data = webClient.DownloadData(url);

string str = Encoding.GetEncoding("Windows-1252").GetString(data);
XDocument xd = XDocument.Parse(str);