java - Big XML file and OutOfMemoryError

Question

I’m trying to parse a XML file up to 500 mb in java. I tried to use SAX but it gives me this error java.lang.OutOfMemoryError: Java heap space at com.sun.org.apache.xerces.internal.util.XMLStringBuffer.append(Unknown Source) Can you help me? Thanks a lot. P.S. Smaller XML files works just fine

Answer 1

Most likely you're not using SAX correctly, or your application isn't suited for stream processing.

The whole point of SAX is to avoid keeping the entire XML structure in memory, but that's only possible if you can process the XML in small chunks without keeping much context, and if the result of the processing either is much smaller than the processed XML (so that it does not use too much memory either) or can itself be passed on to a recipient or written to disk continuously.

Edit: It's also possible that you simply have a memory leak, i.e. you're holding on to data that you don't need anymore , preventing it from getting garbage collected. If you use any Lists, Maps or Sets for processing the XML, make sure that anything you add to them while processing one chunk of XML is removed before you start the next chunk.