How to open large XML files
Jun 9, 2010

Problem: Open large xml files without loading the whole xml file in memory (more than 2 GB) Resolution:  When we think of working with any XML file, we normally think of using XML Document (DOM) or DataSet.LoadXML(). Some programmers also use XPathDocument to load the xml file. When we use the above objects, we are loading the files into the system memory. The problem is that, if the size of the xml file is for e.g. 5 GB to 7 GB, we have to load the complete file in System’s Memory. This will cost us systems memory and will throw “System out of Memory Exception”. The best approach to process such large files is to avoid loading the files into the memory. .NET framework is provided with XmlTextReader class.  XmlTextReader helps us processing the xml file line by line. In this way we are not loading the complete xml file into the memory but processing the file line by line, node by node. Below is an example of how to use XMLTextReader class: –
XmlTextReader AgencyTextReader = new XmlTextReader(filename);
AgencyTextReader.WhitespaceHandling = WhitespaceHandling.None;

while (AgencyTextReader.Read())
{
if (AgencyTextReader.NodeType == XmlNodeType.Element &&
AgencyTextReader.LocalName == "award" &&
AgencyTextReader.IsStartElement() == true)
{
ProcessAwardNode(AgencyTextReader);
AgencyTextReader.Skip();
}
}

private void ProcessAwardNode(XmlTextReader AwardReader)
{

XmlDocument AwardXmlDoc = new XmlDocument();

AwardXmlDoc.LoadXml(AwardReader.ReadOuterXml());

// we can use xpath as below

agencyID =AwardXmlDoc.SelectSingleNode("award/agencyID").InnerText;

}
   

Leave a Reply

Your email address will not be published. Required fields are marked *

*

Related Posts

Entity Framework 7 Beta Version 8 Launched

What is Entity Framework? Entity framework is an ORM that generates business objects…

Jun 2, 2016

Solve MS Ajax Script loading error in Safari & Chrome

Safari 3x and Google Chrome uses javascript engine called WebKit and which causes…

Jan 16, 2008

Solve: Partial rendering updates the page but CSS styles are lost.

In IE,  if you have a style-sheet updated to certain component (server component) and if…

Nov 10, 2008
Top

Copyright © Prakash Software Solutions Pvt. Ltd. All Rights Reserved.