Download AVR 8.0 example
XML is starting to live up to its initial hype and truly be ubiquitious. Now that more and more vendors and business partners are passing XML around like it was a cheap bottle of wine at a campfire, the need to process XML with AVR is quickly growing.
One such task that often gets asked about is how to read an XML document and populate an ASNA DataGate file with it (ie, a data file available through DataGate which includes files on the iSeries, SQL Server, and hte local ASNA DB). This example shows how to do that.
The XML input file it uses is shown below (while this test file has only two "records" in it, the example would work with an XML file of any size). This file is rather simple, but generally these kinds of files are. And, unless the file grows to highly unwieldly nested levels, parsing complexity doesn't necessary grow as XML complexity does. The techniques presented in this article would well for moderately complex documents.
<?xml version="1.0"?>
<File>
<Member>
<Record>
<CMCustNo>100</CMCustNo>
<CMName>Binary Data Services</CMName>
<CMAddr1>112 East Hamish Street</CMAddr1>
<CMCity>Gotham City</CMCity>
<CMState>IL</CMState>
<CMCntry>US</CMCntry>
<CMPostCode>45678</CMPostCode>
<CMActive>1</CMActive>
<CMFax>8761238765</CMFax>
<CMPhone>(234) 123-7979</CMPhone>
</Record>
<Record>
<CMCustNo>200</CMCustNo>
<CMName>Carlton Industries</CMName>
<CMAddr1>13 West Lovely Street</CMAddr1>
<CMCity>Pinehurst</CMCity>
<CMState>NC</CMState>
<CMCntry>US</CMCntry>
<CMPostCode>34567</CMPostCode>
<CMActive>1</CMActive>
<CMFax>4087651292</CMFax>
<CMPhone>(456) 198-3590</CMPhone>
</Record>
</Member>
</File>
This article uses .NET's intrinsic XmlReader object to read the XML. The XmlReader is an often overlooked alternative to the XML DOM (document object model). While I find the XML DOM a little bit easier to use (perhaps not because it actually is easier but it's how I first learned to process XML) than the XmlReader, the XmlReader is much faster (by order of magnitutes as the document you're reading grows in size) than the DOM. There is good news and bad news here: the good news is that the XmlReader is very fast. Unlike the XML DOM, the XmlReader doesn't load the entire XML document into memory before processing it. It reads. on a foward-only basis, from top to bottom. The bad news is that the XmlReader can't be used to update the XML document and you can't randomly read through the document (like you can with the XML DOM). The XmlReader starts at the top and reads to the bottom. Generally, for the task of reading an XML document using its contents to update a data file, the XmlReader is very well suited.
The ReadXml class
To process the XML and perform the file adds, I wrote the following ReadXml class. It only has one public member, ImportFromXml(). ImportFromXml() reads through the XML document. and for each record, it it stores fields name and the field values in a NameValueCollection (from the System.Collections.Specialized namespace). The instance of this object is named Buffer. After each record has been read, Buffer is passed to WriteRecord() to write a new record to the data file.
The System.Xml.XmlReader is what actually lets us traverse our way through the Xml document. The XmlReader object is created on line 34 and traversing its contents starts on line 38. Each XML line read has a NodeType. For example <File>, <Member>. and <Record> lines are type XmlNodeType.Element types and </File>, </Member>. and </Record> lines are type XmlNodeType.EndElement. The element value File, Member, or Record can be read as the Reader.Name property. Interogating each line type is how you manage traverse your way through the document. In this example, having read the <Record> line signals that we're reading to read a record's fields (InRecord = *True) and having read the </Record> line signals that we're done reading fields for a record.
As field values are read (known because InRecord is true), the field name and its value are stored in an instance of NameValueCollection named Buffer. When the last record is read (determined in lines 61-65), Buffer is passed to the WriteRecord subroutine to populate record fields and write the new record.
Importing data from foreign sources can often cause problem with unique key values. In this example, I am ignoring the customer number value specified in the XML document and generating a new unique customer number with the GetNextCustomer routine (lines 110-127). You may need to process potential duplicate keys in some other fashion.
Do note that there is no error handling in this program. In the real world, you'll want to provide some pretty extensive error handling, especially in the Write record routine where the field assignments (lines 94-102) take place. There is potential for runtime error here because you're never really sure what values the XML document will provide (it may provide something really silly like characters for a numeric value, for example). Consider adding both good error handling and perhaps even a log that can help you determine what when wrong when it goes wrong (because sometimes you know it will!).
ReadXml class
0001 Using System
0002 Using System.Text
0003 Using System.Xml
0004 Using System.Collections.Specialized
0005 Using System.Data
0006 Using System.Text.RegularExpressions
0007
0008 BegClass ReadXml Access(*Public)
0009 DclDB pgmDB DBName( "*Public/DG Net Local" )
0010
0011 DclDiskFile Cust +
0012 Type( *Update ) +
0013 Org( *Indexed ) +
0014 Prefix( Cust_ ) +
0015 File( "Examples/CMastNewL1" ) +
0016 DB( pgmDB ) +
0017 ImpOpen( *No ) +
0018 AddRec( *Yes )
0019
0020 BegFunc ImportFromXml Type( *Boolean ) Access( *Public )
0021 //
0022 // Read the XML document.
0023 //
0024 DclSrParm XmlFileName Type( *String )
0025
0026 DclFld Reader Type( XmlReader )
0027 DclFld InRecord Type( *Boolean )
0028 DclFld Buffer Type( NameValueCollection ) New()
0029
0030 Connect pgmDB
0031 Open Cust
0032
0033 // Create a new XmlReader object for given XML file.
0034 Reader = XmlReader.Create( XmlFileName )
0035 // Move to XML content (past directives, etc).
0036 Reader.MoveToContent()
0037
0038 DoWhile ( Reader.Read() )
0039 // Look for first element of a new record.
0040 If ( Reader.NodeType = XmlNodeType.Element )
0041 If ( Reader.Name = "Record" )
0042 // You are now in a record's elements.
0043 InRecord = *True
0044 // Clear the field buffer.
0045 Buffer.Clear()
0046 Iterate
0047 EndIf
0048
0049 If ( InRecord )
0050 // Save the field name and its value in the Buffer collection.
0051 // The Buffer object, an instance of NameValueCollection stores
0052 // a value pair (a key and an associated value). In this case,
0053 // the key is the field name and the value is the field value.
0054 // If you're familiar with Web apps, the QueryString is also
0055 // an instance of a NameValueCollection.
0056 Buffer.Add( Reader.Name, Reader.ReadElementContentAsString() )
0057 EndIf
0058 EndIf
0059
0060 // Done reading data for this record.
0061 If ( Reader.NodeType = XmlNodeType.EndElement ) AND +
0062 ( Reader.Name = "Record" )
0063 // Write the record.
0064 WriteRecord( Buffer )
0065 EndIf
0066 EndDo
0067
0068 Reader.Close()
0069
0070 Close Cust
0071 Disconnect pgmDB
0072
0073 // In this magic world, there are _never_ errors!
0074 LeaveSr *True
0075 EndFunc
0076
0077 BegFunc WriteRecord Type( *Boolean )
0078 DclSrParm Buffer Type( NameValueCollection )
0079 //
0080 // Write the data file record from the NameValueCollection contents.
0081 //
0082 // This simple example provides NO error handling! In the real
0083 // world you'll want to wrap up some pretty robust error handling
0084 // around the data assignments. Few things are more frustrating than
0085 // having the import blow up on the 11000th record! Have your error
0086 // handling error track what XML row you're on and what field failed.
0087 //
0088 // Note that something usually has to be done to generate unique keys
0089 // for data imported from foreign sources. In this case, I'm calling
0090 // GetNextCustomerNumber() to get the next available customer number
0091 // and ignoring the number provided by the XML input. You may need
0092 // to process potential duplicate keys with other means.
0093 //
0094 Cust_CMCustNo = GetNextCustomerNumber()
0095 Cust_CMName = Buffer[ "CMName" ]
0096 Cust_CMAddr1 = Buffer[ "CMAddr1" ]
0097 Cust_CMState = Buffer[ "CMState" ]
0098 Cust_CMCntry = Buffer[ "CMCntry" ]
0099 Cust_CMPostCode = Buffer[ "CMPostCode" ]
0100 Cust_CMActive = Buffer[ "CMActive" ]
0101 Cust_CMFax = Buffer[ "CMFax" ]
0102 Cust_CMPhone = Buffer[ "CMPhone" ]
0103
0104 Write Cust
0105
0106 // In this magic world, there are _never_ errors!
0107 LeaveSr *True
0108 EndFunc
0109
0110 BegFunc GetNextCustomerNumber Type( *Integer8 )
0111 //
0112 // Get the next customer number.
0113 //
0114 DclFld BOF Type( *Boolean )
0115
0116 SetLL Cust Key( *End )
0117
0118 // Read last record.
0119 ReadP Cust Eof( BOF ) Access( *NoLock )
0120 If ( BOF )
0121 // If file is empty, return 100.
0122 LeaveSr Value( 100 )
0123 Else
0124 // Otherwise, return last customer number + 100.
0125 LeaveSr Value( Cust_CMCustNo + 100 )
0126 EndIf
0127 EndFunc
0128
0129 EndClass
The code below shows how the XML file is processed using the ReadXml class. In the downloadable example, the the OpenFileDialog control is used to let the user locate the file interactively.
BegSr ProcessXmlFile
DclSrParm XmlFileName Type( *String )
// Instance XML processing class.
DclFld rx Type( ReadXml ) New()
// Import from XML.
rx.ImportFromXml( XmlFileName )
EndSr