Uploaded image for project: 'HPCC'
  1. HPCC
  2. HPCC-23079

When empty string provided as header and/or footer for an XML output, all engines set the wrong value to headerLength and/or footerLength metadata.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 5.4.0
    • Fix Version/s: 7.6.10, 7.4.34
    • Component/s: Thor
    • Labels:
      None

      Description

      If an XML file write out with 'HEADING('' ,'')' parameter then the header and footer are absent, but the logical file metainformation contains: 

      /opt/HPCCSystems/bin/daliadmin . dfsfile thor::spray-test-xmlsimple_xml
      <File directory="/var/lib/HPCCSystems/hpcc-data/thor/thor"
            group="mythor"
            modified="2019-11-04T15:57:22"
            numclusters="1"
            numparts="6"
            partmask="spray-test-xmlsimple_xml._$P$_of_6"
            replicated="0">
       <OrigName>thor::spray-test-xmlsimple_xml</OrigName>
       <Attr accessed="2019-11-04T15:57:22"
             checkSum="2027469177"
             footerLength="11"
             format="utf8n"
             headerLength="10"
             job=""
             kind="xml"
             maxSkew="50344"
             maxSkewPart="0"
             minSkew="10000"
             minSkewPart="1"
             owner=""
             recordCount="3"
             rowTag="Row"
             size="175"
             workunit="W20191104-155719">
        <ECL>{ string3 name, unsigned2 age, boolean good };&#10;</ECL>
        <_rtlType xsi:type="SOAP-ENC:base64">
         UQz1gaekJHtPMgAAAAQAAAADAQEAAAIAAAAAAQ0AQAAGA25hbWUAAAQAAABhZ2UAAQEBAABn
      b29kAAIAAAAA  </_rtlType>
       </Attr>
       <Cluster defaultBaseDir="/var/lib/HPCCSystems/hpcc-data/thor"
                mapFlags="16"
                name="mythor"
                redundancy="0">
        <Group>192.168.1.56*6</Group>
       </Cluster>
       <Part fileCrc="-2027469178"
             modified="2019-11-04T15:57:22"
             node="192.168.1.56"
             num="1"
             recordCount="3"
             size="175"/>
       <Part fileCrc="0"
             modified="2019-11-04T15:57:22"
             node="192.168.1.56"
             num="2"
             recordCount="0"
             size="0"/>
       <Part fileCrc="0"
             modified="2019-11-04T15:57:22"
             node="192.168.1.56"
             num="3"
             recordCount="0"
             size="0"/>
       <Part fileCrc="0"
             modified="2019-11-04T15:57:22"
             node="192.168.1.56"
             num="4"
             recordCount="0"
             size="0"/>
       <Part fileCrc="0"
             modified="2019-11-04T15:57:22"
             node="192.168.1.56"
             num="5"
             recordCount="0"
             size="0"/>
       <Part fileCrc="0"
             modified="2019-11-04T15:57:22"
             node="192.168.1.56"
             num="6"
             recordCount="0"
             size="0"/>
      </File>
       

      A code fragment to reproduce the problem:

      import Std.File AS FileServices;
      
      prefix := 'spray-test-xml';
      
      dropzonePath := FileServices.GetDefaultDropZone() +'/' : STORED('dropzonePath');
      
      Layout_Person := RECORD
        STRING3  name;
        UNSIGNED2 age;
        BOOLEAN good;
      END;
      
      dsSetup := DATASET([ {'foo', 10, 1},
                             {'bar', 12, 0},
                             {'baz', 32, 1}]
                  ,Layout_Person);
                  
      setupSimpleXmlPrepFileName := prefix + 'simple_xml';
      setupSimpleXmlFile := output(dsSetup, , setupSimpleXmlPrepFileName, XML( HEADING('' ,'')), OVERWRITE);
      
      setupSimpleXmlPrepFileName2 := prefix + 'simple2_xml';
      setupSimpleXmlFile2 := output(dsSetup, , setupSimpleXmlPrepFileName2, XML, OVERWRITE); 
      
      sequential(
        setupSimpleXmlFile,
        output(fileservices.GetLogicalFileAttribute(setupSimpleXmlPrepFileName, 'headerLength'), NAMED('headerLength')),
        output(fileservices.GetLogicalFileAttribute(setupSimpleXmlPrepFileName, 'footerLength'), NAMED('footerLength')),
        
        setupSimpleXmlFile2,
        output(fileservices.GetLogicalFileAttribute(setupSimpleXmlPrepFileName2, 'headerLength'), NAMED('headerLength2')),
        output(fileservices.GetLogicalFileAttribute(setupSimpleXmlPrepFileName2, 'footerLength'), NAMED('footerLength2')),
        );
       
      

       

      Results in ECLWatch Playground:

      Result 1  ( Write out a file with (XML( HEADING('' ,''))) )

      #### line _fileposition_
      1 <Row><name>foo</name><age>10</age><good>true</good></Row> 0
      2 <Row><name>bar</name><age>12</age><good>false</good></Row> 58
      3 <Row><name>baz</name><age>32</age><good>true</good></Row> 117

       headerLength:

      #### headerLength
      ​1 10

      footerLenght:

      ## footerLength
      1 11

       

       

      Result 4  ( Write out a file with ( XML only )

      #### line _fileposition_
      1 <Dataset> 0
      2 <Row><name>foo</name><age>10</age><good>true</good></Row> 10
      3 <Row><name>bar</name><age>12</age><good>false</good></Row> 68
      4 <Row><name>baz</name><age>32</age><good>true</good></Row> 127

       headerLength2:

      #### headerLength
      ​1 10

      footerLenght2:

      ## footerLength
      1 11

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                jakesmith Jake Smith
                Reporter:
                attilavamos Attila Vamos
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Due:
                  Created:
                  Updated:
                  Resolved: