Uploaded image for project: 'HPCC'
  1. HPCC
  2. HPCC-13906

A reliable way to identify the type of file on a cluster

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 5.2.4
    • Fix Version/s: 6.0.0
    • Component/s: ESP
    • Labels:
      None

      Description

      there is currently no way to reliably determine the file type of a file based on the DFUInfo contentType.

      for flat/thor files, it's "flat".
      For csv files written out by thor, it's "flat".
      For sprayed csv/xml file, it's null, and the format field needs to be checked to see if it's "csv" or "xml". Some xml files don't have "xml set, and I need to check the ecl records for xpath definitions. If a file is sprayed as utf-8, the format is "utf-8" and I again need to try and figure out the file type based on the ecl record definiton and/or a sample of the data.

      Request: Add an attribute into DFUInfoResponse that accurately reflects the file type of thor files, csv files created by thor, csv files sprayed to thor regardless of ansi/utf-8 type, and xml files created by thor and sprayed to thor.

        Attachments

          Activity

            People

            • Assignee:
              attilavamos Attila Vamos
              Reporter:
              drealeed Drea Leed
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: