Uploaded image for project: 'JAPI'
  1. JAPI
  2. JAPI-191

Spark-HPCC: Unsigned8 allow integer overflow for IDs

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 7.2.14
    • Component/s: dfsclient
    • Labels:
      None

      Description

      Java does not support an unsigned integer type so reading unsigned8 values from HPCC may cause an integer overflow. Currently BinaryRecordReader checks for this and throws an exception if an integer value will overflow.

      However, unsigned8 values are often used as unique IDs in datasets, in which case overflowing would be acceptable as the overflowed value will still be unique. An option should be exposed to end users to allow them to control if integer overflowing should be allowed when reading a file. It would be good if this could be controlled on a per-column basis.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                rpastrana Rodrigo Pastrana
                Reporter:
                mcmuja01 James McMullan
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: