Uploaded image for project: 'HPCC'
  1. HPCC
  2. HPCC-8969

Prevent very large #option('outputLimit') causing grief to Dali.

    XMLWordPrintable

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.10.8
    • Thor
    • None

    Description

      The default workunit(dali) result limit is 10MB and for good reason, anything genuinely large, should be written to a disk output file and not fill memory on Dali.

      The #option('outputLimit', <MB>) is there where it is awkward to avoid, sometimes that's because the workunit results are created internally by the codegen - even in those cases, the limit should not just be upped, it should be regarded as a hint to look at the ECL and recode it where possible.

      To prevent GB's of data being written by misuse of #option('outputLimit') - impose an upper hard limit, which it cannot be set above.

      Attachments

        Activity

          People

            jakesmith Jake Smith
            jakesmith Jake Smith
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: