Uploaded image for project: 'HPCC'
  1. HPCC
  2. HPCC-25133

Decrease upper dali workunit result limit

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 7.12.20
    • Component/s: hthor, Roxie, Thor
    • Labels:

      Description

      The default limit for a dali workunit result is 10MB.
      This should be enough, anything bigger should not be targeted as a dali workunit result, and should instead be written to disk.
      This limit can be raised by using #option('outputLimitMb', <mb>) to a upper limit of 2GB.

      This is way too high, and can cause significant performance problems not only to Dali, but to the whole environment.

      Reduce this limit to something much more conservative, e.g. 100MB.

      There are times, when workunit results are implicitly generated by the code generator, by some ECL coding patterns.
      If this limit is hit under those circumstances, it hints that the ECL needs to be looked at, and refactored to avoid an inefficient pattern.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              jakesmith Jake Smith
              Reporter:
              jakesmith Jake Smith
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: