Uploaded image for project: 'HPCC'
  1. HPCC
  2. HPCC-17415

Performance Improvement for GROUP(..., ALL)

    XMLWordPrintable

    Details

      Description

      Currently a GROUP(..., ALL) translates to a SORT followed by a LOCAL GROUP. It would be more efficient if it was DISTRIBUTE, LOCAL SORT, LOCAL GROUP. It also tends to create less skew which is beneficial for subsequent operations.

      One of the tests I ran went from 68 minutes down to 4 minutes with only this change.

        Attachments

          Activity

            People

            • Assignee:
              ghalliday Gavin Halliday
              Reporter:
              dustinskaggs Dustin Skaggs
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: