Uploaded image for project: 'HPCC'
  1. HPCC
  2. HPCC-10144

Add a global sort and distribution optimizer

    Details

    • Type: Suggestion
    • Status: Accepted
    • Priority: Not specified
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Code Generator
    • Labels:

      Description

      I was sure there was an existing issue, but I haven't been able to find it.

      The idea is to look at the entire graph, and track which sorts and distributions are actually used by downstream activities. That information can then be used to remove sorts/distributions that aren't actually needed, or add UNORDERED/UNSTABLE flags to different activities.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                anybody Available for anyone
                Reporter:
                ghalliday Gavin Halliday
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated: