Uploaded image for project: 'HPCC'
  1. HPCC
  2. HPCC-20057

Revisit through aggregate

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Unresourced
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Code Generator, Roxie, Thor
    • Labels:
      None

      Description

      Currently if a dataset is processed and aggregated it creates a splitter.  That can be very inefficient since it links all the rows and has to correctly buffer.

      Previously (and still in hthor?) there is a through aggregate which aggregated the results and saved it as a result as a side-effect.  It had various problems - especially with activities being pruned, and not all rows being read.

      A better version would be to generate an activity which had two outputs - one the unmodified dataset and the second the aggregate.  That way it would cleanly fit into the current activity structure, but have the advantage of not introducing splitters.  It is likely to have significant benefits in child queries.

       

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              ghalliday Gavin Halliday
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: