Uploaded image for project: 'HPCC'
  1. HPCC
  2. HPCC-23297

Expose DFU workunit option "noCommon" - that prevents pullers being commoned up if on same target

    XMLWordPrintable

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 7.6.20
    • DFU Server
    • None

    Description

      When pulling there is normally a pulling process per partition up to a maximum number of concurrent pullers.
      However, if the target pullers are on the same host, they will be dedupped, meaning that if multiple partition parts are destined for the same node, there will only be 1 pull process (ftslave).

      That means that if the source parts on various remote hosts, they will be read in series, not concurrently, which depending on the relative speeds of the target and network speeds/latencies, can significantly effect the throughput.

      There is an option to prevent this commoning up "noCommon", but it is not exposed as an option either in dfuplus or esp/eclwatch.

      Attachments

        Issue Links

          Activity

            People

              jakesmith Jake Smith
              jakesmith Jake Smith
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: