Uploaded image for project: 'HPCC'
  1. HPCC
  2. HPCC-23297

Expose DFU workunit option "noCommon" - that prevents pullers being commoned up if on same target

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 7.6.20
    • Component/s: DFU Server
    • Labels:
      None

      Description

      When pulling there is normally a pulling process per partition up to a maximum number of concurrent pullers.
      However, if the target pullers are on the same host, they will be dedupped, meaning that if multiple partition parts are destined for the same node, there will only be 1 pull process (ftslave).

      That means that if the source parts on various remote hosts, they will be read in series, not concurrently, which depending on the relative speeds of the target and network speeds/latencies, can significantly effect the throughput.

      There is an option to prevent this commoning up "noCommon", but it is not exposed as an option either in dfuplus or esp/eclwatch.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                jakesmith Jake Smith
                Reporter:
                jakesmith Jake Smith
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: