Uploaded image for project: 'HPCC'
  1. HPCC
  2. HPCC-11883

Backupnode inefficiently reads Dali file meta data

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Timed Out
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: BackUpNode
    • Labels:
      None
    • Compatibility:
      Minor

      Description

      Backupnode is run on every Thor cluster, where there are multiple thor's on the same node group, it is run on the primary Thor.
      Currently it is scheduled via cron, it should run as a service likeXREF, see : HPCC-11273.

      The 1st stage of backupnode, is to gather file part information from Dali and find logical files that are part of the cluster it's running on.
      To do so, it scans every File to check if they are resident on the relevant nodes.
      In a large environment with 1000's of files, this is expensive and wasteful as the same thing is done on all the other primary Thor's in the environment.

      If backupnode becomes a service of Sasha, it could/should at least do this stage once for all clusters in one sweep.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              jakesmith Jake Smith
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: