Uploaded image for project: 'HPCC'
  1. HPCC
  2. HPCC-29250

Provide more details of cpu/memory for a subgraph

    XMLWordPrintable

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 9.2.0, 9.0.12, 8.12.24
    • Thor
    • None

    Description

      There are various stats that could be gathered and reported for thor subgraphs to help determine the best instance types to run on. A list of stats and why they might be useful...

      Thor: (per subgraph/graph)

      • Memory high water mark – c heap and roxiemem. => Decrease/increase the size of the instances.
      • Blocked time
      • K8s startup time => Contact MS
      • Waiting for a Thor instance to be available. => Increase number of thors
      • Graph initialisation time, child query reinitialization. => Optimize platform
      • File lookup time, and all other dali requests => Optimize platform
      • Waiting for a file lock – and any over a threshold. => Re-examine the jobs are submitted
      • Unknown blocking – all “external” calls.
      • Size of parent context. => Possible issues with the ecl (e.g. x in set..)
      • Cpu utilization: user, sys, elapsed times, #context switches. => change the instance type.
      • Network utilization. Total MB transferred. Average wait time? Dropped packets. => Instance types, faster network, number of nodes, structure of query?
      • Index Page cached v non cached disk reads. (Count and time.) =>

      Attachments

        Activity

          People

            ghalliday Gavin Halliday
            ghalliday Gavin Halliday
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: