Uploaded image for project: 'HPCC'
  1. HPCC
  2. HPCC-12918

Graph control should improve output of the new statistics



    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.2.0
    • Component/s: WS-ECL
    • Labels:


      With version 5.2. the stats come out in a new format. They are now consistently named, and that naming should allow improved formatting of the results.

      • Each statistic has a root name - e.g., RowsProcessed.
      • Each statistic instance has a prefix which indicates the type of the value:

      "Time" - An elapsed time (raw value is in ns)
      "When" - A timestamp (stored to the microsecond)
      "Num" - a count of a number of occurrences
      "Size" - a quantity of memory or disk (raw in bytes)
      "Load" - measure of cpu activity (1/1M core)
      "Skew" - Skew per-10thousand. Range is -10,000 to +infinity
      "Node" - A node number within a cluster (0 = master)
      "Per" - A percentage (stored as parts per million) [Not currently used in a graph]

      Each raw statistic typically has some derived values. For a stat <Type><Name> you may get the following stats (e.g. Type = Num, Value = RowsProcessed):

      <Type><Name> e.g, NumRowsProcessed

      • the raw stat that is being recorded

      <Type>Min<Name> e.g, NumMinRowsProcessed

      • the value of the stat on the node with the minimum value

      <Type>Max<Name> e.g, NumMaxRowsProcessed

      • the value of the stat on the node with the maximum value

      <Type>Avg<Name> e.g, NumAvgRowsProcessed

      • the average value of the stat over all nodesvalue

      Skew<Name> e.g, SkewRowsProcessed

      • the skew in the value of the stat over all the nodes

      SkewMin<Name> e.g, SkewMinRowsProcessed

      • the skew of the stat on the node with the minimum value

      SkewMax<Name> e.g, SkewMaxRowsProcessed

      • the skew of the stat on the node with the maximum value

      NodeMin<Name> e.g, NodeMinRowsProcessed

      • the index of the node with the minimum value

      NodeMax<Name> e.g, NodeMaxRowsProcessed

      • the index of the node with the maximum value

      <Type>Delta<Name> e.g, NumDeltaRowsProcessed

      • the range in the value of the stats (where skew makes no sense because not zero based - e.g., timestamp).

      As far as the graph control goes:

      • By examining the prefix of the stat you can determine the type - even the derived stats. That might be useful for filtering.
      • It should be possible to create a table of raw stats by the derived values.
      • You should never get two stats with the same name, but different type prefixes e.g., NumRowsProcessed and WhenRowsProcessed, since the derived stats would be ambiguous.
      • https://track.hpccsystems.com/browse/HPCC-12885 adds the legacy information in a backwards compatible format. Once the graph control fully supports the new format, the old information should be stripped - possibly requiring an option in the SOAPCALL.

      Gordon Smith currently the system is adding formatted values to the graph. Would it be better if it always added unformatted values, and left all the formatting up to the graph control?




            gordonsmith Gordon Smith
            ghalliday Gavin Halliday
            0 Vote for this issue
            1 Start watching this issue