Uploaded image for project: 'HPCC'
  1. HPCC
  2. HPCC-26889

Instrumentation to collect queue metrics for auto scaling

    XMLWordPrintable

Details

    • Improvement
    • Status: Accepted
    • Major
    • Resolution: Unresolved
    • None
    • None
    • Dali
    • None
    • 2022q2
    • Allow users and operators to see metrics for all the active queues.

    Description

      Add instrumentation to necessary components to collect metrics for auto scaling of queue consumers.

      Some metrics may not be centralized in that different components collect the same value. Use of meta data allows the monitoring system to combine these into a single value for each collection period.

      All metrics follow the naming convention as outlined in devdoc/Metrics.rst. For all metrics required below, the scope is hpcc.

      Metrics:

      name: hpcc.queue_items.length
      type:  gauge
      units: SMeasureCount
      desc: Current count of items waiting in the queue to be processed.

      name: hpcc.queue_items.in_flight
      type: gauge
      units: SMeasureCount
      desc: Current count of items being processed.

      name: hpcc.queue.blocked_waiting
      type: gauge
      units: SMeasureCount
      desc: Current number of processes blocked waiting for a queue item to process.

      All metrics shall include meta data for the queue name. The meta data key shall be queue_name

       

      Attachments

        Activity

          People

            ghalliday Gavin Halliday
            kenrowland Kenneth A Rowland
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: