Uploaded image for project: 'HPCC'
  1. HPCC
  2. HPCC-24273

Improve WUListQueries performance on clusters with several thousand roxie queries

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 7.10.0, 7.6.54, 7.8.26
    • Component/s: EclWatch
    • Labels:
      None

      Description

      When viewing queries on clusters that have several thousand roxie queries (for example http://ramps_dr_esp:8010/?Widget=QuerySetQueryWidget#/stub/PublishedQueries) , the performance of WUListQueries slows to a crawl. Sometimes it times out.

       

      This is occurring on a 6.4.x cluster, but looking at the code in both the 6.4 and master branches of the hpcc platform, there doesn't appear to be anything that jumps out as a perfromance enhancement, so the problem probably still exists in 7.x.

       

      https://github.com/hpcc-systems/HPCC-Platform/blob/e7ead5c2ee3304c35113906a6774fc5878f6f5fa/esp/services/ws_workunits/ws_workunitsQuerySets.cpp#L1432  (master branch)

      https://github.com/hpcc-systems/HPCC-Platform/blob/candidate-6.4.26/esp/services/ws_workunits/ws_workunitsQuerySets.cpp#L1546 (6.4.26 branch)

       

      Which calls this :

      https://github.com/hpcc-systems/HPCC-Platform/blob/0660e4a2fec76c9457770eda0e0077282be0e969/common/workunit/workunit.cpp#L5323  (master branch)

      https://github.com/hpcc-systems/HPCC-Platform/blob/candidate-6.4.26/common/workunit/workunit.cpp#L2476 (6.4.26 branch)

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                jakesmith Jake Smith
                Reporter:
                drealeed Drea Leed
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: