Uploaded image for project: 'HPCC'
  1. HPCC
  2. HPCC-24269

Investigate cost/speed of default use of WUQueryDetails

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Not specified
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 7.10.6
    • Component/s: ESP
    • Labels:
      None

      Description

      A number of environments and jobs use esp services from SOAPCALLS to manage the validity and sometimes deployment of queries to Roxie.

      For example RAMPS uses WUQueryDetails in order to obtain the workunit a deployed query is based upon, and in a separate Esp service call, discover if there is a more current compiled version of that query.

      WUQueryDetails is an expensive request and does a lot more than just retrieve the workunit id, e.g. it also collates all files and looks them in Dali (with the associated locking that entails).

      Queries performing multiple WUQueryDetails (and other similar esp services) are therefore costly to themselves, to Dali and can have an impact on the performance of the whole environment.

      Therefore it is important that esp services called from queries like this are efficient and perform the minimal requirements necessary, either by allowing additional options to prevent them doing unwanted tasks, or by created new service calls which are designed specifically to perform the task required.

      In the RAMPS case, I think what is really wanted is a single call that asks whether a deployed query is up to date , and that it checks the query wuid against any newer compiled query.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                wangkx Kevin Wang
                Reporter:
                jakesmith Jake Smith
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: