Uploaded image for project: 'HPCC'
  1. HPCC
  2. HPCC-26443

Define offical way to limit query memory vs other memory (e.g. for 3rd party use)

    XMLWordPrintable

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 8.4.0
    • Helm, Thor
    • None

    Description

      There's a need to greater control in the way that then engines allocate memory (particularly on Thor perhaps).
      At the moment Thor reserves all memory for roxiemem based on the *resource limits defined in the yaml. In fact 90% of this memory leaving the rest for heap/OS etc.

      Introduce a section which allows specific amounts to be set aside for "query" (roxiemem) and "thirdParty" (reserved such that hpcc will try not to consume it itself).

      "query" and "thirdParty" are just labels, and we could if wanted extend to e.g. "python", "java" etc.
      Potentially, if we can find a way to support it, these values could be fed down to the 3rd parties involves, in order to restrict their memory usage.

      Attachments

        Issue Links

          Activity

            People

              jakesmith Jake Smith
              jakesmith Jake Smith
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: