Uploaded image for project: 'HPCC'
  1. HPCC
  2. HPCC-26600

Allow hthor job memory to be based on pod resource and other memory options/consolidation

    XMLWordPrintable

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 8.4.0
    • hthor, Thor
    • None

    Description

      Currently in K8s, hthor jobs are fixed to a hard-wired 300MB per query, regardless of the "resources" specification of the configuration.
      Allow hthor jobs to use most of the resource pod memory.
      To make consistent with Thor
      1) Pickup current config options in baremetal (workunit option "hthorMemoryLimit" and environment.xml option "defaultMemoryLimitMB"), and use as fixed memory definition.
      2) Use container based resource memory "resources" if available.
      3) If using resource memory, restrict to a maximum percentage of total memory (to allow for heap/other uses), default 90%, but configurable via "maxMemPercentage"
      4) Like Thor, add a memory section which can define how much of the resource memory should be dedicated to "query" (HPCC/roxiemem) vs thirdParty.
      5) Allow these settings to be overriden via workunit #options per query.

      Attachments

        Issue Links

          Activity

            People

              jakesmith Jake Smith
              jakesmith Jake Smith
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: