Uploaded image for project: 'HPCC'
  1. HPCC
  2. HPCC-16244

Investigate removing link decrement for new heap

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 6.4.0
    • Component/s: Memory Manager
    • Labels:
      None

      Description

      For the scanning allocator another possibility is to use two magic values:

      -1 = the row is freed and the link count is up to date
      -2 = the row is freed but the link count hasn't been decremented.

      This would potentially allow the release to be completely atomic operation free, and also remove the atomic operation from the allocate.

      However the link count is currently used to determine:

      • Is the heap free (for reclaiming the memory)
      • Is the heap full (to avoid allocating from it)
      • Is the heap reasonably full (to extend rather than scan).

      Assuming an extra flag PotentialFree is set, how could the functions above be coded?

      Note, each of these functions should only be called within a spin lock - which means that potentialFree cannot be cleared, and rows cannot be mapped from -2 to -1 by any other concurrent thread.

      IsHeapFree:

      if (PotentialFree)
      {
         PotentialFree = false;
         Walk through all heap entries
             if (-2) then set to -1 and inc toRelease.
             if (toRelease == count-1) break
         decrement count by toRelease
      }
      if (count ==1) then ...
      

      Is HeapFull (similar to hasAtLeastN below):

      if (count != max)
         return false;
      if (!PotentialFree)
         return true;
      Walk through all heap entries
             if (-2) then return false;
      //Don't care about releases occuring in parallel.
      return true;
      

      HeapHasAtleastN

      if (count + n < max)
         return false;
      if (!PotentialFree)
         return true;
      Walk through all heap entries
             if (-2) then
                available++; 
                if (count - available + n < max)
                   return false;
      return true;
      

      Need to carefully check through the code to see if there are any other uses of count that need adapting.

      Also experiment with always extending before scanning with the old code - since something like that is likely to be required for this situation.

        Attachments

          Activity

            People

            • Assignee:
              ghalliday Gavin Halliday
              Reporter:
              ghalliday Gavin Halliday
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: