Uploaded image for project: 'HPCC'
  1. HPCC
  2. HPCC-9572

Fix Hash dedup row leak if flushing buckets after spilling started

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Not specified
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.0
    • Component/s: Thor
    • Labels:
      None

      Description

      Once a bucket has spilt, rows are streamed to disk, but the bucket HT will be used to store input rows (without expanding the HT), so that the bucket can still dedup some incoming rows.
      A callback is installed so that the memory manager can flush out these rows at any time.

      This flushing was unprotected, meaning it could clear the HT table and reset the count, whilst another thread was adding rows. The net result, it lost track of some rows added during this period, causing row leaks.

      NB: At the end of a query the memory manager is reset, so the leaks are recovered.

        Attachments

          Activity

            People

            • Assignee:
              jakesmith Jake Smith
              Reporter:
              jakesmith Jake Smith
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: