Uploaded image for project: 'HPCC'
  1. HPCC
  2. HPCC-19906

Roxiemem 'Invalid pointer' hit during local join

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 7.0.0, 6.4.20
    • Fix Version/s: 6.4.22
    • Component/s: Thor
    • Labels:
      None

      Description

      Whilst loading the input of a join, if roxiemem's OOM callback mechanism calls to spill gathered rows whilst the input is finishing, it appears it can either corrupt or double free rows that are owned by the loading container and transferred to a stream used by the join. Leading to a 'Invalid pointer' error.

      Extensive examination of the code hasn't revealed exactly how this can occur, but a race condition bug has been spotted which could mean that whilst spilling the rows could be transferred to the streaming output, potentially this could happen mid spill, though in the cases seen it happens during the sort phase and by the time the save is called the rows have been swapped out.

      Despite this unintentional overlap and swapping of the owned rows from the loading container to a spilling stream, I can't see how this leads to a double free and the 'Invalid pointer'.
      It can however, if the timing is unfortunate, lead to the spill happening, but the stream fetched appearing empty, meaning the join input could have all or some[a spills worth] of missing rows.

        Attachments

          Activity

            People

            • Assignee:
              jakesmith Jake Smith
              Reporter:
              jakesmith Jake Smith
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: