Uploaded image for project: 'HPCC'
  1. HPCC
  2. HPCC-24198

Ensure noseek index builds do not buffer endless leaf nodes

    Details

    • Type: Improvement
    • Status: Unresourced
    • Priority: Not specified
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: JHTree
    • Labels:
      None

      Description

      A problem with the noseek code if the first row in an index has a blob, and no other rows have blob (because the associated fields are all zero length).  

      In this case all the leaf nodes will be buffered in memory until the end of the index.

      I suspect adding some code to flushNode (keybuild.cpp)  to add activeBlobNode to the pendingBlob list if (activeBlobNode->getFpos() - prevLeafNode->getFPos()) is > some threshold e.g. 1Mb would be the cleanest fix.

       

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                ghalliday Gavin Halliday
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated: