Uploaded image for project: 'HPCC'
  1. HPCC
  2. HPCC-24198

Ensure noseek index builds do not buffer endless leaf nodes

    XMLWordPrintable

Details

    • Improvement
    • Status: Unresourced
    • Not specified
    • Resolution: Unresolved
    • None
    • None
    • JHTree
    • None

    Description

      A problem with the noseek code if the first row in an index has a blob, and no other rows have blob (because the associated fields are all zero length).  

      In this case all the leaf nodes will be buffered in memory until the end of the index.

      I suspect adding some code to flushNode (keybuild.cpp)  to add activeBlobNode to the pendingBlob list if (activeBlobNode->getFpos() - prevLeafNode->getFPos()) is > some threshold e.g. 1Mb would be the cleanest fix.

       

       

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              ghalliday Gavin Halliday
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated: