Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
Minor
-
2020q1
-
Refactor index format so that it can be entirely streamed without needing to seek back to update the header. This will help support external file stores where seeking is not allowed.
Description
Currently when an index is built the header of the file is updated after everything else. This means that the output cannot be done through a purely streaming interface. We should investigate what information is being updated, and allow this step to be removed. Possibilities include defaulting to the last node if the offsets are not set up, or duplicating the header of the file at the end.
Requires more investigation.
