Details
Description
Currently the key merging code compares all fields when merging rows from multiple sources. It currently does this using a memcmp on the maximum length of the key record. This has several problems:
- It could access uninitialised data.
- The payload may not be sorted in a memcmp order - so may return rows out of order
- If field translation is in use there is no guarantee that the rows will be ordered
I propose changing the comparison so that it only compared the key fields. If they are identical the row from the lower part is returned.
I think the key merger is currently used in the following situations:
a) Smart stepping. This should be unaffected by the change.
b) Merging results from local indexes. This will potentially change.
c) Keyed join in thor. I suspect this is for convenience. For global keys it may be more efficient to use a key manager that does not compare - and takes rows from parts in order.
Attachments
Issue Links
- relates to
-
HPCC-19900 Project indexes to reduce fields transferred
-
- Accepted
-