Uploaded image for project: 'HPCC'
  1. HPCC
  2. HPCC-19564

Partition keys issue

    XMLWordPrintable

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • 7.0.0
    • 7.0.0
    • JHTree
    • None

    Description

      Was getting some missing values on FETCH test - I think the cause is a difference of opinion on how the partition hash is interprted.

      Generated code when creating the index has:

                      virtual unsigned hash(const void * _self) override {
                              const unsigned char * self = (const unsigned char *) _self;
                              unsigned long long vT5;
                              vT5 = rtlHash64Data(10U,self + 0U,14695981039346656037LLU);
                              return (unsigned)vT5;
                      }
      

      but calculation of partition for picking the index has:

          virtual unsigned getPartition() override
          {
              if (partitionFieldMask)
              {
                  hash64_t hash = HASH64_INIT;
                  if (getBloomHash(partitionFieldMask, segs, hash))
                      return (hash % indexParts) + 1;
              }
              return 0;
          }
      

      I think this means that the distribution will truncate the hash to 32-bits before doing the modulus, but the getPartition will do the modulus at 64-bits. They may therefore give different answers

      Attachments

        Activity

          People

            richardkchapman Richard Chapman
            richardkchapman Richard Chapman
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: