Uploaded image for project: 'HPCC'
  1. HPCC
  2. HPCC-9884

ecl publish does not copy top-level-key to roxie even if timestamp has changed

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.0.2
    • Component/s: None
    • Labels:
      None

      Description

      I have an environment with the following settings:
      ================================
      allFilesDynamic=false
      copyResources=true
      useRemoteResources=true
      number of roxie nodes= 3
      number of thor slaves= 3
      ================================

      Steps to reproduce:
      1) ecl run --target=thor icecream_data.ecl
      2) ecl publish --target=roxie -A --name=icecream_query icecream_query.ecl
      3) Get the timestamps and md5sums for hpcc-data/thor directory and hpcc-data/roxie dir

      ====thor====

      loch@node020021:~$ md5sum /var/lib/HPCCSystems/hpcc-data/thor/thor/icecream/key/*
      bbd66f0bb7ae4fdf56f612af39728f50  /var/lib/HPCCSystems/hpcc-data/thor/thor/icecream/key/icecream_name_password._1_of_4
      42c82cf774aa6bcc8b6afa1508b24a96  /var/lib/HPCCSystems/hpcc-data/thor/thor/icecream/key/icecream_name_password_2._1_of_4
      2699349525a62e350280102712863351  /var/lib/HPCCSystems/hpcc-data/thor/thor/icecream/key/icecream_name_password_2._4_of_4
      c6f1cee0a1413e666731716b71f52397  /var/lib/HPCCSystems/hpcc-data/thor/thor/icecream/key/icecream_name_password_3._1_of_4
      6c5697eda99d396be7debf0423f2b862  /var/lib/HPCCSystems/hpcc-data/thor/thor/icecream/key/icecream_name_password_3._4_of_4
      d22e3346f4e5b50b633e30e4b6bfde29  /var/lib/HPCCSystems/hpcc-data/thor/thor/icecream/key/icecream_name_password._4_of_4
      loch@node020021:~$ ls -ltr /var/lib/HPCCSystems/hpcc-data/thor/thor/icecream/key/*
      -rw-r--r-- 1 hpcc hpcc 24576 Aug 22 07:24 /var/lib/HPCCSystems/hpcc-data/thor/thor/icecream/key/icecream_name_password._1_of_4
      -rw-r--r-- 1 hpcc hpcc 32768 Aug 22 07:24 /var/lib/HPCCSystems/hpcc-data/thor/thor/icecream/key/icecream_name_password._4_of_4
      -rw-r--r-- 1 hpcc hpcc 24576 Aug 22 07:24 /var/lib/HPCCSystems/hpcc-data/thor/thor/icecream/key/icecream_name_password_2._1_of_4
      -rw-r--r-- 1 hpcc hpcc 32768 Aug 22 07:24 /var/lib/HPCCSystems/hpcc-data/thor/thor/icecream/key/icecream_name_password_2._4_of_4
      -rw-r--r-- 1 hpcc hpcc 24576 Aug 22 07:24 /var/lib/HPCCSystems/hpcc-data/thor/thor/icecream/key/icecream_name_password_3._1_of_4
      -rw-r--r-- 1 hpcc hpcc 32768 Aug 22 07:24 /var/lib/HPCCSystems/hpcc-data/thor/thor/icecream/key/icecream_name_password_3._4_of_4
      
      
      

      ====roxie====

      loch@node020021:~$ md5sum /var/lib/HPCCSystems/hpcc-data/roxie/thor/icecream/key/*
      1d440344e084b7645c08803c67d14da6  /var/lib/HPCCSystems/hpcc-data/roxie/thor/icecream/key/icecream_name_password._2_of_4
      2a679fbb6cbbab99cd0d7dc3914d1687  /var/lib/HPCCSystems/hpcc-data/roxie/thor/icecream/key/icecream_name_password_3._2_of_4
      6c5697eda99d396be7debf0423f2b862  /var/lib/HPCCSystems/hpcc-data/roxie/thor/icecream/key/icecream_name_password_3._4_of_4
      d22e3346f4e5b50b633e30e4b6bfde29  /var/lib/HPCCSystems/hpcc-data/roxie/thor/icecream/key/icecream_name_password._4_of_4
      loch@node020021:~$ ls -ltr /var/lib/HPCCSystems/hpcc-data/roxie/thor/icecream/key/*
      -rw-r--r-- 1 hpcc hpcc 32768 Aug 22 07:24 /var/lib/HPCCSystems/hpcc-data/roxie/thor/icecream/key/icecream_name_password._4_of_4
      -rw-r--r-- 1 hpcc hpcc 32768 Aug 22 07:24 /var/lib/HPCCSystems/hpcc-data/roxie/thor/icecream/key/icecream_name_password_3._4_of_4
      -rw-r--r-- 1 hpcc hpcc 24576 Aug 22 07:24 /var/lib/HPCCSystems/hpcc-data/roxie/thor/icecream/key/icecream_name_password._2_of_4
      -rw-r--r-- 1 hpcc hpcc 24576 Aug 22 07:24 /var/lib/HPCCSystems/hpcc-data/roxie/thor/icecream/key/icecream_name_password_3._2_of_4
      

      4) Delete the Roxie Query
      5) rerun ecl run --target=thor icecream_data.ecl
      6) Get the md5sum and timestamps of the hpcc-data/thor directory

      ====thor====

      loch@node020021:~$ ls -ltr /var/lib/HPCCSystems/hpcc-data/thor/thor/icecream/key/*
      -rw-r--r-- 1 hpcc hpcc 24576 Aug 22 07:40 /var/lib/HPCCSystems/hpcc-data/thor/thor/icecream/key/icecream_name_password._1_of_4
      -rw-r--r-- 1 hpcc hpcc 32768 Aug 22 07:40 /var/lib/HPCCSystems/hpcc-data/thor/thor/icecream/key/icecream_name_password._4_of_4
      -rw-r--r-- 1 hpcc hpcc 24576 Aug 22 07:40 /var/lib/HPCCSystems/hpcc-data/thor/thor/icecream/key/icecream_name_password_2._1_of_4
      -rw-r--r-- 1 hpcc hpcc 32768 Aug 22 07:40 /var/lib/HPCCSystems/hpcc-data/thor/thor/icecream/key/icecream_name_password_2._4_of_4
      -rw-r--r-- 1 hpcc hpcc 24576 Aug 22 07:40 /var/lib/HPCCSystems/hpcc-data/thor/thor/icecream/key/icecream_name_password_3._1_of_4
      -rw-r--r-- 1 hpcc hpcc 32768 Aug 22 07:40 /var/lib/HPCCSystems/hpcc-data/thor/thor/icecream/key/icecream_name_password_3._4_of_4
      loch@node020021:~$ md5sum /var/lib/HPCCSystems/hpcc-data/thor/thor/icecream/key/*
      bbd66f0bb7ae4fdf56f612af39728f50  /var/lib/HPCCSystems/hpcc-data/thor/thor/icecream/key/icecream_name_password._1_of_4
      42c82cf774aa6bcc8b6afa1508b24a96  /var/lib/HPCCSystems/hpcc-data/thor/thor/icecream/key/icecream_name_password_2._1_of_4
      2699349525a62e350280102712863351  /var/lib/HPCCSystems/hpcc-data/thor/thor/icecream/key/icecream_name_password_2._4_of_4
      c6f1cee0a1413e666731716b71f52397  /var/lib/HPCCSystems/hpcc-data/thor/thor/icecream/key/icecream_name_password_3._1_of_4
      6c5697eda99d396be7debf0423f2b862  /var/lib/HPCCSystems/hpcc-data/thor/thor/icecream/key/icecream_name_password_3._4_of_4
      d22e3346f4e5b50b633e30e4b6bfde29  /var/lib/HPCCSystems/hpcc-data/thor/thor/icecream/key/icecream_name_password._4_of_4
      

      7) republish the query again
      8) look at the information under hpcc-data/roxie

      ====roxie====

      loch@node020021:~$ ls -ltr /var/lib/HPCCSystems/hpcc-data/roxie/thor/icecream/key/*
      -rw-r--r-- 1 hpcc hpcc 32768 Aug 22 07:24 /var/lib/HPCCSystems/hpcc-data/roxie/thor/icecream/key/icecream_name_password._4_of_4
      -rw-r--r-- 1 hpcc hpcc 32768 Aug 22 07:24 /var/lib/HPCCSystems/hpcc-data/roxie/thor/icecream/key/icecream_name_password_3._4_of_4
      -rw-r--r-- 1 hpcc hpcc 24576 Aug 22 07:40 /var/lib/HPCCSystems/hpcc-data/roxie/thor/icecream/key/icecream_name_password._2_of_4
      -rw-r--r-- 1 hpcc hpcc 24576 Aug 22 07:40 /var/lib/HPCCSystems/hpcc-data/roxie/thor/icecream/key/icecream_name_password_3._2_of_4
      loch@node020021:~$ md5sum /var/lib/HPCCSystems/hpcc-data/roxie/thor/icecream/key/*
      1d440344e084b7645c08803c67d14da6  /var/lib/HPCCSystems/hpcc-data/roxie/thor/icecream/key/icecream_name_password._2_of_4
      2a679fbb6cbbab99cd0d7dc3914d1687  /var/lib/HPCCSystems/hpcc-data/roxie/thor/icecream/key/icecream_name_password_3._2_of_4
      6c5697eda99d396be7debf0423f2b862  /var/lib/HPCCSystems/hpcc-data/roxie/thor/icecream/key/icecream_name_password_3._4_of_4
      d22e3346f4e5b50b633e30e4b6bfde29  /var/lib/HPCCSystems/hpcc-data/roxie/thor/icecream/key/icecream_name_password._4_of_4
      
      
      

      Richard Chapman

        Attachments

          Activity

            People

            Assignee:
            richardkchapman Richard Chapman
            Reporter:
            cloln Chris Lo
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: