Uploaded image for project: 'HPCC'
  1. HPCC
  2. HPCC-14492

Redis Plugin - increase regression test stability

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Trivial
    • Resolution: Fixed
    • Affects Version/s: 6.0.0
    • Fix Version/s: 6.0.0
    • Component/s: Plugins
    • Labels:
      None

      Description

      //Test Publish and Subscribe
      //SUM(NOFOLD(s1 + s2), a) uses two threads - this test relies on this fact to work!
      INTEGER N2 := 1000;
      subDS := DATASET(N2, TRANSFORM({ integer a }, SELF.a := (INTEGER)myRedis.Subscribe('PubSubTest' + (STRING)COUNTER)));
      
      INTEGER pub2(STRING channel) := FUNCTION
              sl := SEQUENTIAL(
                  Std.System.Debug.Sleep(2),
                  myRedis.Publish(channel, '3')//This pub is the one read by the sub.
                  );
              value :=  myRedis.Publish(channel, '10000');//This pub isn't read by the sub, however the returned subscription count is present in the sum
           RETURN WHEN(value, sl, BEFORE);
      END;
      pubDS2 := DATASET(N2, TRANSFORM({ integer a }, SELF.a := pub2('PubSubTest' + (STRING)COUNTER)));
      OUTPUT(SUM(NOFOLD(subDS + pubDS2), a));
      

      The following test occasionally becomes unsynchronized between the subscriptions and publishes as shown in the redis MONITOR output:

      1446773145.007342 [0 127.0.0.1:42957] "AUTH" "foobared"
      1446773145.007563 [0 127.0.0.1:42957] "SUBSCRIBE" "PubSubTest179"
      1446773145.009523 [0 127.0.0.1:42779] "PUBLISH" "PubSubTest179" "3"
      1446773145.009705 [0 127.0.0.1:42779] "PUBLISH" "PubSubTest179" "10000"
      1446773145.010064 [0 127.0.0.1:42958] "AUTH" "foobared"
      1446773145.010255 [0 127.0.0.1:42958] "SUBSCRIBE" "PubSubTest180"
      1446773145.012000 [0 127.0.0.1:42779] "PUBLISH" "PubSubTest180" "3"
      1446773145.012180 [0 127.0.0.1:42779] "PUBLISH" "PubSubTest180" "10000"
      1446773145.016091 [0 127.0.0.1:42779] "PUBLISH" "PubSubTest181" "3"
      1446773145.016143 [0 127.0.0.1:42959] "AUTH" "foobared"
      1446773145.016313 [0 127.0.0.1:42779] "PUBLISH" "PubSubTest181" "10000"
      1446773145.016360 [0 127.0.0.1:42959] "SUBSCRIBE" "PubSubTest181"
      1446773145.018781 [0 127.0.0.1:42779] "PUBLISH" "PubSubTest182" "3"
      1446773145.019018 [0 127.0.0.1:42779] "PUBLISH" "PubSubTest182" "10000"
      1446773145.021332 [0 127.0.0.1:42779] "PUBLISH" "PubSubTest183" "3"
      1446773145.021788 [0 127.0.0.1:42779] "PUBLISH" "PubSubTest183" "10000"
      1446773145.024136 [0 127.0.0.1:42779] "PUBLISH" "PubSubTest184" "3"
      1446773145.024309 [0 127.0.0.1:42779] "PUBLISH" "PubSubTest184" "10000"
      1446773145.026661 [0 127.0.0.1:42779] "PUBLISH" "PubSubTest185" "3"
      1446773145.026866 [0 127.0.0.1:42779] "PUBLISH" "PubSubTest185" "10000"
      1446773145.029221 [0 127.0.0.1:42779] "PUBLISH" "PubSubTest186" "3"
      1446773145.029374 [0 127.0.0.1:42779] "PUBLISH" "PubSubTest186" "10000"
      1446773145.031828 [0 127.0.0.1:42779] "PUBLISH" "PubSubTest187" "3"
      1446773145.032033 [0 127.0.0.1:42779] "PUBLISH" "PubSubTest187" "10000"

      The culprit being PubSubTest181 where it subscribes after the publish rather than before. Increasing the sleep before the publish from 2ms -> 10ms should alleviate this though will now extend the total tests running time by 16s (when doing so for the other similar test also).

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                jamienoss Jamie Noss
                Reporter:
                jamienoss Jamie Noss
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: