Uploaded image for project: 'HPCC'
  1. HPCC
  2. HPCC-20364

Unhandled exception if jlog throws an exception whilst writing to log

    Details

    • Type: Bug
    • Status: Scheduled
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 6.4.6, 7.0.x
    • Fix Version/s: None
    • Component/s: Code Generator, Thor
    • Labels:
      None
    • Environment:
      Fails slightly differently on different clusters (different platform versions), but fails consistently on a given cluster
    • Compatibility:
      Minor

      Description

      When loop depth exceeds a certain threshold (varies with data size and nature), the job fails.

      On platform dev (latest install), it produces the following error:

      In this case, if I run with a loop depth of 63, it runs fine.  If I run with a loop depth of 64, the job runs for a much longer time, and then terminates with the above error.  The 8-bit boundary is coincidental, as I find a different boundary when I change the details of the data.

      As the loop depth increases, there is very little data to process.  In this case, only about 100 records are being processed by the end of the loop.

      I suspect that this is a memory leak because on ALPHADEV (6.4.x version), it reports an out-of-memory error on one or more slaves when it occurs.  On PlatformDev (7.0.x), it reports the above error, which does not specify cause.

      I have also seen it fail in a running state – it stays running for hours with no error indication.  During this time, ECL watch seems dramatically slowed down, with many UI timeouts.

      ZAP Report is too big to attach.  WU is W20180820-105634 on PlatformDev http://10.173.160.101:8010.  I can re-run when you like, or send ZAP by some other means.

      It is 61 mb.

       

      Note:  This is the issue originally hit by Lily.  I was able to reproduce it by adding some non-separable data to my tests, which causes the looping to the configured maxDepth.  This lets me control the loop depth explicitly.

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                jakesmith Jake Smith
                Reporter:
                rdev Roger Dev
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Due:
                  Created:
                  Updated: