Uploaded image for project: 'HPCC'
  1. HPCC
  2. HPCC-20864

Race condition if child query activity requests the whole (global) loop input dataset

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 7.0.2
    • Component/s: Thor
    • Labels:
      None

      Description

      If a child query activity on initialization (e.g. in onStart ), requests the global (from all slaves) loop input dataset, there is currently a race condition which can result in an assert being seen:

      "ERROR: assert(!readers) failed - file: thgraph.cpp, line 119"
      

      It is being caused by the loop iteration synchronization happening slightly too early, before the pending loop dataset is added into the result.
      As a result, the master can request a result before it is setup, then when the setup method is called it hits the assert.

      This is the stack hit:

      0004FB27 2018-10-30 02:47:22.342 24707  5143 "Backtrace:"
      0004FB28 2018-10-30 02:47:22.342 24707  5143 "  /opt/HPCCSystems/lib/libjlib.so(_Z16printStackReportx+0x3c) [0x7fa9f0b04d1c]"
      0004FB29 2018-10-30 02:47:22.343 24707  5143 "  /opt/HPCCSystems/lib/libjlib.so(_Z20raiseAssertExceptionPKcS0_j+0x83) [0x7fa9f0b05053]"
      0004FB2A 2018-10-30 02:47:22.343 24707  5143 "  /opt/HPCCSystems/lib/libgraph_lcr.so(_ZN16CThorGraphResult15setResultStreamEP21IRowWriterMultiReadery+0x47) [0x7fa9f57f4d07]"
      0004FB2B 2018-10-30 02:47:22.343 24707  5143 "  /opt/HPCCSystems/lib/libgraph_lcr.so(_ZN19CThorBoundLoopGraph7executeER13CActivityBasejP17IThorGraphResultsP21IRowWriterMultiReaderyjPKh+0x61) [0x7fa9f57f45b1]"
      0004FB2C 2018-10-30 02:47:22.343 24707  5143 "  /opt/HPCCSystems/lib/libactivityslaves_lcr.so(_ZN18CLoopSlaveActivity10getNextRowEb+0x402) [0x7fa9f6201182]"
      0004FB2D 2018-10-30 02:47:22.343 24707  5143 "  /opt/HPCCSystems/lib/libactivityslaves_lcr.so(_ZN18CLoopSlaveActivity14CNextRowFeeder10threadmainEv+0x54) [0x7fa9f6201694]"
      0004FB2E 2018-10-30 02:47:22.343 24707  5143 "  ./thorslave_mythor(_ZN9CThreaded3runEv+0x10) [0x417310]"
      0004FB2F 2018-10-30 02:47:22.343 24707  5143 "  /opt/HPCCSystems/lib/libjlib.so(_ZN6Thread5beginEv+0x2c) [0x7fa9f0baac5c]"
      0004FB30 2018-10-30 02:47:22.343 24707  5143 "  /opt/HPCCSystems/lib/libjlib.so(_ZN6Thread11_threadmainEPv+0x1e) [0x7fa9f0bac7ce]"
      0004FB31 2018-10-30 02:47:22.343 24707  5143 "  /lib64/libpthread.so.0(+0x7aa1) [0x7fa9eef58aa1]"
      

        Attachments

          Activity

            People

            • Assignee:
              jakesmith Jake Smith
              Reporter:
              jakesmith Jake Smith
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: