Uploaded image for project: 'HPCC'
  1. HPCC
  2. HPCC-17209

Thor job coring in global JOIN/DENORMALIZE if LHS is pre-sorted.

    XMLWordPrintable

    Details

    • Type: Regression
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 6.2.10
    • Fix Version/s: 6.2.10
    • Component/s: Thor
    • Labels:
      None
    • Environment:
      Alpha dev

      Description

      Jobs that formerly ran successfully on 6.2.6 are now encountering core dumps with stack traces like:

      00001239 2017-03-13 16:35:09.336 17458 17458 ",Progress,Thor,StartSubgraph,thor20_166_sta02,W20170313-161815,11,214,thor20_166_sta,thor20_166_staging_dev.thor,thor20_166_staging_qc.thor,thor20_166_stagi
      ng_prod.thor,thor20_166_staging_scrub.thor,thor20_166_sta_eclcc_dev.thor,thor20_166_staging_qc_scrub.thor"
      0000123A 2017-03-13 16:35:09.336 17458 32633 "process - activity(ch=0, denormalize, 217)"
      0000123B 2017-03-13 16:35:09.349 17458 32633 "Sort setup cosort=true, needconnect=true has key serializer - activity(ch=0, denormalize, 217)"
      0000123C 2017-03-13 16:35:09.349 17458 32633 "================================================"
      0000123D 2017-03-13 16:35:09.349 17458 32633 "Signal: 11 Segmentation fault"
      0000123E 2017-03-13 16:35:09.349 17458 32633 "Fault IP: 00007F9E22A128B2"
      0000123F 2017-03-13 16:35:09.349 17458 32633 "Accessing: 0000000000000010"
      00001240 2017-03-13 16:35:09.349 17458 32633 "Registers:"
      00001241 2017-03-13 16:35:09.349 17458 32633 "EAX:0000000000000000 EBX:0000000004B9CE90 ECX:0000000000000000 EDX:00007F9E223A24D0 ESI:0000000000000000 EDI:00007F9E14000F20"
      00001242 2017-03-13 16:35:09.349 17458 32633 "CS:EIP:0033:00007F9E22A128B2"
      00001243 2017-03-13 16:35:09.349 17458 32633 " ESP:00007F9CAE5AE350 EBP:00007F9E14001A80"
      00001244 2017-03-13 16:35:09.349 17458 32633 "Stack[00007F9CAE5AE350]: 00007F9E14007A20 21BFB4D100007F9E 00007F9E21BFB4D1 049DA7B800007F9E 00000000049DA7B8 0000000000000000 0000000000000000 14000F200000
      0000"
      00001245 2017-03-13 16:35:09.349 17458 32633 "Stack[00007F9CAE5AE370]: 00007F9E14000F20 B17271E000007F9E 00007F9CB17271E0 000000D900007F9C 00000000000000D9 0000008000000000 0000000000000080 049D90C00000
      0000"
      00001246 2017-03-13 16:35:09.349 17458 32633 "Stack[00007F9CAE5AE390]: 00000000049D90C0 221674F800000000 00007F9E221674F8 049DA7C000007F9E 00000000049DA7C0 04B92F1000000000 0000000004B92F10 AE5AE4400000
      0000"
      00001247 2017-03-13 16:35:09.349 17458 32633 "Stack[00007F9CAE5AE3B0]: 00007F9CAE5AE440 14001A8000007F9C 00007F9E14001A80 049DA7C000007F9E 00000000049DA7C0 049DADB000000000 00000000049DADB0 000000000000
      0000"
      00001248 2017-03-13 16:35:09.349 17458 32633 "Stack[00007F9CAE5AE3D0]: 0000000000000000 22E7752B00000000 00007F9E22E7752B 0000000000007F9E 0000000000000000 14001A8000000000 00007F9E14001A80 AE5AE4000000
      7F9E"
      00001249 2017-03-13 16:35:09.349 17458 32633 "Stack[00007F9CAE5AE3F0]: 00007F9CAE5AE400 14000F2000007F9C 00007F9E14000F20 049DA7B000007F9E 00000000049DA7B0 1CC5EE0000000000 00007F9E1CC5EE00 000000000000
      7F9E"
      0000124A 2017-03-13 16:35:09.349 17458 32633 "Stack[00007F9CAE5AE410]: 0000000000000000 0000000000000000 0000000000000000 01929F8000000000 00A0000001929F80 049DA7B800A00000 00000000049DA7B8 000000130000
      0000"
      0000124B 2017-03-13 16:35:09.349 17458 32633 "Stack[00007F9CAE5AE430]: 0000000000000013 0000001300000000 0000000000000013 01929F8000000000 0000000001929F80 1FD4FD9000000000 00007F9E1FD4FD90 AE5AE4400000
      7F9E"
      0000124C 2017-03-13 16:35:09.349 17458 32633 "Backtrace:"
      0000124D 2017-03-13 16:35:09.350 17458 32633 " /opt/HPCCSystems/lib/libjlib.so(+0xd0958) [0x7f9e1d36e958]"
      0000124E 2017-03-13 16:35:09.350 17458 32633 " /opt/HPCCSystems/lib/libjlib.so(_Z13excsighandleriP7siginfoPv+0x33c) [0x7f9e1d3705cc]"
      0000124F 2017-03-13 16:35:09.350 17458 32633 " /lib64/libpthread.so.0(+0xf7e0) [0x7f9e1c0897e0]"
      00001250 2017-03-13 16:35:09.350 17458 32633 " /opt/HPCCSystems/lib/libthorsort_lcr.so(ZN11CSortMaster9SortSetupEP18IThorRowInterfacesP8ICompareP18ISortKeySerializerbbPKcS1+0xc2) [0x7f9e22a128b2]"
      00001251 2017-03-13 16:35:09.350 17458 32633 " /opt/HPCCSystems/lib/libactivitymasters_lcr.so(_ZN18JoinActivityMaster7processEv+0x46b) [0x7f9e22e7752b]"
      00001252 2017-03-13 16:35:09.350 17458 32633 " /opt/HPCCSystems/lib/libgraphmaster_lcr.so(_ZN15CMasterActivity4mainEv+0x10) [0x7f9e227d8d90]"
      00001253 2017-03-13 16:35:09.350 17458 32633 " /mnt/disk1/var/lib/HPCCSystems/thor20_166_sta02/thormaster_thor20_166_sta02(_ZN9CThreaded3runEv+0x10) [0x40ecb0]"
      00001254 2017-03-13 16:35:09.350 17458 32633 " /opt/HPCCSystems/lib/libjlib.so(_ZN6Thread5beginEv+0x2c) [0x7f9e1d40e01c]"
      00001255 2017-03-13 16:35:09.350 17458 32633 " /opt/HPCCSystems/lib/libjlib.so(_ZN6Thread11_threadmainEPv+0x1e) [0x7f9e1d40f9de]"
      00001256 2017-03-13 16:35:09.350 17458 32633 " /lib64/libpthread.so.0(+0x7aa1) [0x7f9e1c081aa1]"
      00001257 2017-03-13 16:35:09.350 17458 32633 " /lib64/libc.so.6(clone+0x6d) [0x7f9e1bdceaad]"

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                jakesmith Jake Smith
                Reporter:
                kev77log Kevin Logemann
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: