Uploaded image for project: 'HPCC'
  1. HPCC
  2. HPCC-18689

segfault possibly due to mismatched ESPs

    XMLWordPrintable

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 6.2.22
    • 6.4.6
    • ESP
    • None
    • Dataland Staging

    Description

       

      Core

      10.241.12.204:/var/lib/HPCCSystems/staging_esp/core.10721

       

      Log

       

      10.241.12.204:/var/log/HPCCSystems/staging_esp/esp.log
      00259D6E 2017-11-06 13:56:20.335 10721 54106 "Signal:    11 Segmentation fault"
      00259D6F 2017-11-06 13:56:20.335 10721 54106 "Fault IP:  000000000040E636"
      00259D70 2017-11-06 13:56:20.335 10721 54106 "Accessing: 0000000000000028"
      00259D71 2017-11-06 13:56:20.335 10721 54106 "Registers:"
      00259D72 2017-11-06 13:56:20.335 10721 54106 "EAX:0000000000000000  EBX:00007F9C9E9AAD70  ECX:0000000000000009  EDX:0000000000000009  ESI:00007F9C9E9AAD70  EDI:00007F9C9E9AAD70"
      00259D73 2017-11-06 13:56:20.335 10721 54106 "CS:EIP:0033:000000000040E636"
      00259D74 2017-11-06 13:56:20.335 10721 54106 "   ESP:00007F9C04FFB408  EBP:0000000000F223B8"
      00259D75 2017-11-06 13:56:20.335 10721 54106 "Stack[00007F9C04FFB408]: 00007F9D0A67AF7D 4D67B3B300007F9D 00007F9C4D67B3B3 00F223B800007F9C 0000000000F223B8 9E9AAD7000000000 00007F9C9E9AAD70 0A67B7F600007F9C"
      00259D76 2017-11-06 13:56:20.335 10721 54106 "Stack[00007F9C04FFB428]: 00007F9D0A67B7F6 0000000700007F9D 0000000000000007 00F223B800000000 0000000000F223B8 9F0CF18000000000 00007F9C9F0CF180 0C6B6A8000007F9C"
      00259D77 2017-11-06 13:56:20.335 10721 54106 "Stack[00007F9C04FFB448]: 00007F9D0C6B6A80 C729555000007F9D 00007F9CC7295550 04FFB4A000007F9C 00007F9C04FFB4A0 C7568F4000007F9C 00007F9CC7568F40 C729556700007F9C"
      00259D78 2017-11-06 13:56:20.335 10721 54106 "Stack[00007F9C04FFB468]: 00000000C7295567 C639C87000000000 00007F9CC639C870 0000000100007F9C 0000000800000001 0C8DA33000000008 00007F9D0C8DA330 0000000100007F9D"
      00259D79 2017-11-06 13:56:20.335 10721 54106 "Stack[00007F9C04FFB488]: 0000000100000001 00F223B800000001 0000000000F223B8 9E9AAD7000000000 00007F9C9E9AAD70 0000000700007F9C 0000000000000007 0000000000000000"
      00259D7A 2017-11-06 13:56:20.335 10721 54106 "Stack[00007F9C04FFB4A8]: 0000000000000000 0000006E00000000 000000000000006E 0000000000000000 0000000000000000 04FF000000000000 00007F9C04FF0000 0000000000007F9C"
      00259D7B 2017-11-06 13:56:20.335 10721 54106 "Stack[00007F9C04FFB4C8]: 0000000000000000 C651E4B000000000 00007F9CC651E4B0 C41B507000007F9C 00007F9CC41B5070 0000007C00007F9C 00007F9C0000007C 0000000000007F9C"
      00259D7C 2017-11-06 13:56:20.335 10721 54106 "Stack[00007F9C04FFB4E8]: 0000000000000000 4D66235100000000 00007F9C4D662351 0C6A339C00007F9C 00007F9D0C6A339C FFFFFFFF00007F9D 00000006FFFFFFFF C651E4B000000006"
      00259D7D 2017-11-06 13:56:20.335 10721 54106 "Backtrace:"
      00259D7E 2017-11-06 13:56:20.337 10721 54106 "  /opt/HPCCSystems/lib/libjlib.so(+0xcf048) [0x7f9d0a5dd048]"
      00259D7F 2017-11-06 13:56:20.337 10721 54106 "  /opt/HPCCSystems/lib/libjlib.so(_Z13excsighandleriP9siginfo_tPv+0x33c) [0x7f9d0a5decbc]"
      00259D80 2017-11-06 13:56:20.337 10721 54106 "  /usr/lib/jvm/jre/lib/amd64/server/libjvm.so(+0x7fd3eb) [0x7f9ce67f03eb]"
      00259D81 2017-11-06 13:56:20.337 10721 54106 "  /usr/lib/jvm/jre/lib/amd64/server/libjvm.so(JVM_handle_linux_signal+0x1a1) [0x7f9ce67f4f71]"
      00259D82 2017-11-06 13:56:20.337 10721 54106 "  /usr/lib64/libpthread.so.0(+0xf370) [0x7f9d08e45370]"
      00259D83 2017-11-06 13:56:20.337 10721 54106 "  esp(_ZNK9HashTable18getHashFromElementEPKv+0x6) [0x40e636]"
      00259D84 2017-11-06 13:56:20.337 10721 54106 "  /opt/HPCCSystems/lib/libjlib.so(_ZNK14SuperHashTable11doFindExactEPKv+0x2d) [0x7f9d0a67af7d]"
      00259D85 2017-11-06 13:56:20.337 10721 54106 "  /opt/HPCCSystems/lib/libjlib.so(_ZNK14SuperHashTable4nextEPKv+0x16) [0x7f9d0a67b7f6]"
      00259D86 2017-11-06 13:56:20.337 10721 54106 "  /opt/HPCCSystems/lib/libesphttp.so(_ZN10SecHandler25validateSecFeaturesAccessER11MapStringToI14SecAccessFlagsS1_15MappingStringToIS1_S1_EEb+0x130) [0x7f9d0c6b6a80]"
      00259D87 2017-11-06 13:56:20.337 10721 54106 "  /opt/HPCCSystems/lib/libesphttp.so(ZN14EspHttpBinding19onFeaturesAuthorizeER11IEspContextR11MapStringToI14SecAccessFlagsS3_15MappingStringToIS3_S3_EEPKcS9+0x2b) [0x7f9d0c67a94b]"
      00259D88 2017-11-06 13:56:20.337 10721 54106 "  /opt/HPCCSystems/lib/libws_workunits.so(ZN12ws_workunits23CWsWorkunitsSoapBinding14processRequestEP11IRpcMessageS2+0xa6dc) [0x7f9c4d48a2d6]"
      00259D89 2017-11-06 13:56:20.337 10721 54106 "  /opt/HPCCSystems/lib/libesphttp.so(ZN12CSoapService14processRequestER12ISoapMessageS1+0x7cf) [0x7f9d0c6ab80f]"
      00259D8A 2017-11-06 13:56:20.337 10721 54106 "  /opt/HPCCSystems/lib/libesphttp.so(_ZN16CHttpSoapBinding17HandleSoapRequestEP12CHttpRequestP13CHttpResponse+0x290) [0x7f9d0c69fe00]"
      00259D8B 2017-11-06 13:56:20.337 10721 54106 "  /opt/HPCCSystems/lib/libesphttp.so(_ZN16CHttpSoapBinding13onSoapRequestEP12CHttpRequestP13CHttpResponse+0x20) [0x7f9d0c6a0df0]"
      00259D8C 2017-11-06 13:56:20.337 10721 54106 "  /opt/HPCCSystems/lib/libesphttp.so(_ZN14CEspHttpServer14processRequestEv+0x8df) [0x7f9d0c6883bf]"
      00259D8D 2017-11-06 13:56:20.337 10721 54106 "  /opt/HPCCSystems/lib/libesphttp.so(_ZN11CHttpThread9onRequestEv+0x154) [0x7f9d0c683c24]"
      00259D8E 2017-11-06 13:56:20.337 10721 54106 "  /opt/HPCCSystems/lib/libesphttp.so(_ZN18CEspProtocolThread3runEv+0x21) [0x7f9d0c6b4eb1]"
      00259D8F 2017-11-06 13:56:20.337 10721 54106 "  /opt/HPCCSystems/lib/libjlib.so(_ZN6Thread5beginEv+0x2c) [0x7f9d0a67c22c]"
      00259D90 2017-11-06 13:56:20.337 10721 54106 "  /opt/HPCCSystems/lib/libjlib.so(_ZN6Thread11_threadmainEPv+0x1e) [0x7f9d0a67dbfe]"
      00259D91 2017-11-06 13:56:20.337 10721 54106 "  /usr/lib64/libpthread.so.0(+0x7dc5) [0x7f9d08e3ddc5]"
      00259D92 2017-11-06 13:56:20.337 10721 54106 "  /usr/lib64/libc.so.6(clone+0x6d) [0x7f9d08b6c73d]"
       
      

       

      Kevin Wang mentioned:

      ESP crashed when authorizing WU feature access for: SOAP method <WUResult> from vchikte_prod@172.25.241.220

       

      Tony Kirk mentioned:

      I see Vani has workunits on Dev (6.4.4-1) and Sta clusters (6.2.22-1).  Could there be some interface conflict because of mixed environment, her hitting Staging ESP for Dev workunits from 6.4.x IDE, etc?

       

      Rodrigo Pastrana mentioned:

      This may be caused by the mismatch in ESPs (as Tony mentioned), but regardless we need to look deeper and prevent this issue in the future.

      Attachments

        Activity

          People

            rpastrana Rodrigo Pastrana
            rwagner42 Russell Wagner
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: