Uploaded image for project: 'HPCC'
  1. HPCC
  2. HPCC-21542

ESP server runs "Segmentation fault" during retrieve a '01db_deleteswitch.ecl Performance Suite test case related work unit information.

    XMLWordPrintable

    Details

    • Type: Regression
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 7.0.14
    • Fix Version/s: 7.0.20
    • Component/s: Core Libraries, ESP
    • Labels:
      None
    • Environment:
      OBT Performance test

      Description

      In the result analyzing phase of OBT Performance Test, the process retrieve the list of workunits executed on the given day and engine with this (example) query:

       http://10.241.40.12:8010/WsWorkunits/WUQuery.json?PageSize=2500&Sortby=Jobname&Cluster=roxie&Jobname=*-190224-*

      Then it queries every workunit details to get exact execution time. with this kind of URL:

      http://10.241.40.12:8010/WsWorkunits/WUInfo.json?Wuid=W20190224-073918

       

      From 14th of February every WUInfo query related to 01db_deleteswitch.ecl workunit failed:

      This site can’t be reached
      10.241.40.12 refused to connect. 

      and the reason is the ESP server crash.:

      000000CC 2019-02-25 07:24:49.766   338  1603 "HTTP First Line: GET /WsWorkunits/WUInfo.json?Wuid=W20190224-073918 HTTP/1.1"
      000000CD 2019-02-25 07:24:49.766   338  1603 "GET /WsWorkunits/WUInfo.json, from unknown@10.145.172.84"
      000000CE 2019-02-25 07:24:49.791   338  1603 "WUInfo: W20190224-073918"
      000000CF 2019-02-25 07:24:49.800   338  1603 "================================================"
      000000D0 2019-02-25 07:24:49.800   338  1603 "Program:   10.241.40.12:/opt/HPCCSystems/bin/esp"
      000000D1 2019-02-25 07:24:49.800   338  1603 "Signal:    11 Segmentation fault"
      000000D2 2019-02-25 07:24:49.800   338  1603 "Fault IP:  00007FD38C4D5779"
      000000D3 2019-02-25 07:24:49.800   338  1603 "Accessing: 0000000000000000"
      000000D4 2019-02-25 07:24:49.800   338  1603 "Backtrace:"
      000000D5 2019-02-25 07:24:49.827   338  1603 "  /opt/HPCCSystems/lib/libjlib.so(_ZN12StatsScopeId12setScopeTextEPKcPS1_+0x259) [0x7fd38c4d5779]"
      000000D6 2019-02-25 07:24:49.827   338  1603 "  /opt/HPCCSystems/lib/libjlib.so(_ZNK11ScopeFilter7compareEPKc+0x147) [0x7fd38c4d8f97]"
      000000D7 2019-02-25 07:24:49.827   338  1603 "  /opt/HPCCSystems/lib/libworkunit.so(_ZNK14CLocalWorkUnit16getScopeIteratorERK13WuScopeFilter+0x911) [0x7fd390803cd1]"
      000000D8 2019-02-25 07:24:49.827   338  1603 "  /opt/HPCCSystems/lib/libworkunit.so(_Z18aggregateStatisticR16StatsAggregationP14IConstWorkUnitRK13WuScopeFilter13StatisticKind+0x38) [0x7fd39080b248]"
      000000D9 2019-02-25 07:24:49.827   338  1603 "  /opt/HPCCSystems/lib/libws_workunits.so(_ZN12ws_workunits8WsWuInfo16getTotalThorTimeEv+0x5b) [0x7fd34efa9bbb]"
      000000DA 2019-02-25 07:24:49.827   338  1603 "  /opt/HPCCSystems/lib/libws_workunits.so(_ZN12ws_workunits8WsWuInfo9getCommonER15IEspECLWorkunitm+0x2b0) [0x7fd34efa9fe0]"
      000000DB 2019-02-25 07:24:49.827   338  1603 "  /opt/HPCCSystems/lib/libws_workunits.so(_ZN12ws_workunits8WsWuInfo7getInfoER15IEspECLWorkunitm+0x1c) [0x7fd34efaf07c]"
      000000DC 2019-02-25 07:24:49.827   338  1603 "  /opt/HPCCSystems/lib/libws_workunits.so(_ZN14CWsWorkunitsEx8onWUInfoER11IEspContextR17IEspWUInfoRequestR18IEspWUInfoResponse+0x371) [0x7fd34ef911a1]"
      000000DD 2019-02-25 07:24:49.827   338  1603 "  /opt/HPCCSystems/lib/libws_workunits.so(_ZN12ws_workunits23CWsWorkunitsSoapBinding17onGetInstantQueryER11IEspContextP12CHttpRequestP13CHttpResponsePKcS8_+0x8b92) [0x7fd34eddf55e]"
      000000DE 2019-02-25 07:24:49.827   338  1603 "  /opt/HPCCSystems/lib/libesphttp.so(_ZN14EspHttpBinding5onGetEP12CHttpRequestP13CHttpResponse+0x1d8) [0x7fd391b6e878]"
      000000DF 2019-02-25 07:24:49.827   338  1603 "  /opt/HPCCSystems/lib/libws_workunits.so(_ZN25CWsWorkunitsSoapBindingEx5onGetEP12CHttpRequestP13CHttpResponse+0x98d) [0x7fd34efc514d]"
      000000E0 2019-02-25 07:24:49.827   338  1603 "  /opt/HPCCSystems/lib/libesphttp.so(_ZN14CEspHttpServer14processRequestEv+0x919) [0x7fd391b7f0c9]"
      000000E1 2019-02-25 07:24:49.827   338  1603 "  /opt/HPCCSystems/lib/libesphttp.so(_ZN11CHttpThread9onRequestEv+0xb2) [0x7fd391b74672]"
      000000E2 2019-02-25 07:24:49.827   338  1603 "  /opt/HPCCSystems/lib/libesphttp.so(_ZN18CEspProtocolThread3runEv+0x21) [0x7fd391baf561]"
      000000E3 2019-02-25 07:24:49.827   338  1603 "  /opt/HPCCSystems/lib/libjlib.so(_ZN6Thread5beginEv+0x2c) [0x7fd38c4e49cc]"
      000000E4 2019-02-25 07:24:49.827   338  1603 "  /opt/HPCCSystems/lib/libjlib.so(_ZN6Thread11_threadmainEPv+0x1e) [0x7fd38c4e5f3e]"
      000000E5 2019-02-25 07:24:49.827   338  1603 "  /lib64/libpthread.so.0(+0x7dd5) [0x7fd38a79fdd5]"
      000000E6 2019-02-25 07:24:49.827   338  1603 "  /lib64/libc.so.6(clone+0x6d) [0x7fd38a4c8ead]"
      000000E7 2019-02-25 07:24:49.827   338  1603 "Registers:"
      000000E8 2019-02-25 07:24:49.827   338  1603 "EAX:0000000000000001  EBX:0000000001E32DE8  ECX:0000000000000016  EDX:0000000000000008  ESI:0000000001E32DE9  EDI:0000000001E32DD6"
      000000E9 2019-02-25 07:24:49.827   338  1603 "R8 :00656C69466C6163  R9 :0000000000000012  R10:00007FD322FF9260  R11:00007FD38C4D54F0"
      000000EA 2019-02-25 07:24:49.827   338  1603 "R12:0000000000000000  R13:0000000000000000  R14:0000000001E32DD6  R15:00007FD314006160"
      000000EB 2019-02-25 07:24:49.827   338  1603 "CS:EIP:0033:00007FD38C4D5779"
      000000EC 2019-02-25 07:24:49.827   338  1603 "   ESP:00007FD322FF9800  EBP:00007FD322FF9830"
      000000ED 2019-02-25 07:24:49.827   338  1603 "Stack[00007FD322FF9800]: 00007FD314005980 0000001600007FD3 0000000000000016 22FF99E000000000 00007FD322FF99E0 0000000400007FD3 0000000000000004 0000000000000000"
      000000EE 2019-02-25 07:24:49.827   338  1603 "Stack[00007FD322FF9820]: 0000000000000000 8C4D8F9700000000 00007FD38C4D8F97 0000000000007FD3 0000000000000000 1400496000000000 00007FD314004960 0000000B00007FD3"
      000000EF 2019-02-25 07:24:49.827   338  1603 "Stack[00007FD322FF9840]: 000000000000000B 1400615000000000 00007FD314006150 0000001600007FD3 0000000000000016 140077A000000000 00007FD3140077A0 1400615000007FD3"
      000000F0 2019-02-25 07:24:49.827   338  1603 "Stack[00007FD322FF9860]: 00007FD314006150 0000001D00007FD3 000000000000001D 22FF99E000000000 00007FD322FF99E0 90803CD100007FD3 00007FD390803CD1 0000000000007FD3"
      000000F1 2019-02-25 07:24:49.827   338  1603 "Stack[00007FD322FF9880]: 0000000000000000 22FF9A5800000000 00007FD322FF9A58 0000000A00007FD3 000000000000000A 0000000000000000 0000000000000000 140049F000000000"
      000000F2 2019-02-25 07:24:49.827   338  1603 "Stack[00007FD322FF98A0]: 00007FD3140049F0 8C4283B400007FD3 00007FD38C4283B4 22FF99E000007FD3 00007FD322FF99E0 907F5B5200007FD3 00007FD3907F5B52 140057E000007FD3"
      000000F3 2019-02-25 07:24:49.827   338  1603 "Stack[00007FD322FF98C0]: 00007FD3140057E0 0000000000007FD3 7FFFFFE900000000 22FF99E07FFFFFE9 00007FD322FF99E0 9080BC7C00007FD3 00007FD39080BC7C 0000000000007FD3"
      000000F4 2019-02-25 07:24:49.827   338  1603 "Stack[00007FD322FF98E0]: 0000000000000000 22FF993000000000 00007FD322FF9930 22FF990000007FD3 00007FD322FF9900 22FF99E000007FD3 00007FD322FF99E0 7461747300007FD3"

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                ghalliday Gavin Halliday
                Reporter:
                attilavamos Attila Vamos
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: