Uploaded image for project: 'HPCC'
  1. HPCC
  2. HPCC-23270

Dafilesrv segfault (possibly due to invalid keyfilter syntax)

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: New
    • Priority: Not specified
    • Resolution: Unresolved
    • Affects Version/s: 7.4.16
    • Fix Version/s: None
    • Component/s: dafilesrv
    • Labels:
      None
    • Environment:
      160 cluster
      dev-roxie07-neut-160001is 10.173.160.1

      Description

      Segfault detected via ELK:
      Dec 10, 2019 @ 16:42:50.4182019-12-10 16:42:50.090
      Segmentation fault 11
      dev-roxie07-neut-160001 /var/log/HPCCSystems/mydafilesrv/DAFILESRV.2019_12_10.log 000000F3 35784 37515

      000000ED 2019-12-10 16:36:49.507 35784 37515 "jsocket(2,1601) wait_read POLLERR 11 l:7601 r:10.176.151.85:60627 err = 104 : A!:7601 -> 10.176.151.85:60627 (11)"
      000000EE 2019-12-10 16:36:49.507 35784 37515 "CRemoteClientHandler::main wait_read error"
      000000EF 2019-12-10 16:37:06.219 35784 35788 "SYS: LPT=0 APT=1012 PU= 4% MU= 18% MAL=246001664 MMP=245047296 SBK=954368 TOT=240588K RAM=2330528K SWP=0K
      Throttler(stdCmdThrotlter) - limit=20, delayMs=5000, cpuThreshold=75, queueLimit=1000
      Throttler(stdCmdThrotlter): statistics since 16:27:06
      Total delay of 0.00 seconds
      Requests currently queued: 0
      Throttler(slowCmdThrotlter) - limit=20, delayMs=5000, cpuThreshold=75, queueLimit=1000
      Throttler(slowCmdThrotlter): statistics since 16:27:06
      Total delay of 0.00 seconds
      Requests currently queued: 0"
      000000F0 2019-12-10 16:37:06.221 35784 35788 "NIC: [eth0] rxp/s=620.2 rxk/s=65.6 txp/s=76.4 txk/s=19.8 rxerrs=0 rxdrps=0 txerrs=0 txdrps=0 CPU: usr=2 sys=1 iow=0 idle=95"
      000000F1 2019-12-10 16:42:50.090 35784 37515 "================================================"
      000000F2 2019-12-10 16:42:50.090 35784 37515 "Program: 10.173.160.1:/opt/HPCCSystems/bin/dafilesrv"
      000000F3 2019-12-10 16:42:50.090 35784 37515 "Signal: 11 Segmentation fault"
      000000F4 2019-12-10 16:42:50.091 35784 37515 "Fault IP: 00007F62F51CB561"
      000000F5 2019-12-10 16:42:50.091 35784 37515 "Accessing: 0000000000000000"
      000000F6 2019-12-10 16:42:50.091 35784 37515 "Backtrace:"
      000000F7 2019-12-10 16:42:50.099 35784 37515 " /opt/HPCCSystems/lib/libeclrtl.so(_ZN9RowFilter9addFilterERK9RtlRecordPKc+0x31) [0x7f62f51cb561]"
      000000F8 2019-12-10 16:42:50.099 35784 37515 " /opt/HPCCSystems/lib/libdafsserver.so(_ZN23CRemoteDiskBaseActivity14setupInputMetaERK13IPropertyTreeP15IOutputMetaData+0xe0) [0x7f62f6e0af20]"
      000000F9 2019-12-10 16:42:50.100 35784 37515 " /opt/HPCCSystems/lib/libdafsserver.so(_ZN23CRemoteDiskReadActivityC1ER13IPropertyTreeP15IFileDescriptor+0x24b) [0x7f62f6e0bc6b]"
      000000FA 2019-12-10 16:42:50.100 35784 37515 " /opt/HPCCSystems/lib/libdafsserver.so(Z20createRemoteActivityR13IPropertyTreebPKS+0x333) [0x7f62f6e04973]"
      000000FB 2019-12-10 16:42:50.100 35784 37515 " /opt/HPCCSystems/lib/libdafsserver.so(_ZN17CRemoteFileServer15cmdStreamCommonEP13IPropertyTreeR12MemoryBufferS3_RNS_20CRemoteClientHandlerE+0x12f) [0x7f62f6e12a9f]"
      000000FC 2019-12-10 16:42:50.100 35784 37515 " /opt/HPCCSystems/lib/libdafsserver.so(_ZN17CRemoteFileServer19cmdStreamReadCommonER12MemoryBufferS1_RNS_20CRemoteClientHandlerE+0x4f) [0x7f62f6e1384f]"
      000000FD 2019-12-10 16:42:50.100 35784 37515 " /opt/HPCCSystems/lib/libdafsserver.so(_ZN17CRemoteFileServer14processCommandEhR12MemoryBufferS1_PNS_20CRemoteClientHandlerEPNS_10CThrottlerE+0x30d) [0x7f62f6e15e7d]"
      000000FE 2019-12-10 16:42:50.100 35784 37515 " /opt/HPCCSystems/lib/libdafsserver.so(_ZN17CRemoteFileServer10CThrottler10addCommandEhR12MemoryBufferPNS_20CRemoteClientHandlerE+0x369) [0x7f62f6e18499]"
      000000FF 2019-12-10 16:42:50.100 35784 37515 " /opt/HPCCSystems/lib/libdafsserver.so(_ZN17CRemoteFileServer17cCommandProcessor10threadmainEv+0x1e1) [0x7f62f6e18d21]"
      00000100 2019-12-10 16:42:50.100 35784 37515 " /opt/HPCCSystems/lib/libjlib.so(_ZThn16_N20CPooledThreadWrapper3runEv+0xa5) [0x7f62f277b3f5]"
      00000101 2019-12-10 16:42:50.100 35784 37515 " /opt/HPCCSystems/lib/libjlib.so(_ZN6Thread5beginEv+0x28) [0x7f62f2772608]"
      00000102 2019-12-10 16:42:50.100 35784 37515 " /opt/HPCCSystems/lib/libjlib.so(_ZN6Thread11_threadmainEPv+0x1d) [0x7f62f2771c8d]"
      00000103 2019-12-10 16:42:50.100 35784 37515 " /lib64/libpthread.so.0(+0x7dc5) [0x7f62f081edc5]"
      00000104 2019-12-10 16:42:50.100 35784 37515 " /lib64/libc.so.6(clone+0x6d) [0x7f62f054d73d]"
      00000105 2019-12-10 16:42:50.100 35784 37515 "Registers:"
      00000106 2019-12-10 16:42:50.100 35784 37515 "EAX:00007F62E0046E40 EBX:00007F62E002B8F8 ECX:0000000000000001 EDX:0000000000000000 ESI:000000000412174C EDI:0000000000000000"
      00000107 2019-12-10 16:42:50.100 35784 37515 "R8 :0000000000000008 R9 :0000000000000040 R10:0000000000000005 R11:00007F62F26A0130"
      00000108 2019-12-10 16:42:50.100 35784 37515 "R12:00007F62E002B8F8 R13:00007F62E002B918 R14:00007F62E0034BB0 R15:00007F62E0000F00"
      00000109 2019-12-10 16:42:50.100 35784 37515 "CS:EIP:0033:00007F62F51CB561"
      0000010A 2019-12-10 16:42:50.100 35784 37515 " ESP:00007F62EE62FE00 EBP:0000000000000000"
      0000010B 2019-12-10 16:42:50.100 35784 37515 "Stack[00007F62EE62FE00]: 0000000000000030 E0000DD000000000 00007F62E0000DD0 E002B8B000007F62 00007F62E002B8B0 F6E0AF2000007F62 00007F62F6E0AF20 E002B8B000007F62"
      0000010C 2019-12-10 16:42:50.100 35784 37515 "Stack[00007F62EE62FE20]: 00007F62E002B8B0 E0000C7000007F62 00007F62E0000C70 E002B8C800007F62 00007F62E002B8C8 F6E0BC6B00007F62 00007F62F6E0BC6B 011790F600007F62"
      0000010D 2019-12-10 16:42:50.100 35784 37515 "Stack[00007F62EE62FE40]: 00000000011790F6 E0000C7000000000 00007F62E0000C70 E002B8B000007F62 00007F62E002B8B0 E0048E1000007F62 00007F62E0048E10 0000000000007F62"
      0000010E 2019-12-10 16:42:50.100 35784 37515 "Stack[00007F62EE62FE60]: 0000000000000000 F6E0497300000000 00007F62F6E04973 EE63033000007F62 00007F62EE630330 011E386000007F62 00000000011E3860 E0000BA000000000"
      0000010F 2019-12-10 16:42:50.100 35784 37515 "Stack[00007F62EE62FE80]: 00007F62E0000BA0 0000000000007F62 0000000000000000 0000000100000000 0000000000000001 F6E12A9F00000000 00007F62F6E12A9F 119D00E100007F62"
      00000110 2019-12-10 16:42:50.100 35784 37515 "Stack[00007F62EE62FEA0]: FFFF809D119D00E1 00000000FFFF809D 0000000000000000 011EA6B000000000 00000000011EA6B0 E001347100000000 00007F62E0013471 EE63037000007F62"
      00000111 2019-12-10 16:42:50.100 35784 37515 "Stack[00007F62EE62FEC0]: 00007F62EE630370 0000000000007F62 0000000000000000 E0000D9000000000 00007F62E0000D90 EE63033000007F62 00007F62EE630330 0000000000007F62"
      00000112 2019-12-10 16:42:50.100 35784 37515 "Stack[00007F62EE62FEE0]: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 00003A3900000000"
      00000113 2019-12-10 16:42:50.100 35784 37515 "ThreadList:
      7F62EEE32700 140062891386624 35788: CMemoryUsageReporter
      7F62EE631700 140062882993920 37515: CRemoteFileServerPool
      7F62EDE30700 140062874601216 37516: Member of thread pool: CRemoteFileServerPool
      7F62ED62F700 140062866208512 37534: CSocketBaseThread
      7F62ECE2E700 140062857815808 37535: Member of thread pool: CRemoteFileServerPool
      7F62DFFFF700 140062641616640 37536: Member of thread pool: CRemoteFileServerPool
      7F62DF7FE700 140062633223936 37537: Member of thread pool: CRemoteFileServerPool
      7F62DEFFD700 140062624831232 37538: Member of thread pool: CRemoteFileServerPool
      7F62DE7FC700 140062616438528 37539: Member of thread pool: CRemoteFileServerPool
      7F62DDFFB700 140062608045824 37540: Member of thread pool: CRemoteFileServerPool
      7F62DD7FA700 140062599653120 37541: Member of thread pool: CRemoteFileServerPool
      7F62DCFF9700 140062591260416 37542: Member of thread pool: CRemoteFileServerPool
      7F62CFFFF700 140062373181184 37543: Member of thread pool: CRemoteFileServerPool
      7F62CF7FE700 140062364788480 37544: Member of thread pool: CRemoteFileServerPool
      7F62CEFFD700 140062356395776 37545: Member of thread pool: CRemoteFileServerPool
      7F62CE7FC700 140062348003072 37546: Member of thread pool: CRemoteFileServerPool
      7F62CDFFB700 140062339610368 37547: Member of thread pool: CRemoteFileServerPool
      7F62CD7FA700 140062331217664 37548: Member of thread pool: CRemoteFileServerPool
      7F62CCFF9700 140062322824960 37549: Member of thread pool: CRemoteFileServerPool
      7F62C7FFF700 140062238963456 37550: Member of thread pool: CRemoteFileServerPool
      7F62C77FE700 140062230570752 37551: Member of thread pool: CRemoteFileServerPool
      7F62C6FFD700 140062222178048 37552: Member of thread pool: CRemoteFileServerPool
      "
      00000001 2019-12-10 16:42:55.328 46308 46308 "Dafilesrv starting - Build internal_7.6.17-closedown12102019015901[remotes/origin/candidate-7.6.x-0-gd171a5-dirty]"

        Attachments

          Activity

            People

            Assignee:
            jakesmith Jake Smith
            Reporter:
            rpastrana Rodrigo Pastrana
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Dates

              Created:
              Updated: