Uploaded image for project: 'HPCC'
  1. HPCC
  2. HPCC-24651

Thor init scripts may kill processes from unrelated Thor's

    XMLWordPrintable

Details

    • Bug
    • Status: Accepted
    • Major
    • Resolution: Unresolved
    • None
    • None
    • Init system, Thor
    • None
    • Point
    • Not applicable

    Description

      The thor init scripts use killall to send a signal to terminate.
      If there are multiple Thor's co-existing on the same nodes, and the process names have the same 15 character prefix, this can cause one Thor to kill the processes of the other Thor unintentionally.

      This is reproducible on Ubuntu 20.04, but does not seem to happen on some other distros, e.g. Ubuntu 18.04

      The approach does seem unsafe though, according to the killdocs + other docs, in that the process name may be truncated to 15 chars, and killall will match all truncated processes.

      In the case where this was seen, 2 Thor clusers were defined "mythor" and "mythor". That meant that the slave process names were "thorslave_mythor" or "thorslave_mythor2", i.e. larger than 15 chars.

      mythor's init scripts use of killall "thorslave_mythor" causes the slaves of mythor2 to also die, because of the truncation issue.

      Attachments

        Activity

          People

            Michael-Gardner Michael Gardner
            jakesmith Jake Smith
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated: