There are a lot of reasons why it would be preferable to start Thor "on demand" for each job rather than leave it running waiting for next job. These include:
- Can run thor as a user corresponding to the job owner rather than always using hpcc user
- Can more easily size thor according to the job needs
- Cloud-friendly - not paying for machines while idle
- Recovery from last spill may be easier to implement
- Likely to be more reliable
- Isolates jobs from side effects of previous jobs (e.g. Java VM or Python artefacts, resource leaks from C++ code (or platform bugs) etc
Main potential downside is that the startup time needs to be short enough to be insignificant for overall thor job times. We COULD consider continuing without a restart if there is another job waiting and some conditions were met, though you may lose some of the above advantages if you do.