This was seen when working on : https://github.com/hpcc-systems/HPCC-Platform/pull/10620
Initially the PR.
The smoketest failed with 'Directory not empty' exceptions in various regression suite queries.
However, I could not reproduce the problem locally - my regression suite runs passed.
It was later discovered that in fact although my local suite runs passed, the workunits they ran had failed with the same exceptions. i.e. the regression suite had labelled them "Pass", when they should have been marked "Failure".
It seem the problem is inconsistent, sometimes the RS will capture the job exception and report failure other times it will report Pass.