Details
-
New Feature
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
Description
When 3rd party services like Istio and Linkerd service meshes, inject sidecars into our pods, there is currently no clean/automatic way to tell them to stop.
Consequently our dynamically launches K8s job pods never terminate, as the injected sidecar container keep running indefinitely.
This is a known issue and K8s plan to rollout a solution (see https://github.com/kubernetes/enhancements/issues/753), but it has been in discussion for a long time, and a release with an implemented solution isn't imminent.
HPCC-24548 allowed for a command to be injected after our main process in the job.
That can be used to tell the Istio sidecar to quit, because they specifically provide a mechanism to do so, via listening for a 'quitquitquit' request.
However, that isn't an option for others, in particular Linkerd.
Another solution is to have the containers in a Job pod, share the same process space, such that a process with elevated permissions can signal (SIGTERM) directly to the sidecar processes.
Attachments
Issue Links
- relates to
-
HPCC-24548 HPCC Pods hanging with istio envoy (sidecar) running
-
- Resolved
-