Linux - reasons for SIGSTOP and how to deal with it?
<p>I have a Python script which is running bash scripts. I need to be able to kill the bash script if it seems to be infinite and it also has to be run in chroot jail because the script might be dangerous. I run it with <code>psutil.Popen()</code> and leave it running for two seconds. If it does not end naturally, I send <code>SIGKILL</code> to it and all of its possible children.</p>

<p>The problem is that if I kill one script due to overtime execution and run another one,  the main (Python) script receives a <code>SIGSTOP</code>. On my local machine, I made a really stupid solution: the Python script wrote its PID to a file at startup and then I run another script, which was sending <code>SIGCONT</code> every second to the PID which was stored in the file. This has two problems: it is really stupid, but even worse is that it refuses to work on the server - <code>SIGCONT</code> just does nothing there.</p>

<p>The sequence is: Python script runs a bash script responsive for the jail and that bash script runs the possibly dangerous and/or infinite script. This script might have some children as well.</p>

<p>The relevant parts of the codes:</p>

<p><strong>Main python script</strong></p>

<pre><code>    p = psutil.Popen(["bash", mode, script_path, self.TESTENV_ROOT])
    start = time.time()

    while True:
        if p.status() == psutil.STATUS_ZOMBIE:
            # process ended naturally
            duration = time.time() - start
            self.stdout.write("Script finished, execution time: {}s".format(duration))
            break

        if time.time() &gt; start + run_limit:
            children = p.children(recursive=True)
            for child in children:
                child.kill()
            p.kill()
            duration = None
            self.stdout.write("Script exceeded maximum time ({}s) and was killed.".format(run_limit))
            break

        time.sleep(0.01)

    os.kill(os.getpid(), 17)  # SIGCHLD
    return duration
</code></pre>

<p><strong>Running script in chroot</strong> ($1 is the script to be run in the chroot jail, $2 is the jail path)</p>

<pre><code>#!/usr/bin/env bash

# copy script to chroot environment
cp "$1" "$2/prepare.sh"

# run script
chmod u+x "$2/prepare.sh"
echo './prepare.sh' | chroot "$2"
rm "$2/prepare.sh"
</code></pre>

<p><strong>Example prepare.sh script</strong></p>

<pre><code>#!/bin/bash
echo asdf &gt; file
</code></pre>

<p>I spent some time trying to solve the issue. I found out that this script (which is not using chroot jail to run bash scripts) is working perfectly:</p>

<pre><code>import psutil
import os
import time

while True:
    if os.path.exists("infinite.sh"):
        p = psutil.Popen(["bash","infinite.sh"])
        start = time.time()

        while True:
            if p.status() == psutil.STATUS_ZOMBIE:
                # process ended naturally
                break

            if time.time() &gt; start + 2:
                # process needs too much time and has to be killed
                children = p.children(recursive=True)
                for child in children:
                    child.kill()

                p.kill()
                break

        os.remove("infinite.sh")
        os.kill(os.getpid(), 17)
</code></pre>

<p>My questions are:</p>

<ul>
<li>Why am I receiving <code>SIGSTOP</code>s? Is it due to the chroot jail?</li>
<li>Is there any better way of hadling my problem than by running the "waking up" script?</li>
</ul>

<p>Thanks for your ideas.</p>

<p><strong>EDIT:</strong> I found out that I am sigstopped at the moment I run the first script after I killed an overtime one. No matter if I use <code>os.system</code> or <code>psutil.Popen</code>.</p>

<p><strong>EDIT2:</strong> I did even more investigation and the critical line is <code>echo './prepare.sh' | chroot "$2"</code> in the bash script controlling the chroot jail. The question now is, what the hell is wrong with it?</p>

<p><strong>EDIT3:</strong> <a href="http://linux-kernel.vger.kernel.narkive.com/v8Jhd6WB/fork-sh-hello-microbenchmark-performance-in-chroot" rel="nofollow">This</a> might be a related problem, if it helps someone.</p>

<p>I'm pretty sure you're running this on Mac OS and not Linux. Why? You're sending signal <code>17</code> to your main python process instead of using:</p>

<pre><code>import signal
signal.SIGCHLD
</code></pre>

<p>I believe you have a handler for signal <code>17</code> which is supposed to respawn the jailed process in response to this signal.<br>
But <code>signal.SIGCHLD == 17</code> on Linux and <code>signal.SIGCHLD == 20</code> on Mac OS.  </p>

<p>Now the answer for your question is:<br>
<strong><code>signal.SIGSTOP == 17</code> on Mac OS</strong>.<br>
Yes, your process sends <code>SIGSTOP</code> to itself with <code>os.kill(os.getpid(), 17)</code><br>
<a href="https://developer.apple.com/library/ios/documentation/System/Conceptual/ManPages_iPhoneOS/man3/signal.3.html" rel="nofollow">Mac OS signal man page</a></p>

<p><strong>EDIT:</strong><br>
Actually it can also happen on Linux since <a href="http://man7.org/linux/man-pages/man7/signal.7.html" rel="nofollow">Linux signal man page</a> says that POSIX standard allows signal <code>17</code> to be either <code>SIGUSR2</code>, <code>SIGCHLD</code> or <code>SIGSTOP</code>. Therefore I strongly recommend using constants from <code>signal</code> module of the standard library instead of hardcoded signal numbers.</p>

python linux bash shell