How to debug external .py functions run from Jupyter/IPython notebook
<p>My Jupyter/IPython notebook executes functions in an external .py.</p>

<p>I need to set breakpoints within these functions, inspect variables, single step, etc.</p>

<p>It just isn't practical to use a combination of <code>print</code> statements and throwing exceptions to early-exit a cell.</p>

<p>I need some kind of workflow.</p>

<p>Is it possible to hook up some third-party editor/IDE to view the .py and somehow connect it to the Python runtime Jupyter/IPython is using?</p>

<p>So that if I set a breakpoint in my external .py using my IDE and execute a cell in the notebook which encounters said breakpoint, I can continue to navigate manually from within the IDE.</p>

<p>EDIT: I've found <a href="https://pypi.python.org/pypi/ipdb" rel="nofollow">https://pypi.python.org/pypi/ipdb</a> <a href="https://www.quora.com/What-are-your-favorite-tricks-for-IPython-Notebook" rel="nofollow">https://www.quora.com/What-are-your-favorite-tricks-for-IPython-Notebook</a></p>

<p>EDIT <a href="https://www.youtube.com/watch?v=Jb2HHOahvcE" rel="nofollow">https://www.youtube.com/watch?v=Jb2HHOahvcE</a> &lt;-- this video is getting close to what I'm after, I just can't quite see how to put it all together. That video demonstrates spyder which is an IDE with an IPython prompt... I wonder if maybe I can run my notebook through the prompt and debug it.</p>

<p>EDIT: It looks as though PyCharm does exactly what I'm after: <a href="https://www.jetbrains.com/help/pycharm/2016.1/tutorial-using-ipython-jupyter-notebook-with-pycharm.html" rel="nofollow">https://www.jetbrains.com/help/pycharm/2016.1/tutorial-using-ipython-jupyter-notebook-with-pycharm.html</a> </p>

<p>EDIT: I'm in the middle of trying to get PyCharm to behave. I will provide the details in an answer if I sort it out.</p>

<blockquote>
  <p>Is it possible to hook up some third-party editor/IDE to view the .py
  and somehow connect it to the Python runtime Jupyter/IPython is using?</p>
</blockquote>

<p>Yes, <strong>it's possible</strong>.</p>

<h2>Using <a href="https://www.gnu.org/software/emacs/" rel="nofollow">Emacs</a> as your third party Python IDE.</h2>

<p>You can open a file side-by-side with the iPython runtime. It is possible to send definitions from the external file directly to the prompt using key shortcuts, and <strong>set breakpoints, single step and inspect variables</strong> from the iPython prompt using the standard python debugger pdb.</p>

<p>Auto-completion works fine. Simply writing <code>import XXX</code> makes Emacs aware of all functions within the <code>XXX</code> module.  </p>

<p>Here is an example screenshot:
<a href="http://i.stack.imgur.com/17GSY.png" rel="nofollow"><img src="http://i.stack.imgur.com/17GSY.png" alt="Spacemacs"></a></p>

<p>And these are the steps I followed to get to the screenshot.* </p>

<ol>
<li><strong>Open file</strong> playground.py on Emacs.</li>
<li>Press <code>C-c C-p</code> to <strong>fire up an iPython process and connect file to it</strong>. Note that the iPython process is fully functional. I can call all magic methods on it.</li>
<li>Press <code>M-m m d b</code> to <strong>include the breakpoint</strong> in the line I want (in form of a pdb import)</li>
<li>Press <code>C-M-x</code> to <strong>send the definition to the iPython prompt</strong>. Note in the screenshot that I am calling <code>unique_everseen</code> without ever having typed it into the prompt. The definition was sent directly from the file (that is why we have an empty cell <a href="http://i.stack.imgur.com/17GSY.png" rel="nofollow">2</a>, as a visual feedback that Emacs did something).</li>
<li><strong>Execute the function from iPython prompt so as to trigger the breakpoint.</strong> I get information telling me what line I am and enter the debugger automatically.</li>
<li>I <strong>inspect variables</strong> to see what is going on.</li>
<li>I press <code>n</code> to <strong>step to the next</strong> instruction.</li>
<li>I click on the filename (displayed in red on the prompt) and it takes my cursor directly to that line within the playground.py file. Note the black triangles next to the line numbers telling me where the stepper is in. Pretty nice.</li>
</ol>

<p>Now, <em>I haven't even scratched the surface</em> of what Emacs can offer.
If this interests you I'd suggest looking into it. It's really powerful and has a great, very helpful and active community.</p>

<p>To get all Python configuration running out of the box without having to configure anything yourself simply install <a href="https://github.com/syl20bnr/spacemacs" rel="nofollow">Spacemacs</a>, a popular distribution of Emacs (what anaconda is to Python, Spacemacs is to Emacs). The install instructions can be found under the link. </p>

<p><strong>Using Emacs to edit iPython notebooks.</strong></p>

<p>A package called EIN (Emacs iPython Notebooks) "provides a IPython Notebook client and integrated REPL in Emacs." 
To get it working you open your iPython notebook server from any terminal with <code>ipython notebook</code>, and call the function <code>ein:notebooklist</code>.</p>

<p>More information can be found under the project's page <a href="https://tkf.github.io/emacs-ipython-notebook/" rel="nofollow">here</a>, and on the Spacemacs iPython-Notebook layer page <a href="https://github.com/syl20bnr/spacemacs/tree/master/layers/%2Blang/ipython-notebook" rel="nofollow">here</a>.</p>

<p>In particular, from EIN documentation:</p>

<ol>
<li>Copy/paste cells, even to/from different notebooks.</li>
<li>Console integration: You can easily connect to a kernel via the console application. This enables you to start debugging in the same kernel. It is even possible to connect to a console over ssh. </li>
<li>An IPython kernel can be "connected" to any Emacs buffer. This enables you to evaluate a buffer or buffer region using the same kernel as the notebook. Notebook goodies such as tooltip help, help browser and code completion are available in these buffers.</li>
<li>Jump to definition (go to the definition by hitting M-. over an object).</li>
</ol>

<p>All functionality described above for the simple iPython + file workflow is available for iPython notebooks (*.ipynb).</p>

<p>Screenshots <a href="https://github.com/millejoh/emacs-ipython-notebook/wiki/Screenshots" rel="nofollow">here.</a></p>

python debugging ipython jupyter-notebook spyder