My boss asked a great question at our first weekly ESO-python tutorial
session: What does a good ipython debugging workflow look like?
The one advantage I had found in IDL was that everything is a script, which
means that everything can be debugged in the same way: add a stop statement
at the relevant line of code. Of course, that debugging model breaks apart
badly once you start writing complex programs and using things like common
blocks.
In ipython, there is a beautiful debugger that is far more feature-rich than
the IDL equivalent. It can be activated simply:
will toggle the interactive ipython debugger. Then, if you run into a code
crash, you will be dumped out at an ipdb> prompt, with access to a
fully-functional python prompt in the local namespace / environment of the
crash point. You can also move up and down the function hierarchy with the u
and d commands.
The existence of the ipdb debugger suggests a natural approach for those
familiar with IDL debugging: simply add a raise statement wherever you would
normally have put a stop statement. There are a few key differences,
however: in IDL, you could just .continue to run the rest of the code as if
no stop occurred; in python the same is not possible with the default ipdb.
However, the ipdb package allows the insertion of breakpoints just like in IDL:
import ipdb; ipdb.set_trace()
From within one of these breakpoints, the .continue command will continue on
to the next breakpoint, as you might expect. The n and s commands will go
to the next line in the code, while s will allow you to drop into called functions (how?)
Postmortem debugging is also awesome. If you have run a command and gotten a traceback,
you can retroactively enter the debugger:
but as with the normal traceback debugger, one cannot continue afterward.
Reloading Code
Python code is typically distributed in packages managed with a setup.py
script. These are the most convenient way to install, use, and distribute
code, but they are not ideal for debugging.
When debugging a normal script, something you could run by invoking
or
%run myfile.py
# in the local namespace
%run -i myfile.py
# with the python debugger active:
%run -d myfile.py
can easily be debugged by running it line-by-line. The documentation
can be accessed via the %magic command.
For debugging "compiled" packages, though, more thought is needed.
If you are working on your own package, you can use setuptools to enable the
python setup.py develop command, which installs a symbolic link to the
source code directory - meaning any changes you make are immediately reflected
in the python source path. This does not mean that any changes are
recognized in the local python environment, though!
If you are in an active ipython session, you need to reload packages to
see their results. As far as I know, you can never "reload" the source code
underlying an already-instantiated class, so you have to remake any class
instances you want to examine.
The reload command is tricky to use. You will only see changes to the source code
if you import the exact package in which the code is stored. For example, if you have a file structure like this:
mypackage/
__init__.py
core.py
and you do reload(mypackage), that will effectively reload only the source
code in __init__.py. If the code you want to use is called myfunction
and it lives in core.py, you can reload that source code by doing
reload(mypackage.core). Reloading mypackage may have no effect. So
the key for developing packages is finding the right module to reload!
IPython has a deepreload package
intended to recursively reload functions; it may work but I had trouble in the
past.
Tests
It's generally much better to have a test suite enabled with unit tests for
each component of your code. This is the approach adopted by most open-source
projects and many industrial code developers.
Tests are invoked with the command py.test with a variety of command
options. Among my favorites are py.test --tb=short, which gives a much
less verbose traceback, py.test --pastebin=failed, which posts any failure
results to pastebin for easy sharing, py.test -p packagename (or python
setup.py test -P packagename in astropy) to select tests for a specific
package. A great deal more options can be found in the pytest docs.
A test suite, if properly constructed, can also be run from with in ipython:
import astroquery
astroquery.test('eso')
Postscript
Some useful related links:
