a life of coding

Monday, November 05, 2007

Debugging Python

Above all else, my greatest annoyance with python is the lack of good documentation and defaults. I bet that there is a group somewhere that knows everything there is to know about python... I beg of them, write that knowledge down!

Here's an example. I want to debug a faulty program:
x = [1,2,3]

def foo():
sum = 0
for i in range(4):
sum += x[i]

import pdb
I'm then prompted by pdb: (> replaced with ])
] [string](1)?()
Good, now I'm in the debugger. To get things started, I type "c", return, and get:
Traceback (most recent call last):
File "[stdin]", line 1, in ?
File "c:\Python24\lib\pdb.py", line 996, in run
Pdb().run(statement, globals, locals)
File "c:\Python24\lib\bdb.py", line 366, in run
exec cmd in globals, locals
File "[string]", line 1, in ?
File "[stdin]", line 4, in foo
IndexError: list index out of range
You should notice that the last line is the standard python prompt (with the aforementioned character replacement to make blogger behave), not the debugger. I have finished my debugging session due to an error. Gee, it sure would have been nice to debug that, since my state is now lost! I was already using the debugger to run this code, why did it not catch this exception?

It turns out that exception handling is done by sys.excepthook, and pdb.run doesn't set the excepthook. Some searching turned up two options. The first is simple but crude - add the following to site-packages/sitecustomize.py:

Thomas Heller
import pdb, sys, traceback
def info(type, value, tb):
traceback.print_exception(type, value, tb)
sys.excepthook = info
The second is more sophisticated, and checks for interactive mode:

ActiveState Python Cookbook.
# code snippet, to be included in 'sitecustomize.py'
import sys

def info(type, value, tb):
if hasattr(sys, 'ps1') or not sys.stderr.isatty():
# we are in interactive mode or we don't have a tty-like
# device, so we call the default hook
sys.__excepthook__(type, value, tb)
import traceback, pdb
# we are NOT in interactive mode, print the exception...
traceback.print_exception(type, value, tb)
# ...then start the debugger in post-mortem mode.

sys.excepthook = info
Here's what I'm currently running:
# code snippet, to be included in 'sitecustomize.py'
import sys

def info(type, value, tb):
if (#hasattr(sys, "ps1") or
not sys.stderr.isatty() or
not sys.stdin.isatty()):
# stdin or stderr is redirected, just do the normal thing
original_hook(type, value, tb)
# a terminal is attached and stderr is not redirected, debug
import traceback, pdb
traceback.print_exception(type, value, tb)

original_hook = sys.excepthook
if sys.excepthook == sys.__excepthook__:
# if someone already patched excepthook, let them win
sys.excepthook = info
The original ActiveState script doesn't debug if you are running in interactive mode. To me, this makes no sense at all - thats a case where I specifically want to debug.

Alas, if the debugger shows an obvious error that could be fixed, python exceptions cannot be resumed. The code listed here will let you see the stack and move around in it, but your program is no longer running and can never finish where it left off. Allowing this requires call/cc, and as far as I can tell there is no plan to ever support that in python.

If this was helpful, or should be changed, let me know. I've just started using it myself.


  • Have you considered Iron Python? Despite some of my objections to Visual Studio, the debugger is fairly good. I haven't actually tried it, but it seems like it might be worth a look, in lieu of fixing the actual Python debugger.

    By Blogger peterp, At 11/6/07 6:05 PM  

  • When I think of lightweight, embeddable, high level languages, somehow a version of python written on top of the CLR just don't come to mind. This is really something that the python community should address, but they don't seem to care much about it. Ultimately it looks like we'll be using a lot more JavaScript, which has a lot better debugging support.

    By Blogger ynniv, At 11/6/07 8:05 PM  

  • You can also try overriding pdb to stop on exceptions at the offending line. Very briefly:
    import pdb
    class MyPdb(pdb.Pdb):
    def dispatch_exception(self, frame, arg):
    self.user_exception(frame, arg)
    if self.quitting: raise BdbQuit
    return self.trace_dispatch

    python faultyprogram.py
    ] c
    IndexError: 'list index out of range'
    ] where
    -> faulty.py
    -> sum += x[i]
    ] print i
    Of course, you could put this in a private pdb on your PYTHONPATH.

    By Anonymous GregFrascadore, At 1/24/09 7:35 PM  

Post a Comment


Create a Link

<< Home