Debugging Python

Above all else, my greatest annoyance with python is the lack of good documentation and defaults. I bet that there is a group somewhere that knows everything there is to know about python... I beg of them, write that knowledge down!

Here's an example. I want to debug a faulty program:

x = [1,2,3]

def foo():
sum = 0
for i in range(4):
 sum += x[i]

import pdb
pdb.run("foo()")

I'm then prompted by pdb: (> replaced with ])

] [string](1)?()
(Pdb)

Good, now I'm in the debugger. To get things started, I type "c", return, and get:

Traceback (most recent call last):
File "[stdin]", line 1, in ?
File "c:\Python24\lib\pdb.py", line 996, in run
Pdb().run(statement, globals, locals)
File "c:\Python24\lib\bdb.py", line 366, in run
exec cmd in globals, locals
File "[string]", line 1, in ?
File "[stdin]", line 4, in foo
IndexError: list index out of range
]]]

You should notice that the last line is the standard python prompt (with the aforementioned character replacement to make blogger behave), not the debugger. I have finished my debugging session due to an error. Gee, it sure would have been nice to debug that, since my state is now lost! I was already using the debugger to run this code, why did it not catch this exception?

It turns out that exception handling is done by sys.excepthook, and pdb.run doesn't set the excepthook. Some searching turned up two options. The first is simple but crude - add the following to site-packages/sitecustomize.py:

Thomas Heller

import pdb, sys, traceback
def info(type, value, tb):
    traceback.print_exception(type, value, tb)
    pdb.pm()
sys.excepthook = info

The second is more sophisticated, and checks for interactive mode:

ActiveState Python Cookbook.

# code snippet, to be included in 'sitecustomize.py'
import sys

def info(type, value, tb):
   if hasattr(sys, 'ps1') or not sys.stderr.isatty():
      # we are in interactive mode or we don't have a tty-like
      # device, so we call the default hook
      sys.__excepthook__(type, value, tb)
   else:
      import traceback, pdb
      # we are NOT in interactive mode, print the exception...
      traceback.print_exception(type, value, tb)
      print
      # ...then start the debugger in post-mortem mode.
      pdb.pm()

sys.excepthook = info

Here's what I'm currently running:

# code snippet, to be included in 'sitecustomize.py'
import sys

def info(type, value, tb):
   if (#hasattr(sys, "ps1") or
       not sys.stderr.isatty() or 
       not sys.stdin.isatty()):
       # stdin or stderr is redirected, just do the normal thing
       original_hook(type, value, tb)
   else:
       # a terminal is attached and stderr is not redirected, debug 
       import traceback, pdb
       traceback.print_exception(type, value, tb)
       print
       pdb.pm()
       #traceback.print_stack()

original_hook = sys.excepthook
if sys.excepthook == sys.__excepthook__:
    # if someone already patched excepthook, let them win
    sys.excepthook = info

The original ActiveState script doesn't debug if you are running in interactive mode. To me, this makes no sense at all - thats a case where I specifically want to debug.

Alas, if the debugger shows an obvious error that could be fixed, python exceptions cannot be resumed. The code listed here will let you see the stack and move around in it, but your program is no longer running and can never finish where it left off. Allowing this requires call/cc, and as far as I can tell there is no plan to ever support that in python.

If this was helpful, or should be changed, let me know. I've just started using it myself.