Learning with Python 2nd Edition documentation (2023)

Different kinds of errors can occur in a program, and it is useful todistinguish among them in order to track them down more quickly:

  1. Syntax errors are produced by Python when it is translating the source codeinto byte code. They usually indicate that there is something wrong with thesyntax of the program. Example: Omitting the colon at the end of a defstatement yields the somewhat redundant message SyntaxError: invalidsyntax.
  2. Runtime errors are produced by the runtime system if something goes wrongwhile the program is running. Most runtime error messages includeinformation about where the error occurred and what functions wereexecuting. Example: An infinite recursion eventually causes a runtime errorof maximum recursion depth exceeded.
  3. Semantic errors are problems with a program that compiles and runs butdoesn’t do the right thing. Example: An expression may not be evaluated inthe order you expect, yielding an unexpected result.

The first step in debugging is to figure out which kind of error you aredealing with. Although the following sections are organized by error type, sometechniques are applicable in more than one situation.

Syntax errors

Syntax errors are usually easy to fix once you figure out what they are.Unfortunately, the error messages are often not helpful. The most commonmessages are SyntaxError: invalid syntax and SyntaxError: invalidtoken, neither of which is very informative.

On the other hand, the message does tell you where in the program the problemoccurred. Actually, it tells you where Python noticed a problem, which is notnecessarily where the error is. Sometimes the error is prior to the location ofthe error message, often on the preceding line.

If you are building the program incrementally, you should have a good ideaabout where the error is. It will be in the last line you added.

If you are copying code from a book, start by comparing your code to the book’scode very carefully. Check every character. At the same time, remember that thebook might be wrong, so if you see something that looks like a syntax error, itmight be.

Here are some ways to avoid the most common syntax errors:

  1. Make sure you are not using a Python keyword for a variable name.
  2. Check that you have a colon at the end of the header of every compoundstatement, including for, while, if, and def statements.
  3. Check that indentation is consistent. You may indent with either spaces ortabs but it’s best not to mix them. Each level should be nested the sameamount.
  4. Make sure that any strings in the code have matching quotation marks.
  5. If you have multiline strings with triple quotes (single or double), makesure you have terminated the string properly. An unterminated string maycause an invalid token error at the end of your program, or it may treatthe following part of the program as a string until it comes to the nextstring. In the second case, it might not produce an error message at all!
  6. An unclosed bracket – (, {, or [ – makes Python continue with the nextline as part of the current statement. Generally, an error occurs almostimmediately in the next line.
  7. Check for the classic = instead of == inside a conditional.

If nothing works, move on to the next section...

I can’t get my program to run no matter what I do.

If the compiler says there is an error and you don’t see it, that might bebecause you and the compiler are not looking at the same code. Check yourprogramming environment to make sure that the program you are editing is theone Python is trying to run. If you are not sure, try putting an obvious anddeliberate syntax error at the beginning of the program. Now run (or import) itagain. If the compiler doesn’t find the new error, there is probably somethingwrong with the way your environment is set up.

If this happens, one approach is to start again with a new program like Hello,World!, and make sure you can get a known program to run. Then gradually addthe pieces of the new program to the working one.

Runtime errors

Once your program is syntactically correct, Python can import it and at leaststart running it. What could possibly go wrong?

My program does absolutely nothing.

This problem is most common when your file consists of functions and classesbut does not actually invoke anything to start execution. This may beintentional if you only plan to import this module to supply classes andfunctions.

If it is not intentional, make sure that you are invoking a function to startexecution, or execute one from the interactive prompt. Also see the Flow ofExecution section below.

My program hangs.

If a program stops and seems to be doing nothing, we say it is hanging. Oftenthat means that it is caught in an infinite loop or an infinite recursion.

  1. If there is a particular loop that you suspect is the problem, add aprint statement immediately before the loop that says entering the loopand another immediately after that says exiting the loop.
  2. Run the program. If you get the first message and not the second, you’ve gotan infinite loop. Go to the Infinite Loop section below.
  3. Most of the time, an infinite recursion will cause the program to run for awhile and then produce a RuntimeError: Maximum recursion depth exceedederror. If that happens, go to the Infinite Recursionsection below.
  4. If you are not getting this error but you suspect there is a problem with arecursive method or function, you can still use the techniques in theInfinite Recursion section.
  5. If neither of those steps works, start testing other loops and otherrecursive functions and methods.
  6. If that doesn’t work, then it is possible that you don’t understand the flowof execution in your program. Go to the Flow of Execution section below.

Infinite Loop

If you think you have an infinite loop and you think you know what loop iscausing the problem, add a print statement at the end of the loop thatprints the values of the variables in the condition and the value of thecondition.

For example:

while x > 0 and y < 0: # do something to x # do something to y print "x: ", x print "y: ", y print "condition: ", (x > 0 and y < 0)

Now when you run the program, you will see three lines of output for each timethrough the loop. The last time through the loop, the condition should befalse. If the loop keeps going, you will be able to see the values of xand y, and you might figure out why they are not being updated correctly.

Infinite Recursion

Most of the time, an infinite recursion will cause the program to run for awhile and then produce a Maximum recursion depth exceeded error.

If you suspect that a function or method is causing an infinite recursion,start by checking to make sure that there is a base case. In other words,there should be some condition that will cause the function or method to returnwithout making a recursive invocation. If not, then you need to rethink thealgorithm and identify a base case.

If there is a base case but the program doesn’t seem to be reaching it, add aprint statement at the beginning of the function or method that prints theparameters. Now when you run the program, you will see a few lines of outputevery time the function or method is invoked, and you will see the parameters.If the parameters are not moving toward the base case, you will get some ideasabout why not.

Flow of Execution

If you are not sure how the flow of execution is moving through your program,add print statements to the beginning of each function with a message likeentering function foo, where foo is the name of the function.

Now when you run the program, it will print a trace of each function as it isinvoked.

When I run the program I get an exception.

If something goes wrong during runtime, Python prints a message that includesthe name of the exception, the line of the program where the problem occurred,and a traceback.

The traceback identifies the function that is currently running, and then thefunction that invoked it, and then the function that invoked that, and so on.In other words, it traces the path of function invocations that got you towhere you are. It also includes the line number in your file where each ofthese calls occurs.

The first step is to examine the place in the program where the error occurredand see if you can figure out what happened. These are some of the most commonruntime errors:

You are trying to use a variable that doesn’t exist in the currentenvironment. Remember that local variables are local. You cannot refer tothem from outside the function where they are defined.

There are several possible causes:

  1. You are trying to use a value improperly. Example: indexing astring, list, or tuple with something other than an integer.
  2. There is a mismatch between the items in a format string and theitems passed for conversion. This can happen if either the number ofitems does not match or an invalid conversion is called for.
  3. You are passing the wrong number of arguments to a function ormethod. For methods, look at the method definition and check that thefirst parameter is self. Then look at the method invocation; makesure you are invoking the method on an object with the right type andproviding the other arguments correctly.
You are trying to access an element of a dictionary using a key value thatthe dictionary does not contain.
You are trying to access an attribute or method that does not exist.
The index you are using to access a list, string, or tuple is greater thanits length minus one. Immediately before the site of the error, add aprint statement to display the value of the index and the length of thearray. Is the array the right size? Is the index the right value?

I added so many print statements I get inundated with output.

One of the problems with using print statements for debugging isthat you can end up buried in output. There are two ways to proceed:simplify the output or simplify the program.

To simplify the output, you can remove or comment out printstatements that aren’t helping, or combine them, or format the outputso it is easier to understand.

To simplify the program, there are several things you can do. First,scale down the problem the program is working on. For example, if youare sorting an array, sort a small array. If the program takes inputfrom the user, give it the simplest input that causes the problem.

Second, clean up the program. Remove dead code and reorganize theprogram to make it as easy to read as possible. For example, if yoususpect that the problem is in a deeply nested part of the program,try rewriting that part with simpler structure. If you suspect a largefunction, try splitting it into smaller functions and testing themseparately.

Often the process of finding the minimal test case leads you to thebug. If you find that a program works in one situation but not inanother, that gives you a clue about what is going on.

Similarly, rewriting a piece of code can help you find subtle bugs. Ifyou make a change that you think doesn’t affect the program, and itdoes, that can tip you off.

Semantic errors

In some ways, semantic errors are the hardest to debug, because thecompiler and the runtime system provide no information about what iswrong. Only you know what the program is supposed to do, and only youknow that it isn’t doing it.

The first step is to make a connection between the program text andthe behavior you are seeing. You need a hypothesis about what theprogram is actually doing. One of the things that makes that hard isthat computers run so fast.

You will often wish that you could slow the program down to humanspeed, and with some debuggers you can. But the time it takes toinsert a few well-placed print statements is often short compared tosetting up the debugger, inserting and removing breakpoints, andwalking the program to where the error is occurring.

My program doesn’t work.

You should ask yourself these questions:

  1. Is there something the program was supposed to do but which doesn’tseem to be happening? Find the section of the code that performs thatfunction and make sure it is executing when you think it should.
  2. Is something happening that shouldn’t? Find code in your programthat performs that function and see if it is executing when itshouldn’t.
  3. Is a section of code producing an effect that is not what youexpected? Make sure that you understand the code in question,especially if it involves invocations to functions or methods in otherPython modules. Read the documentation for the functions you invoke.Try them out by writing simple test cases and checking the results.

In order to program, you need to have a mental model of how programswork. If you write a program that doesn’t do what you expect, veryoften the problem is not in the program; it’s in your mental model.

The best way to correct your mental model is to break the program intoits components (usually the functions and methods) and test eachcomponent independently. Once you find the discrepancy between yourmodel and reality, you can solve the problem.

Of course, you should be building and testing components as youdevelop the program. If you encounter a problem, there should be onlya small amount of new code that is not known to be correct.

I’ve got a big hairy expression and it doesn’t do what I expect.

Writing complex expressions is fine as long as they are readable, butthey can be hard to debug. It is often a good idea to break a complexexpression into a series of assignments to temporary variables.

For example:

self.hands[i].addCard (self.hands[self.findNeighbor(i)].popCard())

This can be rewritten as:

neighbor = self.findNeighbor (i)pickedCard = self.hands[neighbor].popCard()self.hands[i].addCard (pickedCard)

The explicit version is easier to read because the variable names provideadditional documentation, and it is easier to debug because you can check thetypes of the intermediate variables and display their values.

Another problem that can occur with big expressions is that the order ofevaluation may not be what you expect. For example, if you are translating theexpression x/2pi into Python, you might write:

y = x / 2 * math.pi;

That is not correct because multiplication and division have the sameprecedence and are evaluated from left to right. So this expression computes(x/2)pi.

A good way to debug expressions is to add parentheses to make the order ofevaluation explicit:

y = x / (2 * math.pi);

Whenever you are not sure of the order of evaluation, use parentheses. Notonly will the program be correct (in the sense of doing what you intended), itwill also be more readable for other people who haven’t memorized the rules ofprecedence.

I’ve got a function or method that doesn’t return what I expect.

If you have a return statement with a complex expression, you don’t have achance to print the return value before returning. Again, you can use atemporary variable. For example, instead of:

return self.hands[i].removeMatches()

you could write:

count = self.hands[i].removeMatches()return count

Now you have the opportunity to display the value of count beforereturning.

I’m really, really stuck and I need help.

First, try getting away from the computer for a few minutes. Computers emitwaves that affect the brain, causing these effects:

  1. Frustration and/or rage.
  2. Superstitious beliefs ( the computer hates me ) and magical thinking ( theprogram only works when I wear my hat backward ).
  3. Random-walk programming (the attempt to program by writing every possibleprogram and choosing the one that does the right thing).

If you find yourself suffering from any of these symptoms, get up and go for awalk. When you are calm, think about the program. What is it doing? What aresome possible causes of that behavior? When was the last time you had a workingprogram, and what did you do next?

Sometimes it just takes time to find a bug. We often find bugs when we are awayfrom the computer and let our minds wander. Some of the best places to findbugs are trains, showers, and in bed, just before you fall asleep.

No, I really need help.

It happens. Even the best programmers occasionally get stuck. Sometimes youwork on a program so long that you can’t see the error. A fresh pair of eyesis just the thing.

Before you bring someone else in, make sure you have exhausted the techniquesdescribed here. Your program should be as simple as possible, and you should beworking on the smallest input that causes the error. You should have printstatements in the appropriate places (and the output they produce should becomprehensible). You should understand the problem well enough to describe itconcisely.

When you bring someone in to help, be sure to give them the information theyneed:

  1. If there is an error message, what is it and what part of the program doesit indicate?
  2. What was the last thing you did before this error occurred? What were thelast lines of code that you wrote, or what is the new test case that fails?
  3. What have you tried so far, and what have you learned?

When you find the bug, take a second to think about what you could have done tofind it faster. Next time you see something similar, you will be able to findthe bug more quickly.

Remember, the goal is not just to make the program work. The goal is to learnhow to make the program work.

© Copyright 2010, Jeffrey Elkner, Allen B. Downey and Chris Meyers. Created using Sphinx 1.2.2.

Top Articles
Latest Posts
Article information

Author: Barbera Armstrong

Last Updated: 01/23/2023

Views: 5906

Rating: 4.9 / 5 (79 voted)

Reviews: 86% of readers found this page helpful

Author information

Name: Barbera Armstrong

Birthday: 1992-09-12

Address: Suite 993 99852 Daugherty Causeway, Ritchiehaven, VT 49630

Phone: +5026838435397

Job: National Engineer

Hobby: Listening to music, Board games, Photography, Ice skating, LARPing, Kite flying, Rugby

Introduction: My name is Barbera Armstrong, I am a lovely, delightful, cooperative, funny, enchanting, vivacious, tender person who loves writing and wants to share my knowledge and understanding with you.