for loop - python looping through input file

Question

My question is related to file-input in Python, using open(). I have a text file mytext.txt with 3 lines. I am trying to do two things with this file: print the lines, and print the number of lines.

I tried the following code:

input_file = open('mytext.txt', 'r')
count_lines = 0
for line in input_file:
    print line
for line in input_file:
    count_lines += 1
print 'number of lines:', count_lines

Result: it prints the 3 lines correctly, but prints "number of lines: 0" (instead of 3)

---

I found two ways to solve it, and get it to print 3:

1) I use one loop instead of two

input_file = open('mytext.txt', 'r')
count_lines = 0
for line in input_file:
    print line
    count_lines += 1
print 'number of lines:', count_lines

2) after the first loop, I define input_file again

input_file = open('mytext.txt', 'r')
count_lines = 0
for line in input_file:
    print line
input_file = open('mytext.txt', 'r')
for line in input_file:
    count_lines += 1
print 'number of lines:', count_lines

---

To me, it seems like the definition input_file = ... is valid for only one looping, as if it was deleted after I use it for a loop. But I don't understand why, probably it is not 100% clear to me yet, how variable = open(filename) treated in Python.

By the way, I see that in this case it is better to use only one loop. However, I feel I have to get this question clear, since there might be cases when I can/must make use of it.

Answer 1

The file handle is an iterator. After iterating over the file, the pointer will be positioned at EOF (end of file) and the iterator will raise StopIteration which exits the loop. If you try to use an iterator for a file where the pointer is at EOF it will just raise StopIteration and exit: that is why it counts zero in the second loop. You can rewind the file pointer with input_file.seek(0) without reopening it.

That said, counting lines in the same loop is more I/O efficient, otherwise you have to read the whole file from disk a second time just to count the lines. This is a very common pattern:

with open('filename.ext') as input_file:
    for i, line in enumerate(input_file):
        print line,
print "{0} line(s) printed".format(i+1)

In Python 2.5, the file object has been equipped with __enter__ and __exit__ to address the with statement interface. This is syntactic sugar for something like:

input_file = open('filename.txt')
try:
    for i, line in enumerate(input_file):
        print line,
finally:
    input_file.close()
print "{0} line(s) printed".format(i+1)

I think cPython will close file handles when they get garbage collected, but I'm not sure this holds true for every implementation - IMHO it is better practice to explicitly close resource handles.