Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
769 views
in Technique[技术] by (71.8m points)

python csv reader ignore blank row

Im using the pythons csv reader . How can I use the following code in such a way that it ignores blank lines.

import csv
f1 = open ("ted.csv")
oldFile1 = csv.reader(f1, delimiter=',', quotechar='"')
oldList1 = list(oldFile1)
f2 = open ("ted2.csv")
newFile2 = csv.reader(f2, delimiter=',', quotechar='"')
newList2 = list(newFile2)

f1.close()
f2.close()

with open("ted.csv") as f1, open("ted2.csv") as f2, open('foo.csv', 'w') as out:
     r1, r2 = csv.reader(f1), csv.reader(f2)
     st = set((row[0], row[3]) for row in r1)
     wr = csv.writer(out)
     for row in (row for row in r2 if (row[0],row[3]) not in st):
           wr.writerow(row)
See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

If your blanks are always on the first line, then Marius' answer is simplest. If you have n blanks at the beginning or you just want to skip some number of lines you can use itertools.islice().

Skip first N lines

Suppose you want to skip over the first 4 lines (blank lines or not):

from itertools import islice
with open('csv2.csv', 'r') as f1, open('out.csv', 'w') as out:
    filt_f1 = islice(f1, 4, None)
    r1 = csv.reader(filt_f1)
    wr = csv.writer(out)
    for line in r1:
        ...

Blank lines throughout

If you have blank lines scattered throughout your files then you can filter them out with itertools.filterfalse.

import csv
from itertools import filterfalse
from itertools import chain

with open('csv1.csv', 'r') as f1, open('csv2.csv', 'r') as f2, open('out.csv', 'w') as out:
    # create an iterator without lines that start with '
'
    filt_f1 = filterfalse(lambda line: line.startswith('
'), f1)
    filt_f2 = filterfalse(lambda line: line.startswith('
'), f2)

    # csv.reader consumes the filtered iterators
    r1, r2 = csv.reader(filt_f1), csv.reader(filt_f2)
    wr = csv.writer(out)

    # here insert your logic, I just write both to the same file
    for line in chain(r1, r2):
        wr.writerow(line)

Where csv1.csv is:

time,name,location
12345,Jean,Montreal

12346,Peter,Chicago

1234589,Doug,Boston

and csv2.csv (note: not shown here, but csv2.csv has 4 blank lines at the top of the file):

123457,Scott,San Diego

123458,Jen,Miami

123459,Robert,Sacramento

output out.csv does not have blank lines throughout:

time,name,location
12345,Jean,Montreal
12346,Peter,Chicago
1234589,Doug,Boston
123457,Scott,San Diego
123458,Jen,Miami
123459,Robert,Sacramento

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...