I have a gzip
which is approximately 90 GB. This is well within disk space, but far larger than RAM.
How can I import this into a pandas dataframe? I tried the following in the command line:
# start with Python 3.4.5
import pandas as pd
filename = 'filename.gzip' # size 90 GB
df = read_table(filename, compression='gzip')
However, after several minutes, Python shuts down with Kill 9
.
After defining the database object df
, I was planning to save it into HDF5.
What is the correct way to do this? How can I use pandas.read_table()
to do this?
See Question&Answers more detail:
os 与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…