I am trying to ptrepack a HDF file that was created with pandas HDFStore pytables interface.
The main index of the dataframe was time but I made some more columns data_columns
so that I can filter for data on-disk via these data_columns.
Now I would like to sort the HDF file by one of those columns (because the selection is too slow for my taste, 84 GB file), using ptrepack with the sortby
option like so:
()[maye@luna4 .../nominal]$ ptrepack --chunkshape=auto --propindexes --complevel=9 --complib=blosc --sortby=clat C9.h5 C9_sorted.h5
and I get the error message:
()[maye@luna4 .../nominal]$ Problems doing the copy from 'C9.h5:/' to
'C9_sorted.h5:/' The error was --> :
Field clat
must have associated a 'full' index in table /df/table
(Table(390557601,)) ''
. The destination file looks like: C9_sorted.h5
(File) '' Last modif.: 'Fri Jul 26 18:17:56 2013' Object Tree: /
(RootGroup) '' /df (Group) '' /df/table (Table(0,), shuffle, blosc(9))
''
Traceback (most recent call last): File
"/usr/local/epd/bin/ptrepack", line 10, in
sys.exit(main()) File "/usr/local/epd/lib/python2.7/site-packages/tables/scripts/ptrepack.py",
line 480, in main
upgradeflavors=upgradeflavors) File "/usr/local/epd/lib/python2.7/site-packages/tables/scripts/ptrepack.py",
line 225, in copyChildren
raise RuntimeError("Please check that the node names are not " RuntimeError: Please check that the node names are not duplicated in
destination, and if so, add the --overwrite-nodes flag if desired. In
particular, pay attention that rootUEP is not fooling you.
Does this mean, that I can not sort a HDF file by an index column, because they are not 'full' indexes?
See Question&Answers more detail:
os 与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…