This is a string I'm getting out of pandas.DataFrame.to_json()
, putting it into redis, getting it out of redis elsewhere, and trying to read it via pandas.read_json()
:
DFJ {"args":{"0":"[]","1":"[]","2":"[]","3":"[]","4":"[]","5":"[]","6":"[]","7":"[]"},"date":{"0":1385944439000000000,"1":1385944439000000000,"2":1385944440000000000,"3":1385944440000000000,"4":1385944440000000000,"5":1385944440000000000,"6":1385944440000000000,"7":1385944440000000000},"host":{"0":"yy38.segm1.org","1":"kyy1.segm1.org","2":"yy10.segm1.org","3":"yy24.segm1.org","4":"yy24.segm1.org","5":"yy34.segm1.org","6":"yy15.segm1.org","7":"yy15.segm1.org"},"kwargs":{"0":"{}","1":"{}","2":"{}","3":"{}","4":"{}","5":"{}","6":"{}","7":"{}"},"operation":{"0":"x_gbinf","1":"x_initobj","2":"x_gobjParams","3":"gtfull","4":"x_gbinf","5":"gxyzinf","6":"deletemfg","7":"gxyzinf"},"thingy":{"0":"a13yy38","1":"a19kyy1","2":"a14yy10","3":"a14yy24","4":"a14yy24","5":"a12yy34","6":"a15yy15","7":"a15yy15"},"status":{"0":-101,"1":1,"2":-101,"3":-101,"4":-101,"5":-101,"6":1,"7":-101},"time":{"0":0.000801,"1":0.003244,"2":0.002247,"3":0.002787,"4":0.001067,"5":0.002652,"6":0.004371,"7":0.000602}}
It seems like it does not have any unicode in it. Yet on trying to .read_json()
it I get:
Traceback (most recent call last):
File "./sqlprofile.py", line 160, in <module>
maybe_save_dataframes(rconn, configd, results)
File "./sqlprofile.py", line 140, in maybe_save_dataframes
h5store.append(out_queue, df)
File "/home/username/anaconda/lib/python2.7/site-packages/pandas/io/pytables.py", line 658, in append
self._write_to_group(key, value, table=True, append=True, **kwargs)
File "/home/username/anaconda/lib/python2.7/site-packages/pandas/io/pytables.py", line 923, in _write_to_group
s.write(obj = value, append=append, complib=complib, **kwargs)
File "/home/username/anaconda/lib/python2.7/site-packages/pandas/io/pytables.py", line 2985, in write
**kwargs)
File "/home/username/anaconda/lib/python2.7/site-packages/pandas/io/pytables.py", line 2717, in create_axes
raise e
TypeError: [unicode] is not implemented as a table column
> /home/username/anaconda/lib/python2.7/site-packages/pandas/io/pytables.py(2717)create_axes()
-> raise e
(Pdb) locals()
This is what I'm getting in locals()
- it seems that append_axis
(column names?) values are unicode. Why?
{'append_axis': [u'args', u'date', u'host', u'kwargs', u'operation', u'thingy', u'status', u'time'], 'existing_table': None, 'blocks': [FloatBlock: [time], 1 x 8, dtype float64, ObjectBlock: [args, host, kwargs, operation, thingy], 5 x 8, dtype object, IntBlock: [status], 1 x 8, dtype int64, DatetimeBlock: [date], 1 x 8, dtype datetime64[ns]], 'axis': 1, 'self': frame_table (typ->appendable,nrows->None,ncols->1,indexers->[index]), 'axes': [0], 'kwargs': {}, 'klass': <class 'pandas.io.pytables.DataCol'>, 'block_obj': args date host kwargs operation thingy status time
0 [] 2013-12-02 00:33:59 yy38.segm1.org {} x_gbinf a13yy38 -101 0.000801
1 [] 2013-12-02 00:33:59 kyy1.segm1.org {} x_initobj a19kyy1 1 0.003244
2 [] 2013-12-02 00:34:00 yy10.segm1.org {} x_gobjParams a14yy10 -101 0.002247
3 [] 2013-12-02 00:34:00 yy24.segm1.org {} gtfull a14yy24 -101 0.002787
4 [] 2013-12-02 00:34:00 yy24.segm1.org {} x_gbinf a14yy24 -101 0.001067
5 [] 2013-12-02 00:34:00 yy34.segm1.org {} gxyzinf a12yy34 -101 0.002652
6 [] 2013-12-02 00:34:00 yy15.segm1.org {} deletemfg a15yy15 1 0.004371
7 [] 2013-12-02 00:34:00 yy15.segm1.org {} gxyzinf a15yy15 -101 0.000602, 'axis_labels': [u'args', u'date', u'host', u'kwargs', u'operation', u'thingy', u'status', u'time'], 'nan_rep': 'nan', 'data_columns': [], 'obj': args date host kwargs operation thingy status time
0 [] 2013-12-02 00:33:59 yy38.segm1.org {} x_gbinf a13yy38 -101 0.000801
1 [] 2013-12-02 00:33:59 kyy1.segm1.org {} x_initobj a19kyy1 1 0.003244
2 [] 2013-12-02 00:34:00 yy10.segm1.org {} x_gobjParams a14yy10 -101 0.002247
3 [] 2013-12-02 00:34:00 yy24.segm1.org {} gtfull a14yy24 -101 0.002787
4 [] 2013-12-02 00:34:00 yy24.segm1.org {} x_gbinf a14yy24 -101 0.001067
5 [] 2013-12-02 00:34:00 yy34.segm1.org {} gxyzinf a12yy34 -101 0.002652
6 [] 2013-12-02 00:34:00 yy15.segm1.org {} deletemfg a15yy15 1 0.004371
7 [] 2013-12-02 00:34:00 yy15.segm1.org {} gxyzinf a15yy15 -101 0.000602, 'validate': True, 'a': (1, [u'args', u'date', u'host', u'kwargs', u'operation', u'thingy', u'status', u'time']), 'index_axes_map': {0: name->index,cname->index,axis->0,pos->0,kind->integer}, 'b': ObjectBlock: [args, host, kwargs, operation, thingy], 5 x 8, dtype object, 'e': TypeError('[unicode] is not implemented as a table column',), 'name': None, 'existing_col': None, 'j': 2, 'i': 1, 'min_itemsize': None, 'col': name->values_block_1,cname->values_block_1,dtype->None,shape->None}
How can I fix that? Is this a bug in Pandas / pytables?
Environment:
Python 2.7
pandas==0.12.0
tables==3.0.0
See Question&Answers more detail:
os