The traceback is:
Traceback (most recent call last):
File "stack26739733.py", line 30, in <module>
datarray = rf.append_fields(datarray, 'DateTime', data=mydatetime, usemask=False, dtypes=mydatetime.dtype)
File "/usr/local/lib/python2.7/site-packages/numpy/lib/recfunctions.py", line 641, in append_fields
dtype=base.dtype.descr + data.dtype.descr)
File "/usr/local/lib/python2.7/site-packages/numpy/ma/extras.py", line 163, in masked_all
mask=np.ones(shape, make_mask_descr(dtype)))
File "/usr/local/lib/python2.7/site-packages/numpy/ma/core.py", line 2644, in __new__
_data = ndarray.view(_data, cls)
File "/usr/local/lib/python2.7/site-packages/numpy/ma/core.py", line 2800, in __array_finalize__
self._fill_value = _check_fill_value(None, self.dtype)
File "/usr/local/lib/python2.7/site-packages/numpy/ma/core.py", line 402, in _check_fill_value
dtype=ndtype,)
ValueError: Error parsing datetime string "?" at position 0
So this append function constructs a masked array (ma
), and checks the 'fill_value' for the appended 'dtype'. Apparently _check_fill_value
doesn't understand the datetime
dtype. Looks like an incompatibility between masked array and datetime. A numpy
bug report might be in order.
Here's a simple, do-it-yourself append:
dt1 = np.dtype(datarray.dtype.descr + mydatetime.dtype.descr)
newarray = np.empty(datarray.shape, dtype=dt1)
for n in datarray.dtype.names:
newarray[n] = datarray[n]
newarray['f3'] = mydatetime
I construct an empty array with a union dtype. Then I copy the data from both datarray
and mydatetime
field by field. Since the number of fields is normally quite small compared to the shape
, this copy is quite fast. I'm pretty sure the rf
function does the same.
'f3'
is the default name of the added field. You can change that when creating dt1
.
The result is:
array([(0, 0, 0, datetime.datetime(1990, 1, 1, 0, 0)),
(1, 1, 1, datetime.datetime(1990, 1, 1, 0, 0, 1, 1000)),
(2, 2, 2, datetime.datetime(1990, 1, 1, 0, 0, 2, 2000)),
...
(9, 9, 9, datetime.datetime(1990, 1, 1, 0, 0, 9, 9000))],
dtype=[('a', '<i4'), ('b', '<i4'), ('c', '<i4'), ('f3', '<M8[ms]')])
Turning this newarray
into a masked array produces the same _check_fill_value
error.
np.ma.masked_array(newarray)