python - Numpy savetxt method produces strange hex characters

Question

I am have some strange issue with the numpy package in Python 3.8.6. I am trying to write numpy arrays of float64, but sometimes it writes instead hexadecimal characters to the file: x00x00...

Here is the code I am running:

#!/usr/bin/env python
# coding: utf-8
import numpy as np
from pathlib import Path
import shutil as sh

resdir = Path('./results/')
if not resdir.is_dir():
    resdir.mkdir(parents=True)

trajdir = resdir / 'traj'
if trajdir.is_dir():
  sh.rmtree(trajdir)
trajdir.mkdir()

phi = np.zeros([2**12,2], dtype=np.float_)
itermax = 100
idump_fmt = '{:03d}'
# loop
# dump
for i in range(1, itermax):
    print("i = {:d}".format(i))
    fname = 'i' + idump_fmt.format(i) + '.dat'
    np.savetxt(trajdir / fname, np.zeros(phi.shape, dtype=phi.dtype))
    np.loadtxt(trajdir / fname).T

I am running this code from within a docker container. I have a docker image based on the Dockerfile below, and I am running the command:

docker run --rm -it -v $PWD:/home/python/shared -w /home/python/shared savetxt-test:latest python test_savetxt.py

where savetxt-test:latest is the name of my docker image and test_savetxt.py is the file containing the above code.

Not always, but maybe once every 2-3 times, the above command produces the following error:

Traceback (most recent call last):
 36   File "test_savetxt.py", line 25, in <module>
 37     np.loadtxt(trajdir / fname).T
 38   File "/usr/local/lib/python3.8/site-packages/numpy/lib/npyio.py", line 1139, in loadtxt
 39     for x in read_data(_loadtxt_chunksize):
 40   File "/usr/local/lib/python3.8/site-packages/numpy/lib/npyio.py", line 1067, in read_data
 41     items = [conv(val) for (conv, val) in zip(converters, vals)]
 42   File "/usr/local/lib/python3.8/site-packages/numpy/lib/npyio.py", line 1067, in <listcomp>
 43     items = [conv(val) for (conv, val) in zip(converters, vals)]
 44   File "/usr/local/lib/python3.8/site-packages/numpy/lib/npyio.py", line 763, in floatconv
 45     return float(x)
 46 ValueError: could not convert string to float: 'x00x00x00x00x00x00x[...]

I am using Mac OS 11.1, and when I run this code directly in my system (not through a docker container) I don't experience this problem. I am suspecting that this could be related to a mounting issue with docker, but I am not sure how to get to the bottom of this. Any idea?

Dockerfile:

FROM python:3.8-slim-buster

########## preconfig ##########
USER root
ENV DEBIAN_FRONTEND=noninteractive
RUN export DEBIAN_FRONTEND

RUN apt-get update 
    && apt-get -y install libssl-dev 
    && apt-get clean 
    && pip install wheel

RUN pip install --upgrade pip

# Install dependencies
RUN apt-get update && apt-get install -y less

########## python ##########
RUN pip install numpy

########################
#add new sudo user
RUN apt-get install -y sudo
ENV USERNAME python
ENV GROUPNAME python
ENV UID 1000
ENV GID 1000
RUN groupadd --gid $GID $GROUPNAME && 
     useradd -g $GROUPNAME -u $UID -m $USERNAME && 
     echo "$USERNAME:$USERNAME" | chpasswd && 
     usermod --shell /bin/bash $USERNAME && 
     usermod -aG sudo $USERNAME && 
     echo "$USERNAME ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers.d/$USERNAME && 
     chmod 0440 /etc/sudoers.d/$USERNAME

USER $USERNAME

########## docker behavior ##########
CMD /bin/bash

question from:https://stackoverflow.com/questions/65852398/numpy-savetxt-method-produces-strange-hex-characters

Answer 1

Actually it turns out this was caused by the iCloud Drive syncing. Once turned off, I no longer had this issue.