Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
382 views
in Technique[技术] by (71.8m points)

python - How to append to .mat file using scipy.io.savemat?

so when I use the savemat command it tends to overwrite the file. Is there a possible way to append instead of overwriting? I know a work-around would be to put everything into a list, and then convert it to a dictionary. That won't work for me because I am trying to be RAM efficient. Doing online search I found this How NOT to overwrite the .mat file when using scipy.io.savemat()?

that won't work either because it involves pulling a data file in your ram memory to append it every single loop, which seems stupid from the speed perspective.

I thought about appending to a numpy binary file, then pulling that in and saving it to a .mat file? I am not sure if this would be more RAM efficient than the first option though.

thanks!

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

According to savemat docs:

file_name : str or file-like object

So you can open the file in append mode, and write, e.g.

io.savemat('temp.mat',{'data':np.ones(10)})  # write
with open('temp.mat','ab') as f:
    io.savemat(f, {'newdata':np.arange(5)})   # append
print io.loadmat('temp.mat').keys()           # read
# dict_keys(['data', '__globals__', 'newdata', '__header__', '__version__'])

Or the full description:

{'data': array([[ 1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.]]),
 '__globals__': [],
 'newdata': array([[0, 1, 2, 3, 4]]),
 '__header__': b'MATLAB 5.0 MAT-file Platform: posix, Created on: Fri Mar 13 14:14:33 2015',
 '__version__': '1.0'}

A note in https://github.com/scipy/scipy/blob/master/scipy/io/matlab/mio5.py#L34 suggests that there is a problem with appending when there's a function in the data file, but this indicates that appending isn't a problem if we are just saving arrays. But maybe further search of the scipy issues is in order.


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...