python - MemoryError When Resize Mnist data set images

Question

I am new on deep learning.I am trying to change mnist images from 28*28 into 224 * 224.

So I decided to use resize method. After importing MNIST dataset I try to resized it:

(X_train, y_train), (X_test, y_test) = mnist.load_data()

x_train_small = tf.image.resize(X_train, (224,224)).numpy() 

But I got this error

MemoryError: Unable to allocate 11.2 GiB for an array with shape (60000, 224, 224, 1) and data type float32

My computer is old and I have just 16gig ram. How can I resize Mnist data set ?

question from:https://stackoverflow.com/questions/66050659/memoryerror-when-resize-mnist-data-set-images

Answer 1

Consider using a tf.data.Dataset and resize images on the fly, as batches pass:

import tensorflow as tf

(X_train, y_train), (X_test, y_test) = tf.keras.datasets.mnist.load_data()

resize = lambda x, y: (tf.image.resize(tf.expand_dims(x, -1), (224, 224)), y)

train_ds = tf.data.Dataset.from_tensor_slices((X_train, y_train)).map(resize)

for image, label in train_ds.take(5):
    print(image.shape)

(224, 224, 1)
(224, 224, 1)
(224, 224, 1)
(224, 224, 1)
(224, 224, 1)

You can pass this dataset directly to model.fit(train_ds)