tensorflow - How can I properly train a model to predict a moving average using LSTM in keras?

Question

I'm learning how to train RNN model on Keras and I was expecting that training a model to predict the Moving Average of the last N steps would be quite easy.

I have a time series with thousands of steps and I'm able to create a model and train it with batches of data.

If I train it with the following model though, the test set predictions differ a lot from real values. (batch = 30, moving average window = 10)

inputs = tf.keras.Input(shape=(batch_length, num_features))
x = tf.keras.layers.LSTM(10, return_sequences=False)(inputs)
outputs = tf.keras.layers.Dense(num_labels)(x)
model = tf.keras.Model(inputs=inputs, outputs=outputs, name="test_model") 

To be able to get good predictions, I need to add another layer of TimeDistributed, getting 2D predictions instead of 1D ones (I get one prediction per each time step)

inputs = tf.keras.Input(shape=(batch_length, num_features))
x = tf.keras.layers.LSTM(10, return_sequences=True)(inputs)
x = tf.keras.layers.TimeDistributed(tf.keras.layers.Dense(num_labels))(x)
outputs = tf.keras.layers.Dense(num_labels)(x)
model = tf.keras.Model(inputs=inputs, outputs=outputs, name="test_model")

question from:https://stackoverflow.com/questions/65862277/how-can-i-properly-train-a-model-to-predict-a-moving-average-using-lstm-in-keras

Answer 1

I suggest that if your goal is to give as input the last 10 timesteps and have as a prediction the moving average to try a regressor model with Densely Connected layers rather than an RNN. (Linear activation with regularization might work well enough)

That option would be cheaper to train and run than an LSTM