tensorflow - How to average summaries over multiple batches?

Question

Assuming I have a bunch of summaries defined like:

loss = ...
tf.scalar_summary("loss", loss)
# ...
summaries = tf.merge_all_summaries()

I can evaluate the summaries tensor every few steps on the training data and pass the result to a SummaryWriter. The result will be noisy summaries, because they're only computed on one batch.

However, I would like to compute the summaries on the entire validation dataset. Of course, I can't pass the validation dataset as a single batch, because it would be too big. So, I'll get summary outputs for each validation batch.

Is there a way to average those summaries so that it appears as if the summaries have been computed on the entire validation set?

Answer 1

Do the averaging of your measure in Python and create a new Summary object for each mean. Here is what I do:

accuracies = []

# Calculate your measure over as many batches as you need
for batch in validation_set:
  accuracies.append(sess.run([training_op]))

# Take the mean of you measure
accuracy = np.mean(accuracies)

# Create a new Summary object with your measure
summary = tf.Summary()
summary.value.add(tag="%sAccuracy" % prefix, simple_value=accuracy)

# Add it to the Tensorboard summary writer
# Make sure to specify a step parameter to get nice graphs over time
summary_writer.add_summary(summary, global_step)