python - How do i create Confusion matrix of predicted and ground truth labels with Tensorflow?

Question

I have implemented a Nueral Network model for a classification with the help of using TensorFlow. But, i don't know how can i able to draw confusion matrix by using predicted scores (accuracy). I am not an expert of TensorFlow and still in learning phase. Here i pasted my code below please tell me how can i write a code for making confusion from the following code:

# Launch the graph
with tf.Session() as sess:
sess.run(init)

# Set logs writer into folder /tmp/tensorflow_logs
#summary_writer = tf.train.SummaryWriter('/tmp/tensorflow_logs', graph_def=sess.graph_def)

# Training cycle
for epoch in range(training_epochs):
    avg_cost = 0.
    total_batch = int(X_train.shape[0]/batch_size)

    # Loop over total length of batches
    for i in range(total_batch):  
        #picking up random batches from training set of specific size
        batch_xs, batch_ys = w2v_utils.nextBatch(X_train, y_train, batch_size)
        # Fit training using batch data
        sess.run(optimizer, feed_dict={x: batch_xs, y: batch_ys})
        # Compute average loss
        avg_cost += sess.run(cost, feed_dict={x: batch_xs, y: batch_ys})/total_batch
        # Write logs at every iteration
        #summary_str = sess.run(merged_summary_op, feed_dict={x: batch_xs, y: batch_ys})
        #summary_writer.add_summary(summary_str, epoch*total_batch + i)

    #append loss
    loss_history.append(avg_cost)

    # Display logs per epoch step
    if (epoch % display_step == 0):           
        correct_prediction = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1))            
        # Calculate training  accuracy
        accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))
        trainAccuracy = accuracy.eval({x: X_train, y: y_train})
        train_acc_history.append(trainAccuracy)           
        # Calculate validation  accuracy
        valAccuracy = accuracy.eval({x: X_val, y: y_val})
        val_acc_history.append(valAccuracy) 
        print "Epoch:", '%04d' % (epoch+1), "cost=", "{:.9f}".format(avg_cost), "train=",trainAccuracy,"val=", valAccuracy

print "Optimization Finished!"
# Test model
correct_prediction = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1))
# Calculate accuracy
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))
print "Final Training Accuracy:", accuracy.eval({x: X_train, y: y_train})
print "Final Test Accuracy:", accuracy.eval({x: X_test, y: y_test})
print "Final Gold Accuracy:", accuracy.eval({x: X_gold, y: y_gold})

Up till now, i am able to print predicted scores but failed to implement confusion matrix please help. Note:(I am using one hot vectors for representing my labels)

Answer 1

If you want to produce a confusion matrix, and then later precision and recall, you first need to get your counts of true positives, true negatives, false positives and false negatives. Here is how:

For better readibility, I wrote the code very verbose.

def evaluation(logits,labels):
"Returns correct predictions, and 4 values needed for precision, recall and F1 score"


    # Step 1:
    # Let's create 2 vectors that will contain boolean values, and will describe our labels

    is_label_one = tf.cast(labels, dtype=tf.bool)
    is_label_zero = tf.logical_not(is_label_one)
    # Imagine that labels = [0,1]
    # Then
    # is_label_one = [False,True]
    # is_label_zero = [True,False]

    # Step 2:
    # get the prediction and false prediction vectors. correct_prediction is something that you choose within your model.
    correct_prediction = tf.nn.in_top_k(logits, labels, 1, name="correct_answers")
    false_prediction = tf.logical_not(correct_prediction)

    # Step 3:
    # get the 4 metrics by comparing boolean vectors
    # TRUE POSITIVES
    true_positives = tf.reduce_sum(tf.to_int32(tf.logical_and(correct_prediction,is_label_one)))

    # FALSE POSITIVES
    false_positives = tf.reduce_sum(tf.to_int32(tf.logical_and(false_prediction, is_label_zero)))

    # TRUE NEGATIVES
    true_negatives = tf.reduce_sum(tf.to_int32(tf.logical_and(correct_prediction, is_label_zero)))

    # FALSE NEGATIVES
    false_negatives = tf.reduce_sum(tf.to_int32(tf.logical_and(false_prediction, is_label_one)))


return true_positives, false_positives, true_negatives, false_negatives

# Now you can do something like this in your session:

true_positives, 
false_positives, 
true_negatives, 
false_negatives = sess.run(evaluation(logits,labels), feed_dict=feed_dict)

# you can print the confusion matrix using the 4 values from above, or get precision and recall:
precision = float(true_positives) / float(true_positives+false_positives)
recall = float(true_positives) / float(true_positives+false_negatives)

# or F1 score:
F1_score = 2 * ( precision * recall ) / ( precision+recall )