keras - How to profile CPU usage of a Python script?

Question

Ideally what I want is to record the CPU usage of a Python script that is executing a deep neural net Keras model. I'm looking for the CPU equivalent of memory_profiler, which provides the memory consumption of a process.

I have looked at using psutil (suggested in this answer) which would indicate my script could contain some variant of

p = psutil.Process(current_pid)
p.cpu_percent()

but the problem is the important function call I really want to capture the effect of would be the inference stage of the model

model.predict(x_test)

and if I run psutil before/after this step the CPU usage recorded won't be a true reflection of the CPU usage of the process.

So then I was thinking could I use something like top/htop to log the CPU usage of the script to some file, capturing the fluctuating CPU usage while the process is running, and then calculate an average (or something similar) after the fact. The issue I see with this, however, is don't I need to know the PID to utilise top, so how can I use top to monitor the script before it is executed (and hasn't even been assigned a PID)?

I can see this highly-ranked answer suggests cProfile which gives the running time of functions within a script. Although this isn't exactly what I want I do notice that it returns the total number of CPU seconds, which would at least let me compare CPU usage in that regard.

Answer 1

You can run model.predict(x_test) in a subprocess and log its CPU usage simultaneously in the main process. For example,

import time
import multiprocessing as mp
import psutil
import numpy as np
from keras.models import load_model

def run_predict():
    model = load_model('1.h5')
    x_test = np.random.rand(10000, 1000)
    time.sleep(1)

    for _ in range(3):
        model.predict(x_test)
        time.sleep(0.5)

def monitor(target):
    worker_process = mp.Process(target=target)
    worker_process.start()
    p = psutil.Process(worker_process.pid)

    # log cpu usage of `worker_process` every 10 ms
    cpu_percents = []
    while worker_process.is_alive():
        cpu_percents.append(p.cpu_percent())
        time.sleep(0.01)

    worker_process.join()
    return cpu_percents

cpu_percents = monitor(target=run_predict)

The values in cpu_percents for the above script would be something like: