linux - How to make FIO replay a trace with multiple thread

Question

I'm trying to use fio to replay some block traces.

The job file I wrote looks like:

[global]
name=replay
filename=/dev/md0
direct=1 
ioengine=psync

[replay]
read_iolog=iolog.fio
replay_no_stall=0 
write_lat_log=replay_metrics
numjobs=1

The key here is I want to use "psync" as the ioengine, and replay the iolog. However, with psync, fio seems to ignore "replay_no_stall" option, which ignore the timestamp in the iolog.

And by setting numjobs to be 4, fio seems to make 4 copies of the same workload, instead of using 4 threads to split the workload.

So, how could I make fio with psync respect the timestamp, and use multiple threads to replay the trace?

Answer 1

Without seeing a small problem snippet of the iolog itself I can't say why the replay is always going as fast as possible. Be aware that waits are in milliseconds and successive waits in the iolog MUST increase if the later ones are to have an effect (as they are relative to the start of the job itself and not to each other or the previous I/O). See the "Trace file format v2" section of the HOWTO for more details. This problem sounds like a good question for the fio mailing list (but as it's a question please don't put it in the bug tracker).

numjobs is documented as only creating clones in the HOWTO so your experience matches the documented behaviour.

Sadly fio replay currently (end of 2016) doesn't work in a way that a single replay file can be arbitrarily split among multiple jobs and you need multiple jobs to have fio use multiple threads/processes. If you don't mind the fact that you will lose I/O ordering between jobs you could split the iolog into 4 pieces and create a job that uses each of the new iolog files.