python - How to to iterate through a specific time range in a logfile?

Question

For example: [2017-04-14 03:56:22,085109]

If this is the time where event A happens, I want to go 15 minutes before this line in the logfile which will have thousands of line and I want to iterate through everyline in that period and look for specific keywords. Every line in the logfile has the time stamp with the same format.

Answer 1

You can filter the desired lines with a start and end time.

main.py

from datetime import datetime, timedelta

event_time = datetime.strptime('2017-04-14 03:56:22,085109', '%Y-%m-%d %X,%f')
event_start = event_time - timedelta(minutes=15)


def line_date(line):
    return datetime.strptime(line[1:27], '%Y-%m-%d %X,%f')


with open('example.log', 'r') as myfile:
    lines = filter(lambda x: event_start <= line_date(x) <= event_time,
                   myfile.readlines())


print(lines)

example.log

[2017-04-13 03:56:22,085109] My old log
[2017-04-14 03:55:22,085109] My log in less than 15 minutes ago
[2017-04-14 03:56:22,085109] My important event
[2017-04-14 03:57:22,085109] Log after my important event

But I recommend you to use python3 (instead of python2). filter returns an iterator, instead of the full list.