bash - grep -vf too slow with large files

Question

I am trying filter data from data.txt using patterns stored in a file filter.txt. Like below,

grep -v -f filter.txt data.txt > op.txt

This grep takes more than 10-15 minutes for 30-40K lines in filter.txt and ~300K lines in data.txt.

Is there any way to speed up this?

data.txt

data1
data2
data3

filter.txt

data1

op.txt

data2
data3

This works with solution provided by codeforester but fails when filter.txt is empty.

Answer 1

Based on Inian's solution in the related post, this awk command should solve your issue:

awk 'FNR==NR {hash[$0]; next} !($0 in hash)' filter.txt data.txt > op.txt