I'm a bit confused whether I should remove the robots meta tag, if I want search engines to follow my robots.txt rules.
If the robots meta-tag (index, follow) exists on the page, will search engines then ignore my robots.txt file and index the specified disallowed URLs in my robots.txt anyway?
The reason why I'm asking about this, is that search engines (Google mainly) still indexes disallowed pages from my website.
See Question&Answers more detail:
os 与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…