How do I detect what language a text is written in using NLTK?
The examples I've seen use nltk.detect, but when I've installed it on my mac, I cannot find this package.
nltk.detect
Have you come across the following code snippet?
english_vocab = set(w.lower() for w in nltk.corpus.words.words()) text_vocab = set(w.lower() for w in text if w.lower().isalpha()) unusual = text_vocab.difference(english_vocab)
from http://groups.google.com/group/nltk-users/browse_thread/thread/a5f52af2cbc4cfeb?pli=1&safe=active
Or the following demo file?
https://web.archive.org/web/20120202055535/http://code.google.com/p/nltk/source/browse/trunk/nltk_contrib/nltk_contrib/misc/langid.py
1.4m articles
1.4m replys
5 comments
57.0k users