I'm getting
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 0: ordinal not in range(128)
when I pass text coming from a MySQL database, which I am accessing using SQLAlchemy, to this function:
re.compile(ur"<([^>]+)>", flags=re.UNICODE).sub(u" ", s)
The database encoding is utf-8 and I am even passing the encoding to the create_engine function of SQLAlchemy.
Edit:
This is how I am querying the database:
doc = session.query(Document).get(doc_id)
s = doc.title
By suggestion, I passed s.decode('utf-8') to sub
. The error above disappeared, but I get a different error for a different document:
UnicodeDecodeError: 'utf8' codec can't decode byte 0xeb in position 449: invalid continuation byte
The database table is defined like this:
CREATE TABLE `articles` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`title` varchar(255) DEFAULT NULL,
`cdate` datetime DEFAULT NULL,
`link` varchar(255) DEFAULT NULL,
`content` text,
UNIQUE KEY `id` (`id`),
UNIQUE KEY `link_idx` (`link`)
) ENGINE=InnoDB AUTO_INCREMENT=4127834 DEFAULT CHARSET=utf8;
Any help would be greatly appreciated
See Question&Answers more detail:
os 与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…