Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
239 views
in Technique[技术] by (71.8m points)

python 3.2 UnicodeEncodeError: 'charmap' codec can't encode character 'u2013' in position 9629: character maps to <undefined>

I'm trying to make a script that gets data out from an sqlite3 database, but I have run in to a problem.

The field in the database is of type text and the contains a html formated text. see the text below

<html>
<head>
<title>Yahoo!</title>
</head>
<body>
<style type="text/css">
html {}
.yshortcuts {border-bottom:none !important;}
.ReadMsgBody {width:100%;}
.ExternalClass{width:100%;}
</style>
<table cellpadding="0" cellspacing="0" bgcolor="#ffffff">    
<tr>
<td width="550" valign="top" align="left">

    <table cellpadding="0" cellspacing="0" width="500">
        <tr>
            <td colspan="3"><img        src="http://mail.yimg.com/nq/assets/sharedmessages/v1/us/logo.gif" width="292" height="51" style="display:block;" border="0" alt="Yahoo! Mail"></td>
        </tr>
        <tr>
            <td rowspan="3" width="1" bgcolor="#c7c4ca"></td>
            <td width="498" height="1" bgcolor="#c7c4ca"></td>
            <td rowspan="3" width="1" bgcolor="#c7c4ca"></td>
        </tr>
        <tr>
            <td width="498" valign="top" align="left">
            <table cellpadding="0" cellspacing="0">
                <tr>
                    <td width="498" bgcolor="#61399d" align="left" valign="top">
                    <table cellspacing="0" cellpadding="0"><tr><td height="24"></td></tr></table>
                    <div style="font-family:Arial, Helvetica, sans-serif;font-size:23px;line-height:27px;margin-bottom:10px;color:#ffffff;margin-left:15px;"><span style="color:#ffffff;text-decoration:none;font-weight:bold;line-height:27px;">V?lkommen till Yahoo! Mail.</span></div>
                    <div style="font-family:Arial, Helvetica, sans-serif;font-size:22px;line-height:26px;margin-bottom:1px;color:#ffffff;margin-left:15px;margin-bottom:7px;margin-right:15px;">Ansluta och dela g?r snabbt och enkelt och ?r tillg?ngligt ?verallt.</div>
                    </td>
                </tr>
                <tr>
                    <td><img src="http://mail.yimg.com/nq/assets/sharedmessages/v1/all/b1.gif" width="498" height="18" style="display:block;" border="0"></td>
                </tr>
            </table>
            <table cellpadding="0" cellspacing="0" width="498">
                <tr>
                    <td width="292" valign="top">
                    <table cellpadding="0" cellspacing="0">
                        <tr>
                            <td><img src="http://mail.yimg.com/nq/assets/sharedmessages/v1/all/grad.gif" width="292" height="9" style="display:block;"></td>
                        </tr>
                        <tr>
                            <td width="292" bgcolor="#ffffff" align="left" valign="top">
                            <table cellspacing="0" cellpadding="0"><tr><td height="11"></td></tr></table>
                            <div style="margin-left:15px;">                  
                                <div style="font-family:Arial, Helvetica, sans-serif;font-size:14px;line-height:18px;color:#333333;margin-bottom:11px;font-weight:bold;">Det ?r l?tt som en pl?tt att komma ig?ng.</div>
                                <table cellpadding="0" cellspacing="0" width="267">
                                    <tr>
                                        <td width="16" align="left" valign="top"><div style="font-family:Arial, Helvetica, sans-serif;font-size:14px;line-height:16px;color:#61399d;margin-bottom:9px;font-weight:bold;">1. </div></td>
                                        <td align="left" valign="top"><div style="font-family:Arial, Helvetica, sans-serif;font-size:13px;line-height:16px;color:#61399d;margin-bottom:9px;"><a rel="nofollow" target="_blank" href="http://us-mg999.mail.yahoo.com/neo/launch?action=contacts" style="text-decoration:underline;color:#61399d;"><span>L?gg till alla dina kontakter p? en plats</span></a>.</div></td>
                                    </tr>
                                    <tr>
                                        <td align="left" valign="top"><div style="font-family:Arial, Helvetica, sans-serif;font-size:14px;line-height:16px;color:#61399d;margin-bottom:9px;font-weight:bold;">2. </div></td>
                                        <td align="left" valign="top"><div style="font-family:Arial, Helvetica, sans-serif;font-size:13px;line-height:16px;color:#61399d;margin-bottom:9px;"><a rel="nofollow" target="_blank" href="http://mrd.mail.yahoo.com/themes" style="text-decoration:underline;color:#61399d;"><span>Anpassa din nya inkorg</span></a>.</div></td>
                                    </tr>
                                    <tr>
                                        <td align="left" valign="top"><div style="font-family:Arial, Helvetica, sans-serif;font-size:14px;line-height:16px;color:#61399d;margin-bottom:9px;font-weight:bold;">3. </div></td>
                                        <td align="left" valign="top"><div style="font-family:Arial, Helvetica, sans-serif;font-size:13px;line-height:16px;color:#61399d;"><a rel="nofollow" target="_blank" href="http://se.overview.mail.yahoo.com/mobile" style="text-decoration:underline;color:#61399d;"><span>Anslut ?verallt p? dina mobila enheter</span></a>.</div></td>
                                    </tr>
                                </table>

                            </div>
                            </td>
                        </tr>
                        <tr><td height="13"></td></tr>
                    </table>
                    </td>
                    <td width="196" valign="top">
                    <table cellpadding="0" cellspacing="0">
                        <tr>
                            <td width="1" bgcolor="#fbfbfd" valign="top"><img src="http://mail.yimg.com/nq/assets/sharedmessages/v1/all/g1.gif" width="1" height="21" style="display:block;"></td>
                            <td width="1" bgcolor="#f5f6fa" valign="top"><img src="http://mail.yimg.com/nq/assets/sharedmessages/v1/all/g2.gif" width="1" height="21" style="display:block;"></td>
                            <td width="1" bgcolor="#e8eaf1" valign="top"><img src="http://mail.yimg.com/nq/assets/sharedmessages/v1/all/g3.gif" width="1" height="21" style="display:block;"></td>
                            <td width="1" bgcolor="#d4d4d4"></td>
                            <td width="186" bgcolor="#f0f0f0" align="left" valign="top">  
                            <table cellspacing="0" cellpadding="0"><tr><td height="3">   </td></tr></table>
                            <div style="margin-left:11px;">
                            <div style="font-family:Arial, Helvetica, sans-serif;font-size:13px;line-height:16px;color:#333333;margin-bottom:9px;"><b>Info f?r dig:</b></div>
                            <div style="font-family:Arial, Helvetica, sans-serif;font-size:12px;color:#43494e;line-height:18px;margin-bottom:10px;">
                            Yahoo!-ID och e-postadress:<br />
                            <div style="font-family:Arial, Helvetica, sans-serif;font-size:12px;color:#43494e;line-height:18px;">
                            H?ll ditt konto och inst?llningar aktuella. <br><a rel="nofollow" target="_blank" href="https://edit.yahoo.com/config/eval_profile" style="text-decoration:underline;color:#61399d;"><span>Mitt konto</span></a> 
                            </div>
                            </div>
                            <table cellspacing="0" cellpadding="0"><tr><td height="20"></td></tr></table>
                            </td>
                            <td width="1" bgcolor="#dbdbdb"></td>
                            <td width="1" bgcolor="#ced2de"></td>
                            <td width="1" bgcolor="#dbdfed"></td>
                            <td width="1" bgcolor="#e8ebf3"></td>
                            <td width="1" bgcolor="#f3f4f9"></td>
                            <td width="1" bgcolor="#fafbfc"></td>
                        </tr>
                        <tr>
                            <td colspan="11"><img src="http://mail.yimg.com/nq/assets/sharedmessages/v1/all/b2.gif" width="196" height="8" style="display:block;" border="0"></td>
                        </tr>
                        <tr><td height="13"></td></tr>
                    </table>
                    </td>
                    <td width="10"></td>
                </tr>
            </table>
            </td>
        </tr>
        <tr>
            <td width="498" height="1" bgcolor="#c7c4ca"></td>
        </tr>
    </table>
    <table cellpadding="0" cellspacing="0" width="500">
        <tr>
            <td align="center" valign="top">
            <table cellspacing="0" cellpadding="0"><tr><td height="10"></td></tr></table>
                <div style="font-family:Arial, Helvetica, sans-serif;font-size:11px;line-height:18px;margin-bottom:10px;">
                <a rel="nofollow" target="_blank" href="http://info.yahoo.com/legal/se/yahoo/utos.html" style="text-decoration:underline;color:#61399d;">Yahoo! Villkor f?r anv?ndning</a>&nbsp;&nbsp;&nbsp;|&nbsp;&nbsp;&nbsp;<a rel="nofollow" target="_blank" href="http://info.yahoo.com/legal/se/yahoo/mail/atos.html" style="text-decoration:underline;color:#61399d;">Yahoo! Mail –Villkor f?r anv?ndning</a>&nbsp;&nbsp;&nbsp;|&nbsp;&nbsp;&nbsp;<a rel="nofollow" target="_blank" href="http://info.yahoo.com/privacy/se/yahoo/details.html" style="text-decoration:underline;color:#61399d;">Yahoo! Sekretesspolicy</a>
                </div>
            </td>
        </tr>
        <tr>
            <td align="left" valign="top">
                <div style="font-family:Arial, Helvetica, sans-serif;font-size:11px;line-height:14px;color:#545454;margin-left:16px;margin-right:14px;">Var god svara inte p? detta meddelande. Detta ?r ett servicemeddelande som r?r din anv?ndning av Yahoo! Mail. Om du vill veta mer om Yahoo!s anv?ndning av personlig information, inklusive anv?ndning av webb-beacons i HTML-baserad e-post, kan du l?sa v?r Yahoo! Sekretesspolicy. Yahoo!s adress ?r 701 First Avenue, Sunnyvale, CA 94089, USA.<br /><br />RefID: lp-1037111</div>
            </td>
        </tr>
    </table>





    </td>
</tr>
</table>
<img width="1" height="1" src="http://pclick.internal.yahoo.com/p/s=2143684696">
</body>
</html>`

and the python code that try to extract the data is as follows.

>>> import sqlite3
>&

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

When you open the file you want to write to, open it with a specific encoding that can handle all the characters.

with open('filename', 'w', encoding='utf-8') as f:
    print(r['body'], file=f)

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

1.4m articles

1.4m replys

5 comments

57.0k users

...