UTF-8 is designed for this.
def split_utf8(s, n):
"""Split UTF-8 s into chunks of maximum length n."""
while len(s) > n:
k = n
while (ord(s[k]) & 0xc0) == 0x80:
k -= 1
yield s[:k]
s = s[k:]
yield s
Not tested. But you find a place to split, then backtrack until you reach the beginning of a character.
However, if a user might ever want to see an individual chunk, you may want to split on grapheme cluster boundaries instead. This is significantly more complicated, but not intractable. For example, in "é"
, you might not want to split apart the "e"
and the "′"
. Or you might not care, as long as they get stuck together again in the end.
与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…