python - Can't run pip: UnicodeDecodeError

Question

I have trouble using pip. For example:

pip install numpy --upgrade

Gives me the following error:

Collecting numpy
  Using cached numpy-1.10.2.tar.gz
        Exception:
    Traceback (most recent call last):
      File "/usr/local/lib/python2.7/dist-packages/pip/basecommand.py", line 211, in main
        status = self.run(options, args)
      File "/usr/local/lib/python2.7/dist-packages/pip/commands/install.py", line 305, in run
        wb.build(autobuilding=True)
      File "/usr/local/lib/python2.7/dist-packages/pip/wheel.py", line 705, in build
        self.requirement_set.prepare_files(self.finder)
      File "/usr/local/lib/python2.7/dist-packages/pip/req/req_set.py", line 334, in prepare_files
        functools.partial(self._prepare_file, finder))
      File "/usr/local/lib/python2.7/dist-packages/pip/req/req_set.py", line 321, in _walk_req_to_install
        more_reqs = handler(req_to_install)
      File "/usr/local/lib/python2.7/dist-packages/pip/req/req_set.py", line 505, in _prepare_file
        abstract_dist.prep_for_dist()
      File "/usr/local/lib/python2.7/dist-packages/pip/req/req_set.py", line 123, in prep_for_dist
        self.req_to_install.run_egg_info()
      File "/usr/local/lib/python2.7/dist-packages/pip/req/req_install.py", line 376, in run_egg_info
        self.setup_py, self.name,
      File "/usr/local/lib/python2.7/dist-packages/pip/req/req_install.py", line 347, in setup_py
        import setuptools  # noqa
      File "/usr/local/lib/python2.7/dist-packages/setuptools/__init__.py", line 12, in <module>
        from setuptools.extension import Extension
      File "/usr/local/lib/python2.7/dist-packages/setuptools/extension.py", line 8, in <module>
        from .dist import _get_unpatched
      File "/usr/local/lib/python2.7/dist-packages/setuptools/dist.py", line 19, in <module>
        import pkg_resources
      File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 3138, in <module>
        @_call_aside
      File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 3124, in _call_aside
        f(*args, **kwargs)
      File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 3151, in _initialize_master_working_set
        working_set = WorkingSet._build_master()
      File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 652, in _build_master
        ws = cls()
      File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 645, in __init__
        self.add_entry(entry)
      File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 701, in add_entry
        for dist in find_distributions(entry, True):
      File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2139, in find_on_path
        path_item, entry, metadata, precedence=DEVELOP_DIST
      File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2521, in from_location
        py_version=py_version, platform=platform, **kw
      File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2835, in _reload_version
        md_version = _version_from_file(self._get_metadata(self.PKG_INFO))
      File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2486, in _version_from_file
        line = next(iter(version_lines), '')
      File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2654, in _get_metadata
        for line in self.get_metadata_lines(name):
      File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2030, in get_metadata_lines
        return yield_lines(self.get_metadata(name))
      File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2025, in get_metadata
        metadata = f.read()
      File "/usr/lib/python2.7/codecs.py", line 296, in decode
        (result, consumed) = self._buffer_decode(data, self.errors, final)
    UnicodeDecodeError: 'utf8' codec can't decode byte 0xb6 in position 147: invalid start byte

Here are some clues:

(i) I have the same error when I try to run Spyder. It also appears when I try to to install other packages wtih pip, pandas for example.

(ii) I have the feeling, that this is related to the default encoding since sys.getdefaultencoding gives me 'ascii'

Note that it works well if I do it in a virtualenv.

I'm new to ubuntu so I might have done someting wrong.

Setup: python 2.7.6; pip 7.1.2; ubuntu 14.04.03.

Thank you for your help.

Answer 1

I had the same issue. In my case, it comes from a non-standard character in a module description.

I added a

print f.path

in the script

/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py

before line 2025, which allowed me to identify the file which was raising an error. It appeared to be the file

/usr/lib/pymodules/python2.7/rpl-1.5.5.egg-info

whose author has a name containing the ? character, which can not be read. I simply replaced the "G?ran" with "Goran" in this file and it fixed the problem.

hope this helps.