I have a class MyClass
defined in my_module
. MyClass
has a method pickle_myself
which pickles the instance of the class in question:
def pickle_myself(self, pkl_file_path):
with open(pkl_file_path, 'w+') as f:
pkl.dump(self, f, protocol=2)
I have made sure that my_module
is in PYTHONPATH
. In the interpreter, executing __import__('my_module')
works fine:
>>> __import__('my_module')
<module 'my_module' from 'A:my_stuffmy_module.pyc'>
However, when eventually loading the file, I get:
File "A:Anacondalibpickle.py", line 1128, in find_class
__import__(module)
ImportError: No module named my_module
Some things I have made sure of:
EDIT -- A toy example that reproduces the error:
The example itself is spread over a bunch of files.
First, we have the module ball
(stored in a file called ball.py
):
class Ball():
def __init__(self, ball_radius):
self.ball_radius = ball_radius
def say_hello(self):
print "Hi, I'm a ball with radius {}!".format(self.ball_radius)
Then, we have the module test_environment
:
import os
import ball
#import dill as pkl
import pickle as pkl
class Environment():
def __init__(self, store_dir, num_balls, default_ball_radius):
self.store_dir = store_dir
self.balls_in_environment = [ball.Ball(default_ball_radius) for x in range(num_balls)]
def persist(self):
pkl_file_path = os.path.join(self.store_dir, "test_stored_env.pkl")
with open(pkl_file_path, 'w+') as f:
pkl.dump(self, f, protocol=2)
Then, we have a module that has functions to make environments, persist them, and load them, called make_persist_load
:
import os
import test_environment
#import pickle as pkl
import dill as pkl
def make_env_and_persist():
cwd = os.getcwd()
my_env = test_environment.Environment(cwd, 5, 5)
my_env.persist()
def load_env(store_path):
stored_env = None
with open(store_path, 'rb') as pkl_f:
stored_env = pkl.load(pkl_f)
return stored_env
Then we have a script to put it all together, in test_serialization.py
:
import os
import make_persist_load
MAKE_AND_PERSIST = True
LOAD = (not MAKE_AND_PERSIST)
cwd = os.getcwd()
store_path = os.path.join(cwd, "test_stored_env.pkl")
if MAKE_AND_PERSIST == True:
make_persist_load.make_env_and_persist()
if LOAD == True:
loaded_env = make_persist_load.load_env(store_path)
In order to make it easy to use this toy example, I have put it all up on in a Github repository that simply needs to be cloned into your directory of choice.. Please see the README
containing instructions, which I also reproduce here:
Instructions:
1) Clone repository into a directory.
2) Add repository directory to PYTHONPATH.
3) Open up test_serialization.py
, and set the variable MAKE_AND_PERSIST
to True
. Run the script in an interpreter.
4) Close the previous interpreter instance, and start up a new one. In test_serialization.py
, change MAKE_AND_PERSIST
to False
, and this will programmatically set LOAD
to True
. Run the script in an interpreter, causing ImportError: No module named test_environment
.
5) By default, the test is set to use dill, instead of pickle. In order to change this, go into test_environment.py
and make_persist_load.py
, to change imports as required.
EDIT: after switching to dill '0.2.5.dev0', dill.detect.trace(True)
output
C2: test_environment.Environment
# C2
D2: <dict object at 0x000000000A9BDAE8>
C2: ball.Ball
# C2
D2: <dict object at 0x000000000AA25048>
# D2
D2: <dict object at 0x000000000AA25268>
# D2
D2: <dict object at 0x000000000A9BD598>
# D2
D2: <dict object at 0x000000000A9BD9D8>
# D2
D2: <dict object at 0x000000000A9B0BF8>
# D2
# D2
EDIT: the toy example works perfectly well when run on Mac/Ubuntu (i.e. Unix-like systems?). It only fails on Windows.
See Question&Answers more detail:
os