python - Using a fake mongoDB for pytest testing

Question

I have code that connects to a MongoDB Client and I'm trying to test it. For testing, I don't want to connect to the actual client, so I'm trying to figure out make a fake one for testing purposes. The basic flow of the code is I have a function somewhere the creates a pymongo client, then queries that and makes a dict that is used elsewhere.

I want to write some tests using pytest that will test different functions and classes that will call get_stuff. My problem is that get_stuff calls mongo() which is what actually makes the connection to the database. I was trying to just use pytest.fixture(autouse=True) and mongomock.MongoClient() to replace mongo().

But this isn't replacing the mongo_stuff.mongo(). Is there some way I can tell pytest to replace a function so my fixture is called instead of the actual function? I thought making the fixture would put my testing mongo() higher priority in the namespace than the function in the actual module.

Here is an example file structure with my example:

.
├── project
│?? ├── __init__.py
│?? ├── mongo_stuff
│?? │?? ├── __init__.py
│?? │?? └── mongo_stuff.py
│?? └── working_class
│??     ├── __init__.py
│??     └── somewhere_else.py
└── testing
    ├── __init__.py
    └── test_stuff.py

mongo_stuff.py

import pymongo

def mongo():
    return pymongo.MongoClient(connection_params)

def get_stuff():
    db = mongo()  # Makes the connection using another function
    stuff = query_function(db)  # Does the query and makes a dict
    return result

somewhere_else.py

from project.mongo_stuff import mongo_stuff

mongo_dict = mongo_stuff.get_stuff()

test_stuff.py

import pytest
import mongomock

@pytest.fixture(autouse=True)
def patch_mongo(monkeypatch):
    db = mongomock.MongoClient()
    def fake_mongo():
        return db
    monkeypatch.setattr('project.mongo_stuff.mongo', fake_mongo)

from poject.working_class import working_class  # This starts by calling project.mongo_stuff.mongo_stuff.get_stuff()

And this will currently give me a connection error since the connection params in mongo_stuff.py are only made to work in the production environment. If I put the import statement from test_stuff.py into a test function, then it works fine and mongomock db will be used in the testing enviornment. I also tried change the setattr to monkeypatch.setattr('project.working_class.mongo_stuff.mongo', fake_mongo) which also does not work.

Answer 1

You're halfway there: you have created a mock for the db client, now you have to patch the mongo_stuff.mongo function to return the mock instead of a real connection:

@pytest.fixture(autouse=True)
def patch_mongo(monkeypatch):
    db = mongomock.MongoClient()
    def fake_mongo():
        return db
    monkeypatch.setattr('mongo_stuff.mongo', fake_mongo)

Edit:

The reason why you get the connection error is that you are importing somewhere_else on module level in test_stuff, and somewhere_else runs connection code also on module level. So patching with fixtures will come too late and will have no effect. You have to patch the mongo client before the import of somewhere_else if you want to import on module level. This will avoid the error raise, but is extremely ugly:

from project.mongo_stuff import mongo_stuff
import mongomock
import pytest

from unittest.mock import patch

with patch.object(mongo_stuff, 'mongo', return_value=mongomock.MongoClient()):

    from project.working_class import somewhere_else


@patch.object(mongo_stuff, 'mongo', return_value=mongomock.MongoClient())
def test_db1(mocked_mongo):
    mongo_stuff.mongo()
    assert True


@patch.object(mongo_stuff, 'mongo', return_value=mongomock.MongoClient())
def test_db2(mocked_mongo):
    somewhere_else.foo()
    assert True

You should rather avoid running code on module level when possible, or run the imports that execute code on module level inside the tests (as you already found out in the comments).