orm - Memory leaks Symfony2 Doctrine2 / exceed memory limit

Question

I have a lot of trouble with the combination of symfony2 and doctrine2. I have to deal with huge datasets (around 2-3 million write and read) and have to do a lot of additional effort to avoid running out of memory.

I figgured out 2 main points, that "leak"ing memory (they are actually not really leaking, but allocating a lot).

1. The Entitymanager entity storage (I don't know the real name of this one) it seems like it keeps all processed entities and you have to clear this storage regularly with

   $entityManager->clear()

2. The Doctrine QueryCache - it caches all used Queries and the only configuration I found was, that you are able to decide what kind of Cache you wanna use. I didn't find a global disable neither a useful flag for each query to disable it. So usually I disable it for every query object with the function

    $qb = $repository->createQueryBuilder($a);
    $query = $qb->getQuery();
    $query->useQueryCache(false);
    $query->execute();
    

So.. that's all I figured out right now.. My questions are:

Is there a easy way to deny some objects from the Entitymanagerstorage? Is there a way to set the querycache use in the entitymanager? Can I configure this caching behaviors somewhere in the Symfony/doctrine configuration?

Would be very cool if someone has some nice tips for me.. otherwise this may help some rookie..

cya

Answer 1

As stated by the Doctrine Configuration Reference by default logging of the SQL connection is set to the value of kernel.debug, so if you have instantiated AppKernel with debug set to true the SQL commands get stored in memory for each iteration.

You should either instantiate AppKernel to false, set logging to false in you config YML, or either set the SQLLogger manually to null before using the EntityManager

$em->getConnection()->getConfiguration()->setSQLLogger(null);