If I wasn't using a DI container, I wouldn't have to reference EntityFramework library in my MVC3 app, only my business layer which would reference my DAL/Repo layer.
Yes, that's exactly the situation DI works so hard to avoid :)
With tightly coupled code, each library may only have a few references, but these again have other references, creating a deep graph of dependencies, like this:
Because the dependency graph is deep, it means that most libraries drag along a lot of other dependencies - e.g. in the diagram, Library C drags along Library H, Library E, Library J, Library M, Library K and Library N. This makes it harder to reuse each library independently from the rest - for example in unit testing.
However, in a loosely coupled application, by moving all the references to the Composition Root, the dependency graph is severely flattened:
As illustrated by the green color, it's now possible to reuse Library C without dragging along any unwanted dependencies.
However, all that said, with many DI Containers, you don't have to add hard references to all required libraries. Instead, you can use late binding either in the form of convention-based assembly-scanning (preferred) or XML configuration.
When you do that, however, you must remember to copy the assemblies to the application's bin folder, because that no longer happens automatically. Personally, I rarely find it worth that extra effort.
A more elaborate version of this answer can be found in this excerpt from my book Dependency Injection, Principles, Practices, Patterns.