This observation is easy to confirm (though whether the change is due to Big Sur or to Xcode 12 is a bit unclear).
You can show the Project Navigator in one window tab and the Debug Navigator in another window tab, but you cannot show the Project Navigator in one window tab and no Navigator in another window tab.
Moreover (you didn't mention this), in the second window tab you cannot widen and narrow the Navigator pane. It shows at just one width and it either shows or it does not. (Xcode 12.4 may have fixed this part.)
An easy workaround is to use two windows on the same project (File > New > Window) rather than two window tabs within the same window (File > New > Window Tab).
But it might be worth filing a bug with Apple, as this is definitely a major change in behavior. It certainly feels buggy to me.
与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…