git - Nesting repositories and make them independently pullable/deployable?

Question

I'm not sure if the following is a reasonable thing to do (and would appreciate to be told if not) but I would like to have nested repositories with a structure similar as in my tree below.

github
│
└── main-repo
    ├── django
    │   ├── django-project-1
    │   └── django-project-2
    └── react
        ├── react-project-1
        └── react-project-2

The idea behind this is to have a centralized directory in my computer for my practice projects, easy to find, access and maintain.

I would as well like to have this same (mirroring) tree structure in GitHub for the same reasons described above, rather than having multiple GH repos.

While I can do that as a single repository with subdirectories, I would like all these sub-projects to be 'independent', meaning to be able to pull them separately, and even deploy them separately for my practice purposes.

So far I tried initializing them independently but I don't think that's doing for me what I'm looking for (i.e. I still cannot pull from GitHub a subdirectory so initialized).

I also looked at submodules, but unless I'm wrong I don't think they are the right option for what I'm trying to do.

Is this a doable thing?

question from:https://stackoverflow.com/questions/65871594/nesting-repositories-and-make-them-independently-pullable-deployable

Answer 1

I would as well like to have this same (mirroring) tree structure in GitHub for the same reasons described above, rather than having multiple GH repos.

The way to do this is to carry all the histories in the one repo and check them out as submodules. That's what you've got, and what submodules are: independently-updated histories you're interested in as part of some larger effort. The helper command doesn't have automation for this, but the setup's easy.

So here's a full reset that constructs the repo setup you describe while avoiding assuming anything about details you haven't specified here, to make the commands below work no matter what.

cd `mktemp -d`; git init         # we're going to build a specific branch setup
(cd /path/to/main-repo; find -name .git -prune -o -print | cpio -pd ~-)  
for d in {django,react}/*-project-*; do 
        (cd $d; git init; git add .; git commit -m-)
        git submodule add ./$d
done

and now you've got your nested-repos setup, but with each project added as a submodule. To hoist the histories into your main repo,

for d in {django,react}/*-project-*; do 
        git fetch ./$d
        git branch ${d##*/} FETCH_HEAD
done

and now you've got a perfectly-ordinary branch for each checked-out tip in your project repos. Edit the .gitmodules file and set all the url's to ./ just for the sake of cleanliness.

Now commit the result.

When you clone the repo this built, you'll get the main branch and a branch for each submodule i.e. project history. To set up the subdirectories aka projects aka project submodules,

git worktree add django/django-project-1 django-project-1
# etc.

or to automate it as was done above,

for d in {django,react}/*-project-*/; do
        git worktree add $d ${d##*/}
done

and anywhere in here you can do a git submodule init. Now you've got a freshly-cloned repo carrying all your independent project histories as submodules, all in one neat package.