automation - Python script to search for a file using partial name in a set of folders and sort them into a new folder structure

Question

I am trying out python scripting for automation for the first time. I had a set of files in a particular folder structure

source1
    -date1 
        -date1_time1_xyz.csv
        -date1_time3_abc.doc
    -date2 
        -date2_time1_xyz.csv
        -date2_time3_abc.doc
source2
    -date1
        -date1_time1_dfg.csv
        -date1_time2_abc.doc
    -date2
        -date2_time1_xyz.csv
        -date2_time3_abc.doc

But I want a folder structure that would group all the .csv of all the dates together into one folder called meta and .doc of all the dates together to another folder called docs like this:

date1
    -meta
        -date1_time1_xyz.csv
        -date1_time1_dfg.csv
    -docs
        -date2_time3_abc.doc
        -date2_time2_abc.doc

I wrote a script to sort these videos into my required folder structure! So I thought I would share my solution (maybe it might help someone). All comments for improvement are welcome!

question from:https://stackoverflow.com/questions/65830893/python-script-to-search-for-a-file-using-partial-name-in-a-set-of-folders-and-so

Answer 1

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Created on Wed Jan 20 09:57:51 2021

Example usage: 

#specify source folder
ogpath = '/home/data 
(data should contain folders source1, source2, so on..)

#specify destination folder
new_path = '/home/tester/new'

#specify which recording to aggregate
filename = "date1_time1"


"""
import os, shutil 
from pathlib import Path
import glob

#specify source folder
ogpath = '/home/data'

#specify destination folder
new_path = '/home/tester/new'

#specify which files to aggregate
filename = "date1_time1"

#create new folder for sorted data
dst_dirpath = os.path.join(new_path,'sorted_data')
print(dst_dirpath)
if not os.path.isdir(dst_dirpath):
    os.mkdir(dst_dirpath)

#create new path for each date
record_path = os.path.join(dst_dirpath, filename[:4]) #take only date1 to create the folder
if not os.path.isdir(record_path):
    os.mkdir(record_path)

#create sub path for meta
meta_path = os.path.join(record_path,'meta')
if not os.path.isdir(meta_path):
    os.mkdir(meta_path)

#create sub path for docs
docs_path = os.path.join(record_path,'docs')
if not os.path.isdir(docs_path):
    os.mkdir(docs_path)    

#search for files with same name in all folders
for file in glob.glob(ogpath+'/**/'+filename[:4]+'/*'+filename+'*.doc'): 
    shutil.copy(file, docs_path)

for file in glob.glob(ogpath+'/**/**/*'+filename+'*.csv'):
    shutil.copy(file, meta_path)