Python pandas dataframe failing to successfully execute to_csv() in run mode while successfully executes in debug mode

Question

I'm using python pandas to create a relatively large dataframe and storing it as a csv file on windows operating system. The dataframe is relatively large in terms of 1000s of rows * 100 columns. The problem I'm seeing is inconsistency in creating the file and storing it during execution without any errors. The valid file name is created dynamically and the data changes every time, but I can get windows to create and save the same dataframe as a csv in debug mode with single stepping, i.e. when I give a gap between the dataframe.to_csv() function and next step, but in run-time irrespective of the delay the file is sometimes generated and sometimes not.

Could someone advise the best way to debug the erratic behavior and what might be the root cause ? I'm sure the data or the file name is not the culprit.

import pandas as pd    
# logic for the valid pd_csv_filename creation 
# logic for the valid dataframe df1 creation & manipulation 

time.sleep(x)
df1.to_csv(pd_csv_fileName, mode='a')
time.sleep(x)

# Check if the file is created

The csv file is not opened during the execution.

question from:https://stackoverflow.com/questions/65844412/python-pandas-dataframe-failing-to-successfully-execute-to-csv-in-run-mode-whi

Answer 1

Waitting for answers