python - data appending as dataframe in memory

Question

I have data like this df:

,load,mem,Loc,Rem
0,1.0,7.48,6.7,pencil
1,1.0,8.020022,7.602671,book

df1:

0,log_no
1,1.2165430890794655

which I am appending at each iteration and it attaches like below with command

    df.to_csv(out_path,  mode='a', header=True)
    df1.to_csv(out_path,  mode='a', header=True)

and it results as csv below:

,load,mem,Loc,Rem
0,1.0,7.48,6.7,pencil
1,1.0,8.020022,7.602671,book
0,log_no
1,1.2165430890794655

Than, I have to import it again with

pd.read_csv()

I want to keep it appended as dataframe as df3 in memory to use it further instead of exporting and importing again. Is it possible? I tried to append several ways, but didn't succeed.

question from:https://stackoverflow.com/questions/65935690/data-appending-as-dataframe-in-memory

Answer 1

Sure! Just use .append() method:

df3 = pd.DataFrame()

And then, inside loop:

df3 = df3.append(df)
df3 = df3.append(df1)