I need to split few csv files based on a given time. In these files the time values are in seconds and given in 'Time' column.
For example, if I want to split aaa.csv
file in 0.1 seconds, then the first set of rows with time 0.0 to 0.1 (No 1 to 8 in attached file) needs to get written into aaa1.csv
, then the rows with time greater than 0.1 to 0.2 (No. 9 to 21 in attached file) to aaa2.csv
so on...(basically multiples of the given time).
Output files needs to get the same name as input file along with a number at the end. And output files need to get written into a different location/folder.
Time value need to be a variable. So at a time I can split in 0.1 sec and at another time I can split the file in 0.7sec so on.
How can I write a python script for this please? The file looks like the following (entire 119K file can be downloaded from https://fil.email/vnsZsp7b):
No.,Time,Length
1,0,146
2,0.006752,116
3,0.019767,156
4,0.039635,144
5,0.06009,147
6,0.069165,138
7,0.0797,133
8,0.099397,135
9,0.120142,135
10,0.139721,148
11,0.1401,126
12,0.1401,120
13,0.140101,123
14,0.140101,120
15,0.141294,118
16,0.141295,118
17,0.141295,114
18,0.144909,118
19,0.160639,119
20,0.161214,152
21,0.185625,143
... etc
AFTER @Serafeim 's answer, I tried this:
import pandas as pd
import numpy as np
import glob
import os
path = '/root/Desktop/TT1/'
mystep = 0.4
for filename in glob(os.path.join(path, '*.csv')):
df = pd.read_csv(filename)
def data_splitter(df):
max_time = df['Time'].max() # get max value of Time for the current csv file (df)
myrange= np.arange(0, max_time, mystep) # build the threshold range
for k in range(len(myrange)):
# build the upper values
temp = df[(df['Time'] >= myrange[k]) & (df['Time'] < myrange[k] + mystep)]
#temp.to_csv("/root/Desktop/T1/xx_{}.csv".format(k))
temp.to_csv("/root/Desktop/T1/{}_{}.csv".format(filename, k))
data_splitter(df)
See Question&Answers more detail:
os