python - Curious( and seemingly unexplainable?) problem when replacing array of chars with pandas.where()

Question

In preparations for machine learning tasks on EEG-data i am trying to learn Pandas, and i have struggled alot with some basic challenges.

Brief explaination of what i am trying to do: I have

time_series - dataframe with the eegdata for 64 electrodes(340 000*64)

time_stamps - dataframe with timestamps for the EEG-data(340 000*1)

marker_stamps - dataframe with timestamps for when stimuli was presented to the subject(30*1)

markers - dataframe with markers for what stimuli was presented(30*2)(char)

since stimuli was presented 3 seconds at a time, I wanted to make a mask that shows at what timestamps stimuli was on the screen(e.g. 340 000*2,char) which later could be merged with time_series and time_stamps for a compact dataframe with all needed information.

I realized that to do this i still had to loop through the marker_stamps(please correct me if know how I could do it without looping). So I made a dataframe(340 000*2) with ['n','n'](coined as "idle" in contrast to the different stimuli. Then I looped through the markers and use pandas.where() to mask with the correct stimuli markers. Which looks like this:

a = pd.DataFrame('n', index=range(desc['sample_count']), columns=['0', '1'])
for i in range(5,35):
    cond = ((time_stamps - marker_stamps.iat[i,0]) > 3.0) | ((time_stamps - marker_stamps.iat[i,0]) < 0.0)
    a = a.where(cond.iloc[:,0], [markers.iloc[i,0], markers.iloc[i,1]] , axis=0)
    # a = a.where(cond.iloc[:,0], markers.iloc[i,0:1] , axis=0) ### why does this not work?

So the problem is marked with a comment in the code above: why does the commented line not work, should they not give the excact same output?

Pictures of the results below(exported to csv->screenshot in VSCode). The correct result first, the wrong result last.

So my questions are:

First and foremost: Why do i need to split and put together the array with markes for it to be added correctly?

Secondly: Is my overall method reasonable? could this be done in a vectorized manner?

question from:https://stackoverflow.com/questions/65921562/curious-and-seemingly-unexplainable-problem-when-replacing-array-of-chars-wit

Answer 1

Waitting for answers