pandas - Issue checking for missing datetime value in series

Question

I'm trying to create a derived column based on two conditions being met for values in existing columns. One of the conditions that needs to be met is that that value for one of the columns cannot have a datetime value that is missing (NaT). I keep receiving an attribute error that the Timestamp object has no attribute isnull and I cannot understand how to fix it.

I checked that my conditional statement was correct by filtering my DataFrame on the conditions that I'm trying to include and that was successful.

Here is a sample of what my df contains:

Sample df

I'm choosing to create a function that I can apply using df.apply() because this is a data cleaning process i'll be doing regularly.

I'm trying to create a new field titled "case_start_time" with the following conditions:

Code used in function:

def case_start(df):
    if df[(df['procedure_type_zc'] == 'Infusion') & (df['line_start_time'].isnull() )]:
        return df['check_in']
    else:
        return 'Undefined'

And when applying this function to df to create a new field:

df['case_start_time'] = df.apply(case_start, axis = 1)

I receive the following error:

AttributeError: ("'Timestamp' object has no attribute 'isnull'", 'occurred at index 0')

These are the dtypes for the values in my df:

csn                           int64
line_start_time      datetime64[ns]
procedure_type_zc            object
dtype: object

After doing some research I found that I can apply .isnull() to a datetime value in pandas which is why i'm not sure how to resolve the error.

This is the code that I used to filter the DataFrame for both conditions:

missing_line_time = sample_df[ (sample_df['procedure_type_zc'] == 'Infusion') & (sample_df['line_start_time'].isnull()) ]

Based on the image I attached with the sample_df, this logic is correct.

Answer 1

I was running into a similar problem. This worked for me:

instead of using:

(sample_df['line_start_time'].isnull())

use:

(sample_df['line_start_time'] is pd.NaT)

hopefully that at least gets rid of your current error.