python 3.x - showing multiple data which meets certain conditions-issue in PANDAS

Question

I am working in PANDAS with Python and I am looking at a weather CSV file. I am able to pull data from it with no problem. However, I am not able to pull data that meets certain criteria such as when to show which days have the temperature above 100 degrees.

I have this as my code so far:

import pandas as pd
import numpy as np 
import matplotlib.pyplot as plt 

df = pd.read_csv('csv/weather.csv')

print(df[[df.MaxTemperatureF > 100 ]])

That last line is where I think I have my problem. The error traceback that I now get, after doing the steps below, is the following:

Traceback (most recent call last):
File "weather.py", line 40, in <module>
print(df[df['MaxTemperatureF' > 100]])
TypeError: unorderable types: str() > int()
Mikes-MBP-2:dataframes mikecuddy$ python3 weather.py
Traceback (most recent call last):
File "weather.py", line 41, in <module>
print(df[[df.MaxTemperatureF > 100 ]])
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-     
packages/pandas/core/frame.py", line 1991, in __getitem__
return self._getitem_array(key)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-  
packages/pandas/core/frame.py", line 2028, in _getitem_array
(len(key), len(self.index)))
 ValueError: Item wrong length 1 instead of 360.

I have been doing a tutorial at: http://www.gregreda.com/2013/10/26/working-with-pandas-dataframes/ Again any help would be great! Thank you!

df.info() information:

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 360 entries, 0 to 359
Data columns (total 23 columns):
PST                           360 non-null object
MaxTemperatureF               359 non-null float64
Mean TemperatureF             359 non-null float64
Min TemperatureF              359 non-null float64
Max Dew PointF                359 non-null float64
MeanDew PointF                359 non-null float64
Min DewpointF                 359 non-null float64
Max Humidity                  359 non-null float64
Mean Humidity                359 non-null float64
Min Humidity                 359 non-null float64
Max Sea Level PressureIn     359 non-null float64
Mean Sea Level PressureIn    359 non-null float64
Min Sea Level PressureIn     359 non-null float64
Max VisibilityMiles          355 non-null float64
Mean VisibilityMiles         355 non-null float64
Min VisibilityMiles          355 non-null float64
Max Wind SpeedMPH            359 non-null float64
Mean Wind SpeedMPH           359 non-null float64
Max Gust SpeedMPH            211 non-null float64
PrecipitationIn               360 non-null float64
CloudCover                   343 non-null float64
Events                       18 non-null object
WindDirDegrees               360 non-null int64
dtypes: float64(20), int64(1), object(2)
memory usage: 64.8+ KB
None

Answer 1

For the max temperature you can specify a converter function:

df = pd.read_csv('csv/weather.csv', converters={'MaxTemperatureF':float})

Edit: as @ptrj mentions in a comment you can do this to substitute np.nan for string values in the MaxTemperatureF column:

df = pd.read_csv('csv/weather.csv', 
                 converters={'MaxTemperatureF':
                             lambda x: try: return float(x); 
                                       except ValueError: return np.nan;})

Edit2: @ptrj's solution since he can't write it up in a comment...

def my_conv(x): 
    try: 
        return float(x)
    except ValueError: 
        return np.nan

df = pd.read_csv('csv/weather.csv', converters={'MaxTemperatureF': my_conv})

Other things:

• If the first row of the csv file has the headers then don't pass header=0.
• Being that you already have the header now you don't need to specify cols=...
• The default sep is ',' so you don't need to specify that.