python - select range of values for all columns in pandas dataframe

Question

I have a dataframe 'DF', part of which looks like this:

I want to select only the values between 0 and 0.01, to form a new dataframe(with blanks where the value was over 0.01)

To do this, i tried:

similarity = []
    for x in DF:
        similarity.append([DF[DF.between(0, 0.01).any(axis=1)]])
        simdf = pd.DataFrame(similarity)            
        
    simdf.to_csv("similarity.csv")

However, i get the error AttributeError: 'DataFrame' object has no attribute 'between'

How do i select a range of values and create a new data frame with these?

question from:https://stackoverflow.com/questions/65836606/select-range-of-values-for-all-columns-in-pandas-dataframe

Answer 1

Just do the two comparisons:

df_new = df[(df>0) & (df<0.01)]

Example:

import pandas as pd

df = pd.DataFrame({"a":[0,2,4,54,56,4],"b":[4,5,7,12,3,4]})

print(df[(df>5) & (df<33)])

    a     b
0 NaN   NaN
1 NaN   NaN
2 NaN   7.0
3 NaN  12.0
4 NaN   NaN
5 NaN   NaN

If want blank string instead of NaN:

df[(df>5) & (df<33)].fillna("")