python - Pandas: How to use applymap/apply function with arguements to a dataframe without looping

Question

Background

Link 1 shows that apply can be applied to a Series. I want to use the apply function on a subset of a DataFrame without looping through the columns.

Example code

Creating a sample DataFrame of size 7, 7

def f_test_df(n_rows, n_cols):
    df1 = pd.DataFrame(np.random.rand(n_rows, n_cols))
    df = df1.applymap(lambda x: round(x*10))
    return df 

np.random.seed(seed=1)
df1 = f_test_df(7, 7)

Desired function is should return the same value if the number is within a predefined range, else based on whether it's on the lower or upper side of the limit, corresponding values should be returned. The function to be applied is as follows:

def f_bounds(x, lower, upper):
    if x < lower:
        return 'lower'
    elif x > upper:
        return 'upper'
    else:
        return x

The selected portion of the DataFrame where a function needs to be applied

df1.loc[2:5, 2:5]

Applying the function:

lower = 2
upper = 5
df1.loc[2:5, 2:5].apply(f_bounds, args=(lower, upper))

I encountered the following error:

ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().

Hence, I changed the approach and used looping across the columns, as shown below (which works well):

for j in range(2, 5):
    print(df1.loc[2:5, j].apply(f_bounds, args=(lower, upper)))

Other approaches that were not tested

Link 2 Referring to answer 2 here, advised against using applymap with arguments. So, I did not use applymap because the function requires 2 additional arguments. Readers please note, applymap has been used in the answer.

Desired outcome

I want to implement this function requiring arguements without looping over the columns to a dataframe.

question from:https://stackoverflow.com/questions/66058705/pandas-how-to-use-applymap-apply-function-with-arguements-to-a-dataframe-withou

Answer 1

You can directly use using applymap with a lambda function that takes in the parameters on the window of the DataFrame. Then you can update the view directly to update the original DataFrame -

df1.loc[2:5, 2:5] = df1.loc[2:5, 2:5].applymap(lambda x: f_bounds(x, lower, upper))
print(df1)

    0  1      2      3      4      5  6
0   4  7      0      3      1      1  2
1   3  4      5      4      7      2  9
2   0  7      4  upper  lower      2  8
3  10  3  upper  upper  upper  lower  0
4   2  9  lower      4  upper      5  7
5   3  7  upper  lower  upper  upper  7
6   3  8      1      4      9      3  3

---

EDIT:

Here is another way to do what you are trying to do without using apply or applymap

cond1 = df1[(df1.loc[2:5, 2:5]<lower)].notna()
cond2 = df1[(df1.loc[2:5, 2:5]>upper)].notna() 
df1_new = df1.where(~cond1, 'lower').where(~cond2, 'upper')
print(df1_new)

    0  1      2      3      4      5  6
0   4  7      0      3      1      1  2
1   3  4      5      4      7      2  9
2   0  7      4  upper  lower      2  8
3  10  3  upper  upper  upper  lower  0
4   2  9  lower      4  upper      5  7
5   3  7  upper  lower  upper  upper  7
6   3  8      1      4      9      3  3