I am trying to figure out how I can apply cumulative functions to objects. For numbers there are several alternatives like cumsum
and cumcount
. There is also df.expanding which can be used with apply
. But the functions I pass to apply
do not work on objects.
import pandas as pd
df = pd.DataFrame({"C1": [1, 2, 3, 4],
"C2": [{"A"}, {"B"}, {"C"}, {"D"}],
"C3": ["A", "B", "C", "D"],
"C4": [["A"], ["B"], ["C"], ["D"]]})
df
Out:
C1 C2 C3 C4
0 1 {A} A [A]
1 2 {B} B [B]
2 3 {C} C [C]
3 4 {D} D [D]
In the dataframe I have integer values, sets, strings and lists. Now, if I try expanding().apply(sum)
I have the cumulative sum:
df.expanding().apply(sum)
Out[69]:
C1 C2 C3 C4
0 1.0 {A} A [A]
1 3.0 {B} B [B]
2 6.0 {C} C [C]
3 10.0 {D} D [D]
My expectation was, since summation is defined on lists and strings, I would get something like this:
C1 C2 C3 C4
0 1.0 {A} A [A]
1 3.0 {B} AB [A, B]
2 6.0 {C} ABC [A, B, C]
3 10.0 {D} ABCD [A, B, C, D]
I also tried something like this:
df.expanding().apply(lambda r: reduce(lambda x, y: x+y**2, r))
Out:
C1 C2 C3 C4
0 1.0 {A} A [A]
1 5.0 {B} B [B]
2 14.0 {C} C [C]
3 30.0 {D} D [D]
It works as I expect: previous result is x
and the current row value is y
. But I cannot reduce using x.union(y)
, for example.
So, my question is: Are there any alternatives to expanding
that I can use on objects? The example is just to show that expanding().apply()
is not working on object dtypes. I am looking for a general solution that supports applying functions to those two inputs: previous result and the current element.
See Question&Answers more detail:
os