python - Better approach to f.when in Pyspark

Question

I'm trying to use a case when in Pyspark and I've found 2 ways to do it:

1st way:

f.expr('count(distinct case when client_ticket==1 then ticket_id else null end)').alias('tickets_clientpay'),
            

2nd way (now is producing error) :

 f.when((f.col('client_ticket')==1),f.countDistinct('ticket_id')).alias('tickets_lclientpay'),

I'm personally more confortable with 2nd way, the problem is that I'm getting errors all the time because of the (). Can someone explain me the logic to put () and where? I'm a bit lost and I'm losing time trying attemps all the time.

Thanks!

question from:https://stackoverflow.com/questions/65843943/better-approach-to-f-when-in-pyspark

Answer 1

You can use this syntax. Put countDistinct outside when, not inside.

f.countDistinct(
    f.when(
        f.col('client_ticket') == 1,
        f.col('ticket_id')
    )
).alias('tickets_lclientpay')