How to use Pyspark UDF to update a particular column in Mysql table?

Question

I have a dataframe which contains two column 'mobile' & 'totalcount' and I want to update 'totalcount' column of the dataframe in mysql table in which 'totalcount' column is already present. I was doing this with simple 'for' loop but it's taking too long to update. How can I speed up my process or how can I use UDF functions to do the same?

for row in df.rdd.collect():
    query = f"UPDATE dummy.member_report SET totalcount = '{row.totalcount}' WHERE mobile = '{row.mobile}'"        
    Mysql_insert(spark,query)
    print("finsished")

question from:https://stackoverflow.com/questions/66045820/how-to-use-pyspark-udf-to-update-a-particular-column-in-mysql-table

Answer 1

Waitting for answers