I want to save dataframe rows in the different S3 buckets. Let's assume all buckets exist. I have a simple dataframe:
Please use partitionBy clause as -
partitionBy
df.write. partitionBy("tenantId"). parquet("the root path")
It will create distinct folder by names "tenant1" and "tenant2" and put respective rows inside.
1.4m articles
1.4m replys
5 comments
56.9k users