How to loop through a table to make API call?

Question

I have a table in my database that I need to loop through and make and API call. It's only three columns but about 300,000 rows. I need to grab all three columns and pass it in the requests.get() call.

Is there a way of doing that without putting into an RDD? Or if putting it into an RDD is the only way, how can it be done? Sorry a bit new to spark so I'm working on my own.

question from:https://stackoverflow.com/questions/65835879/how-to-loop-through-a-table-to-make-api-call

Answer 1

Waitting for answers