I am using python 3.7 and Spark 2.4.7 I am not sure what the best way to do this is. I have a dataframe with a url in one of the columns, and I want to download the contents of that url and put it in a new column. Can someone point me in the right direction on how to do this?I looked at the UDFs and they seem confusing to me. Also, is there a good way to rate limit the number of calls I make per second?
- Spark 2.4.7 Harry Jamison
- Re: Spark 2.4.7 Varun Shah
- Re: Spark 2.4.7 Harry Jamison
- Re: Spark 2.4.7 Mich Talebzadeh
- Re: Spark 2.4.7 Mich Talebzadeh