Do any of the Spark SQL 1.x or 2.0 api’s allow reading from a rest endpoint
consuming json or xml?
For example, in the 2.0 context I’ve tried the following attempts with varying
errors:
val conf = new SparkConf().setAppName("http test").setMaster("local[2]")
val builder = SparkSession.builder.config(conf)
val spark = builder.getOrCreate()
val df1 = spark.read.json("http://server/ws/v1/blah";)
val df2 = spark.read.format("json").load("http://server/ws/v1/blah";)
val df3 = spark.read.format("json").textFile("http://server/ws/v1/blah";)
So far I haven’t seen anything but a reference to someone @ Databricks that
might have displayed something like this at a meetup/conference but I couldn’t
find any code or databricks projects.
Anyone else done this & could provide a quick example?
Thanks,
Robert
