AFAICT, we can use spark.sql(s"select $name ..."), name is a value in
Scala context[1].
--
Cheers,
-z
[1] https://docs.scala-lang.org/overviews/core/string-interpolation.html
On Fri, 17 Apr 2020 00:10:59 +0100
Mich Talebzadeh wrote:
> Thanks Patrick,
>
> The partition broadcastId is static
Thanks Patrick,
The partition broadcastId is static as defined as a value below
val broadcastValue = "123456789" // I assume this will be sent as a
constant for the batch
// Create a DF on top of XML
val df = spark.read.
format("com.databricks.spark.xml").
What happens if you change your insert statement to be
INSERT INTO TABLE michtest.BroadcastStaging PARTITION (broadcastId =
broadcastValue, brand)
and then add the value for brand into the select as
SELECT
ocis_party_id AS partyId
, target_mobile_no AS phoneNumber
,
Hi,
I have an XML file that is read into Spark using Databa bricks jar file
spark-xml_2.11-0.9.0.jar
Doing some tests
This is the format of XML (one row here)
//*
SKY
0123456789
123456789
XYZ
GLX
12345678
*/
OK I am trying to insert data into a hive partitioned table through
