Re: Connect the two tables in spark sql
If you are using Spark-sql as opposed to spark-shell, then you can just use
UNION as in SQL for this. Pretty straight forward.
SELECT * from TABLE_A
UNION
SELECT * from TABLE_B
ORDER BY COLUMN_A, COLUMN_B;
Example
spark-sql> SELECT * FROM dummy where id = 1
> UNION
> SELECT * FROM dummy2 where id = 10
> ORDER by id;
HTH
Dr Mich Talebzadeh
LinkedIn *
https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
http://talebzadehmich.wordpress.com
On 2 March 2016 at 06:40, Mao, Wei <wei@intel.com> wrote:
> It should be a “union” operation instead of “join”.
>
>
>
> And besides from Ted’s answer, if you are working with DataSet API:
>
>
>
> *def **union(other: Dataset[**T**]): Dataset[**T**] = withPlan[**T**](other){
> (left, right) =>*
>
>
>
> Thanks,
>
> William
>
>
>
> *From:* Ted Yu [mailto:yuzhih...@gmail.com]
> *Sent:* Wednesday, March 2, 2016 11:41 AM
> *To:* Angel Angel
> *Cc:* user
> *Subject:* Re: Connect the two tables in spark sql
>
>
>
> You only showed one record from each table.
>
>
>
> Have you looked at the following method in DataFrame ?
>
>
>
> def unionAll(other: DataFrame): DataFrame = withPlan {
>
>
>
> On Tue, Mar 1, 2016 at 7:13 PM, Angel Angel <areyouange...@gmail.com>
> wrote:
>
> Hello Sir/Madam,
>
>
>
> I am using the spark sql for the data operation.
>
>
>
> I have two tables with the same fields.
>
>
>
> Table 1
>
> name
>
> address
>
> phone Number
>
> sagar
>
> india
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Table 2
>
> name
>
> address
>
> phone Number
>
> jaya
>
> india
>
> 222
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> I want to join this tables like the following way
>
>
> Result Table
>
> name
>
> address
>
> phone Number
>
> jaya
>
> india
>
> 222
>
> sagar
>
> india
>
>
>
>
>
>
>
> How can i join this table. I tried using the join command it add the table
> to left side.
>
>
>
> Please help me to solve this query.
>
>
>
> Thanks,
>
>
>
RE: Connect the two tables in spark sql
It should be a “union” operation instead of “join”.
And besides from Ted’s answer, if you are working with DataSet API:
def union(other: Dataset[T]): Dataset[T] = withPlan[T](other){ (left, right) =>
Thanks,
William
From: Ted Yu [mailto:yuzhih...@gmail.com]
Sent: Wednesday, March 2, 2016 11:41 AM
To: Angel Angel
Cc: user
Subject: Re: Connect the two tables in spark sql
You only showed one record from each table.
Have you looked at the following method in DataFrame ?
def unionAll(other: DataFrame): DataFrame = withPlan {
On Tue, Mar 1, 2016 at 7:13 PM, Angel Angel
<areyouange...@gmail.com<mailto:areyouange...@gmail.com>> wrote:
Hello Sir/Madam,
I am using the spark sql for the data operation.
I have two tables with the same fields.
Table 1
name
address
phone Number
sagar
india
Table 2
name
address
phone Number
jaya
india
222
I want to join this tables like the following way
Result Table
name
address
phone Number
jaya
india
222
sagar
india
How can i join this table. I tried using the join command it add the table to
left side.
Please help me to solve this query.
Thanks,
Re: Connect the two tables in spark sql
You only showed one record from each table.
Have you looked at the following method in DataFrame ?
def unionAll(other: DataFrame): DataFrame = withPlan {
On Tue, Mar 1, 2016 at 7:13 PM, Angel Angel wrote:
> Hello Sir/Madam,
>
> I am using the spark sql for the data operation.
>
> I have two tables with the same fields.
>
> Table 1
> name address phone Number
> sagar india
>
>
>
>
>
> Table 2
> name address phone Number
> jaya india 222
>
>
>
>
> I want to join this tables like the following way
>
>
> Result Table
> name address phone Number
> jaya india 222
> sagar india
>
> How can i join this table. I tried using the join command it add the table
> to left side.
>
> Please help me to solve this query.
>
> Thanks,
>
Connect the two tables in spark sql
Hello Sir/Madam, I am using the spark sql for the data operation. I have two tables with the same fields. Table 1 name address phone Number sagar india Table 2 name address phone Number jaya india 222 I want to join this tables like the following way Result Table name address phone Number jaya india 222 sagar india How can i join this table. I tried using the join command it add the table to left side. Please help me to solve this query. Thanks,
