Re: How to populate all possible combination values in columns using Spark SQL

[Edgardo Szrajber]

Once you pivot you can use CASE WHEN to calculate the True/false, then you can 
agregate.Bentzi

Sent from Yahoo Mail on Android 
 
  On Sat, May 9, 2020 at 21:18, Aakash Basu wrote:  
 I know how to pivot, but how to aggregate and pivot and build those True and 
False combination is the doubt.
On Fri 8 May, 2020, 1:31 PM Edgardo Szrajber,  wrote:

Have you checked the pivot function?Bentzi

Sent from Yahoo Mail on Android 
 
  On Thu, May 7, 2020 at 22:46, Aakash Basu wrote:  
 Hi,
I've updated the SO question with masked data, added year column and other 
requirement. Please take a look.
Hope this helps in solving the problem.
Thanks and regards,AB
On Thu 7 May, 2020, 10:59 AM Sonal Goyal,  wrote:

As mentioned in the comments on SO, can you provide a (masked) sample of the 
data? It will be easier to see what you are trying to do if you add the year 
column
Thanks,
Sonal
Nube Technologies 





On Thu, May 7, 2020 at 10:26 AM Aakash Basu  wrote:

Hi,
I've described the problem in Stack Overflow with a lot of detailing, can you 
kindly check and help if possible?
https://stackoverflow.com/q/61643910/5536733

I'd be absolutely fine if someone solves it using Spark SQL APIs rather than 
plain spark SQL query.
Thanks,Aakash.

  

  

Re: How to populate all possible combination values in columns using Spark SQL

[Aakash Basu]

I know how to pivot, but how to aggregate and pivot and build those True
and False combination is the doubt.

On Fri 8 May, 2020, 1:31 PM Edgardo Szrajber,  wrote:

> Have you checked the pivot function?
> Bentzi
>
> Sent from Yahoo Mail on Android
> 
>
> On Thu, May 7, 2020 at 22:46, Aakash Basu
>  wrote:
> Hi,
>
> I've updated the SO question with masked data, added year column and other
> requirement. Please take a look.
>
> Hope this helps in solving the problem.
>
> Thanks and regards,
> AB
>
> On Thu 7 May, 2020, 10:59 AM Sonal Goyal,  wrote:
>
> As mentioned in the comments on SO, can you provide a (masked) sample of
> the data? It will be easier to see what you are trying to do if you add the
> year column
>
> Thanks,
> Sonal
> Nube Technologies 
>
> 
>
>
>
>
> On Thu, May 7, 2020 at 10:26 AM Aakash Basu 
> wrote:
>
> Hi,
>
> I've described the problem in Stack Overflow with a lot of detailing, can
> you kindly check and help if possible?
>
> https://stackoverflow.com/q/61643910/5536733
>
> I'd be absolutely fine if someone solves it using Spark SQL APIs rather
> than plain spark SQL query.
>
> Thanks,
> Aakash.
>
>

Re: How to populate all possible combination values in columns using Spark SQL

[Edgardo Szrajber]

Have you checked the pivot function?Bentzi

Sent from Yahoo Mail on Android 
 
  On Thu, May 7, 2020 at 22:46, Aakash Basu wrote:  
 Hi,
I've updated the SO question with masked data, added year column and other 
requirement. Please take a look.
Hope this helps in solving the problem.
Thanks and regards,AB
On Thu 7 May, 2020, 10:59 AM Sonal Goyal,  wrote:

As mentioned in the comments on SO, can you provide a (masked) sample of the 
data? It will be easier to see what you are trying to do if you add the year 
column
Thanks,
Sonal
Nube Technologies 





On Thu, May 7, 2020 at 10:26 AM Aakash Basu  wrote:

Hi,
I've described the problem in Stack Overflow with a lot of detailing, can you 
kindly check and help if possible?
https://stackoverflow.com/q/61643910/5536733

I'd be absolutely fine if someone solves it using Spark SQL APIs rather than 
plain spark SQL query.
Thanks,Aakash.

  

Re: How to populate all possible combination values in columns using Spark SQL

[Aakash Basu]

Hi,

I've updated the SO question with masked data, added year column and other
requirement. Please take a look.

Hope this helps in solving the problem.

Thanks and regards,
AB

On Thu 7 May, 2020, 10:59 AM Sonal Goyal,  wrote:

> As mentioned in the comments on SO, can you provide a (masked) sample of
> the data? It will be easier to see what you are trying to do if you add the
> year column
>
> Thanks,
> Sonal
> Nube Technologies 
>
> 
>
>
>
>
> On Thu, May 7, 2020 at 10:26 AM Aakash Basu 
> wrote:
>
>> Hi,
>>
>> I've described the problem in Stack Overflow with a lot of detailing, can
>> you kindly check and help if possible?
>>
>> https://stackoverflow.com/q/61643910/5536733
>>
>> I'd be absolutely fine if someone solves it using Spark SQL APIs rather
>> than plain spark SQL query.
>>
>> Thanks,
>> Aakash.
>>
>

Re: How to populate all possible combination values in columns using Spark SQL

[Sonal Goyal]

As mentioned in the comments on SO, can you provide a (masked) sample of
the data? It will be easier to see what you are trying to do if you add the
year column

Thanks,
Sonal
Nube Technologies 






On Thu, May 7, 2020 at 10:26 AM Aakash Basu 
wrote:

> Hi,
>
> I've described the problem in Stack Overflow with a lot of detailing, can
> you kindly check and help if possible?
>
> https://stackoverflow.com/q/61643910/5536733
>
> I'd be absolutely fine if someone solves it using Spark SQL APIs rather
> than plain spark SQL query.
>
> Thanks,
> Aakash.
>

How to populate all possible combination values in columns using Spark SQL

[Aakash Basu]

Hi,

I've described the problem in Stack Overflow with a lot of detailing, can
you kindly check and help if possible?

https://stackoverflow.com/q/61643910/5536733

I'd be absolutely fine if someone solves it using Spark SQL APIs rather
than plain spark SQL query.

Thanks,
Aakash.