RE: Drill 1.12 Unsupported type LIST

2017-12-18 Thread Lee, David 
Nevermind.. Somehow enable `exec.enable_union_type` got set to TRUE.

-Original Message-
From: Lee, David 
Sent: Monday, December 18, 2017 2:57 PM
To: user@drill.apache.org
Subject: Drill 1.12 Unsupported type LIST

After upgrading from 1.11 to 1.12 I'm getting the following error all the time 
now trying to convert JSON to Parquet files.
 
SYSTEM ERROR: UnsupportedOperationException: Unsupported type LIST Fragment 0:0

I even tried the following which also errored out:

A. Unarchived an old JSON file and tried to reconvert it under 1.12 (failed).
B. Converted an old parquet files to json (works). Tried to convert json 
(0_0_0.json) back to parquet (failed).


This message may contain information that is confidential or privileged. If you 
are not the intended recipient, please advise the sender immediately and delete 
this message. See 
http://www.blackrock.com/corporate/en-us/compliance/email-disclaimers for 
further information.  Please refer to 
http://www.blackrock.com/corporate/en-us/compliance/privacy-policy for more 
information about BlackRock’s Privacy Policy.

For a list of BlackRock's office addresses worldwide, see 
http://www.blackrock.com/corporate/en-us/about-us/contacts-locations.

© 2017 BlackRock, Inc. All rights reserved.

Drill 1.12 Unsupported type LIST

2017-12-18 Thread Lee, David 
After upgrading from 1.11 to 1.12 I'm getting the following error all the time 
now trying to convert JSON to Parquet files.
 
SYSTEM ERROR: UnsupportedOperationException: Unsupported type LIST Fragment 0:0

I even tried the following which also errored out:

A. Unarchived an old JSON file and tried to reconvert it under 1.12 (failed).
B. Converted an old parquet files to json (works). Tried to convert json 
(0_0_0.json) back to parquet (failed).


This message may contain information that is confidential or privileged. If you 
are not the intended recipient, please advise the sender immediately and delete 
this message. See 
http://www.blackrock.com/corporate/en-us/compliance/email-disclaimers for 
further information.  Please refer to 
http://www.blackrock.com/corporate/en-us/compliance/privacy-policy for more 
information about BlackRock’s Privacy Policy.

For a list of BlackRock's office addresses worldwide, see 
http://www.blackrock.com/corporate/en-us/about-us/contacts-locations.

© 2017 BlackRock, Inc. All rights reserved.

Re: Drill Push to Tableau, Error -

2017-12-18 Thread Andries Engelbrecht 
Does Tableau automatically generate the DML or is it user generated?
Is the table used for multiple sessions or only for a single session?
Does other session create the same/similar tables for use?

As Kunal mentioned temporary tables may the way to go.

--Andries


On 12/17/17, 9:36 AM, "Kunal Khatua"  wrote:

It looks like your system is trying to create the attributes table 
concurrently. Based on the name of the table, my hunch is that your workflow(s) 
is/are trying to generate tables with the same name (attributes) for use in 
reporting. 

The suggested way to work around this is to use a naming convention that 
hints to the report(s) that will leverage the tables.

So, instead of 2 or more reports trying to create the same 'attributes' 
table, how about you try something like .. 'attr_report1' .. 'attr_report2' , 
etc. ?

Also, if these tables are transient in nature, you can consider using the 
'create temp table as ...'  SQL. This will create temporary tables with the 
lifespan of the connection you're working on. The moment you close that 
connection, Drill will clean up. So, if you have multiple connections trying to 
create a temporary 'attributes' table, they will all be isolated. 

Hope that helps. Let us know how else you are using Drill.

Thanks
~ Kunal


-Original Message-
From: Kunal Khatua [mailto:kkha...@mapr.com] 
Sent: Sunday, December 17, 2017 9:30 AM
To: user@drill.apache.org
Subject: FW: Drill Push to Tableau, Error - 

Forwarded from d...@drill.apache.org

From: Spinn, Brandi [mailto:brandi.sp...@siriusxm.com]
Sent: Friday, December 15, 2017 1:46 PM
To: d...@drill.apache.org
Subject: Drill Push to Tableau, Error -

Hello,

We are currently running a project which is utilizing the Drill push to 
Tableau function to be able to work with our data sets, we are already working 
with Tableau regarding our needs and determined that this is our best course of 
action considering how large data sets are - over 2 million rows per day.

At the moment we have several visualizations we have published, but we are 
running in an issue each morning where some of them are not updating according 
to our schedules, and when we review the logs we find some the same type of 
"fatal" errors that do not always allow the visualizations to update.

I have reached out to our account rep through Tableau for possible 
guidance, however, you might be a more appropriate resource; any help you could 
provide would be appreciated, I have not been able to find much via the 
internets.

Below is a sample of the error codes we are seeing, please let me know you 
are able to assist or if you need any additional information. Thank you!

Auto_AOD_Content:288 - create table dfs.tmp.attributes as select distinct 
channels.marketingname as channelmarketingname, channels.streamingname as 
channelstreamingname, channels.channelguid as channelGuid, channels.channelid 
as channelId, categories.category_name as channelcategory, CASE when 
music.channel_guid is not null then 'Music' else null end as genre_Music, CASE 
when news.channel_guid is not null then 'News' else null end as genre_News, 
CASE when sports.channel_guid is not null then 'Sports' else null end as 
genre_Sports,  CASE when talk.channel_guid is not null then 'Talk' else null 
end as genre_Talk, CASE when howard.channel_guid is not null then 'Howard' else 
null end as genre_Howard, categories.channel_name channelName from 
dfs.root.`/SXM/archive/parsed/Channel-parsed-type2/2017-12-14*` channels join 
dfs.root.`/SXM/archive/parsed/category-parsed-type2/2017-12-14*` categories on 
categories.channel_guid=channels.channelguid left join (select channel_guid 
from dfs.root.`/SXM/archive/parsed/category-parsed-type2/2017-12-14*` where 
supercategory_name = 'Music') music on channels.channelguid = 
music.channel_guid left join (select channel_guid from 
dfs.root.`/SXM/archive/parsed/category-parsed-type2/2017-12-14*` where 
supercategory_name = 'News') news on channels.channelguid = news.channel_guid 
left join (select channel_guid from 
dfs.root.`/SXM/archive/parsed/category-parsed-type2/2017-12-14*` where 
supercategory_name = 'Sports') sports on channels.channelguid = 
sports.channel_guid left join (select channel_guid from 
dfs.root.`/SXM/archive/parsed/category-parsed-type2/2017-12-14*` where 
supercategory_name = 'Talk') talk on channels.channelguid = talk.channel_guid 
left join (select channel_guid from 
dfs.root.`/SXM/archive/parsed/category-parsed-type2/2017-12-14*` where 
supercategory_name = 'Howard') howard on channels.channelguid = 
howard.channel_guid

Auto_AOD_Content:304 - There was a SQL ERROR on DRILL side

Auto_AOD_Content:305 - VALIDATION ERROR: A table or view with given name 
[attributes] already exists in schema [dfs.tmp]

[SECURITY] CVE-2017-12630 Apache Drill XSS vulnerability

2017-12-18 Thread Arina Ielchiieva 
*CVE-2017-12630 Apache Drill XSS vulnerability*

*Severity*: Important

*Vendor:* The Apache Software Foundation

*Versions Affected:*
Apache Drill 1.11.0 and earlier

*Description*
In Apache Drill 1.11.0 and earlier when submitting form from Query page
users are able to pass arbitrary script or HTML which will take effect on
Profile page afterwards.

Example:
After submitting special script that returns cookie information from Query
page, malicious user may obtain this information from Profile page
afterwards.

*Mitigation:*
Users of the affected versions should upgrade to Apache Drill to 1.12.0 and
later.

*Credit:*
Sanjog Panda

Kind regards
Arina

[ANNOUNCE] Apache Drill 1.12.0 Released

2017-12-18 Thread Arina Ielchiieva 
On behalf of the Apache Drill community, I am happy to announce the release
of Apache Drill 1.12.0.

For information about Apache Drill, and to get involved, visit the project
website [1].

This release of Drill provides the following new features and improvements:

Kafka and OpenTSDB Storage Plugins (DRILL-4779, DRILL-5337)
Queue-Based Memory Assignment for Buffering Operators (Throttling)
(DRILL-5716)
Networking Functions (DRILL-5834)
SSL Support (DRILL-5431)
Network Encryption Support (DRILL-5682)
Access to Paths Outside the Current Workspace (DRILL-5964)

For the full list please see release notes [2].

The binary and source artifacts are available here [3].

Thanks to everyone in the community who contributed to this release!

1. https://drill.apache.org/
2. https://drill.apache.org/docs/apache-drill-1-12-0-release-notes/
3. https://drill.apache.org/download/