Is there any plans for supporting UDAnF?

[吴朱华]

Hi guys:

Some of my clients are consider switching from vertica to impala, but they
really need the UDAnF to speed up the query. Is there any plans for
supporting UDAnF?^_^

Can a committer please carry the +2 on this change?

[Lars Volker]

Can a committer please carry the +2 on this change?

https://gerrit.cloudera.org/#/c/5453/

Patch set 2 had a +2 from Dan.

Thanks, Lars

Re: Do we have something like git-clang-format for python/PEP8?

[Martin Grund (Das Grundprinzip.de)]

There is something called autoflake8 it can't solve all issues due to the
way that python only has limited cases for line continuations.

Martin

On Thu, Dec 15, 2016, 13:36 Lars Volker  wrote:

> Hi all,
>
> do we have a tool to format only the changed lines in python files
> according to PEP8, similar to what git-clang-format does for c++ files?
>
> Thanks, Lars
>

[Toolchain-CR] Add autotools packages as build prerequisites

[Tim Armstrong (Code Review)]

Tim Armstrong has posted comments on this change.

Change subject: Add autotools packages as build prerequisites
..


Patch Set 1: Code-Review+2

-- 
To view, visit http://gerrit.cloudera.org:8080/5524
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I43218d1bdd4d660f1e50774a90de1733b4be10ee
Gerrit-PatchSet: 1
Gerrit-Project: Toolchain
Gerrit-Branch: master
Gerrit-Owner: Henry Robinson 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-HasComments: No

[Toolchain-CR] IMPALA-4652: Add crcutil to toolchain

[Henry Robinson (Code Review)]

Henry Robinson has posted comments on this change.

Change subject: IMPALA-4652: Add crcutil to toolchain
..


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/5522/2//COMMIT_MSG
Commit Message:

PS2, Line 13: The package builds on all supported platforms, except for CentOS 
5.X due
: to the outdated autotools on that platform. As a workaround, the 
script
: will download and locally install autoconf / automake / libtool 
if it
: detects it is on CentOS 5.X. Doing so only adds a few seconds to 
the build.
Will remove this.


-- 
To view, visit http://gerrit.cloudera.org:8080/5522
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ibf8b39082914b1b2932b4ce7efbd2cc4f5f69743
Gerrit-PatchSet: 2
Gerrit-Project: Toolchain
Gerrit-Branch: master
Gerrit-Owner: Henry Robinson 
Gerrit-Reviewer: Henry Robinson 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-HasComments: Yes

[Toolchain-CR] Add autotools packages as build prerequisites

[Henry Robinson (Code Review)]

Henry Robinson has uploaded a new change for review.

  http://gerrit.cloudera.org:8080/5524

Change subject: Add autotools packages as build prerequisites
..

Add autotools packages as build prerequisites

* Add autoconf, automake and libtool.
* Versions are those installed in Ubuntu 14.04.

Change-Id: I43218d1bdd4d660f1e50774a90de1733b4be10ee
---
M init.sh
A source/autoconf/build.sh
A source/automake/build.sh
A source/libtool/build.sh
4 files changed, 132 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Toolchain refs/changes/24/5524/1
-- 
To view, visit http://gerrit.cloudera.org:8080/5524
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I43218d1bdd4d660f1e50774a90de1733b4be10ee
Gerrit-PatchSet: 1
Gerrit-Project: Toolchain
Gerrit-Branch: master
Gerrit-Owner: Henry Robinson 

[Toolchain-CR] IMPALA-4652: Add crcutil to toolchain

[Henry Robinson (Code Review)]

Henry Robinson has uploaded a new patch set (#2).

Change subject: IMPALA-4652: Add crcutil to toolchain
..

IMPALA-4652: Add crcutil to toolchain

Note: crcutil does not have any release tarballs, so we borrow the one
built for Apache Kudu. As a result, the build version is a git hash,
rather than e.g. 1.0.

The package builds on all supported platforms, except for CentOS 5.X due
to the outdated autotools on that platform. As a workaround, the script
will download and locally install autoconf / automake / libtool if it
detects it is on CentOS 5.X. Doing so only adds a few seconds to the build.

Change-Id: Ibf8b39082914b1b2932b4ce7efbd2cc4f5f69743
---
M buildall.sh
A source/crcutil/build.sh
2 files changed, 44 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Toolchain refs/changes/22/5522/2
-- 
To view, visit http://gerrit.cloudera.org:8080/5522
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ibf8b39082914b1b2932b4ce7efbd2cc4f5f69743
Gerrit-PatchSet: 2
Gerrit-Project: Toolchain
Gerrit-Branch: master
Gerrit-Owner: Henry Robinson 
Gerrit-Reviewer: Henry Robinson 
Gerrit-Reviewer: Tim Armstrong 

[Toolchain-CR] IMPALA-4652: Add crcutil to toolchain

[Tim Armstrong (Code Review)]

Tim Armstrong has posted comments on this change.

Change subject: IMPALA-4652: Add crcutil to toolchain
..


Patch Set 1:

IMO we should just add autotools as a toolchain package. We could install it 
and add it to the path as part of init.sh like cmake and binutils.

We don't really have a firm guarantee that other distros come with specific 
versions of autotools so that would be another step towards the toolchain 
builds being more reproducible

-- 
To view, visit http://gerrit.cloudera.org:8080/5522
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ibf8b39082914b1b2932b4ce7efbd2cc4f5f69743
Gerrit-PatchSet: 1
Gerrit-Project: Toolchain
Gerrit-Branch: master
Gerrit-Owner: Henry Robinson 
Gerrit-Reviewer: Henry Robinson 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-HasComments: No

[Toolchain-CR] IMPALA-4652: Add crcutil to toolchain

[Henry Robinson (Code Review)]

Henry Robinson has uploaded a new change for review.

  http://gerrit.cloudera.org:8080/5522

Change subject: IMPALA-4652: Add crcutil to toolchain
..

IMPALA-4652: Add crcutil to toolchain

Note: crcutil does not have any release tarballs, so we borrow the one
built for Apache Kudu. As a result, the build version is a git hash,
rather than e.g. 1.0.

The package builds on all supported platforms, except for CentOS 5.X due
to the outdated autotools on that platform. As a workaround, the script
will download and locally install autoconf / automake / libtool if it
detects it is on CentOS 5.X. Doing so only adds a few seconds to the build.

Change-Id: Ibf8b39082914b1b2932b4ce7efbd2cc4f5f69743
---
M buildall.sh
A source/crcutil/build.sh
2 files changed, 73 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Toolchain refs/changes/22/5522/1
-- 
To view, visit http://gerrit.cloudera.org:8080/5522
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ibf8b39082914b1b2932b4ce7efbd2cc4f5f69743
Gerrit-PatchSet: 1
Gerrit-Project: Toolchain
Gerrit-Branch: master
Gerrit-Owner: Henry Robinson 

[Toolchain-CR] IMPALA-4477: Bump Kudu to latest commit on master (e018a83)

[Matthew Jacobs (Code Review)]

Matthew Jacobs has submitted this change and it was merged.

Change subject: IMPALA-4477: Bump Kudu to latest commit on master (e018a83)
..


IMPALA-4477: Bump Kudu to latest commit on master (e018a83)

e018a837ce4808dfeb828ed3ac268fa4cac93a8f

Change-Id: I6fb47f30dc6c6478a125d5d4df5be11b5797e2df
---
M buildall.sh
1 file changed, 5 insertions(+), 2 deletions(-)

Approvals:
  Lars Volker: Looks good to me, approved
  Matthew Jacobs: Verified



-- 
To view, visit http://gerrit.cloudera.org:8080/5460
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I6fb47f30dc6c6478a125d5d4df5be11b5797e2df
Gerrit-PatchSet: 2
Gerrit-Project: Toolchain
Gerrit-Branch: master
Gerrit-Owner: Matthew Jacobs 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Matthew Jacobs 

[Toolchain-CR] IMPALA-4477: Bump Kudu to latest commit on master (e018a83)

[Matthew Jacobs (Code Review)]

Matthew Jacobs has posted comments on this change.

Change subject: IMPALA-4477: Bump Kudu to latest commit on master (e018a83)
..


Patch Set 2: Verified+1

-- 
To view, visit http://gerrit.cloudera.org:8080/5460
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I6fb47f30dc6c6478a125d5d4df5be11b5797e2df
Gerrit-PatchSet: 2
Gerrit-Project: Toolchain
Gerrit-Branch: master
Gerrit-Owner: Matthew Jacobs 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Matthew Jacobs 
Gerrit-HasComments: No

[Toolchain-CR] IMPALA-4549: consistently enforce that boost gregorian year <= 9999

[Tim Armstrong (Code Review)]

Hello Taras Bobrovytsky,

I'd like you to reexamine a change.  Please visit

http://gerrit.cloudera.org:8080/5264

to look at the new patch set (#4).

Change subject: IMPALA-4549: consistently enforce that boost gregorian year <= 

..

IMPALA-4549: consistently enforce that boost gregorian year <= 

Adds a Boost patch that makes the documentation and code consistent on
whether the max supported year is  or 1. The documentation and
the max_date value are both , so fix cases where it is 1.

I also filed a bug against boost https://svn.boost.org/trac/boost/ticket/12630

Change-Id: I65e4912b407ae43e281e8fb8153c89cd12e2e237
---
M buildall.sh
A source/boost/boost-1.57.0-patches/0001-greg-year-range.patch
2 files changed, 34 insertions(+), 4 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Toolchain refs/changes/64/5264/4
-- 
To view, visit http://gerrit.cloudera.org:8080/5264
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I65e4912b407ae43e281e8fb8153c89cd12e2e237
Gerrit-PatchSet: 4
Gerrit-Project: Toolchain
Gerrit-Branch: master
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Taras Bobrovytsky 
Gerrit-Reviewer: Tim Armstrong 

Re: Impala-Alluxio

[黄志]

one more thing, how Impala access s3? use the DFSClient in HDFS?Have you 
added code for accessing s3? Or you just simply add a hadoop-aws jar?
Is there book about the code design of Impala? For example, what the fe 
nodule mean and for?And what the be module mean?

在 2016年12月15日星期四 UTC+8下午2:15:45，黄志写道：
>
> Hi, I  see the fe module uses hadoop-aws.jar to support S3, and I see the 
> wrapper class in it, can I use the same way to use Alluxio? and is there 
> any need to add code in Impala?
>
> 在 2016年12月15日星期四 UTC+8下午1:07:02，黄志写道：
>>
>> I see it!Thanks,I want to know how Impala uses S3, I did't see the code 
>> in Impala about choosing s3 and invoke s3's API
>>
>> 在 2016年12月14日星期三 UTC+8下午11:47:33，Tim Armstrong写道：
>>>
>>> See https://wiki.apache.org/hadoop/AmazonS3 and S3AFilesystem.java in 
>>> the Hadoop source tree.
>>>
>>> It looks like Alluxio also has a similar filesystem implementation. I 
>>> don't know anything about Alluxio or how well it works in practce, just 
>>> giving pointers in case you are interested in developing the support.
>>>
>>> On Tue, Dec 13, 2016 at 10:01 PM, 黄志  wrote:
>>>
 Hi, I am interested in this, I saw Impala use boto3 to implement 
 S3Client, but Alluxio only offer Java API,how do we use it?and where is 
 S3AFileSystem?

 在 2016年12月9日星期五 UTC+8上午1:34:42，Tim Armstrong写道：
>
> It looks like Alluxio provides some kind of Hadoop Filesystem API 
> wrapper: 
> http://www.alluxio.org/docs/master/en/File-System-API.html#hadoop-api. 
> I don't know if it's been upstreamed into Apache Hadoop.
>
> We do use similar Hadoop Filesystem API adapters to provide S3 and 
> Isilon support without massive code changes so it seems likely that it's 
> possible to do the same for Alluxio. Our experience with S3 and Isilon 
> was 
> that it required additional work to get it to work and perform well 
> because 
> different filesystems have different quirks even if they implement the 
> same 
> interface.
>
> It would require at least some small code changes to switch it on 
> (e.g. you can grep for S3AFilesystem to see how we turned on S3 support). 
> Please let us know if you experiment with it and have any success (or no 
> success).
>
> - Tim
>
> On Thu, Dec 8, 2016 at 9:07 AM, Jim Apple  wrote:
>
>> I don't know that Impala has been tested with Alluxio.
>>
>> This mailing list is deprecated; please use 
>> d...@impala.incubator.apache.org
>>
>>
>> On Wednesday, December 7, 2016 at 11:08:59 PM UTC-8, 
>> andong@gmail.com wrote:
>>>
>>> Hi
>>> How are you configure the impala on the alluxio;
>>>
>>> Alluxio: Open Source Memory Speed Virtual Distributed Storage
>>> Alluxio: http://alluxio.org/
>>>
>> -- 
>> You received this message because you are subscribed to the Google 
>> Groups "Impala Dev" group.
>> To unsubscribe from this group and stop receiving emails from it, 
>> send an email to impala-dev+...@cloudera.org.
>>
>
> -- 
 You received this message because you are subscribed to the Google 
 Groups "Impala Dev" group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to impala-dev+...@cloudera.org.

>>>
>>>