[jira] [Commented] (HIVE-11595) refactor ORC footer reading to make it usable from outside
[ https://issues.apache.org/jira/browse/HIVE-11595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14720978#comment-14720978 ] Lefty Leverenz commented on HIVE-11595: --- Any doc needed for this? > refactor ORC footer reading to make it usable from outside > -- > > Key: HIVE-11595 > URL: https://issues.apache.org/jira/browse/HIVE-11595 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 2.0.0 > > Attachments: HIVE-10595.patch, HIVE-11595.01.patch, > HIVE-11595.02.patch, HIVE-11595.03.patch, HIVE-11595.04.patch > > > If ORC footer is read from cache, we want to parse it without having the > reader, opening a file, etc. I thought it would be as simple as protobuf > parseFrom bytes, but apparently there's bunch of stuff going on there. It > needs to be accessible via something like parseFrom(ByteBuffer), or similar. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11595) refactor ORC footer reading to make it usable from outside
[
https://issues.apache.org/jira/browse/HIVE-11595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14720883#comment-14720883
]
Hive QA commented on HIVE-11595:
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12753038/HIVE-11595.04.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 9380 tests executed
*Failed tests:*
{noformat}
org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5106/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5106/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5106/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12753038 - PreCommit-HIVE-TRUNK-Build
> refactor ORC footer reading to make it usable from outside
> --
>
> Key: HIVE-11595
> URL: https://issues.apache.org/jira/browse/HIVE-11595
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-10595.patch, HIVE-11595.01.patch,
> HIVE-11595.02.patch, HIVE-11595.03.patch, HIVE-11595.04.patch
>
>
> If ORC footer is read from cache, we want to parse it without having the
> reader, opening a file, etc. I thought it would be as simple as protobuf
> parseFrom bytes, but apparently there's bunch of stuff going on there. It
> needs to be accessible via something like parseFrom(ByteBuffer), or similar.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-11595) refactor ORC footer reading to make it usable from outside
[ https://issues.apache.org/jira/browse/HIVE-11595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14720790#comment-14720790 ] Sergey Shelukhin commented on HIVE-11595: - Will commit after HiveQA > refactor ORC footer reading to make it usable from outside > -- > > Key: HIVE-11595 > URL: https://issues.apache.org/jira/browse/HIVE-11595 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-10595.patch, HIVE-11595.01.patch, > HIVE-11595.02.patch, HIVE-11595.03.patch, HIVE-11595.04.patch > > > If ORC footer is read from cache, we want to parse it without having the > reader, opening a file, etc. I thought it would be as simple as protobuf > parseFrom bytes, but apparently there's bunch of stuff going on there. It > needs to be accessible via something like parseFrom(ByteBuffer), or similar. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11595) refactor ORC footer reading to make it usable from outside
[ https://issues.apache.org/jira/browse/HIVE-11595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14720332#comment-14720332 ] Prasanth Jayachandran commented on HIVE-11595: -- +1 on the new change > refactor ORC footer reading to make it usable from outside > -- > > Key: HIVE-11595 > URL: https://issues.apache.org/jira/browse/HIVE-11595 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-10595.patch, HIVE-11595.01.patch, > HIVE-11595.02.patch, HIVE-11595.03.patch, HIVE-11595.04.patch > > > If ORC footer is read from cache, we want to parse it without having the > reader, opening a file, etc. I thought it would be as simple as protobuf > parseFrom bytes, but apparently there's bunch of stuff going on there. It > needs to be accessible via something like parseFrom(ByteBuffer), or similar. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11595) refactor ORC footer reading to make it usable from outside
[ https://issues.apache.org/jira/browse/HIVE-11595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14717818#comment-14717818 ] Prasanth Jayachandran commented on HIVE-11595: -- +1 > refactor ORC footer reading to make it usable from outside > -- > > Key: HIVE-11595 > URL: https://issues.apache.org/jira/browse/HIVE-11595 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-10595.patch, HIVE-11595.01.patch, > HIVE-11595.02.patch, HIVE-11595.03.patch > > > If ORC footer is read from cache, we want to parse it without having the > reader, opening a file, etc. I thought it would be as simple as protobuf > parseFrom bytes, but apparently there's bunch of stuff going on there. It > needs to be accessible via something like parseFrom(ByteBuffer), or similar. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11595) refactor ORC footer reading to make it usable from outside
[ https://issues.apache.org/jira/browse/HIVE-11595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14717814#comment-14717814 ] Sergey Shelukhin commented on HIVE-11595: - Full footer buffer is not always available, see the comments. I considered that, but the omnly way around this is to have "fake" full footer buffer with fake offsets into it (ie the buffer will only have the footer and offsets wil cover the entire buffer), but it's hacky > refactor ORC footer reading to make it usable from outside > -- > > Key: HIVE-11595 > URL: https://issues.apache.org/jira/browse/HIVE-11595 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-10595.patch, HIVE-11595.01.patch, > HIVE-11595.02.patch, HIVE-11595.03.patch > > > If ORC footer is read from cache, we want to parse it without having the > reader, opening a file, etc. I thought it would be as simple as protobuf > parseFrom bytes, but apparently there's bunch of stuff going on there. It > needs to be accessible via something like parseFrom(ByteBuffer), or similar. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11595) refactor ORC footer reading to make it usable from outside
[ https://issues.apache.org/jira/browse/HIVE-11595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14717456#comment-14717456 ] Prasanth Jayachandran commented on HIVE-11595: -- Mostly looks good. I am concerned about having references to two copies of footer (footerBuffer and fullFooterBuffer). I am guessing footerBuffer is subset of fullFooterBuffer (includes metadata + ps). Can we store the postscript length and footer length in the FileMetaInfo? So that, we can seek to postscript length - footer length and read footer length bytes to extract the footer alone. > refactor ORC footer reading to make it usable from outside > -- > > Key: HIVE-11595 > URL: https://issues.apache.org/jira/browse/HIVE-11595 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-10595.patch, HIVE-11595.01.patch, > HIVE-11595.02.patch, HIVE-11595.03.patch > > > If ORC footer is read from cache, we want to parse it without having the > reader, opening a file, etc. I thought it would be as simple as protobuf > parseFrom bytes, but apparently there's bunch of stuff going on there. It > needs to be accessible via something like parseFrom(ByteBuffer), or similar. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11595) refactor ORC footer reading to make it usable from outside
[ https://issues.apache.org/jira/browse/HIVE-11595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14717414#comment-14717414 ] Sergey Shelukhin commented on HIVE-11595: - [~prasanth_j] ping? > refactor ORC footer reading to make it usable from outside > -- > > Key: HIVE-11595 > URL: https://issues.apache.org/jira/browse/HIVE-11595 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-10595.patch, HIVE-11595.01.patch, > HIVE-11595.02.patch, HIVE-11595.03.patch > > > If ORC footer is read from cache, we want to parse it without having the > reader, opening a file, etc. I thought it would be as simple as protobuf > parseFrom bytes, but apparently there's bunch of stuff going on there. It > needs to be accessible via something like parseFrom(ByteBuffer), or similar. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11595) refactor ORC footer reading to make it usable from outside
[ https://issues.apache.org/jira/browse/HIVE-11595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14710245#comment-14710245 ] Sergey Shelukhin commented on HIVE-11595: - [~prasanth_j] failures are unrelated :) > refactor ORC footer reading to make it usable from outside > -- > > Key: HIVE-11595 > URL: https://issues.apache.org/jira/browse/HIVE-11595 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-10595.patch, HIVE-11595.01.patch, > HIVE-11595.02.patch, HIVE-11595.03.patch > > > If ORC footer is read from cache, we want to parse it without having the > reader, opening a file, etc. I thought it would be as simple as protobuf > parseFrom bytes, but apparently there's bunch of stuff going on there. It > needs to be accessible via something like parseFrom(ByteBuffer), or similar. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11595) refactor ORC footer reading to make it usable from outside
[
https://issues.apache.org/jira/browse/HIVE-11595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14710168#comment-14710168
]
Hive QA commented on HIVE-11595:
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12752073/HIVE-11595.03.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 9377 tests executed
*Failed tests:*
{noformat}
org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5055/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5055/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5055/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12752073 - PreCommit-HIVE-TRUNK-Build
> refactor ORC footer reading to make it usable from outside
> --
>
> Key: HIVE-11595
> URL: https://issues.apache.org/jira/browse/HIVE-11595
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-10595.patch, HIVE-11595.01.patch,
> HIVE-11595.02.patch, HIVE-11595.03.patch
>
>
> If ORC footer is read from cache, we want to parse it without having the
> reader, opening a file, etc. I thought it would be as simple as protobuf
> parseFrom bytes, but apparently there's bunch of stuff going on there. It
> needs to be accessible via something like parseFrom(ByteBuffer), or similar.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-11595) refactor ORC footer reading to make it usable from outside
[
https://issues.apache.org/jira/browse/HIVE-11595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14709836#comment-14709836
]
Hive QA commented on HIVE-11595:
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12752057/HIVE-11595.02.patch
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5053/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5053/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5053/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ [[ -n /usr/java/jdk1.7.0_45-cloudera ]]
+ export JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
+ JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
+ export
PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.7.0_45-cloudera/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin
+
PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.7.0_45-cloudera/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-TRUNK-Build-5053/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
92bd50e..f4aac7e branch-1 -> origin/branch-1
9d9dd72..5e16d53 hbase-metastore -> origin/hbase-metastore
a16bbd4..dd2bdfc master -> origin/master
+ git reset --hard HEAD
HEAD is now at a16bbd4 HIVE-11176 : Caused by: java.lang.ClassCastException:
org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct cannot be cast to
[Ljava.lang.Object; (Navis via Ashutosh Chauhan)
+ git clean -f -d
Removing ql/src/test/queries/clientpositive/pointlookup.q
Removing ql/src/test/queries/clientpositive/pointlookup2.q
Removing ql/src/test/results/clientpositive/pointlookup.q.out
Removing ql/src/test/results/clientpositive/pointlookup2.q.out
+ git checkout master
Already on 'master'
Your branch is behind 'origin/master' by 4 commits, and can be fast-forwarded.
+ git reset --hard origin/master
HEAD is now at dd2bdfc HIVE-11469 : Update doc for InstanceCache to clearly
define the contract on the SeedObject (Swarnim Kulkarni via Ashutosh Chauhan)
+ git merge --ff-only origin/master
Already up-to-date.
+ git gc
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh
/data/hive-ptest/working/scratch/build.patch
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12752057 - PreCommit-HIVE-TRUNK-Build
> refactor ORC footer reading to make it usable from outside
> --
>
> Key: HIVE-11595
> URL: https://issues.apache.org/jira/browse/HIVE-11595
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-10595.patch, HIVE-11595.01.patch,
> HIVE-11595.02.patch
>
>
> If ORC footer is read from cache, we want to parse it without having the
> reader, opening a file, etc. I thought it would be as simple as protobuf
> parseFrom bytes, but apparently there's bunch of stuff going on there. It
> needs to be accessible via something like parseFrom(ByteBuffer), or similar.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-11595) refactor ORC footer reading to make it usable from outside
[ https://issues.apache.org/jira/browse/HIVE-11595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14709751#comment-14709751 ] Sergey Shelukhin commented on HIVE-11595: - 1-2 done, although it is misleading cause the buffer contains the entire footer structure, incl. metadata and PS, not just OrcProto.Footer. 3-4 sure 5 there's a comment in the class. It could be changed to expand FileMetaInfo but FileMetaInfo is serialized in splits, so it would be confusing because the newly added fields would be missing on the other side (they are only used during split generation) > refactor ORC footer reading to make it usable from outside > -- > > Key: HIVE-11595 > URL: https://issues.apache.org/jira/browse/HIVE-11595 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-10595.patch, HIVE-11595.01.patch > > > If ORC footer is read from cache, we want to parse it without having the > reader, opening a file, etc. I thought it would be as simple as protobuf > parseFrom bytes, but apparently there's bunch of stuff going on there. It > needs to be accessible via something like parseFrom(ByteBuffer), or similar. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11595) refactor ORC footer reading to make it usable from outside
[ https://issues.apache.org/jira/browse/HIVE-11595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14705461#comment-14705461 ] Prasanth Jayachandran commented on HIVE-11595: -- 1) Can you rename the reader api from getSerializedFileMetadata() to getSerializedFooter()? 2) Also the variable fullFooterBuffer to serializedFooter? 3) Can you revert the signature of getAndCheckPostScript() to use Path instead of Object? I am assuming metastore has enough information about where the ByteBuffer came from (i.e, the path that the ByteBuffer belongs). It will be good to throw exception with path information instead of just "Byte buffer" for which we won't have any clue. You can add another helper for extractMetaInfoFromFooter that accepts Path as parameter. 4) Rename getAndCheckPostScript() to extractPostScript() to be inline with other extract methods? 5) What is the purpose of FooterInfo class? Apart from serialized footer and metadata (serialized or non serialized?) what other information are stored in the metastore? > refactor ORC footer reading to make it usable from outside > -- > > Key: HIVE-11595 > URL: https://issues.apache.org/jira/browse/HIVE-11595 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-10595.patch, HIVE-11595.01.patch > > > If ORC footer is read from cache, we want to parse it without having the > reader, opening a file, etc. I thought it would be as simple as protobuf > parseFrom bytes, but apparently there's bunch of stuff going on there. It > needs to be accessible via something like parseFrom(ByteBuffer), or similar. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11595) refactor ORC footer reading to make it usable from outside
[
https://issues.apache.org/jira/browse/HIVE-11595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14704289#comment-14704289
]
Hive QA commented on HIVE-11595:
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12751377/HIVE-11595.01.patch
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5019/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5019/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5019/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ [[ -n /usr/java/jdk1.7.0_45-cloudera ]]
+ export JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
+ JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
+ export
PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.7.0_45-cloudera/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin
+
PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.7.0_45-cloudera/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-TRUNK-Build-5019/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at ab03dc9 HIVE-11502: Map side aggregation is extremely slow
(Yongzhi Chen, reviewed by Chao Sun)
+ git clean -f -d
+ git checkout master
Already on 'master'
+ git reset --hard origin/master
HEAD is now at ab03dc9 HIVE-11502: Map side aggregation is extremely slow
(Yongzhi Chen, reviewed by Chao Sun)
+ git merge --ff-only origin/master
Already up-to-date.
+ git gc
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh
/data/hive-ptest/working/scratch/build.patch
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12751377 - PreCommit-HIVE-TRUNK-Build
> refactor ORC footer reading to make it usable from outside
> --
>
> Key: HIVE-11595
> URL: https://issues.apache.org/jira/browse/HIVE-11595
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-10595.patch, HIVE-11595.01.patch
>
>
> If ORC footer is read from cache, we want to parse it without having the
> reader, opening a file, etc. I thought it would be as simple as protobuf
> parseFrom bytes, but apparently there's bunch of stuff going on there. It
> needs to be accessible via something like parseFrom(ByteBuffer), or similar.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-11595) refactor ORC footer reading to make it usable from outside
[ https://issues.apache.org/jira/browse/HIVE-11595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14704046#comment-14704046 ] Sergey Shelukhin commented on HIVE-11595: - [~prasanth_j] ping? > refactor ORC footer reading to make it usable from outside > -- > > Key: HIVE-11595 > URL: https://issues.apache.org/jira/browse/HIVE-11595 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-10595.patch, HIVE-11595.01.patch > > > If ORC footer is read from cache, we want to parse it without having the > reader, opening a file, etc. I thought it would be as simple as protobuf > parseFrom bytes, but apparently there's bunch of stuff going on there. It > needs to be accessible via something like parseFrom(ByteBuffer), or similar. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
