Re: Review Request 45836: JDBC: async wait should happen during fetch for results

[Thejas Nair]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45836/#review127953
---




jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java 


shouldn't resultset be closed here ?



jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java (line 257)


mark as @limitedprivate and @evolving ?



jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java (line 259)


add a note that - users should call those getUpdateCount or ResultSet.next 
(depending on whether query returns results) to ensure that query completes 
successfully.
Calling another execute* method, or close before query completion would 
result in the async query getting killed if it is not already finished.



jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java (line 312)


getUpdateCount or ResultSet.next() would result in another RPC call. 
Can we call keep track of the Statement complete status as a Statement 
member to avoid that ?



service/src/java/org/apache/hive/service/cli/operation/Operation.java (line 208)


just use the condition directly here ?

- if( !states.contains(state) ) {
... 
}


- Thejas Nair


On April 9, 2016, 12:37 a.m., Vaibhav Gumashta wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45836/
> ---
> 
> (Updated April 9, 2016, 12:37 a.m.)
> 
> 
> Review request for hive and Thejas Nair.
> 
> 
> Bugs: HIVE-6535
> https://issues.apache.org/jira/browse/HIVE-6535
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/HIVE-6535
> 
> 
> Diffs
> -
> 
>   jdbc/src/java/org/apache/hive/jdbc/HiveQueryResultSet.java 8f67209 
>   jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java b4dba44 
>   
> service/src/java/org/apache/hive/service/cli/operation/GetCatalogsOperation.java
>  8868ec1 
>   
> service/src/java/org/apache/hive/service/cli/operation/GetColumnsOperation.java
>  35b6c52 
>   
> service/src/java/org/apache/hive/service/cli/operation/GetFunctionsOperation.java
>  8db2e62 
>   
> service/src/java/org/apache/hive/service/cli/operation/GetSchemasOperation.java
>  d6f6280 
>   
> service/src/java/org/apache/hive/service/cli/operation/GetTableTypesOperation.java
>  a09b39a 
>   
> service/src/java/org/apache/hive/service/cli/operation/GetTablesOperation.java
>  740b851 
>   
> service/src/java/org/apache/hive/service/cli/operation/GetTypeInfoOperation.java
>  2a0fec2 
>   service/src/java/org/apache/hive/service/cli/operation/Operation.java 
> d9a273b 
>   service/src/java/org/apache/hive/service/cli/operation/SQLOperation.java 
> 04d816a 
> 
> Diff: https://reviews.apache.org/r/45836/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Vaibhav Gumashta
> 
>

[jira] [Created] (HIVE-13468) branch-2 build is broken

[Sergey Shelukhin (JIRA)]

Sergey Shelukhin created HIVE-13468:
---

 Summary: branch-2 build is broken
 Key: HIVE-13468
 URL: https://issues.apache.org/jira/browse/HIVE-13468
 Project: Hive
  Issue Type: Bug
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin


HIVE-13401 backport



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] hive pull request: HIVE-13418

[thejasmn]

GitHub user thejasmn opened a pull request:

https://github.com/apache/hive/pull/69

HIVE-13418



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/thejasmn/hive HIVE-13418

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/hive/pull/69.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #69


commit 9afad78243c0eeedd7571ac7961f177ebf20e771
Author: Thejas Nair 
Date:   2016-04-08T06:35:25Z

set x-forwarded-for

commit 400406a0765253f14e570061375923431d7f304c
Author: Thejas Nair 
Date:   2016-04-08T06:38:02Z

set forwarded address in HiveAuthzContext

commit ef438d7498cac59a665b92c5d3e5fffb6bbdac19
Author: Thejas Nair 
Date:   2016-04-08T21:23:55Z

add test in TestThriftHttpCLIService

commit a475bf1d077acf7335f4efcbcdd6bce7e75017fb
Author: Thejas Nair 
Date:   2016-04-08T21:47:34Z

rename impls of ThriftCLIServiceTest

commit eb6982c9f013f02df26ff7ea8d78e658224c4f95
Author: Thejas Nair 
Date:   2016-04-08T21:48:38Z

reorganize   ThriftCLIServiceTest tests

commit a3cac6ef692dcd1c89405e0cead4a0d949613122
Author: Thejas Nair 
Date:   2016-04-08T21:53:47Z

rename test class

commit e31cd18d7fd9be2ba0373949fa2e39d19a4aa943
Author: Thejas Nair 
Date:   2016-04-08T21:53:58Z

new classname

commit c48a21fab62f11f17213f2680cd414e69e155398
Author: Thejas Nair 
Date:   2016-04-09T00:17:55Z

test now checks the forwarded ips passed on

commit 131cd7208cc8e244a312253d63a250d7541f0a90
Author: Thejas Nair 
Date:   2016-04-09T00:19:04Z

fix test imports

commit ac227e05d931a906987a53cfcccf31b37fa8b95e
Author: Thejas Nair 
Date:   2016-04-09T00:40:07Z

fix test compile, post rebase




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Review Request 45836: JDBC: async wait should happen during fetch for results

[Vaibhav Gumashta]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45836/
---

(Updated April 9, 2016, 12:37 a.m.)


Review request for hive and Thejas Nair.


Bugs: HIVE-6535
https://issues.apache.org/jira/browse/HIVE-6535


Repository: hive-git


Description
---

https://issues.apache.org/jira/browse/HIVE-6535


Diffs (updated)
-

  jdbc/src/java/org/apache/hive/jdbc/HiveQueryResultSet.java 8f67209 
  jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java b4dba44 
  
service/src/java/org/apache/hive/service/cli/operation/GetCatalogsOperation.java
 8868ec1 
  
service/src/java/org/apache/hive/service/cli/operation/GetColumnsOperation.java 
35b6c52 
  
service/src/java/org/apache/hive/service/cli/operation/GetFunctionsOperation.java
 8db2e62 
  
service/src/java/org/apache/hive/service/cli/operation/GetSchemasOperation.java 
d6f6280 
  
service/src/java/org/apache/hive/service/cli/operation/GetTableTypesOperation.java
 a09b39a 
  
service/src/java/org/apache/hive/service/cli/operation/GetTablesOperation.java 
740b851 
  
service/src/java/org/apache/hive/service/cli/operation/GetTypeInfoOperation.java
 2a0fec2 
  service/src/java/org/apache/hive/service/cli/operation/Operation.java d9a273b 
  service/src/java/org/apache/hive/service/cli/operation/SQLOperation.java 
04d816a 

Diff: https://reviews.apache.org/r/45836/diff/


Testing
---


Thanks,

Vaibhav Gumashta

[jira] [Created] (HIVE-13467) Show llap info on hs2 ui when available

[Gunther Hagleitner (JIRA)]

Gunther Hagleitner created HIVE-13467:
-

 Summary: Show llap info on hs2 ui when available
 Key: HIVE-13467
 URL: https://issues.apache.org/jira/browse/HIVE-13467
 Project: Hive
  Issue Type: Bug
  Components: llap
Reporter: Gunther Hagleitner
Assignee: Gunther Hagleitner


When llap is on and hs2 is configured with access to an llap cluster, HS2 UI 
should show some status of the daemons and provide a mechanism to click through 
to their respective UIs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[discuss] jdk8 support

[Mohit Sabharwal]

Hi folks,

Oracle EOL'ed (ended public updates) for Java 7 in April 2015.
In order to support Java 8, we fixed quite a bunch of tests in
HIVE-8607 (*) early last year. However, since our pre-commit tests run
on JDK7 only, any JDK8 test failures are getting ignored. As a result,
the count has crept up

back
from zero to 125.

Some options to address this:

(1) Run pre-commit tests on both JDK7 and JDK8, in both 1.x
and 2.x: This will further slow down the pre-commit run.

(2) Alternate pre-commit test runs between JDK7 and JDK8 (in
both 1.x and 2.x): It's a cheap hack. More messy, though failures
won't get ignored.

(3) Drop support for JDK7 in Hive 2.x, i.e. build and test Hive 2.x on
JDK8 only. For Hive 1.x, continue running JDK7 pre-commit tests.
The pros are:
- Simple test matrix.
- No need to slow down test run or maintain version specific golden files
(**).
- 2.x looks like the logical place to move to JDK8.
- Users transitioning to JDK8 for all other services do not have to
maintain
multiple java versions on the cluster.

Option (3) looks most attractive to me.

Moving to JDK8 also lines us up better for Java 9, which is on the
horizon (Oracle will end public updates for Java 8 in Sep 2017)

Around 100 of the latest crop of failures are due to one cause
(HIVE-13409).
I can take a pass at triaging the rest if there is consensus around what
to do overall.

Thanks,
Mohit

(*) Most test failures are due to JDK8 using a different hash function for
HashMap  compared to JDK7. This results in (mostly benign, but hard
to debug) ordering differences in q-file output related to serialization
order of map entries, numbering of stages in query plan, etc.

(**) In some cases, hash function related ordering differences originate
in external libraries like Avro, antlr, json ,Thrift's map, etc. for which
code
changes are either more involved or led to more test failures.  To address
this,
we added support for version specific golden files (HIVE-9109). Currently,
there
are ~40 golden files with ".java1.8.out" extension.

[jira] [Created] (HIVE-13466) LLAP: Added unit tests for ZookeeperRegistry and remove unused methods

[Prasanth Jayachandran (JIRA)]

Prasanth Jayachandran created HIVE-13466:


 Summary: LLAP: Added unit tests for ZookeeperRegistry and remove 
unused methods
 Key: HIVE-13466
 URL: https://issues.apache.org/jira/browse/HIVE-13466
 Project: Hive
  Issue Type: Bug
Affects Versions: 2.1.0
Reporter: Prasanth Jayachandran
Assignee: Prasanth Jayachandran


Unit tests needs to be added for ZK based registry. Also some methods like 
scanForNodeChanges are no longer required for ZK registry. We need to cleanup 
such methods. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-13465) Add ZK settings to MiniLlapCluster clusterSpecificConfiguration

[Jason Dere (JIRA)]

Jason Dere created HIVE-13465:
-

 Summary: Add ZK settings to MiniLlapCluster 
clusterSpecificConfiguration
 Key: HIVE-13465
 URL: https://issues.apache.org/jira/browse/HIVE-13465
 Project: Hive
  Issue Type: Bug
  Components: llap, Testing Infrastructure
Reporter: Jason Dere
Assignee: Jason Dere


HIVE-13365 added zookeeper support to MiniLlapCluster. These settings should 
also be added to the clusterSpecificConfiguration to make whoever created the 
mini cluster gets their confs updated with this info.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-13464) Backport changes to storage-api into branch 2 for release into 2.0.1

[Owen O'Malley (JIRA)]

Owen O'Malley created HIVE-13464:


 Summary: Backport changes to storage-api into branch 2 for release 
into 2.0.1
 Key: HIVE-13464
 URL: https://issues.apache.org/jira/browse/HIVE-13464
 Project: Hive
  Issue Type: Bug
  Components: storage-api
Reporter: Owen O'Malley
Assignee: Owen O'Malley
 Fix For: 2.0.1


To release ORC as a separate project, backporting the safe changes for 
storage-api to 2.0.1 will minimize the disruption.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-13463) Fix ImportSemanticAnalyzer to allow for different src/dst filesystems

[Zach York (JIRA)]

Zach York created HIVE-13463:


 Summary: Fix ImportSemanticAnalyzer to allow for different src/dst 
filesystems
 Key: HIVE-13463
 URL: https://issues.apache.org/jira/browse/HIVE-13463
 Project: Hive
  Issue Type: Bug
  Components: Import/Export
Affects Versions: 2.0.0
Reporter: Zach York
Assignee: Zach York


In ImportSemanticAnalyzer, there is an assumption that the src filesystem for 
import and the final location are on the same filesystem. Therefore the check 
for emptiness and getExternalTmpLocation will be looking on the wrong 
filesystem and will cause an error. The output path should be fed into 
getExternalTmpLocation to get a temporary file on the correct filesystem. The 
check for emptiness should use the output filesystem.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-13462) HiveResultSetMetaData.getPrecision() fails for NULL columns

[Jason Dere (JIRA)]

Jason Dere created HIVE-13462:
-

 Summary: HiveResultSetMetaData.getPrecision() fails for NULL 
columns
 Key: HIVE-13462
 URL: https://issues.apache.org/jira/browse/HIVE-13462
 Project: Hive
  Issue Type: Bug
  Components: JDBC
Reporter: Jason Dere
Assignee: Jason Dere


Can happen if you have a null in the select clause, for example "select null, 
key, value from src"

{noformat}
java.sql.SQLException: Unrecognized column type: NULL
at 
org.apache.hive.jdbc.JdbcColumn.typeStringToHiveType(JdbcColumn.java:160)
at 
org.apache.hive.jdbc.HiveResultSetMetaData.getHiveType(HiveResultSetMetaData.java:48)
at 
org.apache.hive.jdbc.HiveResultSetMetaData.getPrecision(HiveResultSetMetaData.java:86)
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-13461) LLAP output format service not actually registered in LLAP registry

[Jason Dere (JIRA)]

Jason Dere created HIVE-13461:
-

 Summary: LLAP output format service not actually registered in 
LLAP registry
 Key: HIVE-13461
 URL: https://issues.apache.org/jira/browse/HIVE-13461
 Project: Hive
  Issue Type: Sub-task
  Components: llap
Reporter: Jason Dere
Assignee: Jason Dere
 Fix For: llap


Should have been done in HIVE-13305, but missed this part.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 45238: HIVE-9660 store end offset of compressed data for RG in RowIndex in ORC

[Sergey Shelukhin]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45238/
---

(Updated April 8, 2016, 6:07 p.m.)


Review request for hive and Prasanth_J.


Repository: hive-git


Description
---

see jira


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 95c5c0e 
  data/conf/hive-log4j2.properties 6bace1f 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/OrcEncodedDataReader.java
 fb0867d 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/metadata/OrcStripeMetadata.java
 82187bd 
  orc/src/gen/protobuf-java/org/apache/orc/OrcProto.java 24715c3 
  orc/src/java/org/apache/orc/OrcConf.java 6fcbb72 
  orc/src/java/org/apache/orc/OrcFile.java 85506ff 
  orc/src/java/org/apache/orc/TypeDescription.java bd900ac 
  orc/src/java/org/apache/orc/impl/BitFieldWriter.java aa5f886 
  orc/src/java/org/apache/orc/impl/IntegerWriter.java 419054f 
  orc/src/java/org/apache/orc/impl/OutStream.java 81662cc 
  orc/src/java/org/apache/orc/impl/RunLengthByteWriter.java 09108b2 
  orc/src/java/org/apache/orc/impl/RunLengthIntegerWriter.java 3e5f2e2 
  orc/src/java/org/apache/orc/impl/RunLengthIntegerWriterV2.java fab2801 
  orc/src/java/org/apache/orc/impl/SerializationUtils.java 2e5a59b 
  orc/src/java/org/apache/orc/impl/WriterImpl.java f8afe06 
  orc/src/protobuf/orc_proto.proto f4935b4 
  orc/src/test/org/apache/orc/impl/TestBitFieldReader.java e4c6f6b 
  orc/src/test/org/apache/orc/impl/TestBitPack.java f2d3d64 
  orc/src/test/org/apache/orc/impl/TestInStream.java 9e65345 
  orc/src/test/org/apache/orc/impl/TestIntegerCompressionReader.java 399f35e 
  orc/src/test/org/apache/orc/impl/TestOutStream.java e9614d5 
  orc/src/test/org/apache/orc/impl/TestRunLengthByteReader.java a14bef1 
  orc/src/test/org/apache/orc/impl/TestRunLengthIntegerReader.java 28239ba 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/DagUtils.java 8aca779 
  ql/src/java/org/apache/hadoop/hive/ql/hooks/PostExecOrcFileDump.java d5d1370 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/FileDump.java 9c2f88f 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/JsonFileDump.java 00de545 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReaderUtils.java 8a73948 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedReader.java 
4d09dcd 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedReaderImpl.java 
f4cfa53 
  ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestInputOutputFormat.java 
85923a8 
  ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestJsonFileDump.java acf232d 
  ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestStringDictionary.java 
41a211b 
  ql/src/test/queries/clientpositive/orc_lengths.q PRE-CREATION 
  ql/src/test/resources/orc-file-dump-bloomfilter.out 18fd2fb 
  ql/src/test/resources/orc-file-dump-bloomfilter2.out fa5cc2d 
  ql/src/test/resources/orc-file-dump-dictionary-threshold.out 17a964b 
  ql/src/test/resources/orc-file-dump.json bf654a1 
  ql/src/test/resources/orc-file-dump.out 70f7fbd 
  ql/src/test/resources/orc-file-has-null.out e98a73f 
  ql/src/test/results/clientpositive/acid_globallimit.q.out d43ed42 
  ql/src/test/results/clientpositive/alter_merge_orc.q.out b5a6d04 
  ql/src/test/results/clientpositive/alter_merge_stats_orc.q.out 0d5ba01 
  ql/src/test/results/clientpositive/annotate_stats_part.q.out 131cf6a 
  ql/src/test/results/clientpositive/annotate_stats_table.q.out 6db4ded 
  ql/src/test/results/clientpositive/columnStatsUpdateForStatsOptimizer_2.q.out 
179bc66 
  ql/src/test/results/clientpositive/dynpart_sort_opt_vectorization.q.out 
d03bfe4 
  ql/src/test/results/clientpositive/dynpart_sort_optimization2.q.out 3b24a2e 
  ql/src/test/results/clientpositive/extrapolate_part_stats_full.q.out a30c356 
  ql/src/test/results/clientpositive/extrapolate_part_stats_partial.q.out 
4e589b8 
  ql/src/test/results/clientpositive/extrapolate_part_stats_partial_ndv.q.out 
3185f70 
  ql/src/test/results/clientpositive/llap/llap_nullscan.q.out e17d721 
  ql/src/test/results/clientpositive/orc_analyze.q.out 87855fa 
  ql/src/test/results/clientpositive/orc_file_dump.q.out a97f6de 
  ql/src/test/results/clientpositive/orc_lengths.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/orc_llap.q.out 6fc73b7 
  ql/src/test/results/clientpositive/orc_merge10.q.out cf70dcf 
  ql/src/test/results/clientpositive/orc_merge11.q.out 8a4d8e9 
  ql/src/test/results/clientpositive/orc_merge12.q.out f23be5a 
  ql/src/test/results/clientpositive/schema_evol_stats.q.out 63dab2e 
  ql/src/test/results/clientpositive/spark/alter_merge_orc.q.out b5a6d04 
  ql/src/test/results/clientpositive/spark/alter_merge_stats_orc.q.out 0d5ba01 
  ql/src/test/results/clientpositive/spark/bucket5.q.out 5baf054 
  
ql/src/test/results/clientpositive/spark/infer_bucket_sort_map_operators.q.out 
a343d93 
  

[jira] [Created] (HIVE-13460) ANALYZE TABLE COMPUTE STATISTICS FAILED max key length is 1000 bytes

[Aleksey Vovchenko (JIRA)]

Aleksey Vovchenko created HIVE-13460:


 Summary: ANALYZE TABLE COMPUTE STATISTICS FAILED max key length is 
1000 bytes
 Key: HIVE-13460
 URL: https://issues.apache.org/jira/browse/HIVE-13460
 Project: Hive
  Issue Type: Bug
  Components: Hive
Affects Versions: 1.0.1
Reporter: Aleksey Vovchenko
Assignee: Aleksey Vovchenko


When Hive configured to Store Statistics in MySQL we have next error:
{noformat} 
2016-04-08 15:53:28,047 ERROR [main]: jdbc.JDBCStatsPublisher 
(JDBCStatsPublisher.java:init(316)) - Error during JDBC initialization.
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Specified key was 
too long; max key length is 767 bytes
{noformat} 

If set MySql properties as:

{noformat} 
set global innodb_large_prefix = ON;
set global innodb_file_format = BARRACUDA;
{noformat} 

Now we have next Error:

{noformat} 
2016-04-08 15:56:05,552 ERROR [main]: jdbc.JDBCStatsPublisher 
(JDBCStatsPublisher.java:init(316)) - Error during JDBC initialization.
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Specified key was 
too long; max key length is 3072 bytes
{noformat} 


 As a result of my investigation I figured out that MySQL does not allow to 
create primary key with size more than 3072 bytes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Masters Thesis on False Positives in Test Failures

[Kevin van den Bekerom]

Dear Developers of the Apache HBase project,



My name is Kevin van den Bekerom and I am currently doing my Master's
research on the topic of false alarms in test code. I would like to ask the
input of the HBase development team categorizing test code bugs.



My research is based on a recent paper by Arash et al. (
http://salt.ece.ubc.ca/publications/docs/icsme15.pdf). They conducted an
empirical study, categorizing "test code bugs" in Apache software projects,
e.g. semantic, flaky, environmental, etc. A "test code bug" is a failing
test, where the System Under Test is correct, but the test code is
incorrect. To identify test code bugs they looked at issues in JIRA, and
checked if the fixing commit was only in the test code. Only fixed issues
were counted and categorised.



My goal is to replicate their results using a different approach, i.e. ask
developers that were involved in the issue and/or fix how they would
categorize it.  For the HBase project they counted 440 test code bugs.
Insight into false positives can therefore be very relevant for your
project. Note that the authors only sampled a number of identified test
code bugs for individual inspection.


I would like to ask the HBase team’s participation in categorizing the
various test bugs. I will provide a list of JIRA IDs which are identified
as test code bugs and an initial list of categories (assertion fault,
obsolete assertion, test dependency, etc) to aid in
the categorisation process with short explanations. In my belief, the
developers that worked on the issue are the one's that are most capable of
categorizing the issue. Please let me know if this project looks
interesting to you and if you are willing to help me out.



As a next step I will look for common patterns in identified test code bugs
and my aim is to extent static source code analysis techniques to be also
suited to find test code bugs. I am of course very happy to share my
findings with the team.



Hope to hear from you!



With kind regards,

-- 
*Kevin van den Bekerom* | Intern

+31 6 21 33 93 85 | kvandenbeke...@sig.eu
Software Improvement Group | www.sig.eu

Re: Review Request 45062: HIVE-13241 LLAP: Incremental Caching marks some small chunks as "incomplete CB"

[Lefty Leverenz]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45062/#review127765
---




common/src/java/org/apache/hadoop/hive/conf/HiveConf.java (lines 2551 - 2554)


It isn't necessary to include "(Hive 2.1?)" – eventually, the description 
would have to be updated and anyway the release number can be found on 
HIVE-9660.

Also, please spell out RG.  Thanks.


- Lefty Leverenz


On March 18, 2016, 11:18 p.m., Sergey Shelukhin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45062/
> ---
> 
> (Updated March 18, 2016, 11:18 p.m.)
> 
> 
> Review request for hive, Gopal V and Prasanth_J.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> see jira
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 98c6372 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/cache/EvictionDispatcher.java
>  bae571e 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/cli/LlapOptionsProcessor.java
>  c292b37 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapIoImpl.java 
> dbee823 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/OrcEncodedDataReader.java
>  eb251a8 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/metadata/OrcFileEstimateErrors.java
>  PRE-CREATION 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/metadata/OrcMetadataCache.java
>  e970137 
>   
> llap-server/src/test/org/apache/hadoop/hive/llap/cache/TestOrcMetadataCache.java
>  901e58a 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedReaderImpl.java 
> 29b51ec 
>   
> storage-api/src/java/org/apache/hadoop/hive/common/io/encoded/EncodedColumnBatch.java
>  ddba889 
> 
> Diff: https://reviews.apache.org/r/45062/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sergey Shelukhin
> 
>

Re: Review Request 45238: HIVE-9660 store end offset of compressed data for RG in RowIndex in ORC

[Lefty Leverenz]

> On March 24, 2016, 1:45 a.m., Lefty Leverenz wrote:
> > orc/src/java/org/apache/orc/OrcConf.java, lines 100-102
> > 
> >
> > Please spell out RG in the description.
> 
> Lefty Leverenz wrote:
> Nudge:  please spell out RG in parameter descriptions in HiveConf.java 
> and OrcConf.java.  Thanks.

Thanks for the fixes, Sergey.  +1 for parameter descriptions in HiveConf.java 
and OrcConf.java.


- Lefty


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45238/#review125176
---


On April 7, 2016, 11:46 p.m., Sergey Shelukhin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45238/
> ---
> 
> (Updated April 7, 2016, 11:46 p.m.)
> 
> 
> Review request for hive and Prasanth_J.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> see jira
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 95c5c0e 
>   data/conf/hive-log4j2.properties 6bace1f 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/OrcEncodedDataReader.java
>  fb0867d 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/metadata/OrcStripeMetadata.java
>  82187bd 
>   orc/src/gen/protobuf-java/org/apache/orc/OrcProto.java 24715c3 
>   orc/src/java/org/apache/orc/OrcConf.java 6fcbb72 
>   orc/src/java/org/apache/orc/OrcFile.java 85506ff 
>   orc/src/java/org/apache/orc/TypeDescription.java bd900ac 
>   orc/src/java/org/apache/orc/impl/BitFieldWriter.java aa5f886 
>   orc/src/java/org/apache/orc/impl/IntegerWriter.java 419054f 
>   orc/src/java/org/apache/orc/impl/OutStream.java 81662cc 
>   orc/src/java/org/apache/orc/impl/RunLengthByteWriter.java 09108b2 
>   orc/src/java/org/apache/orc/impl/RunLengthIntegerWriter.java 3e5f2e2 
>   orc/src/java/org/apache/orc/impl/RunLengthIntegerWriterV2.java fab2801 
>   orc/src/java/org/apache/orc/impl/SerializationUtils.java 2e5a59b 
>   orc/src/java/org/apache/orc/impl/WriterImpl.java f8afe06 
>   orc/src/protobuf/orc_proto.proto f4935b4 
>   orc/src/test/org/apache/orc/impl/TestBitFieldReader.java e4c6f6b 
>   orc/src/test/org/apache/orc/impl/TestBitPack.java f2d3d64 
>   orc/src/test/org/apache/orc/impl/TestInStream.java 9e65345 
>   orc/src/test/org/apache/orc/impl/TestIntegerCompressionReader.java 399f35e 
>   orc/src/test/org/apache/orc/impl/TestOutStream.java e9614d5 
>   orc/src/test/org/apache/orc/impl/TestRunLengthByteReader.java a14bef1 
>   orc/src/test/org/apache/orc/impl/TestRunLengthIntegerReader.java 28239ba 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/tez/DagUtils.java 8aca779 
>   ql/src/java/org/apache/hadoop/hive/ql/hooks/PostExecOrcFileDump.java 
> d5d1370 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/FileDump.java 9c2f88f 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/JsonFileDump.java 00de545 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReaderUtils.java 8a73948 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedReader.java 
> 4d09dcd 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedReaderImpl.java 
> f4cfa53 
>   ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestInputOutputFormat.java 
> 85923a8 
>   ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestJsonFileDump.java acf232d 
>   ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestStringDictionary.java 
> 41a211b 
>   ql/src/test/queries/clientpositive/orc_lengths.q PRE-CREATION 
>   ql/src/test/resources/orc-file-dump-bloomfilter.out 18fd2fb 
>   ql/src/test/resources/orc-file-dump-bloomfilter2.out fa5cc2d 
>   ql/src/test/resources/orc-file-dump-dictionary-threshold.out 17a964b 
>   ql/src/test/resources/orc-file-dump.json bf654a1 
>   ql/src/test/resources/orc-file-dump.out 70f7fbd 
>   ql/src/test/resources/orc-file-has-null.out e98a73f 
>   ql/src/test/results/clientpositive/acid_globallimit.q.out d43ed42 
>   ql/src/test/results/clientpositive/alter_merge_orc.q.out b5a6d04 
>   ql/src/test/results/clientpositive/alter_merge_stats_orc.q.out 0d5ba01 
>   ql/src/test/results/clientpositive/annotate_stats_part.q.out 131cf6a 
>   ql/src/test/results/clientpositive/annotate_stats_table.q.out 6db4ded 
>   
> ql/src/test/results/clientpositive/columnStatsUpdateForStatsOptimizer_2.q.out 
> 179bc66 
>   ql/src/test/results/clientpositive/dynpart_sort_opt_vectorization.q.out 
> d03bfe4 
>   ql/src/test/results/clientpositive/dynpart_sort_optimization2.q.out 3b24a2e 
>   ql/src/test/results/clientpositive/extrapolate_part_stats_full.q.out 
> a30c356 
>   ql/src/test/results/clientpositive/extrapolate_part_stats_partial.q.out 
> 4e589b8 
>   ql/src/test/results/clientpositive/extrapolate_part_stats_partial_ndv.q.out 
> 3185f70 
>