[jira] [Commented] (HIVE-5221) Issue in column type with data type as BINARY
[ https://issues.apache.org/jira/browse/HIVE-5221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13818176#comment-13818176 ] Ashutosh Chauhan commented on HIVE-5221: [~brocknoland] I think you have a valid point. I think we should do same on write side as we are now doing on read side, ie if user wants to store binary data in text file (which is weird, but anyways) he need to use an udf to encode it before passing it to serde. Than, we are consistent on both read and write side, asking user to use appropriate udfs for encoding/decoding and setting serde free from any assumptions. [~kamrul] Can you make changes in LazyUtils.writePrimitiveUtf8 so that it doesn't do any encoding? Issue in column type with data type as BINARY - Key: HIVE-5221 URL: https://issues.apache.org/jira/browse/HIVE-5221 Project: Hive Issue Type: Bug Reporter: Arun Vasu Assignee: Mohammad Kamrul Islam Priority: Critical Attachments: HIVE-5221.1.patch, HIVE-5221.2.patch Hi, I am using Hive 10. When I create an external table with column type as Binary, the query result on the table is showing some junk values for the column with binary datatype. Please find below the query I have used to create the table: CREATE EXTERNAL TABLE BOOL1(NB BOOLEAN,email STRING, bitfld BINARY) ROW FORMAT DELIMITED FIELDS TERMINATED BY '^' LINES TERMINATED BY '\n' STORED AS TEXTFILE LOCATION '/user/hivetables/testbinary'; The query I have used is : select * from bool1 The sample data in the hdfs file is: 0^a...@abc.com^001 1^a...@abc.com^010 ^a...@abc.com^011 ^a...@abc.com^100 t^a...@abc.com^101 f^a...@abc.com^110 true^a...@abc.com^111 false^a...@abc.com^001 123^^01100010 12344^^0111 Please share your inputs if it is possible. Thanks, Arun -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5221) Issue in column type with data type as BINARY
[ https://issues.apache.org/jira/browse/HIVE-5221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13813334#comment-13813334 ] Brock Noland commented on HIVE-5221: We can wait for HiveQA but it looks like a test in TestLazySimpleSerDe might fail as it's encoding base 64 data and expecting it to be decoded? One more question, which perhaps isn't related, but it looks like LazyUtils.writePrimitiveUTF8 is also encoding data as Base64. Thinking about it... doesn't Lazy Serde have to use base64 to store BINARY data? Issue in column type with data type as BINARY - Key: HIVE-5221 URL: https://issues.apache.org/jira/browse/HIVE-5221 Project: Hive Issue Type: Bug Reporter: Arun Vasu Assignee: Mohammad Kamrul Islam Priority: Critical Attachments: HIVE-5221.1.patch, HIVE-5221.2.patch Hi, I am using Hive 10. When I create an external table with column type as Binary, the query result on the table is showing some junk values for the column with binary datatype. Please find below the query I have used to create the table: CREATE EXTERNAL TABLE BOOL1(NB BOOLEAN,email STRING, bitfld BINARY) ROW FORMAT DELIMITED FIELDS TERMINATED BY '^' LINES TERMINATED BY '\n' STORED AS TEXTFILE LOCATION '/user/hivetables/testbinary'; The query I have used is : select * from bool1 The sample data in the hdfs file is: 0^a...@abc.com^001 1^a...@abc.com^010 ^a...@abc.com^011 ^a...@abc.com^100 t^a...@abc.com^101 f^a...@abc.com^110 true^a...@abc.com^111 false^a...@abc.com^001 123^^01100010 12344^^0111 Please share your inputs if it is possible. Thanks, Arun -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5221) Issue in column type with data type as BINARY
[ https://issues.apache.org/jira/browse/HIVE-5221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13813335#comment-13813335 ] Brock Noland commented on HIVE-5221: bq. doesn't Lazy Serde have to use base64 to store BINARY data? meaning when dealing with text files isn't base64 required for BINARY. Issue in column type with data type as BINARY - Key: HIVE-5221 URL: https://issues.apache.org/jira/browse/HIVE-5221 Project: Hive Issue Type: Bug Reporter: Arun Vasu Assignee: Mohammad Kamrul Islam Priority: Critical Attachments: HIVE-5221.1.patch, HIVE-5221.2.patch Hi, I am using Hive 10. When I create an external table with column type as Binary, the query result on the table is showing some junk values for the column with binary datatype. Please find below the query I have used to create the table: CREATE EXTERNAL TABLE BOOL1(NB BOOLEAN,email STRING, bitfld BINARY) ROW FORMAT DELIMITED FIELDS TERMINATED BY '^' LINES TERMINATED BY '\n' STORED AS TEXTFILE LOCATION '/user/hivetables/testbinary'; The query I have used is : select * from bool1 The sample data in the hdfs file is: 0^a...@abc.com^001 1^a...@abc.com^010 ^a...@abc.com^011 ^a...@abc.com^100 t^a...@abc.com^101 f^a...@abc.com^110 true^a...@abc.com^111 false^a...@abc.com^001 123^^01100010 12344^^0111 Please share your inputs if it is possible. Thanks, Arun -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5221) Issue in column type with data type as BINARY
[
https://issues.apache.org/jira/browse/HIVE-5221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13813537#comment-13813537
]
Hive QA commented on HIVE-5221:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12611731/HIVE-5221.2.patch
{color:red}ERROR:{color} -1 due to 16 failed/errored test(s), 4551 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ba_table1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ba_table2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ba_table3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ba_table_udfs
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ba_table_union
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_binary_table_bincolserde
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_binary_table_colserde
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_compute_stats_binary
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_load_binary_data
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_printf
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_reflect2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_substr
org.apache.hadoop.hive.serde2.lazy.TestLazyPrimitive.testLazyBinary
org.apache.hcatalog.pig.TestHCatStorer.testStoreFuncAllSimpleTypes
org.apache.hive.hcatalog.pig.TestHCatStorer.testStoreFuncAllSimpleTypes
org.apache.hive.jdbc.TestJdbcDriver2.testDataTypes
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/134/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/134/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests failed with: TestsFailedException: 16 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12611731
Issue in column type with data type as BINARY
-
Key: HIVE-5221
URL: https://issues.apache.org/jira/browse/HIVE-5221
Project: Hive
Issue Type: Bug
Reporter: Arun Vasu
Assignee: Mohammad Kamrul Islam
Priority: Critical
Attachments: HIVE-5221.1.patch, HIVE-5221.2.patch
Hi,
I am using Hive 10. When I create an external table with column type as
Binary, the query result on the table is showing some junk values for the
column with binary datatype.
Please find below the query I have used to create the table:
CREATE EXTERNAL TABLE BOOL1(NB BOOLEAN,email STRING, bitfld BINARY)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '^'
LINES TERMINATED BY '\n'
STORED AS TEXTFILE
LOCATION '/user/hivetables/testbinary';
The query I have used is : select * from bool1
The sample data in the hdfs file is:
0^a...@abc.com^001
1^a...@abc.com^010
^a...@abc.com^011
^a...@abc.com^100
t^a...@abc.com^101
f^a...@abc.com^110
true^a...@abc.com^111
false^a...@abc.com^001
123^^01100010
12344^^0111
Please share your inputs if it is possible.
Thanks,
Arun
--
This message was sent by Atlassian JIRA
(v6.1#6144)
