[jira] [Commented] (HIVE-2722) GenericUDFUtils.findText use CharBuffer other than ByteBuffer will be better
[
https://issues.apache.org/jira/browse/HIVE-2722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16759752#comment-16759752
]
Peter Vary commented on HIVE-2722:
--
Then it seems like this problem is solved :)
> GenericUDFUtils.findText use CharBuffer other than ByteBuffer will be better
>
>
> Key: HIVE-2722
> URL: https://issues.apache.org/jira/browse/HIVE-2722
> Project: Hive
> Issue Type: Improvement
> Components: UDF
>Affects Versions: 0.8.0, 0.9.0
> Environment: Linux zongren-VirtualBox 3.0.0-14-generic #23-Ubuntu SMP
> Mon Nov 21 20:34:47 UTC 2011 i686 i686 i386 GNU/Linux
> java version "1.6.0_25"
> hadoop-0.20.2-cdh3u0
> hive-0.7.0-cdh3u0
>Reporter: caofangkun
>Assignee: Mani M
>Priority: Minor
> Labels: udf
> Attachments: HIVE-2722.patch, udf_instr_1.q
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> select instr("中文字符测试-第一行","-") from testTable limit 1;
> result:19 (one Chinese Character was considered as 3 Unicode bits)
> select substr("中文字符测试-第一行",1,2) from testTable limit 1;
> result: "中文" (one Chinese Character was considered as 1 Unicode Unit )
> instr should considered one chinese character as one Unicode Unit too.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-2722) GenericUDFUtils.findText use CharBuffer other than ByteBuffer will be better
[
https://issues.apache.org/jira/browse/HIVE-2722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16753892#comment-16753892
]
Mani M commented on HIVE-2722:
--
HI [~pvary]
I have checked this query in HIVE 3.1.1 version and the output is
*{color:#FF}showing as 7{color}* (as given below).
So I think patch is not required.
{code:java}
hive> select instr("中文字符测试-第一行","-") from lang_det limit 1;
OK
7
Time taken: 2.683 seconds, Fetched: 1 row(s)
{code}
> GenericUDFUtils.findText use CharBuffer other than ByteBuffer will be better
>
>
> Key: HIVE-2722
> URL: https://issues.apache.org/jira/browse/HIVE-2722
> Project: Hive
> Issue Type: Improvement
> Components: UDF
>Affects Versions: 0.8.0, 0.9.0
> Environment: Linux zongren-VirtualBox 3.0.0-14-generic #23-Ubuntu SMP
> Mon Nov 21 20:34:47 UTC 2011 i686 i686 i386 GNU/Linux
> java version "1.6.0_25"
> hadoop-0.20.2-cdh3u0
> hive-0.7.0-cdh3u0
>Reporter: caofangkun
>Assignee: Mani M
>Priority: Minor
> Labels: udf
> Attachments: HIVE-2722.patch, udf_instr_1.q
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> select instr("中文字符测试-第一行","-") from testTable limit 1;
> result:19 (one Chinese Character was considered as 3 Unicode bits)
> select substr("中文字符测试-第一行",1,2) from testTable limit 1;
> result: "中文" (one Chinese Character was considered as 1 Unicode Unit )
> instr should considered one chinese character as one Unicode Unit too.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-2722) GenericUDFUtils.findText use CharBuffer other than ByteBuffer will be better
[
https://issues.apache.org/jira/browse/HIVE-2722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16748534#comment-16748534
]
Mani M commented on HIVE-2722:
--
Ok
> GenericUDFUtils.findText use CharBuffer other than ByteBuffer will be better
>
>
> Key: HIVE-2722
> URL: https://issues.apache.org/jira/browse/HIVE-2722
> Project: Hive
> Issue Type: Improvement
> Components: UDF
>Affects Versions: 0.8.0, 0.9.0
> Environment: Linux zongren-VirtualBox 3.0.0-14-generic #23-Ubuntu SMP
> Mon Nov 21 20:34:47 UTC 2011 i686 i686 i386 GNU/Linux
> java version "1.6.0_25"
> hadoop-0.20.2-cdh3u0
> hive-0.7.0-cdh3u0
>Reporter: caofangkun
>Priority: Minor
> Labels: udf
> Attachments: HIVE-2722.patch, udf_instr_1.q
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> select instr("中文字符测试-第一行","-") from testTable limit 1;
> result:19 (one Chinese Character was considered as 3 Unicode bits)
> select substr("中文字符测试-第一行",1,2) from testTable limit 1;
> result: "中文" (one Chinese Character was considered as 1 Unicode Unit )
> instr should considered one chinese character as one Unicode Unit too.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-2722) GenericUDFUtils.findText use CharBuffer other than ByteBuffer will be better
[
https://issues.apache.org/jira/browse/HIVE-2722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16748505#comment-16748505
]
Peter Vary commented on HIVE-2722:
--
[~rmsm...@gmail.com]: Could you please check that the original problem is still
exists or not on Hive master? What is the result of:
{code}
select instr("中文字符测试-第一行","-") from testTable limit 1;
{code}
If the result should be 7.
Otherwise we still need to patch this.
Thanks,
Peter
> GenericUDFUtils.findText use CharBuffer other than ByteBuffer will be better
>
>
> Key: HIVE-2722
> URL: https://issues.apache.org/jira/browse/HIVE-2722
> Project: Hive
> Issue Type: Improvement
> Components: UDF
>Affects Versions: 0.8.0, 0.9.0
> Environment: Linux zongren-VirtualBox 3.0.0-14-generic #23-Ubuntu SMP
> Mon Nov 21 20:34:47 UTC 2011 i686 i686 i386 GNU/Linux
> java version "1.6.0_25"
> hadoop-0.20.2-cdh3u0
> hive-0.7.0-cdh3u0
>Reporter: caofangkun
>Priority: Minor
> Labels: udf
> Attachments: HIVE-2722.patch, udf_instr_1.q
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> select instr("中文字符测试-第一行","-") from testTable limit 1;
> result:19 (one Chinese Character was considered as 3 Unicode bits)
> select substr("中文字符测试-第一行",1,2) from testTable limit 1;
> result: "中文" (one Chinese Character was considered as 1 Unicode Unit )
> instr should considered one chinese character as one Unicode Unit too.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-2722) GenericUDFUtils.findText use CharBuffer other than ByteBuffer will be better
[
https://issues.apache.org/jira/browse/HIVE-2722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743806#comment-16743806
]
Mani M commented on HIVE-2722:
--
HI [~ashutoshc], [~pvary]
Recent changes were gone to the same file
(ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFUtils.java)- as
available in patch, under HIVE-20204, whether still this ticket need to be
addressed.
> GenericUDFUtils.findText use CharBuffer other than ByteBuffer will be better
>
>
> Key: HIVE-2722
> URL: https://issues.apache.org/jira/browse/HIVE-2722
> Project: Hive
> Issue Type: Improvement
> Components: UDF
>Affects Versions: 0.8.0, 0.9.0
> Environment: Linux zongren-VirtualBox 3.0.0-14-generic #23-Ubuntu SMP
> Mon Nov 21 20:34:47 UTC 2011 i686 i686 i386 GNU/Linux
> java version "1.6.0_25"
> hadoop-0.20.2-cdh3u0
> hive-0.7.0-cdh3u0
>Reporter: caofangkun
>Priority: Minor
> Labels: udf
> Attachments: HIVE-2722.patch, udf_instr_1.q
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> select instr("中文字符测试-第一行","-") from testTable limit 1;
> result:19 (one Chinese Character was considered as 3 Unicode bits)
> select substr("中文字符测试-第一行",1,2) from testTable limit 1;
> result: "中文" (one Chinese Character was considered as 1 Unicode Unit )
> instr should considered one chinese character as one Unicode Unit too.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-2722) GenericUDFUtils.findText use CharBuffer other than ByteBuffer will be better
[
https://issues.apache.org/jira/browse/HIVE-2722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738106#comment-16738106
]
Mani M commented on HIVE-2722:
--
HI [~ashutoshc]
Recent changes were gone to the same file
(ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFUtils.java)- as
available in patch, under HIVE-20204, whether still this ticket need to be
addressed.
> GenericUDFUtils.findText use CharBuffer other than ByteBuffer will be better
>
>
> Key: HIVE-2722
> URL: https://issues.apache.org/jira/browse/HIVE-2722
> Project: Hive
> Issue Type: Improvement
> Components: UDF
>Affects Versions: 0.8.0, 0.9.0
> Environment: Linux zongren-VirtualBox 3.0.0-14-generic #23-Ubuntu SMP
> Mon Nov 21 20:34:47 UTC 2011 i686 i686 i386 GNU/Linux
> java version "1.6.0_25"
> hadoop-0.20.2-cdh3u0
> hive-0.7.0-cdh3u0
>Reporter: caofangkun
>Priority: Minor
> Labels: udf
> Attachments: HIVE-2722.patch, udf_instr_1.q
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> select instr("中文字符测试-第一行","-") from testTable limit 1;
> result:19 (one Chinese Character was considered as 3 Unicode bits)
> select substr("中文字符测试-第一行",1,2) from testTable limit 1;
> result: "中文" (one Chinese Character was considered as 1 Unicode Unit )
> instr should considered one chinese character as one Unicode Unit too.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
