[jira] [Created] (HIVE-13752) Mini HDFS Cluster fails to start on trunk
Xiaobing Zhou created HIVE-13752: Summary: Mini HDFS Cluster fails to start on trunk Key: HIVE-13752 URL: https://issues.apache.org/jira/browse/HIVE-13752 Project: Hive Issue Type: Bug Reporter: Xiaobing Zhou It's been noticed that Mini HDFS Cluster fails to start on trunk, blocking unit tests and Jenkins. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-9760) Killed job is not displayed correctly by failure hook
Xiaobing Zhou created HIVE-9760: --- Summary: Killed job is not displayed correctly by failure hook Key: HIVE-9760 URL: https://issues.apache.org/jira/browse/HIVE-9760 Project: Hive Issue Type: Bug Affects Versions: 1.0.0 Environment: CentOS Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou PROBLEM: After killing the job throug CLI by crtl+c, we are seeing the following exception. Also the job is shown as running with timer still counting on the UI Interrupting... Be patient, this might take some time. Press Ctrl+C again to kill JVM Map 1: 20/40 Reducer 2: 0/3 Reducer 3: 0/1 Map 1: 22/40 Reducer 2: 0/3 Reducer 3: 0/1 Map 1: 25/40 Reducer 2: 0/3 Reducer 3: 0/1 Map 1: 26/40 Reducer 2: 0/3 Reducer 3: 0/1 Exiting the JVM Trying to shutdown DAG Shutting down tez session. Status: Killed Job received Kill while in RUNNING state. Vertex killed, vertexName=Reducer 3, vertexId=vertex_1421602925091_3868_2_00, diagnostics=Vertex received Kill while in RUNNING state., Vertex killed due to user-initiated job kill. failedTasks:0 Vertex killed, vertexName=Map 1, vertexId=vertex_1421602925091_3868_2_02, diagnostics=Vertex received Kill while in RUNNING state., Vertex killed due to user-initiated job kill. failedTasks:0 Vertex killed, vertexName=Reducer 2, vertexId=vertex_1421602925091_3868_2_01, diagnostics=Vertex received Kill while in RUNNING state., Vertex killed due to user-initiated job kill. failedTasks:0 DAG killed due to user-initiated kill. failedVertices:0 killedVertices:3 FAILED: Hive Internal Error: java.lang.IllegalStateException(Shutdown in progress) java.lang.IllegalStateException: Shutdown in progress at java.lang.ApplicationShutdownHooks.add(ApplicationShutdownHooks.java:66) at java.lang.Runtime.addShutdownHook(Runtime.java:211) at org.apache.hadoop.hive.ql.hooks.ATSHook.init(ATSHook.java:71) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at java.lang.Class.newInstance(Class.java:374) at org.apache.hadoop.hive.ql.hooks.HookUtils.getHooks(HookUtils.java:59) at org.apache.hadoop.hive.ql.Driver.getHooks(Driver.java:1177) at org.apache.hadoop.hive.ql.Driver.getHooks(Driver.java:1161) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1316) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1093) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:916) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:906) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:793) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) hive EL Info: 2015-01-21 16:20:51.483-ServerSession(1006268460)-EclipseLink, version: Eclipse Persistence Services - 2.5.1.v20130918-f2b9fc5 EL Info: connection: 2015-01-21 16:20:52.025-ServerSession(1006268460)-file:/usr/lib/hive/lib/xasecure-audit.jar_xa_server_url=jdbc:mysql://bkcttplpd036.verizon.com/xasecure_user=xalogger login successful EL Info: connection: 2015-01-21 16:20:52.082-ServerSession(1006268460)-file:/usr/lib/hive/lib/xasecure-audit.jar_xa_server_url=jdbc:mysql://bkcttplpd036.verizon.com/xasecure_user=xalogger logout successful -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9480) Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
[
https://issues.apache.org/jira/browse/HIVE-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaobing Zhou updated HIVE-9480:
Attachment: (was: HIVE-9480.5.patch)
Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
Key: HIVE-9480
URL: https://issues.apache.org/jira/browse/HIVE-9480
Project: Hive
Issue Type: Bug
Components: UDF
Affects Versions: 0.14.0
Reporter: Xiaobing Zhou
Assignee: Xiaobing Zhou
Attachments: HIVE-9480.1.patch, HIVE-9480.3.patch, HIVE-9480.4.patch
Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to
do date/timestamp related computation. This JIRA is to track such an
implementation. Choose to impl TRUNC, a more standard way to get the first
day of a a month, e.g., SELECT TRUNC('2009-12-12', 'MM'); will return
2009-12-01, SELECT TRUNC('2009-12-12', 'YEAR'); will return 2009-01-01.
BTW, this TRUNC is not as feature complete as aligned with Oracle one. only
'MM' and 'YEAR' are supported as format, however, it's a base to add on other
formats.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9480) Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
[
https://issues.apache.org/jira/browse/HIVE-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaobing Zhou updated HIVE-9480:
Attachment: HIVE-9480.6.patch
Thanks [~jdere]. Here's V6 after rebase.
Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
Key: HIVE-9480
URL: https://issues.apache.org/jira/browse/HIVE-9480
Project: Hive
Issue Type: Bug
Components: UDF
Affects Versions: 0.14.0
Reporter: Xiaobing Zhou
Assignee: Xiaobing Zhou
Attachments: HIVE-9480.1.patch, HIVE-9480.3.patch, HIVE-9480.4.patch,
HIVE-9480.5.patch, HIVE-9480.6.patch
Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to
do date/timestamp related computation. This JIRA is to track such an
implementation. Choose to impl TRUNC, a more standard way to get the first
day of a a month, e.g., SELECT TRUNC('2009-12-12', 'MM'); will return
2009-12-01, SELECT TRUNC('2009-12-12', 'YEAR'); will return 2009-01-01.
BTW, this TRUNC is not as feature complete as aligned with Oracle one. only
'MM' and 'YEAR' are supported as format, however, it's a base to add on other
formats.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9480) Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
[
https://issues.apache.org/jira/browse/HIVE-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaobing Zhou updated HIVE-9480:
Attachment: HIVE-9480.5.patch
Re-submit patch in order to trigger UT(s) run. Don't know why it's not
triggered by previous patch submission.
Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
Key: HIVE-9480
URL: https://issues.apache.org/jira/browse/HIVE-9480
Project: Hive
Issue Type: Bug
Components: UDF
Affects Versions: 0.14.0
Reporter: Xiaobing Zhou
Assignee: Xiaobing Zhou
Attachments: HIVE-9480.1.patch, HIVE-9480.3.patch, HIVE-9480.4.patch,
HIVE-9480.5.patch
Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to
do date/timestamp related computation. This JIRA is to track such an
implementation. Choose to impl TRUNC, a more standard way to get the first
day of a a month, e.g., SELECT TRUNC('2009-12-12', 'MM'); will return
2009-12-01, SELECT TRUNC('2009-12-12', 'YEAR'); will return 2009-01-01.
BTW, this TRUNC is not as feature complete as aligned with Oracle one. only
'MM' and 'YEAR' are supported as format, however, it's a base to add on other
formats.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9741) Refactor MetaStoreDirectSql by using getProductName instead of querying DB to determine DbType
[ https://issues.apache.org/jira/browse/HIVE-9741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329884#comment-14329884 ] Xiaobing Zhou commented on HIVE-9741: - Thanks [~ashutoshc], will do that in upcoming patch. Refactor MetaStoreDirectSql by using getProductName instead of querying DB to determine DbType -- Key: HIVE-9741 URL: https://issues.apache.org/jira/browse/HIVE-9741 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 1.0.0 Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Attachments: HIVE-9741.1.patch, HIVE-9741.2.patch MetaStoreDirectSql constructor is querying DB to determine dbType. which leads to too many DB queries to make metastore slow or hanging if MetaStoreDirectSql constructor is frequently called. This is to propose getProductName to get dbType info. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9741) Refactor MetaStoreDirectSql by using getProductName instead of querying DB to determine DbType
[ https://issues.apache.org/jira/browse/HIVE-9741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9741: Attachment: HIVE-9741.2.patch Patch V2. Refactor MetaStoreDirectSql by using getProductName instead of querying DB to determine DbType -- Key: HIVE-9741 URL: https://issues.apache.org/jira/browse/HIVE-9741 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 1.0.0 Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Attachments: HIVE-9741.1.patch, HIVE-9741.2.patch MetaStoreDirectSql constructor is querying DB to determine dbType. which leads to too many DB queries to make metastore slow or hanging if MetaStoreDirectSql constructor is frequently called. This is to propose getProductName to get dbType info. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9746) Refactor ATSHook constructor to avoid issues of twice CTRL+C terminating running query
[ https://issues.apache.org/jira/browse/HIVE-9746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9746: Description: Run a hive query and then hit Ctrl+C twice. This kills the hive query instantly before recording with ATS that is finished. Hive query should attempt recording with ATS before going down. (was: Run a hive query and then hit Ctrl+C twice. This kills the hive query instantly before recording with ATS that is has finished. Hive query should attempt recording with ATS before going down. Hive query should record with ATS when Ctrl+C is pressed) Refactor ATSHook constructor to avoid issues of twice CTRL+C terminating running query -- Key: HIVE-9746 URL: https://issues.apache.org/jira/browse/HIVE-9746 Project: Hive Issue Type: Bug Affects Versions: 1.0.0 Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Attachments: Jobs1.png, Jobs2.png Run a hive query and then hit Ctrl+C twice. This kills the hive query instantly before recording with ATS that is finished. Hive query should attempt recording with ATS before going down. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-9746) Refactor ATSHook constructor to avoid issues of twice CTRL+C terminating running query
Xiaobing Zhou created HIVE-9746: --- Summary: Refactor ATSHook constructor to avoid issues of twice CTRL+C terminating running query Key: HIVE-9746 URL: https://issues.apache.org/jira/browse/HIVE-9746 Project: Hive Issue Type: Bug Affects Versions: 1.0.0 Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9480) Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
[
https://issues.apache.org/jira/browse/HIVE-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329964#comment-14329964
]
Xiaobing Zhou commented on HIVE-9480:
-
V7 addressed latest comments.
Thanks [~apivovarov] and [~jdere] for reviews.
Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
Key: HIVE-9480
URL: https://issues.apache.org/jira/browse/HIVE-9480
Project: Hive
Issue Type: Bug
Components: UDF
Affects Versions: 0.14.0
Reporter: Xiaobing Zhou
Assignee: Xiaobing Zhou
Attachments: HIVE-9480.1.patch, HIVE-9480.3.patch, HIVE-9480.4.patch,
HIVE-9480.5.patch, HIVE-9480.6.patch, HIVE-9480.7.patch
Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to
do date/timestamp related computation. This JIRA is to track such an
implementation. Choose to impl TRUNC, a more standard way to get the first
day of a a month, e.g., SELECT TRUNC('2009-12-12', 'MM'); will return
2009-12-01, SELECT TRUNC('2009-12-12', 'YEAR'); will return 2009-01-01.
BTW, this TRUNC is not as feature complete as aligned with Oracle one. only
'MM' and 'YEAR' are supported as format, however, it's a base to add on other
formats.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9480) Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
[
https://issues.apache.org/jira/browse/HIVE-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaobing Zhou updated HIVE-9480:
Attachment: HIVE-9480.7.patch
Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
Key: HIVE-9480
URL: https://issues.apache.org/jira/browse/HIVE-9480
Project: Hive
Issue Type: Bug
Components: UDF
Affects Versions: 0.14.0
Reporter: Xiaobing Zhou
Assignee: Xiaobing Zhou
Attachments: HIVE-9480.1.patch, HIVE-9480.3.patch, HIVE-9480.4.patch,
HIVE-9480.5.patch, HIVE-9480.6.patch, HIVE-9480.7.patch
Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to
do date/timestamp related computation. This JIRA is to track such an
implementation. Choose to impl TRUNC, a more standard way to get the first
day of a a month, e.g., SELECT TRUNC('2009-12-12', 'MM'); will return
2009-12-01, SELECT TRUNC('2009-12-12', 'YEAR'); will return 2009-01-01.
BTW, this TRUNC is not as feature complete as aligned with Oracle one. only
'MM' and 'YEAR' are supported as format, however, it's a base to add on other
formats.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9741) Refactor MetaStoreDirectSql by using getProductName instead of querying DB to determine DbType
[ https://issues.apache.org/jira/browse/HIVE-9741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9741: Attachment: HIVE-9741.1.patch Refactor MetaStoreDirectSql by using getProductName instead of querying DB to determine DbType -- Key: HIVE-9741 URL: https://issues.apache.org/jira/browse/HIVE-9741 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 1.0.0 Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Attachments: HIVE-9741.1.patch MetaStoreDirectSql constructor is querying DB to determine dbType. which leads to too many DB queries to make metastore slow or hanging if MetaStoreDirectSql constructor is frequently called. This is to propose getProductName to get dbType info. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9480) Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
[
https://issues.apache.org/jira/browse/HIVE-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaobing Zhou updated HIVE-9480:
Description:
Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do
date/timestamp related computation. This JIRA is to track such an
implementation. Choose to impl TRUNC, a more standard way to get the first day
of a a month, e.g., SELECT TRUNC('2009-12-12', 'MM'); will return 2009-12-01,
SELECT TRUNC('2009-12-12', 'YEAR'); will return 2009-01-01.
BTW, this TRUNC is not as feature complete as aligned with Oracle one. only
'MM' and 'YEAR' are supported as format, however, it's a base to add on other
formats.
was:
Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do
date/timestamp related computation. This JIRA is to track such an
implementation. Choose to impl TRUNC, a more standard way to get the first day
of a a month, e.g., SELECT TRUNC('2009-01-12', 'MM') FROM src LIMIT 1; will
return 2009-01-01
BTW, this TRUNC is not as feature complete as aligned with Oracle one. 'MM' is
the only format supported, however, it's a base to add on other formats.
Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
Key: HIVE-9480
URL: https://issues.apache.org/jira/browse/HIVE-9480
Project: Hive
Issue Type: Bug
Components: UDF
Affects Versions: 0.14.0
Reporter: Xiaobing Zhou
Assignee: Xiaobing Zhou
Attachments: HIVE-9480.1.patch, HIVE-9480.3.patch, HIVE-9480.4.patch,
HIVE-9480.5.patch
Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to
do date/timestamp related computation. This JIRA is to track such an
implementation. Choose to impl TRUNC, a more standard way to get the first
day of a a month, e.g., SELECT TRUNC('2009-12-12', 'MM'); will return
2009-12-01, SELECT TRUNC('2009-12-12', 'YEAR'); will return 2009-01-01.
BTW, this TRUNC is not as feature complete as aligned with Oracle one. only
'MM' and 'YEAR' are supported as format, however, it's a base to add on other
formats.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-9741) Refactor MetaStoreDirectSql by using getProductName instead of querying DB to determine DbType
Xiaobing Zhou created HIVE-9741: --- Summary: Refactor MetaStoreDirectSql by using getProductName instead of querying DB to determine DbType Key: HIVE-9741 URL: https://issues.apache.org/jira/browse/HIVE-9741 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 1.0.0 Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Attachments: HIVE-9741.1.patch MetaStoreDirectSql constructor is querying DB to determine dbType. which leads to too many DB queries to make metastore slow or hanging if MetaStoreDirectSql constructor is frequently called. This is to propose getProductName to get dbType info. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9746) Refactor ATSHook constructor to avoid issues of twice CTRL+C terminating running query
[ https://issues.apache.org/jira/browse/HIVE-9746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9746: Attachment: Jobs2.png Jobs1.png Refactor ATSHook constructor to avoid issues of twice CTRL+C terminating running query -- Key: HIVE-9746 URL: https://issues.apache.org/jira/browse/HIVE-9746 Project: Hive Issue Type: Bug Affects Versions: 1.0.0 Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Attachments: Jobs1.png, Jobs2.png Run a hive query and then hit Ctrl+C twice. This kills the hive query instantly before recording with ATS that is has finished. Hive query should attempt recording with ATS before going down. Hive query should record with ATS when Ctrl+C is pressed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9746) Refactor ATSHook constructor to avoid issues of twice CTRL+C terminating running query
[ https://issues.apache.org/jira/browse/HIVE-9746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9746: Description: Run a hive query and then hit Ctrl+C twice. This kills the hive query instantly before recording with ATS that is has finished. Hive query should attempt recording with ATS before going down. Hive query should record with ATS when Ctrl+C is pressed Refactor ATSHook constructor to avoid issues of twice CTRL+C terminating running query -- Key: HIVE-9746 URL: https://issues.apache.org/jira/browse/HIVE-9746 Project: Hive Issue Type: Bug Affects Versions: 1.0.0 Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Run a hive query and then hit Ctrl+C twice. This kills the hive query instantly before recording with ATS that is has finished. Hive query should attempt recording with ATS before going down. Hive query should record with ATS when Ctrl+C is pressed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9480) Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
[
https://issues.apache.org/jira/browse/HIVE-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14328422#comment-14328422
]
Xiaobing Zhou commented on HIVE-9480:
-
[~apivovarov] [~jdere] can you review it? Thanks.
Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
Key: HIVE-9480
URL: https://issues.apache.org/jira/browse/HIVE-9480
Project: Hive
Issue Type: Bug
Components: UDF
Affects Versions: 0.14.0
Reporter: Xiaobing Zhou
Assignee: Xiaobing Zhou
Attachments: HIVE-9480.1.patch, HIVE-9480.3.patch, HIVE-9480.4.patch,
HIVE-9480.5.patch
Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to
do date/timestamp related computation. This JIRA is to track such an
implementation. Choose to impl TRUNC, a more standard way to get the first
day of a a month, e.g., SELECT TRUNC('2009-01-12', 'MM') FROM src LIMIT 1;
will return 2009-01-01
BTW, this TRUNC is not as feature complete as aligned with Oracle one. 'MM'
is the only format supported, however, it's a base to add on other formats.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9480) Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
[
https://issues.apache.org/jira/browse/HIVE-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaobing Zhou updated HIVE-9480:
Attachment: HIVE-9480.5.patch
Patch V5 addressed review comments.
Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
Key: HIVE-9480
URL: https://issues.apache.org/jira/browse/HIVE-9480
Project: Hive
Issue Type: Bug
Components: UDF
Affects Versions: 0.14.0
Reporter: Xiaobing Zhou
Assignee: Xiaobing Zhou
Attachments: HIVE-9480.1.patch, HIVE-9480.3.patch, HIVE-9480.4.patch,
HIVE-9480.5.patch
Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to
do date/timestamp related computation. This JIRA is to track such an
implementation. Choose to impl TRUNC, a more standard way to get the first
day of a a month, e.g., SELECT TRUNC('2009-01-12', 'MM') FROM src LIMIT 1;
will return 2009-01-01
BTW, this TRUNC is not as feature complete as aligned with Oracle one. 'MM'
is the only format supported, however, it's a base to add on other formats.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-9642) Hive metastore client retries don't happen consistently for all api calls
Xiaobing Zhou created HIVE-9642: --- Summary: Hive metastore client retries don't happen consistently for all api calls Key: HIVE-9642 URL: https://issues.apache.org/jira/browse/HIVE-9642 Project: Hive Issue Type: Bug Affects Versions: 1.0.0 Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou When org.apache.thrift.transport.TTransportException is thrown for issues like socket timeout, the retry via RetryingMetaStoreClient happens only in certain cases. Retry happens for the getDatabase call in but not for getAllDatabases(). The reason is RetryingMetaStoreClient checks for TTransportException being the cause for InvocationTargetException. But in case of some calls such as getAllDatabases in HiveMetastoreClient, all exceptions get wrapped in a MetaException. We should remove this unnecessary wrapping of exceptions for certain functions in HMC. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9480) Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
[ https://issues.apache.org/jira/browse/HIVE-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9480: Summary: Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY (was: Implement FIRST_DAY as compared with LAST_DAY already supported in HIVE) Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY Key: HIVE-9480 URL: https://issues.apache.org/jira/browse/HIVE-9480 Project: Hive Issue Type: Bug Components: UDF Affects Versions: 0.14.0 Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Attachments: HIVE-9480.1.patch, HIVE-9480.3.patch Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do date/timestamp related computation. This JIRA is to track such an implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9480) Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
[ https://issues.apache.org/jira/browse/HIVE-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9480: Attachment: HIVE-9480.4.patch Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY Key: HIVE-9480 URL: https://issues.apache.org/jira/browse/HIVE-9480 Project: Hive Issue Type: Bug Components: UDF Affects Versions: 0.14.0 Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Attachments: HIVE-9480.1.patch, HIVE-9480.3.patch, HIVE-9480.4.patch Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do date/timestamp related computation. This JIRA is to track such an implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9480) Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
[
https://issues.apache.org/jira/browse/HIVE-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaobing Zhou updated HIVE-9480:
Description:
Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do
date/timestamp related computation. This JIRA is to track such an
implementation. Choose to impl TRUNC, a more standard way to get the first day
of a a month, e.g., SELECT TRUNC('2009-01-12', 'MM') FROM src LIMIT 1; will
return 2009-01-01
However, this TRUNC is not as feature complete as aligned with Oracle one. 'MM'
is the only format supported.
was:Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary
to do date/timestamp related computation. This JIRA is to track such an
implementation.
Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
Key: HIVE-9480
URL: https://issues.apache.org/jira/browse/HIVE-9480
Project: Hive
Issue Type: Bug
Components: UDF
Affects Versions: 0.14.0
Reporter: Xiaobing Zhou
Assignee: Xiaobing Zhou
Attachments: HIVE-9480.1.patch, HIVE-9480.3.patch, HIVE-9480.4.patch
Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to
do date/timestamp related computation. This JIRA is to track such an
implementation. Choose to impl TRUNC, a more standard way to get the first
day of a a month, e.g., SELECT TRUNC('2009-01-12', 'MM') FROM src LIMIT 1;
will return 2009-01-01
However, this TRUNC is not as feature complete as aligned with Oracle one.
'MM' is the only format supported.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9480) Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
[ https://issues.apache.org/jira/browse/HIVE-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9480: Status: Patch Available (was: Open) Made patch V4. Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY Key: HIVE-9480 URL: https://issues.apache.org/jira/browse/HIVE-9480 Project: Hive Issue Type: Bug Components: UDF Affects Versions: 0.14.0 Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Attachments: HIVE-9480.1.patch, HIVE-9480.3.patch, HIVE-9480.4.patch Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do date/timestamp related computation. This JIRA is to track such an implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9518) Implement MONTHS_BETWEEN aligned with Oracle one
[ https://issues.apache.org/jira/browse/HIVE-9518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14308268#comment-14308268 ] Xiaobing Zhou commented on HIVE-9518: - The failure is irrelevant. Implement MONTHS_BETWEEN aligned with Oracle one Key: HIVE-9518 URL: https://issues.apache.org/jira/browse/HIVE-9518 Project: Hive Issue Type: Bug Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Attachments: HIVE-9518.1.patch, HIVE-9518.2.patch, HIVE-9518.3.patch This is used to track work to build Oracle like months_between. Here's semantics: MONTHS_BETWEEN returns number of months between dates date1 and date2. If date1 is later than date2, then the result is positive. If date1 is earlier than date2, then the result is negative. If date1 and date2 are either the same days of the month or both last days of months, then the result is always an integer. Otherwise Oracle Database calculates the fractional portion of the result based on a 31-day month and considers the difference in time components date1 and date2. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9480) Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
[
https://issues.apache.org/jira/browse/HIVE-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14308304#comment-14308304
]
Xiaobing Zhou commented on HIVE-9480:
-
[~apivovarov] yes, I can do that in next patch for YEAR. Could you please
review V4?
Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
Key: HIVE-9480
URL: https://issues.apache.org/jira/browse/HIVE-9480
Project: Hive
Issue Type: Bug
Components: UDF
Affects Versions: 0.14.0
Reporter: Xiaobing Zhou
Assignee: Xiaobing Zhou
Attachments: HIVE-9480.1.patch, HIVE-9480.3.patch, HIVE-9480.4.patch
Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to
do date/timestamp related computation. This JIRA is to track such an
implementation. Choose to impl TRUNC, a more standard way to get the first
day of a a month, e.g., SELECT TRUNC('2009-01-12', 'MM') FROM src LIMIT 1;
will return 2009-01-01
BTW, this TRUNC is not as feature complete as aligned with Oracle one. 'MM'
is the only format supported, however, it's a base to add on other formats.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9480) Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
[
https://issues.apache.org/jira/browse/HIVE-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaobing Zhou updated HIVE-9480:
Description:
Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do
date/timestamp related computation. This JIRA is to track such an
implementation. Choose to impl TRUNC, a more standard way to get the first day
of a a month, e.g., SELECT TRUNC('2009-01-12', 'MM') FROM src LIMIT 1; will
return 2009-01-01
BTW, this TRUNC is not as feature complete as aligned with Oracle one. 'MM' is
the only format supported, however, it's a base to add on other formats.
was:
Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do
date/timestamp related computation. This JIRA is to track such an
implementation. Choose to impl TRUNC, a more standard way to get the first day
of a a month, e.g., SELECT TRUNC('2009-01-12', 'MM') FROM src LIMIT 1; will
return 2009-01-01
However, this TRUNC is not as feature complete as aligned with Oracle one. 'MM'
is the only format supported.
Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
Key: HIVE-9480
URL: https://issues.apache.org/jira/browse/HIVE-9480
Project: Hive
Issue Type: Bug
Components: UDF
Affects Versions: 0.14.0
Reporter: Xiaobing Zhou
Assignee: Xiaobing Zhou
Attachments: HIVE-9480.1.patch, HIVE-9480.3.patch, HIVE-9480.4.patch
Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to
do date/timestamp related computation. This JIRA is to track such an
implementation. Choose to impl TRUNC, a more standard way to get the first
day of a a month, e.g., SELECT TRUNC('2009-01-12', 'MM') FROM src LIMIT 1;
will return 2009-01-01
BTW, this TRUNC is not as feature complete as aligned with Oracle one. 'MM'
is the only format supported, however, it's a base to add on other formats.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9518) Implement MONTHS_BETWEEN aligned with Oracle one
[ https://issues.apache.org/jira/browse/HIVE-9518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14306081#comment-14306081 ] Xiaobing Zhou commented on HIVE-9518: - Patch V3. [~jdere] and [~apivovarov] can you review it? Thanks. Implement MONTHS_BETWEEN aligned with Oracle one Key: HIVE-9518 URL: https://issues.apache.org/jira/browse/HIVE-9518 Project: Hive Issue Type: Bug Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Attachments: HIVE-9518.1.patch, HIVE-9518.2.patch, HIVE-9518.3.patch This is used to track work to build Oracle like months_between. Here's semantics: MONTHS_BETWEEN returns number of months between dates date1 and date2. If date1 is later than date2, then the result is positive. If date1 is earlier than date2, then the result is negative. If date1 and date2 are either the same days of the month or both last days of months, then the result is always an integer. Otherwise Oracle Database calculates the fractional portion of the result based on a 31-day month and considers the difference in time components date1 and date2. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9518) Implement MONTHS_BETWEEN aligned with Oracle one
[ https://issues.apache.org/jira/browse/HIVE-9518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9518: Attachment: HIVE-9518.3.patch Implement MONTHS_BETWEEN aligned with Oracle one Key: HIVE-9518 URL: https://issues.apache.org/jira/browse/HIVE-9518 Project: Hive Issue Type: Bug Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Attachments: HIVE-9518.1.patch, HIVE-9518.2.patch, HIVE-9518.3.patch This is used to track work to build Oracle like months_between. Here's semantics: MONTHS_BETWEEN returns number of months between dates date1 and date2. If date1 is later than date2, then the result is positive. If date1 is earlier than date2, then the result is negative. If date1 and date2 are either the same days of the month or both last days of months, then the result is always an integer. Otherwise Oracle Database calculates the fractional portion of the result based on a 31-day month and considers the difference in time components date1 and date2. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9518) Implement MONTHS_BETWEEN aligned with Oracle one
[ https://issues.apache.org/jira/browse/HIVE-9518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14306124#comment-14306124 ] Xiaobing Zhou commented on HIVE-9518: - Just did without --no-prefix. Fail to upload it to RB with --no-prefix. Implement MONTHS_BETWEEN aligned with Oracle one Key: HIVE-9518 URL: https://issues.apache.org/jira/browse/HIVE-9518 Project: Hive Issue Type: Bug Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Attachments: HIVE-9518.1.patch, HIVE-9518.2.patch, HIVE-9518.3.patch This is used to track work to build Oracle like months_between. Here's semantics: MONTHS_BETWEEN returns number of months between dates date1 and date2. If date1 is later than date2, then the result is positive. If date1 is earlier than date2, then the result is negative. If date1 and date2 are either the same days of the month or both last days of months, then the result is always an integer. Otherwise Oracle Database calculates the fractional portion of the result based on a 31-day month and considers the difference in time components date1 and date2. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9518) Implement MONTHS_BETWEEN aligned with Oracle one
[ https://issues.apache.org/jira/browse/HIVE-9518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9518: Attachment: HIVE-9518.3.patch Implement MONTHS_BETWEEN aligned with Oracle one Key: HIVE-9518 URL: https://issues.apache.org/jira/browse/HIVE-9518 Project: Hive Issue Type: Bug Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Attachments: HIVE-9518.1.patch, HIVE-9518.2.patch, HIVE-9518.3.patch This is used to track work to build Oracle like months_between. Here's semantics: MONTHS_BETWEEN returns number of months between dates date1 and date2. If date1 is later than date2, then the result is positive. If date1 is earlier than date2, then the result is negative. If date1 and date2 are either the same days of the month or both last days of months, then the result is always an integer. Otherwise Oracle Database calculates the fractional portion of the result based on a 31-day month and considers the difference in time components date1 and date2. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9518) Implement MONTHS_BETWEEN aligned with Oracle one
[ https://issues.apache.org/jira/browse/HIVE-9518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9518: Attachment: (was: HIVE-9518.3.patch) Implement MONTHS_BETWEEN aligned with Oracle one Key: HIVE-9518 URL: https://issues.apache.org/jira/browse/HIVE-9518 Project: Hive Issue Type: Bug Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Attachments: HIVE-9518.1.patch, HIVE-9518.2.patch, HIVE-9518.3.patch This is used to track work to build Oracle like months_between. Here's semantics: MONTHS_BETWEEN returns number of months between dates date1 and date2. If date1 is later than date2, then the result is positive. If date1 is earlier than date2, then the result is negative. If date1 and date2 are either the same days of the month or both last days of months, then the result is always an integer. Otherwise Oracle Database calculates the fractional portion of the result based on a 31-day month and considers the difference in time components date1 and date2. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9472) Implement 7 simple UDFs added to Hive
[
https://issues.apache.org/jira/browse/HIVE-9472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14298066#comment-14298066
]
Xiaobing Zhou commented on HIVE-9472:
-
This work is split to separate JIRA(s) for tracking. SEE ALSO 'relates to'
links.
Implement 7 simple UDFs added to Hive
-
Key: HIVE-9472
URL: https://issues.apache.org/jira/browse/HIVE-9472
Project: Hive
Issue Type: Bug
Components: UDF
Affects Versions: 0.14.0
Reporter: Xiaobing Zhou
Assignee: Xiaobing Zhou
PROBLEM: Hive doesn't not have the following built-in functions.
1) MONTHS_BETWEEN(d1,d2)
2) LAST_DAY(d)
3) FIRST_DAY(d)
4) ADD_MONTHS (date, integer)
5) CHR()
6) REPLACE(char, search_string, replacement_string)
7) TRANSLATE(expr, from_string, to_string)
Without these, not able to do certain analysis.
USE CASE:
1) MONTHS_BETWEEN(d1,d2)
2) LAST_DAY(d)
3) FIRST_DAY(d)
4) ADD_MONTHS (date, integer)
ADD_MONTHS returns the date date plus integer months. If date is the last day
of the month or if the resulting month has fewer days than the day component
of date, then the result is the last day of the resulting month. Otherwise,
the result has the same day component as date.
5) CHR()
CHR returns the character having the binary equivalent to n (where n is
between 0 and 255)as a VARCHAR2 value
6) REPLACE(char, search_string, replacement_string)
REPLACE returns char with every occurrence of search_string replaced with
replacement_string.
SELECT REPLACE('JACK and JUE','J','BL') Changes;
Changes
--
BLACK and BLUE
7) TRANSLATE(expr, from_string, to_string)
TRANSLATE returns expr with all occurrences of each character in from_string
replaced by its corresponding character in to_string.
SELECT TRANSLATE('2KR 229', '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ ',
'99XX-') (CHAR(26)) License;
License
---
9XX-999
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9518) Implement MONTHS_BETWEEN aligned with Oracle one
[ https://issues.apache.org/jira/browse/HIVE-9518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9518: Description: This is used to track work to build a Oracle like months_between. Here's semantics: MONTHS_BETWEEN returns number of months between dates date1 and date2. If date1 is later than date2, then the result is positive. If date1 is earlier than date2, then the result is negative. If date1 and date2 are either the same days of the month or both last days of months, then the result is always an integer. Otherwise Oracle Database calculates the fractional portion of the result based on a 31-day month and considers the difference in time components date1 and date2. Implement MONTHS_BETWEEN aligned with Oracle one Key: HIVE-9518 URL: https://issues.apache.org/jira/browse/HIVE-9518 Project: Hive Issue Type: Bug Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou This is used to track work to build a Oracle like months_between. Here's semantics: MONTHS_BETWEEN returns number of months between dates date1 and date2. If date1 is later than date2, then the result is positive. If date1 is earlier than date2, then the result is negative. If date1 and date2 are either the same days of the month or both last days of months, then the result is always an integer. Otherwise Oracle Database calculates the fractional portion of the result based on a 31-day month and considers the difference in time components date1 and date2. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9518) Implement MONTHS_BETWEEN aligned with Oracle one
[ https://issues.apache.org/jira/browse/HIVE-9518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9518: Description: This is used to track work to build Oracle like months_between. Here's semantics: MONTHS_BETWEEN returns number of months between dates date1 and date2. If date1 is later than date2, then the result is positive. If date1 is earlier than date2, then the result is negative. If date1 and date2 are either the same days of the month or both last days of months, then the result is always an integer. Otherwise Oracle Database calculates the fractional portion of the result based on a 31-day month and considers the difference in time components date1 and date2. was: This is used to track work to build a Oracle like months_between. Here's semantics: MONTHS_BETWEEN returns number of months between dates date1 and date2. If date1 is later than date2, then the result is positive. If date1 is earlier than date2, then the result is negative. If date1 and date2 are either the same days of the month or both last days of months, then the result is always an integer. Otherwise Oracle Database calculates the fractional portion of the result based on a 31-day month and considers the difference in time components date1 and date2. Implement MONTHS_BETWEEN aligned with Oracle one Key: HIVE-9518 URL: https://issues.apache.org/jira/browse/HIVE-9518 Project: Hive Issue Type: Bug Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Attachments: HIVE-9518.1.patch This is used to track work to build Oracle like months_between. Here's semantics: MONTHS_BETWEEN returns number of months between dates date1 and date2. If date1 is later than date2, then the result is positive. If date1 is earlier than date2, then the result is negative. If date1 and date2 are either the same days of the month or both last days of months, then the result is always an integer. Otherwise Oracle Database calculates the fractional portion of the result based on a 31-day month and considers the difference in time components date1 and date2. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9518) Implement MONTHS_BETWEEN aligned with Oracle one
[ https://issues.apache.org/jira/browse/HIVE-9518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9518: Attachment: HIVE-9518.2.patch Implement MONTHS_BETWEEN aligned with Oracle one Key: HIVE-9518 URL: https://issues.apache.org/jira/browse/HIVE-9518 Project: Hive Issue Type: Bug Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Attachments: HIVE-9518.1.patch, HIVE-9518.2.patch This is used to track work to build Oracle like months_between. Here's semantics: MONTHS_BETWEEN returns number of months between dates date1 and date2. If date1 is later than date2, then the result is positive. If date1 is earlier than date2, then the result is negative. If date1 and date2 are either the same days of the month or both last days of months, then the result is always an integer. Otherwise Oracle Database calculates the fractional portion of the result based on a 31-day month and considers the difference in time components date1 and date2. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-9518) Implement MONTHS_BETWEEN aligned with Oracle one
Xiaobing Zhou created HIVE-9518: --- Summary: Implement MONTHS_BETWEEN aligned with Oracle one Key: HIVE-9518 URL: https://issues.apache.org/jira/browse/HIVE-9518 Project: Hive Issue Type: Bug Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9480) Implement FIRST_DAY as compared with LAST_DAY already supported in HIVE
[ https://issues.apache.org/jira/browse/HIVE-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9480: Attachment: HIVE-9480.3.patch Patch V3, added license comments. Implement FIRST_DAY as compared with LAST_DAY already supported in HIVE --- Key: HIVE-9480 URL: https://issues.apache.org/jira/browse/HIVE-9480 Project: Hive Issue Type: Bug Components: UDF Affects Versions: 0.14.0 Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Attachments: HIVE-9480.1.patch, HIVE-9480.3.patch Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do date/timestamp related computation. This JIRA is to track such an implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9518) Implement MONTHS_BETWEEN aligned with Oracle one
[ https://issues.apache.org/jira/browse/HIVE-9518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9518: Attachment: HIVE-9518.1.patch Implement MONTHS_BETWEEN aligned with Oracle one Key: HIVE-9518 URL: https://issues.apache.org/jira/browse/HIVE-9518 Project: Hive Issue Type: Bug Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Attachments: HIVE-9518.1.patch This is used to track work to build a Oracle like months_between. Here's semantics: MONTHS_BETWEEN returns number of months between dates date1 and date2. If date1 is later than date2, then the result is positive. If date1 is earlier than date2, then the result is negative. If date1 and date2 are either the same days of the month or both last days of months, then the result is always an integer. Otherwise Oracle Database calculates the fractional portion of the result based on a 31-day month and considers the difference in time components date1 and date2. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9518) Implement MONTHS_BETWEEN aligned with Oracle one
[ https://issues.apache.org/jira/browse/HIVE-9518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14298064#comment-14298064 ] Xiaobing Zhou commented on HIVE-9518: - [~apivovarov] [~jdere] [~ashutoshc] Could you please look at it for review? Thanks! Implement MONTHS_BETWEEN aligned with Oracle one Key: HIVE-9518 URL: https://issues.apache.org/jira/browse/HIVE-9518 Project: Hive Issue Type: Bug Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Attachments: HIVE-9518.1.patch, HIVE-9518.2.patch This is used to track work to build Oracle like months_between. Here's semantics: MONTHS_BETWEEN returns number of months between dates date1 and date2. If date1 is later than date2, then the result is positive. If date1 is earlier than date2, then the result is negative. If date1 and date2 are either the same days of the month or both last days of months, then the result is always an integer. Otherwise Oracle Database calculates the fractional portion of the result based on a 31-day month and considers the difference in time components date1 and date2. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9456) Make Hive support unicode with MSSQL as Metastore backend
[
https://issues.apache.org/jira/browse/HIVE-9456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14294244#comment-14294244
]
Xiaobing Zhou commented on HIVE-9456:
-
[~ekoifman] Here's test case.
{noformat}
1. New empty metastore's created with 0.14 schema.
2. upgrade-0.14.0-to-0.15.0.mssql.sql is run manually to get 0.14 metastore
upgraded to 0.15.
3. Double check it to make sure every orginal varchar column changed to
nvarchar.
{noformat}
As you suggested, it'd be better to do test in context HIVE/HIVE CLI/, e.g.,
1. New empty metastore's created with 0.14 schema.
2. create some hive tables/load real data
3. shutdown metastore server
4. run schemetool to do upgrade
5. start metastore server
6. query those tables inited in step 2 to make sure things are correct.
I will do that. Thanks!
Make Hive support unicode with MSSQL as Metastore backend
-
Key: HIVE-9456
URL: https://issues.apache.org/jira/browse/HIVE-9456
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.14.0
Reporter: Xiaobing Zhou
Assignee: Xiaobing Zhou
Attachments: HIVE-9456.1.patch
There are significant issues when Hive uses MSSQL as metastore backend to
support unicode, since MSSQL handles varchar and nvarchar datatypes
differently. Hive 0.14 metastore mssql script DDL was using varchar as
datatype, which can't handle multi-bytes/unicode characters, e.g., Chinese
chars. This JIRA is going to track implementation of unicode support in that
case.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9480) Implement FIRST_DAY as compared with LAST_DAY already supported in HIVE
[ https://issues.apache.org/jira/browse/HIVE-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14294315#comment-14294315 ] Xiaobing Zhou commented on HIVE-9480: - Since most of logic for FIRST_DAY and LAST_DAY are common. It'd be better to reuse LAST_DAY impl and make some abstractions to simplify FIRST_DAY impl. Implement FIRST_DAY as compared with LAST_DAY already supported in HIVE --- Key: HIVE-9480 URL: https://issues.apache.org/jira/browse/HIVE-9480 Project: Hive Issue Type: Bug Components: UDF Affects Versions: 0.14.0 Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do date/timestamp related computation. This JIRA is to track such an implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9480) Implement FIRST_DAY as compared with LAST_DAY already supported in HIVE
[ https://issues.apache.org/jira/browse/HIVE-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9480: Description: Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do date/timestamp related computation. This JIRA is to track such an implementation. (was: Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to date/timestamp related computation. This JIRA is to track such an implementation.) Implement FIRST_DAY as compared with LAST_DAY already supported in HIVE --- Key: HIVE-9480 URL: https://issues.apache.org/jira/browse/HIVE-9480 Project: Hive Issue Type: Bug Components: UDF Affects Versions: 0.14.0 Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do date/timestamp related computation. This JIRA is to track such an implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-9480) Implement FIRST_DAY as compared with LAST_DAY already supported in HIVE
Xiaobing Zhou created HIVE-9480: --- Summary: Implement FIRST_DAY as compared with LAST_DAY already supported in HIVE Key: HIVE-9480 URL: https://issues.apache.org/jira/browse/HIVE-9480 Project: Hive Issue Type: Bug Components: UDF Affects Versions: 0.14.0 Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to date/timestamp related computation. This JIRA is to track such an implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9480) Implement FIRST_DAY as compared with LAST_DAY already supported in HIVE
[ https://issues.apache.org/jira/browse/HIVE-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9480: Attachment: HIVE-9480.1.patch Made patch v1. [~jdere] could you please have it reviewed? Thanks! Implement FIRST_DAY as compared with LAST_DAY already supported in HIVE --- Key: HIVE-9480 URL: https://issues.apache.org/jira/browse/HIVE-9480 Project: Hive Issue Type: Bug Components: UDF Affects Versions: 0.14.0 Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Attachments: HIVE-9480.1.patch Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to do date/timestamp related computation. This JIRA is to track such an implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-9472) Implement 7 simple UDFs added to Hive
Xiaobing Zhou created HIVE-9472:
---
Summary: Implement 7 simple UDFs added to Hive
Key: HIVE-9472
URL: https://issues.apache.org/jira/browse/HIVE-9472
Project: Hive
Issue Type: Bug
Components: UDF
Affects Versions: 0.14.0
Reporter: Xiaobing Zhou
PROBLEM: Hive doesn't not have the following built-in functions.
1) MONTHS_BETWEEN(d1,d2)
2) LAST_DAY(d)
3) FIRST_DAY(d)
4) ADD_MONTHS (date, integer)
5) CHR()
6) REPLACE(char, search_string, replacement_string)
7) TRANSLATE(expr, from_string, to_string)
Without these, not able to do certain analysis.
USE CASE:
1) MONTHS_BETWEEN(d1,d2)
2) LAST_DAY(d)
3) FIRST_DAY(d)
4) ADD_MONTHS (date, integer)
ADD_MONTHS returns the date date plus integer months. If date is the last day
of the month or if the resulting month has fewer days than the day component of
date, then the result is the last day of the resulting month. Otherwise, the
result has the same day component as date.
5) CHR()
CHR returns the character having the binary equivalent to n (where n is between
0 and 255)as a VARCHAR2 value
6) REPLACE(char, search_string, replacement_string)
REPLACE returns char with every occurrence of search_string replaced with
replacement_string.
SELECT REPLACE('JACK and JUE','J','BL') Changes;
Changes
--
BLACK and BLUE
7) TRANSLATE(expr, from_string, to_string)
TRANSLATE returns expr with all occurrences of each character in from_string
replaced by its corresponding character in to_string.
SELECT TRANSLATE('2KR 229', '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ ',
'99XX-') (CHAR(26)) License;
License
---
9XX-999
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HIVE-9472) Implement 7 simple UDFs added to Hive
[
https://issues.apache.org/jira/browse/HIVE-9472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaobing Zhou reassigned HIVE-9472:
---
Assignee: Xiaobing Zhou
Implement 7 simple UDFs added to Hive
-
Key: HIVE-9472
URL: https://issues.apache.org/jira/browse/HIVE-9472
Project: Hive
Issue Type: Bug
Components: UDF
Affects Versions: 0.14.0
Reporter: Xiaobing Zhou
Assignee: Xiaobing Zhou
PROBLEM: Hive doesn't not have the following built-in functions.
1) MONTHS_BETWEEN(d1,d2)
2) LAST_DAY(d)
3) FIRST_DAY(d)
4) ADD_MONTHS (date, integer)
5) CHR()
6) REPLACE(char, search_string, replacement_string)
7) TRANSLATE(expr, from_string, to_string)
Without these, not able to do certain analysis.
USE CASE:
1) MONTHS_BETWEEN(d1,d2)
2) LAST_DAY(d)
3) FIRST_DAY(d)
4) ADD_MONTHS (date, integer)
ADD_MONTHS returns the date date plus integer months. If date is the last day
of the month or if the resulting month has fewer days than the day component
of date, then the result is the last day of the resulting month. Otherwise,
the result has the same day component as date.
5) CHR()
CHR returns the character having the binary equivalent to n (where n is
between 0 and 255)as a VARCHAR2 value
6) REPLACE(char, search_string, replacement_string)
REPLACE returns char with every occurrence of search_string replaced with
replacement_string.
SELECT REPLACE('JACK and JUE','J','BL') Changes;
Changes
--
BLACK and BLUE
7) TRANSLATE(expr, from_string, to_string)
TRANSLATE returns expr with all occurrences of each character in from_string
replaced by its corresponding character in to_string.
SELECT TRANSLATE('2KR 229', '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ ',
'99XX-') (CHAR(26)) License;
License
---
9XX-999
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9472) Implement 7 simple UDFs added to Hive
[
https://issues.apache.org/jira/browse/HIVE-9472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14292607#comment-14292607
]
Xiaobing Zhou commented on HIVE-9472:
-
Checked trunk, there's a UDF 'last_day' already supported.
Implement 7 simple UDFs added to Hive
-
Key: HIVE-9472
URL: https://issues.apache.org/jira/browse/HIVE-9472
Project: Hive
Issue Type: Bug
Components: UDF
Affects Versions: 0.14.0
Reporter: Xiaobing Zhou
Assignee: Xiaobing Zhou
PROBLEM: Hive doesn't not have the following built-in functions.
1) MONTHS_BETWEEN(d1,d2)
2) LAST_DAY(d)
3) FIRST_DAY(d)
4) ADD_MONTHS (date, integer)
5) CHR()
6) REPLACE(char, search_string, replacement_string)
7) TRANSLATE(expr, from_string, to_string)
Without these, not able to do certain analysis.
USE CASE:
1) MONTHS_BETWEEN(d1,d2)
2) LAST_DAY(d)
3) FIRST_DAY(d)
4) ADD_MONTHS (date, integer)
ADD_MONTHS returns the date date plus integer months. If date is the last day
of the month or if the resulting month has fewer days than the day component
of date, then the result is the last day of the resulting month. Otherwise,
the result has the same day component as date.
5) CHR()
CHR returns the character having the binary equivalent to n (where n is
between 0 and 255)as a VARCHAR2 value
6) REPLACE(char, search_string, replacement_string)
REPLACE returns char with every occurrence of search_string replaced with
replacement_string.
SELECT REPLACE('JACK and JUE','J','BL') Changes;
Changes
--
BLACK and BLUE
7) TRANSLATE(expr, from_string, to_string)
TRANSLATE returns expr with all occurrences of each character in from_string
replaced by its corresponding character in to_string.
SELECT TRANSLATE('2KR 229', '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ ',
'99XX-') (CHAR(26)) License;
License
---
9XX-999
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9456) Make Hive support unicode with MSSQL as Metastore backend
[ https://issues.apache.org/jira/browse/HIVE-9456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9456: Description: There are significant issues when Hive uses MSSQL as (was: hive-schema-0.15.0.mssql.sql has PARTITIONS.PART_NAME as NVARCHAR but COMPLETED_TXN_COMPONENTS.CTC_PARTITON, COMPACTION_QUEUE.CQ_PARTITION, HIVE_LOCKS.HL_PARTITION, TXN_COMPONENTS.TC_PARTITION all use VARCHAR. This cannot be right since they all store the same value. the same is true of hive-schema-0.14.0.mssql.sql and the two corresponding hvie-txn-schema-... files.) Make Hive support unicode with MSSQL as Metastore backend - Key: HIVE-9456 URL: https://issues.apache.org/jira/browse/HIVE-9456 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.14.0 Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou There are significant issues when Hive uses MSSQL as -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-9456) Make Hive support unicode with MSSQL as Metastore backend
Xiaobing Zhou created HIVE-9456: --- Summary: Make Hive support unicode with MSSQL as Metastore backend Key: HIVE-9456 URL: https://issues.apache.org/jira/browse/HIVE-9456 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.14.0 Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou hive-schema-0.15.0.mssql.sql has PARTITIONS.PART_NAME as NVARCHAR but COMPLETED_TXN_COMPONENTS.CTC_PARTITON, COMPACTION_QUEUE.CQ_PARTITION, HIVE_LOCKS.HL_PARTITION, TXN_COMPONENTS.TC_PARTITION all use VARCHAR. This cannot be right since they all store the same value. the same is true of hive-schema-0.14.0.mssql.sql and the two corresponding hvie-txn-schema-... files. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9456) Make Hive support unicode with MSSQL as Metastore backend
[ https://issues.apache.org/jira/browse/HIVE-9456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9456: Attachment: HIVE-9456.1.patch Made a patch. Could anyone have it reviewed? Thanks! Make Hive support unicode with MSSQL as Metastore backend - Key: HIVE-9456 URL: https://issues.apache.org/jira/browse/HIVE-9456 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.14.0 Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Attachments: HIVE-9456.1.patch There are significant issues when Hive uses MSSQL as metastore backend to support unicode, since MSSQL handles varchar and nvarchar datatypes differently. Hive 0.14 metastore mssql script DDL was using varchar as datatype, which can't handle multi-bytes/unicode characters, e.g., Chinese chars. This JIRA is going to track implementation of unicode support in that case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HIVE-9208) MetaStore DB schema inconsistent for MS SQL Server in use of varchar/nvarchar
[ https://issues.apache.org/jira/browse/HIVE-9208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou resolved HIVE-9208. - Resolution: Won't Fix HIVE-9456 is going to change all varchar to nvarchar in 0.15 and provide upgrade script form 0.14 to 0.15, so there's no consistency issue like the case. Marked it as won't fix. MetaStore DB schema inconsistent for MS SQL Server in use of varchar/nvarchar - Key: HIVE-9208 URL: https://issues.apache.org/jira/browse/HIVE-9208 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.14.0 Reporter: Eugene Koifman Assignee: Xiaobing Zhou Attachments: HIVE-9208.1.patch, HIVE-9208.2.patch hive-schema-0.15.0.mssql.sql has PARTITIONS.PART_NAME as NVARCHAR but COMPLETED_TXN_COMPONENTS.CTC_PARTITON, COMPACTION_QUEUE.CQ_PARTITION, HIVE_LOCKS.HL_PARTITION, TXN_COMPONENTS.TC_PARTITION all use VARCHAR. This cannot be right since they all store the same value. the same is true of hive-schema-0.14.0.mssql.sql and the two corresponding hvie-txn-schema-... files. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9456) Make Hive support unicode with MSSQL as Metastore backend
[ https://issues.apache.org/jira/browse/HIVE-9456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9456: Description: There are significant issues when Hive uses MSSQL as metastore backend to support unicode, since MSSQL handles varchar and nvarchar datatypes differently. Hive 0.14 metastore mssql script DDL was using varchar as datatype, which can't handle multi-bytes/unicode characters, e.g., Chinese chars. This JIRA is going to track implementation of unicode support in that case. (was: There are significant issues when Hive uses MSSQL as ) Make Hive support unicode with MSSQL as Metastore backend - Key: HIVE-9456 URL: https://issues.apache.org/jira/browse/HIVE-9456 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.14.0 Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou There are significant issues when Hive uses MSSQL as metastore backend to support unicode, since MSSQL handles varchar and nvarchar datatypes differently. Hive 0.14 metastore mssql script DDL was using varchar as datatype, which can't handle multi-bytes/unicode characters, e.g., Chinese chars. This JIRA is going to track implementation of unicode support in that case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9208) MetaStore DB schema inconsistent for MS SQL Server in use of varchar/nvarchar
[ https://issues.apache.org/jira/browse/HIVE-9208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14288271#comment-14288271 ] Xiaobing Zhou commented on HIVE-9208: - I don't think we need to change length, right? MetaStore DB schema inconsistent for MS SQL Server in use of varchar/nvarchar - Key: HIVE-9208 URL: https://issues.apache.org/jira/browse/HIVE-9208 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.14.0 Reporter: Eugene Koifman Assignee: Xiaobing Zhou Attachments: HIVE-9208.1.patch, HIVE-9208.2.patch hive-schema-0.15.0.mssql.sql has PARTITIONS.PART_NAME as NVARCHAR but COMPLETED_TXN_COMPONENTS.CTC_PARTITON, COMPACTION_QUEUE.CQ_PARTITION, HIVE_LOCKS.HL_PARTITION, TXN_COMPONENTS.TC_PARTITION all use VARCHAR. This cannot be right since they all store the same value. the same is true of hive-schema-0.14.0.mssql.sql and the two corresponding hvie-txn-schema-... files. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9208) MetaStore DB schema inconsistent for MS SQL Server in use of varchar/nvarchar
[ https://issues.apache.org/jira/browse/HIVE-9208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14288275#comment-14288275 ] Xiaobing Zhou commented on HIVE-9208: - They are partition name with multi-bytes support. MetaStore DB schema inconsistent for MS SQL Server in use of varchar/nvarchar - Key: HIVE-9208 URL: https://issues.apache.org/jira/browse/HIVE-9208 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.14.0 Reporter: Eugene Koifman Assignee: Xiaobing Zhou Attachments: HIVE-9208.1.patch, HIVE-9208.2.patch hive-schema-0.15.0.mssql.sql has PARTITIONS.PART_NAME as NVARCHAR but COMPLETED_TXN_COMPONENTS.CTC_PARTITON, COMPACTION_QUEUE.CQ_PARTITION, HIVE_LOCKS.HL_PARTITION, TXN_COMPONENTS.TC_PARTITION all use VARCHAR. This cannot be right since they all store the same value. the same is true of hive-schema-0.14.0.mssql.sql and the two corresponding hvie-txn-schema-... files. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9208) MetaStore DB schema inconsistent for MS SQL Server in use of varchar/nvarchar
[ https://issues.apache.org/jira/browse/HIVE-9208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9208: Attachment: HIVE-9208.2.patch Made 2nd patch that keeps partition name and SDS location type consistent. [~ekoifman] can you review it? Thanks! MetaStore DB schema inconsistent for MS SQL Server in use of varchar/nvarchar - Key: HIVE-9208 URL: https://issues.apache.org/jira/browse/HIVE-9208 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.14.0 Reporter: Eugene Koifman Assignee: Xiaobing Zhou Attachments: HIVE-9208.1.patch, HIVE-9208.2.patch hive-schema-0.15.0.mssql.sql has PARTITIONS.PART_NAME as NVARCHAR but COMPLETED_TXN_COMPONENTS.CTC_PARTITON, COMPACTION_QUEUE.CQ_PARTITION, HIVE_LOCKS.HL_PARTITION, TXN_COMPONENTS.TC_PARTITION all use VARCHAR. This cannot be right since they all store the same value. the same is true of hive-schema-0.14.0.mssql.sql and the two corresponding hvie-txn-schema-... files. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9208) MetaStore DB schema inconsistent for MS SQL Server in use of varchar/nvarchar
[ https://issues.apache.org/jira/browse/HIVE-9208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9208: Attachment: HIVE-9208.1.patch Made a patch. [~ekoifman] can you have it reviewed, thanks! MetaStore DB schema inconsistent for MS SQL Server in use of varchar/nvarchar - Key: HIVE-9208 URL: https://issues.apache.org/jira/browse/HIVE-9208 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.14.0 Reporter: Eugene Koifman Assignee: Xiaobing Zhou Attachments: HIVE-9208.1.patch hive-schema-0.15.0.mssql.sql has PARTITIONS.PART_NAME as NVARCHAR but COMPLETED_TXN_COMPONENTS.CTC_PARTITON, COMPACTION_QUEUE.CQ_PARTITION, HIVE_LOCKS.HL_PARTITION, TXN_COMPONENTS.TC_PARTITION all use VARCHAR. This cannot be right since they all store the same value. the same is true of hive-schema-0.14.0.mssql.sql and the two corresponding hvie-txn-schema-... files. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9208) MetaStore DB schema inconsistent for MS SQL Server in use of varchar/nvarchar
[ https://issues.apache.org/jira/browse/HIVE-9208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276150#comment-14276150 ] Xiaobing Zhou commented on HIVE-9208: - Thanks [~ekoifman]! I'd take a look at them to get all partition name replications. MetaStore DB schema inconsistent for MS SQL Server in use of varchar/nvarchar - Key: HIVE-9208 URL: https://issues.apache.org/jira/browse/HIVE-9208 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.14.0 Reporter: Eugene Koifman Assignee: Xiaobing Zhou Attachments: HIVE-9208.1.patch hive-schema-0.15.0.mssql.sql has PARTITIONS.PART_NAME as NVARCHAR but COMPLETED_TXN_COMPONENTS.CTC_PARTITON, COMPACTION_QUEUE.CQ_PARTITION, HIVE_LOCKS.HL_PARTITION, TXN_COMPONENTS.TC_PARTITION all use VARCHAR. This cannot be right since they all store the same value. the same is true of hive-schema-0.14.0.mssql.sql and the two corresponding hvie-txn-schema-... files. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HIVE-8812) TestMinimrCliDriver failure if run in the same command as TestHBaseNegativeCliDriver
[
https://issues.apache.org/jira/browse/HIVE-8812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaobing Zhou reassigned HIVE-8812:
---
Assignee: Xiaobing Zhou (was: Jason Dere)
TestMinimrCliDriver failure if run in the same command as
TestHBaseNegativeCliDriver
Key: HIVE-8812
URL: https://issues.apache.org/jira/browse/HIVE-8812
Project: Hive
Issue Type: Bug
Components: Testing Infrastructure
Reporter: Jason Dere
Assignee: Xiaobing Zhou
Attachments: HIVE-8812.1.patch
TestHBaseNegativeCliDriver does not drop all of the tables created during the
test. The next CliDriver test in line (TestMinimrCliDriver) tries to clean up
the tables created by the test HBase instance and fails:
{format}
junit.framework.AssertionFailedError: Unexpected exception in setup
at junit.framework.Assert.fail(Assert.java:57)
at junit.framework.TestCase.fail(TestCase.java:227)
at
org.apache.hadoop.hive.cli.TestMinimrCliDriver.setUp(TestMinimrCliDriver.java:71)
{noformat}
This only happens if you run these tests in the same maven command.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8340) HiveServer2 service doesn't stop backend jvm process, which prevents follow-up service start.
[ https://issues.apache.org/jira/browse/HIVE-8340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14207223#comment-14207223 ] Xiaobing Zhou commented on HIVE-8340: - [~szehon] and [~leftylev], I guess, this should be documented. Users are allowed to set that parameter in hiveserver2.xml. HiveServer2 service doesn't stop backend jvm process, which prevents follow-up service start. - Key: HIVE-8340 URL: https://issues.apache.org/jira/browse/HIVE-8340 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.14.0 Environment: Windows Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Priority: Critical Labels: TODOC14 Fix For: 0.14.0 Attachments: HIVE-8340.1.patch, HIVE-8340.2.patch, HIVE-8340.3.patch, HIVE-8340.4.patch On stopping the HS2 service from the services tab, it only kills the root process and does not kill the child java process. As a result resources are not freed and this throws an error on restarting from command line. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8812) TestMinimrCliDriver failure if run in the same command as TestHBaseNegativeCliDriver
[
https://issues.apache.org/jira/browse/HIVE-8812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14207242#comment-14207242
]
Xiaobing Zhou commented on HIVE-8812:
-
+1
TestMinimrCliDriver failure if run in the same command as
TestHBaseNegativeCliDriver
Key: HIVE-8812
URL: https://issues.apache.org/jira/browse/HIVE-8812
Project: Hive
Issue Type: Bug
Components: Testing Infrastructure
Reporter: Jason Dere
Assignee: Jason Dere
Attachments: HIVE-8812.1.patch
TestHBaseNegativeCliDriver does not drop all of the tables created during the
test. The next CliDriver test in line (TestMinimrCliDriver) tries to clean up
the tables created by the test HBase instance and fails:
{format}
junit.framework.AssertionFailedError: Unexpected exception in setup
at junit.framework.Assert.fail(Assert.java:57)
at junit.framework.TestCase.fail(TestCase.java:227)
at
org.apache.hadoop.hive.cli.TestMinimrCliDriver.setUp(TestMinimrCliDriver.java:71)
{noformat}
This only happens if you run these tests in the same maven command.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8550) Hive cannot load data into partitioned table with Unicode key
[ https://issues.apache.org/jira/browse/HIVE-8550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14198880#comment-14198880 ] Xiaobing Zhou commented on HIVE-8550: - Non-Windows installations already supported unicode keys. Yes, this is for Windows. Thanks. Hive cannot load data into partitioned table with Unicode key - Key: HIVE-8550 URL: https://issues.apache.org/jira/browse/HIVE-8550 Project: Hive Issue Type: Bug Affects Versions: 0.14.0 Environment: Windows Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Priority: Critical Fix For: 0.14.0 Attachments: CreatePartitionedTable.hql, HIVE-8550.0.14.1.patch, HIVE-8550.1.patch, HIVE-8550.2.patch, HIVE-8550.3.patch, LoadIntoPartitionedTable.hql, partitioned.txt Steps to reproduce: 1) Copy the file partitioned.txt to the root folder of your HDFS root dir. Copy the two hql files to your local directory. 2) Open Hive CLI. 3) Run: hive source path to CreatePartitionedTable.hql; 4) Run hive source path to LoadIntoPartitionedTable.hql; The following error will be shown: hive source C:\Scripts\partition\LoadIntoPartitionedTable.hql; Loading data to table default.mypartitioned partition (tag=䶵) Failed with exception null FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8737) setEnv is not portable, which fails TestCliDriverMethods#testprocessInitFiles on Windows
[
https://issues.apache.org/jira/browse/HIVE-8737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaobing Zhou updated HIVE-8737:
Description:
repro:
{noformat}
mvn test -Phadoop-2 -Dtest=TestCliDriverMethods#testprocessInitFiles
{noformat}
setEnv tries to do JVM wide system variables changes, previous approach is not
portable.
was:
repro:
{noformat}
mvn test -Phadoop-2 -Dmaven.surefire.debug
-Dtest=TestCliDriverMethods#testprocessInitFiles
{noformat}
setEnv tries to do JVM wide system variables changes, previous approach is not
portable.
setEnv is not portable, which fails TestCliDriverMethods#testprocessInitFiles
on Windows
Key: HIVE-8737
URL: https://issues.apache.org/jira/browse/HIVE-8737
Project: Hive
Issue Type: Bug
Affects Versions: 0.14.0
Environment: Windows
Reporter: Xiaobing Zhou
Assignee: Xiaobing Zhou
Attachments: HIVE-8737.1.patch, HIVE-8737.2.patch
repro:
{noformat}
mvn test -Phadoop-2 -Dtest=TestCliDriverMethods#testprocessInitFiles
{noformat}
setEnv tries to do JVM wide system variables changes, previous approach is
not portable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-8737) setEnv is not portable, which fails TestCliDriverMethods#testprocessInitFiles on Windows
[
https://issues.apache.org/jira/browse/HIVE-8737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaobing Zhou updated HIVE-8737:
Attachment: HIVE-8737.3.patch
V3 patch to deal with the exception above. Forgot to recompile on Linux.
setEnv is not portable, which fails TestCliDriverMethods#testprocessInitFiles
on Windows
Key: HIVE-8737
URL: https://issues.apache.org/jira/browse/HIVE-8737
Project: Hive
Issue Type: Bug
Affects Versions: 0.14.0
Environment: Windows
Reporter: Xiaobing Zhou
Assignee: Xiaobing Zhou
Attachments: HIVE-8737.1.patch, HIVE-8737.2.patch, HIVE-8737.3.patch
repro:
{noformat}
mvn test -Phadoop-2 -Dtest=TestCliDriverMethods#testprocessInitFiles
{noformat}
setEnv tries to do JVM wide system variables changes, previous approach is
not portable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-8737) setEnv is not portable, which fails TestCliDriverMethods#testprocessInitFiles on Windows
[
https://issues.apache.org/jira/browse/HIVE-8737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaobing Zhou updated HIVE-8737:
Status: Patch Available (was: Open)
setEnv is not portable, which fails TestCliDriverMethods#testprocessInitFiles
on Windows
Key: HIVE-8737
URL: https://issues.apache.org/jira/browse/HIVE-8737
Project: Hive
Issue Type: Bug
Affects Versions: 0.14.0
Environment: Windows
Reporter: Xiaobing Zhou
Assignee: Xiaobing Zhou
Attachments: HIVE-8737.1.patch, HIVE-8737.2.patch, HIVE-8737.3.patch
repro:
{noformat}
mvn test -Phadoop-2 -Dtest=TestCliDriverMethods#testprocessInitFiles
{noformat}
setEnv tries to do JVM wide system variables changes, previous approach is
not portable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8703) More Windows unit test fixes
[ https://issues.apache.org/jira/browse/HIVE-8703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14196489#comment-14196489 ] Xiaobing Zhou commented on HIVE-8703: - Yes, test only changes. More Windows unit test fixes Key: HIVE-8703 URL: https://issues.apache.org/jira/browse/HIVE-8703 Project: Hive Issue Type: Bug Components: Tests, Windows Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-8703.1.patch, HIVE-8703.2.patch - TestStorageBasedMetastoreAuthorizationReads - needs to call WindowsPathUtil.convertPathsFromWindowsToHdfs() - TestAuthorizationApiAuthorizer - created role should have a name. This was causing TestLocationQueries to fail when run together because TestLocationQueries was unable to drop a role with a null name. This one fails on Unix as well. - create_like.q, stats_noscan_2.q: system:hive.root wasn't working on Windows, change test to use system:test.tmp.dir. - Also update the golden files for a few Windows-only .q file tests -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-8737) setEnv is not portable, which fails TestCliDriverMethods#testprocessInitFiles on Windows
Xiaobing Zhou created HIVE-8737: --- Summary: setEnv is not portable, which fails TestCliDriverMethods#testprocessInitFiles on Windows Key: HIVE-8737 URL: https://issues.apache.org/jira/browse/HIVE-8737 Project: Hive Issue Type: Bug Affects Versions: 0.14.0 Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8737) setEnv is not portable, which fails TestCliDriverMethods#testprocessInitFiles on Windows
[ https://issues.apache.org/jira/browse/HIVE-8737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-8737: Environment: Windows setEnv is not portable, which fails TestCliDriverMethods#testprocessInitFiles on Windows Key: HIVE-8737 URL: https://issues.apache.org/jira/browse/HIVE-8737 Project: Hive Issue Type: Bug Affects Versions: 0.14.0 Environment: Windows Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8737) setEnv is not portable, which fails TestCliDriverMethods#testprocessInitFiles on Windows
[
https://issues.apache.org/jira/browse/HIVE-8737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaobing Zhou updated HIVE-8737:
Description:
repro:
{noformat}
mvn test -Phadoop-2 -Dmaven.surefire.debug
-Dtest=TestCliDriverMethods#testprocessInitFiles
{noformat}
setEnv tries to do JVM wide system variables changes, previous approach is not
portable.
was:
repro:
{noformat}
mvn test -Phadoop-2 -Dmaven.surefire.debug
-Dtest=TestCliDriverMethods#testprocessInitFiles
{noformat}
setEnv tries to do JVM wide system variables change, previous approach is not
portable.
setEnv is not portable, which fails TestCliDriverMethods#testprocessInitFiles
on Windows
Key: HIVE-8737
URL: https://issues.apache.org/jira/browse/HIVE-8737
Project: Hive
Issue Type: Bug
Affects Versions: 0.14.0
Environment: Windows
Reporter: Xiaobing Zhou
Assignee: Xiaobing Zhou
repro:
{noformat}
mvn test -Phadoop-2 -Dmaven.surefire.debug
-Dtest=TestCliDriverMethods#testprocessInitFiles
{noformat}
setEnv tries to do JVM wide system variables changes, previous approach is
not portable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-8737) setEnv is not portable, which fails TestCliDriverMethods#testprocessInitFiles on Windows
[
https://issues.apache.org/jira/browse/HIVE-8737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaobing Zhou updated HIVE-8737:
Description:
repro:
{noformat}
mvn test -Phadoop-2 -Dmaven.surefire.debug
-Dtest=TestCliDriverMethods#testprocessInitFiles
{noformat}
setEnv tries to do JVM wide system variables change, previous approach is not
portable.
setEnv is not portable, which fails TestCliDriverMethods#testprocessInitFiles
on Windows
Key: HIVE-8737
URL: https://issues.apache.org/jira/browse/HIVE-8737
Project: Hive
Issue Type: Bug
Affects Versions: 0.14.0
Environment: Windows
Reporter: Xiaobing Zhou
Assignee: Xiaobing Zhou
repro:
{noformat}
mvn test -Phadoop-2 -Dmaven.surefire.debug
-Dtest=TestCliDriverMethods#testprocessInitFiles
{noformat}
setEnv tries to do JVM wide system variables change, previous approach is not
portable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-8737) setEnv is not portable, which fails TestCliDriverMethods#testprocessInitFiles on Windows
[
https://issues.apache.org/jira/browse/HIVE-8737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaobing Zhou updated HIVE-8737:
Attachment: HIVE-8737.1.patch
Made a patch. [~jdere] can you review it? Thanks!
setEnv is not portable, which fails TestCliDriverMethods#testprocessInitFiles
on Windows
Key: HIVE-8737
URL: https://issues.apache.org/jira/browse/HIVE-8737
Project: Hive
Issue Type: Bug
Affects Versions: 0.14.0
Environment: Windows
Reporter: Xiaobing Zhou
Assignee: Xiaobing Zhou
Attachments: HIVE-8737.1.patch
repro:
{noformat}
mvn test -Phadoop-2 -Dmaven.surefire.debug
-Dtest=TestCliDriverMethods#testprocessInitFiles
{noformat}
setEnv tries to do JVM wide system variables changes, previous approach is
not portable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-8737) setEnv is not portable, which fails TestCliDriverMethods#testprocessInitFiles on Windows
[
https://issues.apache.org/jira/browse/HIVE-8737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaobing Zhou updated HIVE-8737:
Attachment: HIVE-8737.2.patch
setEnv is not portable, which fails TestCliDriverMethods#testprocessInitFiles
on Windows
Key: HIVE-8737
URL: https://issues.apache.org/jira/browse/HIVE-8737
Project: Hive
Issue Type: Bug
Affects Versions: 0.14.0
Environment: Windows
Reporter: Xiaobing Zhou
Assignee: Xiaobing Zhou
Attachments: HIVE-8737.1.patch, HIVE-8737.2.patch
repro:
{noformat}
mvn test -Phadoop-2 -Dmaven.surefire.debug
-Dtest=TestCliDriverMethods#testprocessInitFiles
{noformat}
setEnv tries to do JVM wide system variables changes, previous approach is
not portable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8737) setEnv is not portable, which fails TestCliDriverMethods#testprocessInitFiles on Windows
[
https://issues.apache.org/jira/browse/HIVE-8737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14197272#comment-14197272
]
Xiaobing Zhou commented on HIVE-8737:
-
Made v2 and did code format adjustment.
setEnv is not portable, which fails TestCliDriverMethods#testprocessInitFiles
on Windows
Key: HIVE-8737
URL: https://issues.apache.org/jira/browse/HIVE-8737
Project: Hive
Issue Type: Bug
Affects Versions: 0.14.0
Environment: Windows
Reporter: Xiaobing Zhou
Assignee: Xiaobing Zhou
Attachments: HIVE-8737.1.patch, HIVE-8737.2.patch
repro:
{noformat}
mvn test -Phadoop-2 -Dmaven.surefire.debug
-Dtest=TestCliDriverMethods#testprocessInitFiles
{noformat}
setEnv tries to do JVM wide system variables changes, previous approach is
not portable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8550) Hive cannot load data into partitioned table with Unicode key
[ https://issues.apache.org/jira/browse/HIVE-8550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14194796#comment-14194796 ] Xiaobing Zhou commented on HIVE-8550: - [~leftylev] I think we can claim unicode partitioned table is supported from 0.14 and above, including fresh install and upgrade from 0.13 to 0.14. Thanks. Hive cannot load data into partitioned table with Unicode key - Key: HIVE-8550 URL: https://issues.apache.org/jira/browse/HIVE-8550 Project: Hive Issue Type: Bug Affects Versions: 0.14.0 Environment: Windows Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Priority: Critical Fix For: 0.14.0 Attachments: CreatePartitionedTable.hql, HIVE-8550.0.14.1.patch, HIVE-8550.1.patch, HIVE-8550.2.patch, HIVE-8550.3.patch, LoadIntoPartitionedTable.hql, partitioned.txt Steps to reproduce: 1) Copy the file partitioned.txt to the root folder of your HDFS root dir. Copy the two hql files to your local directory. 2) Open Hive CLI. 3) Run: hive source path to CreatePartitionedTable.hql; 4) Run hive source path to LoadIntoPartitionedTable.hql; The following error will be shown: hive source C:\Scripts\partition\LoadIntoPartitionedTable.hql; Loading data to table default.mypartitioned partition (tag=䶵) Failed with exception null FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8584) Setting hive.exec.orc.default.compress to ZLIB will lead to orc file size delta byte(s) shorter on Windows than Linux
[ https://issues.apache.org/jira/browse/HIVE-8584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14194799#comment-14194799 ] Xiaobing Zhou commented on HIVE-8584: - Thanks [~leftylev], no docs need to be addressed. Setting hive.exec.orc.default.compress to ZLIB will lead to orc file size delta byte(s) shorter on Windows than Linux - Key: HIVE-8584 URL: https://issues.apache.org/jira/browse/HIVE-8584 Project: Hive Issue Type: Bug Affects Versions: 0.14.0 Environment: Windows Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Priority: Minor Attachments: HIVE-8584.1.patch, orc-win-none-1.dump, orc-win-none-2.dump, orc-win-snappy-1.dump, orc-win-snappy-2.dump, orc-win-zlib-1.dump, orc-win-zlib-2.dump, orc_analyze.q repo steps: 1. run query orc_analyze.q 2. hive --orcfiledump target_orc_file_generated run 1 and 2 on PST timezone on Linux, and one more time on other timezone e.g. CST on Windows. Compare two target orc file dumping. Windows orc file is 1 byte shorter than Linux one. That's the case even if running 1 and 2 on Windows for different timezones, however, no problem on Linux. The issue only exists by using ZLIB mode, eventually OS native compression lib is used. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8715) Hive 14 upgrade scripts can fail for statistics if database was created using auto-create
[ https://issues.apache.org/jira/browse/HIVE-8715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14195331#comment-14195331 ] Xiaobing Zhou commented on HIVE-8715: - +1. Just tested mssql script, it works fine. Actually ''identifier_name'' or [identifier_name] both are fine. They deals with those id with empty spaces in the middle, e.g. ''identifier name'' or [identifier name]. Hive 14 upgrade scripts can fail for statistics if database was created using auto-create - Key: HIVE-8715 URL: https://issues.apache.org/jira/browse/HIVE-8715 Project: Hive Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Priority: Critical Fix For: 0.14.0 Attachments: HIVE-8715.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-8717) Space character is not ascii encoded in golden file, which fails two unit tests.
Xiaobing Zhou created HIVE-8717: --- Summary: Space character is not ascii encoded in golden file, which fails two unit tests. Key: HIVE-8717 URL: https://issues.apache.org/jira/browse/HIVE-8717 Project: Hive Issue Type: Bug Affects Versions: 0.14.0 Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8717) Space character is not ascii encoded in golden file, which fails several unit tests.
[ https://issues.apache.org/jira/browse/HIVE-8717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-8717: Summary: Space character is not ascii encoded in golden file, which fails several unit tests. (was: Space character is not ascii encoded in golden file, which fails two unit tests.) Space character is not ascii encoded in golden file, which fails several unit tests. Key: HIVE-8717 URL: https://issues.apache.org/jira/browse/HIVE-8717 Project: Hive Issue Type: Bug Affects Versions: 0.14.0 Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8717) Space character is not ascii encoded in golden file, which fails several unit tests.
[ https://issues.apache.org/jira/browse/HIVE-8717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-8717: Environment: Windows Space character is not ascii encoded in golden file, which fails several unit tests. Key: HIVE-8717 URL: https://issues.apache.org/jira/browse/HIVE-8717 Project: Hive Issue Type: Bug Affects Versions: 0.14.0 Environment: Windows Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8717) PreExecutePrinter.printEntities behaves differently on Windows vs. Linux, which fails two UTs
[ https://issues.apache.org/jira/browse/HIVE-8717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-8717: Summary: PreExecutePrinter.printEntities behaves differently on Windows vs. Linux, which fails two UTs (was: Space character is not ascii encoded in golden file, which fails several unit tests.) PreExecutePrinter.printEntities behaves differently on Windows vs. Linux, which fails two UTs - Key: HIVE-8717 URL: https://issues.apache.org/jira/browse/HIVE-8717 Project: Hive Issue Type: Bug Affects Versions: 0.14.0 Environment: Windows Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8717) PreExecutePrinter.printEntities behaves differently on Windows vs. Linux, which fails two UTs
[
https://issues.apache.org/jira/browse/HIVE-8717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaobing Zhou updated HIVE-8717:
Description:
repro:
{noformat}
mvn test -Phadoop-2
-Dtest=TestMiniTezCliDriver#testCliDriver_vector_partitioned_date_time
mvn test -Phadoop-2
-Dtest=TestCliDriver#testCliDriver_vector_partitioned_date_time
{noformat}
{noformat}
POSTHOOK: Output:
default@flights_tiny_orc_partitioned_timestamp@fl_time=2010-10-20 07%3A00%3A00
{noformat}
is the entry in golden file, however, .q.out on Windows outputs
{noformat}
POSTHOOK: Output:
default@flights_tiny_orc_partitioned_timestamp@fl_time=2010-10-20%2007%3A00%3A00
{noformat}. There's %20 added, which is ascii encoding of space char. These
diff(s) fail the UT(s)
PreExecutePrinter.printEntities behaves differently on Windows vs. Linux,
which fails two UTs
-
Key: HIVE-8717
URL: https://issues.apache.org/jira/browse/HIVE-8717
Project: Hive
Issue Type: Bug
Affects Versions: 0.14.0
Environment: Windows
Reporter: Xiaobing Zhou
Assignee: Xiaobing Zhou
repro:
{noformat}
mvn test -Phadoop-2
-Dtest=TestMiniTezCliDriver#testCliDriver_vector_partitioned_date_time
mvn test -Phadoop-2
-Dtest=TestCliDriver#testCliDriver_vector_partitioned_date_time
{noformat}
{noformat}
POSTHOOK: Output:
default@flights_tiny_orc_partitioned_timestamp@fl_time=2010-10-20 07%3A00%3A00
{noformat}
is the entry in golden file, however, .q.out on Windows outputs
{noformat}
POSTHOOK: Output:
default@flights_tiny_orc_partitioned_timestamp@fl_time=2010-10-20%2007%3A00%3A00
{noformat}. There's %20 added, which is ascii encoding of space char. These
diff(s) fail the UT(s)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8703) More Windows unit test fixes
[ https://issues.apache.org/jira/browse/HIVE-8703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14195414#comment-14195414 ] Xiaobing Zhou commented on HIVE-8703: - +1, [~hagleitn] can you get this into trunk as well as 0.14? Thanks! More Windows unit test fixes Key: HIVE-8703 URL: https://issues.apache.org/jira/browse/HIVE-8703 Project: Hive Issue Type: Bug Components: Tests, Windows Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-8703.1.patch, HIVE-8703.2.patch - TestStorageBasedMetastoreAuthorizationReads - needs to call WindowsPathUtil.convertPathsFromWindowsToHdfs() - TestAuthorizationApiAuthorizer - created role should have a name. This was causing TestLocationQueries to fail when run together because TestLocationQueries was unable to drop a role with a null name. This one fails on Unix as well. - create_like.q, stats_noscan_2.q: system:hive.root wasn't working on Windows, change test to use system:test.tmp.dir. - Also update the golden files for a few Windows-only .q file tests -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-8695) TestJdbcWithMiniKdc.testNegativeTokenAuth fails on non-expected error messages
Xiaobing Zhou created HIVE-8695: --- Summary: TestJdbcWithMiniKdc.testNegativeTokenAuth fails on non-expected error messages Key: HIVE-8695 URL: https://issues.apache.org/jira/browse/HIVE-8695 Project: Hive Issue Type: Bug Affects Versions: 0.14.0 Reporter: Xiaobing Zhou -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8695) TestJdbcWithMiniKdc.testNegativeTokenAuth fails on non-expected error messages
[
https://issues.apache.org/jira/browse/HIVE-8695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaobing Zhou updated HIVE-8695:
Description:
repo steps:
{noformat}
run mvn test -Phadoop-2 -Dtest=TestJdbcWithMiniKdc#testNegativeTokenAuth
{noformat}
, it fails since '*Failed to validate proxy privilege*' is expected error
message and cause message, however, '*Error retrieving delegation token for
user*' and '*is not allowed to impersonate*' are the returned exception.
TestJdbcWithMiniKdc.testNegativeTokenAuth fails on non-expected error messages
--
Key: HIVE-8695
URL: https://issues.apache.org/jira/browse/HIVE-8695
Project: Hive
Issue Type: Bug
Affects Versions: 0.14.0
Reporter: Xiaobing Zhou
repo steps:
{noformat}
run mvn test -Phadoop-2 -Dtest=TestJdbcWithMiniKdc#testNegativeTokenAuth
{noformat}
, it fails since '*Failed to validate proxy privilege*' is expected error
message and cause message, however, '*Error retrieving delegation token for
user*' and '*is not allowed to impersonate*' are the returned exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-8695) TestJdbcWithMiniKdc.testNegativeTokenAuth fails on non-expected error messages
[
https://issues.apache.org/jira/browse/HIVE-8695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaobing Zhou updated HIVE-8695:
Attachment: HIVE-8695.1.patch
After check, this is a result of HIVE-8557. Made a patch. Can anyone please
review it? Thanks!
TestJdbcWithMiniKdc.testNegativeTokenAuth fails on non-expected error messages
--
Key: HIVE-8695
URL: https://issues.apache.org/jira/browse/HIVE-8695
Project: Hive
Issue Type: Bug
Affects Versions: 0.14.0
Reporter: Xiaobing Zhou
Attachments: HIVE-8695.1.patch
repo steps:
{noformat}
run mvn test -Phadoop-2 -Dtest=TestJdbcWithMiniKdc#testNegativeTokenAuth
{noformat}
, it fails since '*Failed to validate proxy privilege*' is expected error
message and cause message, however, '*Error retrieving delegation token for
user*' and '*is not allowed to impersonate*' are the returned exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8695) TestJdbcWithMiniKdc.testNegativeTokenAuth fails on non-expected error messages
[
https://issues.apache.org/jira/browse/HIVE-8695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14193572#comment-14193572
]
Xiaobing Zhou commented on HIVE-8695:
-
[~thejas] is it safe to do this change in this patch, since you were working on
HIVE-8557? Thanks!
TestJdbcWithMiniKdc.testNegativeTokenAuth fails on non-expected error messages
--
Key: HIVE-8695
URL: https://issues.apache.org/jira/browse/HIVE-8695
Project: Hive
Issue Type: Bug
Affects Versions: 0.14.0
Reporter: Xiaobing Zhou
Attachments: HIVE-8695.1.patch
repo steps:
{noformat}
run mvn test -Phadoop-2 -Dtest=TestJdbcWithMiniKdc#testNegativeTokenAuth
{noformat}
, it fails since '*Failed to validate proxy privilege*' is expected error
message and cause message, however, '*Error retrieving delegation token for
user*' and '*is not allowed to impersonate*' are the returned exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HIVE-8584) Setting hive.exec.orc.default.compress to ZLIB will lead to orc file size delta byte(s) shorter on Windows than Linux
[ https://issues.apache.org/jira/browse/HIVE-8584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou resolved HIVE-8584. - Resolution: Invalid Setting hive.exec.orc.default.compress to ZLIB will lead to orc file size delta byte(s) shorter on Windows than Linux - Key: HIVE-8584 URL: https://issues.apache.org/jira/browse/HIVE-8584 Project: Hive Issue Type: Bug Affects Versions: 0.14.0 Environment: Windows Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Priority: Minor Attachments: HIVE-8584.1.patch, orc-win-none-1.dump, orc-win-none-2.dump, orc-win-snappy-1.dump, orc-win-snappy-2.dump, orc-win-zlib-1.dump, orc-win-zlib-2.dump, orc_analyze.q repo steps: 1. run query orc_analyze.q 2. hive --orcfiledump target_orc_file_generated run 1 and 2 on PST timezone on Linux, and one more time on other timezone e.g. CST on Windows. Compare two target orc file dumping. Windows orc file is 1 byte shorter than Linux one. That's the case even if running 1 and 2 on Windows for different timezones, however, no problem on Linux. The issue only exists by using ZLIB mode, eventually OS native compression lib is used. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8584) Setting hive.exec.orc.default.compress to ZLIB will lead to orc file size delta byte(s) shorter on Windows than Linux
[ https://issues.apache.org/jira/browse/HIVE-8584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14193581#comment-14193581 ] Xiaobing Zhou commented on HIVE-8584: - Thanks all for comments. After deep investigation, ZLIB mode actually works fine for both platform if qtest output is exact same on both. There are other reasons led to output diff, which will be tracked by other JIRA. I'd mark this as invalid. Setting hive.exec.orc.default.compress to ZLIB will lead to orc file size delta byte(s) shorter on Windows than Linux - Key: HIVE-8584 URL: https://issues.apache.org/jira/browse/HIVE-8584 Project: Hive Issue Type: Bug Affects Versions: 0.14.0 Environment: Windows Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Priority: Minor Attachments: HIVE-8584.1.patch, orc-win-none-1.dump, orc-win-none-2.dump, orc-win-snappy-1.dump, orc-win-snappy-2.dump, orc-win-zlib-1.dump, orc-win-zlib-2.dump, orc_analyze.q repo steps: 1. run query orc_analyze.q 2. hive --orcfiledump target_orc_file_generated run 1 and 2 on PST timezone on Linux, and one more time on other timezone e.g. CST on Windows. Compare two target orc file dumping. Windows orc file is 1 byte shorter than Linux one. That's the case even if running 1 and 2 on Windows for different timezones, however, no problem on Linux. The issue only exists by using ZLIB mode, eventually OS native compression lib is used. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HIVE-7276) BaseSemanticAnalyzer.unescapeSQLString fails to parse Windows like path
[ https://issues.apache.org/jira/browse/HIVE-7276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou resolved HIVE-7276. - Resolution: Cannot Reproduce Resolved it since it's not reproducible any more. BaseSemanticAnalyzer.unescapeSQLString fails to parse Windows like path --- Key: HIVE-7276 URL: https://issues.apache.org/jira/browse/HIVE-7276 Project: Hive Issue Type: Bug Components: Query Processor, Windows Affects Versions: 0.13.0 Environment: Windows Server 2008 R2 Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Priority: Critical BaseSemanticAnalyzer.unescapeSQLString fails to parse windows-like path, e.g. C:\Users\xzhou\hworks. This will cause a large quantity of queries on windows to fail. For example, 'C:\Users\xzhou\hworks\workspace\hwx-hive-ws\hive\hcatalog\core\target\tmp\hive-junit-960740885870900' will be parsed as 'C:Usersxzhouhworksworkspacehwx-hive-wshivehcatalogcore arget mphive-junit-960740885870900', since \ is interpreted as start char in unicode string, e.g. \002 for delimiter, and thus swallowed. \0, \b, \n, \r, \t, \Z, and so on within normal Windows like path will also be swallowed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-7511) Hive: output is incorrect if there are UTF-8 characters in where clause of a hive select query.
[ https://issues.apache.org/jira/browse/HIVE-7511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14193589#comment-14193589 ] Xiaobing Zhou commented on HIVE-7511: - This can be resolved by applying java options, like -Dfile.encoding=UTF-8. Setting it as env variable(_JAVA_OPTIONS=-Dfile.encoding=UTF-8) or passing as java start argument both work fine. Hive: output is incorrect if there are UTF-8 characters in where clause of a hive select query. --- Key: HIVE-7511 URL: https://issues.apache.org/jira/browse/HIVE-7511 Project: Hive Issue Type: Bug Affects Versions: 0.13.0 Environment: Windows Server 2008 R2 Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Priority: Critical Attachments: HIVE-7511.1.patch When we put UTF-8 characters in where clause of a hive query the results are empty for where content like '%丄%' and results contain all rows for where content not like '%丄%'; even when few rows contain this character. Steps to reproduce: 1. Save a file called data.txt in the root container. The contents of the files are as follows. 190 丄f齄啊c狛䶴h䶴c狝 899 d狜狜㐁geg阿狚ea䶴eead狜e 137 齄鼾h狝ge㐀狛g狚阿 21﨩﨩e㐀c狛鼾d䶴﨨 767 﨩c﨩g狜㐁狜狛齄阿﨩狚齄﨨䶵狝﨨 281 﨨㐀啊aga啊c狝e鼾鼾 573 㐁䶴hc﨨b狝㐁﨩䶴狜丄hc齄 966 䶴丄狜﨨e狝eb狜㐁c㐀鼾﨩丄ga狚丄 565 䶵㐀﨩㐀bb狛ehd丄ea丄㐀 778 﨩㐁阿﨨狚bbea丄䶵丄狚鼾狚a䶵 363 gd齄a鼾a䶴b㐁㐁fg鼾 822 a阿狜䶵h䶵e狛h﨩gac狜阿㐀啊b 338 b齄㐁ff阿e狜e㐀ba齄 2. Execute the following queries to setup the table. a. CREATE TABLE hivetable(row INT, content STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY ' t' LOCATION '/hivetable'; b. LOAD DATA INPATH 'wasb:///data.txt' OVERWRITE INTO TABLE hivetable; 3. create a query file query.hql with following contents INSERT OVERWRITE DIRECTORY 'wasb:///hiveoutput' select * from hivetable where content like '%丄%'; 4. even though few rows contains this character the output is empty. 5. change the contents of query.hql to INSERT OVERWRITE DIRECTORY 'wasb:///hiveoutput' select * from hivetable where content not like '%丄%'; 6. The output contains all rows including those containing the given character. 7. Similar results are observed when using where content = '丄f齄啊c狛䶴h䶴c狝'; 8. We get expected results when using where content like '%a%'; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8665) Fix misc unit tests on Windows
[ https://issues.apache.org/jira/browse/HIVE-8665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14192579#comment-14192579 ] Xiaobing Zhou commented on HIVE-8665: - [~thejas] can you get that committed to trunk and 0.14. We need this in Champlain. Thanks! Fix misc unit tests on Windows -- Key: HIVE-8665 URL: https://issues.apache.org/jira/browse/HIVE-8665 Project: Hive Issue Type: Bug Components: Windows Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-8665.1.patch Several junit tests failing on Windows for misc reasons (path issues, resources need to be closed before file can be deleted, etc). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8550) Hive cannot load data into partitioned table with Unicode key
[
https://issues.apache.org/jira/browse/HIVE-8550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14190433#comment-14190433
]
Xiaobing Zhou commented on HIVE-8550:
-
Yes [~jdere], tested both, see also review board. Pasted tests done here:
{noformat}
Legacy system upgrading:
1. stop all services
2. drop HIVE DB
3. manually create HIVE DB
4. enable datanucleus.autoCreateSchema
5. start all services
6. all tables are created by autoCreateSchema
7. run 004-HIVE-8550.mssql.sql to do upgrade
Fresh install:
1. stop all services
2. drop HIVE DB
3. manually create HIVE DB
4. run hive-schema-0.14.0.mssql.sql to create all tables
5. disable datanucleus.autoCreateSchema
6. start all services
It fixed the issue.
{noformat}
Hive cannot load data into partitioned table with Unicode key
-
Key: HIVE-8550
URL: https://issues.apache.org/jira/browse/HIVE-8550
Project: Hive
Issue Type: Bug
Affects Versions: 0.14.0
Environment: Windows
Reporter: Xiaobing Zhou
Assignee: Xiaobing Zhou
Priority: Critical
Attachments: CreatePartitionedTable.hql, HIVE-8550.1.patch,
HIVE-8550.2.patch, HIVE-8550.3.patch, LoadIntoPartitionedTable.hql,
partitioned.txt
Steps to reproduce:
1) Copy the file partitioned.txt to the root folder of your HDFS root dir.
Copy the two hql files to your local directory.
2) Open Hive CLI.
3) Run:
hive source path to CreatePartitionedTable.hql;
4) Run
hive source path to LoadIntoPartitionedTable.hql;
The following error will be shown:
hive source C:\Scripts\partition\LoadIntoPartitionedTable.hql;
Loading data to table default.mypartitioned partition (tag=䶵)
Failed with exception null
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.MoveTask
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8668) mssql sql script has carriage returns
[
https://issues.apache.org/jira/browse/HIVE-8668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14190720#comment-14190720
]
Xiaobing Zhou commented on HIVE-8668:
-
[~hagleitn] Sorry for latency, just saw this thread. Are there anythings I can
help?
mssql sql script has carriage returns
-
Key: HIVE-8668
URL: https://issues.apache.org/jira/browse/HIVE-8668
Project: Hive
Issue Type: Bug
Affects Versions: 0.14.0
Reporter: Brock Noland
Assignee: Brock Noland
Priority: Blocker
Fix For: 0.14.0
Attachments: HIVE-8668.patch, HIVE-8668.patch
This is breaking patches generated by {{svn merge}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8668) mssql sql script has carriage returns
[
https://issues.apache.org/jira/browse/HIVE-8668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14190729#comment-14190729
]
Xiaobing Zhou commented on HIVE-8668:
-
Actually I just executed queries in hive-txn-schema-0.14.0.mssql.sql on Windows
by text copy-paste as well as sqlcmd CLI. Both work well.
mssql sql script has carriage returns
-
Key: HIVE-8668
URL: https://issues.apache.org/jira/browse/HIVE-8668
Project: Hive
Issue Type: Bug
Affects Versions: 0.14.0
Reporter: Brock Noland
Assignee: Brock Noland
Priority: Blocker
Fix For: 0.14.0
Attachments: HIVE-8668.patch, HIVE-8668.patch
This is breaking patches generated by {{svn merge}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8550) Hive cannot load data into partitioned table with Unicode key
[ https://issues.apache.org/jira/browse/HIVE-8550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14190736#comment-14190736 ] Xiaobing Zhou commented on HIVE-8550: - Thanks [~hagleitn]. We can put a hold on 0.14 patch since [~jdere] proposed to do some verifications that make sure index on PART_NAME works well even after nvarchar change, although we think so. Hive cannot load data into partitioned table with Unicode key - Key: HIVE-8550 URL: https://issues.apache.org/jira/browse/HIVE-8550 Project: Hive Issue Type: Bug Affects Versions: 0.14.0 Environment: Windows Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Priority: Critical Attachments: CreatePartitionedTable.hql, HIVE-8550.1.patch, HIVE-8550.2.patch, HIVE-8550.3.patch, LoadIntoPartitionedTable.hql, partitioned.txt Steps to reproduce: 1) Copy the file partitioned.txt to the root folder of your HDFS root dir. Copy the two hql files to your local directory. 2) Open Hive CLI. 3) Run: hive source path to CreatePartitionedTable.hql; 4) Run hive source path to LoadIntoPartitionedTable.hql; The following error will be shown: hive source C:\Scripts\partition\LoadIntoPartitionedTable.hql; Loading data to table default.mypartitioned partition (tag=䶵) Failed with exception null FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8550) Hive cannot load data into partitioned table with Unicode key
[
https://issues.apache.org/jira/browse/HIVE-8550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14191083#comment-14191083
]
Xiaobing Zhou commented on HIVE-8550:
-
Confirmed two concerns from [~jdere]:
1. legacy system(varchar typed column) with non-unicode partition data in DB,
running nvarchar upgrade, patch works well for unicode partitioned table
creation and data loading.
2. run some samples queries on PARTITIONS table to make sure index is still
working after upgrade.
For case 1, smoothy upgrade is seen; For case 2, I ran query on MSSQL,
{noformat}
select * from dbo.PARTITIONS where part_name = 'ds=2008-04-08/hr=11' and TBL_ID
= 7;
{noformat}
, the execution plans include index seek on the [PARTITIONS].[UNIQUEPARTITION]
which is NonClustered index on part_name and TBL_ID. This is good certificate
that index is working well after upgrade.
So It's safe to commit it.
Hive cannot load data into partitioned table with Unicode key
-
Key: HIVE-8550
URL: https://issues.apache.org/jira/browse/HIVE-8550
Project: Hive
Issue Type: Bug
Affects Versions: 0.14.0
Environment: Windows
Reporter: Xiaobing Zhou
Assignee: Xiaobing Zhou
Priority: Critical
Attachments: CreatePartitionedTable.hql, HIVE-8550.1.patch,
HIVE-8550.2.patch, HIVE-8550.3.patch, LoadIntoPartitionedTable.hql,
partitioned.txt
Steps to reproduce:
1) Copy the file partitioned.txt to the root folder of your HDFS root dir.
Copy the two hql files to your local directory.
2) Open Hive CLI.
3) Run:
hive source path to CreatePartitionedTable.hql;
4) Run
hive source path to LoadIntoPartitionedTable.hql;
The following error will be shown:
hive source C:\Scripts\partition\LoadIntoPartitionedTable.hql;
Loading data to table default.mypartitioned partition (tag=䶵)
Failed with exception null
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.MoveTask
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-8550) Hive cannot load data into partitioned table with Unicode key
[ https://issues.apache.org/jira/browse/HIVE-8550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-8550: Attachment: HIVE-8550.0.14.1.patch Made a patch for 0.14. [~hagleitn] can you get that into 0.14, thanks! Hive cannot load data into partitioned table with Unicode key - Key: HIVE-8550 URL: https://issues.apache.org/jira/browse/HIVE-8550 Project: Hive Issue Type: Bug Affects Versions: 0.14.0 Environment: Windows Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Priority: Critical Attachments: CreatePartitionedTable.hql, HIVE-8550.0.14.1.patch, HIVE-8550.1.patch, HIVE-8550.2.patch, HIVE-8550.3.patch, LoadIntoPartitionedTable.hql, partitioned.txt Steps to reproduce: 1) Copy the file partitioned.txt to the root folder of your HDFS root dir. Copy the two hql files to your local directory. 2) Open Hive CLI. 3) Run: hive source path to CreatePartitionedTable.hql; 4) Run hive source path to LoadIntoPartitionedTable.hql; The following error will be shown: hive source C:\Scripts\partition\LoadIntoPartitionedTable.hql; Loading data to table default.mypartitioned partition (tag=䶵) Failed with exception null FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8550) Hive cannot load data into partitioned table with Unicode key
[ https://issues.apache.org/jira/browse/HIVE-8550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-8550: Attachment: HIVE-8550.1.patch Made a patch. Can anyone please review it? Thanks! Hive cannot load data into partitioned table with Unicode key - Key: HIVE-8550 URL: https://issues.apache.org/jira/browse/HIVE-8550 Project: Hive Issue Type: Bug Affects Versions: 0.14.0 Environment: Windows Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Priority: Critical Attachments: CreatePartitionedTable.hql, HIVE-8550.1.patch, LoadIntoPartitionedTable.hql, partitioned.txt Steps to reproduce: 1) Copy the file partitioned.txt to the root folder of your HDFS root dir. Copy the two hql files to your local directory. 2) Open Hive CLI. 3) Run: hive source path to CreatePartitionedTable.hql; 4) Run hive source path to LoadIntoPartitionedTable.hql; The following error will be shown: hive source C:\Scripts\partition\LoadIntoPartitionedTable.hql; Loading data to table default.mypartitioned partition (tag=䶵) Failed with exception null FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8550) Hive cannot load data into partitioned table with Unicode key
[ https://issues.apache.org/jira/browse/HIVE-8550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-8550: Attachment: HIVE-8550.2.patch Made the 2nd patch! Considered 0.14 fresh install. Hive cannot load data into partitioned table with Unicode key - Key: HIVE-8550 URL: https://issues.apache.org/jira/browse/HIVE-8550 Project: Hive Issue Type: Bug Affects Versions: 0.14.0 Environment: Windows Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Priority: Critical Attachments: CreatePartitionedTable.hql, HIVE-8550.1.patch, HIVE-8550.2.patch, LoadIntoPartitionedTable.hql, partitioned.txt Steps to reproduce: 1) Copy the file partitioned.txt to the root folder of your HDFS root dir. Copy the two hql files to your local directory. 2) Open Hive CLI. 3) Run: hive source path to CreatePartitionedTable.hql; 4) Run hive source path to LoadIntoPartitionedTable.hql; The following error will be shown: hive source C:\Scripts\partition\LoadIntoPartitionedTable.hql; Loading data to table default.mypartitioned partition (tag=䶵) Failed with exception null FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8550) Hive cannot load data into partitioned table with Unicode key
[ https://issues.apache.org/jira/browse/HIVE-8550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-8550: Status: Patch Available (was: In Progress) Hive cannot load data into partitioned table with Unicode key - Key: HIVE-8550 URL: https://issues.apache.org/jira/browse/HIVE-8550 Project: Hive Issue Type: Bug Affects Versions: 0.14.0 Environment: Windows Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Priority: Critical Attachments: CreatePartitionedTable.hql, HIVE-8550.1.patch, HIVE-8550.2.patch, LoadIntoPartitionedTable.hql, partitioned.txt Steps to reproduce: 1) Copy the file partitioned.txt to the root folder of your HDFS root dir. Copy the two hql files to your local directory. 2) Open Hive CLI. 3) Run: hive source path to CreatePartitionedTable.hql; 4) Run hive source path to LoadIntoPartitionedTable.hql; The following error will be shown: hive source C:\Scripts\partition\LoadIntoPartitionedTable.hql; Loading data to table default.mypartitioned partition (tag=䶵) Failed with exception null FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (HIVE-8550) Hive cannot load data into partitioned table with Unicode key
[ https://issues.apache.org/jira/browse/HIVE-8550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-8550 started by Xiaobing Zhou. --- Hive cannot load data into partitioned table with Unicode key - Key: HIVE-8550 URL: https://issues.apache.org/jira/browse/HIVE-8550 Project: Hive Issue Type: Bug Affects Versions: 0.14.0 Environment: Windows Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Priority: Critical Attachments: CreatePartitionedTable.hql, HIVE-8550.1.patch, HIVE-8550.2.patch, LoadIntoPartitionedTable.hql, partitioned.txt Steps to reproduce: 1) Copy the file partitioned.txt to the root folder of your HDFS root dir. Copy the two hql files to your local directory. 2) Open Hive CLI. 3) Run: hive source path to CreatePartitionedTable.hql; 4) Run hive source path to LoadIntoPartitionedTable.hql; The following error will be shown: hive source C:\Scripts\partition\LoadIntoPartitionedTable.hql; Loading data to table default.mypartitioned partition (tag=䶵) Failed with exception null FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask -- This message was sent by Atlassian JIRA (v6.3.4#6332)
