[jira] [Assigned] (HIVE-23037) Print Logging Information for Exception in AcidUtils tryListLocatedHdfsStatus
[ https://issues.apache.org/jira/browse/HIVE-23037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor reassigned HIVE-23037: - > Print Logging Information for Exception in AcidUtils tryListLocatedHdfsStatus > - > > Key: HIVE-23037 > URL: https://issues.apache.org/jira/browse/HIVE-23037 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HIVE-23037.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23007) Do Not Consider Client Session For Default Fetch Size
[
https://issues.apache.org/jira/browse/HIVE-23007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23007:
--
Description:
Right now, when a client makes a new session, it's value override's the
server's value for {{hive.server2.thrift.resultset.default.fetch.size}}. This
is a server-side configuration and the client should not dictate what the
server's default (preferred) size is.
* Remove superfluous null check
* Use a Collection's Singleton Map instead of creating a full-sized HashMap
* Code simplification
was:
* Remove superfluous null check
* Use a Collection's Singleton Map instead of creating a full-sized HashMap
* Code simplification
> Do Not Consider Client Session For Default Fetch Size
> -
>
> Key: HIVE-23007
> URL: https://issues.apache.org/jira/browse/HIVE-23007
> Project: Hive
> Issue Type: Sub-task
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-23007.1.patch, HIVE-23007.2.patch,
> HIVE-23017.3.patch
>
>
> Right now, when a client makes a new session, it's value override's the
> server's value for {{hive.server2.thrift.resultset.default.fetch.size}}.
> This is a server-side configuration and the client should not dictate what
> the server's default (preferred) size is.
> * Remove superfluous null check
> * Use a Collection's Singleton Map instead of creating a full-sized HashMap
> * Code simplification
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23007) Do Not Consider Client Session For Default Fetch Size
[ https://issues.apache.org/jira/browse/HIVE-23007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-23007: -- Summary: Do Not Consider Client Session For Default Fetch Size (was: Cleanup Default Fetch Size If One Is Not Sent By Client) > Do Not Consider Client Session For Default Fetch Size > - > > Key: HIVE-23007 > URL: https://issues.apache.org/jira/browse/HIVE-23007 > Project: Hive > Issue Type: Sub-task >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-23007.1.patch, HIVE-23007.2.patch, > HIVE-23017.3.patch > > > * Remove superfluous null check > * Use a Collection's Singleton Map instead of creating a full-sized HashMap > * Code simplification -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23007) Do Not Consider Client Session For Default Fetch Size
[ https://issues.apache.org/jira/browse/HIVE-23007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-23007: -- Attachment: HIVE-23017.3.patch > Do Not Consider Client Session For Default Fetch Size > - > > Key: HIVE-23007 > URL: https://issues.apache.org/jira/browse/HIVE-23007 > Project: Hive > Issue Type: Sub-task >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-23007.1.patch, HIVE-23007.2.patch, > HIVE-23017.3.patch > > > * Remove superfluous null check > * Use a Collection's Singleton Map instead of creating a full-sized HashMap > * Code simplification -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23017) Use Inherited Logger in Tasks
[
https://issues.apache.org/jira/browse/HIVE-23017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23017:
--
Attachment: (was: HIVE-23017.1.patch)
> Use Inherited Logger in Tasks
> -
>
> Key: HIVE-23017
> URL: https://issues.apache.org/jira/browse/HIVE-23017
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-23017.1.patch
>
>
> {code:java|title=FetchTask}
> public class FetchTask extends Task implements Serializable {
> private static final long serialVersionUID = 1L;
> private int maxRows = 100;
> private FetchOperator fetch;
> private ListSinkOperator sink;
> private int totalRows;
> private static transient final Logger LOG =
> LoggerFactory.getLogger(FetchTask.class);
> JobConf job = null;
> {code}
> The Tasks class is base class for many other. Cleaner to simply make the
> {{Logger}} {{protected}} and usable by other classes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23017) Use Inherited Logger in Tasks
[
https://issues.apache.org/jira/browse/HIVE-23017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23017:
--
Attachment: HIVE-23017.1.patch
> Use Inherited Logger in Tasks
> -
>
> Key: HIVE-23017
> URL: https://issues.apache.org/jira/browse/HIVE-23017
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-23017.1.patch
>
>
> {code:java|title=FetchTask}
> public class FetchTask extends Task implements Serializable {
> private static final long serialVersionUID = 1L;
> private int maxRows = 100;
> private FetchOperator fetch;
> private ListSinkOperator sink;
> private int totalRows;
> private static transient final Logger LOG =
> LoggerFactory.getLogger(FetchTask.class);
> JobConf job = null;
> {code}
> The Tasks class is base class for many other. Cleaner to simply make the
> {{Logger}} {{protected}} and usable by other classes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23017) Use Inherited Logger in Tasks
[
https://issues.apache.org/jira/browse/HIVE-23017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23017:
--
Status: Patch Available (was: Open)
> Use Inherited Logger in Tasks
> -
>
> Key: HIVE-23017
> URL: https://issues.apache.org/jira/browse/HIVE-23017
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-23017.1.patch
>
>
> {code:java|title=FetchTask}
> public class FetchTask extends Task implements Serializable {
> private static final long serialVersionUID = 1L;
> private int maxRows = 100;
> private FetchOperator fetch;
> private ListSinkOperator sink;
> private int totalRows;
> private static transient final Logger LOG =
> LoggerFactory.getLogger(FetchTask.class);
> JobConf job = null;
> {code}
> The Tasks class is base class for many other. Cleaner to simply make the
> {{Logger}} {{protected}} and usable by other classes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23017) Use Inherited Logger in Tasks
[
https://issues.apache.org/jira/browse/HIVE-23017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23017:
--
Attachment: HIVE-23017.1.patch
> Use Inherited Logger in Tasks
> -
>
> Key: HIVE-23017
> URL: https://issues.apache.org/jira/browse/HIVE-23017
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-23017.1.patch
>
>
> {code:java|title=FetchTask}
> public class FetchTask extends Task implements Serializable {
> private static final long serialVersionUID = 1L;
> private int maxRows = 100;
> private FetchOperator fetch;
> private ListSinkOperator sink;
> private int totalRows;
> private static transient final Logger LOG =
> LoggerFactory.getLogger(FetchTask.class);
> JobConf job = null;
> {code}
> The Tasks class is base class for many other. Cleaner to simply make the
> {{Logger}} {{protected}} and usable by other classes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-23017) Use Inherited Logger in Tasks
[
https://issues.apache.org/jira/browse/HIVE-23017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor reassigned HIVE-23017:
-
Assignee: David Mollitor
> Use Inherited Logger in Tasks
> -
>
> Key: HIVE-23017
> URL: https://issues.apache.org/jira/browse/HIVE-23017
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
>
> {code:java|title=FetchTask}
> public class FetchTask extends Task implements Serializable {
> private static final long serialVersionUID = 1L;
> private int maxRows = 100;
> private FetchOperator fetch;
> private ListSinkOperator sink;
> private int totalRows;
> private static transient final Logger LOG =
> LoggerFactory.getLogger(FetchTask.class);
> JobConf job = null;
> {code}
> The Tasks class is base class for many other. Cleaner to simply make the
> {{Logger}} {{protected}} and usable by other classes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23017) Use Inherited Logger in Tasks
[
https://issues.apache.org/jira/browse/HIVE-23017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23017:
--
Labels: (was: n00b newbie noob)
> Use Inherited Logger in Tasks
> -
>
> Key: HIVE-23017
> URL: https://issues.apache.org/jira/browse/HIVE-23017
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Priority: Minor
>
> {code:java|title=FetchTask}
> public class FetchTask extends Task implements Serializable {
> private static final long serialVersionUID = 1L;
> private int maxRows = 100;
> private FetchOperator fetch;
> private ListSinkOperator sink;
> private int totalRows;
> private static transient final Logger LOG =
> LoggerFactory.getLogger(FetchTask.class);
> JobConf job = null;
> {code}
> The Tasks class is base class for many other. Cleaner to simply make the
> {{Logger}} {{protected}} and usable by other classes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23017) Use Inherited Logger in Tasks
[
https://issues.apache.org/jira/browse/HIVE-23017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23017:
--
Description:
{code:java|title=FetchTask}
public class FetchTask extends Task implements Serializable {
private static final long serialVersionUID = 1L;
private int maxRows = 100;
private FetchOperator fetch;
private ListSinkOperator sink;
private int totalRows;
private static transient final Logger LOG =
LoggerFactory.getLogger(FetchTask.class);
JobConf job = null;
{code}
The Tasks class is base class for many other. Cleaner to simply make the
{{Logger}} {{protected}} and usable by other classes.
was:
{code:java|title=FetchTask}
public class FetchTask extends Task implements Serializable {
private static final long serialVersionUID = 1L;
private int maxRows = 100;
private FetchOperator fetch;
private ListSinkOperator sink;
private int totalRows;
private static transient final Logger LOG =
LoggerFactory.getLogger(FetchTask.class);
JobConf job = null;
{code}
The Tasks class is base class for many other. Cleaner to simply make the
{{Logger}} usable by other classes.
> Use Inherited Logger in Tasks
> -
>
> Key: HIVE-23017
> URL: https://issues.apache.org/jira/browse/HIVE-23017
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Priority: Minor
> Labels: n00b, newbie, noob
>
> {code:java|title=FetchTask}
> public class FetchTask extends Task implements Serializable {
> private static final long serialVersionUID = 1L;
> private int maxRows = 100;
> private FetchOperator fetch;
> private ListSinkOperator sink;
> private int totalRows;
> private static transient final Logger LOG =
> LoggerFactory.getLogger(FetchTask.class);
> JobConf job = null;
> {code}
> The Tasks class is base class for many other. Cleaner to simply make the
> {{Logger}} {{protected}} and usable by other classes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23017) Use Inherited Logger in Tasks
[
https://issues.apache.org/jira/browse/HIVE-23017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23017:
--
Description:
{code:java|title=FetchTask}
public class FetchTask extends Task implements Serializable {
private static final long serialVersionUID = 1L;
private int maxRows = 100;
private FetchOperator fetch;
private ListSinkOperator sink;
private int totalRows;
private static transient final Logger LOG =
LoggerFactory.getLogger(FetchTask.class);
JobConf job = null;
{code}
The Tasks class is base class for many other. Cleaner to simply make the
{{Logger}} usable by other classes.
was:
{code:java|title=FetchTask}
public class FetchTask extends Task implements Serializable {
private static final long serialVersionUID = 1L;
private int maxRows = 100;
private FetchOperator fetch;
private ListSinkOperator sink;
private int totalRows;
private static transient final Logger LOG =
LoggerFactory.getLogger(FetchTask.class);
JobConf job = null;
{code}
There is not need for this {{Logger}} to be transient. Please remove as it is
useless overheard.
> Use Inherited Logger in Tasks
> -
>
> Key: HIVE-23017
> URL: https://issues.apache.org/jira/browse/HIVE-23017
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Priority: Minor
> Labels: n00b, newbie, noob
>
> {code:java|title=FetchTask}
> public class FetchTask extends Task implements Serializable {
> private static final long serialVersionUID = 1L;
> private int maxRows = 100;
> private FetchOperator fetch;
> private ListSinkOperator sink;
> private int totalRows;
> private static transient final Logger LOG =
> LoggerFactory.getLogger(FetchTask.class);
> JobConf job = null;
> {code}
> The Tasks class is base class for many other. Cleaner to simply make the
> {{Logger}} usable by other classes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23017) Use Inherited Logger in Tasks
[
https://issues.apache.org/jira/browse/HIVE-23017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23017:
--
Summary: Use Inherited Logger in Tasks (was: Remove Superfluous
'Transient' Keyword From FetchTask)
> Use Inherited Logger in Tasks
> -
>
> Key: HIVE-23017
> URL: https://issues.apache.org/jira/browse/HIVE-23017
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Priority: Minor
> Labels: n00b, newbie, noob
>
> {code:java|title=FetchTask}
> public class FetchTask extends Task implements Serializable {
> private static final long serialVersionUID = 1L;
> private int maxRows = 100;
> private FetchOperator fetch;
> private ListSinkOperator sink;
> private int totalRows;
> private static transient final Logger LOG =
> LoggerFactory.getLogger(FetchTask.class);
> JobConf job = null;
> {code}
> There is not need for this {{Logger}} to be transient. Please remove as it
> is useless overheard.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22762) Leap day is incorrectly parsed during cast in Hive
[
https://issues.apache.org/jira/browse/HIVE-22762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-22762:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
Pushed to master. Thank you for the patch [~klcopp]!
> Leap day is incorrectly parsed during cast in Hive
> --
>
> Key: HIVE-22762
> URL: https://issues.apache.org/jira/browse/HIVE-22762
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-22762.01.patch, HIVE-22762.01.patch,
> HIVE-22762.01.patch, HIVE-22762.01.patch, HIVE-22762.02.patch,
> HIVE-22762.03.patch, HIVE-22762.03.patch, HIVE-22762.04.patch,
> HIVE-22762.05.patch, HIVE-22762.06.patch, HIVE-22762.07.patch,
> HIVE-22762.08.patch, HIVE-22762.09.patch, HIVE-22762.10.patch
>
>
> While casting a string to a date with a custom date format having day token
> before year and moth tokens, the date is parsed incorrectly for leap days.
> h3. How to reproduce
> Execute {code}select cast("29 02 0" as date format "dd mm rr"){code} with
> Hive. The query results in *2020-02-28*, incorrectly.
>
> Executing the another cast with a slightly modified representation of the
> date (day is preceded by year and moth) is however correctly parsed:
> {code}select cast("0 02 29" as date format "rr mm dd"){code}
> It returns *2020-02-29*.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23017) Remove Superfluous 'Transient' Keyword From FetchTask
[
https://issues.apache.org/jira/browse/HIVE-23017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23017:
--
Priority: Minor (was: Trivial)
> Remove Superfluous 'Transient' Keyword From FetchTask
> -
>
> Key: HIVE-23017
> URL: https://issues.apache.org/jira/browse/HIVE-23017
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Priority: Minor
> Labels: n00b, newbie, noob
>
> {code:java|title=FetchTask}
> public class FetchTask extends Task implements Serializable {
> private static final long serialVersionUID = 1L;
> private int maxRows = 100;
> private FetchOperator fetch;
> private ListSinkOperator sink;
> private int totalRows;
> private static transient final Logger LOG =
> LoggerFactory.getLogger(FetchTask.class);
> JobConf job = null;
> {code}
> There is not need for this {{Logger}} to be transient. Please remove as it
> is useless overheard.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23005) Consider Default JDBC Fetch Size From HS2
[
https://issues.apache.org/jira/browse/HIVE-23005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23005:
--
Status: Open (was: Patch Available)
> Consider Default JDBC Fetch Size From HS2
> -
>
> Key: HIVE-23005
> URL: https://issues.apache.org/jira/browse/HIVE-23005
> Project: Hive
> Issue Type: Sub-task
> Components: JDBC
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-23005.1.patch, HIVE-23005.2.patch,
> HIVE-23005.3.patch, HIVE-23005.4.patch, HIVE-23005.5.patch
>
>
> # Create two fetch sizes: and "init" fetch size and a "default" fetch size.
> The "init" fetch size comes from the JDBC connection string _fetchSize_ (if
> present) and the "default" fetch size comes from
> _HIVE_SERVER2_THRIFT_RESULTSET_DEFAULT_FETCH_SIZE_ the server response to the
> open session request.
> # When a {{Statement}} is created, its starting fetch size is the "init"
> fetch size (may be 0)
> # Manually setting the fetch size on the {{Statement}} to 0, sets the fetch
> size to be the server default
> Setting to zero defaults to the server's instructed default and adheres to:
> bq. If the value specified is zero, then the hint is ignored. The default
> value is zero.
> That is to say, if the fetch size is 0, the default from the server is used,
> otherwise the user can pass a 'hint' and that will be the number of rows
> fetched instead.
> https://docs.oracle.com/javase/8/docs/api/java/sql/Statement.html#setFetchSize-int-
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23005) Consider Default JDBC Fetch Size From HS2
[
https://issues.apache.org/jira/browse/HIVE-23005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23005:
--
Status: Patch Available (was: Open)
> Consider Default JDBC Fetch Size From HS2
> -
>
> Key: HIVE-23005
> URL: https://issues.apache.org/jira/browse/HIVE-23005
> Project: Hive
> Issue Type: Sub-task
> Components: JDBC
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-23005.1.patch, HIVE-23005.2.patch,
> HIVE-23005.3.patch, HIVE-23005.4.patch, HIVE-23005.5.patch
>
>
> # Create two fetch sizes: and "init" fetch size and a "default" fetch size.
> The "init" fetch size comes from the JDBC connection string _fetchSize_ (if
> present) and the "default" fetch size comes from
> _HIVE_SERVER2_THRIFT_RESULTSET_DEFAULT_FETCH_SIZE_ the server response to the
> open session request.
> # When a {{Statement}} is created, its starting fetch size is the "init"
> fetch size (may be 0)
> # Manually setting the fetch size on the {{Statement}} to 0, sets the fetch
> size to be the server default
> Setting to zero defaults to the server's instructed default and adheres to:
> bq. If the value specified is zero, then the hint is ignored. The default
> value is zero.
> That is to say, if the fetch size is 0, the default from the server is used,
> otherwise the user can pass a 'hint' and that will be the number of rows
> fetched instead.
> https://docs.oracle.com/javase/8/docs/api/java/sql/Statement.html#setFetchSize-int-
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23005) Consider Default JDBC Fetch Size From HS2
[
https://issues.apache.org/jira/browse/HIVE-23005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23005:
--
Attachment: HIVE-23005.5.patch
> Consider Default JDBC Fetch Size From HS2
> -
>
> Key: HIVE-23005
> URL: https://issues.apache.org/jira/browse/HIVE-23005
> Project: Hive
> Issue Type: Sub-task
> Components: JDBC
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-23005.1.patch, HIVE-23005.2.patch,
> HIVE-23005.3.patch, HIVE-23005.4.patch, HIVE-23005.5.patch
>
>
> # Create two fetch sizes: and "init" fetch size and a "default" fetch size.
> The "init" fetch size comes from the JDBC connection string _fetchSize_ (if
> present) and the "default" fetch size comes from
> _HIVE_SERVER2_THRIFT_RESULTSET_DEFAULT_FETCH_SIZE_ the server response to the
> open session request.
> # When a {{Statement}} is created, its starting fetch size is the "init"
> fetch size (may be 0)
> # Manually setting the fetch size on the {{Statement}} to 0, sets the fetch
> size to be the server default
> Setting to zero defaults to the server's instructed default and adheres to:
> bq. If the value specified is zero, then the hint is ignored. The default
> value is zero.
> That is to say, if the fetch size is 0, the default from the server is used,
> otherwise the user can pass a 'hint' and that will be the number of rows
> fetched instead.
> https://docs.oracle.com/javase/8/docs/api/java/sql/Statement.html#setFetchSize-int-
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22853) Allow JDBC FetchSize to Be Set in Beeline
[ https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058161#comment-17058161 ] David Mollitor commented on HIVE-22853: --- [~ngangam] Can you please review this patch? :) > Allow JDBC FetchSize to Be Set in Beeline > - > > Key: HIVE-22853 > URL: https://issues.apache.org/jira/browse/HIVE-22853 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 4.0.0 >Reporter: Naveen Gangam >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-22853.2.patch, HIVE-22853.3.patch, > HIVE-22853.4.patch, HIVE-22853.5.patch, HIVE-22853.6.patch, > HIVE-22853.7.patch, HIVE-22853.patch > > > Currently beeline uses a hard coded default of 1000 rows for fetchSize. This > default value is different from what the server has set. While the beeline > user can reset the value via set command, its cumbersome to change the > workloads. > Rather it should default to the server-side value and set should be used to > override within the session. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23005) Consider Default JDBC Fetch Size From HS2
[
https://issues.apache.org/jira/browse/HIVE-23005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23005:
--
Attachment: HIVE-23005.4.patch
> Consider Default JDBC Fetch Size From HS2
> -
>
> Key: HIVE-23005
> URL: https://issues.apache.org/jira/browse/HIVE-23005
> Project: Hive
> Issue Type: Sub-task
> Components: JDBC
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-23005.1.patch, HIVE-23005.2.patch,
> HIVE-23005.3.patch, HIVE-23005.4.patch
>
>
> # Create two fetch sizes: and "init" fetch size and a "default" fetch size.
> The "init" fetch size comes from the JDBC connection string _fetchSize_ (if
> present) and the "default" fetch size comes from
> _HIVE_SERVER2_THRIFT_RESULTSET_DEFAULT_FETCH_SIZE_ the server response to the
> open session request.
> # When a {{Statement}} is created, its starting fetch size is the "init"
> fetch size (may be 0)
> # Manually setting the fetch size on the {{Statement}} to 0, sets the fetch
> size to be the server default
> Setting to zero defaults to the server's instructed default and adheres to:
> bq. If the value specified is zero, then the hint is ignored. The default
> value is zero.
> That is to say, if the fetch size is 0, the default from the server is used,
> otherwise the user can pass a 'hint' and that will be the number of rows
> fetched instead.
> https://docs.oracle.com/javase/8/docs/api/java/sql/Statement.html#setFetchSize-int-
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23016) Extract JdbcConnectionParams from Utils Class
[ https://issues.apache.org/jira/browse/HIVE-23016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-23016: -- Description: And make it its own class. https://github.com/apache/hive/blob/4700e210ef7945278c4eb313c9ebd810b0224da1/jdbc/src/java/org/apache/hive/jdbc/Utils.java#L72 was:And make it its own class. > Extract JdbcConnectionParams from Utils Class > - > > Key: HIVE-23016 > URL: https://issues.apache.org/jira/browse/HIVE-23016 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Priority: Minor > Labels: n00b, newbie, noob > > And make it its own class. > https://github.com/apache/hive/blob/4700e210ef7945278c4eb313c9ebd810b0224da1/jdbc/src/java/org/apache/hive/jdbc/Utils.java#L72 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23007) Cleanup Default Fetch Size If One Is Not Sent By Client
[ https://issues.apache.org/jira/browse/HIVE-23007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-23007: -- Description: * Remove superfluous null check * Use a Collection's Singleton Map instead of creating a full-sized HashMap * Code simplification > Cleanup Default Fetch Size If One Is Not Sent By Client > --- > > Key: HIVE-23007 > URL: https://issues.apache.org/jira/browse/HIVE-23007 > Project: Hive > Issue Type: Sub-task >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-23007.1.patch, HIVE-23007.2.patch > > > * Remove superfluous null check > * Use a Collection's Singleton Map instead of creating a full-sized HashMap > * Code simplification -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23007) Cleanup Default Fetch Size If One Is Not Sent By Client
[ https://issues.apache.org/jira/browse/HIVE-23007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-23007: -- Summary: Cleanup Default Fetch Size If One Is Not Sent By Client (was: Server Should Return Default Fetch Size If One Is Not Sent By Client) > Cleanup Default Fetch Size If One Is Not Sent By Client > --- > > Key: HIVE-23007 > URL: https://issues.apache.org/jira/browse/HIVE-23007 > Project: Hive > Issue Type: Sub-task >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-23007.1.patch, HIVE-23007.2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22762) Leap day is incorrectly parsed during cast in Hive
[
https://issues.apache.org/jira/browse/HIVE-22762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057046#comment-17057046
]
David Mollitor commented on HIVE-22762:
---
+1
> Leap day is incorrectly parsed during cast in Hive
> --
>
> Key: HIVE-22762
> URL: https://issues.apache.org/jira/browse/HIVE-22762
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-22762.01.patch, HIVE-22762.01.patch,
> HIVE-22762.01.patch, HIVE-22762.01.patch, HIVE-22762.02.patch,
> HIVE-22762.03.patch, HIVE-22762.03.patch, HIVE-22762.04.patch,
> HIVE-22762.05.patch, HIVE-22762.06.patch, HIVE-22762.07.patch,
> HIVE-22762.08.patch, HIVE-22762.09.patch, HIVE-22762.10.patch
>
>
> While casting a string to a date with a custom date format having day token
> before year and moth tokens, the date is parsed incorrectly for leap days.
> h3. How to reproduce
> Execute {code}select cast("29 02 0" as date format "dd mm rr"){code} with
> Hive. The query results in *2020-02-28*, incorrectly.
>
> Executing the another cast with a slightly modified representation of the
> date (day is preceded by year and moth) is however correctly parsed:
> {code}select cast("0 02 29" as date format "rr mm dd"){code}
> It returns *2020-02-29*.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22853) Allow JDBC FetchSize to Be Set in Beeline
[ https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22853: -- Summary: Allow JDBC FetchSize to Be Set in Beeline (was: Beeline should use HS2 server defaults for fetchSize) > Allow JDBC FetchSize to Be Set in Beeline > - > > Key: HIVE-22853 > URL: https://issues.apache.org/jira/browse/HIVE-22853 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 4.0.0 >Reporter: Naveen Gangam >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-22853.2.patch, HIVE-22853.3.patch, > HIVE-22853.4.patch, HIVE-22853.5.patch, HIVE-22853.6.patch, > HIVE-22853.7.patch, HIVE-22853.patch > > > Currently beeline uses a hard coded default of 1000 rows for fetchSize. This > default value is different from what the server has set. While the beeline > user can reset the value via set command, its cumbersome to change the > workloads. > Rather it should default to the server-side value and set should be used to > override within the session. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23005) Consider Default JDBC Fetch Size From HS2
[
https://issues.apache.org/jira/browse/HIVE-23005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23005:
--
Attachment: HIVE-23005.3.patch
> Consider Default JDBC Fetch Size From HS2
> -
>
> Key: HIVE-23005
> URL: https://issues.apache.org/jira/browse/HIVE-23005
> Project: Hive
> Issue Type: Sub-task
> Components: JDBC
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-23005.1.patch, HIVE-23005.2.patch,
> HIVE-23005.3.patch
>
>
> # Create two fetch sizes: and "init" fetch size and a "default" fetch size.
> The "init" fetch size comes from the JDBC connection string _fetchSize_ (if
> present) and the "default" fetch size comes from
> _HIVE_SERVER2_THRIFT_RESULTSET_DEFAULT_FETCH_SIZE_ the server response to the
> open session request.
> # When a {{Statement}} is created, its starting fetch size is the "init"
> fetch size (may be 0)
> # Manually setting the fetch size on the {{Statement}} to 0, sets the fetch
> size to be the server default
> Setting to zero defaults to the server's instructed default and adheres to:
> bq. If the value specified is zero, then the hint is ignored. The default
> value is zero.
> That is to say, if the fetch size is 0, the default from the server is used,
> otherwise the user can pass a 'hint' and that will be the number of rows
> fetched instead.
> https://docs.oracle.com/javase/8/docs/api/java/sql/Statement.html#setFetchSize-int-
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22853) Beeline should use HS2 server defaults for fetchSize
[ https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22853: -- Status: Open (was: Patch Available) > Beeline should use HS2 server defaults for fetchSize > > > Key: HIVE-22853 > URL: https://issues.apache.org/jira/browse/HIVE-22853 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 4.0.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Major > Attachments: HIVE-22853.2.patch, HIVE-22853.3.patch, > HIVE-22853.4.patch, HIVE-22853.5.patch, HIVE-22853.6.patch, > HIVE-22853.7.patch, HIVE-22853.patch > > > Currently beeline uses a hard coded default of 1000 rows for fetchSize. This > default value is different from what the server has set. While the beeline > user can reset the value via set command, its cumbersome to change the > workloads. > Rather it should default to the server-side value and set should be used to > override within the session. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22853) Beeline should use HS2 server defaults for fetchSize
[ https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22853: -- Attachment: HIVE-22853.7.patch > Beeline should use HS2 server defaults for fetchSize > > > Key: HIVE-22853 > URL: https://issues.apache.org/jira/browse/HIVE-22853 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 4.0.0 >Reporter: Naveen Gangam >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-22853.2.patch, HIVE-22853.3.patch, > HIVE-22853.4.patch, HIVE-22853.5.patch, HIVE-22853.6.patch, > HIVE-22853.7.patch, HIVE-22853.patch > > > Currently beeline uses a hard coded default of 1000 rows for fetchSize. This > default value is different from what the server has set. While the beeline > user can reset the value via set command, its cumbersome to change the > workloads. > Rather it should default to the server-side value and set should be used to > override within the session. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22853) Beeline should use HS2 server defaults for fetchSize
[ https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22853: -- Status: Patch Available (was: Open) > Beeline should use HS2 server defaults for fetchSize > > > Key: HIVE-22853 > URL: https://issues.apache.org/jira/browse/HIVE-22853 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 4.0.0 >Reporter: Naveen Gangam >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-22853.2.patch, HIVE-22853.3.patch, > HIVE-22853.4.patch, HIVE-22853.5.patch, HIVE-22853.6.patch, > HIVE-22853.7.patch, HIVE-22853.patch > > > Currently beeline uses a hard coded default of 1000 rows for fetchSize. This > default value is different from what the server has set. While the beeline > user can reset the value via set command, its cumbersome to change the > workloads. > Rather it should default to the server-side value and set should be used to > override within the session. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-22853) Beeline should use HS2 server defaults for fetchSize
[ https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor reassigned HIVE-22853: - Assignee: David Mollitor (was: Naveen Gangam) > Beeline should use HS2 server defaults for fetchSize > > > Key: HIVE-22853 > URL: https://issues.apache.org/jira/browse/HIVE-22853 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 4.0.0 >Reporter: Naveen Gangam >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-22853.2.patch, HIVE-22853.3.patch, > HIVE-22853.4.patch, HIVE-22853.5.patch, HIVE-22853.6.patch, > HIVE-22853.7.patch, HIVE-22853.patch > > > Currently beeline uses a hard coded default of 1000 rows for fetchSize. This > default value is different from what the server has set. While the beeline > user can reset the value via set command, its cumbersome to change the > workloads. > Rather it should default to the server-side value and set should be used to > override within the session. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23005) Consider Default JDBC Fetch Size From HS2
[
https://issues.apache.org/jira/browse/HIVE-23005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23005:
--
Attachment: (was: HIVE-23005.2.patch)
> Consider Default JDBC Fetch Size From HS2
> -
>
> Key: HIVE-23005
> URL: https://issues.apache.org/jira/browse/HIVE-23005
> Project: Hive
> Issue Type: Sub-task
> Components: JDBC
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-23005.1.patch, HIVE-23005.2.patch
>
>
> # Create two fetch sizes: and "init" fetch size and a "default" fetch size.
> The "init" fetch size comes from the JDBC connection string _fetchSize_ (if
> present) and the "default" fetch size comes from
> _HIVE_SERVER2_THRIFT_RESULTSET_DEFAULT_FETCH_SIZE_ the server response to the
> open session request.
> # When a {{Statement}} is created, its starting fetch size is the "init"
> fetch size (may be 0)
> # Manually setting the fetch size on the {{Statement}} to 0, sets the fetch
> size to be the server default
> Setting to zero defaults to the server's instructed default and adheres to:
> bq. If the value specified is zero, then the hint is ignored. The default
> value is zero.
> That is to say, if the fetch size is 0, the default from the server is used,
> otherwise the user can pass a 'hint' and that will be the number of rows
> fetched instead.
> https://docs.oracle.com/javase/8/docs/api/java/sql/Statement.html#setFetchSize-int-
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23005) Consider Default JDBC Fetch Size From HS2
[
https://issues.apache.org/jira/browse/HIVE-23005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23005:
--
Status: Patch Available (was: Open)
Was not setting fetch size with hint of 0 correctly in constructor. Supplied
new patch.
> Consider Default JDBC Fetch Size From HS2
> -
>
> Key: HIVE-23005
> URL: https://issues.apache.org/jira/browse/HIVE-23005
> Project: Hive
> Issue Type: Sub-task
> Components: JDBC
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-23005.1.patch, HIVE-23005.2.patch
>
>
> # Create two fetch sizes: and "init" fetch size and a "default" fetch size.
> The "init" fetch size comes from the JDBC connection string _fetchSize_ (if
> present) and the "default" fetch size comes from
> _HIVE_SERVER2_THRIFT_RESULTSET_DEFAULT_FETCH_SIZE_ the server response to the
> open session request.
> # When a {{Statement}} is created, its starting fetch size is the "init"
> fetch size (may be 0)
> # Manually setting the fetch size on the {{Statement}} to 0, sets the fetch
> size to be the server default
> Setting to zero defaults to the server's instructed default and adheres to:
> bq. If the value specified is zero, then the hint is ignored. The default
> value is zero.
> That is to say, if the fetch size is 0, the default from the server is used,
> otherwise the user can pass a 'hint' and that will be the number of rows
> fetched instead.
> https://docs.oracle.com/javase/8/docs/api/java/sql/Statement.html#setFetchSize-int-
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23005) Consider Default JDBC Fetch Size From HS2
[
https://issues.apache.org/jira/browse/HIVE-23005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23005:
--
Attachment: HIVE-23005.2.patch
> Consider Default JDBC Fetch Size From HS2
> -
>
> Key: HIVE-23005
> URL: https://issues.apache.org/jira/browse/HIVE-23005
> Project: Hive
> Issue Type: Sub-task
> Components: JDBC
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-23005.1.patch, HIVE-23005.2.patch
>
>
> # Create two fetch sizes: and "init" fetch size and a "default" fetch size.
> The "init" fetch size comes from the JDBC connection string _fetchSize_ (if
> present) and the "default" fetch size comes from
> _HIVE_SERVER2_THRIFT_RESULTSET_DEFAULT_FETCH_SIZE_ the server response to the
> open session request.
> # When a {{Statement}} is created, its starting fetch size is the "init"
> fetch size (may be 0)
> # Manually setting the fetch size on the {{Statement}} to 0, sets the fetch
> size to be the server default
> Setting to zero defaults to the server's instructed default and adheres to:
> bq. If the value specified is zero, then the hint is ignored. The default
> value is zero.
> That is to say, if the fetch size is 0, the default from the server is used,
> otherwise the user can pass a 'hint' and that will be the number of rows
> fetched instead.
> https://docs.oracle.com/javase/8/docs/api/java/sql/Statement.html#setFetchSize-int-
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23005) Consider Default JDBC Fetch Size From HS2
[
https://issues.apache.org/jira/browse/HIVE-23005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23005:
--
Attachment: HIVE-23005.2.patch
> Consider Default JDBC Fetch Size From HS2
> -
>
> Key: HIVE-23005
> URL: https://issues.apache.org/jira/browse/HIVE-23005
> Project: Hive
> Issue Type: Sub-task
> Components: JDBC
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-23005.1.patch, HIVE-23005.2.patch
>
>
> # Create two fetch sizes: and "init" fetch size and a "default" fetch size.
> The "init" fetch size comes from the JDBC connection string _fetchSize_ (if
> present) and the "default" fetch size comes from
> _HIVE_SERVER2_THRIFT_RESULTSET_DEFAULT_FETCH_SIZE_ the server response to the
> open session request.
> # When a {{Statement}} is created, its starting fetch size is the "init"
> fetch size (may be 0)
> # Manually setting the fetch size on the {{Statement}} to 0, sets the fetch
> size to be the server default
> Setting to zero defaults to the server's instructed default and adheres to:
> bq. If the value specified is zero, then the hint is ignored. The default
> value is zero.
> That is to say, if the fetch size is 0, the default from the server is used,
> otherwise the user can pass a 'hint' and that will be the number of rows
> fetched instead.
> https://docs.oracle.com/javase/8/docs/api/java/sql/Statement.html#setFetchSize-int-
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23005) Consider Default JDBC Fetch Size From HS2
[
https://issues.apache.org/jira/browse/HIVE-23005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23005:
--
Status: Open (was: Patch Available)
> Consider Default JDBC Fetch Size From HS2
> -
>
> Key: HIVE-23005
> URL: https://issues.apache.org/jira/browse/HIVE-23005
> Project: Hive
> Issue Type: Sub-task
> Components: JDBC
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-23005.1.patch, HIVE-23005.2.patch
>
>
> # Create two fetch sizes: and "init" fetch size and a "default" fetch size.
> The "init" fetch size comes from the JDBC connection string _fetchSize_ (if
> present) and the "default" fetch size comes from
> _HIVE_SERVER2_THRIFT_RESULTSET_DEFAULT_FETCH_SIZE_ the server response to the
> open session request.
> # When a {{Statement}} is created, its starting fetch size is the "init"
> fetch size (may be 0)
> # Manually setting the fetch size on the {{Statement}} to 0, sets the fetch
> size to be the server default
> Setting to zero defaults to the server's instructed default and adheres to:
> bq. If the value specified is zero, then the hint is ignored. The default
> value is zero.
> That is to say, if the fetch size is 0, the default from the server is used,
> otherwise the user can pass a 'hint' and that will be the number of rows
> fetched instead.
> https://docs.oracle.com/javase/8/docs/api/java/sql/Statement.html#setFetchSize-int-
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23007) Server Should Return Default Fetch Size If One Is Not Sent By Client
[ https://issues.apache.org/jira/browse/HIVE-23007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-23007: -- Status: Patch Available (was: Open) > Server Should Return Default Fetch Size If One Is Not Sent By Client > > > Key: HIVE-23007 > URL: https://issues.apache.org/jira/browse/HIVE-23007 > Project: Hive > Issue Type: Sub-task >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-23007.1.patch, HIVE-23007.2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23007) Server Should Return Default Fetch Size If One Is Not Sent By Client
[ https://issues.apache.org/jira/browse/HIVE-23007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-23007: -- Attachment: HIVE-23007.2.patch > Server Should Return Default Fetch Size If One Is Not Sent By Client > > > Key: HIVE-23007 > URL: https://issues.apache.org/jira/browse/HIVE-23007 > Project: Hive > Issue Type: Sub-task >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-23007.1.patch, HIVE-23007.2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23007) Server Should Return Default Fetch Size If One Is Not Sent By Client
[ https://issues.apache.org/jira/browse/HIVE-23007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-23007: -- Status: Open (was: Patch Available) > Server Should Return Default Fetch Size If One Is Not Sent By Client > > > Key: HIVE-23007 > URL: https://issues.apache.org/jira/browse/HIVE-23007 > Project: Hive > Issue Type: Sub-task >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-23007.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23007) Server Should Return Default Fetch Size If One Is Not Sent By Client
[ https://issues.apache.org/jira/browse/HIVE-23007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-23007: -- Status: Patch Available (was: Open) > Server Should Return Default Fetch Size If One Is Not Sent By Client > > > Key: HIVE-23007 > URL: https://issues.apache.org/jira/browse/HIVE-23007 > Project: Hive > Issue Type: Sub-task >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-23007.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-23007) Server Should Return Default Fetch Size If One Is Not Sent By Client
[ https://issues.apache.org/jira/browse/HIVE-23007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor reassigned HIVE-23007: - > Server Should Return Default Fetch Size If One Is Not Sent By Client > > > Key: HIVE-23007 > URL: https://issues.apache.org/jira/browse/HIVE-23007 > Project: Hive > Issue Type: Sub-task >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-23007.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23007) Server Should Return Default Fetch Size If One Is Not Sent By Client
[ https://issues.apache.org/jira/browse/HIVE-23007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-23007: -- Attachment: HIVE-23007.1.patch > Server Should Return Default Fetch Size If One Is Not Sent By Client > > > Key: HIVE-23007 > URL: https://issues.apache.org/jira/browse/HIVE-23007 > Project: Hive > Issue Type: Sub-task >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-23007.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23005) Consider Default JDBC Fetch Size From HS2
[
https://issues.apache.org/jira/browse/HIVE-23005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23005:
--
Attachment: HIVE-23005.1.patch
> Consider Default JDBC Fetch Size From HS2
> -
>
> Key: HIVE-23005
> URL: https://issues.apache.org/jira/browse/HIVE-23005
> Project: Hive
> Issue Type: Sub-task
> Components: JDBC
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-23005.1.patch
>
>
> # Create two fetch sizes: and "init" fetch size and a "default" fetch size.
> The "init" fetch size comes from the JDBC connection string _fetchSize_ (if
> present) and the "default" fetch size comes from
> _HIVE_SERVER2_THRIFT_RESULTSET_DEFAULT_FETCH_SIZE_ the server response to the
> open session request.
> # When a {{Statement}} is created, its starting fetch size is the "init"
> fetch size (may be 0)
> # Manually setting the fetch size on the {{Statement}} to 0, sets the fetch
> size to be the server default
> Setting to zero defaults to the server's instructed default and adheres to:
> bq. If the value specified is zero, then the hint is ignored. The default
> value is zero.
> That is to say, if the fetch size is 0, the default from the server is used,
> otherwise the user can pass a 'hint' and that will be the number of rows
> fetched instead.
> https://docs.oracle.com/javase/8/docs/api/java/sql/Statement.html#setFetchSize-int-
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23005) Consider Default JDBC Fetch Size From HS2
[
https://issues.apache.org/jira/browse/HIVE-23005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23005:
--
Attachment: (was: HIVE-23005.1.patch)
> Consider Default JDBC Fetch Size From HS2
> -
>
> Key: HIVE-23005
> URL: https://issues.apache.org/jira/browse/HIVE-23005
> Project: Hive
> Issue Type: Sub-task
> Components: JDBC
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
>
> # Create two fetch sizes: and "init" fetch size and a "default" fetch size.
> The "init" fetch size comes from the JDBC connection string _fetchSize_ (if
> present) and the "default" fetch size comes from
> _HIVE_SERVER2_THRIFT_RESULTSET_DEFAULT_FETCH_SIZE_ the server response to the
> open session request.
> # When a {{Statement}} is created, its starting fetch size is the "init"
> fetch size (may be 0)
> # Manually setting the fetch size on the {{Statement}} to 0, sets the fetch
> size to be the server default
> Setting to zero defaults to the server's instructed default and adheres to:
> bq. If the value specified is zero, then the hint is ignored. The default
> value is zero.
> That is to say, if the fetch size is 0, the default from the server is used,
> otherwise the user can pass a 'hint' and that will be the number of rows
> fetched instead.
> https://docs.oracle.com/javase/8/docs/api/java/sql/Statement.html#setFetchSize-int-
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23005) Consider Default JDBC Fetch Size From HS2
[
https://issues.apache.org/jira/browse/HIVE-23005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23005:
--
Description:
# Create two fetch sizes: and "init" fetch size and a "default" fetch size. The
"init" fetch size comes from the JDBC connection string _fetchSize_ (if
present) and the "default" fetch size comes from
_HIVE_SERVER2_THRIFT_RESULTSET_DEFAULT_FETCH_SIZE_ the server response to the
open session request.
# When a {{Statement}} is created, its starting fetch size is the "init" fetch
size (may be 0)
# Manually setting the fetch size on the {{Statement}} to 0, sets the fetch
size to be the server default
Setting to zero defaults to the server's instructed default and adheres to:
bq. If the value specified is zero, then the hint is ignored. The default value
is zero.
That is to say, if the fetch size is 0, the default from the server is used,
otherwise the user can pass a 'hint' and that will be the number of rows
fetched instead.
https://docs.oracle.com/javase/8/docs/api/java/sql/Statement.html#setFetchSize-int-
was:
# Create two fetch sizes: and "init" fetch size and a "default" fetch size. The
"init" fetch size comes from the JDBC connection string _fetchSize_ (if
present) and the "default" fetch size comes from
_HIVE_SERVER2_THRIFT_RESULTSET_DEFAULT_FETCH_SIZE_ the server response to the
open session request.
# When a {{Statement}} is created, its starting fetch size is the "init" fetch
size (may be 0)
# Manually setting the fetch size on the {{Statement}} to 0, sets the fetch
size to be the server default
> Consider Default JDBC Fetch Size From HS2
> -
>
> Key: HIVE-23005
> URL: https://issues.apache.org/jira/browse/HIVE-23005
> Project: Hive
> Issue Type: Sub-task
> Components: JDBC
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-23005.1.patch
>
>
> # Create two fetch sizes: and "init" fetch size and a "default" fetch size.
> The "init" fetch size comes from the JDBC connection string _fetchSize_ (if
> present) and the "default" fetch size comes from
> _HIVE_SERVER2_THRIFT_RESULTSET_DEFAULT_FETCH_SIZE_ the server response to the
> open session request.
> # When a {{Statement}} is created, its starting fetch size is the "init"
> fetch size (may be 0)
> # Manually setting the fetch size on the {{Statement}} to 0, sets the fetch
> size to be the server default
> Setting to zero defaults to the server's instructed default and adheres to:
> bq. If the value specified is zero, then the hint is ignored. The default
> value is zero.
> That is to say, if the fetch size is 0, the default from the server is used,
> otherwise the user can pass a 'hint' and that will be the number of rows
> fetched instead.
> https://docs.oracle.com/javase/8/docs/api/java/sql/Statement.html#setFetchSize-int-
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23005) Consider Default JDBC Fetch Size From HS2
[
https://issues.apache.org/jira/browse/HIVE-23005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23005:
--
Description:
# Create two fetch sizes: and "init" fetch size and a "default" fetch size. The
"init" fetch size comes from the JDBC connection string _fetchSize_ (if
present) and the "default" fetch size comes from
_HIVE_SERVER2_THRIFT_RESULTSET_DEFAULT_FETCH_SIZE_ the server response to the
open session request.
# When a {{Statement}} is created, its starting fetch size is the "init" fetch
size (may be 0)
# Manually setting the fetch size on the {{Statement}} to 0, sets the fetch
size to be the server default
> Consider Default JDBC Fetch Size From HS2
> -
>
> Key: HIVE-23005
> URL: https://issues.apache.org/jira/browse/HIVE-23005
> Project: Hive
> Issue Type: Sub-task
> Components: JDBC
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-23005.1.patch
>
>
> # Create two fetch sizes: and "init" fetch size and a "default" fetch size.
> The "init" fetch size comes from the JDBC connection string _fetchSize_ (if
> present) and the "default" fetch size comes from
> _HIVE_SERVER2_THRIFT_RESULTSET_DEFAULT_FETCH_SIZE_ the server response to the
> open session request.
> # When a {{Statement}} is created, its starting fetch size is the "init"
> fetch size (may be 0)
> # Manually setting the fetch size on the {{Statement}} to 0, sets the fetch
> size to be the server default
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23005) Consider Default JDBC Fetch Size From HS2
[ https://issues.apache.org/jira/browse/HIVE-23005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-23005: -- Status: Patch Available (was: Open) > Consider Default JDBC Fetch Size From HS2 > - > > Key: HIVE-23005 > URL: https://issues.apache.org/jira/browse/HIVE-23005 > Project: Hive > Issue Type: Sub-task > Components: JDBC >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-23005.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23005) Consider Default JDBC Fetch Size From HS2
[ https://issues.apache.org/jira/browse/HIVE-23005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-23005: -- Attachment: HIVE-23005.1.patch > Consider Default JDBC Fetch Size From HS2 > - > > Key: HIVE-23005 > URL: https://issues.apache.org/jira/browse/HIVE-23005 > Project: Hive > Issue Type: Sub-task > Components: JDBC >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-23005.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23005) Consider Default JDBC Fetch Size From HS2
[ https://issues.apache.org/jira/browse/HIVE-23005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-23005: -- Status: Open (was: Patch Available) > Consider Default JDBC Fetch Size From HS2 > - > > Key: HIVE-23005 > URL: https://issues.apache.org/jira/browse/HIVE-23005 > Project: Hive > Issue Type: Sub-task > Components: JDBC >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23005) Consider Default JDBC Fetch Size From HS2
[ https://issues.apache.org/jira/browse/HIVE-23005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-23005: -- Attachment: (was: HIVE-23005.1.patch) > Consider Default JDBC Fetch Size From HS2 > - > > Key: HIVE-23005 > URL: https://issues.apache.org/jira/browse/HIVE-23005 > Project: Hive > Issue Type: Sub-task > Components: JDBC >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23005) Consider Default JDBC Fetch Size From HS2
[ https://issues.apache.org/jira/browse/HIVE-23005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-23005: -- Status: Patch Available (was: Open) > Consider Default JDBC Fetch Size From HS2 > - > > Key: HIVE-23005 > URL: https://issues.apache.org/jira/browse/HIVE-23005 > Project: Hive > Issue Type: Sub-task > Components: JDBC >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23005) Consider Default JDBC Fetch Size From HS2
[ https://issues.apache.org/jira/browse/HIVE-23005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-23005: -- Attachment: HIVE-23005.1.patch > Consider Default JDBC Fetch Size From HS2 > - > > Key: HIVE-23005 > URL: https://issues.apache.org/jira/browse/HIVE-23005 > Project: Hive > Issue Type: Sub-task > Components: JDBC >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-23005.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-23005) Consider Default JDBC Fetch Size From HS2
[ https://issues.apache.org/jira/browse/HIVE-23005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor reassigned HIVE-23005: - > Consider Default JDBC Fetch Size From HS2 > - > > Key: HIVE-23005 > URL: https://issues.apache.org/jira/browse/HIVE-23005 > Project: Hive > Issue Type: Sub-task > Components: JDBC >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22853) Beeline should use HS2 server defaults for fetchSize
[
https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17056022#comment-17056022
]
David Mollitor commented on HIVE-22853:
---
[~ngangam] OK. I put together a patch that I think addresses part of this JIRA,
but I can't seem to attach it for some reason. Maybe because it's assigned to
you and in "patch" state. I'm going to open a sub-task.
This JIRA is really two parts and should probably be 2 JIRAs:
# Update Hive JDBC to use server default fetch size
# Update Beeline to overwrite the fetch size when supplied on the command line
I do worry #2 is too specific to Beeline. If this is a good feature to have, it
should be included in the JDBC library and not just hacked into beeline in this
adhoc manner, though implementing it well would be tricky and probably not
worth the reward.
My patch does the following:
# Create two fetch sizes: and "init" fetch size and a "default" fetch size.
The "init" fetch size comes from the JDBC connection string _fetchSize_ (if
present) and the "default" fetch size comes from the server response to the
open session request.
# When a connection is made, it's starting fetch size is the "init" fetch size
(may be 0)
# Manually setting the fetch size to 0, sets the fetch size to be the server
default
This is a sensible thing to do any for the JDBC library and any application
that uses JDBC benefits, not just beeline (which has limited use).
For beeline-specific features, I think it's wonky to be parsing (regexing) a
SQL statement and then trying to decipher a HiveServer setting. I would much
rather see something specific to beeline that doesn't need to change if the HS2
or JDBC changes.
Let there be a Hive specific feature and a Beeline specific feature:
{code:java}
set hive.server2.thrift.resultset.default.fetch.size=200;
!set fetchsize 200
{code}
[http://sqlline.sourceforge.net/#sect_command_properties]
> Beeline should use HS2 server defaults for fetchSize
>
>
> Key: HIVE-22853
> URL: https://issues.apache.org/jira/browse/HIVE-22853
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 4.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Major
> Attachments: HIVE-22853.2.patch, HIVE-22853.3.patch,
> HIVE-22853.4.patch, HIVE-22853.5.patch, HIVE-22853.6.patch, HIVE-22853.patch
>
>
> Currently beeline uses a hard coded default of 1000 rows for fetchSize. This
> default value is different from what the server has set. While the beeline
> user can reset the value via set command, its cumbersome to change the
> workloads.
> Rather it should default to the server-side value and set should be used to
> override within the session.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22853) Beeline should use HS2 server defaults for fetchSize
[
https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17055945#comment-17055945
]
David Mollitor commented on HIVE-22853:
---
[~ngangam] I think you're going to want to {{trim()}} this too:
{code:java}
Matcher matcher = setPattern.matcher(sql.toLowerCase());
{code}
> Beeline should use HS2 server defaults for fetchSize
>
>
> Key: HIVE-22853
> URL: https://issues.apache.org/jira/browse/HIVE-22853
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 4.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Major
> Attachments: HIVE-22853.2.patch, HIVE-22853.3.patch,
> HIVE-22853.4.patch, HIVE-22853.5.patch, HIVE-22853.6.patch, HIVE-22853.patch
>
>
> Currently beeline uses a hard coded default of 1000 rows for fetchSize. This
> default value is different from what the server has set. While the beeline
> user can reset the value via set command, its cumbersome to change the
> workloads.
> Rather it should default to the server-side value and set should be used to
> override within the session.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (HIVE-22993) Include Bloom Filter in Column Statistics to Better Estimate nDV
[
https://issues.apache.org/jira/browse/HIVE-22993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17053607#comment-17053607
]
David Mollitor edited comment on HIVE-22993 at 3/6/20, 5:24 PM:
[~gopalv] Thanks. Do you know what JIRA introduced this change? I have been
testing on HDP 3.1
Edit: Can this BIT_VECTOR field be applied to this request for better stats on
INSERT?
was (Author: belugabehr):
[~gopalv] Thanks. Do you know what JIRA introduced this change? I have been
testing on HDP 3.1
> Include Bloom Filter in Column Statistics to Better Estimate nDV
>
>
> Key: HIVE-22993
> URL: https://issues.apache.org/jira/browse/HIVE-22993
> Project: Hive
> Issue Type: Improvement
> Components: CBO, Statistics
>Reporter: David Mollitor
>Priority: Major
>
> When performing an INSERT statement, Hive has no way to determine the number
> of distinct values since the distinct values themselves are not recorded.
> {code:sql}
> create table test_mm(`id` int, `my_dt` date);
> insert into test_mm values (1, "2018-10-01"), (2, "2018-10-01"), (3,
> "2018-10-01"),
> (4, "2017-10-01"), (5, "2017-10-01"), (6, "2017-10-01"),
> (7, "2010-10-01"), (8, "2010-10-01"), (9, "2010-10-01"),
> (10, "1998-10-01"), (11, "1998-10-01"), (12, "1998-10-01");
> DESCRIBE FORMATTED test_mm my_dt;
> -- distinct_count: 4
> insert into test_mm values (13, "2030-10-01"), (14, "2030-10-01"), (15,
> "2030-10-01");
> DESCRIBE FORMATTED test_mm my_dt;
> -- distinct_count: 4
> {code}
> The first INSERT statement sees that there are 0 records, so it makes sense
> that any distinct values marked in the statistics. However, for the second
> INSERT, Hive has no idea if "2030-10-01" is distinct, so the distinct_count
> is unchanged. By introducing a bloom filter for column statistics, the
> second INSERT may be able to determine that "2030-10-01" is indeed unique and
> update the distinct_count accordingly.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22993) Include Bloom Filter in Column Statistics to Better Estimate nDV
[
https://issues.apache.org/jira/browse/HIVE-22993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17053607#comment-17053607
]
David Mollitor commented on HIVE-22993:
---
[~gopalv] Thanks. Do you know what JIRA introduced this change? I have been
testing on HDP 3.1
> Include Bloom Filter in Column Statistics to Better Estimate nDV
>
>
> Key: HIVE-22993
> URL: https://issues.apache.org/jira/browse/HIVE-22993
> Project: Hive
> Issue Type: Improvement
> Components: CBO, Statistics
>Reporter: David Mollitor
>Priority: Major
>
> When performing an INSERT statement, Hive has no way to determine the number
> of distinct values since the distinct values themselves are not recorded.
> {code:sql}
> create table test_mm(`id` int, `my_dt` date);
> insert into test_mm values (1, "2018-10-01"), (2, "2018-10-01"), (3,
> "2018-10-01"),
> (4, "2017-10-01"), (5, "2017-10-01"), (6, "2017-10-01"),
> (7, "2010-10-01"), (8, "2010-10-01"), (9, "2010-10-01"),
> (10, "1998-10-01"), (11, "1998-10-01"), (12, "1998-10-01");
> DESCRIBE FORMATTED test_mm my_dt;
> -- distinct_count: 4
> insert into test_mm values (13, "2030-10-01"), (14, "2030-10-01"), (15,
> "2030-10-01");
> DESCRIBE FORMATTED test_mm my_dt;
> -- distinct_count: 4
> {code}
> The first INSERT statement sees that there are 0 records, so it makes sense
> that any distinct values marked in the statistics. However, for the second
> INSERT, Hive has no idea if "2030-10-01" is distinct, so the distinct_count
> is unchanged. By introducing a bloom filter for column statistics, the
> second INSERT may be able to determine that "2030-10-01" is indeed unique and
> update the distinct_count accordingly.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22673) Replace Base64 in contrib Package
[ https://issues.apache.org/jira/browse/HIVE-22673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22673: -- Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master. Thanks [~kgyrtkirk] for the review! > Replace Base64 in contrib Package > - > > Key: HIVE-22673 > URL: https://issues.apache.org/jira/browse/HIVE-22673 > Project: Hive > Issue Type: Sub-task >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HIVE-22673.1.patch, HIVE-22673.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22673) Replace Base64 in contrib Package
[ https://issues.apache.org/jira/browse/HIVE-22673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22673: -- Attachment: (was: HIVE-22673.1.patch) > Replace Base64 in contrib Package > - > > Key: HIVE-22673 > URL: https://issues.apache.org/jira/browse/HIVE-22673 > Project: Hive > Issue Type: Sub-task >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HIVE-22673.1.patch, HIVE-22673.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22762) Leap day is incorrectly parsed during cast in Hive
[
https://issues.apache.org/jira/browse/HIVE-22762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17053490#comment-17053490
]
David Mollitor commented on HIVE-22762:
---
+1 pending checkstyle fixes and testing
> Leap day is incorrectly parsed during cast in Hive
> --
>
> Key: HIVE-22762
> URL: https://issues.apache.org/jira/browse/HIVE-22762
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-22762.01.patch, HIVE-22762.01.patch,
> HIVE-22762.01.patch, HIVE-22762.01.patch, HIVE-22762.02.patch,
> HIVE-22762.03.patch, HIVE-22762.03.patch, HIVE-22762.04.patch,
> HIVE-22762.05.patch, HIVE-22762.06.patch, HIVE-22762.07.patch
>
>
> While casting a string to a date with a custom date format having day token
> before year and moth tokens, the date is parsed incorrectly for leap days.
> h3. How to reproduce
> Execute {code}select cast("29 02 0" as date format "dd mm rr"){code} with
> Hive. The query results in *2020-02-28*, incorrectly.
>
> Executing the another cast with a slightly modified representation of the
> date (day is preceded by year and moth) is however correctly parsed:
> {code}select cast("0 02 29" as date format "rr mm dd"){code}
> It returns *2020-02-29*.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22762) Leap day is incorrectly parsed during cast in Hive
[
https://issues.apache.org/jira/browse/HIVE-22762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17052295#comment-17052295
]
David Mollitor commented on HIVE-22762:
---
[~klcopp] Hey, sorry, one more correction:
{code:java}
+// Create Timestamp
+LocalDateTime ldt = LocalDateTime.ofInstant(Instant.EPOCH, ZoneOffset.UTC);
+for (Pair pair : tokenValueList) {
+ TemporalField temporalField = ((Token) pair.getLeft()).temporalField;
+ int value = (int) pair.getRight();
{code}
You shouldn't need to cast any more since you're using Pair
> Leap day is incorrectly parsed during cast in Hive
> --
>
> Key: HIVE-22762
> URL: https://issues.apache.org/jira/browse/HIVE-22762
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-22762.01.patch, HIVE-22762.01.patch,
> HIVE-22762.01.patch, HIVE-22762.01.patch, HIVE-22762.02.patch,
> HIVE-22762.03.patch, HIVE-22762.03.patch, HIVE-22762.04.patch,
> HIVE-22762.05.patch
>
>
> While casting a string to a date with a custom date format having day token
> before year and moth tokens, the date is parsed incorrectly for leap days.
> h3. How to reproduce
> Execute {code}select cast("29 02 0" as date format "dd mm rr"){code} with
> Hive. The query results in *2020-02-28*, incorrectly.
>
> Executing the another cast with a slightly modified representation of the
> date (day is preceded by year and moth) is however correctly parsed:
> {code}select cast("0 02 29" as date format "rr mm dd"){code}
> It returns *2020-02-29*.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22762) Leap day is incorrectly parsed during cast in Hive
[
https://issues.apache.org/jira/browse/HIVE-22762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17050360#comment-17050360
]
David Mollitor commented on HIVE-22762:
---
{code:java}
// add @throws JavaDoc
private Timestamp getTimestampFromValues(List temporalValues) {
if (temporalTokens.size() != temporalValues.size()) {
// use Guava Preconditions.checkState(boolean)
}
// Use parameters for ImmutablePair here (and removed future casts)
List tokensList = new ArrayList<>();
// Instead of sorting then reversing, just sort in reverse order :)
//
https://docs.oracle.com/javase/8/docs/api/java/util/Collections.html#reverseOrder-java.util.Comparator-
tokensList.sort((Comparator) (o1, o2) -> {
Token token1 = ((ImmutablePair) o1).left;
Token token2 = ((ImmutablePair) o2).left;
return token1.temporalField.getBaseUnit().getDuration()
.compareTo(token2.temporalField.getBaseUnit().getDuration());
});
Collections.reverse(tokensList);
}
{code}
Rather than adding a new token list that captures all the temporal tokens, I
would rather see that the method accept a list of tokens and a list of values.
The tokens can be filtered and sorted in the method. This way there is only
ever a single list to keep track of and users (methods) can filter however they
want on the fly.
{code:java}
List tokens = temporalTokens.stream()
.filter(token-> isNumericTemporalToken(token.type) ||
isCharacterTemporalToken(token.type))
.collect(Collectors.toList());
{code}
> Leap day is incorrectly parsed during cast in Hive
> --
>
> Key: HIVE-22762
> URL: https://issues.apache.org/jira/browse/HIVE-22762
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-22762.01.patch, HIVE-22762.01.patch,
> HIVE-22762.01.patch, HIVE-22762.01.patch, HIVE-22762.02.patch,
> HIVE-22762.03.patch, HIVE-22762.03.patch
>
>
> While casting a string to a date with a custom date format having day token
> before year and moth tokens, the date is parsed incorrectly for leap days.
> h3. How to reproduce
> Execute {code}select cast("29 02 0" as date format "dd mm rr"){code} with
> Hive. The query results in *2020-02-28*, incorrectly.
>
> Executing the another cast with a slightly modified representation of the
> date (day is preceded by year and moth) is however correctly parsed:
> {code}select cast("0 02 29" as date format "rr mm dd"){code}
> It returns *2020-02-29*.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22762) Leap day is incorrectly parsed during cast in Hive
[
https://issues.apache.org/jira/browse/HIVE-22762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049616#comment-17049616
]
David Mollitor commented on HIVE-22762:
---
Got it finally, not sure why I needed to add {{-p0}}
{code:none}
apache@apache-VirtualBox:~/hive/hive$ git apply -p0 HIVE-22762.03.patch
{code}
> Leap day is incorrectly parsed during cast in Hive
> --
>
> Key: HIVE-22762
> URL: https://issues.apache.org/jira/browse/HIVE-22762
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-22762.01.patch, HIVE-22762.01.patch,
> HIVE-22762.01.patch, HIVE-22762.01.patch, HIVE-22762.02.patch,
> HIVE-22762.03.patch, HIVE-22762.03.patch
>
>
> While casting a string to a date with a custom date format having day token
> before year and moth tokens, the date is parsed incorrectly for leap days.
> h3. How to reproduce
> Execute {code}select cast("29 02 0" as date format "dd mm rr"){code} with
> Hive. The query results in *2020-02-28*, incorrectly.
>
> Executing the another cast with a slightly modified representation of the
> date (day is preceded by year and moth) is however correctly parsed:
> {code}select cast("0 02 29" as date format "rr mm dd"){code}
> It returns *2020-02-29*.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22762) Leap day is incorrectly parsed during cast in Hive
[
https://issues.apache.org/jira/browse/HIVE-22762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049611#comment-17049611
]
David Mollitor commented on HIVE-22762:
---
[~klcopp] How are you creating these patches? I seem to have problems applying
your patches.
{code:none}
apache@apache-VirtualBox:~/hive/hive$ git branch
branch-3
branch-3.1
* master
apache@apache-VirtualBox:~/hive/hive$ git apply HIVE-22762.03.patch
error:
src/java/org/apache/hadoop/hive/common/format/datetime/HiveSqlDateTimeFormatter.java:
No such file or directory
error:
src/test/org/apache/hadoop/hive/common/format/datetime/TestHiveSqlDateTimeFormatter.java:
No such file or directory
{code}
> Leap day is incorrectly parsed during cast in Hive
> --
>
> Key: HIVE-22762
> URL: https://issues.apache.org/jira/browse/HIVE-22762
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-22762.01.patch, HIVE-22762.01.patch,
> HIVE-22762.01.patch, HIVE-22762.01.patch, HIVE-22762.02.patch,
> HIVE-22762.03.patch, HIVE-22762.03.patch
>
>
> While casting a string to a date with a custom date format having day token
> before year and moth tokens, the date is parsed incorrectly for leap days.
> h3. How to reproduce
> Execute {code}select cast("29 02 0" as date format "dd mm rr"){code} with
> Hive. The query results in *2020-02-28*, incorrectly.
>
> Executing the another cast with a slightly modified representation of the
> date (day is preceded by year and moth) is however correctly parsed:
> {code}select cast("0 02 29" as date format "rr mm dd"){code}
> It returns *2020-02-29*.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22853) Beeline should use HS2 server defaults for fetchSize
[
https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049292#comment-17049292
]
David Mollitor commented on HIVE-22853:
---
{code:java|title=Utils.java}
+ static int serverDefaultFetchSize = -1;
{code}
Please do not store state in a utility class.
> Beeline should use HS2 server defaults for fetchSize
>
>
> Key: HIVE-22853
> URL: https://issues.apache.org/jira/browse/HIVE-22853
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 4.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Major
> Attachments: HIVE-22853.2.patch, HIVE-22853.3.patch,
> HIVE-22853.4.patch, HIVE-22853.5.patch, HIVE-22853.patch
>
>
> Currently beeline uses a hard coded default of 1000 rows for fetchSize. This
> default value is different from what the server has set. While the beeline
> user can reset the value via set command, its cumbersome to change the
> workloads.
> Rather it should default to the server-side value and set should be used to
> override within the session.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22853) Beeline should use HS2 server defaults for fetchSize
[
https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049276#comment-17049276
]
David Mollitor commented on HIVE-22853:
---
{code:java}
Pattern setPattern =
Pattern.compile("(\\s*)(set)(\\s+)(\\S+)(\\s*)=(\\s*)(-?\\d+)(\\s*)(;*)");
{code}
Do not bother capturing 7 groups. Just capture the two groups that matter:
{code:java}
Pattern setPattern = Pattern.compile("\\s*set\\s+(\\S+)\\s*=\\s*(-?\\d+)\\s*;*);
{code}
Do not worry about beginning and trailing spaces:
{code:java}
Pattern setPattern = Pattern.compile("set\\s+(\\S+)\\s*=\\s*(-?\\d+)\\s*;);
Matcher matcher = setPattern.matcher(sql.toLowerCase().trim());
{code}
> Beeline should use HS2 server defaults for fetchSize
>
>
> Key: HIVE-22853
> URL: https://issues.apache.org/jira/browse/HIVE-22853
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 4.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Major
> Attachments: HIVE-22853.2.patch, HIVE-22853.3.patch,
> HIVE-22853.4.patch, HIVE-22853.5.patch, HIVE-22853.patch
>
>
> Currently beeline uses a hard coded default of 1000 rows for fetchSize. This
> default value is different from what the server has set. While the beeline
> user can reset the value via set command, its cumbersome to change the
> workloads.
> Rather it should default to the server-side value and set should be used to
> override within the session.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22853) Beeline should use HS2 server defaults for fetchSize
[
https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049271#comment-17049271
]
David Mollitor commented on HIVE-22853:
---
{code:java|title=BeeLine.java}
+ public int fetchSize = -1;
+ public void setFetchSize(int fetchRows) {
+if (fetchRows <= 0) { // if set to 0 or less, use server side default
+ this.fetchSize = Utils.getServerDefaultFetchSize();
+} else {
+ this.fetchSize = fetchRows;
+}
+ }
+
if (getOpts().timeout > -1) {
stmnt.setQueryTimeout(getOpts().timeout);
}
if (signalHandler != null) {
signalHandler.setStatement(stmnt);
}
+if (fetchSize > -1) {
+ stmnt.setFetchSize(fetchSize);
+}
{code}
# "-1" is an invalid value for fetch size, please make the default "0" so that
it lines up with the JDBC specs. The {{setFetchSize}} method makes sure the
value is always valid.
# Remove the blank line change to minimize the patch
# Do not check for a valid fetchSize, just set it
{{stmnt.setFetchSize(fetchSize);}}}
> Beeline should use HS2 server defaults for fetchSize
>
>
> Key: HIVE-22853
> URL: https://issues.apache.org/jira/browse/HIVE-22853
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 4.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Major
> Attachments: HIVE-22853.2.patch, HIVE-22853.3.patch,
> HIVE-22853.4.patch, HIVE-22853.5.patch, HIVE-22853.patch
>
>
> Currently beeline uses a hard coded default of 1000 rows for fetchSize. This
> default value is different from what the server has set. While the beeline
> user can reset the value via set command, its cumbersome to change the
> workloads.
> Rather it should default to the server-side value and set should be used to
> override within the session.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (HIVE-22853) Beeline should use HS2 server defaults for fetchSize
[
https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049271#comment-17049271
]
David Mollitor edited comment on HIVE-22853 at 3/2/20 2:55 PM:
---
{code:java|title=BeeLine.java}
+ public int fetchSize = -1;
+ public void setFetchSize(int fetchRows) {
+if (fetchRows <= 0) { // if set to 0 or less, use server side default
+ this.fetchSize = Utils.getServerDefaultFetchSize();
+} else {
+ this.fetchSize = fetchRows;
+}
+ }
+
if (getOpts().timeout > -1) {
stmnt.setQueryTimeout(getOpts().timeout);
}
if (signalHandler != null) {
signalHandler.setStatement(stmnt);
}
+if (fetchSize > -1) {
+ stmnt.setFetchSize(fetchSize);
+}
{code}
# "-1" is an invalid value for fetch size, please make the default "0" so that
it lines up with the JDBC specs. The {{setFetchSize}} method makes sure the
value is always valid.
# Remove the blank line change to minimize the patch
# Do not check for a valid fetchSize, just set it
{{stmnt.setFetchSize(fetchSize);}}
was (Author: belugabehr):
{code:java|title=BeeLine.java}
+ public int fetchSize = -1;
+ public void setFetchSize(int fetchRows) {
+if (fetchRows <= 0) { // if set to 0 or less, use server side default
+ this.fetchSize = Utils.getServerDefaultFetchSize();
+} else {
+ this.fetchSize = fetchRows;
+}
+ }
+
if (getOpts().timeout > -1) {
stmnt.setQueryTimeout(getOpts().timeout);
}
if (signalHandler != null) {
signalHandler.setStatement(stmnt);
}
+if (fetchSize > -1) {
+ stmnt.setFetchSize(fetchSize);
+}
{code}
# "-1" is an invalid value for fetch size, please make the default "0" so that
it lines up with the JDBC specs. The {{setFetchSize}} method makes sure the
value is always valid.
# Remove the blank line change to minimize the patch
# Do not check for a valid fetchSize, just set it
{{stmnt.setFetchSize(fetchSize);}}}
> Beeline should use HS2 server defaults for fetchSize
>
>
> Key: HIVE-22853
> URL: https://issues.apache.org/jira/browse/HIVE-22853
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 4.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Major
> Attachments: HIVE-22853.2.patch, HIVE-22853.3.patch,
> HIVE-22853.4.patch, HIVE-22853.5.patch, HIVE-22853.patch
>
>
> Currently beeline uses a hard coded default of 1000 rows for fetchSize. This
> default value is different from what the server has set. While the beeline
> user can reset the value via set command, its cumbersome to change the
> workloads.
> Rather it should default to the server-side value and set should be used to
> override within the session.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22763) 0 is accepted in 12-hour format during timestamp cast
[
https://issues.apache.org/jira/browse/HIVE-22763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045601#comment-17045601
]
David Mollitor commented on HIVE-22763:
---
Excellent. Thanks. +1
> 0 is accepted in 12-hour format during timestamp cast
> -
>
> Key: HIVE-22763
> URL: https://issues.apache.org/jira/browse/HIVE-22763
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-22763.01.patch, HIVE-22763.01.patch,
> HIVE-22763.01.patch, HIVE-22763.01.patch, HIVE-22763.01.patch,
> HIVE-22763.01.patch, HIVE-22763.02.patch, HIVE-22763.03.patch,
> HIVE-22763.04.patch, HIVE-22763.04.patch
>
>
> Having a timestamp string in 12-hour format can be parsed if the hour is 0,
> however, based on the [design
> document|https://docs.google.com/document/d/1V7k6-lrPGW7_uhqM-FhKl3QsxwCRy69v2KIxPsGjc1k/edit],
> it should be rejected.
> h3. How to reproduce
> Run {code}select cast("2020-01-01 0 am 00" as timestamp format "-mm-dd
> hh12 p.m. ss"){code}
> It shouldn' t be parsed, as the hour component is 0.
> h3. Spec
> ||Pattern||Meaning||Additional details||
> |HH12|Hour of day (1-12)|Same as HH|
> |HH|Hour of day (1-12)|{panel:borderStyle=none}
> - One digit inputs are possible in a string to datetime conversion but needs
> to be surrounded by separators.
> - In a datetime to string conversion one digit hours are prefixed with a zero.
> - Error if provided hour is not between 1 and 12.
> - Displaying an unformatted timestamp in Impala uses the HH24 format
> regardless if it was created using HH12.
> - If no AM/PM provided then defaults to AM.
> - In string to datetime conversion, conflicts with S and
> HH24.{panel:borderStyle=none}|
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22853) Beeline should use HS2 server defaults for fetchSize
[ https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044468#comment-17044468 ] David Mollitor commented on HIVE-22853: --- Same thing in HiveConnection.java, the fetchSize should default to 0. > Beeline should use HS2 server defaults for fetchSize > > > Key: HIVE-22853 > URL: https://issues.apache.org/jira/browse/HIVE-22853 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 4.0.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Major > Attachments: HIVE-22853.2.patch, HIVE-22853.3.patch, HIVE-22853.patch > > > Currently beeline uses a hard coded default of 1000 rows for fetchSize. This > default value is different from what the server has set. While the beeline > user can reset the value via set command, its cumbersome to change the > workloads. > Rather it should default to the server-side value and set should be used to > override within the session. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22853) Beeline should use HS2 server defaults for fetchSize
[
https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044466#comment-17044466
]
David Mollitor commented on HIVE-22853:
---
{quote}
if we follow the same spec as Oracle, we should treat less than 0 as invalid,
and default to 0 and treat zero as no-limit.
{quote}
I do not think the is correct. From the docs: _If the value specified is zero,
then the hint is ignored._
I think this means simply that the driver can use whatever default value it
wants since the client application has not provided any kind of hint.
So, actually the 'default' value should be 0 in BeeLine.java and should be
treated as 1000 (current default) in the Driver itself... this behavior is
correctly implemented in the driver:
https://github.com/apache/hive/blob/037eacea46371015a7f9894c5a9ccfb9708d5c56/jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java#L811
> Beeline should use HS2 server defaults for fetchSize
>
>
> Key: HIVE-22853
> URL: https://issues.apache.org/jira/browse/HIVE-22853
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 4.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Major
> Attachments: HIVE-22853.2.patch, HIVE-22853.3.patch, HIVE-22853.patch
>
>
> Currently beeline uses a hard coded default of 1000 rows for fetchSize. This
> default value is different from what the server has set. While the beeline
> user can reset the value via set command, its cumbersome to change the
> workloads.
> Rather it should default to the server-side value and set should be used to
> override within the session.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (HIVE-22853) Beeline should use HS2 server defaults for fetchSize
[
https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17043763#comment-17043763
]
David Mollitor edited comment on HIVE-22853 at 2/24/20 6:56 PM:
OK. I just you that there is value here, I'm not sure I see it.
If you want to do this, I don't think the current approach is correct.
[https://docs.oracle.com/javase/8/docs/api/java/sql/Statement.html#setFetchSize-int-]
{quote}
If the value specified is zero, then the hint is ignored. The default value is
zero.
SQLException - the condition rows >= 0 is not satisfied.
{quote}
So, I think you should adhere more closely to this spec to avoid ambiguity and
you don't need to check for a valid value. Just always set the fetch size on
the statement. Also think about what is the behavior if the user attempts to
set something less than zero? Right now the patch ignores the value, I'm not
sure that's the best approach. Better to validate the user supplied value and
log an error if the value is invalid otherwise user will not know that their
change didn't take affect.
Also, the proposed code is too brittle. I would like to see this actually
parse the statement and then look for the value instead of just this simple,
brittle string match. Would HiveServer2 accept the following?
{code:none}
set hive.server2.thrift.resultset.default.fetch.size=1;
set hive.server2.thrift.resultset.default.fetch.size =1
set hive.server2.thrift.resultset.default.fetch.size = 1;
{code}
Would HS2 accept these? Would your solution?
was (Author: belugabehr):
OK. I just you that there is value here, I'm not sure I see it.
If you want to do this, I don't think the current approach is correct.
[https://docs.oracle.com/javase/8/docs/api/java/sql/Statement.html#setFetchSize-int-]
{quote}
If the value specified is zero, then the hint is ignored. The default value is
zero.
SQLException - the condition rows >= 0 is not satisfied.
{quote}
So, I think you should adhere more closely to this spec to avoid ambiguity and
you don't need to check for a valid value. Just always set the fetch size on
the statement. Also think about what is the behavior if the user attempts to
set something less than zero? Right now the patch ignores the value, I'm not
sure that's the best approach. Better to validate the user supplied value and
log an error if the value is invalid otherwise user will not know that their
change didn't take affect.
Also, the proposed code is too brittle. I would like to see this actually
parse the statement and then look for the value instead of just this simple,
brittle string match. Would HiveServer2 accept the following?
{code:none}
set hive.server2.thrift.resultset.default.fetch.size=1;
set hive.server2.thrift.resultset.default.fetch.size=1;
{code}
> Beeline should use HS2 server defaults for fetchSize
>
>
> Key: HIVE-22853
> URL: https://issues.apache.org/jira/browse/HIVE-22853
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 4.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Major
> Attachments: HIVE-22853.2.patch, HIVE-22853.3.patch, HIVE-22853.patch
>
>
> Currently beeline uses a hard coded default of 1000 rows for fetchSize. This
> default value is different from what the server has set. While the beeline
> user can reset the value via set command, its cumbersome to change the
> workloads.
> Rather it should default to the server-side value and set should be used to
> override within the session.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22853) Beeline should use HS2 server defaults for fetchSize
[
https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17043763#comment-17043763
]
David Mollitor commented on HIVE-22853:
---
OK. I just you that there is value here, I'm not sure I see it.
If you want to do this, I don't think the current approach is correct.
[https://docs.oracle.com/javase/8/docs/api/java/sql/Statement.html#setFetchSize-int-]
{quote}
If the value specified is zero, then the hint is ignored. The default value is
zero.
SQLException - the condition rows >= 0 is not satisfied.
{quote}
So, I think you should adhere more closely to this spec to avoid ambiguity and
you don't need to check for a valid value. Just always set the fetch size on
the statement. Also think about what is the behavior if the user attempts to
set something less than zero? Right now the patch ignores the value, I'm not
sure that's the best approach. Better to validate the user supplied value and
log an error if the value is invalid otherwise user will not know that their
change didn't take affect.
Also, the proposed code is too brittle. I would like to see this actually
parse the statement and then look for the value instead of just this simple,
brittle string match. Would HiveServer2 accept the following?
{code:none}
set hive.server2.thrift.resultset.default.fetch.size=1;
set hive.server2.thrift.resultset.default.fetch.size=1;
{code}
> Beeline should use HS2 server defaults for fetchSize
>
>
> Key: HIVE-22853
> URL: https://issues.apache.org/jira/browse/HIVE-22853
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 4.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Major
> Attachments: HIVE-22853.2.patch, HIVE-22853.3.patch, HIVE-22853.patch
>
>
> Currently beeline uses a hard coded default of 1000 rows for fetchSize. This
> default value is different from what the server has set. While the beeline
> user can reset the value via set command, its cumbersome to change the
> workloads.
> Rather it should default to the server-side value and set should be used to
> override within the session.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22763) 0 is accepted in 12-hour format during timestamp cast
[
https://issues.apache.org/jira/browse/HIVE-22763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17043136#comment-17043136
]
David Mollitor commented on HIVE-22763:
---
... also, please add a proper JavaDoc to this method to include the parameters,
and the fact that it may throw an {{IllegalArgumentException}}
> 0 is accepted in 12-hour format during timestamp cast
> -
>
> Key: HIVE-22763
> URL: https://issues.apache.org/jira/browse/HIVE-22763
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-22763.01.patch, HIVE-22763.01.patch,
> HIVE-22763.01.patch, HIVE-22763.01.patch, HIVE-22763.01.patch,
> HIVE-22763.01.patch, HIVE-22763.02.patch
>
>
> Having a timestamp string in 12-hour format can be parsed if the hour is 0,
> however, based on the [design
> document|https://docs.google.com/document/d/1V7k6-lrPGW7_uhqM-FhKl3QsxwCRy69v2KIxPsGjc1k/edit],
> it should be rejected.
> h3. How to reproduce
> Run {code}select cast("2020-01-01 0 am 00" as timestamp format "-mm-dd
> hh12 p.m. ss"){code}
> It shouldn' t be parsed, as the hour component is 0.
> h3. Spec
> ||Pattern||Meaning||Additional details||
> |HH12|Hour of day (1-12)|Same as HH|
> |HH|Hour of day (1-12)|{panel:borderStyle=none}
> - One digit inputs are possible in a string to datetime conversion but needs
> to be surrounded by separators.
> - In a datetime to string conversion one digit hours are prefixed with a zero.
> - Error if provided hour is not between 1 and 12.
> - Displaying an unformatted timestamp in Impala uses the HH24 format
> regardless if it was created using HH12.
> - If no AM/PM provided then defaults to AM.
> - In string to datetime conversion, conflicts with S and
> HH24.{panel:borderStyle=none}|
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22763) 0 is accepted in 12-hour format during timestamp cast
[
https://issues.apache.org/jira/browse/HIVE-22763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17043135#comment-17043135
]
David Mollitor commented on HIVE-22763:
---
[~klcopp] OK. Please add some comments in the code about this and I'll grant a
+1 :)
Also note that since the "exceptions to the rule" perform an early return (or
throw an exception), there does not need to be if/else statement.
{code:java}
// exceptions to the rule
// Needs a comment (but can be left for a different day since it's outside
the scope of this path)
if (token.temporalField == ChronoField.AMPM_OF_DAY) {
return substring.toLowerCase().startsWith("a") ? AM : PM;
}
// Needs a comment explaining why "12" returns 0 and "0" throws an exception
if (token.temporalField == ChronoField.HOUR_OF_AMPM) {
if ("12".equals(substring)) {
return 0;
}
if ("0".equals(substring)) {
throw new IllegalArgumentException("Value of hour of day (hh/hh12) in
input is 0. "
+ "The value should be between 1 and 12.");
}
}
{code}
> 0 is accepted in 12-hour format during timestamp cast
> -
>
> Key: HIVE-22763
> URL: https://issues.apache.org/jira/browse/HIVE-22763
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-22763.01.patch, HIVE-22763.01.patch,
> HIVE-22763.01.patch, HIVE-22763.01.patch, HIVE-22763.01.patch,
> HIVE-22763.01.patch, HIVE-22763.02.patch
>
>
> Having a timestamp string in 12-hour format can be parsed if the hour is 0,
> however, based on the [design
> document|https://docs.google.com/document/d/1V7k6-lrPGW7_uhqM-FhKl3QsxwCRy69v2KIxPsGjc1k/edit],
> it should be rejected.
> h3. How to reproduce
> Run {code}select cast("2020-01-01 0 am 00" as timestamp format "-mm-dd
> hh12 p.m. ss"){code}
> It shouldn' t be parsed, as the hour component is 0.
> h3. Spec
> ||Pattern||Meaning||Additional details||
> |HH12|Hour of day (1-12)|Same as HH|
> |HH|Hour of day (1-12)|{panel:borderStyle=none}
> - One digit inputs are possible in a string to datetime conversion but needs
> to be surrounded by separators.
> - In a datetime to string conversion one digit hours are prefixed with a zero.
> - Error if provided hour is not between 1 and 12.
> - Displaying an unformatted timestamp in Impala uses the HH24 format
> regardless if it was created using HH12.
> - If no AM/PM provided then defaults to AM.
> - In string to datetime conversion, conflicts with S and
> HH24.{panel:borderStyle=none}|
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22914) Make Hive Connection ZK Interactions Easier to Troubleshoot
[ https://issues.apache.org/jira/browse/HIVE-22914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22914: -- Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) [~pvary] Thanks for the review! Pushed to master. > Make Hive Connection ZK Interactions Easier to Troubleshoot > --- > > Key: HIVE-22914 > URL: https://issues.apache.org/jira/browse/HIVE-22914 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0, 3.1.2 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Fix For: 4.0.0 > > Attachments: HIVE-22914.1.patch, HIVE-22914.1.patch > > > Add better logging and make errors more consistent and meaningful. > Recently was trying to troubleshoot an issue where the ZK namespace of the > client and the HS2 were different and it was way too difficult to diagnose. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22853) Beeline should use HS2 server defaults for fetchSize
[ https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17042352#comment-17042352 ] David Mollitor commented on HIVE-22853: --- [~ngangam] What is the value of being able to change this value in the middle of a session? What is the use case there? Perhaps it's better to simply prohibit users from changing the value mid-session. > Beeline should use HS2 server defaults for fetchSize > > > Key: HIVE-22853 > URL: https://issues.apache.org/jira/browse/HIVE-22853 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 4.0.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Major > Attachments: HIVE-22853.2.patch, HIVE-22853.patch > > > Currently beeline uses a hard coded default of 1000 rows for fetchSize. This > default value is different from what the server has set. While the beeline > user can reset the value via set command, its cumbersome to change the > workloads. > Rather it should default to the server-side value and set should be used to > override within the session. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22914) Make Hive Connection ZK Interactions Easier to Troubleshoot
[ https://issues.apache.org/jira/browse/HIVE-22914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22914: -- Attachment: HIVE-22914.1.patch > Make Hive Connection ZK Interactions Easier to Troubleshoot > --- > > Key: HIVE-22914 > URL: https://issues.apache.org/jira/browse/HIVE-22914 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0, 3.1.2 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HIVE-22914.1.patch, HIVE-22914.1.patch > > > Add better logging and make errors more consistent and meaningful. > Recently was trying to troubleshoot an issue where the ZK namespace of the > client and the HS2 were different and it was way too difficult to diagnose. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22914) Make Hive Connection ZK Interactions Easier to Troubleshoot
[ https://issues.apache.org/jira/browse/HIVE-22914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22914: -- Status: Patch Available (was: Open) > Make Hive Connection ZK Interactions Easier to Troubleshoot > --- > > Key: HIVE-22914 > URL: https://issues.apache.org/jira/browse/HIVE-22914 > Project: Hive > Issue Type: Improvement >Affects Versions: 3.1.2, 4.0.0 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HIVE-22914.1.patch > > > Add better logging and make errors more consistent and meaningful. > Recently was trying to troubleshoot an issue where the ZK namespace of the > client and the HS2 were different and it was way too difficult to diagnose. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22914) Make Hive Connection ZK Interactions Easier to Troubleshoot
[ https://issues.apache.org/jira/browse/HIVE-22914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22914: -- Attachment: HIVE-22914.1.patch > Make Hive Connection ZK Interactions Easier to Troubleshoot > --- > > Key: HIVE-22914 > URL: https://issues.apache.org/jira/browse/HIVE-22914 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0, 3.1.2 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HIVE-22914.1.patch > > > Add better logging and make errors more consistent and meaningful. > Recently was trying to troubleshoot an issue where the ZK namespace of the > client and the HS2 were different and it was way too difficult to diagnose. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-22914) Make Hive Connection ZK Interactions Easier to Troubleshoot
[ https://issues.apache.org/jira/browse/HIVE-22914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor reassigned HIVE-22914: - > Make Hive Connection ZK Interactions Easier to Troubleshoot > --- > > Key: HIVE-22914 > URL: https://issues.apache.org/jira/browse/HIVE-22914 > Project: Hive > Issue Type: Improvement >Affects Versions: 3.1.2, 4.0.0 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > > Add better logging and make errors more consistent and meaningful. > Recently was trying to troubleshoot an issue where the ZK namespace of the > client and the HS2 were different and it was way too difficult to diagnose. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22763) 0 is accepted in 12-hour format during timestamp cast
[
https://issues.apache.org/jira/browse/HIVE-22763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041083#comment-17041083
]
David Mollitor commented on HIVE-22763:
---
[~klcopp] Thanks.
bq. the SQL range is 1-12 (or rather 12, 1..11)
Is it the range [1,11] or [1,12] ? I would think that AM/PM the SQL range is
[1,12] based one "not between 1 and 12." Can you please clarify?
I don't know that I'd care about efficiency too much, if by efficiency you mean
execution speed. This runs in a Spark/Tez/MR context, so the simple solution
is just to add one more task. If however you're interested, I would probably
just return an {{int}} 0 here instead of re-assigning the string value and then
having to parse it.
> 0 is accepted in 12-hour format during timestamp cast
> -
>
> Key: HIVE-22763
> URL: https://issues.apache.org/jira/browse/HIVE-22763
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-22763.01.patch, HIVE-22763.01.patch,
> HIVE-22763.01.patch, HIVE-22763.01.patch, HIVE-22763.01.patch,
> HIVE-22763.01.patch
>
>
> Having a timestamp string in 12-hour format can be parsed if the hour is 0,
> however, based on the [design
> document|https://docs.google.com/document/d/1V7k6-lrPGW7_uhqM-FhKl3QsxwCRy69v2KIxPsGjc1k/edit],
> it should be rejected.
> h3. How to reproduce
> Run {code}select cast("2020-01-01 0 am 00" as timestamp format "-mm-dd
> hh12 p.m. ss"){code}
> It shouldn' t be parsed, as the hour component is 0.
> h3. Spec
> ||Pattern||Meaning||Additional details||
> |HH12|Hour of day (1-12)|Same as HH|
> |HH|Hour of day (1-12)|{panel:borderStyle=none}
> - One digit inputs are possible in a string to datetime conversion but needs
> to be surrounded by separators.
> - In a datetime to string conversion one digit hours are prefixed with a zero.
> - Error if provided hour is not between 1 and 12.
> - Displaying an unformatted timestamp in Impala uses the HH24 format
> regardless if it was created using HH12.
> - If no AM/PM provided then defaults to AM.
> - In string to datetime conversion, conflicts with S and
> HH24.{panel:borderStyle=none}|
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22763) 0 is accepted in 12-hour format during timestamp cast
[
https://issues.apache.org/jira/browse/HIVE-22763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039256#comment-17039256
]
David Mollitor commented on HIVE-22763:
---
I see this in the patch:
{code:java}
if ("12".equals(substring)) {
substring = "0";
}
{code}
Why is it reset to "0" if the value is "12"?
Also there is this existing code... what is this about and shouldn't a value
here of 0 throw an Exception?
{code:java}
private String formatNumericTemporal(int value, Token token) {
...
if (token.temporalField == ChronoField.HOUR_OF_AMPM && value == 0) {
value = 12;
}
...
}
{code}
I'd also caution of doing this validation check in the {{parseNumericTemporal}}
method. The value is parsing correctly: the value is zero. However, there
should be another place in the code to check if this is a valid value for this
Token type.
> 0 is accepted in 12-hour format during timestamp cast
> -
>
> Key: HIVE-22763
> URL: https://issues.apache.org/jira/browse/HIVE-22763
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-22763.01.patch, HIVE-22763.01.patch,
> HIVE-22763.01.patch, HIVE-22763.01.patch, HIVE-22763.01.patch,
> HIVE-22763.01.patch
>
>
> Having a timestamp string in 12-hour format can be parsed if the hour is 0,
> however, based on the [design
> document|https://docs.google.com/document/d/1V7k6-lrPGW7_uhqM-FhKl3QsxwCRy69v2KIxPsGjc1k/edit],
> it should be rejected.
> h3. How to reproduce
> Run {code}select cast("2020-01-01 0 am 00" as timestamp format "-mm-dd
> hh12 p.m. ss"){code}
> It shouldn' t be parsed, as the hour component is 0.
> h3. Spec
> ||Pattern||Meaning||Additional details||
> |HH12|Hour of day (1-12)|Same as HH|
> |HH|Hour of day (1-12)|{panel:borderStyle=none}
> - One digit inputs are possible in a string to datetime conversion but needs
> to be surrounded by separators.
> - In a datetime to string conversion one digit hours are prefixed with a zero.
> - Error if provided hour is not between 1 and 12.
> - Displaying an unformatted timestamp in Impala uses the HH24 format
> regardless if it was created using HH12.
> - If no AM/PM provided then defaults to AM.
> - In string to datetime conversion, conflicts with S and
> HH24.{panel:borderStyle=none}|
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22763) 0 is accepted in 12-hour format during timestamp cast
[
https://issues.apache.org/jira/browse/HIVE-22763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-22763:
--
Description:
Having a timestamp string in 12-hour format can be parsed if the hour is 0,
however, based on the [design
document|https://docs.google.com/document/d/1V7k6-lrPGW7_uhqM-FhKl3QsxwCRy69v2KIxPsGjc1k/edit],
it should be rejected.
h3. How to reproduce
Run {code}select cast("2020-01-01 0 am 00" as timestamp format "-mm-dd hh12
p.m. ss"){code}
It shouldn' t be parsed, as the hour component is 0.
h3. Spec
||Pattern||Meaning||Additional details||
|HH12|Hour of day (1-12)|Same as HH|
|HH|Hour of day (1-12)|{panel:borderStyle=none}
- One digit inputs are possible in a string to datetime conversion but needs to
be surrounded by separators.
- In a datetime to string conversion one digit hours are prefixed with a zero.
- Error if provided hour is not between 1 and 12.
- Displaying an unformatted timestamp in Impala uses the HH24 format regardless
if it was created using HH12.
- If no AM/PM provided then defaults to AM.
- In string to datetime conversion, conflicts with S and
HH24.{panel:borderStyle=none}|
was:
Having a timestamp string in 12-hour format can be parsed if the hour is 0,
however, based on the [design
document|https://docs.google.com/document/d/1V7k6-lrPGW7_uhqM-FhKl3QsxwCRy69v2KIxPsGjc1k/edit],
it should be rejected.
h3. How to reproduce
Run {code}select cast("2020-01-01 0 am 00" as timestamp format "-mm-dd hh12
p.m. ss"){code}
It shouldn' t be parsed, as the hour component is 0.
> 0 is accepted in 12-hour format during timestamp cast
> -
>
> Key: HIVE-22763
> URL: https://issues.apache.org/jira/browse/HIVE-22763
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-22763.01.patch, HIVE-22763.01.patch,
> HIVE-22763.01.patch, HIVE-22763.01.patch, HIVE-22763.01.patch,
> HIVE-22763.01.patch
>
>
> Having a timestamp string in 12-hour format can be parsed if the hour is 0,
> however, based on the [design
> document|https://docs.google.com/document/d/1V7k6-lrPGW7_uhqM-FhKl3QsxwCRy69v2KIxPsGjc1k/edit],
> it should be rejected.
> h3. How to reproduce
> Run {code}select cast("2020-01-01 0 am 00" as timestamp format "-mm-dd
> hh12 p.m. ss"){code}
> It shouldn' t be parsed, as the hour component is 0.
> h3. Spec
> ||Pattern||Meaning||Additional details||
> |HH12|Hour of day (1-12)|Same as HH|
> |HH|Hour of day (1-12)|{panel:borderStyle=none}
> - One digit inputs are possible in a string to datetime conversion but needs
> to be surrounded by separators.
> - In a datetime to string conversion one digit hours are prefixed with a zero.
> - Error if provided hour is not between 1 and 12.
> - Displaying an unformatted timestamp in Impala uses the HH24 format
> regardless if it was created using HH12.
> - If no AM/PM provided then defaults to AM.
> - In string to datetime conversion, conflicts with S and
> HH24.{panel:borderStyle=none}|
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21737) Upgrade Avro to version 1.9.1
[ https://issues.apache.org/jira/browse/HIVE-21737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029052#comment-17029052 ] David Mollitor commented on HIVE-21737: --- Support for Hive-on-Spark is being dropped by Cloudera. Perhaps the first order of business is simply to drop it from Hive. That may pave the way for upgrading Avro. > Upgrade Avro to version 1.9.1 > - > > Key: HIVE-21737 > URL: https://issues.apache.org/jira/browse/HIVE-21737 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ismaël Mejía >Assignee: Fokko Driesprong >Priority: Major > Labels: pull-request-available > Attachments: 0001-HIVE-21737-Bump-Apache-Avro-to-1.9.1.patch > > Time Spent: 1h > Remaining Estimate: 0h > > Avro 1.9.0 was released recently. It brings a lot of fixes including a leaner > version of Avro without Jackson in the public API. Worth the update. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22679) Replace Base64 in metastore-common Package
[ https://issues.apache.org/jira/browse/HIVE-22679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17022189#comment-17022189 ] David Mollitor commented on HIVE-22679: --- [~ngangam] Can you please take a look at this one? > Replace Base64 in metastore-common Package > -- > > Key: HIVE-22679 > URL: https://issues.apache.org/jira/browse/HIVE-22679 > Project: Hive > Issue Type: Sub-task >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HIVE-22679.1.patch, HIVE-22679.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22679) Replace Base64 in metastore-common Package
[ https://issues.apache.org/jira/browse/HIVE-22679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22679: -- Attachment: (was: HIVE-22679.1.patch) > Replace Base64 in metastore-common Package > -- > > Key: HIVE-22679 > URL: https://issues.apache.org/jira/browse/HIVE-22679 > Project: Hive > Issue Type: Sub-task >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HIVE-22679.1.patch, HIVE-22679.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22679) Replace Base64 in metastore-common Package
[ https://issues.apache.org/jira/browse/HIVE-22679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22679: -- Attachment: (was: HIVE-22679.1.patch) > Replace Base64 in metastore-common Package > -- > > Key: HIVE-22679 > URL: https://issues.apache.org/jira/browse/HIVE-22679 > Project: Hive > Issue Type: Sub-task >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HIVE-22679.1.patch, HIVE-22679.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22685) Fix TestHiveSqlDateTimeFormatter To Work With New Year 2020
[
https://issues.apache.org/jira/browse/HIVE-22685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-22685:
--
Fix Version/s: 4.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Fix TestHiveSqlDateTimeFormatter To Work With New Year 2020
> ---
>
> Key: HIVE-22685
> URL: https://issues.apache.org/jira/browse/HIVE-22685
> Project: Hive
> Issue Type: Bug
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22685.1.patch, HIVE-22685.2.patch,
> HIVE-22685.3.patch
>
>
> Unit test is now broken (n)(n):(
> {code:java}
> //Tests for these patterns would need changing every decade if done in
> the above way.
> //Thursday of the first week in an ISO year always matches the Gregorian
> year.
> checkParseTimestampIso("IY-IW-ID", "0-01-04", "iw, ", "01, " +
> thisYearString.substring(0, 3) + "0");
> checkParseTimestampIso("I-IW-ID", "0-01-04", "iw, ", "01, " +
> thisYearString.substring(0, 3) + "0");
> {code}
> {code}
> org.junit.ComparisonFailure: expected:<01, 20[1]0> but was:<01, 20[2]0>
> at org.junit.Assert.assertEquals(Assert.java:115)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.hadoop.hive.common.format.datetime.TestHiveSqlDateTimeFormatter.checkParseTimestampIso(TestHiveSqlDateTimeFormatter.java:313)
> at
> org.apache.hadoop.hive.common.format.datetime.TestHiveSqlDateTimeFormatter.testParseTimestamp(TestHiveSqlDateTimeFormatter.java:287)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22685) Fix TestHiveSqlDateTimeFormatter To Work With New Year 2020
[
https://issues.apache.org/jira/browse/HIVE-22685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17019833#comment-17019833
]
David Mollitor commented on HIVE-22685:
---
[~klcopp] [~kuczoram] Thank you for all your help.
Pushed to master.
> Fix TestHiveSqlDateTimeFormatter To Work With New Year 2020
> ---
>
> Key: HIVE-22685
> URL: https://issues.apache.org/jira/browse/HIVE-22685
> Project: Hive
> Issue Type: Bug
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-22685.1.patch, HIVE-22685.2.patch,
> HIVE-22685.3.patch
>
>
> Unit test is now broken (n)(n):(
> {code:java}
> //Tests for these patterns would need changing every decade if done in
> the above way.
> //Thursday of the first week in an ISO year always matches the Gregorian
> year.
> checkParseTimestampIso("IY-IW-ID", "0-01-04", "iw, ", "01, " +
> thisYearString.substring(0, 3) + "0");
> checkParseTimestampIso("I-IW-ID", "0-01-04", "iw, ", "01, " +
> thisYearString.substring(0, 3) + "0");
> {code}
> {code}
> org.junit.ComparisonFailure: expected:<01, 20[1]0> but was:<01, 20[2]0>
> at org.junit.Assert.assertEquals(Assert.java:115)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.hadoop.hive.common.format.datetime.TestHiveSqlDateTimeFormatter.checkParseTimestampIso(TestHiveSqlDateTimeFormatter.java:313)
> at
> org.apache.hadoop.hive.common.format.datetime.TestHiveSqlDateTimeFormatter.testParseTimestamp(TestHiveSqlDateTimeFormatter.java:287)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22685) Fix TestHiveSqlDateTimeFormatter To Work With New Year 2020
[
https://issues.apache.org/jira/browse/HIVE-22685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-22685:
--
Summary: Fix TestHiveSqlDateTimeFormatter To Work With New Year 2020 (was:
TestHiveSqlDateTimeFormatter Now Broken with New Year 2020)
> Fix TestHiveSqlDateTimeFormatter To Work With New Year 2020
> ---
>
> Key: HIVE-22685
> URL: https://issues.apache.org/jira/browse/HIVE-22685
> Project: Hive
> Issue Type: Bug
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-22685.1.patch, HIVE-22685.2.patch,
> HIVE-22685.3.patch
>
>
> Unit test is now broken (n)(n):(
> {code:java}
> //Tests for these patterns would need changing every decade if done in
> the above way.
> //Thursday of the first week in an ISO year always matches the Gregorian
> year.
> checkParseTimestampIso("IY-IW-ID", "0-01-04", "iw, ", "01, " +
> thisYearString.substring(0, 3) + "0");
> checkParseTimestampIso("I-IW-ID", "0-01-04", "iw, ", "01, " +
> thisYearString.substring(0, 3) + "0");
> {code}
> {code}
> org.junit.ComparisonFailure: expected:<01, 20[1]0> but was:<01, 20[2]0>
> at org.junit.Assert.assertEquals(Assert.java:115)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.hadoop.hive.common.format.datetime.TestHiveSqlDateTimeFormatter.checkParseTimestampIso(TestHiveSqlDateTimeFormatter.java:313)
> at
> org.apache.hadoop.hive.common.format.datetime.TestHiveSqlDateTimeFormatter.testParseTimestamp(TestHiveSqlDateTimeFormatter.java:287)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22685) TestHiveSqlDateTimeFormatter Now Broken with New Year 2020
[
https://issues.apache.org/jira/browse/HIVE-22685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017065#comment-17017065
]
David Mollitor commented on HIVE-22685:
---
[~ngangam] [~pvary] Can you please chime in? :)
> TestHiveSqlDateTimeFormatter Now Broken with New Year 2020
> --
>
> Key: HIVE-22685
> URL: https://issues.apache.org/jira/browse/HIVE-22685
> Project: Hive
> Issue Type: Bug
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-22685.1.patch, HIVE-22685.2.patch,
> HIVE-22685.3.patch
>
>
> Unit test is now broken (n)(n):(
> {code:java}
> //Tests for these patterns would need changing every decade if done in
> the above way.
> //Thursday of the first week in an ISO year always matches the Gregorian
> year.
> checkParseTimestampIso("IY-IW-ID", "0-01-04", "iw, ", "01, " +
> thisYearString.substring(0, 3) + "0");
> checkParseTimestampIso("I-IW-ID", "0-01-04", "iw, ", "01, " +
> thisYearString.substring(0, 3) + "0");
> {code}
> {code}
> org.junit.ComparisonFailure: expected:<01, 20[1]0> but was:<01, 20[2]0>
> at org.junit.Assert.assertEquals(Assert.java:115)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.hadoop.hive.common.format.datetime.TestHiveSqlDateTimeFormatter.checkParseTimestampIso(TestHiveSqlDateTimeFormatter.java:313)
> at
> org.apache.hadoop.hive.common.format.datetime.TestHiveSqlDateTimeFormatter.testParseTimestamp(TestHiveSqlDateTimeFormatter.java:287)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22685) TestHiveSqlDateTimeFormatter Now Broken with New Year 2020
[
https://issues.apache.org/jira/browse/HIVE-22685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015230#comment-17015230
]
David Mollitor commented on HIVE-22685:
---
[~klcopp] Thanks. Love the discussion.
I do really believe that hard-coded date is the way to go.
{quote}
testing with dates around 1970 – they aren't as likely to be actually used.
{quote}
If you would like to suggest other dates that you feel are more likely to be
used, let me know and we can update. I think that's kind of the point though,
that there is no such thing as a good date or bad date. However, if for
example there is some sort of edge case, the edge-case should get its own test.
One cannot introduce this edge case test if the date changes with every test
run. For example, there is some comment about {{Thursday of the first week in
an ISO year always matches the Gregorian year.}}. If this is something that
needs testing, I would ask that you put that test back in, after commit, with
the specific date that might cause an issue.
I also think that it's important that the expected outcome doesn't change. For
example:
{code:none}
-checkParseTimestampIso("IY-IW-ID", "0-01-04", "iw, ", "01, " +
thisYearString.substring(0, 3) + "0");
-checkParseTimestampIso("I-IW-ID", "0-01-04", "iw, ", "01, " +
thisYearString.substring(0, 3) + "0");
+checkParseTimestampIso("IY-IW-ID", "0-01-04", "iw, ", "01, 1970");
+checkParseTimestampIso("I-IW-ID", "0-01-04", "iw, ", "01, 1970");
//time patterns are allowed; date patterns are not
{code}
Is there a unit test now testing this unit test that
{{thisYearString.substring(0, 3)}} is correct? The cost of introducing this
extra capability to make testing possible is well worth it.
> TestHiveSqlDateTimeFormatter Now Broken with New Year 2020
> --
>
> Key: HIVE-22685
> URL: https://issues.apache.org/jira/browse/HIVE-22685
> Project: Hive
> Issue Type: Bug
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-22685.1.patch, HIVE-22685.2.patch,
> HIVE-22685.3.patch
>
>
> Unit test is now broken (n)(n):(
> {code:java}
> //Tests for these patterns would need changing every decade if done in
> the above way.
> //Thursday of the first week in an ISO year always matches the Gregorian
> year.
> checkParseTimestampIso("IY-IW-ID", "0-01-04", "iw, ", "01, " +
> thisYearString.substring(0, 3) + "0");
> checkParseTimestampIso("I-IW-ID", "0-01-04", "iw, ", "01, " +
> thisYearString.substring(0, 3) + "0");
> {code}
> {code}
> org.junit.ComparisonFailure: expected:<01, 20[1]0> but was:<01, 20[2]0>
> at org.junit.Assert.assertEquals(Assert.java:115)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.hadoop.hive.common.format.datetime.TestHiveSqlDateTimeFormatter.checkParseTimestampIso(TestHiveSqlDateTimeFormatter.java:313)
> at
> org.apache.hadoop.hive.common.format.datetime.TestHiveSqlDateTimeFormatter.testParseTimestamp(TestHiveSqlDateTimeFormatter.java:287)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22685) TestHiveSqlDateTimeFormatter Now Broken with New Year 2020
[
https://issues.apache.org/jira/browse/HIVE-22685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015112#comment-17015112
]
David Mollitor commented on HIVE-22685:
---
Added a new patch with proposed changes.
> TestHiveSqlDateTimeFormatter Now Broken with New Year 2020
> --
>
> Key: HIVE-22685
> URL: https://issues.apache.org/jira/browse/HIVE-22685
> Project: Hive
> Issue Type: Bug
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-22685.1.patch, HIVE-22685.2.patch,
> HIVE-22685.3.patch
>
>
> Unit test is now broken (n)(n):(
> {code:java}
> //Tests for these patterns would need changing every decade if done in
> the above way.
> //Thursday of the first week in an ISO year always matches the Gregorian
> year.
> checkParseTimestampIso("IY-IW-ID", "0-01-04", "iw, ", "01, " +
> thisYearString.substring(0, 3) + "0");
> checkParseTimestampIso("I-IW-ID", "0-01-04", "iw, ", "01, " +
> thisYearString.substring(0, 3) + "0");
> {code}
> {code}
> org.junit.ComparisonFailure: expected:<01, 20[1]0> but was:<01, 20[2]0>
> at org.junit.Assert.assertEquals(Assert.java:115)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.hadoop.hive.common.format.datetime.TestHiveSqlDateTimeFormatter.checkParseTimestampIso(TestHiveSqlDateTimeFormatter.java:313)
> at
> org.apache.hadoop.hive.common.format.datetime.TestHiveSqlDateTimeFormatter.testParseTimestamp(TestHiveSqlDateTimeFormatter.java:287)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22685) TestHiveSqlDateTimeFormatter Now Broken with New Year 2020
[
https://issues.apache.org/jira/browse/HIVE-22685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-22685:
--
Attachment: HIVE-22685.3.patch
> TestHiveSqlDateTimeFormatter Now Broken with New Year 2020
> --
>
> Key: HIVE-22685
> URL: https://issues.apache.org/jira/browse/HIVE-22685
> Project: Hive
> Issue Type: Bug
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-22685.1.patch, HIVE-22685.2.patch,
> HIVE-22685.3.patch
>
>
> Unit test is now broken (n)(n):(
> {code:java}
> //Tests for these patterns would need changing every decade if done in
> the above way.
> //Thursday of the first week in an ISO year always matches the Gregorian
> year.
> checkParseTimestampIso("IY-IW-ID", "0-01-04", "iw, ", "01, " +
> thisYearString.substring(0, 3) + "0");
> checkParseTimestampIso("I-IW-ID", "0-01-04", "iw, ", "01, " +
> thisYearString.substring(0, 3) + "0");
> {code}
> {code}
> org.junit.ComparisonFailure: expected:<01, 20[1]0> but was:<01, 20[2]0>
> at org.junit.Assert.assertEquals(Assert.java:115)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.hadoop.hive.common.format.datetime.TestHiveSqlDateTimeFormatter.checkParseTimestampIso(TestHiveSqlDateTimeFormatter.java:313)
> at
> org.apache.hadoop.hive.common.format.datetime.TestHiveSqlDateTimeFormatter.testParseTimestamp(TestHiveSqlDateTimeFormatter.java:287)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22685) TestHiveSqlDateTimeFormatter Now Broken with New Year 2020
[
https://issues.apache.org/jira/browse/HIVE-22685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015097#comment-17015097
]
David Mollitor commented on HIVE-22685:
---
[~klcopp] Thanks for the review.
Yes. I see now that it needs to be serialized. Thanks for bringing that to my
attention as I was surprised to see that this is a requirement. OK. Yes, Java
Optional is not Serializeable. While technically they were not intended to be
class variables, I personally think they make a lot of sense as instance
variables because of their idiot-proof-ness. If they are used as intended...
only in Getter/Setter then it adds a lot of boilerplate code that is subject to
fat-fingers. Most importantly, I tend to respect what Google does above all
else:
Their Optional implementation is serializeable so that it can be used as an
instance variable.
[https://guava.dev/releases/19.0/api/docs/com/google/common/base/Optional.html]
But yes, in this case, since it does need to be serializeable, we can use the
Guava optional (my preference) or use native null values.
I still very much encourage that we keep the ability to pass in a
{{LocalDateTime}}. Maybe we can lessen the scope to default package and add
the VisibleForTesting annotation. I think it's not ideal that the values in
the test change every time the test is ran. Sometimes its unavoidable, but in
this case passing in a known value is not too intrusive. If the test fails (as
in this case) the developer has no way to know what the value was at that exact
moment that caused it to fail. Passing in a known value, that does not change,
is ideal for debugging and testing.
> TestHiveSqlDateTimeFormatter Now Broken with New Year 2020
> --
>
> Key: HIVE-22685
> URL: https://issues.apache.org/jira/browse/HIVE-22685
> Project: Hive
> Issue Type: Bug
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-22685.1.patch, HIVE-22685.2.patch
>
>
> Unit test is now broken (n)(n):(
> {code:java}
> //Tests for these patterns would need changing every decade if done in
> the above way.
> //Thursday of the first week in an ISO year always matches the Gregorian
> year.
> checkParseTimestampIso("IY-IW-ID", "0-01-04", "iw, ", "01, " +
> thisYearString.substring(0, 3) + "0");
> checkParseTimestampIso("I-IW-ID", "0-01-04", "iw, ", "01, " +
> thisYearString.substring(0, 3) + "0");
> {code}
> {code}
> org.junit.ComparisonFailure: expected:<01, 20[1]0> but was:<01, 20[2]0>
> at org.junit.Assert.assertEquals(Assert.java:115)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.hadoop.hive.common.format.datetime.TestHiveSqlDateTimeFormatter.checkParseTimestampIso(TestHiveSqlDateTimeFormatter.java:313)
> at
> org.apache.hadoop.hive.common.format.datetime.TestHiveSqlDateTimeFormatter.testParseTimestamp(TestHiveSqlDateTimeFormatter.java:287)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22680) Replace Base64 in druid-handler Package
[ https://issues.apache.org/jira/browse/HIVE-22680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22680: -- Attachment: (was: HIVE-22680.1.patch) > Replace Base64 in druid-handler Package > --- > > Key: HIVE-22680 > URL: https://issues.apache.org/jira/browse/HIVE-22680 > Project: Hive > Issue Type: Sub-task >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HIVE-22680.1.patch, HIVE-22680.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22676) Replace Base64 in hive-service Package
[
https://issues.apache.org/jira/browse/HIVE-22676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17008376#comment-17008376
]
David Mollitor commented on HIVE-22676:
---
Will need to look at this change more:
{code:none}
2019-12-29T15:22:03,106 ERROR [HiveServer2-HttpHandler-Pool: Thread-71]
thrift.ThriftHttpServlet: Error:
org.apache.hive.service.auth.HttpAuthenticationException:
java.lang.IllegalArgumentException: Illegal base64 character 3a
at
org.apache.hive.service.cli.thrift.ThriftHttpServlet.doKerberosAuth(ThriftHttpServlet.java:450)
~[hive-service-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at
org.apache.hive.service.cli.thrift.ThriftHttpServlet.doPost(ThriftHttpServlet.java:201)
[hive-service-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
[javax.servlet-api-3.1.0.jar:3.1.0]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
[javax.servlet-api-3.1.0.jar:3.1.0]
at
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848)
[jetty-runner-9.3.27.v20190418.jar:9.3.27.v20190418]
{code}
> Replace Base64 in hive-service Package
> --
>
> Key: HIVE-22676
> URL: https://issues.apache.org/jira/browse/HIVE-22676
> Project: Hive
> Issue Type: Sub-task
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-22676.1.patch, HIVE-22676.2.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22679) Replace Base64 in metastore-common Package
[ https://issues.apache.org/jira/browse/HIVE-22679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22679: -- Attachment: HIVE-22679.1.patch > Replace Base64 in metastore-common Package > -- > > Key: HIVE-22679 > URL: https://issues.apache.org/jira/browse/HIVE-22679 > Project: Hive > Issue Type: Sub-task >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HIVE-22679.1.patch, HIVE-22679.1.patch, > HIVE-22679.1.patch, HIVE-22679.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22678) Run Eclipse Cleanup Against hive-accumulo-handler Module
[ https://issues.apache.org/jira/browse/HIVE-22678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17008373#comment-17008373 ] David Mollitor commented on HIVE-22678: --- [~mgergely] Are you able to take a swing at reviewing this one too? Thanks! > Run Eclipse Cleanup Against hive-accumulo-handler Module > > > Key: HIVE-22678 > URL: https://issues.apache.org/jira/browse/HIVE-22678 > Project: Hive > Issue Type: Sub-task >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HIVE-22678.1.patch, HIVE-22678.2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22673) Replace Base64 in contrib Package
[ https://issues.apache.org/jira/browse/HIVE-22673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17008367#comment-17008367 ] David Mollitor commented on HIVE-22673: --- [~ashutoshc] Can you also take a look at this one since you've been so helpful with my other JIRAs regarding this change? Thanks! > Replace Base64 in contrib Package > - > > Key: HIVE-22673 > URL: https://issues.apache.org/jira/browse/HIVE-22673 > Project: Hive > Issue Type: Sub-task >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HIVE-22673.1.patch, HIVE-22673.1.patch, > HIVE-22673.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
