subject:"\[jira\] \[Commented\] \(HIVE\-2693\) Add DECIMAL data type"

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2013-02-01 Thread Mark Grover (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13569234#comment-13569234
 ] 

Mark Grover commented on HIVE-2693:
---

Thanks for committing this, Carl!

FYI, I have updated the wiki pages with info about Decimal type.
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Types
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Fix For: 0.11.0

 Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch, 
 HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch, 
 HIVE-2693-13.patch, HIVE-2693-14.patch, HIVE-2693-15.patch, 
 HIVE-2693-16.patch, HIVE-2693-17.patch, HIVE-2693-18.patch, 
 HIVE-2693-19.patch, HIVE-2693-1.patch.txt, HIVE-2693-20.patch, 
 HIVE-2693-21.patch, HIVE-2693-22.patch, HIVE-2693-23.patch, 
 HIVE-2693-all.patch, HIVE-2693.D7683.1.patch, HIVE-2693-fix.patch, 
 HIVE-2693.patch, HIVE-2693-take3.patch, HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2013-01-14 Thread Gunther Hagleitner (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13552495#comment-13552495
 ] 

Gunther Hagleitner commented on HIVE-2693:
--

Mark: Thanks for the clarification. I had misunderstood you earlier. I think 
you do bring up an important point about controlling representation, however, I 
think my point about best fixing this with an optional scoped decimal is still 
valid.

The example of the poor C++ thrift developer eventually got me thinking: Why 
would setting the scale in round even have any effect? We strip trailing zeros 
internally from all decimals, so 0.00, 0E-99 should always come out as simply 0.

(I think we've talked about this before, we have to strip zeros, because 
otherwise sorting, joins, group-bys etc would all internally distinguish by 
value and scale instead of just value, which would be bad because we allow 
mixed-scale input and any operation performed on decimals can change the scale.)

So given that, the reason you are seeing the sticky scale on round() is this: 
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6480539

A bug in the JVM: stripTrailingZeros doesn't work for the number 0.

I've put a workaround for that bug in the code (simply storing BigDecimal.ZERO 
when the value is 0) and that gives at least consistent behavior between 
floor, ceiling and round, however that still does not allow you to control the 
representation. For that (given that we have to strip internally), I still 
suggest another jira that let's you define types that do specifically that.

It does also help the C++ coder: With this round(mycol, 2) would give you at 
most 2 decimal digits (instead of potentially unlimited many). So if they init 
that buffer right, they should be fine.

So can you take a look at patch 23? It only special-cases the ZERO value (and 
removes the meaningless setting to original scale in round). Does that seem 
reasonable?

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch, 
 HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch, 
 HIVE-2693-13.patch, HIVE-2693-14.patch, HIVE-2693-15.patch, 
 HIVE-2693-16.patch, HIVE-2693-17.patch, HIVE-2693-18.patch, 
 HIVE-2693-19.patch, HIVE-2693-1.patch.txt, HIVE-2693-20.patch, 
 HIVE-2693-21.patch, HIVE-2693-22.patch, HIVE-2693-all.patch, 
 HIVE-2693.D7683.1.patch, HIVE-2693-fix.patch, HIVE-2693.patch, 
 HIVE-2693-take3.patch, HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2013-01-14 Thread Carl Steinbach (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13553400#comment-13553400
 ] 

Carl Steinbach commented on HIVE-2693:
--

+1 on patch 23. Running tests. Will commit if everything passes.

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch, 
 HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch, 
 HIVE-2693-13.patch, HIVE-2693-14.patch, HIVE-2693-15.patch, 
 HIVE-2693-16.patch, HIVE-2693-17.patch, HIVE-2693-18.patch, 
 HIVE-2693-19.patch, HIVE-2693-1.patch.txt, HIVE-2693-20.patch, 
 HIVE-2693-21.patch, HIVE-2693-22.patch, HIVE-2693-23.patch, 
 HIVE-2693-all.patch, HIVE-2693.D7683.1.patch, HIVE-2693-fix.patch, 
 HIVE-2693.patch, HIVE-2693-take3.patch, HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2013-01-13 Thread Mark Grover (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13552151#comment-13552151
 ] 

Mark Grover commented on HIVE-2693:
---

[~hagleitn], I see what you mean. The reason I had changed the type was to be 
consistent with other methods in the UDF in terms of scale of the return 
type, albeit at the cost of losing precision. In any case, given what you found 
in documentation, I agree that it makes sense to retain the type to decimal. 

However, the scale of the return value in floor/ceil/round UDFs for decimal 
type methods seems incorrect to me. We should be not be setting the return 
values back to the original scale.

This seems to lead to inconsistencies in ceil/floor and a bug in round UDF.
For example,
{code}
mysql select ceil(-0.33) from t;
+-+
| ceil(-0.33) |
+-+
|   0 |
+-+
1 row in set (0.00 sec)
{code}
However, patch 21 gives ceil(-0.33) as 0.00. With the proposed change, the 
result in hive would be consistent in representation with MySQL.

Here is what seems like a bug in round UDF:
In MySQL:
round(1E-99,2)=0.00
But, hive gives
round(1E-99,2)=0E-99
which seems to be the incorrect representation.

The proposed change will make hive give the correct result (consistent with 
MySQL): round(1E-99,2)=0.00

What do you think? If you agree, I can upload the new patch. Please let me 
know. Thanks!

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch, 
 HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch, 
 HIVE-2693-13.patch, HIVE-2693-14.patch, HIVE-2693-15.patch, 
 HIVE-2693-16.patch, HIVE-2693-17.patch, HIVE-2693-18.patch, 
 HIVE-2693-19.patch, HIVE-2693-1.patch.txt, HIVE-2693-20.patch, 
 HIVE-2693-21.patch, HIVE-2693-22.patch, HIVE-2693-all.patch, 
 HIVE-2693.D7683.1.patch, HIVE-2693-fix.patch, HIVE-2693.patch, 
 HIVE-2693-take3.patch, HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2013-01-13 Thread Namit Jain (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13552219#comment-13552219
 ] 

Namit Jain commented on HIVE-2693:
--

Either approach is fine, as long as we document it clearly.

Intuitively, returning a long makes sense, barring the corner cases.
But, I am open.


 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch, 
 HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch, 
 HIVE-2693-13.patch, HIVE-2693-14.patch, HIVE-2693-15.patch, 
 HIVE-2693-16.patch, HIVE-2693-17.patch, HIVE-2693-18.patch, 
 HIVE-2693-19.patch, HIVE-2693-1.patch.txt, HIVE-2693-20.patch, 
 HIVE-2693-21.patch, HIVE-2693-22.patch, HIVE-2693-all.patch, 
 HIVE-2693.D7683.1.patch, HIVE-2693-fix.patch, HIVE-2693.patch, 
 HIVE-2693-take3.patch, HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2013-01-13 Thread Mark Grover (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13552312#comment-13552312
 ] 

Mark Grover commented on HIVE-2693:
---

Gunther: thanks for the feedback! I understand what you are saying about the 
different representation for the same number. That's why I referred to the 
scale change as inconsistencies for floor/ceil but a bug for round. I called 
them inconsistencies for floor/ceil because they are the exact same numbers and 
like we have discussed before the decimal patch considers same numbers with 
different representation equal (by using compareTo() instead of equals()). 
However, here is why I considered it a bug for round because by definition 
round(x,d) rounds number x to d decimal places. While 0.00, 0 and 0E-99 are all 
the same number (with different representation), only one of them has 2 decimal 
places as expected by the result of round(1E-99, 2). Here is an example where 
it could cause problems: if someone was using the thrift client to issue Hive 
queries from C++, and issued a query like this:
{code}
select round(mycol,2) from mytable;
{code}
and split the output based on the decimal point to obtain the fractional part; 
they would expect the fractional part to fit in a string of length of 2. 
However, given the present implementation, that's not the case.

Consistency with MySQL aside, I don't think we should be setting the value back 
to the original scale, especially in round.

Having said the above, I agree that they are just different representations of 
the same number so if you feel strongly about not changing this, I happily +1 
patch 21.

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch, 
 HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch, 
 HIVE-2693-13.patch, HIVE-2693-14.patch, HIVE-2693-15.patch, 
 HIVE-2693-16.patch, HIVE-2693-17.patch, HIVE-2693-18.patch, 
 HIVE-2693-19.patch, HIVE-2693-1.patch.txt, HIVE-2693-20.patch, 
 HIVE-2693-21.patch, HIVE-2693-22.patch, HIVE-2693-all.patch, 
 HIVE-2693.D7683.1.patch, HIVE-2693-fix.patch, HIVE-2693.patch, 
 HIVE-2693-take3.patch, HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2013-01-12 Thread Namit Jain (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551969#comment-13551969
 ] 

Namit Jain commented on HIVE-2693:
--

All, Does https://reviews.facebook.net/D7779 contain the latest patch ?

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch, 
 HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch, 
 HIVE-2693-13.patch, HIVE-2693-14.patch, HIVE-2693-15.patch, 
 HIVE-2693-16.patch, HIVE-2693-17.patch, HIVE-2693-18.patch, 
 HIVE-2693-19.patch, HIVE-2693-1.patch.txt, HIVE-2693-20.patch, 
 HIVE-2693-21.patch, HIVE-2693-all.patch, HIVE-2693.D7683.1.patch, 
 HIVE-2693-fix.patch, HIVE-2693.patch, HIVE-2693-take3.patch, 
 HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2013-01-12 Thread Namit Jain (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551970#comment-13551970
 ] 

Namit Jain commented on HIVE-2693:
--

I see that some comments from Ashutosh have not been addressed.

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch, 
 HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch, 
 HIVE-2693-13.patch, HIVE-2693-14.patch, HIVE-2693-15.patch, 
 HIVE-2693-16.patch, HIVE-2693-17.patch, HIVE-2693-18.patch, 
 HIVE-2693-19.patch, HIVE-2693-1.patch.txt, HIVE-2693-20.patch, 
 HIVE-2693-21.patch, HIVE-2693-all.patch, HIVE-2693.D7683.1.patch, 
 HIVE-2693-fix.patch, HIVE-2693.patch, HIVE-2693-take3.patch, 
 HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2013-01-12 Thread Namit Jain (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551974#comment-13551974
 ] 

Namit Jain commented on HIVE-2693:
--

minor comments on phabricator

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch, 
 HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch, 
 HIVE-2693-13.patch, HIVE-2693-14.patch, HIVE-2693-15.patch, 
 HIVE-2693-16.patch, HIVE-2693-17.patch, HIVE-2693-18.patch, 
 HIVE-2693-19.patch, HIVE-2693-1.patch.txt, HIVE-2693-20.patch, 
 HIVE-2693-21.patch, HIVE-2693-all.patch, HIVE-2693.D7683.1.patch, 
 HIVE-2693-fix.patch, HIVE-2693.patch, HIVE-2693-take3.patch, 
 HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2013-01-12 Thread Ashutosh Chauhan (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551998#comment-13551998
 ] 

Ashutosh Chauhan commented on HIVE-2693:


@Namit: I think you looked at phabricator patch, but if you look at the latest 
patch which is attached here on jira HIVE-2693-21.patch my comments has been 
addressed. 

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch, 
 HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch, 
 HIVE-2693-13.patch, HIVE-2693-14.patch, HIVE-2693-15.patch, 
 HIVE-2693-16.patch, HIVE-2693-17.patch, HIVE-2693-18.patch, 
 HIVE-2693-19.patch, HIVE-2693-1.patch.txt, HIVE-2693-20.patch, 
 HIVE-2693-21.patch, HIVE-2693-all.patch, HIVE-2693.D7683.1.patch, 
 HIVE-2693-fix.patch, HIVE-2693.patch, HIVE-2693-take3.patch, 
 HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2013-01-12 Thread Mark Grover (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13552027#comment-13552027
 ] 

Mark Grover commented on HIVE-2693:
---

Thanks for the review, Namit, Carl and Ashutosh!

Ashutosh's comments were addressed in patch 21. I think we missed updating the 
review. Gunther, can you please update the review with patch 21. 

[~namit]: Your review comments about ceil, round, floor, etc. are correct. 
MySQL doesn't return decimal on floor(decimal), etc. So, I will fix that.

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch, 
 HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch, 
 HIVE-2693-13.patch, HIVE-2693-14.patch, HIVE-2693-15.patch, 
 HIVE-2693-16.patch, HIVE-2693-17.patch, HIVE-2693-18.patch, 
 HIVE-2693-19.patch, HIVE-2693-1.patch.txt, HIVE-2693-20.patch, 
 HIVE-2693-21.patch, HIVE-2693-all.patch, HIVE-2693.D7683.1.patch, 
 HIVE-2693-fix.patch, HIVE-2693.patch, HIVE-2693-take3.patch, 
 HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2013-01-12 Thread Gunther Hagleitner (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13552145#comment-13552145
 ] 

Gunther Hagleitner commented on HIVE-2693:
--

I don't think ceil/floor should return long. If the decimal is larger than the 
max long value it will result in truncation and return an incorrect result, 
which isn't necessary if we stay in the same type. As far as compatibility 
goes: Judging from the docs MySQL just promises that exact-value goes to an 
exact-value type and floating point to floating point. Returning decimal would 
be in line with this. Oracle/SQL Server stay in the same datatype.

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch, 
 HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch, 
 HIVE-2693-13.patch, HIVE-2693-14.patch, HIVE-2693-15.patch, 
 HIVE-2693-16.patch, HIVE-2693-17.patch, HIVE-2693-18.patch, 
 HIVE-2693-19.patch, HIVE-2693-1.patch.txt, HIVE-2693-20.patch, 
 HIVE-2693-21.patch, HIVE-2693-22.patch, HIVE-2693-all.patch, 
 HIVE-2693.D7683.1.patch, HIVE-2693-fix.patch, HIVE-2693.patch, 
 HIVE-2693-take3.patch, HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2013-01-12 Thread Gunther Hagleitner (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13552148#comment-13552148
 ] 

Gunther Hagleitner commented on HIVE-2693:
--

As requested: Created https://reviews.facebook.net/D7875

This is the same as HIVE-2693-21.patch, which if we agree on decimal return 
type for floor/ceil is the patch to use. It has the fixes Ashutosh requested.

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch, 
 HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch, 
 HIVE-2693-13.patch, HIVE-2693-14.patch, HIVE-2693-15.patch, 
 HIVE-2693-16.patch, HIVE-2693-17.patch, HIVE-2693-18.patch, 
 HIVE-2693-19.patch, HIVE-2693-1.patch.txt, HIVE-2693-20.patch, 
 HIVE-2693-21.patch, HIVE-2693-22.patch, HIVE-2693-all.patch, 
 HIVE-2693.D7683.1.patch, HIVE-2693-fix.patch, HIVE-2693.patch, 
 HIVE-2693-take3.patch, HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2013-01-11 Thread Carl Steinbach (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13550950#comment-13550950
 ] 

Carl Steinbach commented on HIVE-2693:
--

+1. Changes look good to me.

Ashutosh and Namit: let me know if you want more time to look at this. 
Otherwise I'll plan to commit it in 24 hours. Thanks.

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch, 
 HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch, 
 HIVE-2693-13.patch, HIVE-2693-14.patch, HIVE-2693-15.patch, 
 HIVE-2693-16.patch, HIVE-2693-17.patch, HIVE-2693-18.patch, 
 HIVE-2693-19.patch, HIVE-2693-1.patch.txt, HIVE-2693-20.patch, 
 HIVE-2693-21.patch, HIVE-2693-all.patch, HIVE-2693.D7683.1.patch, 
 HIVE-2693-fix.patch, HIVE-2693.patch, HIVE-2693-take3.patch, 
 HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2013-01-11 Thread Ashutosh Chauhan (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13550965#comment-13550965
 ] 

Ashutosh Chauhan commented on HIVE-2693:


Latest patch looks good to me. +1

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch, 
 HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch, 
 HIVE-2693-13.patch, HIVE-2693-14.patch, HIVE-2693-15.patch, 
 HIVE-2693-16.patch, HIVE-2693-17.patch, HIVE-2693-18.patch, 
 HIVE-2693-19.patch, HIVE-2693-1.patch.txt, HIVE-2693-20.patch, 
 HIVE-2693-21.patch, HIVE-2693-all.patch, HIVE-2693.D7683.1.patch, 
 HIVE-2693-fix.patch, HIVE-2693.patch, HIVE-2693-take3.patch, 
 HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2013-01-11 Thread Namit Jain (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13550970#comment-13550970
 ] 

Namit Jain commented on HIVE-2693:
--

Carl, give me some time. I will try to take a look today/tomorrow.
Can you give me ~2 days ? Go ahead, if you dont hear from me.

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch, 
 HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch, 
 HIVE-2693-13.patch, HIVE-2693-14.patch, HIVE-2693-15.patch, 
 HIVE-2693-16.patch, HIVE-2693-17.patch, HIVE-2693-18.patch, 
 HIVE-2693-19.patch, HIVE-2693-1.patch.txt, HIVE-2693-20.patch, 
 HIVE-2693-21.patch, HIVE-2693-all.patch, HIVE-2693.D7683.1.patch, 
 HIVE-2693-fix.patch, HIVE-2693.patch, HIVE-2693-take3.patch, 
 HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2013-01-07 Thread Mark Grover (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13546256#comment-13546256
 ] 

Mark Grover commented on HIVE-2693:
---

Non-committer +1

Namit, any thoughts on the UDF method selection logic?

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch, 
 HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch, 
 HIVE-2693-13.patch, HIVE-2693-14.patch, HIVE-2693-15.patch, 
 HIVE-2693-16.patch, HIVE-2693-17.patch, HIVE-2693-18.patch, 
 HIVE-2693-19.patch, HIVE-2693-1.patch.txt, HIVE-2693-all.patch, 
 HIVE-2693.D7683.1.patch, HIVE-2693-fix.patch, HIVE-2693.patch, 
 HIVE-2693-take3.patch, HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2013-01-04 Thread Gunther Hagleitner (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13543787#comment-13543787
 ] 

Gunther Hagleitner commented on HIVE-2693:
--

This one should address the comments. Added the requested new tests: 
https://reviews.facebook.net/D7779

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch, 
 HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch, 
 HIVE-2693-13.patch, HIVE-2693-14.patch, HIVE-2693-15.patch, 
 HIVE-2693-16.patch, HIVE-2693-1.patch.txt, HIVE-2693-all.patch, 
 HIVE-2693.D7683.1.patch, HIVE-2693-fix.patch, HIVE-2693.patch, 
 HIVE-2693-take3.patch, HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2013-01-04 Thread Mark Grover (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13544204#comment-13544204
 ] 

Mark Grover commented on HIVE-2693:
---

Some comments on the reviews. 

I agree with Namit that we should deal with the ambiguous UDF issue separately 
in a different JIRA. Created a subtask for that: HIVE-3859

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch, 
 HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch, 
 HIVE-2693-13.patch, HIVE-2693-14.patch, HIVE-2693-15.patch, 
 HIVE-2693-16.patch, HIVE-2693-17.patch, HIVE-2693-1.patch.txt, 
 HIVE-2693-all.patch, HIVE-2693.D7683.1.patch, HIVE-2693-fix.patch, 
 HIVE-2693.patch, HIVE-2693-take3.patch, HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2013-01-04 Thread Gunther Hagleitner (JIRA)

[
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13544227#comment-13544227
]

Gunther Hagleitner commented on HIVE-2693:
--

I thought I had added a response on the ambiguous UDF issue on the review, but
it seems I never submitted that. It's there now.

I think it would be rather cumbersome to remove that piece from the patch. We
would have to remove all the conversion code from the file (including any
implicit conversion of numbers to decimal) and then rewrite all the tests to
work around this. Just to re-introduce this in a new jira and change the tests
back to what they were.

It seems Mark you have an idea of re-writing this piece altogether using
different cost for conversions. I think that's more intrusive (the current code
just adds a case for multiple choices) and could be handled in another jira.

Add DECIMAL data type
-

Key: HIVE-2693
URL: https://issues.apache.org/jira/browse/HIVE-2693
Project: Hive
Issue Type: New Feature
Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch,
HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch,
HIVE-2693-13.patch, HIVE-2693-14.patch, HIVE-2693-15.patch,
HIVE-2693-16.patch, HIVE-2693-17.patch, HIVE-2693-1.patch.txt,
HIVE-2693-all.patch, HIVE-2693.D7683.1.patch, HIVE-2693-fix.patch,
HIVE-2693.patch, HIVE-2693-take3.patch, HIVE-2693-take4.patch

Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice
template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2013-01-03 Thread Gunther Hagleitner (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13543219#comment-13543219
 ] 

Gunther Hagleitner commented on HIVE-2693:
--

Thanks for the review! I am in the process of adding the requested tests and 
taking care of the other comments.

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch, 
 HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch, 
 HIVE-2693-13.patch, HIVE-2693-14.patch, HIVE-2693-15.patch, 
 HIVE-2693-16.patch, HIVE-2693-1.patch.txt, HIVE-2693-all.patch, 
 HIVE-2693.D7683.1.patch, HIVE-2693-fix.patch, HIVE-2693.patch, 
 HIVE-2693-take3.patch, HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2013-01-02 Thread Namit Jain (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13542080#comment-13542080
 ] 

Namit Jain commented on HIVE-2693:
--

Looks mostly good.

Most of the comments are minor - the only major ones are around lack of testing.


 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch, 
 HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch, 
 HIVE-2693-13.patch, HIVE-2693-14.patch, HIVE-2693-15.patch, 
 HIVE-2693-16.patch, HIVE-2693-1.patch.txt, HIVE-2693-all.patch, 
 HIVE-2693.D7683.1.patch, HIVE-2693-fix.patch, HIVE-2693.patch, 
 HIVE-2693-take3.patch, HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-12-28 Thread Gunther Hagleitner (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13540435#comment-13540435
 ] 

Gunther Hagleitner commented on HIVE-2693:
--

Sorry for the noise. Here's the right review: https://reviews.facebook.net/D7689

Tried to fix the fact that some new files show up as copied files in the 
review, but the end result seems right.

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch, 
 HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch, 
 HIVE-2693-13.patch, HIVE-2693-14.patch, HIVE-2693-15.patch, 
 HIVE-2693-16.patch, HIVE-2693-1.patch.txt, HIVE-2693-all.patch, 
 HIVE-2693.D7683.1.patch, HIVE-2693-fix.patch, HIVE-2693.patch, 
 HIVE-2693-take3.patch, HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-12-19 Thread Mark Grover (JIRA)

[
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536251#comment-13536251
]

Mark Grover commented on HIVE-2693:
---

Gunther, to answer your former question. Yes, the trick is to convert double(by
default)- String - BigDecimal in the casting code. I just verified with my
latest changes that cast(3.4 as decimal) gives the correct precise BigDecimal.
An added benefit of this is that where clause are now working. No quotes or
literals required. Let me tidy up a few loose ends and I will post the patch up.

About your latest patch, thanks for the testing more. I agree that we should
use compareTo over equals. There is another place where equals() is being used
in PrimitiveObjectInspectorUtils.java I will change that to compareTo() in my
next patch. Was the nasty bug you are referring to related to equals/compareTo
inconsistency?

Thanks again, patch coming soon.

Add DECIMAL data type
-

Key: HIVE-2693
URL: https://issues.apache.org/jira/browse/HIVE-2693
Project: Hive
Issue Type: New Feature
Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch,
HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch,
HIVE-2693-13.patch, HIVE-2693-14.patch, HIVE-2693-1.patch.txt,
HIVE-2693-all.patch, HIVE-2693-fix.patch, HIVE-2693.patch,
HIVE-2693-take3.patch, HIVE-2693-take4.patch

Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice
template for how to do this.

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-12-19 Thread Gunther Hagleitner (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536334#comment-13536334
 ] 

Gunther Hagleitner commented on HIVE-2693:
--

The nasty bug was that the deserialization code didn't read the final byte. 
Which works fine if you serialize only one column but generates incorrect 
results in the general case.

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch, 
 HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch, 
 HIVE-2693-13.patch, HIVE-2693-14.patch, HIVE-2693-1.patch.txt, 
 HIVE-2693-all.patch, HIVE-2693-fix.patch, HIVE-2693.patch, 
 HIVE-2693-take3.patch, HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-12-19 Thread Mark Grover (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536369#comment-13536369
 ] 

Mark Grover commented on HIVE-2693:
---

New review at https://reviews.facebook.net/D7491


 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch, 
 HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch, 
 HIVE-2693-13.patch, HIVE-2693-14.patch, HIVE-2693-15.patch, 
 HIVE-2693-1.patch.txt, HIVE-2693-all.patch, HIVE-2693-fix.patch, 
 HIVE-2693.patch, HIVE-2693-take3.patch, HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-12-19 Thread Gunther Hagleitner (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536389#comment-13536389
 ] 

Gunther Hagleitner commented on HIVE-2693:
--

Mark,

If the conversion code goes through double - String - BigDecimal, I think 
it's basically rounding the result of the operation. You can achieve the same 
by setting the MathContext in the BigDecimal. That might seem better, but still 
causes issues. For instance: 
BigDecimal(Double.parseDouble(String.valueOf(0.9)) = 1.0 and 
BigDecimal(0.9) = 0.9.

Maybe documenting that we're rounding to a specific precision is the way 
forward, but I still think that not implicitly casting + error is the safer 
choice.

Thoughts?

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch, 
 HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch, 
 HIVE-2693-13.patch, HIVE-2693-14.patch, HIVE-2693-15.patch, 
 HIVE-2693-1.patch.txt, HIVE-2693-all.patch, HIVE-2693-fix.patch, 
 HIVE-2693.patch, HIVE-2693-take3.patch, HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-12-19 Thread Mark Grover (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536632#comment-13536632
 ] 

Mark Grover commented on HIVE-2693:
---

Gunther, I see where you are coming from. However, in my opinion, a closer 
analogy is {{new BigDecimal(Double.valueOf(0.9).toString())}} 
which is what's being done instead of {{new 
BigDecimal(Double.valueOf(0.9))}} which was being done 
previously.

I am inclined towards keeping the present behavior and documenting it. However, 
if you feel strongly about not implicitly casting, I am happy to reconsider.

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch, 
 HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch, 
 HIVE-2693-13.patch, HIVE-2693-14.patch, HIVE-2693-15.patch, 
 HIVE-2693-1.patch.txt, HIVE-2693-all.patch, HIVE-2693-fix.patch, 
 HIVE-2693.patch, HIVE-2693-take3.patch, HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-12-18 Thread Mark Grover (JIRA)

[
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13534942#comment-13534942
]

Mark Grover commented on HIVE-2693:
---

Thanks for contributing, [~hagleitn]! I am just about to take a closer look at
your latest patch but like we discussed offline, a better packed byte (with
radix 256) might be a better solution to a digit (i.e. a radix 10) based
serialization. Since we would need a sentinel character for terminating this
arbitrary long sequence of bytes (regardless of what radix we use), we have 2
options:
A. To use a smaller radix and reserve a particular byte as a terminator
B. To use a larger radix (like 256) and have certain escape character to escape
in the terminator if it appears in the content, similar to what we have for
string.

I would prefer the latter (option B). Any particular concerns from anybody
about that?

Add DECIMAL data type
-

Key: HIVE-2693
URL: https://issues.apache.org/jira/browse/HIVE-2693
Project: Hive
Issue Type: New Feature
Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch,
HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch,
HIVE-2693-1.patch.txt, HIVE-2693-all.patch, HIVE-2693-fix.patch,
HIVE-2693.patch, HIVE-2693-take3.patch, HIVE-2693-take4.patch

Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice
template for how to do this.

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-12-18 Thread Mark Grover (JIRA)

[
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13535044#comment-13535044
]

Mark Grover commented on HIVE-2693:
---

I added some test data and more tests to patch12 and found a few more
interesting issues:
1. I added new tests related to where clauses. The where clause doesn't seem to
be working as expected. I will take another look to see if I am doing something
wrong but that's my first impression anyways.
2. I added more test data where the decimal column has values like 3.14 and
3.140. This is an interesting case since we would like to maintain
compatibility with MySQL where possible. If I remember correctly (from a few
days ago when I tried it), MySQL considers 3.14 and 3.140 to be equivalent.
Therefore, they would be considered the same in equi-join, where clauses, etc.
This addition of a new data led me to see that order by is non-deterministic
when done over a decimal column. Again something, we should look more into.
FWIW, 3.14 are correctly being joined to 3.140 rows, so that's good!
3. I added some more test data with NULLs for the decimal column to make sure
those were being read and handled properly when the table was being loaded.
I will submit a new patch with these added tests and data shortly.

[~hagleitn] About patch 12, do we need to have a separate sign (0) for zero?
Would it not suffice for it to use the same sign as positive numbers? That
would make it consistent with other datatypes as well. Been having a little
busy morning but will review the rest of the patch shortly!

Add DECIMAL data type
-

Key: HIVE-2693
URL: https://issues.apache.org/jira/browse/HIVE-2693
Project: Hive
Issue Type: New Feature
Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch,
HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch,
HIVE-2693-1.patch.txt, HIVE-2693-all.patch, HIVE-2693-fix.patch,
HIVE-2693.patch, HIVE-2693-take3.patch, HIVE-2693-take4.patch

Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice
template for how to do this.

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-12-18 Thread Mark Grover (JIRA)

[
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13535191#comment-13535191
]

Mark Grover commented on HIVE-2693:
---

Ok, took a better look at a patch 12. Looks good overall.
On second thought, radix 10 is good for now. I am not going to vouch for
unnecessary optimization unless there is evidence that we need to.

[~hagleitn]
1. I understand the intent of +/- 2 you are doing to the sign bit. Is that
really necessary? Sure, you can have a sign that is the same as terminator but
the terminator is not considered until you get to the variable length part.
What do you think?
2. Were you able to test out your serialization and deserialization code? We
should make sure it works the way we expect? Do you need some help with testing
that?

So, once we can fix the issues I mentioned in the previous comment (where
clauses, deterministic order by), we should be good to review and post.
Thanks again!

Add DECIMAL data type
-

Key: HIVE-2693
URL: https://issues.apache.org/jira/browse/HIVE-2693
Project: Hive
Issue Type: New Feature
Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch,
HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch,
HIVE-2693-13.patch, HIVE-2693-1.patch.txt, HIVE-2693-all.patch,
HIVE-2693-fix.patch, HIVE-2693.patch, HIVE-2693-take3.patch,
HIVE-2693-take4.patch

Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice
template for how to do this.

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-12-18 Thread Gunther Hagleitner (JIRA)

[
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13535252#comment-13535252
]

Gunther Hagleitner commented on HIVE-2693:
--

Mark, thanks for the additional tests, I'll take a closer look this afternoon.

To answer the questions:

1: I introduced +/-2 when I was at a point in the debugging stage where
paranoia took over. I can remove that, it'll make the code more readable.
2: It is only implicitly tested in all the queries that use a reduce stage. I
agree that a test of just that code would be good. Is there a place in the
current unit tests that already does that/that I could use as a model?

Sign bit: I introduced a value for zero to avoid a factor of negative
infinity. If you lump 0 into either the positive or negative bucket it would
become the number that has an infinite number of zeros before the first
non-zero digit (after the decimal point). MIN_INT might have been an option,
but it seems cleaner to just make the sign have three states (-1,0,1).
BigDecimal class in Java itself for instance more or less randomly defines
precision of 0 (i.e.: number of unscaled digits) as 1.

Non-deterministic order: 3.14 and 3.140 are indeed equal. Their representation
should be exactly the same (1,1,314). Given that, I'm not sure how to
enforce a deterministic order or even what that would be. Are you suggesting
3.14 should always appear before 3.140?

I am worried about your comments about the where clause. I'll take a look at
the tests. But you say it's not working right?

Add DECIMAL data type
-

Key: HIVE-2693
URL: https://issues.apache.org/jira/browse/HIVE-2693
Project: Hive
Issue Type: New Feature
Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch,
HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch,
HIVE-2693-13.patch, HIVE-2693-1.patch.txt, HIVE-2693-all.patch,
HIVE-2693-fix.patch, HIVE-2693.patch, HIVE-2693-take3.patch,
HIVE-2693-take4.patch

Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice
template for how to do this.

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-12-18 Thread Mark Grover (JIRA)

[
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13535407#comment-13535407
]

Mark Grover commented on HIVE-2693:
---

1. Yeah, let's take the +/- 2 out please.
2. I am not aware of any such unit tests at the top of my head. I can try to
poke around the code and see if I find something, will post if I find
something. It probably wouldn't be until tomorrow.

I didn't realize that about the sign bit. Yeah, (-1,0,1) is good then.

Non-deterministic order: You are right they that are equal (and they should
be). However, if you diff patch12 patch13, you will find the order in which
order by displayed 1.0 and 1 got switched. The only thing that changed was me
adding some data. All I would expect is for the order to remain consistent and
deterministic which doesn't seem to be the case presently.

One thing that did stand out was in serialize()
{code}
BigDecimal dec = boi.getPrimitiveJavaObject(o).stripTrailingZeros();
{code}
stripTrailingZeros() seems interesting. I am just handwaving right now, need to
look more before I can assert further but could this be a part of the problem?

Yeah, where clause is not working. The tests didn't give the expected output.
Consequently, I tested using the Hive CLI (which I built after applying the
patch) and it doesn't work on that either. You're welcome to take a look, I
will try to find some time tonight or tomorrow morning to look into this as
well.
These two problems might be related but look at this:
{code}
hive select cast(3.14 as decimal) from decimal_3 limit 1;
3.140124344978758017532527446746826171875
{code}
That doesn't look right to me:-)

Add DECIMAL data type
-

Key: HIVE-2693
URL: https://issues.apache.org/jira/browse/HIVE-2693
Project: Hive
Issue Type: New Feature
Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch,
HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch,
HIVE-2693-13.patch, HIVE-2693-1.patch.txt, HIVE-2693-all.patch,
HIVE-2693-fix.patch, HIVE-2693.patch, HIVE-2693-take3.patch,
HIVE-2693-take4.patch

Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice
template for how to do this.

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-12-18 Thread Gunther Hagleitner (JIRA)

[
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13535453#comment-13535453
]

Gunther Hagleitner commented on HIVE-2693:
--

Ordering: The stripTrailingZeros() does not change the value of the BigDecimal
and therefore doesn't change the serialized output. I don't think this is the
culprit. The order of the output depends on what hadoop does in the
shuffle/sort phase. My guess is that since you changed the input the results
that mapred came up with where different. Hadoop doesn't guarantee stable sort
afaik.

Where clause: Yes, it's one and the same problem. We end up comparing 3.14 with
3.14...1243... Still debugging but it seems we're going through double on
the way to BigDecimal which changes the number.

Add DECIMAL data type
-

Key: HIVE-2693
URL: https://issues.apache.org/jira/browse/HIVE-2693
Project: Hive
Issue Type: New Feature
Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch,
HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch,
HIVE-2693-13.patch, HIVE-2693-1.patch.txt, HIVE-2693-all.patch,
HIVE-2693-fix.patch, HIVE-2693.patch, HIVE-2693-take3.patch,
HIVE-2693-take4.patch

Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice
template for how to do this.

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-12-18 Thread Gunther Hagleitner (JIRA)

[
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13535519#comment-13535519
]

Gunther Hagleitner commented on HIVE-2693:
--

Ok. I think I've convinced myself that the cast statement above is executed
correctly, but the result is definitely not very intuitive.

When you say: cast (3.14 as decimal) you ask hive to convert the double 3.14
to a bigdecimal in unlimited context, i.e. no rounding. Thus 3.141234... is
correct. If you ask cast('3.14' as decimal) you get 3.14. Which is what you
want. Rounding would be another option.

There was a similar issue with big integer: HIVE-2509. They 'solved' the
problem by at least having a shorthand for big int literals (i.e.: 0L).
BigDecimal should at the very least allow something like that too. E.g.: 3.14D
or 3.14BD.

So, should we just introduce a literal shorthand and leave the behavior as is?

Add DECIMAL data type
-

Key: HIVE-2693
URL: https://issues.apache.org/jira/browse/HIVE-2693
Project: Hive
Issue Type: New Feature
Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch,
HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch,
HIVE-2693-13.patch, HIVE-2693-1.patch.txt, HIVE-2693-all.patch,
HIVE-2693-fix.patch, HIVE-2693.patch, HIVE-2693-take3.patch,
HIVE-2693-take4.patch

Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice
template for how to do this.

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-12-18 Thread Gunther Hagleitner (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13535523#comment-13535523
 ] 

Gunther Hagleitner commented on HIVE-2693:
--

One more thing. It would be great to stop auto-converting double to decimal. 
WHERE decimal_column = 3.14 should preferably fail in the semantic analysis, 
so that any user would know that they have to cast from string or use a short 
hand.

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch, 
 HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch, 
 HIVE-2693-13.patch, HIVE-2693-1.patch.txt, HIVE-2693-all.patch, 
 HIVE-2693-fix.patch, HIVE-2693.patch, HIVE-2693-take3.patch, 
 HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-12-18 Thread Mark Grover (JIRA)

[
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13535526#comment-13535526
]

Mark Grover commented on HIVE-2693:
---

Fair enough. I would say let's not worry about the literal in this JIRA. We can
create another JIRA and deal with it separately. I personally don't think it's
a big deal because the 3.14 had it been read from a file in HDFS (when doing
say 'LOCAL DATA IN PATH'), would have been correctly interpreted as a
BigDecimal.

Good point about the hadoop shuffle/sort not being stable, I was hoping it was
but I can see why it's not.

So, that leaves us with a hurdle and a half:
1: where clause
0.5: rigorous testing of serialization and deserialization code for Decimal
type.

Add DECIMAL data type
-

Key: HIVE-2693
URL: https://issues.apache.org/jira/browse/HIVE-2693
Project: Hive
Issue Type: New Feature
Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch,
HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch,
HIVE-2693-13.patch, HIVE-2693-1.patch.txt, HIVE-2693-all.patch,
HIVE-2693-fix.patch, HIVE-2693.patch, HIVE-2693-take3.patch,
HIVE-2693-take4.patch

Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice
template for how to do this.

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-12-18 Thread Mark Grover (JIRA)

[
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13535532#comment-13535532
]

Mark Grover commented on HIVE-2693:
---

That actually explains why where clauses aren't working because we are
comparing BigDecimal with Double.
I tried the same where clause but with a string and that worked:
{code}
SELECT * FROM DECIMAL_3 WHERE key='3.14';
3.143
3.143
3.143
3.140 4
{code}

So, here is the million dollar question. Our users (understandably so) are
going to forget about the quotes. Even if we introduce a new literal as a part
of this JIRA, they are going to forget about the literal. I can think of two
options:
1. Somehow promote the double 3.14 to be a BigDecimal 3.14 (maintaining
precision). Maybe via a double-string-BigDecimal?
2. Throw an error like you suggested.

What do you think [~hagleitn]?

Add DECIMAL data type
-

Key: HIVE-2693
URL: https://issues.apache.org/jira/browse/HIVE-2693
Project: Hive
Issue Type: New Feature
Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch,
HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch,
HIVE-2693-13.patch, HIVE-2693-1.patch.txt, HIVE-2693-all.patch,
HIVE-2693-fix.patch, HIVE-2693.patch, HIVE-2693-take3.patch,
HIVE-2693-take4.patch

Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice
template for how to do this.

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-12-18 Thread Gunther Hagleitner (JIRA)

[
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13535548#comment-13535548
]

Gunther Hagleitner commented on HIVE-2693:
--

Solution Number 1 would be nice, but I don't think we'll be able to do that.
Any time we convert to double in an intermediate step we lose precision - and I
don't think it can be recovered by using string or anything else afterwards.
That seems to be just in the nature of floating point operations. We could
round to fewer digits in the conversion for instance, but that will probably
just cause issues elsewhere.

Not converting to double also seems a bad choice. That would change some very
fundamental behavior in hive. All of a sudden select 3.14 * 2 ... would
return BigDecimal instead of double. Not good.

So, I am in favor of throwing an error. At least we'll fail early and don't
send users on a wild goose chase. Although it's not ideal either: The where
clause issue wouldn't have happened for instance. But we probably don't want to
bar explicit casts (cast (3.14 as decimal)) even though they probably won't
do what anyone thinks they do. Or do we want to disallow them?

Add DECIMAL data type
-

Key: HIVE-2693
URL: https://issues.apache.org/jira/browse/HIVE-2693
Project: Hive
Issue Type: New Feature
Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch,
HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch,
HIVE-2693-13.patch, HIVE-2693-1.patch.txt, HIVE-2693-all.patch,
HIVE-2693-fix.patch, HIVE-2693.patch, HIVE-2693-take3.patch,
HIVE-2693-take4.patch

Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice
template for how to do this.

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-12-17 Thread Gunther Hagleitner (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13534221#comment-13534221
 ] 

Gunther Hagleitner commented on HIVE-2693:
--

It should be possible to serialize a big decimal as 
signscalelengthint-digits. That way the natural ordering should be 
preserved at the binary level.

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch, 
 HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-1.patch.txt, 
 HIVE-2693-all.patch, HIVE-2693-fix.patch, HIVE-2693.patch, 
 HIVE-2693-take3.patch, HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-12-17 Thread Gunther Hagleitner (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13534241#comment-13534241
 ] 

Gunther Hagleitner commented on HIVE-2693:
--

Actually, that's not going to work because of the length. But for this case I 
don't think we need to encode the length. Hive key will put the stuff in a 
byteswritable and skip the length header on comparison.

Also sign has to be 1 for positive 0 for negative (to preserve order) and 
scale has to be negated for negative numbers.


 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch, 
 HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-1.patch.txt, 
 HIVE-2693-all.patch, HIVE-2693-fix.patch, HIVE-2693.patch, 
 HIVE-2693-take3.patch, HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-12-17 Thread Gunther Hagleitner (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13534290#comment-13534290
 ] 

Gunther Hagleitner commented on HIVE-2693:
--

Wrong again, but getting closer. The length of strings/byte arrays are encoded 
with trailing \0 in BinarySortableSerDe. So the encoding should be

sign bytesale intstring of digits

sign byte: 1 for = 0, 0 for 0
scale: scale integer if sign byte 1, -scale integer otherwise
string of digits: Zero terminated string of digits

I'm trying this out right now. 


 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch, 
 HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-1.patch.txt, 
 HIVE-2693-all.patch, HIVE-2693-fix.patch, HIVE-2693.patch, 
 HIVE-2693-take3.patch, HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-12-17 Thread Mark Grover (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13534319#comment-13534319
 ] 

Mark Grover commented on HIVE-2693:
---

Thanks for your comments, [~hagleitn]. Yeah, the right thing moving forward 
would be to update BinarySortableSerDe to support BigDecimal. When I thinking 
of the best way to serialize BigDecimal, the sign and scale part were easy but 
I wasn't able to come up with a space efficient way to store and arbitrary 
number of digits so they are in-order byte sortable. Correct me if I am wrong 
but seems like you are suggesting 1 byte per digit which would work (if the 
lengths are equal) but can be dangerous since we are exploding an arbitrarily 
long integer. Having said that, given Hive's historical philosophy of ignoring 
malicious intent users, I am ok with moving forward with that approach.

On a related note, let's talk about the scale.
13267 has a scale of 0 and digits 13267
132.67 has a scale of 2 and digits 13267
132670 has a scale of -1 and digits 13267

So, it seems like the lower scale always means bigger number for positive 
numbers, so shouldn't we do
{code}
scale: -scale for positive numbers (sign byte 1) and scale for negative 
numbers (sign byte 0)
{code}
instead of 
{code}
scale: scale integer if sign byte 1, -scale integer otherwise
{code}
which you suggested in your previous comment. I am basically asking to flip it 
around since negative numbers have sign byte 0 and positive have sign byte.
BTW, feel free to contact me offline if you want to bounce around some ideas!

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch, 
 HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-1.patch.txt, 
 HIVE-2693-all.patch, HIVE-2693-fix.patch, HIVE-2693.patch, 
 HIVE-2693-take3.patch, HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-12-16 Thread Mark Grover (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13533646#comment-13533646
 ] 

Mark Grover commented on HIVE-2693:
---

I was playing around with this and noticed that group by on Decimal column 
doesn't work because BinarySortableSerDe doesn't support DECIMAL type. It's 
questionable if it would ever support it since BigDecimal is arbitrarily long 
and BinarySortableSerDe dictates data to be serialized so that the value can be 
compared byte by byte with the same order. That would have been ok but it seems 
like {{getReduceKeyTableDesc}} in {{PlanUtils.java}} is hardcoded to use 
BinarySortableSerDe. I will have to poke around some more to see how Group By 
can be made to work with Decimal type (backed by BigDecimal).

In any case, the last patch didn't apply cleanly on trunk, so I fixed some 
merge conflicts and am attaching a new patch (#11) which is a refresh of patch 
10 that applies cleanly on trunk as of today.

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch, 
 HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-1.patch.txt, 
 HIVE-2693-all.patch, HIVE-2693-fix.patch, HIVE-2693.patch, 
 HIVE-2693-take3.patch, HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-11-15 Thread Vikram Dixit K (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13498098#comment-13498098
 ] 

Vikram Dixit K commented on HIVE-2693:
--

Hi Carl, I have already done some work with regard to review comments. I am 
planning to continue to work on this. Let me know how you want to proceed. 
Thanks Vikram.

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch, 
 HIVE-2693-1.patch.txt, HIVE-2693-all.patch, HIVE-2693-fix.patch, 
 HIVE-2693.patch, HIVE-2693-take3.patch, HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-11-15 Thread Ashutosh Chauhan (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13498110#comment-13498110
 ] 

Ashutosh Chauhan commented on HIVE-2693:


Vikram,
Current patch also lacks test coverage. Please add following tests:
a) Test which loads data in the table having decimal column via reading from 
text file.
b) Test which does group-by / join on decimal column.

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch, 
 HIVE-2693-1.patch.txt, HIVE-2693-all.patch, HIVE-2693-fix.patch, 
 HIVE-2693.patch, HIVE-2693-take3.patch, HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-11-14 Thread Ashutosh Chauhan (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13497546#comment-13497546
 ] 

Ashutosh Chauhan commented on HIVE-2693:


Vikram, I left some comments on RB.

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Josh Wills
 Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch, 
 HIVE-2693-all.patch, HIVE-2693-fix.patch, HIVE-2693.patch, 
 HIVE-2693-take3.patch, HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-11-13 Thread Vikram Dixit K (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13496836#comment-13496836
 ] 

Vikram Dixit K commented on HIVE-2693:
--

Review board url:
https://reviews.apache.org/r/8049/

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Josh Wills
 Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch, 
 HIVE-2693-all.patch, HIVE-2693-fix.patch, HIVE-2693.patch, 
 HIVE-2693-take3.patch, HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-11-12 Thread Ashutosh Chauhan (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13495865#comment-13495865
 ] 

Ashutosh Chauhan commented on HIVE-2693:


@Vikram, Patch still doesn't apply. Can you update the patch. Also, note that 
you need to install thrift-0.9.0 now after we updated from thrift-0.7. Also, 
can you please create a phabricator or review board review request?

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: 2693_7.patch, 2693_fix_all_tests1.patch, 
 HIVE-2693-all.patch, HIVE-2693-fix.patch, HIVE-2693.patch, 
 HIVE-2693-take3.patch, HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-11-04 Thread Mark Grover (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13490479#comment-13490479
 ] 

Mark Grover commented on HIVE-2693:
---

Can someone update the review please?

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: 2693_fix_all_tests1.patch, HIVE-2693-all.patch, 
 HIVE-2693-fix.patch, HIVE-2693.patch, HIVE-2693-take3.patch, 
 HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-11-04 Thread Vikram Dixit K (JIRA)

[
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13490481#comment-13490481
]

Vikram Dixit K commented on HIVE-2693:
--

With this latest patch, I see all the failed tests passing. I have just started
a run of the entire test suite just to be sure I did not break anything. There
is one issue however. The mysql round function behavior does not seem to be
reproducible via the bigdecimal round function. The round function takes a math
context object whose scale can only be a positive integer and this results in
some inconsistent results. The currently existing double-based method is
producing the expected results in accordance with the mysql response. I am not
sure which way the coin should drop in this case. Bigdecimal's primary use case
is to have higher precision but, the round operation is not entirely compatible
with the current mysql standard. As an err on the mysql side strategy, I have
taken the approach of sticking to the current double based round api. Based on
consensus in the hive community, I can add back the capability to go the big
decimal route or if anyone knows how to tune the bigdecimal to round the way
mysql does it, I would be happy to correct/use that.

Apart from this change, I have also had to refactor the code in
FunctionRegistry to be able to choose a 'higher denominator' api over a lower
one as alluded to in the comment above. This was required for the logarithm udf
function. I have refactored things in such a way that new apis can easily be
added that uses the already existing hierarchy without the user having to get
caught in these internals.

Add DECIMAL data type
-

Key: HIVE-2693
URL: https://issues.apache.org/jira/browse/HIVE-2693
Project: Hive
Issue Type: New Feature
Components: Query Processor, Types
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
Attachments: 2693_fix_all_tests1.patch, HIVE-2693-all.patch,
HIVE-2693-fix.patch, HIVE-2693.patch, HIVE-2693-take3.patch,
HIVE-2693-take4.patch

Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice
template for how to do this.

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-11-02 Thread Vikram Dixit K (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13489644#comment-13489644
 ] 

Vikram Dixit K commented on HIVE-2693:
--

Hi Josh,

Can you tell me why in the UDFRound.java, the math context is being initialized 
with the formula bd.precision() - (bd.scale() - i.get())? This is causing some 
issues with respect to calls such as round(5, -6) etc. The reason is the 
bigdecimal constructor sets precision by default to the number of digits of the 
integer passed into it. In this case for 5, it is set to 5 and the scale by 
default is 0. This implies that for this case, the formula above results in a 
negative number that the mathcontext constructor cannot take. I am not sure 
what the behavior should be?

Thanks
Vikram.

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: HIVE-2693-all.patch, HIVE-2693-fix.patch, 
 HIVE-2693.patch, HIVE-2693-take3.patch, HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-11-02 Thread Josh Wills (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13489650#comment-13489650
 ] 

Josh Wills commented on HIVE-2693:
--

For the life of me, I do not remember-- if there's something that makes sense, 
by all means, change it up.

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: HIVE-2693-all.patch, HIVE-2693-fix.patch, 
 HIVE-2693.patch, HIVE-2693-take3.patch, HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-11-01 Thread Vikram Dixit K (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13488946#comment-13488946
 ] 

Vikram Dixit K commented on HIVE-2693:
--

Ok. I will look into this error.

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: HIVE-2693-all.patch, HIVE-2693-fix.patch, 
 HIVE-2693.patch, HIVE-2693-take3.patch, HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-10-31 Thread Vikram Dixit K (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13488240#comment-13488240
 ] 

Vikram Dixit K commented on HIVE-2693:
--

Hi Josh, I found the issue. The getMethodInternal api within the 
FunctionRegistry class iterates over all the apis within a class say UDFRound 
and finds the api with the lowest conversion cost to use for the subsequent 
operations. Since the addition of the big decimal evaluate api in the UDFRound 
class, there is now a collision between the Double and Big decimal evaluate 
apis since both have the same conversion cost. One way I think we can fix this 
is to add an enum of conversion precedence in the functionregistry class. The 
precedence order can be:

BigDecimal  Double  Float  Long  Int  boolean. If we see a collision, we 
can resolve it in favor of the broadest (most accepting) object which in this 
case is BigDecimal. Let me know what you think and also if you have time to 
work on it.

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: HIVE-2693-all.patch, HIVE-2693-fix.patch, 
 HIVE-2693.patch, HIVE-2693-take3.patch, HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-10-31 Thread Josh Wills (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13488283#comment-13488283
 ] 

Josh Wills commented on HIVE-2693:
--

Hey Vikram-- I'm a bit swamped at the moment and we're pushing the limits of my 
knowledge of Hive internals. Would you or someone else like to take a crack at 
it?

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: HIVE-2693-all.patch, HIVE-2693-fix.patch, 
 HIVE-2693.patch, HIVE-2693-take3.patch, HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-10-30 Thread Josh Wills (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13487004#comment-13487004
 ] 

Josh Wills commented on HIVE-2693:
--

Comments/feedback?

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: HIVE-2693-all.patch, HIVE-2693-fix.patch, 
 HIVE-2693.patch, HIVE-2693-take3.patch, HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-10-30 Thread Vikram Dixit K (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13487108#comment-13487108
 ] 

Vikram Dixit K commented on HIVE-2693:
--

Hi Josh, I have been able to successfully build and run the tests contained in 
this patch. However, I see a few failures (4 in total) related to some UDF 
tests - udf_div.q, udf_round.q, ql_rewrite_gbtoidx.q, udf7.q.

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: HIVE-2693-all.patch, HIVE-2693-fix.patch, 
 HIVE-2693.patch, HIVE-2693-take3.patch, HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-10-30 Thread Vikram Dixit K (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13487278#comment-13487278
 ] 

Vikram Dixit K commented on HIVE-2693:
--

org.apache.hadoop.hive.ql.parse.SemanticException: Line 3:36 Wrong arguments 
'TOK_NULL': Ambiguous method for class org.apache.hadoop.hive.ql.udf.UDFRound 
with (int, void). Possible choices: _FUNC_(double)  _FUNC_(double, int)  
_FUNC_(decimal, int)
  at 
org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.process(TypeCheckProcFactory.java:914)

Similar errors in the others.

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: HIVE-2693-all.patch, HIVE-2693-fix.patch, 
 HIVE-2693.patch, HIVE-2693-take3.patch, HIVE-2693-take4.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-10-26 Thread Vikram Dixit K (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13485105#comment-13485105
 ] 

Vikram Dixit K commented on HIVE-2693:
--

I tried the latest patch and it builds successfully. However, I cannot run any 
tests because there seems to be something broken in the parser. Posting the 
error below:

[junit] FAILED: ParseException line 1:23 cannot recognize input near 
''/grid/0/hive/BigD/data/files/kv1.txt'' '(' ''2008-04-08''
[junit] 
[junit] java.lang.Exception: load command: LOAD DATA LOCAL INPATH 
'/grid/0/hive/BigD/data/files/kv1.txt' OVERWRITE INTO TABLE srcpart PARTITION 
(ds='2008-04-08',hr='11') failed with exit code= 4
[junit] at 
org.apache.hadoop.hive.ql.QTestUtil.runLoadCmd(QTestUtil.java:468)
[junit] at 
org.apache.hadoop.hive.ql.QTestUtil.createSources(QTestUtil.java:512)
[junit] at 
org.apache.hadoop.hive.cli.TestCliDriver.clinit(TestCliDriver.java:55)


 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: HIVE-2693-all.patch, HIVE-2693-fix.patch, 
 HIVE-2693.patch, HIVE-2693-take3.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-10-26 Thread Josh Wills (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13485144#comment-13485144
 ] 

Josh Wills commented on HIVE-2693:
--

Hrm, I'm not seeing that. I'm just doing 'ant package' followed by 'ant test'-- 
I assume you're doing the same?

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: HIVE-2693-all.patch, HIVE-2693-fix.patch, 
 HIVE-2693.patch, HIVE-2693-take3.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-10-26 Thread Vikram Dixit K (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13485156#comment-13485156
 ] 

Vikram Dixit K commented on HIVE-2693:
--

Yes. For the test however, I am running ant test -Dtestcase=TestCliDriver 
-Dqfile=decimal_1.q. However, I see this error even when I run ant test 
-Dtestcase=TestCliDriver. I ran the compilation step after cleaning up my m2 
and ivy caches.

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: HIVE-2693-all.patch, HIVE-2693-fix.patch, 
 HIVE-2693.patch, HIVE-2693-take3.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-10-25 Thread Vikram Dixit K (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13484567#comment-13484567
 ] 

Vikram Dixit K commented on HIVE-2693:
--

Hi,

I was trying out this patch and found that it is missing some of the classes. 
Specifically BigDecimalWritable, BigDecimalObjectInspector 
(org.apache.hadoop.hive.serde2.objectinspector.primitive.BigDecimalObjectInspector),
 WritableBigDecimalObjectInspector 
(org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableBigDecimalObjectInspector),
 JavaBigDecimalObjectInspector are missing. Would it be possible to update this 
patch. Is anyone working on this? It seems like a local git commit is the 
likely cause for this as Carl pointed out in the review.

Thanks
Vikram.

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: HIVE-2693.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-10-25 Thread Vikram Dixit K (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13484639#comment-13484639
 ] 

Vikram Dixit K commented on HIVE-2693:
--

Hey Josh, Thank you for your response. I still get errors when I apply this 
patch. The class - BigDecimalWritable is missing in the repository. The changes 
specified for this class does not apply.

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: HIVE-2693-all.patch, HIVE-2693.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-10-25 Thread Vikram Dixit K (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13484687#comment-13484687
 ] 

Vikram Dixit K commented on HIVE-2693:
--

Patch applied fine but when I try to compile the change, I see that the 
WritableBigDecimalObjectInspector class is missing.

 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Reporter: Carl Steinbach
Assignee: Prasad Mujumdar
 Attachments: HIVE-2693-all.patch, HIVE-2693-fix.patch, HIVE-2693.patch


 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2693) Add DECIMAL data type

2012-06-11 Thread Carl Steinbach (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13293088#comment-13293088
 ] 

Carl Steinbach commented on HIVE-2693:
--

Review request from a while ago: https://reviews.facebook.net/D1221


 Add DECIMAL data type
 -

 Key: HIVE-2693
 URL: https://issues.apache.org/jira/browse/HIVE-2693
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor, Types
Reporter: Carl Steinbach
Assignee: Josh Wills

 Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice 
 template for how to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

66 matches

Mail list logo