[GitHub] incubator-hivemall issue #46: [HIVEMALL-72] Fix corner-case rescale UDF beha...

[myui]

Github user myui commented on the issue:

https://github.com/apache/incubator-hivemall/pull/46
  
@wangyum Are you agree with the following new behavior of 
[rescale(value,min,max)](https://en.wikipedia.org/wiki/Feature_scaling#Rescaling)?

```
-- new
select rescale(4.2,1.0,3.0),rescale(-0.3, 1.0, 3.0);
> 1.0 0.0

-- old
select rescale(4.2,1.0,3.0),rescale(-0.3, 1.0, 3.0);
> 1.599 -0.65
```

Throwing an exception if value is out of range but exact min/max is 
sometimes unknown beforehand.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hivemall issue #46: [HIVEMALL-72] Fix corner-case rescale UDF beha...

[myui]

Github user myui commented on the issue:

https://github.com/apache/incubator-hivemall/pull/46
  
Oops.. the latter one is expected.

```sql
select rescale(4.2,1.0,3.0),rescale(-0.3, 1.0, 3.0);
> 3.0 1.0

select rescale(4.2,1.0,3.0),rescale(-0.3, 1.0, 3.0);
> 1.0 0.0
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hivemall pull request #46: [HIVEMALL-72] Fix rescale UDF behavior ...

[myui]

GitHub user myui opened a pull request:

https://github.com/apache/incubator-hivemall/pull/46

[HIVEMALL-72] Fix rescale UDF behavior to return range [0.0,1.0]

## What changes were proposed in this pull request?

Fix rescale UDF behavior to return range `[0.0,1.0]`.

## What type of PR is it?

Bug Fix

## What is the Jira issue?

https://issues.apache.org/jira/browse/HIVEMALL-72

## How was this patch tested?

manual tests

```sql
-- Before
select rescale(4.2,1.0,3.0),rescale(-0.3, 1.0, 3.0);
> 1.599 -0.65

-- After
select rescale(4.2,1.0,3.0),rescale(-0.3, 1.0, 3.0);
> 3.0 1.0
```


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/myui/incubator-hivemall HIVEMALL-72

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hivemall/pull/46.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #46


commit 6510edd3930d061272e702b5a6b7fc97b7ce3e52
Author: myui 
Date:   2017-02-16T06:39:36Z

[HIVEMALL-72] Fix rescale UDF behavior to return range [0.0,1.0]




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Closed] (HIVEMALL-71) Handle null values in RescaleUDF.java

[Makoto Yui (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVEMALL-71?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Makoto Yui closed HIVEMALL-71.
--
Resolution: Fixed
  Assignee: Makoto Yui

> Handle null values in RescaleUDF.java
> -
>
> Key: HIVEMALL-71
> URL: https://issues.apache.org/jira/browse/HIVEMALL-71
> Project: Hivemall
>  Issue Type: Improvement
>Reporter: Yuming Wang
>Assignee: Makoto Yui
>
> Change:
> {code:java}
> public FloatWritable evaluate(final float value, final float min,
> final float max) {
> return val(min_max_normalization(value, min, max));
> }
> {code}
> to:
> {code:java}
> @Nullable
> public FloatWritable evaluate(final Float value, final Float min,
> final Float max) throws HiveException {
> if(value == null) {
> return null;
> }
> if(min == null) throw new HiveException("min should not be null");
> if(max == null) throw new HiveException("max should not be null");
> return val(min_max_normalization(value.floatValue(), 
> min.floatValue(), max. floatValue()));
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] incubator-hivemall issue #45: [HIVEMALL-71] Handle null values and add a uni...

[myui]

Github user myui commented on the issue:

https://github.com/apache/incubator-hivemall/pull/45
  
@wangyum Thanks. Merged with some modifications.

```sql
select rescale(v,min,max) from (
select cast(1.2 as float) as v, 1.0 as min, 10.0 as max
union all
select cast(null as float) as v, 1.0 as min, 10.0 as max
union all
select cast(1.2 as double) as v, 1.0 as min, 10.0 as max
union all
select cast(null as double) as v, 1.0 as min, 10.0 as max
union all
select cast(1.2 as double) as v, cast(5.0 as double) as min, cast(10.0 
as float) as max
) t;

0.02228
NULL
0.02228
NULL
-0.76
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hivemall issue #44: [HIVEMALL-65] Update define-all.spark and impo...

[myui]

Github user myui commented on the issue:

https://github.com/apache/incubator-hivemall/pull/44
  
@wangyum Merged. Thank you for your contribution!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hivemall pull request #44: [HIVEMALL-65] Update define-all.spark a...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-hivemall/pull/44


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---