[GitHub] incubator-hivemall issue #105: [WIP][HIVEMALL-24] Scalable field-aware facto...
Github user coveralls commented on the issue: https://github.com/apache/incubator-hivemall/pull/105 [](https://coveralls.io/builds/13065721) Coverage decreased (-0.6%) to 40.213% when pulling **1ecfe82f36a2d8dfb80aba109ac5d317ff2f727d on myui:HIVEMALL-24-2** into **7205de1e959f0d9b96ac756e415d8a8ada7e92af on apache:master**. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hivemall issue #105: [WIP][HIVEMALL-24] Scalable field-aware facto...
Github user coveralls commented on the issue: https://github.com/apache/incubator-hivemall/pull/105 [](https://coveralls.io/builds/13048972) Coverage decreased (-0.3%) to 40.563% when pulling **784ce76cae6acc7682f8c7b90550eae6aee9cb65 on myui:HIVEMALL-24-2** into **7205de1e959f0d9b96ac756e415d8a8ada7e92af on apache:master**. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hivemall issue #105: [WIP][HIVEMALL-24] Scalable field-aware facto...
Github user coveralls commented on the issue: https://github.com/apache/incubator-hivemall/pull/105 [](https://coveralls.io/builds/13048054) Coverage decreased (-0.3%) to 40.56% when pulling **54116c53f3ea0d2263162f84c5dbd8795ae2510b on myui:HIVEMALL-24-2** into **7205de1e959f0d9b96ac756e415d8a8ada7e92af on apache:master**. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hivemall issue #105: [WIP][HIVEMALL-24] Scalable field-aware facto...
Github user coveralls commented on the issue: https://github.com/apache/incubator-hivemall/pull/105 [](https://coveralls.io/builds/13013516) Coverage decreased (-0.6%) to 40.215% when pulling **dba8bf64e3a29ba4da373bfbec01a37439a2cdea on myui:HIVEMALL-24-2** into **7205de1e959f0d9b96ac756e415d8a8ada7e92af on apache:master**. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hivemall issue #105: [WIP][HIVEMALL-24] Scalable field-aware facto...
Github user coveralls commented on the issue: https://github.com/apache/incubator-hivemall/pull/105 [](https://coveralls.io/builds/12990161) Coverage decreased (-0.8%) to 40.02% when pulling **dc7e52549cfc42249f9706056d7a6cf962b573bb on myui:HIVEMALL-24-2** into **7205de1e959f0d9b96ac756e415d8a8ada7e92af on apache:master**. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hivemall issue #105: [WIP][HIVEMALL-24] Scalable field-aware facto...
Github user coveralls commented on the issue: https://github.com/apache/incubator-hivemall/pull/105 [](https://coveralls.io/builds/12983162) Coverage decreased (-0.4%) to 40.466% when pulling **521e2af9f94a5c936d146e47eb93d05859fc0e0e on myui:HIVEMALL-24-2** into **7205de1e959f0d9b96ac756e415d8a8ada7e92af on apache:master**. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hivemall issue #105: [WIP][HIVEMALL-24] Scalable field-aware facto...
Github user coveralls commented on the issue: https://github.com/apache/incubator-hivemall/pull/105 [](https://coveralls.io/builds/12968215) Coverage decreased (-0.4%) to 40.462% when pulling **7ced373691a8d5a95bb67a03afad80a5090ba743 on myui:HIVEMALL-24-2** into **7205de1e959f0d9b96ac756e415d8a8ada7e92af on apache:master**. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hivemall issue #105: [WIP][HIVEMALL-24] Scalable field-aware facto...
Github user coveralls commented on the issue: https://github.com/apache/incubator-hivemall/pull/105 [](https://coveralls.io/builds/12952251) Coverage decreased (-0.4%) to 40.44% when pulling **df03a79926f072de5bae4e65d10ba91d3eaf20c4 on myui:HIVEMALL-24-2** into **7205de1e959f0d9b96ac756e415d8a8ada7e92af on apache:master**. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hivemall issue #105: [WIP][HIVEMALL-24] Scalable field-aware facto...
Github user myui commented on the issue: https://github.com/apache/incubator-hivemall/pull/105 ``` 2017-08-23 06:57:27,685 [INFO] [TezChild] |fm.FieldAwareFactorizationMachineUDTF|: FFMStringFeatureMapModel [bytesAllocated=1.70 GiB, bytesUsed=N/A, numAllocatedW=799,594, numReusedW=8,773, numRemovedW=8,773, numAllocatedV=28,289,552, numReusedV=1,232,381, numRemovedV=1,232,382] 2017-08-23 06:57:27,690 [INFO] [TezChild] |fm.FactorizationMachineUDTF|: Wrote 1000334 records to a temporary file for iterative training: /mnt/yarn/usercache/hadoop/appcache/application_1503463978621_0003/container_1503463978621_0003_01_13/tmp/hivemall_fm2720664696266209509.sgmt (462.1 MiB) 2017-08-23 07:16:11,004 [ERROR] [TezChild] |tez.TezProcessor|: java.lang.OutOfMemoryError: Java heap space at hivemall.utils.collections.maps.Int2LongOpenHashTable.rehash(Int2LongOpenHashTable.java:319) at hivemall.utils.collections.maps.Int2LongOpenHashTable.ensureCapacity(Int2LongOpenHashTable.java:310) at hivemall.utils.collections.maps.Int2LongOpenHashTable.preAddEntry(Int2LongOpenHashTable.java:202) at hivemall.utils.collections.maps.Int2LongOpenHashTable.put(Int2LongOpenHashTable.java:146) at hivemall.fm.FFMStringFeatureMapModel.getV(FFMStringFeatureMapModel.java:139) at hivemall.fm.FieldAwareFactorizationMachineModel.predict(FieldAwareFactorizationMachineModel.java:105) at hivemall.fm.FieldAwareFactorizationMachineUDTF.trainTheta(FieldAwareFactorizationMachineUDTF.java:191) at hivemall.fm.FieldAwareFactorizationMachineUDTF.train(FieldAwareFactorizationMachineUDTF.java:183) at hivemall.fm.FactorizationMachineUDTF.runTrainingIteration(FactorizationMachineUDTF.java:636) at hivemall.fm.FactorizationMachineUDTF.close(FactorizationMachineUDTF.java:425) at hivemall.fm.FieldAwareFactorizationMachineUDTF.close(FieldAwareFactorizationMachineUDTF.java:255) at org.apache.hadoop.hive.ql.exec.UDTFOperator.closeOp(UDTFOperator.java:152) at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:697) at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:711) at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:711) at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:711) at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.close(MapRecordProcessor.java:464) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:199) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:168) at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370) at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73) at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61) at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37) at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) ``` `getV` should not initialize `V` in the second or later iterations. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hivemall issue #105: [WIP][HIVEMALL-24] Scalable field-aware facto...
Github user coveralls commented on the issue: https://github.com/apache/incubator-hivemall/pull/105 [](https://coveralls.io/builds/12950365) Coverage decreased (-0.7%) to 40.118% when pulling **91f296b0338a854cca255637e7322ed538bc70f7 on myui:HIVEMALL-24-2** into **7205de1e959f0d9b96ac756e415d8a8ada7e92af on apache:master**. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hivemall issue #105: [WIP][HIVEMALL-24] Scalable field-aware facto...
Github user myui commented on the issue: https://github.com/apache/incubator-hivemall/pull/105 memory consumption of FFM is estimated as follows: ``` ( 4 + 4*factors + 8+ 4+8) * fields * features (bytes) ~~ ~ ~~~~~ Wi V[k] adagrad ftrl ffm (4+4*4*8+4+8)*39*2^20 bytes = 5.88 GiB ``` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hivemall issue #105: [WIP][HIVEMALL-24] Scalable field-aware facto...
Github user myui commented on the issue: https://github.com/apache/incubator-hivemall/pull/105 ```sql SET mapred.max.split.size=6400; -- use more mappers to avoid OOM in mappers -- Hive on Tez SET tez.task.resource.memory.mb=3072; SET hive.tez.java.opts=-server -Xmx2560m -XX:+PrintGCDetails -XX:+UseNUMA -XX:+UseParallelGC -XX:+HeapDumpOnOutOfMemoryError; -- Hive on MapReuce SET mapreduce.map.memory.mb=3072; SET mapreduce.map.java.opts=-server -Xmx2560m -XX:+PrintGCDetails -XX:+UseNUMA -XX:+UseParallelGC -XX:+HeapDumpOnOutOfMemoryError; -- 46 mappers INSERT OVERWRITE TABLE ffm_model select train_ffm(features, label, "-c -iters 10 -factors 4 -feature_hashing 20") -- 2^20 = 1,048,576 features from train ; ``` ``` Caused by: java.lang.OutOfMemoryError: Java heap space at hivemall.utils.collections.maps.Int2LongOpenHashTable.rehash(Int2LongOpenHashTable.java:320) at hivemall.utils.collections.maps.Int2LongOpenHashTable.ensureCapacity(Int2LongOpenHashTable.java:310) at hivemall.utils.collections.maps.Int2LongOpenHashTable.preAddEntry(Int2LongOpenHashTable.java:202) at hivemall.utils.collections.maps.Int2LongOpenHashTable.put(Int2LongOpenHashTable.java:146) at hivemall.fm.FFMStringFeatureMapModel.getV(FFMStringFeatureMapModel.java:168) at hivemall.fm.FieldAwareFactorizationMachineModel.predict(FieldAwareFactorizationMachineModel.java:89) at hivemall.fm.FieldAwareFactorizationMachineUDTF.trainTheta(FieldAwareFactorizationMachineUDTF.java:194) at hivemall.fm.FieldAwareFactorizationMachineUDTF.train(FieldAwareFactorizationMachineUDTF.java:184) at hivemall.fm.FactorizationMachineUDTF.process(FactorizationMachineUDTF.java:285) at org.apache.hadoop.hive.ql.exec.UDTFOperator.process(UDTFOperator.java:116) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:878) at org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:878) at org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:130) at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:149) at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:489) at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:86) at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:70) at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:360) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:172) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:160) at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370) at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73) at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61) at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37) at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ``` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hivemall issue #105: [WIP][HIVEMALL-24] Scalable field-aware facto...
Github user myui commented on the issue: https://github.com/apache/incubator-hivemall/pull/105 ``` 2017-07-28 11:52:50,625 [ERROR] [TezChild] |tez.TezProcessor|: java.lang.OutOfMemoryError: Java heap space at hivemall.utils.collections.maps.Int2LongOpenHashTable.rehash(Int2LongOpenHashTable.java:320) at hivemall.utils.collections.maps.Int2LongOpenHashTable.ensureCapacity(Int2LongOpenHashTable.java:310) at hivemall.utils.collections.maps.Int2LongOpenHashTable.preAddEntry(Int2LongOpenHashTable.java:202) at hivemall.utils.collections.maps.Int2LongOpenHashTable.put(Int2LongOpenHashTable.java:146) at hivemall.fm.FFMStringFeatureMapModel.getV(FFMStringFeatureMapModel.java:168) at hivemall.fm.FieldAwareFactorizationMachineModel.predict(FieldAwareFactorizationMachineModel.java:89) at hivemall.fm.FieldAwareFactorizationMachineUDTF.trainTheta(FieldAwareFactorizationMachineUDTF.java:194) at hivemall.fm.FieldAwareFactorizationMachineUDTF.train(FieldAwareFactorizationMachineUDTF.java:184) at hivemall.fm.FactorizationMachineUDTF.process(FactorizationMachineUDTF.java:285) at org.apache.hadoop.hive.ql.exec.UDTFOperator.process(UDTFOperator.java:116) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:878) at org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:878) at org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:130) at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:149) at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:489) at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:86) at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:70) at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:360) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:172) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:160) at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370) at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73) at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61) at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37) at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ``` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hivemall issue #105: [WIP][HIVEMALL-24] Scalable field-aware facto...
Github user coveralls commented on the issue: https://github.com/apache/incubator-hivemall/pull/105 [](https://coveralls.io/builds/12592858) Coverage decreased (-0.9%) to 39.952% when pulling **3f24157b4ab1d6fa8c2a1a459bc1f62fff1a7f12 on myui:HIVEMALL-24-2** into **7205de1e959f0d9b96ac756e415d8a8ada7e92af on apache:master**. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hivemall issue #105: [WIP][HIVEMALL-24] Scalable field-aware facto...
Github user myui commented on the issue:
https://github.com/apache/incubator-hivemall/pull/105
```
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
Error while processing row
{"rowid":null,"features":["1:i1:1.791759469228055","2:i2:3.9889840465642745","3:i3:1.9459101490553132","4:i4:1.9459101490553132","5:i5:0.6931471805599453","6:i6:0","7:i7:4.219507705176107","8:i8:1.9459101490553132","9:i9:5.0238805208462765","10:i10:0.6931471805599453","11:i11:2.1972245773362196","13:i13:0","14:c1#05db9164","15:c2#38a947a1","16:c3#1c8bc12d","17:c4#bb729a93","18:c5#25c83c98","19:c6#7e0f","20:c7#0433a7db","21:c8#37e4aa92","22:c9#a73ee510","23:c10#3b08e48b","24:c11#418037d7","25:c12#8f4497cb","26:c13#b0bfed6d","27:c14#07d13a8f","28:c15#f28cbdb6","29:c16#b031d057","30:c17#e5ba7672","31:c18#9d83e7f2","34:c21#21756f76","36:c23#423fab69","37:c24#2d794282"],"label":1}
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:499)
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:86)
... 17 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Invalid
feature value: c1#05db9164
at hivemall.fm.Feature.parseFeatureValue(Feature.java:350)
at hivemall.fm.Feature.parseFFMFeature(Feature.java:224)
at hivemall.fm.Feature.parseFFMFeatures(Feature.java:155)
at
hivemall.fm.FieldAwareFactorizationMachineUDTF.parseFeatures(FieldAwareFactorizationMachineUDTF.java:176)
at
hivemall.fm.FactorizationMachineUDTF.process(FactorizationMachineUDTF.java:271)
at
org.apache.hadoop.hive.ql.exec.UDTFOperator.process(UDTFOperator.java:116)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:878)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:878)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:130)
at
org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:149)
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:489)
... 18 more
Caused by: java.lang.NumberFormatException: For input string: "c1#05db9164"
at
sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:2043)
at sun.misc.FloatingDecimal.parseDouble(FloatingDecimal.java:110)
at java.lang.Double.parseDouble(Double.java:538)
at hivemall.fm.Feature.parseFeatureValue(Feature.java:348)
... 29 more
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hivemall issue #105: [WIP][HIVEMALL-24] Scalable field-aware facto...
Github user myui commented on the issue:
https://github.com/apache/incubator-hivemall/pull/105
```
2017-07-28 08:39:59,134 [ERROR] [TezChild] |tez.TezProcessor|:
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
Hive Runtime Error while processing row
{"rowid":null,"features":["1:i1:1.791759469228055","2:i2:3.9889840465642745","3:i3:1.9459101490553132","4:i4:1.9459101490553132","5:i5:0.6931471805599453","6:i6:0","7:i7:4.219507705176107","8:i8:1.9459101490553132","9:i9:5.0238805208462765","10:i10:0.6931471805599453","11:i11:2.1972245773362196","13:i13:0","14:c1#05db9164","15:c2#38a947a1","16:c3#1c8bc12d","17:c4#bb729a93","18:c5#25c83c98","19:c6#7e0f","20:c7#0433a7db","21:c8#37e4aa92","22:c9#a73ee510","23:c10#3b08e48b","24:c11#418037d7","25:c12#8f4497cb","26:c13#b0bfed6d","27:c14#07d13a8f","28:c15#f28cbdb6","29:c16#b031d057","30:c17#e5ba7672","31:c18#9d83e7f2","34:c21#21756f76","36:c23#423fab69","37:c24#2d794282"],"label":1}
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:95)
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:70)
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:360)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:172)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:160)
at
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
at
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
at
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
at
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
Error while processing row
{"rowid":null,"features":["1:i1:1.791759469228055","2:i2:3.9889840465642745","3:i3:1.9459101490553132","4:i4:1.9459101490553132","5:i5:0.6931471805599453","6:i6:0","7:i7:4.219507705176107","8:i8:1.9459101490553132","9:i9:5.0238805208462765","10:i10:0.6931471805599453","11:i11:2.1972245773362196","13:i13:0","14:c1#05db9164","15:c2#38a947a1","16:c3#1c8bc12d","17:c4#bb729a93","18:c5#25c83c98","19:c6#7e0f","20:c7#0433a7db","21:c8#37e4aa92","22:c9#a73ee510","23:c10#3b08e48b","24:c11#418037d7","25:c12#8f4497cb","26:c13#b0bfed6d","27:c14#07d13a8f","28:c15#f28cbdb6","29:c16#b031d057","30:c17#e5ba7672","31:c18#9d83e7f2","34:c21#21756f76","36:c23#423fab69","37:c24#2d794282"],"label":1}
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:499)
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:86)
... 17 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Invalid
feature value: c1#05db9164
at hivemall.fm.Feature.parseFeatureValue(Feature.java:347)
at hivemall.fm.Feature.parseFFMFeature(Feature.java:224)
at hivemall.fm.Feature.parseFFMFeatures(Feature.java:155)
at
hivemall.fm.FieldAwareFactorizationMachineUDTF.parseFeatures(FieldAwareFactorizationMachineUDTF.java:176)
at
hivemall.fm.FactorizationMachineUDTF.process(FactorizationMachineUDTF.java:271)
at
org.apache.hadoop.hive.ql.exec.UDTFOperator.process(UDTFOperator.java:116)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:878)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:878)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:130)
at
org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:149)
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:489)
... 18 more
Caused by: java.lang.NumberFormatException: For input string: "c1#05db9164"
at
sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:2043)
at sun.misc.FloatingDecimal.parseDouble(FloatingDecimal.java:110)
at
[GitHub] incubator-hivemall issue #105: [WIP][HIVEMALL-24] Scalable field-aware facto...
Github user coveralls commented on the issue: https://github.com/apache/incubator-hivemall/pull/105 [](https://coveralls.io/builds/12576130) Coverage decreased (-0.4%) to 40.391% when pulling **bf13385028cc75d352df72c1632033fa4ce70873 on myui:HIVEMALL-24-2** into **7205de1e959f0d9b96ac756e415d8a8ada7e92af on apache:master**. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hivemall issue #105: [WIP][HIVEMALL-24] Scalable field-aware facto...
Github user myui commented on the issue: https://github.com/apache/incubator-hivemall/pull/105 Usage of `ffm_features` can be found in https://github.com/myui/hivemall/pull/286 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hivemall issue #105: [WIP][HIVEMALL-24] Scalable field-aware facto...
Github user myui commented on the issue:
https://github.com/apache/incubator-hivemall/pull/105
Need to fix the following error:
```
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
Error while processing row (tag=0)
{"key":{"reducesinkkey0":1222736,"reducesinkkey1":"0"},"value":{"_col0":6683,"_col2":1.9218120556728056,"_col3":1.0,"_col5":0.0,"_col6":[21.011486,-6.3776617,4.0689898,0.8347159]}}
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:356)
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:278)
... 16 more
Caused by: org.apache.hadoop.hive.ql.exec.UDFArgumentException: The second
and third arguments (Vij, Vji) must not be null
at
hivemall.fm.FFMPredictGenericUDAF$Evaluator.iterate(FFMPredictGenericUDAF.java:159)
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.aggregate(GenericUDAFEvaluator.java:188)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.updateAggregations(GroupByOperator.java:619)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processHashAggr(GroupByOperator.java:794)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processKey(GroupByOperator.java:700)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.process(GroupByOperator.java:768)
at
org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:878)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
at
org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:878)
at
org.apache.hadoop.hive.ql.exec.CommonJoinOperator.internalForward(CommonJoinOperator.java:647)
at
org.apache.hadoop.hive.ql.exec.CommonJoinOperator.createForwardJoinObject(CommonJoinOperator.java:442)
at
org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genObject(CommonJoinOperator.java:534)
at
org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genObject(CommonJoinOperator.java:531)
at
org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genJoinObject(CommonJoinOperator.java:460)
at
org.apache.hadoop.hive.ql.exec.CommonJoinOperator.checkAndGenObject(CommonJoinOperator.java:761)
at
org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.joinObject(CommonMergeJoinOperator.java:316)
at
org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.joinOneGroup(CommonMergeJoinOperator.java:279)
at
org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.joinOneGroup(CommonMergeJoinOperator.java:272)
at
org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.process(CommonMergeJoinOperator.java:258)
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:347)
... 17 more
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hivemall issue #105: [WIP][HIVEMALL-24] Scalable field-aware facto...
Github user coveralls commented on the issue: https://github.com/apache/incubator-hivemall/pull/105 [](https://coveralls.io/builds/12554772) Coverage decreased (-0.4%) to 40.431% when pulling **d0cff233f37570c68dae839d8b21dbaf49ddcf19 on myui:HIVEMALL-24-2** into **7205de1e959f0d9b96ac756e415d8a8ada7e92af on apache:master**. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hivemall issue #105: [WIP][HIVEMALL-24] Scalable field-aware facto...
Github user myui commented on the issue: https://github.com/apache/incubator-hivemall/pull/105 CI passed in https://github.com/apache/incubator-hivemall/pull/105/commits/dd678891926c6f0efabea54e9d1a7ff3762e36fb ãï¸ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hivemall issue #105: [WIP][HIVEMALL-24] Scalable field-aware facto...
Github user coveralls commented on the issue: https://github.com/apache/incubator-hivemall/pull/105 [](https://coveralls.io/builds/12553865) Coverage decreased (-0.4%) to 40.454% when pulling **dd678891926c6f0efabea54e9d1a7ff3762e36fb on myui:HIVEMALL-24-2** into **7205de1e959f0d9b96ac756e415d8a8ada7e92af on apache:master**. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hivemall issue #105: [WIP][HIVEMALL-24] Scalable field-aware facto...
Github user myui commented on the issue: https://github.com/apache/incubator-hivemall/pull/105 I'll try this one. https://github.com/windup/windup-rulesets/blob/master/pom.xml#L159 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hivemall issue #105: [WIP][HIVEMALL-24] Scalable field-aware facto...
Github user myui commented on the issue: https://github.com/apache/incubator-hivemall/pull/105 @maropu Could you check CI error due to Spark test failure. Travis env has been changed to Ubuntu Trusty and oraclejdk8 version might be changed. `mvn -q scalastyle:check test -Pspark-2.1` is killed by some reason. ``` Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support was removed in 8.0 .. /home/travis/.travis/job_stages: line 57: 2845 Killed mvn -q scalastyle:check test -Pspark-2.1 The command "mvn -q scalastyle:check test -Pspark-2.1" exited with 137. 178.12s$ mvn -q scalastyle:check clean -Pspark-2.0 -pl spark/spark-2.0 -am test -Dtest=none ``` http://equj65.net/tech/java8hotspot/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hivemall issue #105: [WIP][HIVEMALL-24] Scalable field-aware facto...
Github user coveralls commented on the issue: https://github.com/apache/incubator-hivemall/pull/105 [](https://coveralls.io/builds/12543689) Coverage decreased (-0.4%) to 40.454% when pulling **357323a522c594830f6cdfeff4d5c115c6f28dd9 on myui:HIVEMALL-24-2** into **7205de1e959f0d9b96ac756e415d8a8ada7e92af on apache:master**. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hivemall issue #105: [WIP][HIVEMALL-24] Scalable field-aware facto...
Github user coveralls commented on the issue: https://github.com/apache/incubator-hivemall/pull/105 [](https://coveralls.io/builds/12538298) Coverage decreased (-0.4%) to 40.454% when pulling **550bb4e6f0f69cfd25506c34caf67e3c014d5750 on myui:HIVEMALL-24-2** into **7205de1e959f0d9b96ac756e415d8a8ada7e92af on apache:master**. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
