[GitHub] spark pull request #19244: SPARK-22021

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/19244


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19244: SPARK-22021

[srowen]

Github user srowen commented on a diff in the pull request:

https://github.com/apache/spark/pull/19244#discussion_r139281714
  
--- Diff: 
mllib/src/main/scala/org/apache/spark/ml/feature/GenFuncTransformer.scala ---
@@ -0,0 +1,110 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.ml.feature
+
+import scala.util.Random
+
+import org.apache.spark.annotation.Since
+import org.apache.spark.ml.Transformer
+import org.apache.spark.ml.param.Param
+import org.apache.spark.ml.param.ParamMap
+import org.apache.spark.ml.param.shared.HasInputCols
+import org.apache.spark.ml.param.shared.HasOutputCol
+import org.apache.spark.ml.util.DefaultParamsReadable
+import org.apache.spark.ml.util.DefaultParamsWritable
+import org.apache.spark.ml.util.Identifiable
+import org.apache.spark.sql.DataFrame
+import org.apache.spark.sql.Dataset
+import org.apache.spark.sql.Row
+import org.apache.spark.sql.catalyst.encoders.RowEncoder
+import org.apache.spark.sql.types.DoubleType
+import org.apache.spark.sql.types.NumericType
+import org.apache.spark.sql.types.StructField
+import org.apache.spark.sql.types.StructType
+
+import javax.script.ScriptEngineManager
--- End diff --

I think introducing Javascript is a non-starter


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19244: SPARK-22021

[narahari92]

GitHub user narahari92 opened a pull request:

https://github.com/apache/spark/pull/19244

SPARK-22021

## What changes were proposed in this pull request?

This adds a new feature transformer which will take a javascript function 
and based on the logic provided in that, derives new features.

## How was this patch tested?

There is proper unit test written for this transformer and also it's been 
testing in local and yarn-client mode.
Please review http://spark.apache.org/contributing.html before opening a 
pull request.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/narahari92/spark master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/19244.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #19244


commit c3c95523e6f60a22055b968f61a7f525ed13bafa
Author: Hosur Narhari 
Date:   2017-09-15T11:13:15Z

committing GenFuncTransformer

commit 795619f6989a66708c41042a2338e9e343a352d9
Author: Hosur Narhari 
Date:   2017-09-15T11:19:30Z

adding license header to GenFuncTransformerSuite




---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org