[jira] [Commented] (SPARK-24768) Have a built-in AVRO data source implementation

2018-08-23 Thread Antonio Murgia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590441#comment-16590441 ] Antonio Murgia commented on SPARK-24768: Will this support UDT to the extent the parquet

[jira] [Issue Comment Deleted] (SPARK-24772) support reading AVRO logical types - Date

2018-08-23 Thread Antonio Murgia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Antonio Murgia updated SPARK-24772: --- Comment: was deleted (was: Will this support UDT to the extent the parquet reader/writer

[jira] [Commented] (SPARK-24772) support reading AVRO logical types - Date

2018-08-23 Thread Antonio Murgia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590439#comment-16590439 ] Antonio Murgia commented on SPARK-24772: Will this support UDT to the extent the parquet

[jira] [Commented] (SPARK-24862) Spark Encoder is not consistent to scala case class semantic for multiple argument lists

2018-07-21 Thread Antonio Murgia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16551582#comment-16551582 ] Antonio Murgia commented on SPARK-24862: We can check if {{y}} is also synthesized as a field

[jira] [Commented] (SPARK-24862) Spark Encoder is not consistent to scala case class semantic for multiple argument lists

2018-07-20 Thread Antonio Murgia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16551312#comment-16551312 ] Antonio Murgia commented on SPARK-24862: Yeah, they are definitely not supported. Therefore I

[jira] [Created] (SPARK-24862) Spark Encoder is not consistent to scala case class semantic for multiple argument lists

2018-07-19 Thread Antonio Murgia (JIRA)
Antonio Murgia created SPARK-24862: -- Summary: Spark Encoder is not consistent to scala case class semantic for multiple argument lists Key: SPARK-24862 URL: https://issues.apache.org/jira/browse/SPARK-24862

[jira] [Commented] (SPARK-24673) scala sql function from_utc_timestamp second argument could be Column instead of String

2018-07-02 Thread Antonio Murgia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16529479#comment-16529479 ] Antonio Murgia commented on SPARK-24673: I have created a PR, I have added the overload to both

[jira] [Commented] (SPARK-24673) scala sql function from_utc_timestamp second argument could be Column instead of String

2018-06-28 Thread Antonio Murgia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16526307#comment-16526307 ] Antonio Murgia commented on SPARK-24673: Looks doable. Should I go with a method overload,

[jira] [Created] (SPARK-24673) scala sql function from_utc_timestamp second argument could be Column instead of String

2018-06-28 Thread Antonio Murgia (JIRA)
Antonio Murgia created SPARK-24673: -- Summary: scala sql function from_utc_timestamp second argument could be Column instead of String Key: SPARK-24673 URL: https://issues.apache.org/jira/browse/SPARK-24673

[jira] [Commented] (SPARK-15740) Word2VecSuite "big model load / save" caused OOM in maven jenkins builds

2016-06-04 Thread Antonio Murgia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15315394#comment-15315394 ] Antonio Murgia commented on SPARK-15740: As of now the memory requirement would be something like

[jira] [Commented] (SPARK-15740) Word2VecSuite "big model load / save" caused OOM in maven jenkins builds

2016-06-03 Thread Antonio Murgia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313830#comment-15313830 ] Antonio Murgia commented on SPARK-15740: In order to set a small partition size w/r/t

[jira] [Commented] (SPARK-15740) Word2VecSuite "big model load / save" caused OOM in maven jenkins builds

2016-06-03 Thread Antonio Murgia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313754#comment-15313754 ] Antonio Murgia commented on SPARK-15740: Looking into it right now. > Word2VecSuite "big model

[jira] [Created] (SPARK-11993) https://github.com/streamatica/TrafficAnalytics/issues/393#issuecomment-159685855

2015-11-25 Thread Antonio Murgia (JIRA)
Antonio Murgia created SPARK-11993: -- Summary: https://github.com/streamatica/TrafficAnalytics/issues/393#issuecomment-159685855 Key: SPARK-11993 URL: https://issues.apache.org/jira/browse/SPARK-11993

[jira] [Commented] (SPARK-11994) Word2VecModel load and save cause SparkException when model is bigger than spark.kryoserializer.buffer.max

2015-11-25 Thread Antonio Murgia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15028016#comment-15028016 ] Antonio Murgia commented on SPARK-11994: Since `spark.kryoserializer.buffer.max` defaults to

[jira] [Created] (SPARK-11994) Word2VecModel load and save cause SparkException when model is bigger than spark.kryoserializer.buffer.max

2015-11-25 Thread Antonio Murgia (JIRA)
Antonio Murgia created SPARK-11994: -- Summary: Word2VecModel load and save cause SparkException when model is bigger than spark.kryoserializer.buffer.max Key: SPARK-11994 URL:

[jira] [Closed] (SPARK-11993) https://github.com/streamatica/TrafficAnalytics/issues/393#issuecomment-159685855

2015-11-25 Thread Antonio Murgia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Antonio Murgia closed SPARK-11993. -- Resolution: Invalid >

[jira] [Commented] (SPARK-11994) Word2VecModel load and save cause SparkException when model is bigger than spark.kryoserializer.buffer.max

2015-11-25 Thread Antonio Murgia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15027648#comment-15027648 ] Antonio Murgia commented on SPARK-11994: Sure. > Word2VecModel load and save cause

[jira] [Created] (SPARK-11350) There is no best practice to handle warnings or messages produced by Executors in a distributed manner

2015-10-27 Thread Antonio Murgia (JIRA)
Antonio Murgia created SPARK-11350: -- Summary: There is no best practice to handle warnings or messages produced by Executors in a distributed manner Key: SPARK-11350 URL:

[jira] [Updated] (SPARK-10105) Adding most k frequent words parameter to Word2Vec implementation

2015-08-18 Thread Antonio Murgia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Antonio Murgia updated SPARK-10105: --- Description: When training Word2Vec on a really big dataset, it's really hard to evaluate

[jira] [Created] (SPARK-10105) Adding most k frequent words parameter to Word2Vec implementation

2015-08-18 Thread Antonio Murgia (JIRA)
Antonio Murgia created SPARK-10105: -- Summary: Adding most k frequent words parameter to Word2Vec implementation Key: SPARK-10105 URL: https://issues.apache.org/jira/browse/SPARK-10105 Project: Spark

[jira] [Created] (SPARK-10046) Hive warehouse dir not set in current directory when not providing hive-site.xml

2015-08-16 Thread Antonio Murgia (JIRA)
Antonio Murgia created SPARK-10046: -- Summary: Hive warehouse dir not set in current directory when not providing hive-site.xml Key: SPARK-10046 URL: https://issues.apache.org/jira/browse/SPARK-10046