[jira] [Commented] (SPARK-23463) Filter operation fails to handle blank values and evicts rows that even satisfy the filtering condition

2018-02-21 Thread Manan Bakshi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16371864#comment-16371864 ] Manan Bakshi commented on SPARK-23463: -- I will go ahead and resolve this. Thanks!  > Filter

[jira] [Commented] (SPARK-23463) Filter operation fails to handle blank values and evicts rows that even satisfy the filtering condition

2018-02-21 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16371186#comment-16371186 ] Marco Gaido commented on SPARK-23463: - It changed Spark's implicit casting. Probably in 2.1.1

[jira] [Commented] (SPARK-23463) Filter operation fails to handle blank values and evicts rows that even satisfy the filtering condition

2018-02-20 Thread Manan Bakshi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370917#comment-16370917 ] Manan Bakshi commented on SPARK-23463: -- Hi Marco, That makes sense. However, this same code used to

[jira] [Commented] (SPARK-23463) Filter operation fails to handle blank values and evicts rows that even satisfy the filtering condition

2018-02-20 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370046#comment-16370046 ] Marco Gaido commented on SPARK-23463: - Hi [~m.bakshi11]. The problem is very easy. The column `val`

[jira] [Commented] (SPARK-23463) Filter operation fails to handle blank values and evicts rows that even satisfy the filtering condition

2018-02-19 Thread Manan Bakshi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369750#comment-16369750 ] Manan Bakshi commented on SPARK-23463: -- Hi Marco, I realized that if you used following code for

[jira] [Commented] (SPARK-23463) Filter operation fails to handle blank values and evicts rows that even satisfy the filtering condition

2018-02-19 Thread Manan Bakshi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369424#comment-16369424 ] Manan Bakshi commented on SPARK-23463: -- Hi Marco, I have attached a text file along with this

[jira] [Commented] (SPARK-23463) Filter operation fails to handle blank values and evicts rows that even satisfy the filtering condition

2018-02-19 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368968#comment-16368968 ] Marco Gaido commented on SPARK-23463: - sorry, what do you mean by blank values? Which is the type of

[jira] [Commented] (SPARK-23463) Filter operation fails to handle blank values and evicts rows that even satisfy the filtering condition

2018-02-19 Thread Manan Bakshi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368879#comment-16368879 ] Manan Bakshi commented on SPARK-23463: -- I believe that the bug has something to do with CBO that was