Durga Prasad Gunturu created SPARK-18593:
--------------------------------------------
Summary: Unable to use filter function if the data length is one.
Key: SPARK-18593
URL: https://issues.apache.org/jira/browse/SPARK-18593
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 1.6.2
Reporter: Durga Prasad Gunturu
Priority: Minor
when using filter using jdbc.. unable to retrieve data if the column value is
having only value of length one.
for example:
Below code works because the filter condition data is of length > 1
val dataFromTable =
sqlContext.read.jdbc("jdbc:postgresql://localhost:5433/postgres", "stocknames",
defaultProps)
dataFromTable.filter(dataFromTable("grouptype")==="ZP").show()
result:
+------------+----------+--------------------+------+---------+---------+------------+--------------------+----------+
|securitycode|securityid| securityname|status|grouptype|facevalue|
isinno| industry|instrument|
+------------+----------+--------------------+------+---------+---------+------------+--------------------+----------+
| 504370| ZARDIINV|ARDI INVESTMENT &...|Active| ZP| 10.0|
|Misc.Commercial S...| Equity|
| 512221| TARCF|TARRIF CINE &...|Active| ZP|
10.0|INE177E01010|Finance (includin...| Equity|
| 512505| MEENST|MEENAKSHI STEEL I...|Active| ZP| 10.0|
|Iron & Steel ...| Equity|
| 514060| EVERTEX|Evergreen Textile...|Active| ZP|
10.0|INE229N01010| Textiles| Equity|
| 526085| SGARRES|SAGAR TOURIST RES...|Active| ZP| 10.0|NA
| Hotels| Equity|
+------------+----------+--------------------+------+---------+---------+------------+--------------------+----------+
But if the same filter condition code is of length 1, no data is filtered.
eg:
val dataFromTable =
sqlContext.read.jdbc("jdbc:postgresql://localhost:5433/postgres", "stocknames",
defaultProps)
dataFromTable.filter(dataFromTable("grouptype")==="A").show()
not getting any output
+------------+----------+------------+------+---------+---------+------+--------+----------+
|securitycode|securityid|securityname|status|grouptype|facevalue|isinno|industry|instrument|
+------------+----------+------------+------+---------+---------+------+--------+----------+
+------------+----------+------------+------+---------+---------+------+--------+----------+
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
