I am afraid logic is incorrect. that is the reason why it is not working. Dr Mich Talebzadeh
LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* http://talebzadehmich.wordpress.com *Disclaimer:* Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction. On 8 August 2016 at 04:36, Divya Gehlot <divya.htco...@gmail.com> wrote: > I tried with condition expression also but it didn't work :( > > On Aug 8, 2016 11:13 AM, "Chanh Le" <giaosu...@gmail.com> wrote: > >> You should use *df.where(conditionExpr)* which is more convenient to >> express some simple term in SQL. >> >> >> /** >> * Filters rows using the given SQL expression. >> * {{{ >> * peopleDf.where("age > 15") >> * }}} >> * @group dfops >> * @since 1.5.0 >> */ >> def where(conditionExpr: String): DataFrame = { >> filter(Column(SqlParser.parseExpression(conditionExpr))) >> } >> >> >> >> >> >> On Aug 7, 2016, at 10:58 PM, Mich Talebzadeh <mich.talebza...@gmail.com> >> wrote: >> >> although the logic should be col1 <> a && col(1) <> b >> >> to exclude both >> >> Like >> >> df.filter('transactiontype > " ").filter(not('transactiontype ==="DEB") >> && not('transactiontype ==="BGC")).select('transaction >> type).distinct.collect.foreach(println) >> >> HTH >> >> Dr Mich Talebzadeh >> >> >> LinkedIn * >> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw >> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* >> >> >> http://talebzadehmich.wordpress.com >> >> *Disclaimer:* Use it at your own risk. Any and all responsibility for >> any loss, damage or destruction of data or any other property which may >> arise from relying on this email's technical content is explicitly >> disclaimed. The author will in no case be liable for any monetary damages >> arising from such loss, damage or destruction. >> >> >> >> On 7 August 2016 at 16:53, Mich Talebzadeh <mich.talebza...@gmail.com> >> wrote: >> >>> try similar to this >>> >>> df.filter(not('transactiontype ==="DEB") || not('transactiontype >>> ==="CRE")) >>> >>> HTH >>> >>> >>> Dr Mich Talebzadeh >>> >>> >>> LinkedIn * >>> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw >>> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* >>> >>> >>> http://talebzadehmich.wordpress.com >>> >>> *Disclaimer:* Use it at your own risk. Any and all responsibility for >>> any loss, damage or destruction of data or any other property which may >>> arise from relying on this email's technical content is explicitly >>> disclaimed. The author will in no case be liable for any monetary damages >>> arising from such loss, damage or destruction. >>> >>> >>> >>> On 7 August 2016 at 15:43, Divya Gehlot <divya.htco...@gmail.com> wrote: >>> >>>> Hi, >>>> I have use case where I need to use or[||] operator in filter condition. >>>> It seems its not working its taking the condition before the operator >>>> and ignoring the other filter condition after or operator. >>>> As any body faced similar issue . >>>> >>>> Psuedo code : >>>> df.filter(col("colName").notEqual("no_value") || >>>> col("colName").notEqual("")) >>>> >>>> Am I missing something. >>>> Would really appreciate the help. >>>> >>>> >>>> Thanks, >>>> Divya >>>> >>> >>> >> >>
