Hi,
I was able to resolve issue with below conditions.
datapoint_df(Constants.Datapoint.Vin).like("012345")
datapoint_filter_df.filter( datapoint_filter_df(Constants.Datapoint.Vin)
rlike "^([A-Z]|[0-9]|[a-z])+$" ) // for checking alpha numeric.
Thanks,
Asmath
On Tue, Feb 28, 2017 at 10:49 AM, KhajaAsmath Mohammed <
mdkhajaasm...@gmail.com> wrote:
> Hi,
>
> Could anyone please provide me your suggestions on how to resolve the
> issue that I am facing with not contains code on dataframe column.
>
> Here is the code. My dataframe is not getting filtered with below
> conditions. I even tried not and ! on Column. any suggestions?
>
> def filterDatapointRawCountsDF(vin: Column): Column =
>
> {
>
> import org.apache.spark.sql.functions.not
> val filterColumn: Column = {
>
> not(vin.contains("VIN")) ||
> not(vin.contains("Ÿ")) ||
> not(vin.contains("0123456789ABCDEFG"))
>
> }
>
> filterColumn;
>
> }
>
>
> }
>
>
