Re: Structured Streaming + UDF - logic based on checking if a column is present in the Dataframe

2022-02-25 Thread Gourav Sengupta
Hi, can you please let us know the following: 1. the spark version 2. a few samples of input data 3. a few samples of what is the expected output that you want Regards, Gourav Sengupta On Wed, Feb 23, 2022 at 8:43 PM karan alang wrote: > Hello All, > > I'm using StructuredStreaming, and am

Structured Streaming + UDF - logic based on checking if a column is present in the Dataframe

2022-02-23 Thread karan alang
Hello All, I'm using StructuredStreaming, and am trying to use UDF to parse each row. Here is the requirement: - we can get alerts of a particular KPI with type 'major' OR 'critical' - for a KPI, if we get alerts of type 'major' eg _major, and we have a critical alert as well _critical,