Hi.
There is a workaround for that.
You can disable event logs for Spark Streaming applications.
On Tue, Jul 16, 2019 at 1:08 PM raman gugnani
wrote:
> HI ,
>
> I have long running spark streaming jobs.
> Event log directories are getting filled with .inprogress files.
> Is there fix or work
uplication) while Spark is adding more data to it every
minute. (hive query can take > 2 minutes)
Thanks,
Artur Sukhenko
Probably it is wrong to compare StringType and ShortType.
I'll use something like this
df.select(colString, colShort,
colShort.equalTo(colString.cast(DecimalType(38,15.show
On Wed, Feb 6, 2019 at 6:32 PM Artur Sukhenko
wrote:
> scala> df.select(colString, colShort, colShort.e
ing it's casting tier_id to a
> small_int to do the comparison
>
> On Wed, Feb 6, 2019 at 9:31 AM Artur Sukhenko
> wrote:
>
>> Hello guys,
>> I am migrating from Spark 1.6 to 2.2 and have this issue:
>> I am casting string to short and comparing them with equal .
>&g
3.15| 3| true|
+---+---+-----+
scala>
Why is this?
--
--
Artur Sukhenko
