be
>>>>>>> done in UI
>>>>>>> > so maybe doing manually would be the only option. Not sure Spark
>>>>>>> UI will
>>>>>>> > provide the same, haven't used at all.)
>>>>>>> >
&
#x27;s shown as
>>>>>> running) and
>>>>>> > which point to look at.
>>>>>> >
>>>>>> > On Thu, Apr 16, 2020 at 4:29 PM Ruijing Li
>>>>>> wrote:
>>>>>> >
>>>&
--
Cheers,
-z
[1] -- https://linux.die.net/man/2/accept
From: Jungtaek Lim
Sent: Wednesday, April 22, 2020 11:21
To: Ruijing Li
Cc: Gabor Somogyi; Mich Talebzadeh; ZHANG Wei; user
Subject: Re: Spark hangs while reading from jdbc - does nothing Removing Guess
work from trouble shooting
No, that&#x
d (even it's shown as
>>>>>> running) and
>>>>>> > which point to look at.
>>>>>> >
>>>>>> > On Thu, Apr 16, 2020 at 4:29 PM Ruijing Li
>>>>>> wrote:
>>>>>> >
>>>>>>
mp, what should I be looking for to tell where
>>>>> it is
>>>>> > > hanging? Seeing a lot of timed_waiting and waiting on driver.
>>>>> Driver is
>>>>> > > also being blocked by spark UI. If there are no tas
gt;>> > > On Tue, Apr 14, 2020 at 4:49 AM Gabor Somogyi <
>>>> gabor.g.somo...@gmail.com>
>>>> > > wrote:
>>>> > >
>>>> > >> The simplest way is to do thread dump which doesn't require any
>>&
, 2020 at 11:32 AM jane thorpe
>>
>> > >> wrote:
>> > >>
>> > >>> Here a is another tool I use Logic Analyser 7:55
>> > >>> https://youtu.be/LnzuMJLZRdU
>> > >>>
>> > >>> you could take so
gt;> > >
>>> > >> The simplest way is to do thread dump which doesn't require any
>>> fancy
>>> > >> tool (it's available on Spark UI).
>>> > >> Without thread dump it's hard to say anything...
>>> >
; >>
> > >>> Here a is another tool I use Logic Analyser 7:55
> > >>> https://youtu.be/LnzuMJLZRdU
> > >>>
> > >>> you could take some suggestions for improving performance queries.
> > >>>
> https://dzone.c
>>> time of different components cascading through the control flow within an
> >>> application.
> >>> These types of visualizations are useful, and AppOptics has them, but
> >>> they can be difficult to understand for those of us without a PhD."
ful, and AppOptics has them, but
>>> they can be difficult to understand for those of us without a PhD."
>>>
>>> Especially helpful if you want to understand through visualisation and
>>> you do not have a phD.
>>>
>>>
>>> Jane tho
ethor...@aol.com
>>
>>
>> -Original Message-
>> From: jane thorpe
>> To: janethorpe1 ; mich.talebzadeh <
>> mich.talebza...@gmail.com>; liruijing09 ; user <
>> user@spark.apache.org>
>> Sent: Mon, 13 Apr 2020 8:32
>> Subject: Re: Spark hangs
nethorpe1 ; mich.talebzadeh <
> mich.talebza...@gmail.com>; liruijing09 ; user <
> user@spark.apache.org>
> Sent: Mon, 13 Apr 2020 8:32
> Subject: Re: Spark hangs while reading from jdbc - does nothing Removing
> Guess work from trouble shooting
>
thorpe
To: janethorpe1 ; mich.talebzadeh
; liruijing09 ; user
Sent: Mon, 13 Apr 2020 8:32
Subject: Re: Spark hangs while reading from jdbc - does nothing Removing Guess
work from trouble shooting
This tool may be useful for you to trouble shoot your problems away.
https
e
janethor...@aol.com
-Original Message-
From: jane thorpe
To: mich.talebzadeh ; liruijing09
; user
CC: user
Sent: Sun, 12 Apr 2020 4:35
Subject: Re: Spark hangs while reading from jdbc - does nothing
You seem to be implying the error is intermittent.
You seem to be implying data is bein
You seem to be implying the error is intermittent.
You seem to be implying data is being ingested via JDBC. So the connection has
proven itself to be working unless no data is arriving from the JDBC channel
at all. If no data is arriving then one could say it could be the JDBC.
If the e
Hi,
Have you checked your JDBC connections from Spark to Oracle. What is Oracle
saying? Is it doing anything or hanging?
set pagesize
set linesize 140
set heading off
select SUBSTR(name,1,8) || ' sessions as on '||TO_CHAR(CURRENT_DATE, 'MON
DD HH:MI AM') from v$database;
set heading on
Hi all,
I am on spark 2.4.4 and using scala 2.11.12, and running cluster mode on
mesos. I am ingesting from an oracle database using spark.read.jdbc. I am
seeing a strange issue where spark just hangs and does nothing, not
starting any new tasks. Normally this job finishes in 30 stages but
sometim
18 matches
Mail list logo
