Thank you, I +1ed the JIRA, and added my user list comments. This should be easily reproducible and fixable.
On Wed, Jun 1, 2016 at 11:17 AM, Krystal Nguyen <kngu...@maprtech.com> wrote: > Looks like a jira was opened a while back (status is unresolved) regarding > similar issue you are seeing: > https://issues.apache.org/jira/browse/DRILL-2593 > > On Wed, Jun 1, 2016 at 5:52 AM, John Omernik <j...@omernik.com> wrote: > > > Abdel and Jacques - > > > > It is absolutely a bad profile, however, Jacques, it was not a empty file > > profile, instead, it was a profile where the write didn't complete > (likely > > due to my bad cluster state post). > > > > I was able to reproduce it synthetically like this, say you had a profile > > that ended similar to this: > > > > ..."endpoint":{"address":"zeta01.mydomain.net","userPort": > > 20001,"controlPort":20002,"dataPort":20003},"lastUpdate": > > 1464730828126,"lastProgress":1464730828126}]}],"user":"adm"} > > > > (The ... represents the part I removed for brevity in this email, the > rest > > of the profile is below) > > > > We can reproduce the UI profiles issue by removing ,"user":"adm"} > > > > so it ends like this: > > > > ..."endpoint":{"address":"zeta01.mydomain.net","userPort": > > 20001,"controlPort":20002,"dataPort":20003},"lastUpdate": > > 1464730828126,"lastProgress":1464730828126}]}] > > > > > > Let me know if you know of a JIRA, otherwise I'll open one, this is a > pain > > for users, because you have to go through an validate the json on every > > profile to find the one that is breaking your UI. > > > > John > > > > > > > > > > > > Full "Working" Profile: > > > > {"id":{"part1":2932401029014293687,"part2":6412631480783062510},"type":1, > > "start":1464730828036,"end":1464730828131,"query":"alter session set > > `store.parquet.enable_dictionary_encoding` = true","foreman":{"address":" > > zeta01.mydomain.net","userPort":20001,"controlPort":20002," > > dataPort":20003},"state":2,"totalFragments":1,"finishedFragments":0," > > fragmentProfile":[{"majorFragmentId":0,"minorFragmentProfile":[{" > > state":3,"minorFragmentId":0,"operatorProfile":[{" > > inputProfile":[{"records":1,"batches":1,"schemas":1}]," > > operatorId":0,"operatorType":26,"setupNanos":0,"processNanos":1020862," > > peakLocalMemoryAllocated":4530432,"waitNanos":0},{" > > inputProfile":[{"records":1,"batches":1,"schemas":1}]," > > operatorId":0,"operatorType":13,"setupNanos":0,"processNanos":154675," > > peakLocalMemoryAllocated":4530176,"metric":[{"metricId": > > 0,"longValue":59}],"waitNanos":1936593}],"startTime": > > 1464730828114,"endTime":1464730828123,"memoryUsed":0," > > maxMemoryUsed":9060608,"endpoint":{"address":"zeta01.mydomain.net > > ","userPort":20001,"controlPort":20002,"dataPort":20003},"lastUpdate": > > 1464730828126,"lastProgress":1464730828126}]}],"user":"adm"} > > > > > > Full "Broken" Profile: > > > > {"id":{"part1":2932401029014293687,"part2":6412631480783062510},"type":1, > > "start":1464730828036,"end":1464730828131,"query":"alter session set > > `store.parquet.enable_dictionary_encoding` = true","foreman":{"address":" > > zeta01.mydomain.net","userPort":20001,"controlPort":20002," > > dataPort":20003},"state":2,"totalFragments":1,"finishedFragments":0," > > fragmentProfile":[{"majorFragmentId":0,"minorFragmentProfile":[{" > > state":3,"minorFragmentId":0,"operatorProfile":[{" > > inputProfile":[{"records":1,"batches":1,"schemas":1}]," > > operatorId":0,"operatorType":26,"setupNanos":0,"processNanos":1020862," > > peakLocalMemoryAllocated":4530432,"waitNanos":0},{" > > inputProfile":[{"records":1,"batches":1,"schemas":1}]," > > operatorId":0,"operatorType":13,"setupNanos":0,"processNanos":154675," > > peakLocalMemoryAllocated":4530176,"metric":[{"metricId": > > 0,"longValue":59}],"waitNanos":1936593}],"startTime": > > 1464730828114,"endTime":1464730828123,"memoryUsed":0," > > maxMemoryUsed":9060608,"endpoint":{"address":"zeta01.mydomain.net > > ","userPort":20001,"controlPort":20002,"dataPort":20003},"lastUpdate": > > 1464730828126,"lastProgress":1464730828126}]}] > > > > > > On Wed, Jun 1, 2016 at 6:22 AM, John Omernik <j...@omernik.com> wrote: > > > > > I will walk through and look for the corrupt profile. This could be > the > > > result of my "cluster in a bad state problem" (actually, it IS the > result > > > of that, I have Ben doing testing to recreate the issue). Is there a > JIRA > > > for this ? I agree the UI should handle this. > > > > > > > > > On Tuesday, May 31, 2016, Jacques Nadeau <jacq...@dremio.com> wrote: > > > > > >> Odds are one of your profiles is corrupt. Last I checked, there was a > > bug > > >> that a corrupt profile would cause no profiles to show up in list. > Check > > >> if > > >> any of your profiles are zero bytes and delete them. > > >> > > >> The UI should really handle this. > > >> > > >> -- > > >> Jacques Nadeau > > >> CTO and Co-Founder, Dremio > > >> > > >> On Tue, May 31, 2016 at 12:49 PM, John Omernik <j...@omernik.com> > > wrote: > > >> > > >> > I am scratching my head at this one... I made some minor changes to > my > > >> > drill-env.sh to enable gclogging, and was using the profiles in the > > >> webui > > >> > just fine. Due to some previously mentioned issues, I've had to > > restart > > >> > drill bits due to GC issues etc. > > >> > > > >> > Now, while my profiles directory still exists, and my > > >> drill-override.conf > > >> > has not been changed, no profiles now show up in the webui, even > after > > >> > drillbit restarts, and running more queries... The profiles are > still > > >> being > > >> > created (I can see them being added to the same profiles directory) > > just > > >> > nothing shows up in the Web UI... > > >> > > > >> > What could be happening here? > > >> > > > >> > *scratching my head > > >> > > > >> > > > > > > > > > -- > > > Sent from my iThing > > > > > >
