> On Nov. 19, 2019, 6:53 p.m., Slim Bouguerra wrote: > > ql/src/java/org/apache/hadoop/hive/ql/hooks/HiveProtoLoggingHook.java > > Lines 274 (patched) > > <https://reviews.apache.org/r/71784/diff/1/?file=2173907#file2173907line275> > > > > looking at the java code i see that it is using a bounded queue so not > > sure what you mean by unbounded ? > > can you please clarify ? > > Attila Magyar wrote: > No, unfortunatelly it uses an unbounded DelayedWorkQueue internally and > it cannot be changed. > > Slim Bouguerra wrote: > ```java > /** > * Specialized delay queue. To mesh with TPE declarations, this > * class must be declared as a BlockingQueue<Runnable> even though > * it can only hold RunnableScheduledFutures. > */ > static class DelayedWorkQueue extends AbstractQueue<Runnable> > implements BlockingQueue<Runnable> { > > ``` > but it is blocking means that it should block and therefore we get the > RejectedException.
Yes but a BlockingQueue can be created without a size limit. It will still block if I want to take out from an empty queue but it will never be full. So it'll never block when adding an element. / The implementation might use Integer.MAX_VAL as a size limit which is practically almost the same (it'll likely never block or it'll block too late). / - Attila ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71784/#review218679 ----------------------------------------------------------- On Nov. 19, 2019, 3:43 p.m., Attila Magyar wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/71784/ > ----------------------------------------------------------- > > (Updated Nov. 19, 2019, 3:43 p.m.) > > > Review request for hive, Laszlo Bodor, Harish Jaiprakash, Mustafa Iman, and > Panos Garefalakis. > > > Bugs: HIVE-22514 > https://issues.apache.org/jira/browse/HIVE-22514 > > > Repository: hive-git > > > Description > ------- > > HiveProtoLoggingHook uses a ScheduledThreadPoolExecutor to submit writer > tasks and to periodically handle rollover. The builtin > ScheduledThreadPoolExecutor uses a unbounded queue which cannot be replaced > from the outside. If log events are generated at a very fast rate this queue > can grow large. > > Since ScheduledThreadPoolExecutor does not support changing the default > unbounded queue to a bounded one, the queue capacity is checked manually by > the patch. > > > Diffs > ----- > > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java a7687d59004 > ql/src/java/org/apache/hadoop/hive/ql/hooks/HiveProtoLoggingHook.java > 8eab54859bf > ql/src/test/org/apache/hadoop/hive/ql/hooks/TestHiveProtoLoggingHook.java > 450a0b544d6 > > > Diff: https://reviews.apache.org/r/71784/diff/1/ > > > Testing > ------- > > unittest > > > Thanks, > > Attila Magyar > >
