Hi David,
Thank you for joining the mailing list!
>if the lambda function that processes a window into a new stream
encounters an exception but DOES NOT handle it, what is supposed to happen?
By not handling it, I assume you mean something like the following where
the exception is rethrown:
/* In the user's lambda */
try{
// Do some operation
}
catch(IllegalStateException e){
throw e;
}
In this case, what *does* happen, currently, is the exception will
percolate up to the Edgent/Quarks Thread Scheduler and be caught there. I
believe this kills all runtime threads, terminating the application. This
is why you observe all tuple flow to stop after you removed exception
catching from your lambda code.
What *should* happen is that that the windowing library catches the
exception and then stop all batching for that window. This is more graceful
than terminating all threads. The windowing library might look something
like the following:
/* In PartitionImpl */
@Override
public synchronized void process() {
try{
window.getPartitionProcessor().accept(unmodifiableTuples, key);
}
catch(Exception e){
// Clear the ScheduledExecutorService which handles the batch
scheduling. No more batching for this window.
}
}
>I rewrote my lambda function to catch exceptions, and once in a while the
catch clause gets control.
Right, so if your lambda code catches all exceptions and doesn't rethrow
them, the batch scheduler doesn't know that anything is wrong and will
continue to schedule batches. This is why the catch clause gets control
once in a while, and you continue to see tuples downstream from the window.
>It is very inconvenient (from a programming model perspective) for the
lambda functions to have to do exception handling in simple cases
Would you mind providing a brief code/pseudocode example of such a simple
case?
I hope this helps to answer your question.
-Will
