I got one answer on Stackoverflow (sorry for duplicates, I am so curious) https://stackoverflow.com/questions/52479607/apache-beam-why-does-it-write-to-spanner-twice-on-report-failures-mode
I'd love to hear more, of course. :-) 2018년 9월 24일 (월) 오후 6:03, Lukasz Cwik <[email protected]>님이 작성: > +Mairbek Khadikov <[email protected]>, do you know why the writes are > first attempted as a batch and then re-attempted individually if the batch > call failed? > > (Might be worth a code comment to explain the retry behavior) > > On Mon, Sep 24, 2018 at 7:12 AM NaHeon Kim <[email protected]> wrote: > >> Hi community! >> >> I found interesting write operation codes while looking at SpannerIO, and >> want to understand reasons. >> >> On write(WriteToSpannerFn) and REPORT_FAILURES failure mode, it seems >> trying to write failed mutations twice. >> I think it's for logging each mutation's exceptions. Is it a correct >> assumption, and is there any workaround? >> Below, I removed some lines for simplicity. >> >> public void processElement(ProcessContext c) { >> Iterable<MutationGroup> mutations = c.element(); >> boolean tryIndividual = false; >> >> try { >> Iterable<Mutation> batch = Iterables.concat(mutations); >> spannerAccessor.getDatabaseClient().writeAtLeastOnce(batch); >> } catch (SpannerException e) { >> if (failureMode == FailureMode.REPORT_FAILURES) { >> tryIndividual = true; >> } else { >> ... >> } >> } >> if (tryIndividual) { >> for (MutationGroup mg : mutations) { >> try { >> spannerAccessor.getDatabaseClient().writeAtLeastOnce(mg); >> } catch (SpannerException e) { >> LOG.warn("Failed to submit the mutation group", e); >> c.output(failedTag, mg); >> } >> } >> } >> } >> >> Thanks! >> >
