Re: [RFC] Bootloader Buffered Logging
Much appreciated, reviewing the doc now. On Wed, Aug 16, 2023 at 9:08 PM Valentyn Tymofieiev via dev < dev@beam.apache.org> wrote: > Thanks, Jack! left some comments, looking forward to this work! > > On Wed, Aug 16, 2023 at 10:31 AM Robert Burke wrote: > >> I've added some comments but generally +1 on this. >> >> A later change might be able to build from this to ensure the various >> STDErr and STDOut logs from the SDK harness executions are always plumbed >> as described. >> >> But that would take more thought since other incidental logs from the >> users worker binary (sic) might be misconstrued as serious when they were >> largely benign noise previously ignored (since they were invisible). >> >> On Wed, Aug 16, 2023, 9:57 AM Jack McCluskey via dev >> wrote: >> >>> Hey everyone, >>> >>> I've written a small design doc around implementing some buffered >>> logging for the Beam boot.go scripts that is available at >>> https://s.apache.org/beam-buffered-logging. This should help surface >>> errors that occur during worker set-up (like issues with dependency >>> installation) that tend to be logged improperly at INFO. >>> >>> Thanks, >>> >>> Jack McCluskey >>> >>> -- >>> >>> >>> Jack McCluskey >>> SWE - DataPLS PLAT/ Dataflow ML >>> RDU >>> jrmcclus...@google.com >>> >>> >>>
Re: [RFC] Bootloader Buffered Logging
Thanks, Jack! left some comments, looking forward to this work! On Wed, Aug 16, 2023 at 10:31 AM Robert Burke wrote: > I've added some comments but generally +1 on this. > > A later change might be able to build from this to ensure the various > STDErr and STDOut logs from the SDK harness executions are always plumbed > as described. > > But that would take more thought since other incidental logs from the > users worker binary (sic) might be misconstrued as serious when they were > largely benign noise previously ignored (since they were invisible). > > On Wed, Aug 16, 2023, 9:57 AM Jack McCluskey via dev > wrote: > >> Hey everyone, >> >> I've written a small design doc around implementing some buffered logging >> for the Beam boot.go scripts that is available at >> https://s.apache.org/beam-buffered-logging. This should help surface >> errors that occur during worker set-up (like issues with dependency >> installation) that tend to be logged improperly at INFO. >> >> Thanks, >> >> Jack McCluskey >> >> -- >> >> >> Jack McCluskey >> SWE - DataPLS PLAT/ Dataflow ML >> RDU >> jrmcclus...@google.com >> >> >>
Re: [RFC] Bootloader Buffered Logging
I've added some comments but generally +1 on this. A later change might be able to build from this to ensure the various STDErr and STDOut logs from the SDK harness executions are always plumbed as described. But that would take more thought since other incidental logs from the users worker binary (sic) might be misconstrued as serious when they were largely benign noise previously ignored (since they were invisible). On Wed, Aug 16, 2023, 9:57 AM Jack McCluskey via dev wrote: > Hey everyone, > > I've written a small design doc around implementing some buffered logging > for the Beam boot.go scripts that is available at > https://s.apache.org/beam-buffered-logging. This should help surface > errors that occur during worker set-up (like issues with dependency > installation) that tend to be logged improperly at INFO. > > Thanks, > > Jack McCluskey > > -- > > > Jack McCluskey > SWE - DataPLS PLAT/ Dataflow ML > RDU > jrmcclus...@google.com > > >
[RFC] Bootloader Buffered Logging
Hey everyone, I've written a small design doc around implementing some buffered logging for the Beam boot.go scripts that is available at https://s.apache.org/beam-buffered-logging. This should help surface errors that occur during worker set-up (like issues with dependency installation) that tend to be logged improperly at INFO. Thanks, Jack McCluskey -- Jack McCluskey SWE - DataPLS PLAT/ Dataflow ML RDU jrmcclus...@google.com
