Re: [Mojolicious] Minion args
Thanks, Dan! Using Mojo::JSON::MaybeXS solved the problem due to having too large of an argument (pure text) in the args jsonb column. What I can't understand is how it kills my whole Linux system to the point I need to do a hard reset on the OS. On Thu, Apr 26, 2018 at 11:22 PM, Dan Bookwrote: > What backend are you using? > > Note that the args have to be decoded from JSON whenever accessed and when > starting the job, which for pure-perl parsers can be very slow. Perhaps try > loading Mojo::JSON::MaybeXS in your app. Otherwise I'd suggest storing it > in a separate table in a way that doesn't require encoding or decoding, or > using Sereal to serialize it for storage. > > -Dan > > On Fri, Apr 27, 2018 at 12:05 AM, Stefan Adams wrote: > >> Is there a maximum length that I should respect for the args being passed >> to a Minion task? >> >> I have a job that has an arg of 2MB and the state is failed with a result >> of "Worker went away". Using the Minion Admin UI, I clicked on the >> "failed" tasks and it hangs my entire Ubuntu 16.04.3 system! I've had to >> reboot three times now. I have since manually removed that specific job >> from the database and now, so far, all seems to be well, and I can also >> browse failed tasks (there are now 0). >> >> If I shouldn't be passing 2MB via args, should I instead save the 2MB to >> disk and let the job remove the file from disk when the job has finished >> successfully? >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Mojolicious" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to mojolicious+unsubscr...@googlegroups.com. >> To post to this group, send email to mojolicious@googlegroups.com. >> Visit this group at https://groups.google.com/group/mojolicious. >> For more options, visit https://groups.google.com/d/optout. >> > > -- > You received this message because you are subscribed to the Google Groups > "Mojolicious" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to mojolicious+unsubscr...@googlegroups.com. > To post to this group, send email to mojolicious@googlegroups.com. > Visit this group at https://groups.google.com/group/mojolicious. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Mojolicious" group. To unsubscribe from this group and stop receiving emails from it, send an email to mojolicious+unsubscr...@googlegroups.com. To post to this group, send email to mojolicious@googlegroups.com. Visit this group at https://groups.google.com/group/mojolicious. For more options, visit https://groups.google.com/d/optout.
Re: [Mojolicious] Minion args
Thanks, Dan! I'll try that. Yeah I don't need anything fancy, it's just a large text string (a raw email message headers, attachments, and all, actually). I'm using Pg. On Thu, Apr 26, 2018, 11:22 PM Dan Bookwrote: > What backend are you using? > > Note that the args have to be decoded from JSON whenever accessed and when > starting the job, which for pure-perl parsers can be very slow. Perhaps try > loading Mojo::JSON::MaybeXS in your app. Otherwise I'd suggest storing it > in a separate table in a way that doesn't require encoding or decoding, or > using Sereal to serialize it for storage. > > -Dan > > On Fri, Apr 27, 2018 at 12:05 AM, Stefan Adams wrote: > >> Is there a maximum length that I should respect for the args being passed >> to a Minion task? >> >> I have a job that has an arg of 2MB and the state is failed with a result >> of "Worker went away". Using the Minion Admin UI, I clicked on the >> "failed" tasks and it hangs my entire Ubuntu 16.04.3 system! I've had to >> reboot three times now. I have since manually removed that specific job >> from the database and now, so far, all seems to be well, and I can also >> browse failed tasks (there are now 0). >> >> If I shouldn't be passing 2MB via args, should I instead save the 2MB to >> disk and let the job remove the file from disk when the job has finished >> successfully? >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Mojolicious" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to mojolicious+unsubscr...@googlegroups.com. >> To post to this group, send email to mojolicious@googlegroups.com. >> Visit this group at https://groups.google.com/group/mojolicious. >> For more options, visit https://groups.google.com/d/optout. >> > > -- > You received this message because you are subscribed to the Google Groups > "Mojolicious" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to mojolicious+unsubscr...@googlegroups.com. > To post to this group, send email to mojolicious@googlegroups.com. > Visit this group at https://groups.google.com/group/mojolicious. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Mojolicious" group. To unsubscribe from this group and stop receiving emails from it, send an email to mojolicious+unsubscr...@googlegroups.com. To post to this group, send email to mojolicious@googlegroups.com. Visit this group at https://groups.google.com/group/mojolicious. For more options, visit https://groups.google.com/d/optout.
Re: [Mojolicious] Minion args
What backend are you using? Note that the args have to be decoded from JSON whenever accessed and when starting the job, which for pure-perl parsers can be very slow. Perhaps try loading Mojo::JSON::MaybeXS in your app. Otherwise I'd suggest storing it in a separate table in a way that doesn't require encoding or decoding, or using Sereal to serialize it for storage. -Dan On Fri, Apr 27, 2018 at 12:05 AM, Stefan Adamswrote: > Is there a maximum length that I should respect for the args being passed > to a Minion task? > > I have a job that has an arg of 2MB and the state is failed with a result > of "Worker went away". Using the Minion Admin UI, I clicked on the > "failed" tasks and it hangs my entire Ubuntu 16.04.3 system! I've had to > reboot three times now. I have since manually removed that specific job > from the database and now, so far, all seems to be well, and I can also > browse failed tasks (there are now 0). > > If I shouldn't be passing 2MB via args, should I instead save the 2MB to > disk and let the job remove the file from disk when the job has finished > successfully? > > -- > You received this message because you are subscribed to the Google Groups > "Mojolicious" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to mojolicious+unsubscr...@googlegroups.com. > To post to this group, send email to mojolicious@googlegroups.com. > Visit this group at https://groups.google.com/group/mojolicious. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Mojolicious" group. To unsubscribe from this group and stop receiving emails from it, send an email to mojolicious+unsubscr...@googlegroups.com. To post to this group, send email to mojolicious@googlegroups.com. Visit this group at https://groups.google.com/group/mojolicious. For more options, visit https://groups.google.com/d/optout.
[Mojolicious] Minion args
Is there a maximum length that I should respect for the args being passed to a Minion task? I have a job that has an arg of 2MB and the state is failed with a result of "Worker went away". Using the Minion Admin UI, I clicked on the "failed" tasks and it hangs my entire Ubuntu 16.04.3 system! I've had to reboot three times now. I have since manually removed that specific job from the database and now, so far, all seems to be well, and I can also browse failed tasks (there are now 0). If I shouldn't be passing 2MB via args, should I instead save the 2MB to disk and let the job remove the file from disk when the job has finished successfully? -- You received this message because you are subscribed to the Google Groups "Mojolicious" group. To unsubscribe from this group and stop receiving emails from it, send an email to mojolicious+unsubscr...@googlegroups.com. To post to this group, send email to mojolicious@googlegroups.com. Visit this group at https://groups.google.com/group/mojolicious. For more options, visit https://groups.google.com/d/optout.