Am 13/05/2022 um 15:49 schrieb Matthias Heiserer: > == The problem > Upload of files smaller than ~16kb failed. > This was because the code assumed that it would be called > several times, and each time would do a certain action. > When the whole file was in the buffer, this failed because > the function would try parssing the first part in the payload and > then return, instead of parsing the rest of the available data. > > == Why not just modifying the current code a bit? > The code had a lot of nested control statements and a > non-intuitive control flow (phase 0->2->1->1->1 and so on). > > The way the phases and buffer content were checked made it > rather difficult to just fix a few lines. > > == What was changed > * Part headers are parsed with a single regex line each, > which improves code readability. > > * Parsing the content is done in order, so even if the whole data is in the > buffer, > it can be read in one go. Files of arbitrary sizes can be uploaded. > > == Tested with > * Uploaded 0B, 1B, 14KB, 16KB, 1GB, 10GB, 20GB files > > * Tested with all checksums and without > > * Tested on firefox, chromium, and pvesh > > I didn't do any fuzzing or automated upload testing. > > == Drawbacks & Potential issues > * Part headers are hardcoded, adding new ones requries modifying this file > > == does not fix > * upload can still time out > > Signed-off-by: Matthias Heiserer <m.heise...@proxmox.com> > --- > > Note: > Regarding trim, I forgot to answer the mail. > Trim is imo a good name, as several languagues (e.g. rust, javascript) > use trim to mean mean "remove all whitespace, including newlines and such". > I can send a v3 if that's a problem. > > Changes from v1: > * fix whitespace in separate patch > * move trim into inline closure > * correctly call trim > * replace [^\S] with \S in regexes > * improve trim regex: don't replace string > * check for phase 1 once > * remove regex comment > > src/PVE/APIServer/AnyEvent.pm | 146 ++++++++++++++++++---------------- > 1 file changed, 76 insertions(+), 70 deletions(-) > >
applied, with slightly reworking the commit messages subject and Daniel's T-b tag, thanks to both! Note that I made some follow ups trying to improve a few things of your change and the existing code. E.g.: $string in trim wasn't used anywhere, same for $eof in phase 2. The content-disposition extraction could be factored in a small closure to reduce code size and (IMO) legibility, and some other smaller style/formatting stuff. None of that was a blocker, and due to the age of this series (sorry for that!) I really did not want to bother for a v3 and applied changes myself. I re-tested it with various sizes and functionality (checksum), but an additional pair of eyes would be appreciated to ensure no regression snuck in. _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel