I tested and confirmed that the files are not being deleted. Mind opening a bug for that?
For the failed chunk uploads, httpie is timing out. You can set a higher timeout (like you did) or use smaller chunks. I looked into why artifact creation is failing for files < 2GB. The reason is that it's taking too long to calculate the checksums. There are 6 checksum types and each one takes about 4-8 seconds from the command line in my test environment. Calculating the digests in Python seems to add about 1-2 seconds. The default timeout in gunicorn is 30 seconds: Jul 05 14:21:56 pulp3 gunicorn[13691]: [2019-07-05 14:21:56 +0000] [13691] [CRITICAL] WORKER TIMEOUT (pid:29843) Jul 05 14:21:57 pulp3 gunicorn[13691]: [2019-07-05 14:21:57 +0000] [30031] [INFO] Booting worker with pid: 30031 You can raise this timeout or also you can pass in the checksums when creating the artifact[0]. I think the best solution though might be to make artifact creation a background task. [0] http POST :24817/pulp/api/v3/artifacts/ upload=$UPLOAD sha256=abc... David On Fri, Jul 5, 2019 at 9:08 AM Juan Cabrera <[email protected]> wrote: > Hi David > > This morning I made a test sequence and open a ticket > > https://pulp.plan.io/issues/5087 > > Yes, I see that the update file is deleted when the artifact is created. > > When I get the artifact creation error, the upload HREF is not deleted, > which could be normal as there was a error. > > But when I clean all the server uploads using API > > for u in $(http $PORT/pulp/api/v3/uploads/ | jq -r '.results[] | ._href'); > do > echo $u > http DELETE $PORT$u > done > > The files are not deleted. > > Juan > On 5/07/19 12:44, David Davis wrote: > > Juan, > > There is in fact a 2GB limit currently on artifact size. I consider this a > bug and I filed this issue: > > https://pulp.plan.io/issues/4998 > > The file in /var/lib/pulp/upload should be deleted once it's imported as > an artifact. I'm guessing it's maybe not happening since the server is > throwing an error. > > David > > > On Thu, Jul 4, 2019 at 12:31 PM Juan Cabrera <[email protected]> > wrote: > >> Hi, >> >> In my previous mail I forget to say that I updated the Pulp version to >> 3.0.0rc3 >> >> pulp_source_dir: "git+https://github.com/pulp/[email protected]"; >> pulp_plugin_source_dir: >> "git+https://github.com/pulp/[email protected]"; >> pulp_install_plugins: >> pulp-rpm: >> app_label: "rpm" >> source_dir: "git+https://github.com/pulp/[email protected]"; >> >> Juan >> >> _______________________________________________ >> Pulp-list mailing list >> [email protected] >> https://www.redhat.com/mailman/listinfo/pulp-list > > -- > > Juan CABRERA > Correspondant informatique > Département de Mathématiques > > T. 081724919 > [email protected] > http://staff.unamur.be/jbcabrer > > Université de Namur ASBL > Rue de Bruxelles 61 - 5000 Namur > Belgique > > Let’s respect the environment together. > Only print this message if necessary! >
_______________________________________________ Pulp-list mailing list [email protected] https://www.redhat.com/mailman/listinfo/pulp-list
