Jira (PUP-9813) Performance of unpacking modules is extremely slow
Title: Message Title
Chris Suszynski updated an issue
Puppet / PUP-9813
Performance of unpacking modules is extremely slow
Change By:
Chris Suszynski
*Puppet Version: any* *Puppet Server Version: any* *OS Name/Version: any*Installing modules from Puppet Forge is extremelly extremely slow. I have found a culprit. It is a{{Puppet::ModuleTool::Tar::Mini}}implementation.The unpack is * 100x times * slower then executing a \ {{tar xzvf}} system command . ! *Steps to reproduce:*Install a module from Puppet Forge and measure a time:{{time puppet module install puppetlabs-stdlib --version 4.25.1}}Then execute:{{time bash -c 'curl [https://forge.puppet.com/v3/files/puppetlabs-stdlib-4.25.1.tar.gz] -o puppetlabs-stdlib-4.25.1.tar.gz && tar xzvf puppetlabs-stdlib-4.25.1.tar.gz'}}*Desired Behavior:*Extraction of downloaded modules should be about the same performance as system tar xzvf command. It might be at most 2x times slower then system command, but preferably should be about the same. * Temporal solution: * To install modules faster i propose to switch to Gnu implementation as a first choice. Al least until Minitar can be much faster.
Add Comment
This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)
Jira (PUP-9813) Performance of unpacking modules is extremely slow
Title: Message Title
Chris Suszynski updated an issue
Puppet / PUP-9813
Performance of unpacking modules is extremely slow
Change By:
Chris Suszynski
*Puppet Version: any* *Puppet Server Version: any* *OS Name/Version: any*Installing modules from Puppet Forge is extremelly slow. I have found a culprit. It is a{{Puppet::ModuleTool::Tar::Mini}}implementation.The unpack is 100x times slower then executing a \ {{tar xzvf }}system command.*Steps to reproduce:*Install a module from Puppet Forge and measure a time:{{time puppet module install puppetlabs-stdlib --version 4.25.1}}Then execute:{{time bash -c 'curl [https://forge.puppet.com/v3/files/puppetlabs-stdlib-4.25.1.tar.gz] -o puppetlabs-stdlib-4.25.1.tar.gz && tar xzvf puppetlabs-stdlib-4.25.1.tar.gz'}}*Desired Behavior:*Extraction of downloaded modules should be about the same performance as system tar xzvf command. It might be at most 2x times slower then system command, but preferably should be about the same. Temporal solution:To install modules faster i propose to switch to Gnu implementation as a first choice. Al least until Minitar can be much faster.
Add Comment
This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)
Jira (PUP-9813) Performance of unpacking modules is extremely slow
Title: Message Title
Chris Suszynski created an issue
Puppet / PUP-9813
Performance of unpacking modules is extremely slow
Issue Type:
Bug
Affects Versions:
PUP 6.6.0, PUP 5.5.14, PUP 4.10.12
Assignee:
Unassigned
Components:
Modules
Created:
2019/07/03 3:53 PM
Priority:
Normal
Reporter:
Chris Suszynski
Original Estimate:
2 days
Remaining Estimate:
2 days
Puppet Version: any Puppet Server Version: any OS Name/Version: any Installing modules from Puppet Forge is extremelly slow. I have found a culprit. It is a Puppet::ModuleTool::Tar::Mini implementation. The unpack is 100x times slower then executing a {{tar xzvf }}system command. Steps to reproduce: Install a module from Puppet Forge and measure a time: time puppet module install puppetlabs-stdlib --version 4.25.1 Then execute: time bash -c 'curl https://forge.puppet.com/v3/files/puppetlabs-stdlib-4.25.1.tar.gz -o puppetlabs-stdlib-4.25.1.tar.gz && tar xzvf puppetlabs-stdlib-4.25.1.tar.gz' Desired Behavior: Extraction of downloaded modules should be about the same performance as system tar xzvf command. It might be at most 2x times slower then system command, but preferably should be about the same.
Jira (PUP-9787) Unintentional secret reveal while installing modules
Title: Message Title
Chris Suszynski updated an issue
Puppet / PUP-9787
Unintentional secret reveal while installing modules
Change By:
Chris Suszynski
*Puppet Version:* any *Puppet Server Version:* any *OS Name/Version:* any*Actual Behavior:*Puppet Forge is public, and downloading modules don't require authentication. However there are some repositories that can hold modules and require authentication to connect.Those repositories are: * [Artifactory|https://www.jfrog.com/confluence/display/RTF/Puppet+Repositories] (live) * [Nexus (emerging) |https://github.com/wavesoftware/nexus-repository-puppet] (emerging)When installing modules from those repositories user is forced to set his credentials in plain text in URI supported form, for ex.:{noformat} https://admin:s3c...@pkg.acmecorp.com/repository/puppet{noformat} Installing modules with similar module repository being set, reveals those credentials. In fact it's done each time a module is installed, with a message: {noformat} Notice: Preparing to install into /home/jdoe/.puppetlabs/etc/code/modules ...Notice: Downloading from https://jdoe:s3c...@pkg.acmecorp.com/repository/puppet ...{noformat} *Desired Behavior:*Puppet should mask password if given, like this: {noformat} Notice: Preparing to install into /home/jdoe/.puppetlabs/etc/code/modules ...Notice: Downloading from https://jdoe:***@pkg.acmecorp.com/repository/puppet ...{noformat}
Add Comment
This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)
Jira (PUP-9787) Unintentional secret reveal while installing modules
Title: Message Title Chris Suszynski updated an issue Puppet / PUP-9787 Unintentional secret reveal while installing modules Change By: Chris Suszynski Affects Version/s: PUP 4.10.0 Affects Version/s: PUP 4.10.12 Affects Version/s: PUP 5.5.14 Affects Version/s: PUP 6.4.2 Add Comment This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to
Jira (PUP-9787) Unintentional secret reveal while installing modules
Title: Message Title Chris Suszynski updated an issue Puppet / PUP-9787 Unintentional secret reveal while installing modules Change By: Chris Suszynski Acceptance Criteria: * Puppet should not reveal sensitive information while isntalling modules. * Usit Unit tests are added that assure that marking masking is done Add Comment This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.313276.1560984349000.54345.1560984421359%40Atlassian.JIRA. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-9787) Unintentional secret reveal while installing modules
Title: Message Title Chris Suszynski created an issue Puppet / PUP-9787 Unintentional secret reveal while installing modules Issue Type: Bug Affects Versions: PUP 4.10.0 Assignee: Unassigned Components: Modules Created: 2019/06/19 3:45 PM Priority: Minor Reporter: Chris Suszynski Puppet Version: any Puppet Server Version: any OS Name/Version: any Actual Behavior: Puppet Forge is public, and downloading modules don't require authentication. However there are some repositories that can hold modules and require authentication to connect. Those repositories are: Artifactory (live) Nexus (emerging) (emerging) When installing modules from those repositories user is forced to set his credentials in plain text in URI supported form, for ex.: https://admin:s3c...@pkg.acmecorp.com/repository/puppet Installing modules with similar module repository being set, reveals those credentials. In fact it's done each time a module is installed, with a message:
