[jira] [Commented] (CB-13540) cordova-fetch npm install --production clobbers node_modules, causes workflow issues
[ https://issues.apache.org/jira/browse/CB-13540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16246030#comment-16246030 ] Alex Matchneer commented on CB-13540: - OK, getting closer: we use `yarn` (npm alternative that is generally faster and more reliable) to install dependencies. I guess we can use npm if we must, but the rest of our tooling is yarn-based and we figured we'd use it for cordova too. I still haven't pinpointed the exact issue, but I can reproduce it now: 1. rm -rf node_modules 2. yarn install 3. cordova add platform android --verbose {code} cordova :: cordova platform add android --verbose No scripts found for hook "before_platform_add". No version supplied. Retrieving version from config.xml... Grabbing pinned version. Using cordova-fetch for cordova-android@~6.3.0 saving Running command: npm install cordova-android@~6.3.0 --production --save Command finished with error code 0: npm install,cordova-android@~6.3.0,--production,--save Error: Failed to fetch platform cordova-android@~6.3.0 Probably this is either a connection problem, or platform spec is incorrect. Check your connection and platform name/version/URL. Failed to get absolute path to installed module {code} The npm install that internally happens is restructuring the contents of node modules from what yarn installed; in particular, yarn tries to install modules at the top level, including devDependency modules, and I believe the --production flag passed to npm install is causing a bunch of those modules to be deleted in the middle of script execution. Here is a diff on my node_modules directory before and after the first `cordova platform add android`: https://gist.github.com/machty/ae683968198a68359423f06a1339aacd So, I guess the easiest answer is "just use npm and don't try and be clever with yarn", but it seems non-ideal for Cordova to restructure `node_modules` mid-execution, and it makes the integration with other workflows a bit more brittle. I'm gonna continue poking around but wanted to share my findings thus far. > cordova-fetch npm install --production clobbers node_modules, causes workflow > issues > > > Key: CB-13540 > URL: https://issues.apache.org/jira/browse/CB-13540 > Project: Apache Cordova > Issue Type: Bug > Components: cordova-fetch >Reporter: Alex Matchneer >Priority: Minor > Labels: fetch, node_modules, npm > > I have tried and failed to provide a reproduction of this issue and have > hence written/deleted/rewritten this issue many times without submitting, but > now I'm going to just submit it, hope for feedback, and continue to try and > reproduce: > Basically, since this issue (https://issues.apache.org/jira/browse/CB-13303), > cordova-fetch uses the `--production` flag with `npm install`, which tells > npm not to install any "devDependencies". > This seems like a sound approach, but for reasons that I can't reliably > reproduced/put my finger on, it seems like this flag is also aggressively > removing packages from `node_modules`, causing the following workflow issues: > In my cordova project, I've added `cordova` as a dependency to package.json > and encourage my teammates to use `./node_modules/.bin/cordova` rather than > some globally installed `cordova` executable so that all dependencies are > locked down and we don't have to fight the dependency hell of global CLI > dependencies + local package.json dependencies. > Now, when I freshly clone a repo, do an npm install, and then run > `./node_modules/.bin/cordova prepare`, it will non-deterministically fail > with the following output: > {code} > Discovered platform "ios@^4.5.3" in config.xml or package.json. Adding it to > the project > Using cordova-fetch for cordova-ios@^4.5.3 > Error: Failed to fetch platform cordova-ios@^4.5.3 > Probably this is either a connection problem, or platform spec is incorrect. > Check your connection and platform name/version/URL. > Failed to get absolute path to installed module > Error: No platforms added to this project. Please use `cordova platform add > `. > {code} > And when I look at node_modules, most of my dependencies have been deleted, > and in particular, there's no more `node_modules/.bin/cordova` executable. > My best guess is that the `npm install --production --no-save` that > cordova-fetch internally does is nuking a bunch of dependencies considering > "non-production", including the cli folder itself. The remedy is to use a > globally installed cordova CLI executable, which somehow avoids these issues. > Both my local and global cordova executables are 7.7.4. > I will continue to try and provide a consistent reproduction of this issue; > I've tried and failed many times and I'm starting to pu
[jira] [Commented] (CB-13540) cordova-fetch npm install --production clobbers node_modules, causes workflow issues
[ https://issues.apache.org/jira/browse/CB-13540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16245935#comment-16245935 ] Alex Matchneer commented on CB-13540: - npm -v 5.5.1 node -v v7.7.4 cordova -v 7.1.0 I just ran into a similar issue, this time when using `cordova platform add android`, and got the following output: {code} cordova platform add android Using cordova-fetch for cordova-android@~6.3.0 Error: Failed to fetch platform cordova-android@~6.3.0 Probably this is either a connection problem, or platform spec is incorrect. Check your connection and platform name/version/URL. Failed to get absolute path to installed module {code} And then when I ran it again, it ran to completion. Continuing to poke around and hopefully find a scenario that will reproduce. > cordova-fetch npm install --production clobbers node_modules, causes workflow > issues > > > Key: CB-13540 > URL: https://issues.apache.org/jira/browse/CB-13540 > Project: Apache Cordova > Issue Type: Bug > Components: cordova-fetch >Reporter: Alex Matchneer >Priority: Minor > Labels: fetch, node_modules, npm > > I have tried and failed to provide a reproduction of this issue and have > hence written/deleted/rewritten this issue many times without submitting, but > now I'm going to just submit it, hope for feedback, and continue to try and > reproduce: > Basically, since this issue (https://issues.apache.org/jira/browse/CB-13303), > cordova-fetch uses the `--production` flag with `npm install`, which tells > npm not to install any "devDependencies". > This seems like a sound approach, but for reasons that I can't reliably > reproduced/put my finger on, it seems like this flag is also aggressively > removing packages from `node_modules`, causing the following workflow issues: > In my cordova project, I've added `cordova` as a dependency to package.json > and encourage my teammates to use `./node_modules/.bin/cordova` rather than > some globally installed `cordova` executable so that all dependencies are > locked down and we don't have to fight the dependency hell of global CLI > dependencies + local package.json dependencies. > Now, when I freshly clone a repo, do an npm install, and then run > `./node_modules/.bin/cordova prepare`, it will non-deterministically fail > with the following output: > {code} > Discovered platform "ios@^4.5.3" in config.xml or package.json. Adding it to > the project > Using cordova-fetch for cordova-ios@^4.5.3 > Error: Failed to fetch platform cordova-ios@^4.5.3 > Probably this is either a connection problem, or platform spec is incorrect. > Check your connection and platform name/version/URL. > Failed to get absolute path to installed module > Error: No platforms added to this project. Please use `cordova platform add > `. > {code} > And when I look at node_modules, most of my dependencies have been deleted, > and in particular, there's no more `node_modules/.bin/cordova` executable. > My best guess is that the `npm install --production --no-save` that > cordova-fetch internally does is nuking a bunch of dependencies considering > "non-production", including the cli folder itself. The remedy is to use a > globally installed cordova CLI executable, which somehow avoids these issues. > Both my local and global cordova executables are 7.7.4. > I will continue to try and provide a consistent reproduction of this issue; > I've tried and failed many times and I'm starting to pull my hair out. I just > wanted to officially post this issue in case other people have seen it, have > advice, or have had better luck reproducing. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@cordova.apache.org For additional commands, e-mail: issues-h...@cordova.apache.org
[jira] [Comment Edited] (CB-13503) Problems with using plugins from local file system
[ https://issues.apache.org/jira/browse/CB-13503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16241937#comment-16241937 ] Alex Matchneer edited comment on CB-13503 at 11/7/17 12:38 PM: --- I am also seeing this issue on 7.1.0 and have had to resort to the same hacks/scripts as Marco. was (Author: machty): I am also seeing this issue on 7.1.0 and have had to resort to the same hacks as Marco. > Problems with using plugins from local file system > -- > > Key: CB-13503 > URL: https://issues.apache.org/jira/browse/CB-13503 > Project: Apache Cordova > Issue Type: Bug > Components: AllPlatforms, cordova-fetch >Affects Versions: cordova@7.1.0 >Reporter: Mark Veenstra >Priority: Blocker > > I am struggling to get up and running with {{cordova@7.1.0}} sometime now. > Maybe I am doing something wrong here but I can't figure it out. It seems to > me something is going wrong with locally defined plugins that needs to be > installed and the way it is saved into {{package.json}}. > So to get upgraded I first installed the latest version {{npm install -g > cordova@7.1.0}}. Then I did a {{cordova prepare}}. > This {{prepare}} failed, after [some digging and > trying|https://github.com/dpa99c/cordova-diagnostic-plugin/issues/245] it > seemed that some locally plugins that needs to be installed into the project > are interrupting other plugins. So I needed to disable the {{cordova-fetch}}. > I set this globally with {{cordova config set fetch false}}. > After this config change the {{codova prepare}} command went all fine. So I > thought this is it. > But when I tried to let my project be build by my CI server it failed on > installating NPM packages. > Because a {{npm install}} command will install the dependencies defined in > {{package.json}} it will install the cordova plugins also. But in the > beginning my plugins are in there like this: > {code} > { > "name": "myApp", > "version": "1.0.0", > "description": "myApp description", > "dependencies": { > "cordova-android": "6.3.0", > "cordova-browser": "5.0.1", > "cordova-ios": "4.5.2", > "cordova-plugin-myplugin": "resources/cordova-plugin-myplugin", > "cordova-plugin-myplugin-locker": > "resources/cordova-plugin-myplugin-locker" > }, > "devDependencies": { > }, > "scripts": { > }, > "cordova": { > "platforms": [ > "browser", > "android", > "ios" > ], > "plugins": { > "cordova-plugin-myplugin": {}, > "cordova-plugin-myplugin-locker": {} > } > } > } > {code} > With above {{npm install}} fails and tries to get the dependecies from git, > which is incorrect. So I search the {{package.json}} definition and they say > I should put {{file:}} in front of it, like this: > {code} > { > "name": "myApp", > "version": "1.0.0", > "description": "myApp description", > "dependencies": { > "cordova-android": "6.3.0", > "cordova-browser": "5.0.1", > "cordova-ios": "4.5.2", > "cordova-plugin-myplugin": "file:resources/cordova-plugin-myplugin", > "cordova-plugin-myplugin-locker": > "file:resources/cordova-plugin-myplugin-locker" > }, > "devDependencies": { > }, > "scripts": { > }, > "cordova": { > "platforms": [ > "browser", > "android", > "ios" > ], > "plugins": { > "cordova-plugin-myplugin": {}, > "cordova-plugin-myplugin-locker": {} > } > } > } > {code} > Now NPM finds the dependecies OK, but now when I do a {{cordova prepare}} > again it throws an error that the plugin can't be installed from > {{file:resources/cordova-plugin-myplugin}}. > So I get stuck into different needs for different tools. Any ideas? -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@cordova.apache.org For additional commands, e-mail: issues-h...@cordova.apache.org
[jira] [Commented] (CB-13503) Problems with using plugins from local file system
[ https://issues.apache.org/jira/browse/CB-13503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16241937#comment-16241937 ] Alex Matchneer commented on CB-13503: - I am also seeing this issue on 7.1.0 and have had to resort to the same hacks as Marco. > Problems with using plugins from local file system > -- > > Key: CB-13503 > URL: https://issues.apache.org/jira/browse/CB-13503 > Project: Apache Cordova > Issue Type: Bug > Components: AllPlatforms, cordova-fetch >Affects Versions: cordova@7.1.0 >Reporter: Mark Veenstra >Priority: Blocker > > I am struggling to get up and running with {{cordova@7.1.0}} sometime now. > Maybe I am doing something wrong here but I can't figure it out. It seems to > me something is going wrong with locally defined plugins that needs to be > installed and the way it is saved into {{package.json}}. > So to get upgraded I first installed the latest version {{npm install -g > cordova@7.1.0}}. Then I did a {{cordova prepare}}. > This {{prepare}} failed, after [some digging and > trying|https://github.com/dpa99c/cordova-diagnostic-plugin/issues/245] it > seemed that some locally plugins that needs to be installed into the project > are interrupting other plugins. So I needed to disable the {{cordova-fetch}}. > I set this globally with {{cordova config set fetch false}}. > After this config change the {{codova prepare}} command went all fine. So I > thought this is it. > But when I tried to let my project be build by my CI server it failed on > installating NPM packages. > Because a {{npm install}} command will install the dependencies defined in > {{package.json}} it will install the cordova plugins also. But in the > beginning my plugins are in there like this: > {code} > { > "name": "myApp", > "version": "1.0.0", > "description": "myApp description", > "dependencies": { > "cordova-android": "6.3.0", > "cordova-browser": "5.0.1", > "cordova-ios": "4.5.2", > "cordova-plugin-myplugin": "resources/cordova-plugin-myplugin", > "cordova-plugin-myplugin-locker": > "resources/cordova-plugin-myplugin-locker" > }, > "devDependencies": { > }, > "scripts": { > }, > "cordova": { > "platforms": [ > "browser", > "android", > "ios" > ], > "plugins": { > "cordova-plugin-myplugin": {}, > "cordova-plugin-myplugin-locker": {} > } > } > } > {code} > With above {{npm install}} fails and tries to get the dependecies from git, > which is incorrect. So I search the {{package.json}} definition and they say > I should put {{file:}} in front of it, like this: > {code} > { > "name": "myApp", > "version": "1.0.0", > "description": "myApp description", > "dependencies": { > "cordova-android": "6.3.0", > "cordova-browser": "5.0.1", > "cordova-ios": "4.5.2", > "cordova-plugin-myplugin": "file:resources/cordova-plugin-myplugin", > "cordova-plugin-myplugin-locker": > "file:resources/cordova-plugin-myplugin-locker" > }, > "devDependencies": { > }, > "scripts": { > }, > "cordova": { > "platforms": [ > "browser", > "android", > "ios" > ], > "plugins": { > "cordova-plugin-myplugin": {}, > "cordova-plugin-myplugin-locker": {} > } > } > } > {code} > Now NPM finds the dependecies OK, but now when I do a {{cordova prepare}} > again it throws an error that the plugin can't be installed from > {{file:resources/cordova-plugin-myplugin}}. > So I get stuck into different needs for different tools. Any ideas? -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@cordova.apache.org For additional commands, e-mail: issues-h...@cordova.apache.org
[jira] [Created] (CB-13540) cordova-fetch npm install --production clobbers node_modules, causes workflow issues
Alex Matchneer created CB-13540: --- Summary: cordova-fetch npm install --production clobbers node_modules, causes workflow issues Key: CB-13540 URL: https://issues.apache.org/jira/browse/CB-13540 Project: Apache Cordova Issue Type: Bug Components: cordova-fetch Reporter: Alex Matchneer Priority: Minor I have tried and failed to provide a reproduction of this issue and have hence written/deleted/rewritten this issue many times without submitting, but now I'm going to just submit it, hope for feedback, and continue to try and reproduce: Basically, since this issue (https://issues.apache.org/jira/browse/CB-13303), cordova-fetch uses the `--production` flag with `npm install`, which tells npm not to install any "devDependencies". This seems like a sound approach, but for reasons that I can't reliably reproduced/put my finger on, it seems like this flag is also aggressively removing packages from `node_modules`, causing the following workflow issues: In my cordova project, I've added `cordova` as a dependency to package.json and encourage my teammates to use `./node_modules/.bin/cordova` rather than some globally installed `cordova` executable so that all dependencies are locked down and we don't have to fight the dependency hell of global CLI dependencies + local package.json dependencies. Now, when I freshly clone a repo, do an npm install, and then run `./node_modules/.bin/cordova prepare`, it will non-deterministically fail with the following output: {code} Discovered platform "ios@^4.5.3" in config.xml or package.json. Adding it to the project Using cordova-fetch for cordova-ios@^4.5.3 Error: Failed to fetch platform cordova-ios@^4.5.3 Probably this is either a connection problem, or platform spec is incorrect. Check your connection and platform name/version/URL. Failed to get absolute path to installed module Error: No platforms added to this project. Please use `cordova platform add `. {code} And when I look at node_modules, most of my dependencies have been deleted, and in particular, there's no more `node_modules/.bin/cordova` executable. My best guess is that the `npm install --production --no-save` that cordova-fetch internally does is nuking a bunch of dependencies considering "non-production", including the cli folder itself. The remedy is to use a globally installed cordova CLI executable, which somehow avoids these issues. Both my local and global cordova executables are 7.7.4. I will continue to try and provide a consistent reproduction of this issue; I've tried and failed many times and I'm starting to pull my hair out. I just wanted to officially post this issue in case other people have seen it, have advice, or have had better luck reproducing. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@cordova.apache.org For additional commands, e-mail: issues-h...@cordova.apache.org