[jira] [Commented] (CB-13540) cordova-fetch npm install --production clobbers node_modules, causes workflow issues
[
https://issues.apache.org/jira/browse/CB-13540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246030#comment-16246030
]
Alex Matchneer commented on CB-13540:
-
OK, getting closer: we use `yarn` (npm alternative that is generally faster and
more reliable) to install dependencies. I guess we can use npm if we must, but
the rest of our tooling is yarn-based and we figured we'd use it for cordova
too.
I still haven't pinpointed the exact issue, but I can reproduce it now:
1. rm -rf node_modules
2. yarn install
3. cordova add platform android --verbose
{code}
cordova :: cordova platform add android --verbose
No scripts found for hook "before_platform_add".
No version supplied. Retrieving version from config.xml...
Grabbing pinned version.
Using cordova-fetch for cordova-android@~6.3.0
saving
Running command: npm install cordova-android@~6.3.0 --production --save
Command finished with error code 0: npm
install,cordova-android@~6.3.0,--production,--save
Error: Failed to fetch platform cordova-android@~6.3.0
Probably this is either a connection problem, or platform spec is incorrect.
Check your connection and platform name/version/URL.
Failed to get absolute path to installed module
{code}
The npm install that internally happens is restructuring the contents of node
modules from what yarn installed; in particular, yarn tries to install modules
at the top level, including devDependency modules, and I believe the
--production flag passed to npm install is causing a bunch of those modules to
be deleted in the middle of script execution. Here is a diff on my node_modules
directory before and after the first `cordova platform add android`:
https://gist.github.com/machty/ae683968198a68359423f06a1339aacd
So, I guess the easiest answer is "just use npm and don't try and be clever
with yarn", but it seems non-ideal for Cordova to restructure `node_modules`
mid-execution, and it makes the integration with other workflows a bit more
brittle.
I'm gonna continue poking around but wanted to share my findings thus far.
> cordova-fetch npm install --production clobbers node_modules, causes workflow
> issues
>
>
> Key: CB-13540
> URL: https://issues.apache.org/jira/browse/CB-13540
> Project: Apache Cordova
> Issue Type: Bug
> Components: cordova-fetch
>Reporter: Alex Matchneer
>Priority: Minor
> Labels: fetch, node_modules, npm
>
> I have tried and failed to provide a reproduction of this issue and have
> hence written/deleted/rewritten this issue many times without submitting, but
> now I'm going to just submit it, hope for feedback, and continue to try and
> reproduce:
> Basically, since this issue (https://issues.apache.org/jira/browse/CB-13303),
> cordova-fetch uses the `--production` flag with `npm install`, which tells
> npm not to install any "devDependencies".
> This seems like a sound approach, but for reasons that I can't reliably
> reproduced/put my finger on, it seems like this flag is also aggressively
> removing packages from `node_modules`, causing the following workflow issues:
> In my cordova project, I've added `cordova` as a dependency to package.json
> and encourage my teammates to use `./node_modules/.bin/cordova` rather than
> some globally installed `cordova` executable so that all dependencies are
> locked down and we don't have to fight the dependency hell of global CLI
> dependencies + local package.json dependencies.
> Now, when I freshly clone a repo, do an npm install, and then run
> `./node_modules/.bin/cordova prepare`, it will non-deterministically fail
> with the following output:
> {code}
> Discovered platform "ios@^4.5.3" in config.xml or package.json. Adding it to
> the project
> Using cordova-fetch for cordova-ios@^4.5.3
> Error: Failed to fetch platform cordova-ios@^4.5.3
> Probably this is either a connection problem, or platform spec is incorrect.
> Check your connection and platform name/version/URL.
> Failed to get absolute path to installed module
> Error: No platforms added to this project. Please use `cordova platform add
> `.
> {code}
> And when I look at node_modules, most of my dependencies have been deleted,
> and in particular, there's no more `node_modules/.bin/cordova` executable.
> My best guess is that the `npm install --production --no-save` that
> cordova-fetch internally does is nuking a bunch of dependencies considering
> "non-production", including the cli folder itself. The remedy is to use a
> globally installed cordova CLI executable, which somehow avoids these issues.
> Both my local and global cordova executables are 7.7.4.
> I will continue to try and provide a consistent reproduction of this issue;
> I've tried and failed many times and I'm starting to pull my hair out.
[jira] [Commented] (CB-13540) cordova-fetch npm install --production clobbers node_modules, causes workflow issues
[
https://issues.apache.org/jira/browse/CB-13540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245935#comment-16245935
]
Alex Matchneer commented on CB-13540:
-
npm -v
5.5.1
node -v
v7.7.4
cordova -v
7.1.0
I just ran into a similar issue, this time when using `cordova platform add
android`, and got the following output:
{code}
cordova platform add android
Using cordova-fetch for cordova-android@~6.3.0
Error: Failed to fetch platform cordova-android@~6.3.0
Probably this is either a connection problem, or platform spec is incorrect.
Check your connection and platform name/version/URL.
Failed to get absolute path to installed module
{code}
And then when I ran it again, it ran to completion. Continuing to poke around
and hopefully find a scenario that will reproduce.
> cordova-fetch npm install --production clobbers node_modules, causes workflow
> issues
>
>
> Key: CB-13540
> URL: https://issues.apache.org/jira/browse/CB-13540
> Project: Apache Cordova
> Issue Type: Bug
> Components: cordova-fetch
>Reporter: Alex Matchneer
>Priority: Minor
> Labels: fetch, node_modules, npm
>
> I have tried and failed to provide a reproduction of this issue and have
> hence written/deleted/rewritten this issue many times without submitting, but
> now I'm going to just submit it, hope for feedback, and continue to try and
> reproduce:
> Basically, since this issue (https://issues.apache.org/jira/browse/CB-13303),
> cordova-fetch uses the `--production` flag with `npm install`, which tells
> npm not to install any "devDependencies".
> This seems like a sound approach, but for reasons that I can't reliably
> reproduced/put my finger on, it seems like this flag is also aggressively
> removing packages from `node_modules`, causing the following workflow issues:
> In my cordova project, I've added `cordova` as a dependency to package.json
> and encourage my teammates to use `./node_modules/.bin/cordova` rather than
> some globally installed `cordova` executable so that all dependencies are
> locked down and we don't have to fight the dependency hell of global CLI
> dependencies + local package.json dependencies.
> Now, when I freshly clone a repo, do an npm install, and then run
> `./node_modules/.bin/cordova prepare`, it will non-deterministically fail
> with the following output:
> {code}
> Discovered platform "ios@^4.5.3" in config.xml or package.json. Adding it to
> the project
> Using cordova-fetch for cordova-ios@^4.5.3
> Error: Failed to fetch platform cordova-ios@^4.5.3
> Probably this is either a connection problem, or platform spec is incorrect.
> Check your connection and platform name/version/URL.
> Failed to get absolute path to installed module
> Error: No platforms added to this project. Please use `cordova platform add
> `.
> {code}
> And when I look at node_modules, most of my dependencies have been deleted,
> and in particular, there's no more `node_modules/.bin/cordova` executable.
> My best guess is that the `npm install --production --no-save` that
> cordova-fetch internally does is nuking a bunch of dependencies considering
> "non-production", including the cli folder itself. The remedy is to use a
> globally installed cordova CLI executable, which somehow avoids these issues.
> Both my local and global cordova executables are 7.7.4.
> I will continue to try and provide a consistent reproduction of this issue;
> I've tried and failed many times and I'm starting to pull my hair out. I just
> wanted to officially post this issue in case other people have seen it, have
> advice, or have had better luck reproducing.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: issues-unsubscr...@cordova.apache.org
For additional commands, e-mail: issues-h...@cordova.apache.org
[jira] [Comment Edited] (CB-13503) Problems with using plugins from local file system
[
https://issues.apache.org/jira/browse/CB-13503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16241937#comment-16241937
]
Alex Matchneer edited comment on CB-13503 at 11/7/17 12:38 PM:
---
I am also seeing this issue on 7.1.0 and have had to resort to the same
hacks/scripts as Marco.
was (Author: machty):
I am also seeing this issue on 7.1.0 and have had to resort to the same hacks
as Marco.
> Problems with using plugins from local file system
> --
>
> Key: CB-13503
> URL: https://issues.apache.org/jira/browse/CB-13503
> Project: Apache Cordova
> Issue Type: Bug
> Components: AllPlatforms, cordova-fetch
>Affects Versions: cordova@7.1.0
>Reporter: Mark Veenstra
>Priority: Blocker
>
> I am struggling to get up and running with {{cordova@7.1.0}} sometime now.
> Maybe I am doing something wrong here but I can't figure it out. It seems to
> me something is going wrong with locally defined plugins that needs to be
> installed and the way it is saved into {{package.json}}.
> So to get upgraded I first installed the latest version {{npm install -g
> cordova@7.1.0}}. Then I did a {{cordova prepare}}.
> This {{prepare}} failed, after [some digging and
> trying|https://github.com/dpa99c/cordova-diagnostic-plugin/issues/245] it
> seemed that some locally plugins that needs to be installed into the project
> are interrupting other plugins. So I needed to disable the {{cordova-fetch}}.
> I set this globally with {{cordova config set fetch false}}.
> After this config change the {{codova prepare}} command went all fine. So I
> thought this is it.
> But when I tried to let my project be build by my CI server it failed on
> installating NPM packages.
> Because a {{npm install}} command will install the dependencies defined in
> {{package.json}} it will install the cordova plugins also. But in the
> beginning my plugins are in there like this:
> {code}
> {
> "name": "myApp",
> "version": "1.0.0",
> "description": "myApp description",
> "dependencies": {
> "cordova-android": "6.3.0",
> "cordova-browser": "5.0.1",
> "cordova-ios": "4.5.2",
> "cordova-plugin-myplugin": "resources/cordova-plugin-myplugin",
> "cordova-plugin-myplugin-locker":
> "resources/cordova-plugin-myplugin-locker"
> },
> "devDependencies": {
> },
> "scripts": {
> },
> "cordova": {
> "platforms": [
> "browser",
> "android",
> "ios"
> ],
> "plugins": {
> "cordova-plugin-myplugin": {},
> "cordova-plugin-myplugin-locker": {}
> }
> }
> }
> {code}
> With above {{npm install}} fails and tries to get the dependecies from git,
> which is incorrect. So I search the {{package.json}} definition and they say
> I should put {{file:}} in front of it, like this:
> {code}
> {
> "name": "myApp",
> "version": "1.0.0",
> "description": "myApp description",
> "dependencies": {
> "cordova-android": "6.3.0",
> "cordova-browser": "5.0.1",
> "cordova-ios": "4.5.2",
> "cordova-plugin-myplugin": "file:resources/cordova-plugin-myplugin",
> "cordova-plugin-myplugin-locker":
> "file:resources/cordova-plugin-myplugin-locker"
> },
> "devDependencies": {
> },
> "scripts": {
> },
> "cordova": {
> "platforms": [
> "browser",
> "android",
> "ios"
> ],
> "plugins": {
> "cordova-plugin-myplugin": {},
> "cordova-plugin-myplugin-locker": {}
> }
> }
> }
> {code}
> Now NPM finds the dependecies OK, but now when I do a {{cordova prepare}}
> again it throws an error that the plugin can't be installed from
> {{file:resources/cordova-plugin-myplugin}}.
> So I get stuck into different needs for different tools. Any ideas?
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: issues-unsubscr...@cordova.apache.org
For additional commands, e-mail: issues-h...@cordova.apache.org
[jira] [Commented] (CB-13503) Problems with using plugins from local file system
[
https://issues.apache.org/jira/browse/CB-13503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16241937#comment-16241937
]
Alex Matchneer commented on CB-13503:
-
I am also seeing this issue on 7.1.0 and have had to resort to the same hacks
as Marco.
> Problems with using plugins from local file system
> --
>
> Key: CB-13503
> URL: https://issues.apache.org/jira/browse/CB-13503
> Project: Apache Cordova
> Issue Type: Bug
> Components: AllPlatforms, cordova-fetch
>Affects Versions: cordova@7.1.0
>Reporter: Mark Veenstra
>Priority: Blocker
>
> I am struggling to get up and running with {{cordova@7.1.0}} sometime now.
> Maybe I am doing something wrong here but I can't figure it out. It seems to
> me something is going wrong with locally defined plugins that needs to be
> installed and the way it is saved into {{package.json}}.
> So to get upgraded I first installed the latest version {{npm install -g
> cordova@7.1.0}}. Then I did a {{cordova prepare}}.
> This {{prepare}} failed, after [some digging and
> trying|https://github.com/dpa99c/cordova-diagnostic-plugin/issues/245] it
> seemed that some locally plugins that needs to be installed into the project
> are interrupting other plugins. So I needed to disable the {{cordova-fetch}}.
> I set this globally with {{cordova config set fetch false}}.
> After this config change the {{codova prepare}} command went all fine. So I
> thought this is it.
> But when I tried to let my project be build by my CI server it failed on
> installating NPM packages.
> Because a {{npm install}} command will install the dependencies defined in
> {{package.json}} it will install the cordova plugins also. But in the
> beginning my plugins are in there like this:
> {code}
> {
> "name": "myApp",
> "version": "1.0.0",
> "description": "myApp description",
> "dependencies": {
> "cordova-android": "6.3.0",
> "cordova-browser": "5.0.1",
> "cordova-ios": "4.5.2",
> "cordova-plugin-myplugin": "resources/cordova-plugin-myplugin",
> "cordova-plugin-myplugin-locker":
> "resources/cordova-plugin-myplugin-locker"
> },
> "devDependencies": {
> },
> "scripts": {
> },
> "cordova": {
> "platforms": [
> "browser",
> "android",
> "ios"
> ],
> "plugins": {
> "cordova-plugin-myplugin": {},
> "cordova-plugin-myplugin-locker": {}
> }
> }
> }
> {code}
> With above {{npm install}} fails and tries to get the dependecies from git,
> which is incorrect. So I search the {{package.json}} definition and they say
> I should put {{file:}} in front of it, like this:
> {code}
> {
> "name": "myApp",
> "version": "1.0.0",
> "description": "myApp description",
> "dependencies": {
> "cordova-android": "6.3.0",
> "cordova-browser": "5.0.1",
> "cordova-ios": "4.5.2",
> "cordova-plugin-myplugin": "file:resources/cordova-plugin-myplugin",
> "cordova-plugin-myplugin-locker":
> "file:resources/cordova-plugin-myplugin-locker"
> },
> "devDependencies": {
> },
> "scripts": {
> },
> "cordova": {
> "platforms": [
> "browser",
> "android",
> "ios"
> ],
> "plugins": {
> "cordova-plugin-myplugin": {},
> "cordova-plugin-myplugin-locker": {}
> }
> }
> }
> {code}
> Now NPM finds the dependecies OK, but now when I do a {{cordova prepare}}
> again it throws an error that the plugin can't be installed from
> {{file:resources/cordova-plugin-myplugin}}.
> So I get stuck into different needs for different tools. Any ideas?
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: issues-unsubscr...@cordova.apache.org
For additional commands, e-mail: issues-h...@cordova.apache.org
[jira] [Created] (CB-13540) cordova-fetch npm install --production clobbers node_modules, causes workflow issues
Alex Matchneer created CB-13540:
---
Summary: cordova-fetch npm install --production clobbers
node_modules, causes workflow issues
Key: CB-13540
URL: https://issues.apache.org/jira/browse/CB-13540
Project: Apache Cordova
Issue Type: Bug
Components: cordova-fetch
Reporter: Alex Matchneer
Priority: Minor
I have tried and failed to provide a reproduction of this issue and have hence
written/deleted/rewritten this issue many times without submitting, but now I'm
going to just submit it, hope for feedback, and continue to try and reproduce:
Basically, since this issue (https://issues.apache.org/jira/browse/CB-13303),
cordova-fetch uses the `--production` flag with `npm install`, which tells npm
not to install any "devDependencies".
This seems like a sound approach, but for reasons that I can't reliably
reproduced/put my finger on, it seems like this flag is also aggressively
removing packages from `node_modules`, causing the following workflow issues:
In my cordova project, I've added `cordova` as a dependency to package.json and
encourage my teammates to use `./node_modules/.bin/cordova` rather than some
globally installed `cordova` executable so that all dependencies are locked
down and we don't have to fight the dependency hell of global CLI dependencies
+ local package.json dependencies.
Now, when I freshly clone a repo, do an npm install, and then run
`./node_modules/.bin/cordova prepare`, it will non-deterministically fail with
the following output:
{code}
Discovered platform "ios@^4.5.3" in config.xml or package.json. Adding it to
the project
Using cordova-fetch for cordova-ios@^4.5.3
Error: Failed to fetch platform cordova-ios@^4.5.3
Probably this is either a connection problem, or platform spec is incorrect.
Check your connection and platform name/version/URL.
Failed to get absolute path to installed module
Error: No platforms added to this project. Please use `cordova platform add
`.
{code}
And when I look at node_modules, most of my dependencies have been deleted, and
in particular, there's no more `node_modules/.bin/cordova` executable.
My best guess is that the `npm install --production --no-save` that
cordova-fetch internally does is nuking a bunch of dependencies considering
"non-production", including the cli folder itself. The remedy is to use a
globally installed cordova CLI executable, which somehow avoids these issues.
Both my local and global cordova executables are 7.7.4.
I will continue to try and provide a consistent reproduction of this issue;
I've tried and failed many times and I'm starting to pull my hair out. I just
wanted to officially post this issue in case other people have seen it, have
advice, or have had better luck reproducing.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: issues-unsubscr...@cordova.apache.org
For additional commands, e-mail: issues-h...@cordova.apache.org
