> On Nov 20, 2019, at 10:27 PM, Tim Orling <timothy.t.orl...@linux.intel.com> > wrote: > > > >> On Nov 20, 2019, at 1:33 AM, Jean-Marie LEMETAYER >> <jean-marie.lemeta...@savoirfairelinux.com >> <mailto:jean-marie.lemeta...@savoirfairelinux.com>> wrote: >> >> The current NPM support have several issues: >> - The current NPM fetcher downloads the dependency tree but not the other >> fetchers. The 'subdir' parameter was used to fix this issue. >> - They are multiple issues with package names (uppercase, exotic characters, >> scoped packages) even if they are inside the dependencies. >> - The lockdown file generation have issues. When a package depends on >> multiple version of the same package (all versions have the same checksum). >> >> This patchset refactors the NPM support in Yocto: >> - As the NPM algorithm for dependency management is hard to handle, the new >> NPM fetcher downloads only the package source (and not the dependencies, >> like the other fetchers) (patch submitted in the bitbake-devel list). >> - The NPM class handles the dependencies using NPM (and not manually). >> - The NPM recipe creation is simplified to avoid issues. >> - The lockdown file is no more used as it is no longer relevant compared to >> the >> latest shrinkwrap file format. >> >> This patchset may remove some features (lockdown file, license management for >> dependencies) but fixes the majority of the NPM issues. All of these issues >> from the bugzilla.yoctoproject.org <http://bugzilla.yoctoproject.org/> are >> resolved by this patchset: >> #10237, #10760, #11028, #11728, #11902, #12534 >> >> The fetcher and recipetool are now aware of a 'latest' keyword for the >> version >> which allow to build the latest version available on the registry. This >> feature >> fixes the two issues: #10515, #11029 >> >> Moreover the issue #13415 should also be fixed but I cannot test it. >> >> I have tested the recipe creation and builds using a self made example to >> generate build failures: >> - https://github.com/savoirfairelinux/node-server-example >> <https://github.com/savoirfairelinux/node-server-example> >> - https://npmjs.com/package/@savoirfairelinux/node-server-example >> <https://npmjs.com/package/@savoirfairelinux/node-server-example> >> >> The test steps are these ones: >> $ source poky/oe-init-build-env >> $ bitbake-layers add-layer ../meta-openembedded/meta-oe >> $ devtool add >> "npm://registry.npmjs.org;name=@savoirfairelinux/node-server-example;version=latest >> >> <npm://registry.npmjs.org%3Bname%3D@savoirfairelinux/node-server-example;version=latest>” > > I noticed this before but forgot to report it: > > ERROR: Nothing PROVIDES 'c-ares-native' (but > virtual:native:/build/meta-openembedded/meta-oe/recipes-devtools/nodejs/nodejs_10.17.0.bb > DEPENDS on or otherwise requires it). ... > http://layers.openembedded.org/layerindex/recipe/50318/ > <http://layers.openembedded.org/layerindex/recipe/50318/> > $ bitbake-layers add-layer ../meta-openembedded/meta-networking > Which requires > $ bitbake-layers add-layer ../meta-openembedded/meta-python > > > Using tip of bitbake with V3, openembedded-core with V3 and meta-openembedded > master, somehow nodejs-native do_compile broke (at least on qemux86-64) since > the last time I built it shortly after ELC-E… > So now I’ll have to git bisect that…more to come…
See https://errors.yoctoproject.org/Errors/Details/274939/ <https://errors.yoctoproject.org/Errors/Details/274939/> The same error I am seeing. > > FWIW I had previously successfully run your instructions and the tests for > both bitbake and oe-selftest for V2 (distroless, e.g. not with poky). > Something unrelated has broken I suspect. > >> $ devtool build savoirfairelinux-node-server-example >> $ bitbake-layers create-layer ../meta-test >> $ bitbake-layers add-layer ../meta-test >> $ devtool finish savoirfairelinux-node-server-example ../meta-test >> $ echo IMAGE_INSTALL_append = '" savoirfairelinux-node-server-example"' >> >> conf/local.conf >> $ bitbake core-image-minimal >> >> Also the 'devtool add' url could be one of these: >> - >> npm://registry.npmjs.org;name=@savoirfairelinux/node-server-example;version=latest >> >> <npm://registry.npmjs.org%3Bname%3D@savoirfairelinux/node-server-example;version=latest> >> This url uses the new npm fetcher and request the latest version available >> on >> the registry. >> - >> npm://registry.npmjs.org;name=@savoirfairelinux/node-server-example;version=1.0.0 >> >> <npm://registry.npmjs.org%3Bname%3D@savoirfairelinux/node-server-example;version=1.0.0> >> This url uses the new npm fetcher and request a fixed version. >> - git://github.com/savoirfairelinux/node-server-example.git;protocol=https >> <git://github.com/savoirfairelinux/node-server-example.git;protocol=https> >> This url uses the git fetcher. As the dependencies are managed by the NPM >> class, any fetcher can be used. >> >> When this patchset will be merged, I have planned to update the NPM wiki: >> https://wiki.yoctoproject.org/wiki/TipsAndTricks/NPM >> <https://wiki.yoctoproject.org/wiki/TipsAndTricks/NPM> >> >> This patchset is also the base work for the full Angular support in Yocto >> that I >> am preparing. These applications have a huge dependency tree which is ideal >> to >> test the NPM support. >> >> --- V2 >> >> - Add the 'check_network_access' function before each network access to check >> for 'BB_NO_NETWORK' and 'BB_ALLOWED_NETWORKS' variables. >> >> - Add a 'recipetool.RecipetoolTests.test_recipetool_create_npm' test case for >> 'oe-selftest' to test the npm recipe creation. >> >> - Update the 'npm.bbclass' to fetch the dependencies in the 'do_fetch' task. >> The dependencies are cached in a npm cache stored in '${DL_DIR}/npm_cache' >> allowing the dependencies to be saved in the download directory and verify >> on insertion and extraction [1]: >> "All data that passes through the cache is fully verified >> for integrity on both insertion and extraction." >> >> 1: https://docs.npmjs.com/cli/cache.html >> <https://docs.npmjs.com/cli/cache.html> >> >> --- V3 >> >> - Add a test regarding the 'devtool add' and 'devtool build' command: >> - devtool.DevtoolAddTests.test_devtool_add_npm >> >> - Add the license management for dependencies. >> >> - Split the npm workflow to use the bitbake default tasks (do_fetch, >> do_unpack, >> do_compile, do_install). Make sure that only the do_fetch task requires >> network access. >> >> - Split the commits for better understanding. >> >> - Force the rebuild of prebuild addons. >> >> - Use ${nonarch_libdir} instead of ${libdir} in the do_install task. >> >> - These patches can be found here: >> - >> https://github.com/savoirfairelinux/openembedded-core/tree/npm-refactoring-v3 >> >> <https://github.com/savoirfairelinux/openembedded-core/tree/npm-refactoring-v3> >> - https://github.com/savoirfairelinux/poky/tree/npm-refactoring-v3 >> <https://github.com/savoirfairelinux/poky/tree/npm-refactoring-v3> >> >> Jean-Marie LEMETAYER (17): >> devtool: npm: update command line options >> npm.bbclass: refactor the npm class >> recipetool/create_npm.py: refactor the npm recipe creation handler >> lib/oe/package.py: remove unneeded npm_split_package_dirs function >> devtool/standard.py: npm: update the append file >> recipetool/create.py: npm: remove the 'noverify' url parameter >> recipetool/create.py: npm: replace the 'latest' keyword >> npm.bbclass: split the do_compile task >> devtool/standard.py: npm: exclude the node_modules directory >> recipetool/create_npm.py: handle the licenses of the dependencies >> npm.bbclass: restrict the build to be offline >> npm.bbclass: use the local node headers >> recipetool/create_npm.py: convert the shrinkwrap file >> npm.bbclass: force to rebuild the prebuild addons >> devtool/standard.py: npm: configure the NPM_CACHE_DIR to be persistent >> oeqa/selftest/recipetool: add npm recipe creation test >> oeqa/selftest/devtool: add npm recipe build test >> >> meta/classes/npm.bbclass | 224 +++++--- >> meta/lib/oe/package.py | 33 -- >> meta/lib/oeqa/selftest/cases/devtool.py | 20 + >> meta/lib/oeqa/selftest/cases/recipetool.py | 21 + >> scripts/lib/devtool/standard.py | 31 +- >> scripts/lib/recipetool/create.py | 16 +- >> scripts/lib/recipetool/create_npm.py | 565 +++++++++++---------- >> 7 files changed, 510 insertions(+), 400 deletions(-) >> >> -- >> 2.20.1 >> > > -- > _______________________________________________ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > <mailto:Openembedded-core@lists.openembedded.org> > http://lists.openembedded.org/mailman/listinfo/openembedded-core > <http://lists.openembedded.org/mailman/listinfo/openembedded-core>
-- _______________________________________________ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
