Package: release.debian.org Severity: normal User: release.debian....@packages.debian.org Usertags: unblock
Please unblock package opm-common It contains an important fix of the printed version number. It is only blocked because britney thinks that autopkgtests have not run, but they have [ Reason ] It contains an important fix of the printed version number and reduces resource usage during the build. [ Impact ] No impact for users if there no binary rebuilds. In the case of rebuilds resource usage on buildd will be much higher than with new version. [ Tests ] All autopkgtest have run successfully for official architectures with binary packages [ Risks ] None that I can think of [ Checklist ] [X] all changes are documented in the d/changelog [X] I reviewed all changes and I approve them [X] attach debdiff against the package in testing [ Other info ] None unblock opm-common/2022.10+ds-7
diff --git a/debian/changelog b/debian/changelog index 0af409a45..a011537d5 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,3 +1,23 @@ +opm-common (2022.10+ds-7) unstable; urgency=medium + + * d/control: Limit the architectures to known working ones. + + -- Markus Blatt <mar...@dr-blatt.de> Thu, 23 Mar 2023 08:34:59 +0100 + +opm-common (2022.10+ds-6) unstable; urgency=medium + + * Reduce compile time and resources with g++-12 + * d: Fixed version string used in binaries. + + -- Markus Blatt <mar...@dr-blatt.de> Wed, 08 Mar 2023 23:05:15 +0100 + +opm-common (2022.10+ds-5) experimental; urgency=medium + + * Backported fixes for python tests from upstream (to fix mipsel*?) + * d/rules: Try to limit parallel builds to ensure 3GB RAM per process + + -- Markus Blatt <mar...@dr-blatt.de> Sun, 05 Feb 2023 13:59:47 +0100 + opm-common (2022.10+ds-4) unstable; urgency=medium * Make sure copy_python is run before CopyHeaders Closes: #1029440 diff --git a/debian/control b/debian/control index 86112f71a..55debe51b 100644 --- a/debian/control +++ b/debian/control @@ -2,9 +2,9 @@ Source: opm-common Priority: optional Maintainer: Debian Science Maintainers <debian-science-maintain...@lists.alioth.debian.org> Uploaders: Arne Morten Kvarving <arne.morten.kvarv...@sintef.no>, Markus Blatt <mar...@dr-blatt.de> -Build-Depends: cmake (>=3.10), mpi-default-bin, mpi-default-dev, +Build-Depends: cmake (>=3.10), mpi-default-bin, mpi-default-dev, bc, procps, debhelper-compat (= 12), libcjson-dev, libfmt-dev, quilt, dh-python, - pkg-config, git, libtool, doxygen, graphviz, + pkg-config, lsb-release, libtool, doxygen, graphviz, texlive-latex-extra, texlive-latex-recommended, ghostscript, libboost-system-dev, libboost-test-dev, zlib1g-dev, python3-dev, libpython3-dev, python3-numpy, python3-distutils, @@ -19,7 +19,7 @@ Rules-Requires-Root: no Package: libopm-common Pre-Depends: ${misc:Pre-Depends} -Architecture: any +Architecture: amd64 arm64 armel ia64 m68k mips64el mipsel ppc64el riscv64 Multi-Arch: same Depends: ${shlibs:Depends}, ${misc:Depends}, ${python3:Depends} Provides: ${opm:shared-library} @@ -36,7 +36,7 @@ Description: Tools for Eclipse reservoir simulation files -- library Package: libopm-common-bin Pre-Depends: ${misc:Pre-Depends} -Architecture: any +Architecture: amd64 arm64 armel ia64 m68k mips64el mipsel ppc64el riscv64 Multi-Arch: foreign Depends: ${shlibs:Depends}, ${misc:Depends} Replaces: libopm-common1-bin @@ -52,7 +52,7 @@ Description: Tools for Eclipse reservoir simulation files -- utility programs Package: libopm-common-dev Section: libdevel -Architecture: any +Architecture: amd64 arm64 armel ia64 m68k mips64el mipsel ppc64el riscv64 Multi-Arch: same Suggests: libopm-common-doc Replaces: libopm-common1-dev @@ -85,7 +85,7 @@ Description: Tools for Eclipse reservoir simulation files -- documentation Package: python3-opm-common Section: python Pre-Depends: ${misc:Pre-Depends} -Architecture: any +Architecture: amd64 arm64 armel ia64 m68k mips64el mipsel ppc64el riscv64 Depends: ${shlibs:Depends}, ${misc:Depends}, ${python3:Depends}, libopm-common, python3-numpy, python3-decorator Description: Tools for Eclipse reservoir simulation files -- Python wrappers The Open Porous Media (OPM) software suite provides libraries and diff --git a/debian/opm-debian.mk b/debian/opm-debian.mk index 3b74fbe07..ea37112a8 100644 --- a/debian/opm-debian.mk +++ b/debian/opm-debian.mk @@ -2,10 +2,10 @@ include /usr/share/dpkg/architecture.mk include /usr/share/dpkg/pkg-info.mk include /usr/share/dune/dune-debian.env -LSB_RELEASE = lsb_release -r | sed "s/.*:\s*\([^\s]*\).*/\1/" +LSB_RELEASE = lsb_release -d | sed "s/.*:\s\+\(.*\)/\1/" # Needed for reproducable builds as we use __FILE__ export DEB_BUILD_MAINT_OPTIONS += reproducible=+fixfilepath -OPM_DEBIAN_CMAKE_FLAGS += -DBUILD_SHARED_LIBS=1 -DCMAKE_INSTALL_DOCDIR=share/doc/lib$(DEB_SOURCE) -DPYTHON_INSTALL_PREFIX=lib/python3/dist-packages -DOPM_INSTALL_COMPILED_PYTHON=OFF -DUSE_RUNPATH=OFF -DWITH_NATIVE=OFF -DUSE_MPI=ON -DUSE_BASH_COMPLETIONS_DIR=ON -DOPM_BINARY_PACKAGE_VERSION="$(DEB_VENDOR) $(LSB_RELEASE): $(DEB_VERSION)" +OPM_DEBIAN_CMAKE_FLAGS += -DBUILD_SHARED_LIBS=1 -DCMAKE_INSTALL_DOCDIR=share/doc/lib$(DEB_SOURCE) -DPYTHON_INSTALL_PREFIX=lib/python3/dist-packages -DOPM_INSTALL_COMPILED_PYTHON=OFF -DUSE_RUNPATH=OFF -DWITH_NATIVE=OFF -DUSE_MPI=ON -DUSE_BASH_COMPLETIONS_DIR=ON -DOPM_BINARY_PACKAGE_VERSION="$(LSB_RELEASE): $(DEB_VERSION)" OPM_DEBIAN_SHLIB = $(subst ~,.,lib$(DEB_SOURCE)-$(DEB_VERSION_UPSTREAM)) diff --git a/debian/patches/0011-python-Use-assertEqual-instead-of-deprecated-assertE.patch b/debian/patches/0011-python-Use-assertEqual-instead-of-deprecated-assertE.patch new file mode 100644 index 000000000..72eb9524d --- /dev/null +++ b/debian/patches/0011-python-Use-assertEqual-instead-of-deprecated-assertE.patch @@ -0,0 +1,27 @@ +From: Markus Blatt <mar...@dr-blatt.de> +Date: Fri, 13 Jan 2023 16:59:26 +0100 +Subject: [python] Use assertEqual instead of deprecated assertEquals- + +Prevents warnings like +``` +1: test_faults (tests.test_state.TestState2.test_faults) ... /<<PKGBUILDDIR>>/obj-arm-linux-gnueabihf/python/tests/test_state.py:124: DeprecationWarning: Please use assertEqual instead. +1: self.assertEquals([], self.state.faultNames()) +``` +On Debian unstable. +--- + python/tests/test_state.py | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/python/tests/test_state.py b/python/tests/test_state.py +index 9051df7..f756e11 100644 +--- a/python/tests/test_state.py ++++ b/python/tests/test_state.py +@@ -121,7 +121,7 @@ SATNUM + self.state.tables().evaluate(tab, 0, 'NO', 1) + + def test_faults(self): +- self.assertEquals([], self.state.faultNames()) ++ self.assertEqual([], self.state.faultNames()) + parser = Parser() + faultdeck = parser.parse_string(self.FAULTS_DECK) + faultstate = EclipseState(faultdeck) diff --git a/debian/patches/0012-python-Prevent-ResourceWarning-unclosed-file-_io.Tex.patch b/debian/patches/0012-python-Prevent-ResourceWarning-unclosed-file-_io.Tex.patch new file mode 100644 index 000000000..45218786f --- /dev/null +++ b/debian/patches/0012-python-Prevent-ResourceWarning-unclosed-file-_io.Tex.patch @@ -0,0 +1,22 @@ +From: Markus Blatt <mar...@dr-blatt.de> +Date: Fri, 13 Jan 2023 17:06:43 +0100 +Subject: [python] Prevent "ResourceWarning: unclosed file <_io.TextIOWrapper + +--- + python/tests/test_time_vector.py | 3 ++- + 1 file changed, 2 insertions(+), 1 deletion(-) + +diff --git a/python/tests/test_time_vector.py b/python/tests/test_time_vector.py +index 4f27413..6c81782 100644 +--- a/python/tests/test_time_vector.py ++++ b/python/tests/test_time_vector.py +@@ -83,7 +83,8 @@ class TestTimeVector(unittest.TestCase): + datetime.datetime(1998, 8, 1)]) + + def test_str(self): +- tv = TimeVector(datetime.date(1997, 11, 6), base_string = open(test_path("data/schedule/part1.sch")).read()) ++ with open(test_path("data/schedule/part1.sch")) as f: ++ tv = TimeVector(datetime.date(1997, 11, 6), base_string = f.read()) + tv.load(test_path("data/schedule/part3.sch")) + tv.load(test_path("data/schedule/part2.sch")) + diff --git a/debian/patches/0013-python-Run-unit-tests-directly-and-not-via-setup.py-.patch b/debian/patches/0013-python-Run-unit-tests-directly-and-not-via-setup.py-.patch new file mode 100644 index 000000000..c99ed6222 --- /dev/null +++ b/debian/patches/0013-python-Run-unit-tests-directly-and-not-via-setup.py-.patch @@ -0,0 +1,26 @@ +From: Markus Blatt <mar...@dr-blatt.de> +Date: Fri, 13 Jan 2023 18:12:31 +0100 +Subject: [python] Run unit tests directly and not via setup.py (deprecated). + +Thus we get rid of the warning: + +WARNING: Testing via this command is deprecated and will be removed in +a future version. Users looking for a generic test entry point +independent of test runner are encouraged to use tox. +--- + CMakeLists.txt | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/CMakeLists.txt b/CMakeLists.txt +index 9039dc2..d057ece 100644 +--- a/CMakeLists.txt ++++ b/CMakeLists.txt +@@ -477,7 +477,7 @@ if (OPM_ENABLE_PYTHON) + # testing. + add_test(NAME python_tests + WORKING_DIRECTORY ${PROJECT_BINARY_DIR}/python +- COMMAND ${CMAKE_COMMAND} -E env LD_LIBRARY_PATH=${PROJECT_BINARY_DIR}/lib ${Python3_EXECUTABLE} setup.py build_ext --dry-run --build-lib ${PROJECT_BINARY_DIR}/python test ++ COMMAND ${CMAKE_COMMAND} -E env LD_LIBRARY_PATH=${PROJECT_BINARY_DIR}/lib ${Python3_EXECUTABLE} -m unittest discover + ) + + set_target_properties(opmcommon PROPERTIES POSITION_INDEPENDENT_CODE ON) diff --git a/debian/patches/0014-Refactored-ParserInit.cpp-to-reduce-ram-needed-for-c.patch b/debian/patches/0014-Refactored-ParserInit.cpp-to-reduce-ram-needed-for-c.patch new file mode 100644 index 000000000..a5a6b1bde --- /dev/null +++ b/debian/patches/0014-Refactored-ParserInit.cpp-to-reduce-ram-needed-for-c.patch @@ -0,0 +1,225 @@ +From: Markus Blatt <mar...@dr-blatt.de> +Date: Fri, 20 Jan 2023 19:49:37 +0100 +Subject: Refactored ParserInit.cpp to reduce ram needed for compilation + +--- + CMakeLists.txt | 1 + + CopyHeaders.cmake | 3 + + GenerateKeywords.cmake | 8 +- + opm/input/eclipse/Generator/KeywordGenerator.hpp | 2 +- + .../input/eclipse/Generator/KeywordGenerator.cpp | 93 ++++++++++++++++++---- + .../eclipse/Parser/createDefaultKeywordList.cpp | 2 +- + 6 files changed, 90 insertions(+), 19 deletions(-) + +diff --git a/CMakeLists.txt b/CMakeLists.txt +index d057ece..e5ae79e 100644 +--- a/CMakeLists.txt ++++ b/CMakeLists.txt +@@ -136,6 +136,7 @@ macro (sources_hook) + list(INSERT opm-common_SOURCES 0 ${PROJECT_BINARY_DIR}/ParserInit.cpp) + foreach (name A B C D E F G H I J K L M N O P Q R S T U V W X Y Z) + list(INSERT opm-common_SOURCES 0 ${PROJECT_BINARY_DIR}/ParserKeywords/${name}.cpp) ++ list(INSERT opm-common_SOURCES 0 ${PROJECT_BINARY_DIR}/ParserKeywords/ParserInit${name}.cpp) + list(INSERT opm-common_HEADERS 0 ${PROJECT_BINARY_DIR}/include/opm/input/eclipse/Parser/ParserKeywords/${name}.hpp) + endforeach() + if (OPM_ENABLE_EMBEDDED_PYTHON) +diff --git a/CopyHeaders.cmake b/CopyHeaders.cmake +index 044fbb3..4c243e3 100644 +--- a/CopyHeaders.cmake ++++ b/CopyHeaders.cmake +@@ -31,4 +31,7 @@ foreach (name A B C D E F G H I J K L M N O P Q R S T U V W X Y Z) + execute_process(COMMAND ${CMAKE_COMMAND} -E copy_if_different + ${BASE_DIR}/tmp_gen/ParserKeywords/${name}.cpp + ${BASE_DIR}/ParserKeywords/${name}.cpp) ++ execute_process(COMMAND ${CMAKE_COMMAND} -E copy_if_different ++ ${BASE_DIR}/tmp_gen/ParserKeywords/ParserInit${name}.cpp ++ ${BASE_DIR}/ParserKeywords/ParserInit${name}.cpp) + endforeach() +diff --git a/GenerateKeywords.cmake b/GenerateKeywords.cmake +index 16a94c4..9320f92 100644 +--- a/GenerateKeywords.cmake ++++ b/GenerateKeywords.cmake +@@ -54,9 +54,13 @@ set( genkw_argv keyword_list.argv + + foreach (name A B C D E F G H I J K L M N O P Q R S T U V W X Y Z) + list(APPEND _tmp_output ${PROJECT_BINARY_DIR}/tmp_gen/ParserKeywords/${name}.cpp +- ${PROJECT_BINARY_DIR}/tmp_gen/include/opm/input/eclipse/Parser/ParserKeywords/${name}.hpp) ++ ${PROJECT_BINARY_DIR}/tmp_gen/include/opm/input/eclipse/Parser/ParserKeywords/${name}.hpp ++ ${PROJECT_BINARY_DIR}/tmp_gen/ParserKeywords/ParserInit${name}.cpp ++ ${PROJECT_BINARY_DIR}/tmp_gen/include/opm/input/eclipse/Parser/ParserKeywords/ParserInit${name}.hpp) + list(APPEND _target_output ${PROJECT_BINARY_DIR}/ParserKeywords/${name}.cpp +- ${PROJECT_BINARY_DIR}/include/opm/input/eclipse/Parser/ParserKeywords/${name}.hpp) ++ ${PROJECT_BINARY_DIR}/include/opm/input/eclipse/Parser/ParserKeywords/${name}.hpp ++ ${PROJECT_BINARY_DIR}/ParserKeywords/ParserInit${name}.cpp ++ ${PROJECT_BINARY_DIR}/include/opm/input/eclipse/Parser/ParserKeywords/ParserInit${name}.hpp) + endforeach() + + foreach(name TestKeywords.cpp ParserInit.cpp) +diff --git a/opm/input/eclipse/Generator/KeywordGenerator.hpp b/opm/input/eclipse/Generator/KeywordGenerator.hpp +index e571193..7c24f18 100644 +--- a/opm/input/eclipse/Generator/KeywordGenerator.hpp ++++ b/opm/input/eclipse/Generator/KeywordGenerator.hpp +@@ -39,7 +39,7 @@ namespace Opm { + static void updateFile(const std::stringstream& newContent, const std::string& filename); + + void updateBuiltInHeader(const KeywordLoader& loader, const std::string& headerBuildPath, const std::string& headerPath) const; +- void updateInitSource(const KeywordLoader& loader, const std::string& sourceFile ) const; ++ void updateInitSource(const KeywordLoader& loader, const std::string& sourceFile, const std::string& sourcePath) const; + void updateKeywordSource(const KeywordLoader& loader, const std::string& sourceFile ) const; + void updatePybindSource(const KeywordLoader& loader , const std::string& sourceFile ) const; + void updateHeader(const KeywordLoader& loader, const std::string& headerBuildPath, const std::string& headerPath) const; +diff --git a/src/opm/input/eclipse/Generator/KeywordGenerator.cpp b/src/opm/input/eclipse/Generator/KeywordGenerator.cpp +index f2179cf..418f311 100644 +--- a/src/opm/input/eclipse/Generator/KeywordGenerator.cpp ++++ b/src/opm/input/eclipse/Generator/KeywordGenerator.cpp +@@ -65,18 +65,34 @@ namespace Opm { + std::filesystem::create_directories( file.parent_path()); + } + +- void KeywordGenerator::updateFile(const std::stringstream& newContent , const std::string& filename) { +- ensurePath(filename); ++ void ++ updateFile(const std::string& newContent, const std::string& filename) ++ { ++ KeywordGenerator::ensurePath(filename); + std::ofstream outputStream(filename); +- outputStream << newContent.str(); ++ outputStream << newContent; ++ } ++ void ++ KeywordGenerator::updateFile(const std::stringstream& newContent, const std::string& filename) ++ { ++ Opm::updateFile(newContent.str(), filename); + } + +- static void write_file( const std::stringstream& stream, const std::string& file, bool verbose, std::string desc = "source" ) { +- KeywordGenerator::updateFile( stream, file ); +- if( verbose ) ++ void ++ write_file(const std::string& content, const std::string& file, bool verbose, std::string desc = "source") ++ { ++ Opm::updateFile(content, file); ++ if (verbose) + fmt::print("Updated {} file written to {}\n", desc, file); + } + ++ ++ static void ++ write_file(const std::stringstream& stream, const std::string& file, bool verbose, std::string desc = "source") ++ { ++ write_file(stream.str(), file, verbose, desc); ++ } ++ + void KeywordGenerator::updateBuiltInHeader(const KeywordLoader& loader, const std::string& headerBuildPath, const std::string& headerPath) const { + std::stringstream newSource; + newSource << R"(#ifndef PARSER_KEYWORDS_BUILTIN_HPP +@@ -141,7 +157,11 @@ private: + write_file( newSource, final_path, m_verbose, "header" ); + } + +- void KeywordGenerator::updateInitSource(const KeywordLoader& loader , const std::string& sourceFile ) const { ++ void KeywordGenerator::updateInitSource(const KeywordLoader& loader , const std::string& sourceFile, ++ const std::string& sourcePath ) const { ++ std::filesystem::path parserInitSource(sourceFile); ++ std::string stem = parserInitSource.stem(); ++ std::filesystem::path parentPath = parserInitSource.parent_path(); + std::stringstream newSource; + newSource << R"( + #include <opm/input/eclipse/Parser/Parser.hpp> +@@ -150,7 +170,52 @@ private: + + for(const auto& kw_pair : loader) { + const auto& first_char = kw_pair.first; +- newSource << fmt::format("#include <opm/input/eclipse/Parser/ParserKeywords/{}.hpp>\n", first_char); ++ const std::string header = fmt::format(R"( ++#ifndef OPM_PARSER_INIT_{0}_HH ++#define OPM_PARSER_INIT_{0}_HH ++ ++namespace Opm {{ ++class Parser; ++namespace ParserKeywords {{ ++void addDefaultKeywords{0}(Parser& p); ++}} ++}} ++#endif ++)", ++ first_char); ++ auto charHeaderFile = parserInitSource; ++ charHeaderFile.replace_filename( ++ fmt::format("include/opm/input/eclipse/Parser/ParserKeywords/ParserInit{}.hpp", first_char)); ++ write_file(header, charHeaderFile, m_verbose, fmt::format("init header for {}", first_char)); ++ std::stringstream sourceStr; ++ sourceStr << fmt::format(R"( ++#include <opm/input/eclipse/Parser/Parser.hpp> ++#include <opm/input/eclipse/Parser/ParserKeywords/Builtin.hpp> ++#include<opm/input/eclipse/Parser/ParserKeywords/ParserInit{0}.hpp> ++#include <opm/input/eclipse/Parser/ParserKeywords/{0}.hpp> ++ ++namespace Opm {{ ++namespace ParserKeywords {{ ++void addDefaultKeywords{0}(Parser& p){{ ++ Builtin keywords; ++)", ++ first_char); ++ for (const auto& kw_pair : loader) { ++ const auto& keywords = kw_pair.second; ++ for (const auto& kw : keywords) ++ sourceStr << fmt::format(" p.addParserKeyword( keywords.{} );", kw.className()) << std::endl; ++ } ++ sourceStr << R"( ++ ++} ++} ++} ++)"; ++ auto charSourceFile = std::filesystem::path(sourcePath) / fmt::format("ParserInit{}.cpp", first_char); ++ write_file(sourceStr, charSourceFile, m_verbose, fmt::format("init source for {}", first_char)); ++ ++ newSource << fmt::format("#include <opm/input/eclipse/Parser/ParserKeywords/ParserInit{}.hpp>\n", ++ first_char); + } + + newSource << R"( +@@ -158,24 +223,22 @@ namespace Opm { + namespace ParserKeywords { + void addDefaultKeywords(Parser& p); + void addDefaultKeywords(Parser& p) { +- Builtin keywords; + )"; + +- for(const auto& kw_pair : loader) { +- const auto& keywords = kw_pair.second; +- for (const auto& kw: keywords) +- newSource << " p.addParserKeyword( keywords." << kw.className() << " );" << std::endl; ++ for(const auto& [first_char, keywords] : loader) { ++ newSource << fmt::format(" addDefaultKeywords{}(p);", first_char) << std::endl; + } + + newSource << R"( + } + } + void Parser::addDefaultKeywords() { +- ParserKeywords::addDefaultKeywords(*this); ++ ParserKeywords::addDefaultKeywords(*this); + } + } + )"; +- write_file( newSource, sourceFile, m_verbose, "init" ); ++ ++ write_file(newSource, sourceFile, m_verbose, "init"); + } + + +diff --git a/src/opm/input/eclipse/Parser/createDefaultKeywordList.cpp b/src/opm/input/eclipse/Parser/createDefaultKeywordList.cpp +index 9c65405..88c8cf1 100644 +--- a/src/opm/input/eclipse/Parser/createDefaultKeywordList.cpp ++++ b/src/opm/input/eclipse/Parser/createDefaultKeywordList.cpp +@@ -58,7 +58,7 @@ int main(int argc, char ** argv) { + Opm::KeywordGenerator generator( true ); + + generator.updateKeywordSource(loader , source_file_path ); +- generator.updateInitSource(loader , init_file_name ); ++ generator.updateInitSource(loader , init_file_name , source_file_path ); + generator.updateHeader(loader, header_file_base_path, header_file_path ); + generator.updateBuiltInHeader(loader, header_file_base_path, header_file_path ); + generator.updateTest( loader , test_file_name ); diff --git a/debian/patches/0015-Split-Builtin.hpp-into-multiple-compile-units-for-g-.patch b/debian/patches/0015-Split-Builtin.hpp-into-multiple-compile-units-for-g-.patch new file mode 100644 index 000000000..5c394e5b2 --- /dev/null +++ b/debian/patches/0015-Split-Builtin.hpp-into-multiple-compile-units-for-g-.patch @@ -0,0 +1,236 @@ +From: Markus Blatt <mar...@dr-blatt.de> +Date: Wed, 8 Mar 2023 19:47:36 +0100 +Subject: Split Builtin.hpp into multiple compile units for g++-12. + +g++-12 needs quite some memory and time to compile it. +We now use several Builtin*.cpp for the imlementations for different +starting letters of eclipse keywords to use less resources for +compiling. +--- + CMakeLists.txt | 1 + + CopyHeaders.cmake | 3 + + GenerateKeywords.cmake | 2 + + opm/input/eclipse/Generator/KeywordGenerator.hpp | 3 +- + .../input/eclipse/Generator/KeywordGenerator.cpp | 88 ++++++++++++++-------- + .../eclipse/Parser/createDefaultKeywordList.cpp | 3 +- + 6 files changed, 65 insertions(+), 35 deletions(-) + +diff --git a/CMakeLists.txt b/CMakeLists.txt +index e5ae79e..6a00d68 100644 +--- a/CMakeLists.txt ++++ b/CMakeLists.txt +@@ -137,6 +137,7 @@ macro (sources_hook) + foreach (name A B C D E F G H I J K L M N O P Q R S T U V W X Y Z) + list(INSERT opm-common_SOURCES 0 ${PROJECT_BINARY_DIR}/ParserKeywords/${name}.cpp) + list(INSERT opm-common_SOURCES 0 ${PROJECT_BINARY_DIR}/ParserKeywords/ParserInit${name}.cpp) ++ list(INSERT opm-common_SOURCES 0 ${PROJECT_BINARY_DIR}/ParserKeywords/Builtin${name}.cpp) + list(INSERT opm-common_HEADERS 0 ${PROJECT_BINARY_DIR}/include/opm/input/eclipse/Parser/ParserKeywords/${name}.hpp) + endforeach() + if (OPM_ENABLE_EMBEDDED_PYTHON) +diff --git a/CopyHeaders.cmake b/CopyHeaders.cmake +index 4c243e3..faf9f4e 100644 +--- a/CopyHeaders.cmake ++++ b/CopyHeaders.cmake +@@ -34,4 +34,7 @@ foreach (name A B C D E F G H I J K L M N O P Q R S T U V W X Y Z) + execute_process(COMMAND ${CMAKE_COMMAND} -E copy_if_different + ${BASE_DIR}/tmp_gen/ParserKeywords/ParserInit${name}.cpp + ${BASE_DIR}/ParserKeywords/ParserInit${name}.cpp) ++ execute_process(COMMAND ${CMAKE_COMMAND} -E copy_if_different ++ ${BASE_DIR}/tmp_gen/ParserKeywords/Builtin${name}.cpp ++ ${BASE_DIR}/ParserKeywords/Builtin${name}.cpp) + endforeach() +diff --git a/GenerateKeywords.cmake b/GenerateKeywords.cmake +index 9320f92..ac21320 100644 +--- a/GenerateKeywords.cmake ++++ b/GenerateKeywords.cmake +@@ -56,10 +56,12 @@ foreach (name A B C D E F G H I J K L M N O P Q R S T U V W X Y Z) + list(APPEND _tmp_output ${PROJECT_BINARY_DIR}/tmp_gen/ParserKeywords/${name}.cpp + ${PROJECT_BINARY_DIR}/tmp_gen/include/opm/input/eclipse/Parser/ParserKeywords/${name}.hpp + ${PROJECT_BINARY_DIR}/tmp_gen/ParserKeywords/ParserInit${name}.cpp ++ ${PROJECT_BINARY_DIR}/tmp_gen/ParserKeywords/Builtin${name}.cpp + ${PROJECT_BINARY_DIR}/tmp_gen/include/opm/input/eclipse/Parser/ParserKeywords/ParserInit${name}.hpp) + list(APPEND _target_output ${PROJECT_BINARY_DIR}/ParserKeywords/${name}.cpp + ${PROJECT_BINARY_DIR}/include/opm/input/eclipse/Parser/ParserKeywords/${name}.hpp + ${PROJECT_BINARY_DIR}/ParserKeywords/ParserInit${name}.cpp ++ ${PROJECT_BINARY_DIR}/ParserKeywords/Builtin${name}.cpp + ${PROJECT_BINARY_DIR}/include/opm/input/eclipse/Parser/ParserKeywords/ParserInit${name}.hpp) + endforeach() + +diff --git a/opm/input/eclipse/Generator/KeywordGenerator.hpp b/opm/input/eclipse/Generator/KeywordGenerator.hpp +index 7c24f18..aa7af85 100644 +--- a/opm/input/eclipse/Generator/KeywordGenerator.hpp ++++ b/opm/input/eclipse/Generator/KeywordGenerator.hpp +@@ -38,7 +38,8 @@ namespace Opm { + static std::string headerHeader( const std::string& ); + static void updateFile(const std::stringstream& newContent, const std::string& filename); + +- void updateBuiltInHeader(const KeywordLoader& loader, const std::string& headerBuildPath, const std::string& headerPath) const; ++ void updateBuiltInHeader(const KeywordLoader& loader, const std::string& headerBuildPath, ++ const std::string& headerPath, const std::string& sourcePath) const; + void updateInitSource(const KeywordLoader& loader, const std::string& sourceFile, const std::string& sourcePath) const; + void updateKeywordSource(const KeywordLoader& loader, const std::string& sourceFile ) const; + void updatePybindSource(const KeywordLoader& loader , const std::string& sourceFile ) const; +diff --git a/src/opm/input/eclipse/Generator/KeywordGenerator.cpp b/src/opm/input/eclipse/Generator/KeywordGenerator.cpp +index 418f311..b1a2dd7 100644 +--- a/src/opm/input/eclipse/Generator/KeywordGenerator.cpp ++++ b/src/opm/input/eclipse/Generator/KeywordGenerator.cpp +@@ -93,60 +93,79 @@ namespace Opm { + write_file(stream.str(), file, verbose, desc); + } + +- void KeywordGenerator::updateBuiltInHeader(const KeywordLoader& loader, const std::string& headerBuildPath, const std::string& headerPath) const { +- std::stringstream newSource; +- newSource << R"(#ifndef PARSER_KEYWORDS_BUILTIN_HPP ++ void KeywordGenerator::updateBuiltInHeader(const KeywordLoader& loader, const std::string& headerBuildPath, const std::string& headerPath, ++ const std::string& sourcePath ) const { ++ std::stringstream newHeader; ++ std::map<char, std::stringstream> newSources; ++ ++ newHeader << R"(#ifndef PARSER_KEYWORDS_BUILTIN_HPP + #define PARSER_KEYWORDS_BUILTIN_HPP + #include <unordered_map> + #include <fmt/format.h> ++#include <opm/input/eclipse/Parser/ParserKeyword.hpp> + )"; + + for(const auto& kw_pair : loader) { + const auto& first_char = kw_pair.first; +- newSource << fmt::format("#include <opm/input/eclipse/Parser/ParserKeywords/{}.hpp>\n", first_char); ++ newSources[first_char] << fmt::format("#include <opm/input/eclipse/Parser/ParserKeywords/{}.hpp>\n", first_char) ++ << fmt::format("#include <{}/Builtin.hpp>\n", headerPath) ++ << "namespace Opm { namespace ParserKeywords {\n"; + } + +- newSource << R"( ++ newHeader << R"( + namespace Opm { + namespace ParserKeywords { + struct Builtin { + )"; + for(const auto& kw_pair : loader) { + const auto& keywords = kw_pair.second; ++ auto& source = newSources[kw_pair.first]; + for (const auto& kw: keywords) +- newSource << fmt::format(" const ::Opm::ParserKeywords::{0} {0};\n", kw.className()); +- } +- +- for(const auto& kw_pair : loader) { +- const auto& keywords = kw_pair.second; +- for (const auto& kw: keywords) +- newSource << fmt::format(" const ::Opm::ParserKeyword& get_{0}() {{ return this->{0}; }};\n",kw.className()); ++ { ++ newHeader << fmt::format(" const ::Opm::ParserKeyword get_{0}();\n",kw.className()); ++ source << fmt::format("const ::Opm::ParserKeyword Builtin::get_{0}() {{ return {0}(); }};\n",kw.className()); ++ } + } + +- newSource << R"( +- const ::Opm::ParserKeyword& operator[](const std::string& keyword) const { +- if (this->keywords.empty()) { ++ newHeader << R"( ++ const ::Opm::ParserKeyword& operator[](const std::string& keyword) const { ++ if (this->keywords.empty()) { + )"; ++ std::stringstream declareEmplace; + + for(const auto& kw_pair : loader) { + const auto& keywords = kw_pair.second; ++ auto& source = newSources[kw_pair.first]; ++ newHeader << fmt::format(" emplace{}(this->keywords);\n", kw_pair.first); ++ source << fmt::format(R"( ++void Builtin::emplace{}([[maybe_unused]] std::unordered_map<std::string, ::Opm::ParserKeyword>& keywords) const {{ ++)", ++ kw_pair.first); ++ declareEmplace << fmt::format(R"( ++ void emplace{}(std::unordered_map<std::string, ::Opm::ParserKeyword>& keywords) const; ++)", ++ kw_pair.first); ++ + for (const auto& kw: keywords) +- newSource << fmt::format(" this->keywords.emplace(\"{0}\", this->{0});\n", kw.className()); ++ source << fmt::format(" keywords.emplace(\"{0}\", {0}());\n", kw.className()); ++ source <<"}\n"; ++ source <<"} }\n"; + } + +- newSource << R"( } +- const auto kw_iter = this->keywords.find(keyword); +- if (kw_iter == this->keywords.end()) +- throw std::invalid_argument(fmt::format("No builtin keyword: {}", keyword)); +- return kw_iter->second; +-} ++ newHeader << R"( } ++ const auto kw_iter = this->keywords.find(keyword); ++ if (kw_iter == this->keywords.end()) ++ throw std::invalid_argument(fmt::format("No builtin keyword: {}", keyword)); ++ return kw_iter->second; ++ } + +- const ::Opm::ParserKeyword& getKeyword(const std::string& keyword) const { return this->operator[](keyword); } ++ const ::Opm::ParserKeyword& getKeyword(const std::string& keyword) const { return this->operator[](keyword); } + )"; + +- newSource << R"( +-private: +- mutable std::unordered_map<std::string, ::Opm::ParserKeyword> keywords; ++ newHeader << "private:\n"; ++ newHeader << declareEmplace.str(); ++ newHeader << R"( ++ mutable std::unordered_map<std::string, ::Opm::ParserKeyword> keywords; + }; + } + } +@@ -154,7 +173,13 @@ private: + )"; + + const auto final_path = headerBuildPath + headerPath+ "/Builtin.hpp"; +- write_file( newSource, final_path, m_verbose, "header" ); ++ write_file( newHeader, final_path, m_verbose, "header" ); ++ for(auto&& [first_char, source]: newSources) ++ { ++ auto sourceFile = std::filesystem::path(sourcePath) / fmt::format("Builtin{}.cpp", ++ first_char); ++ write_file(source, sourceFile, m_verbose, fmt::format("builtin source for {}", first_char)); ++ } + } + + void KeywordGenerator::updateInitSource(const KeywordLoader& loader , const std::string& sourceFile, +@@ -190,21 +215,18 @@ void addDefaultKeywords{0}(Parser& p); + std::stringstream sourceStr; + sourceStr << fmt::format(R"( + #include <opm/input/eclipse/Parser/Parser.hpp> +-#include <opm/input/eclipse/Parser/ParserKeywords/Builtin.hpp> + #include<opm/input/eclipse/Parser/ParserKeywords/ParserInit{0}.hpp> + #include <opm/input/eclipse/Parser/ParserKeywords/{0}.hpp> + + namespace Opm {{ + namespace ParserKeywords {{ +-void addDefaultKeywords{0}(Parser& p){{ +- Builtin keywords; ++void addDefaultKeywords{0}([[maybe_unused]] Parser& p){{ ++ //Builtin keywords; + )", + first_char); +- for (const auto& kw_pair : loader) { + const auto& keywords = kw_pair.second; + for (const auto& kw : keywords) +- sourceStr << fmt::format(" p.addParserKeyword( keywords.{} );", kw.className()) << std::endl; +- } ++ sourceStr << fmt::format(" p.addParserKeyword( {}() );", kw.className()) << std::endl; + sourceStr << R"( + + } +diff --git a/src/opm/input/eclipse/Parser/createDefaultKeywordList.cpp b/src/opm/input/eclipse/Parser/createDefaultKeywordList.cpp +index 88c8cf1..ec8d73d 100644 +--- a/src/opm/input/eclipse/Parser/createDefaultKeywordList.cpp ++++ b/src/opm/input/eclipse/Parser/createDefaultKeywordList.cpp +@@ -60,7 +60,8 @@ int main(int argc, char ** argv) { + generator.updateKeywordSource(loader , source_file_path ); + generator.updateInitSource(loader , init_file_name , source_file_path ); + generator.updateHeader(loader, header_file_base_path, header_file_path ); +- generator.updateBuiltInHeader(loader, header_file_base_path, header_file_path ); ++ generator.updateBuiltInHeader(loader, header_file_base_path, header_file_path, ++ source_file_path ); + generator.updateTest( loader , test_file_name ); + if (argc >= 8) + generator.updatePybindSource(loader , argv[7]); diff --git a/debian/patches/0016-Added-missing-include-to-ParserTests.cpp.patch b/debian/patches/0016-Added-missing-include-to-ParserTests.cpp.patch new file mode 100644 index 000000000..f663a4aef --- /dev/null +++ b/debian/patches/0016-Added-missing-include-to-ParserTests.cpp.patch @@ -0,0 +1,29 @@ +From: Markus Blatt <mar...@dr-blatt.de> +Date: Wed, 8 Mar 2023 19:48:38 +0100 +Subject: Added missing include to ParserTests.cpp + +--- + tests/parser/ParserTests.cpp | 3 ++- + 1 file changed, 2 insertions(+), 1 deletion(-) + +diff --git a/tests/parser/ParserTests.cpp b/tests/parser/ParserTests.cpp +index 4ab0ecf..85b3086 100644 +--- a/tests/parser/ParserTests.cpp ++++ b/tests/parser/ParserTests.cpp +@@ -35,6 +35,7 @@ + #include <opm/input/eclipse/Parser/ParserKeyword.hpp> + #include <opm/input/eclipse/Parser/ParserKeywords/A.hpp> + #include <opm/input/eclipse/Parser/ParserKeywords/S.hpp> ++#include <opm/input/eclipse/Parser/ParserKeywords/T.hpp> + #include <opm/input/eclipse/Parser/ParserKeywords/Builtin.hpp> + #include <opm/input/eclipse/Parser/ParserRecord.hpp> + +@@ -2355,7 +2356,7 @@ GUIDERAT + )"; + + BOOST_CHECK_EQUAL( parser.size(), 0 ); +- parser.addParserKeyword( builtin.GUIDERAT ); ++ parser.addParserKeyword( builtin.get_GUIDERAT() ); + BOOST_CHECK_EQUAL( parser.size(), 1 ); + auto deck = parser.parseString(deck_string); + BOOST_CHECK( deck.hasKeyword("GUIDERAT") ); diff --git a/debian/patches/series b/debian/patches/series index 735303415..0a9f71111 100644 --- a/debian/patches/series +++ b/debian/patches/series @@ -8,3 +8,9 @@ 0008-cmake-Limit-dimensions-for-Alberta.patch 0009-cmake-Also-search-for-Alberta-with-pkgconfig-to-be-D.patch 0011-Make-sure-copy_python-is-run-before-CopyHeaders-Clos.patch +0011-python-Use-assertEqual-instead-of-deprecated-assertE.patch +0012-python-Prevent-ResourceWarning-unclosed-file-_io.Tex.patch +0013-python-Run-unit-tests-directly-and-not-via-setup.py-.patch +0014-Refactored-ParserInit.cpp-to-reduce-ram-needed-for-c.patch +0015-Split-Builtin.hpp-into-multiple-compile-units-for-g-.patch +0016-Added-missing-include-to-ParserTests.cpp.patch diff --git a/debian/rules b/debian/rules index 71031ce82..6b310e5e9 100755 --- a/debian/rules +++ b/debian/rules @@ -7,10 +7,14 @@ include /usr/share/dpkg/architecture.mk OPM_DEBIAN_CMAKE_FLAGS = -DOPM_ENABLE_PYTHON=1 -DOPM_INSTALL_PYTHON=1 -DPYTHON_EXECUTABLE=/usr/bin/python3 -DOPM_INSTALL_COMPILED_PYTHON=OFF -DOPM_ENABLE_EMBEDDED_PYTHON=1 - +need_gb_ram_per_process=3 +free_ram=$(shell free -g | sed -n 2p| sed "s/ \+/ /g"| cut -d " " -f 2) +max_procs=$(shell echo "$(free_ram)/$(need_gb_ram_per_process)" | bc) +parallel_procs=$(shell if test "$(max_procs)" -lt "1"; then echo 1; else echo "$(max_procs)"; fi) %: - dh $@ --with python3 + echo "ram in gb: $(free_ram), needed ram per thread: $(need_gb_ram_per_process), threads: $(parallel_procs)" + dh $@ --with python3 --max-parallel=$(parallel_procs) include debian/opm-debian.mk include debian/opm-lib-debian.mk # for makeshlibs