Hi Masatake, I checked the GPL license dependencies, some of them are required for ClickHouse, it'll also be a lot of work to keep track on it’s dependencies’ license in the future, so we may not be able to include it in Bigtop.
Thanks for your help! Best Regards, Zhiguo Wu > On Aug 25, 2022, at 18:28, Masatake Iwasaki <[email protected]> wrote: > >> mariadb-connector-c binary file is included in build directory, will this >> affect? > > Hmm. If it is optional contrib binary and not part of ClickHouse itself, > excluding it on packaging should be safer. > > On 2022/08/25 15:19, 吴治国 wrote: >> Thanks Masatake, >> mariadb-connector-c binary file is included in build directory, will this >> affect? >> ``` >> [root@30bdb332571b ~]# ls -R >> ./ClickHouse/build/contrib/mariadb-connector-c-cmake/ >> ./ClickHouse/build/contrib/mariadb-connector-c-cmake/: >> CMakeFiles cmake_install.cmake CTestTestfile.cmake include-private >> include-public libmariadb lib_mariadbclient.a >> ./ClickHouse/build/contrib/mariadb-connector-c-cmake/CMakeFiles: >> _mariadbclient.dir >> ./ClickHouse/build/contrib/mariadb-connector-c-cmake/CMakeFiles/_mariadbclient.dir: >> __ libmariadb >> ./ClickHouse/build/contrib/mariadb-connector-c-cmake/CMakeFiles/_mariadbclient.dir/__: >> mariadb-connector-c >> ./ClickHouse/build/contrib/mariadb-connector-c-cmake/CMakeFiles/_mariadbclient.dir/__/mariadb-connector-c: >> libmariadb plugins >> ./ClickHouse/build/contrib/mariadb-connector-c-cmake/CMakeFiles/_mariadbclient.dir/__/mariadb-connector-c/libmariadb: >> ma_alloc.c.o ma_context.c.o ma_hash.c.o ma_ll2str.c.o ma_pvio.c.o >> mariadb_rpl.c.o ma_string.c.o >> ma_array.c.o ma_default.c.o ma_init.c.o ma_loaddata.c.o >> mariadb_async.c.o mariadb_stmt.c.o ma_time.c.o >> ma_charset.c.o ma_dtoa.c.o ma_io.c.o ma_net.c.o >> mariadb_charset.c.o ma_sha1.c.o ma_tls.c.o >> ma_compress.c.o ma_errmsg.c.o ma_list.c.o ma_password.c.o >> mariadb_lib.c.o ma_stmt_codec.c.o secure >> ./ClickHouse/build/contrib/mariadb-connector-c-cmake/CMakeFiles/_mariadbclient.dir/__/mariadb-connector-c/libmariadb/secure: >> openssl.c.o openssl_crypt.c.o >> ./ClickHouse/build/contrib/mariadb-connector-c-cmake/CMakeFiles/_mariadbclient.dir/__/mariadb-connector-c/plugins: >> auth connection pvio >> ./ClickHouse/build/contrib/mariadb-connector-c-cmake/CMakeFiles/_mariadbclient.dir/__/mariadb-connector-c/plugins/auth: >> caching_sha2_pw.c.o my_auth.c.o sha256_pw.c.o >> ./ClickHouse/build/contrib/mariadb-connector-c-cmake/CMakeFiles/_mariadbclient.dir/__/mariadb-connector-c/plugins/connection: >> aurora.c.o >> ./ClickHouse/build/contrib/mariadb-connector-c-cmake/CMakeFiles/_mariadbclient.dir/__/mariadb-connector-c/plugins/pvio: >> pvio_socket.c.o >> ./ClickHouse/build/contrib/mariadb-connector-c-cmake/CMakeFiles/_mariadbclient.dir/libmariadb: >> ma_client_plugin.c.o >> ./ClickHouse/build/contrib/mariadb-connector-c-cmake/include-private: >> config.h ma_config.h >> ./ClickHouse/build/contrib/mariadb-connector-c-cmake/include-public: >> mariadb_version.h >> ./ClickHouse/build/contrib/mariadb-connector-c-cmake/libmariadb: >> ma_client_plugin.c >> ``` >> Best Regards, >> Zhiguo Wu >>> On Aug 25, 2022, at 09:30, Masatake Iwasaki <[email protected]> >>> wrote: >>> >>> Hi Zhiguo, >>> >>>>> GPL-2.0 License: >>>>> https://github.com/ClickHouse/rocksdb >>>>> https://github.com/xz-mirror/xz >>>>> https://github.com/facebook/zstd >>>>> >>>>> GPL-3.0 License: >>>>> https://github.com/xz-mirror/xz >>>>> >>>>> LGPL-2.1 License: >>>>> https://github.com/ClickHouse/lemmagen-c >>>>> https://github.com/ClickHouse/libgsasl >>>>> https://github.com/ClickHouse/mariadb-connector-c >>>>> https://github.com/ClickHouse/UnixODBC >>> >>> I think if the GPL libraries are linked to the single binary of ClickHouse, >>> it must be redistributed under GPL. >>> >>> | [1]: https://github.com/ClickHouse/ClickHouse/issues/39899 >>> | We use a portable binary - a single binary without dependencies that runs >>> on any Linux distribution, and is wrapped in .deb, .rpm, .tgz as well. No >>> need to have more than a single universal build. >>> >>> Since RocksDB is dual-licensed under GPLv2 and Apache 2.0, >>> we can choose Apache 2.0 license. >>> I'm not sure about other products listed above. >>> >>> Thanks, >>> Masatake Iwasaki >>> >>> On 2022/08/15 22:55, 吴治国 wrote: >>>> Hello everyone, >>>> Looks like ClickHouse community are ok with this[1], but they still have >>>> different opinions on writing our own build scripts. >>>> What do you think? >>>> Can I continue work on this? >>>> Best Regards, >>>> Zhiguo Wu >>>> [1]: https://github.com/ClickHouse/ClickHouse/issues/39899 >>>>> On Aug 5, 2022, at 11:44, 吴治国 <[email protected]> wrote: >>>>> >>>>> Hi everyone, >>>>> >>>>> I've checked all licenses used by its git submodules, below is the list, >>>>> will this affect? >>>>> >>>>> Apache License 2.0: >>>>> https://github.com/ClickHouse/AMQP-CPP >>>>> https://github.com/ClickHouse/NuRaft >>>>> https://github.com/abseil/abseil-cpp >>>>> https://github.com/ClickHouse/arrow >>>>> https://github.com/ClickHouse/avro >>>>> https://github.com/ClickHouse/aws-sdk-cpp >>>>> https://github.com/ClickHouse/aws-c-common >>>>> https://github.com/ClickHouse/aws-c-event-stream >>>>> https://github.com/ClickHouse/aws-checksums >>>>> https://github.com/ClickHouse/cpp-driver >>>>> https://github.com/ClickHouse/cctz >>>>> https://github.com/ClickHouse/cld2 >>>>> https://github.com/ClickHouse/datasketches-cpp >>>>> https://github.com/ClickHouse/dragonbox >>>>> https://github.com/fastfloat/fast_float >>>>> https://github.com/ClickHouse/flatbuffers >>>>> https://github.com/ClickHouse/grpc >>>>> https://github.com/ClickHouse/h3 >>>>> https://github.com/ClickHouse/hive-metastore >>>>> https://github.com/ClickHouse/libhdfs3 >>>>> https://github.com/google/libprotobuf-mutator >>>>> https://github.com/ClickHouse/orc >>>>> https://github.com/ClickHouse/s2geometry >>>>> https://github.com/simdjson/simdjson >>>>> https://github.com/apache/thrift >>>>> >>>>> MIT License: >>>>> https://github.com/ClickHouse/azure-sdk-for-cpp >>>>> https://github.com/google/brotli >>>>> https://github.com/fastfloat/fast_float >>>>> https://github.com/ClickHouse/fastops >>>>> https://github.com/Neargye/magic_enum >>>>> https://github.com/ClickHouse/nanodbc >>>>> https://github.com/ClickHouse/sentry-native >>>>> https://github.com/ClickHouse/yaml-cpp >>>>> >>>>> BSL-1.0 License: >>>>> https://github.com/ClickHouse/boost >>>>> https://github.com/ClickHouse/dragonbox >>>>> https://github.com/danlark1/miniselect >>>>> https://github.com/msgpack/msgpack-c >>>>> >>>>> BSD-2-Clause License: >>>>> https://github.com/mfontanini/cppkafka >>>>> >>>>> BSD-3-Clause License: >>>>> https://github.com/google/double-conversion >>>>> https://github.com/google/googletest >>>>> https://github.com/ClickHouse/hyperscan >>>>> https://github.com/ClickHouse/libpqxx >>>>> https://github.com/ClickHouse/libstemmer_c >>>>> https://github.com/google/re2 >>>>> https://github.com/sparsehash/sparsehash-c11 >>>>> https://github.com/azadkuh/sqlite-amalgamation >>>>> >>>>> GPL-2.0 License: >>>>> https://github.com/ClickHouse/rocksdb >>>>> https://github.com/xz-mirror/xz >>>>> https://github.com/facebook/zstd >>>>> >>>>> GPL-3.0 License: >>>>> https://github.com/xz-mirror/xz >>>>> >>>>> LGPL-2.1 License: >>>>> https://github.com/ClickHouse/lemmagen-c >>>>> https://github.com/ClickHouse/libgsasl >>>>> https://github.com/ClickHouse/mariadb-connector-c >>>>> https://github.com/ClickHouse/UnixODBC >>>>> >>>>> CC-BY-4.0 License: >>>>> https://github.com/ClickHouse/libuv >>>>> >>>>> Zlib License: >>>>> https://github.com/ClickHouse/zlib-ng >>>>> >>>>> Unknown License: >>>>> https://github.com/ClickHouse/Turbo-Base64 >>>>> https://github.com/ClickHouse/boringssl >>>>> https://github.com/ClickHouse/bzip2 >>>>> https://github.com/capnproto/capnproto >>>>> https://github.com/RoaringBitmap/CRoaring >>>>> https://github.com/curl/curl >>>>> https://github.com/ClickHouse/cyrus-sasl >>>>> https://github.com/fmtlib/fmt >>>>> https://github.com/ClickHouse/hyperscan >>>>> https://github.com/ClickHouse/icudata >>>>> https://github.com/jemalloc/jemalloc >>>>> https://github.com/ClickHouse/libcpuid >>>>> https://github.com/ClickHouse/libcxx >>>>> https://github.com/ClickHouse/libcxxabi >>>>> https://github.com/ClickHouse/librdkafka >>>>> https://github.com/ClickHouse/libunwind >>>>> https://github.com/ClickHouse/libuv >>>>> https://github.com/lz4/lz4 >>>>> https://github.com/zlib-ng/minizip-ng >>>>> https://github.com/msgpack/msgpack-c >>>>> https://github.com/ClickHouse/openldap >>>>> https://github.com/ClickHouse/poco >>>>> https://github.com/ClickHouse/protobuf >>>>> https://github.com/ClickHouse/rapidjson >>>>> https://github.com/ClickHouse/replxx >>>>> https://github.com/ClickHouse/snappy >>>>> >>>>> No License: >>>>> https://github.com/unicode-org/icu >>>>> https://github.com/ClickHouse/krb5 >>>>> https://github.com/ClickHouse/libpq >>>>> https://github.com/GNOME/libxml2 >>>>> https://github.com/ClickHouse/llvm >>>>> https://github.com/ClickHouse/nlp-data >>>>> https://github.com/ClickHouse/sysroot >>>>> https://github.com/ClickHouse/wordnet-blast >>>>> >>>>>> On Aug 4, 2022, at 15:25, 吴治国 <[email protected]> wrote: >>>>>> >>>>>> Thanks you all for your opinions, >>>>>> >>>>>> I have created a discussion on the ClickHouse GitHub page[1], let's wait >>>>>> for their response. >>>>>> >>>>>> Best Regards, >>>>>> Zhiguo Wu >>>>>> >>>>>> [1]: https://github.com/ClickHouse/ClickHouse/discussions/39889 >>>>>> >>>>>>> On Aug 4, 2022, at 14:26, Evans Ye <[email protected]> wrote: >>>>>>> >>>>>>> If both Yuqi questions can be answered I'm open to the clickhouse >>>>>>> addition >>>>>>> as long as there's a maintainer for it. >>>>>>> >>>>>>> For Distro coverage, I'd like to raise a discussion. >>>>>>> I'm thinking maybe we can have two layers of support: core packages and >>>>>>> extra packages. >>>>>>> Core packages should be fully supported for all the Bigtop Distros, >>>>>>> while >>>>>>> extra packages' support can be selective (or open for completion). >>>>>>> Since we do have the supporting matrix maintained, I think it manages >>>>>>> the user expectation and experience. >>>>>>> Anyhow, any code addition either fully or partially supported should >>>>>>> have >>>>>>> CI & smoke test covered (ex: clickhouse on x86 w/ packaging CI, puppet, >>>>>>> and >>>>>>> smoke test passed). >>>>>>> >>>>>>> - Evans >>>>>>> >>>>>>> Yuqi Gu <[email protected]> 於 2022年8月4日 週四 上午10:55寫道： >>>>>>> >>>>>>>> Agree with Kengo. >>>>>>>> Zhiguo has contributed much to Bigtop including the new Mpack >>>>>>>> provisioner >>>>>>>> which offers on-click deployment to test Bigtop+Ambari+Mpack. >>>>>>>> I believe he would offer the long-term support to maintain ClickHouse >>>>>>>> and >>>>>>>> the Mpack services. >>>>>>>> >>>>>>>> To Zhiguo, >>>>>>>> As mentioned by Masatake, ClickHouse is the non-ASF project. >>>>>>>> Before adding it into Bigtop, could you please make clear that >>>>>>>> 1. if there are any license conflicts. >>>>>>>> 2. if there are any restrictions or concerns from upstream ClickHouse >>>>>>>> community. >>>>>>>> >>>>>>>> >>>>>>>> BRs, >>>>>>>> Yuqi >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Kengo Seki <[email protected]> 于2022年8月2日周二 21:12写道： >>>>>>>> >>>>>>>>> Personally I'd like to encourage Zhiguo to add ClickHouse. >>>>>>>>> He's already done several notable contributions to our codebase, >>>>>>>>> so I think we can expect his long-term commit for keeping ClickHouse >>>>>>>> fresh. >>>>>>>>> In addition, once he gets used to the whole process to add a new >>>>>>>>> component through this development, >>>>>>>>> we also can count on him to add other components and maintain existing >>>>>>>>> ones. >>>>>>>>> >>>>>>>>> Kengo Seki <[email protected]> >>>>>>>>> >>>>>>>>> On Mon, Aug 1, 2022 at 4:17 PM 吴治国 <[email protected]> wrote: >>>>>>>>>> >>>>>>>>>> Is there any conclusion? >>>>>>>>>> >>>>>>>>>> Best Regards, >>>>>>>>>> Zhiguo Wu >>>>>>>>>> >>>>>>>>>>> On Jul 29, 2022, at 17:53, 吴治国 <[email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>> Thanks Masatake, >>>>>>>>>>> >>>>>>>>>>> I also prefer to use packages provide by them, this will reduce our >>>>>>>>> work. >>>>>>>>>>> >>>>>>>>>>> As I'm trying to use Bigtop as Ambari's default stack, it'll be >>>>>>>> better >>>>>>>>> if we use bigtop repo just like mpack, it makes sense to use BIGTOP >>>>>>>>> stack >>>>>>>>> name with BIGTOP repository. >>>>>>>>>>> >>>>>>>>>>> But add components to Bigtop requires build it from source, provide >>>>>>>>> rpm/deb build scripts, smoke test cases, and puppet manifest, which >>>>>>>>> I'm >>>>>>>>> doing for ClickHouse right now. >>>>>>>>>>> >>>>>>>>>>> And, if ClickHouse end up like ELK, I'll remove it at that time. >>>>>>>>>>> >>>>>>>>>>> Best Regards, >>>>>>>>>>> Zhiguo Wu >>>>>>>>>>> >>>>>>>>>>>> On Jul 29, 2022, at 17:21, Masatake Iwasaki < >>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>> Hi Zhiguo, >>>>>>>>>>>> >>>>>>>>>>>> Honestly saying, I'm not positive to add the product to Bigtop. >>>>>>>>>>>> We do not have enough developer's resource even for maintaining >>>>>>>>> current stack. >>>>>>>>>>>> Since it is non-ASF product driven by dedicated company, it may end >>>>>>>>> up in the same way with ELK. >>>>>>>>>>>> >>>>>>>>>>>> In addition, ClickHouse seems to provide packaging stuff on their >>>>>>>>> own[1] by using nFPM[2] >>>>>>>>>>>> and the packages are published[3]. >>>>>>>>>>>> If you want additional coverage, contributing to ClickHouse itself >>>>>>>>> should be the first choice. >>>>>>>>>>>> >>>>>>>>>>>> [1] https://github.com/ClickHouse/ClickHouse/tree/master/packages >>>>>>>>>>>> [2] https://github.com/goreleaser/nfpm >>>>>>>>>>>> [3] https://clickhouse.com/docs/en/getting-started/install/ >>>>>>>>>>>> >>>>>>>>>>>> Regards, >>>>>>>>>>>> Masatake Iwasaki >>>>>>>>>>>> >>>>>>>>>>>> On 2022/07/29 15:25, 吴治国 wrote: >>>>>>>>>>>>> Hi community, as the ClickHouse are becoming more and more popular >>>>>>>>> OLAP engine for bigdata, Although it's not related to Hadoop >>>>>>>>> ecosystem, I >>>>>>>>> still want to add it to Bigtop stack. >>>>>>>>>>>>> But I can only take care of it in some distros(mostly CentOS-7, >>>>>>>>> RockyLinux-8 and Ubuntu-20 for now), I don't know if this is enough, >>>>>>>>> if >>>>>>>>> not, can I add it to Bigtop? Or is there anyone also interested in >>>>>>>>> this? >>>>>>>>>>>>> Best Regards, >>>>>>>>>>>>> Zhiguo Wu >>>>>>>>>> >>>>>>>>> >>>>>>>>
