Hi Florian, As of now there are 8 packages in the rsyslog repo providing the librdkafka++.so.1 dependency. (and one package in epel, that's the last in the list below)
thanks, -Derek. ********** $ yum whatprovides --disablerepo=epel /usr/lib64/librdkafka++.so.1 Loaded plugins: langpacks, product-id, search-disabled-repos, subscription-manager adiscon-librdkafka1-0.8.5-1.x86_64 : The Apache Kafka C library Repo : rsyslog_v8 Matched from: Filename : /usr/lib64/librdkafka++.so.1 adiscon-librdkafka1-0.8.6-1.x86_64 : The Apache Kafka C library Repo : rsyslog_v8 Matched from: Filename : /usr/lib64/librdkafka++.so.1 adisconbuild-librdkafka1-0.9.5-1.x86_64 : The Apache Kafka C library Repo : rsyslog_v8 Matched from: Filename : /usr/lib64/librdkafka++.so.1 adisconbuild-librdkafka1-0.11.0-1.x86_64 : The Apache Kafka C library Repo : rsyslog_v8 Matched from: Filename : /usr/lib64/librdkafka++.so.1 adisconbuild-librdkafka1-0.11.0-2.x86_64 : The Apache Kafka C library Repo : rsyslog_v8 Matched from: Filename : /usr/lib64/librdkafka++.so.1 adisconbuild-librdkafka1-0.11.1-1.x86_64 : The Apache Kafka C library Repo : rsyslog_v8 Matched from: Filename : /usr/lib64/librdkafka++.so.1 adisconbuild-librdkafka1-0.11.3-1.x86_64 : The Apache Kafka C library Repo : rsyslog_v8 Matched from: Filename : /usr/lib64/librdkafka++.so.1 adisconbuild-librdkafka1-0.11.4-1.x86_64 : The Apache Kafka C library Repo : rsyslog_v8 Matched from: Filename : /usr/lib64/librdkafka++.so.1 librdkafka-0.11.3-1.el7.x86_64 : The Apache Kafka C library Repo : @epel Matched from: Filename : /usr/lib64/librdkafka++.so.1 On Fri, Apr 20, 2018 at 11:53 AM, Derek DiFilippo <de...@jsonar.com> wrote: > Hi Florian: > > Thanks for removing that package. > > So you know, there are still others in the rsyslog repo that satisfy the > same dependency > We still see a conflict but now with one of the adiscon-librdkafka > packages. > > The conflict used to be this: > > Transaction check error: > file /usr/lib64/librdkafka++.so.1 conflicts between attempted installs of > librdkafka1-0.8.5-0.x86_64 and librdkafka-0.11.3-1.el7.x86_64 > file /usr/lib64/librdkafka.so.1 conflicts between attempted installs of > librdkafka1-0.8.5-0.x86_64 and librdkafka-0.11.3-1.el7.x86_64 > > and now it's this: > > Transaction check error: > file /usr/lib64/librdkafka++.so.1 conflicts between attempted installs of > adiscon-librdkafka1-0.8.6-1.x86_64 and librdkafka-0.11.3-1.el7.x86_64 > file /usr/lib64/librdkafka.so.1 conflicts between attempted installs of > adiscon-librdkafka1-0.8.6-1.x86_64 and librdkafka-0.11.3-1.el7.x86_64 > > > I'll be that the other packages could be removed as well. Or if necessary > put into a separate internal repository you can use for builds. > > thanks, > -D. > > On Fri, Apr 20, 2018 at 12:57 AM, Florian Riedl <fri...@adiscon.com> > wrote: > >> Derek, quick update. >> >> I removed the librdkafka1 RPM from the repository for EL6 and EL7. >> Since this exact RPM was not used by rsyslog anyway, it seems safe to >> let it go. >> >> I updated the repository accordingly, did a quick install test on my >> lab machine and yum couldn't find it anymore, so I assume this worked >> out right away. >> >> Florian >> >> 2018-04-20 7:46 GMT+02:00 Rainer Gerhards <rgerha...@hq.adiscon.com>: >> > Derek, >> > >> > thanks for the info. This is very useful for me. Obviously I >> > misunderstood how RPMs work in that regard. >> > >> > As as far as librdkafka1 is involved, the root problem is clear. I >> > probably need to review now if we build non-rsyslog components >> > somewhere else which may also cause issues. >> > >> > I think we probably also can have conflicts in regard to support >> > libraries, like libfastjson. This is more or less inevitable. The >> > rsyslog package obviously also conflicts with OS packages. Maybe a >> > warning might be due that folks should not add our repositories if >> > they also use "this and that" component (e.g. another application that >> > requires a different libfastjson)? >> > >> > David: what do you think? >> > >> > Rainer >> > PS: I never was a big fan of doing packaging but it seems to be >> > important enough to justify the troubles associated with it. >> > >> > 2018-04-20 0:36 GMT+02:00 Derek DiFilippo <de...@jsonar.com>: >> >> It's done by file name, not package name. >> >> >> >> To double check I made a simple rpm that builds and packages two of the >> >> examples from the librdkafka repo. >> >> >> >> There's no mention of rsyslog or librdkafka packages anywhere. The >> >> dependencies are driven by the files. >> >> >> >> Two examples below. In the second I disable epel and it finds the >> >> librdkafka1 in the rsyslog repository. >> >> >> >> [ec2-user@ip-172-31-18-125 ~]$ sudo yum install >> >> kafka-conflict-1.1.0-Linux.rpm >> >> Loaded plugins: amazon-id, rhui-lb, search-disabled-repos >> >> Examining kafka-conflict-1.1.0-Linux.rpm: >> kafka-conflict-1.1.0-1.x86_64 >> >> Marking kafka-conflict-1.1.0-Linux.rpm to be installed >> >> Resolving Dependencies >> >> --> Running transaction check >> >> ---> Package kafka-conflict.x86_64 0:1.1.0-1 will be installed >> >> --> Processing Dependency: librdkafka++.so.1()(64bit) for package: >> >> kafka-conflict-1.1.0-1.x86_64 >> >> --> Running transaction check >> >> ---> Package librdkafka.x86_64 0:0.11.3-1.el7 will be installed >> >> --> Finished Dependency Resolution >> >> >> >> Dependencies Resolved >> >> >> >> ============================================================ >> ============================================================ >> ====================== >> >> Package Arch Version >> >> Repository Size >> >> ============================================================ >> ============================================================ >> ====================== >> >> Installing: >> >> kafka-conflict x86_64 1.1.0-1 >> >> /kafka-conflict-1.1.0-Linux 92 k >> >> Installing for dependencies: >> >> librdkafka x86_64 0.11.3-1.el7 >> >> epel 326 k >> >> >> >> Transaction Summary >> >> ============================================================ >> ============================================================ >> ====================== >> >> Install 1 Package (+1 Dependent package) >> >> >> >> Total size: 418 k >> >> Installed size: 1.1 M >> >> >> >> ************ >> >> ************ >> >> >> >> [ec2-user@ip-172-31-18-125 ~]$ sudo yum install --disablerepo=epel >> >> kafka-conflict-1.1.0-Linux.rpm >> >> Loaded plugins: amazon-id, rhui-lb, search-disabled-repos >> >> Examining kafka-conflict-1.1.0-Linux.rpm: >> kafka-conflict-1.1.0-1.x86_64 >> >> Marking kafka-conflict-1.1.0-Linux.rpm to be installed >> >> Resolving Dependencies >> >> --> Running transaction check >> >> ---> Package kafka-conflict.x86_64 0:1.1.0-1 will be installed >> >> --> Processing Dependency: librdkafka++.so.1()(64bit) for package: >> >> kafka-conflict-1.1.0-1.x86_64 >> >> --> Running transaction check >> >> ---> Package librdkafka1.x86_64 0:0.8.5-0 will be installed >> >> --> Finished Dependency Resolution >> >> >> >> Dependencies Resolved >> >> >> >> ============================================================ >> ============================================================ >> ====================== >> >> Package Arch Version >> >> Repository Size >> >> ============================================================ >> ============================================================ >> ====================== >> >> Installing: >> >> kafka-conflict x86_64 1.1.0-1 >> >> /kafka-conflict-1.1.0-Linux 92 k >> >> Installing for dependencies: >> >> librdkafka1 x86_64 0.8.5-0 >> >> rsyslog_v8 100 k >> >> >> >> Transaction Summary >> >> ============================================================ >> ============================================================ >> ====================== >> >> Install 1 Package (+1 Dependent package) >> >> >> >> Total size: 193 k >> >> Total download size: 100 k >> >> Installed size: 366 k >> >> >> >> >> >> >> >> On Thu, Apr 19, 2018 at 2:09 PM, Rainer Gerhards < >> rgerha...@hq.adiscon.com> >> >> wrote: >> >> >> >>> But packages require other packages by package name, not file name - >> or am >> >>> I wrong here? If I am not wrong, it would still mean a manual >> installation? >> >>> >> >>> Rainer >> >>> >> >>> Sent from phone, thus brief. >> >>> >> >>> Derek DiFilippo <de...@jsonar.com> schrieb am Do., 19. Apr. 2018, >> 22:47: >> >>> >> >>> > Well, here's one way to create the bug, but it doesn't have >> anything to >> >>> do >> >>> > with the rsyslog package and it doesn't require any willing >> installation >> >>> of >> >>> > librdkafka1... this is what I've been trying to communicate -- >> apologies >> >>> > that it's been so long winded. >> >>> > >> >>> > Imagine someone had added the rsyslog repo and either had not added >> epel >> >>> or >> >>> > disabled it. >> >>> > >> >>> > Then yum will find librdkafka1 to satisfy the dependency, like so: >> >>> > >> >>> > [ec2-user@ip-172-31-18-125 ~]$ sudo yum whatprovides >> --disablerepo=epel >> >>> > /usr/lib64/librdkafka++.so.1 | grep epel | wc -l >> >>> > 0 >> >>> > [ec2-user@ip-172-31-18-125 ~]$ sudo yum whatprovides >> --disablerepo=epel >> >>> > /usr/lib64/librdkafka++.so.1 | grep rsyslog | wc -l >> >>> > 10 >> >>> > >> >>> > No need to willingly install librdkafka1 or even care about the >> rsyslog >> >>> > package. >> >>> > >> >>> > -D. >> >>> > >> >>> > >> >>> > On Thu, Apr 19, 2018 at 1:34 PM, Rainer Gerhards < >> >>> rgerha...@hq.adiscon.com >> >>> > > >> >>> > wrote: >> >>> > >> >>> > > Sent from phone, thus brief. >> >>> > > >> >>> > > Derek DiFilippo <de...@jsonar.com> schrieb am Do., 19. Apr. 2018, >> >>> 22:05: >> >>> > > >> >>> > > > We install rsyslog by adding the rsyslog repository to >> >>> /etc/yum.repos.d >> >>> > > and >> >>> > > > rely on yum from there. >> >>> > > > >> >>> > > > We do not modify or repackage rsyslog in any way. >> >>> > > > >> >>> > > > Florian can't guarantee anything about librdkafka1, >> unfortunately. >> >>> Once >> >>> > > > you've added the repositories to yum.repos.d, you get the >> following: >> >>> > > > >> >>> > > > [ec2-user@ip-172-31-18-125 ~]$ sudo yum whatprovides >> >>> > > > /usr/lib64/librdkafka++.so.1 | grep rsyslog | wc -l >> >>> > > > 10 >> >>> > > > [ec2-user@ip-172-31-18-125 ~]$ sudo yum whatprovides >> >>> > > > /usr/lib64/librdkafka++.so.1 | grep epel | wc -l >> >>> > > > 1 >> >>> > > > >> >>> > > > Notice -- yum can locate a package that fulfills the dependency >> based >> >>> > on >> >>> > > > file. This is a straight up conflict. >> >>> > > > >> >>> > > > Once again -- we've solved this by building librdkafka from >> source >> >>> and >> >>> > > > linking statically. But, I'm pestering here because if your team >> >>> thinks >> >>> > > > that everything is ok because they gave the rpm package a >> different >> >>> > > > name... that's not the case. >> >>> > > > >> >>> > > >> >>> > > No, it's clear for several hours now that this is wrong and not >> needed. >> >>> > > >> >>> > > BUT: the question is how you experienced that EXCEPT by willingly >> >>> > > installing librdkafka1? >> >>> > > >> >>> > > As far as we know, it is never used nor required by anyone. If it >> is >> >>> > > dragged in by rsyslog, there is a bug inside the rsyslog package. >> So >> >>> > rather >> >>> > > than just removing librdkafka1, I want to make sure we fix the >> bug that >> >>> > > invalidly installs librdkafka1. >> >>> > > >> >>> > > This is what I am after. >> >>> > > >> >>> > > Rainer >> >>> > > >> >>> > > > >> >>> > > > -Derek. >> >>> > > > >> >>> > > > On Thu, Apr 19, 2018 at 12:55 PM, Rainer Gerhards < >> >>> > > > rgerha...@hq.adiscon.com> >> >>> > > > wrote: >> >>> > > > >> >>> > > > > Mmmhhhh. Even then it doesn't totally explain. At least if >> Florian >> >>> is >> >>> > > > > correct in that librdkafka1 was never installed anywhere >> >>> > automatically >> >>> > > > and >> >>> > > > > was never needed by rsyslog. I should call it a day... >> >>> > > > > >> >>> > > > > Rainer >> >>> > > > > >> >>> > > > > Sent from phone, thus brief. >> >>> > > > > >> >>> > > > > Rainer Gerhards <rgerha...@hq.adiscon.com> schrieb am Do., >> 19. >> >>> Apr. >> >>> > > > 2018, >> >>> > > > > 21:46: >> >>> > > > > >> >>> > > > > > Ohhh... Are you a software developer who packages for third >> >>> parties >> >>> > > > (like >> >>> > > > > > we)? Are you concerned the a user of both of our products >> rubs >> >>> into >> >>> > > > this >> >>> > > > > > problem? Then this explains everything... >> >>> > > > > > >> >>> > > > > > I was always under the impression that you are a user who >> >>> installs >> >>> > a >> >>> > > > > > different package in addition to rsyslog. Guess I got that >> wrong, >> >>> > > > > correct? >> >>> > > > > > If so, that explains everything to me... If so, sorry for >> not >> >>> > > > > understanding >> >>> > > > > > the use case. >> >>> > > > > > >> >>> > > > > > Rainer >> >>> > > > > > >> >>> > > > > > Sent from phone, thus brief. >> >>> > > > > > >> >>> > > > > > Derek DiFilippo <de...@jsonar.com> schrieb am Mo., 16. Apr. >> >>> 2018, >> >>> > > > 20:19: >> >>> > > > > > >> >>> > > > > >> Hi Rainer, hi everyone, >> >>> > > > > >> >> >>> > > > > >> In addition to a dependency on rsyslog, one of our >> packages now >> >>> > > > depends >> >>> > > > > on >> >>> > > > > >> librdkafka and librdkafka-devel. >> >>> > > > > >> >> >>> > > > > >> We want to use the librdkafka packages provided by epel, >> nothing >> >>> > > > fancy. >> >>> > > > > >> >> >>> > > > > >> I'm seeing a conflict between the librdkafka1 package in >> the >> >>> > rsyslog >> >>> > > > > repo >> >>> > > > > >> (note the "1" at the end of the library name) and the ones >> >>> > provided >> >>> > > by >> >>> > > > > >> epel. >> >>> > > > > >> >> >>> > > > > >> ... >> >>> > > > > >> ... >> >>> > > > > >> librdkafka x86_64 0.11.3-1.el7 epel >> >>> > > > > >> 326 k >> >>> > > > > >> librdkafka-devel x86_64 0.11.3-1.el7 epel >> >>> > > > > >> 43 k >> >>> > > > > >> librdkafka1 x86_64 0.8.5-0 rsyslog_v8 >> >>> > > > > >> 100 k >> >>> > > > > >> ... >> >>> > > > > >> ... >> >>> > > > > >> >> >>> > > > > >> Transaction check error: >> >>> > > > > >> file /usr/lib64/librdkafka++.so.1 conflicts between >> attempted >> >>> > > > installs >> >>> > > > > >> of >> >>> > > > > >> librdkafka1-0.8.5-0.x86_64 and >> librdkafka-0.11.3-1.el7.x86_64 >> >>> > > > > >> file /usr/lib64/librdkafka.so.1 conflicts between >> attempted >> >>> > > installs >> >>> > > > > of >> >>> > > > > >> librdkafka1-0.8.5-0.x86_64 and >> librdkafka-0.11.3-1.el7.x86_64 >> >>> > > > > >> >> >>> > > > > >> Can librdkafka1 be removed from the rsyslog repo? What is >> the >> >>> > > > rationale >> >>> > > > > >> for >> >>> > > > > >> pinning a version of librdkafka via the librdkafka1 >> package? >> >>> > > > > >> >> >>> > > > > >> If librdkafka1 must stay in the rsyslog repo what do you >> >>> recommend >> >>> > > as >> >>> > > > > the >> >>> > > > > >> best practice for resolving this conflict? >> >>> > > > > >> >> >>> > > > > >> I see a lot of "adisconbuild-librdkafka*" packages in the >> repo. >> >>> > Why >> >>> > > > are >> >>> > > > > >> they there? Is the solution to change the dependency in >> *our* >> >>> > > package >> >>> > > > > form >> >>> > > > > >> librdkafka to adiscon-librdkafka? I might be comfortable >> with >> >>> that >> >>> > > if >> >>> > > > I >> >>> > > > > >> know why the adiscon packages are there. >> >>> > > > > >> >> >>> > > > > >> As a fallback we could add librdkafka as an external >> project via >> >>> > > CMake >> >>> > > > > and >> >>> > > > > >> build from source but I wanted to check in here first >> before >> >>> > > > proceeding. >> >>> > > > > >> >> >>> > > > > >> Apologies if I missed something obvious. I did my best >> googling >> >>> > > > > >> due-diligence and couldn't find anyhing satisfactory. >> >>> > > > > >> >> >>> > > > > >> Thanks for your time & patience! >> >>> > > > > >> _______________________________________________ >> >>> > > > > >> rsyslog mailing list >> >>> > > > > >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >>> > > > > >> http://www.rsyslog.com/professional-services/ >> >>> > > > > >> What's up with rsyslog? Follow >> https://twitter.com/rgerhards >> >>> > > > > >> NOTE WELL: This is a PUBLIC mailing list, posts are >> ARCHIVED by >> >>> a >> >>> > > > myriad >> >>> > > > > >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT >> POST >> >>> if >> >>> > > you >> >>> > > > > >> DON'T LIKE THAT. >> >>> > > > > >> >> >>> > > > > > >> >>> > > > > _______________________________________________ >> >>> > > > > rsyslog mailing list >> >>> > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog >> >>> > > > > http://www.rsyslog.com/professional-services/ >> >>> > > > > What's up with rsyslog? Follow https://twitter.com/rgerhards >> >>> > > > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED >> by a >> >>> > > myriad >> >>> > > > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT >> POST if >> >>> > you >> >>> > > > > DON'T LIKE THAT. >> >>> > > > > >> >>> > > > _______________________________________________ >> >>> > > > rsyslog mailing list >> >>> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog >> >>> > > > http://www.rsyslog.com/professional-services/ >> >>> > > > What's up with rsyslog? Follow https://twitter.com/rgerhards >> >>> > > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by >> a >> >>> > myriad >> >>> > > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST >> if >> >>> you >> >>> > > > DON'T LIKE THAT. >> >>> > > > >> >>> > > _______________________________________________ >> >>> > > rsyslog mailing list >> >>> > > http://lists.adiscon.net/mailman/listinfo/rsyslog >> >>> > > http://www.rsyslog.com/professional-services/ >> >>> > > What's up with rsyslog? Follow https://twitter.com/rgerhards >> >>> > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> >>> myriad >> >>> > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST >> if you >> >>> > > DON'T LIKE THAT. >> >>> > > >> >>> > _______________________________________________ >> >>> > rsyslog mailing list >> >>> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> >>> > http://www.rsyslog.com/professional-services/ >> >>> > What's up with rsyslog? Follow https://twitter.com/rgerhards >> >>> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> myriad >> >>> > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >> you >> >>> > DON'T LIKE THAT. >> >>> > >> >>> _______________________________________________ >> >>> rsyslog mailing list >> >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >>> http://www.rsyslog.com/professional-services/ >> >>> What's up with rsyslog? Follow https://twitter.com/rgerhards >> >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> myriad >> >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> >>> DON'T LIKE THAT. >> >>> >> >> _______________________________________________ >> >> rsyslog mailing list >> >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> http://www.rsyslog.com/professional-services/ >> >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >> you DON'T LIKE THAT. >> > _______________________________________________ >> > rsyslog mailing list >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > http://www.rsyslog.com/professional-services/ >> > What's up with rsyslog? Follow https://twitter.com/rgerhards >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >> you DON'T LIKE THAT. >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> > > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
