Thanks for the info Dennis! I tried 'rpm repo sync sun --force-full' but it didn't help to clear the corrupted entry. How can I drop this on the server side? Should I just directly edit an entry in mongodb?
пт, 2 окт. 2020 г. в 14:39, Dennis Kliban <dkli...@redhat.com>: > Filelists are stored in the database in a compressed form[0]. During > publishing of metadata, each fillist is decompressed before being written > to a file[1]. It's hard to speculate when data corruption occurred. It's > possible that the data was corrupted in the remote repository the first > time that this RPM was downloaded. That metadata could have been fixed > since then, but Pulp thinks it already has the information and is not > trying to re-download. > > [0] > https://github.com/pulp/pulp_rpm/blob/2-master/plugins/pulp_rpm/plugins/db/models.py#L1051 > [1] > https://github.com/pulp/pulp_rpm/blob/2-master/plugins/pulp_rpm/plugins/db/models.py#L1064 > > On Fri, Oct 2, 2020 at 3:27 AM Konstantin M. Khankin < > khankin.konstan...@gmail.com> wrote: > >> Sorry for reviving this thread but I think I found something. >> >> Today I tried to install CentOS 7 and got an error: >> "TypeError: Parsing filelists.xml error: expected '>'" >> >> So I did this: >> >> >>> import xml.etree.ElementTree as ET >> >>> tree = >> ET.parse('/mnt/sysimage/var/tmp/yum.cache/centos_updates/gen/filelists.xml') >> ... >> File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1506, in >> _raiseerror >> raise err >> xml.etree.ElementTree.ParseError: not well-formed (invalid token): line >> 1884123, column 136 >> >> I opened filelists.xml and found that line 1884123 is indeed corrupt and >> looks like this: >> >> <file>/usr/share/javadoc/java-1.8.0-openjdk-1.8.0.111-1.b15.el7_2-debug/api/javax/sql/rowset/serial/compact3-package-frame.html</fil% >> !,^@ 0 >> 480!share/javadoc/java-1.8.0-openjdk-1.8.0.111-1.b15.el7_2-debug/api/javax/sql/rowset/serial/compact3-package-summary.html</file> >> >> I verified another consumer has exactly the same issue with this file. >> >> Do I understand correctly Pulp generates filelists.xml? If so, can it be >> because of a bug in Pulp or should I look for a silent data corruption >> issue on my pulp server? >> >> Thanks! >> >> >> вт, 1 сент. 2020 г. в 17:14, Dennis Kliban <dkli...@redhat.com>: >> >>> The repoview feature requires the sqlite db to be generated. This >>> feature generates HTML pages for browsing the repository in a web browser. >>> I believe that users are still able to browse the repository without it, >>> but in that case the HTML listing of directories is generated by the web >>> server each time a user requests it. >>> >>> On Tue, Sep 1, 2020 at 4:53 AM Konstantin M. Khankin < >>> khankin.konstan...@gmail.com> wrote: >>> >>>> Thank you, that obviously helped. Still would be useful to know the >>>> root cause. >>>> >>>> OTOH, if sqlite creation functionality is not that critical - why would >>>> we ever enable something which only consumes time during sync for no >>>> benefit? >>>> >>>> вт, 1 сент. 2020 г. в 00:26, Dennis Kliban <dkli...@redhat.com>: >>>> >>>>> Looks like everything is up to date. I have no idea what the root >>>>> cause is, but according to this comment[0], you can work around the >>>>> problem >>>>> by disabling the generation of sqlite db. I am not sure what the exact >>>>> effect of this will be for the clients that are consuming this content, >>>>> but >>>>> the repository will be usable. >>>>> >>>>> [0] https://pulp.plan.io/issues/2019#note-2 >>>>> >>>>> On Mon, Aug 31, 2020 at 4:10 PM Konstantin M. Khankin < >>>>> khankin.konstan...@gmail.com> wrote: >>>>> >>>>>> # rpm -qa | grep pulp | sort >>>>>> pulp-admin-client-2.21.3-1.el7.noarch >>>>>> pulp-agent-2.21.3-1.el7.noarch >>>>>> pulp-consumer-client-2.21.3-1.el7.noarch >>>>>> pulp-deb-admin-extensions-1.10.1-1.el7.noarch >>>>>> pulp-deb-plugins-1.10.1-1.el7.noarch >>>>>> pulp-docker-admin-extensions-3.2.6-1.el7.noarch >>>>>> pulp-docker-plugins-3.2.6-1.el7.noarch >>>>>> pulp-puppet-consumer-extensions-2.21.3-1.el7.noarch >>>>>> pulp-puppet-handlers-2.21.3-1.el7.noarch >>>>>> pulp-python-admin-extensions-2.0.4-1.el7.noarch >>>>>> pulp-python-plugins-2.0.4-1.el7.noarch >>>>>> pulp-rpm-admin-extensions-2.21.3-1.el7.noarch >>>>>> pulp-rpm-consumer-extensions-2.21.3-1.el7.noarch >>>>>> pulp-rpm-handlers-2.21.3-1.el7.noarch >>>>>> pulp-rpm-plugins-2.21.3-1.el7.noarch >>>>>> pulp-rpm-yumplugins-2.21.3-1.el7.noarch >>>>>> pulp-selinux-2.21.3-1.el7.noarch >>>>>> pulp-server-2.21.3-1.el7.noarch >>>>>> python-pulp-agent-lib-2.21.3-1.el7.noarch >>>>>> python-pulp-bindings-2.21.3-1.el7.noarch >>>>>> python-pulp-client-lib-2.21.3-1.el7.noarch >>>>>> python-pulp-common-2.21.3-1.el7.noarch >>>>>> python-pulp-deb-common-1.10.1-1.el7.noarch >>>>>> python-pulp-docker-common-3.2.6-1.el7.noarch >>>>>> python-pulp-oid_validation-2.21.3-1.el7.noarch >>>>>> python-pulp-puppet-common-2.21.3-1.el7.noarch >>>>>> python-pulp-python-common-2.0.4-1.el7.noarch >>>>>> python-pulp-repoauth-2.21.3-1.el7.noarch >>>>>> python-pulp-rpm-common-2.21.3-1.el7.noarch >>>>>> >>>>>> # rpm -qa | grep createrepo >>>>>> createrepo-0.9.9-28.el7.noarch >>>>>> createrepo_c-0.10.0-20.el7.x86_64 >>>>>> createrepo_c-libs-0.10.0-20.el7.x86_64 >>>>>> >>>>>> # cat /etc/redhat-release >>>>>> CentOS Linux release 7.8.2003 (Core) >>>>>> >>>>>> пн, 31 авг. 2020 г. в 22:48, Dennis Kliban <dkli...@redhat.com>: >>>>>> >>>>>>> This looks exactly like the issue that was reported here[0]. >>>>>>> >>>>>>> What version of pulp are you using? What version of createrepo_c is >>>>>>> installed? >>>>>>> >>>>>>> [0] https://pulp.plan.io/issues/2019 >>>>>>> >>>>>>> On Mon, Aug 31, 2020 at 3:16 PM Konstantin M. Khankin < >>>>>>> khankin.konstan...@gmail.com> wrote: >>>>>>> >>>>>>>> Hi! >>>>>>>> >>>>>>>> The issue still persists. Could someone take a look please? >>>>>>>> >>>>>>>> Thanks! >>>>>>>> >>>>>>>> ср, 19 авг. 2020 г., 13:15 Konstantin M. Khankin < >>>>>>>> khankin.konstan...@gmail.com>: >>>>>>>> >>>>>>>>> Hi! >>>>>>>>> >>>>>>>>> I found that my pulp2-managed mirror of >>>>>>>>> http://mirror.yandex.ru/centos/7/updates/x86_64 has not been >>>>>>>>> successfully published since April 30. I ran publish manually and >>>>>>>>> received >>>>>>>>> an error: >>>>>>>>> >>>>>>>>> ''' >>>>>>>>> Generating sqlite files >>>>>>>>> [/] >>>>>>>>> ... failed >>>>>>>>> Error occurred during 'sqliterepo_c' execution: Preparing sqlite >>>>>>>>> DBs >>>>>>>>> >>>>>>>>> :: >>>>>>>>> C_CREATEREPOLIB: Critical: cr_xml_parser_generic: parsing error >>>>>>>>> '/var/cache/pulp/reserved_resource_worke...@hive.gsk.loc >>>>>>>>> /12257118-33e7-4294-ad68 >>>>>>>>> >>>>>>>>> -98e0515f2627/repodata/9e958c09c7880d130ef3321332000f43a4e4e701e416801a0f54313f5 >>>>>>>>> 766d03b-filelists.xml.gz': not well-formed (invalid token) >>>>>>>>> Parse error >>>>>>>>> '/var/cache/pulp/reserved_resource_worke...@hive.gsk.loc >>>>>>>>> /12257118-33e7-4294-ad68 >>>>>>>>> >>>>>>>>> -98e0515f2627/repodata/9e958c09c7880d130ef3321332000f43a4e4e701e416801a0f54313f5 >>>>>>>>> 766d03b-filelists.xml.gz' at line: 1884123 (not well-formed >>>>>>>>> (invalid token)) >>>>>>>>> ''' >>>>>>>>> >>>>>>>>> I can't open .sqlite files in /tmp from CLI either: >>>>>>>>> >>>>>>>>> ''' >>>>>>>>> -rw-------. 1 apache apache 54066176 Aug 19 15:12 >>>>>>>>> filelists.211JP0.sqlite >>>>>>>>> -rw-------. 1 apache apache 157097984 Aug 19 15:12 >>>>>>>>> primary.PQ3JP0.sqlite >>>>>>>>> -rw-------. 1 apache apache 0 Aug 19 15:12 >>>>>>>>> other.X201JP0.sqlite >>>>>>>>> >>>>>>>>> [root@hive tmp]# sqlite3 primary.PQ3JP0.sqlite >>>>>>>>> SQLite version 3.7.17 2013-05-20 00:56:22 >>>>>>>>> Enter ".help" for instructions >>>>>>>>> Enter SQL statements terminated with a ";" >>>>>>>>> sqlite> .databases >>>>>>>>> Error: file is encrypted or is not a database >>>>>>>>> ''' >>>>>>>>> >>>>>>>>> I have multiple repos managed by pulp, some of which originate >>>>>>>>> also from mirror.yandex.ru, and they are synced and published >>>>>>>>> normally. --force-full doesn't help. >>>>>>>>> >>>>>>>>> What issue could be here? >>>>>>>>> >>>>>>>>> Thanks! >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Konstantin Khankin >>>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Pulp-list mailing list >>>>>>>> Pulp-list@redhat.com >>>>>>>> https://www.redhat.com/mailman/listinfo/pulp-list >>>>>>> >>>>>>> >>>>>> >>>>>> -- >>>>>> Ханкин Константин >>>>>> >>>>> >>>> >>>> -- >>>> Ханкин Константин >>>> >>> >> >> -- >> Ханкин Константин >> > -- Ханкин Константин
_______________________________________________ Pulp-list mailing list Pulp-list@redhat.com https://www.redhat.com/mailman/listinfo/pulp-list