Could you create a clone of your production system and try upgrading the cloned environment? I suspect that some problems may have been created from the extra migration being created in between all the upgrade attempts on your test system.
Could you provide the SQL schema (without data) from the database of the system that is experiencing the problem? We want to compare to a clean installed pulp 3.3.1 schema. Could you provide the SQL schema (without data) from the database of the 3.2 production system? On Wed, May 13, 2020 at 11:52 AM Bin Li (BLOOMBERG/ 120 PARK) < bli...@bloomberg.net> wrote: > Here is what I have > # django-admin showmigrations > admin > [X] 0001_initial > [X] 0002_logentry_remove_auto_add > [X] 0003_logentry_add_action_flag_choices > auth > [X] 0001_initial > [X] 0002_alter_permission_name_max_length > [X] 0003_alter_user_email_max_length > [X] 0004_alter_user_username_opts > [X] 0005_alter_user_last_login_null > [X] 0006_require_contenttypes_0002 > [X] 0007_alter_validators_add_error_messages > [X] 0008_alter_user_username_max_length > [X] 0009_alter_user_last_name_max_length > [X] 0010_alter_group_name_max_length > [X] 0011_update_proxy_permissions > contenttypes > [X] 0001_initial > [X] 0002_remove_content_type_name > core > [X] 0001_initial > [X] 0002_increase_artifact_size_field > [X] 0003_remove_upload_completed > [X] 0004_add_duplicated_reserved_resources > [X] 0005_progressreport_code > [X] 0006_repository_plugin_managed > [X] 0007_delete_progress_proxies > [X] 0008_published_metadata_as_content > [X] 0009_remove_task_non_fatal_errors > [X] 0010_pulp_fields > [X] 0011_relative_path > [X] 0012_auto_20191104_2000 > [X] 0013_repository_pulp_type > [X] 0014_remove_repository_plugin_managed > [X] 0015_auto_20191112_1426 > [X] 0016_charfield_to_textfield > [X] 0017_remove_task_parent > [X] 0018_auto_20191127_2350 > [X] 0019_add_signing_service_model > [X] 0020_change_publishedartifact_constraints > [X] 0021_add_signing_service_foreign_key > [X] 0022_rename_last_version > [X] 0023_change_exporter_models > [X] 0024_use_local_storage_for_uploads > [X] 0025_task_parent_task > [X] 0026_task_group > [X] 0027_export_backend > [X] 0028_import_importer_pulpimporter_pulpimporterrepository > file > [X] 0001_initial > [X] 0002_file_related_names > [X] 0003_auto_20191014_1721 > [X] 0004_filefilesystemexporter > [X] 0005_filerepository > [X] 0006_delete_filefilesystemexporter > [X] 0007_filefilesystemexporter > rpm > [X] 0001_initial > [X] 0002_updaterecord_reboot_suggested > [X] 0003_DATA_incorrect_json > [X] 0004_add_metadata_signing_service_fk > [X] 0005_optimize_sync > [X] 0006_opensuse_support > [X] 0007_checksum_types > [X] 0008_advisory_pkg_sumtype_as_int > [X] 0009_revision_null > sessions > [X] 0001_initial > > > From: ipan...@redhat.com At: 05/13/20 11:41:13 > To: Bin Li (BLOOMBERG/ 120 PARK ) <bli...@bloomberg.net> > Cc: ttere...@redhat.com, pulp-list@redhat.com > Subject: Re: [Pulp-list] pulp 3.3.1 server error > > Can you confirm that migration 0009_revision_null was applied? > > <https://github.com/pulp/pulp_rpm/pull/1694/files#diff-baf8ad85df1ed2543d715f2a86de6cb4> > > You can see the migrations with this command 'django-admin showmigrations' > > -------- > Regards, > > Ina Panova > Senior Software Engineer| Pulp| Red Hat Inc. > > "Do not go where the path may lead, > go instead where there is no path and leave a trail." > > > On Wed, May 13, 2020 at 4:09 PM Bin Li (BLOOMBERG/ 120 PARK) < > bli...@bloomberg.net> wrote: > >> Hi Ina, >> >> The error was not from migration. I see the error when I try to create a >> new repo. My pulp_installer is on the latest version. Rerun the installer >> doesn't solve the issue. What should I do to fix this? We cannot reset >> database from scratch since we already have a lot data. >> >> From: ipan...@redhat.com At: 05/13/20 09:43:45 >> To: Bin Li (BLOOMBERG/ 120 PARK ) <bli...@bloomberg.net> >> Cc: ttere...@redhat.com, pulp-list@redhat.com >> Subject: Re: [Pulp-list] pulp 3.3.1 server error >> >> This issue was fixed in the 3.3.1 release [0][1] >> >> [0] https://pulp-rpm.readthedocs.io/en/latest/changes.html#bugfixes >> [1] https://pulp.plan.io/issues/6662 >> >> >> -------- >> Regards, >> >> Ina Panova >> Senior Software Engineer| Pulp| Red Hat Inc. >> >> "Do not go where the path may lead, >> go instead where there is no path and leave a trail." >> >> >> On Tue, May 12, 2020 at 11:30 PM Bin Li (BLOOMBERG/ 120 PARK) < >> bli...@bloomberg.net> wrote: >> >>> I removed the 0009_auto.. and rerun the installer. The upgrade was >>> successful. >>> Below is error log. >>> >>> May 12 17:25:05 myhost gunicorn[55426]: pulp: django.request:ERROR: >>> Internal Server Error: /pulp/api/v3/repositories/rpm/rpm/ >>> May 12 17:25:05 myhost gunicorn[55426]: Traceback (most recent call >>> last): >>> May 12 17:25:05 myhost gunicorn[55426]: File >>> "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/backends/utils.py", >>> line 84, in _execute >>> May 12 17:25:05 myhost gunicorn[55426]: return self.cursor.execute(sql, >>> params) >>> May 12 17:25:05 myhost gunicorn[55426]: >>> psycopg2.errors.NotNullViolation: null value in column >>> "last_sync_revision_number" violates not-null constraint >>> May 12 17:25:05 myhost gunicorn[55426]: DETAIL: Failing row contains >>> (0053634d-d0ca-46de-81d4-b8acf19d592e, f, null, null, 0, null, {}). >>> May 12 17:25:05 myhost gunicorn[55426]: The above exception was the >>> direct cause of the following exception: >>> May 12 17:25:05 myhost gunicorn[55426]: Traceback (most recent call >>> last): >>> May 12 17:25:05 myhost gunicorn[55426]: File >>> "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/core/handlers/exception.py", >>> line 34, in inner >>> May 12 17:25:05 myhost gunicorn[55426]: response = get_response(request) >>> May 12 17:25:05 myhost gunicorn[55426]: File >>> "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/core/handlers/base.py", >>> line 115, in _get_response >>> May 12 17:25:05 myhost gunicorn[55426]: response = >>> self.process_exception_by_middleware(e, request) >>> May 12 17:25:05 myhost gunicorn[55426]: File >>> "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/core/handlers/base.py", >>> line 113, in _get_response >>> May 12 17:25:05 myhost gunicorn[55426]: response = >>> wrapped_callback(request, *callback_args, **callback_kwargs) >>> May 12 17:25:05 myhost gunicorn[55426]: File >>> "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/views/decorators/csrf.py", >>> line 54, in wrapped_view >>> May 12 17:25:05 myhost gunicorn[55426]: return view_func(*args, **kwargs) >>> May 12 17:25:05 myhost gunicorn[55426]: File >>> "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/rest_framework/viewsets.py", >>> line 114, in view >>> May 12 17:25:05 myhost gunicorn[55426]: return self.dispatch(request, >>> *args, **kwargs) >>> May 12 17:25:05 myhost gunicorn[55426]: File >>> "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/rest_framework/views.py", >>> line 505, in dispatch >>> May 12 17:25:05 myhost gunicorn[55426]: response = >>> self.handle_exception(exc) >>> May 12 17:25:05 myhost gunicorn[55426]: File >>> "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/rest_framework/views.py", >>> line 465, in handle_exception >>> May 12 17:25:05 myhost gunicorn[55426]: >>> self.raise_uncaught_exception(exc) >>> May 12 17:25:05 myhost gunicorn[55426]: File >>> "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/rest_framework/views.py", >>> line 476, in raise_uncaught_exception >>> May 12 17:25:05 myhost gunicorn[55426]: raise exc >>> May 12 17:25:05 myhost gunicorn[55426]: File >>> "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/rest_framework/views.py", >>> line 502, in dispatch >>> May 12 17:25:05 myhost gunicorn[55426]: response = handler(request, >>> *args, **kwargs) >>> May 12 17:25:05 myhost gunicorn[55426]: File >>> "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/rest_framework/mixins.py", >>> line 19, in create >>> May 12 17:25:05 myhost gunicorn[55426]: self.perform_create(serializer) >>> May 12 17:25:05 myhost gunicorn[55426]: File >>> "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/rest_framework/mixins.py", >>> line 24, in perform_create >>> May 12 17:25:05 myhost gunicorn[55426]: serializer.save() >>> May 12 17:25:05 myhost gunicorn[55426]: File >>> "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/rest_framework/serializers.py", >>> line 213, in save >>> May 12 17:25:05 myhost gunicorn[55426]: self.instance = >>> self.create(validated_data) >>> May 12 17:25:05 myhost gunicorn[55426]: File >>> "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/rest_framework/serializers.py", >>> line 932, in create >>> May 12 17:25:05 myhost gunicorn[55426]: instance = >>> ModelClass._default_manager.create(**validated_data) >>> May 12 17:25:05 myhost gunicorn[55426]: File >>> "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/models/manager.py", >>> line 82, in manager_method >>> May 12 17:25:05 myhost gunicorn[55426]: return >>> getattr(self.get_queryset(), name)(*args, **kwargs) >>> May 12 17:25:05 myhost gunicorn[55426]: File >>> "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/models/query.py", >>> line 422, in create >>> May 12 17:25:05 myhost gunicorn[55426]: obj.save(force_insert=True, >>> using=self.db) >>> May 12 17:25:05 myhost gunicorn[55426]: File >>> "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/pulpcore/app/models/repository.py", >>> line 62, in save >>> May 12 17:25:05 myhost gunicorn[55426]: super().save(*args, **kwargs) >>> May 12 17:25:05 myhost gunicorn[55426]: File >>> "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/pulpcore/app/models/base.py", >>> line 107, in save >>> May 12 17:25:05 myhost gunicorn[55426]: return super().save(*args, >>> **kwargs) >>> May 12 17:25:05 myhost gunicorn[55426]: File >>> "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/models/base.py", >>> line 741, in save >>> May 12 17:25:05 myhost gunicorn[55426]: force_update=force_update, >>> update_fields=update_fields) >>> May 12 17:25:05 myhost gunicorn[55426]: File >>> "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/models/base.py", >>> line 779, in save_base >>> May 12 17:25:05 myhost gunicorn[55426]: force_update, using, >>> update_fields, >>> May 12 17:25:05 myhost gunicorn[55426]: File >>> "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/models/base.py", >>> line 870, in _save_table >>> May 12 17:25:05 myhost gunicorn[55426]: result = >>> self._do_insert(cls._base_manager, using, fields, update_pk, raw) >>> May 12 17:25:05 myhost gunicorn[55426]: File >>> "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/models/base.py", >>> line 908, in _do_insert >>> May 12 17:25:05 myhost gunicorn[55426]: using=using, raw=raw) >>> May 12 17:25:05 myhost gunicorn[55426]: File >>> "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/models/manager.py", >>> line 82, in manager_method >>> May 12 17:25:05 myhost gunicorn[55426]: return >>> getattr(self.get_queryset(), name)(*args, **kwargs) >>> May 12 17:25:05 myhost gunicorn[55426]: File >>> "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/models/query.py", >>> line 1186, in _insert >>> May 12 17:25:05 myhost gunicorn[55426]: return >>> query.get_compiler(using=using).execute_sql(return_id) >>> May 12 17:25:05 myhost gunicorn[55426]: File >>> "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/models/sql/compiler.py", >>> line 1368, in execute_sql >>> May 12 17:25:05 myhost gunicorn[55426]: cursor.execute(sql, params) >>> May 12 17:25:05 myhost gunicorn[55426]: File >>> "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/backends/utils.py", >>> line 67, in execute >>> May 12 17:25:05 myhost gunicorn[55426]: return >>> self._execute_with_wrappers(sql, params, many=False, executor=self._execute) >>> May 12 17:25:05 myhost gunicorn[55426]: File >>> "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/backends/utils.py", >>> line 76, in _execute_with_wrappers >>> May 12 17:25:05 myhost gunicorn[55426]: return executor(sql, params, >>> many, context) >>> May 12 17:25:05 myhost gunicorn[55426]: File >>> "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/backends/utils.py", >>> line 84, in _execute >>> May 12 17:25:05 myhost gunicorn[55426]: return self.cursor.execute(sql, >>> params) >>> May 12 17:25:05 myhost gunicorn[55426]: File >>> "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/utils.py", >>> line 89, in __exit__ >>> May 12 17:25:05 myhost gunicorn[55426]: raise >>> dj_exc_value.with_traceback(traceback) from exc_value >>> May 12 17:25:05 myhost gunicorn[55426]: File >>> "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/backends/utils.py", >>> line 84, in _execute >>> May 12 17:25:05 myhost gunicorn[55426]: return self.cursor.execute(sql, >>> params) >>> May 12 17:25:05 myhost gunicorn[55426]: django.db.utils.IntegrityError: >>> null value in column "last_sync_revision_number" violates not-null >>> constraint >>> May 12 17:25:05 myhost gunicorn[55426]: DETAIL: Failing row contains >>> (0053634d-d0ca-46de-81d4-b8acf19d592e, f, null, null, 0, null, {}). >>> May 12 17:25:05 myhost gunicorn[55426]: 127.0.0.1 - admin >>> [12/May/2020:21:25:05 +0000] "POST /pulp/api/v3/repositories/rpm/rpm/ >>> HTTP/1.1" 500 27 "-" "HTTPie/0.9.4" >>> >>> >>> From: ttere...@redhat.com At: 05/12/20 16:19:31 >>> To: Bin Li (BLOOMBERG/ 120 PARK ) <bli...@bloomberg.net> >>> Cc: pulp-list@redhat.com >>> Subject: Re: [Pulp-list] pulp 3.3.1 server error >>> >>> Was the upgrade successful after you removed the 0009_auto...? Did you >>> run the installer again? >>> >>> If no, please do so. >>> If yes, could you share logs? There will be some traceback showing why >>> 500 error happened. >>> >>> Thank you, >>> Tanya >>> >>> >>> _______________________________________________ >>> Pulp-list mailing list >>> Pulp-list@redhat.com >>> https://www.redhat.com/mailman/listinfo/pulp-list >> >> >> > _______________________________________________ > Pulp-list mailing list > Pulp-list@redhat.com > https://www.redhat.com/mailman/listinfo/pulp-list
_______________________________________________ Pulp-list mailing list Pulp-list@redhat.com https://www.redhat.com/mailman/listinfo/pulp-list
