Adding CONTENT_HOST = 'http://localhost:8080' to setting.py fixed the issue. However, got connection error when try to access a distribution.
# http get http://localhost:8080/pulp/content/bloomberg/systems_third_party_rhel7.2/ http: error: ConnectionError: ('Connection aborted.', error(111, 'Connection refused')) while doing GET request to URL: http://localhost:8080/pulp/content/bloomberg/systems_third_party_rhel7.2/ The port 8080 is not listening. What am I doing wrong? The distribution is as following: { "count": 1, "next": null, "previous": null, "results": [ { "_created": "2019-06-25T18:51:14.803243Z", "_href": "/pulp/api/v3/distributions/rpm/rpm/6cde0eda-ea68-4d22-920d-c62173c4d80e/", "base_path": "bloomberg/systems_third_party_rhel7.2", "base_url": "http://localhost:8080/pulp/content/bloomberg/systems_third_party_rhel7.2";, "content_guard": null, "name": "systems_third_party_rhel7.2", "publication": "/pulp/api/v3/publications/rpm/rpm/894a8109-001b-42fc-8658-4ac28bc52c0e/" } ] } Thanks From: [email protected] At: 06/25/19 14:07:56To: Bin Li (BLOOMBERG/ 120 PARK ) Cc: [email protected] Subject: Re: [Pulp-list] Pulp 3 internal error This is the same bug as I mentioned earlier. You need to set either the PULP_CONTENT_HOST environment variable or CONTENT_HOST setting in /etc/pulp/settings.py On Tue, Jun 25, 2019 at 1:57 PM Bin Li (BLOOMBERG/ 120 PARK) <[email protected]> wrote: Here is the traceback from the log: Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: pulp: django.request:ERROR: Internal Server Error: /pulp/api/v3/tasks/ Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: Traceback (most recent call last): Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: File "/opt/python/3.6.5/lib/python3.6/site-packages/django/core/handlers/exception.py", line 34, in inner Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: response = get_response(request) Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: File "/opt/python/3.6.5/lib/python3.6/site-packages/django/core/handlers/base.py", line 115, in _get_response Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: response = self.process_exception_by_middleware(e, request) Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: File "/opt/python/3.6.5/lib/python3.6/site-packages/django/core/handlers/base.py", line 113, in _get_response Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: response = wrapped_callback(request, *callback_args, **callback_kwargs) Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: File "/opt/python/3.6.5/lib/python3.6/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: return view_func(*args, **kwargs) Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: File "/opt/python/3.6.5/lib/python3.6/site-packages/rest_framework/viewsets.py", line 116, in view Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: return self.dispatch(request, *args, **kwargs) Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: File "/opt/python/3.6.5/lib/python3.6/site-packages/rest_framework/views.py", line 495, in dispatch Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: response = self.handle_exception(exc) Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: File "/opt/python/3.6.5/lib/python3.6/site-packages/rest_framework/views.py", line 455, in handle_exception Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: self.raise_uncaught_exception(exc) Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: File "/opt/python/3.6.5/lib/python3.6/site-packages/rest_framework/views.py", line 492, in dispatch Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: response = handler(request, *args, **kwargs) Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: File "/opt/python/3.6.5/lib/python3.6/site-packages/rest_framework/mixins.py", line 45, in list Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: return self.get_paginated_response(serializer.data) Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: File "/opt/python/3.6.5/lib/python3.6/site-packages/rest_framework/serializers.py", line 768, in data Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: ret = super(ListSerializer, self).data Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: File "/opt/python/3.6.5/lib/python3.6/site-packages/rest_framework/serializers.py", line 262, in data Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: self._data = self.to_representation(self.instance) Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: File "/opt/python/3.6.5/lib/python3.6/site-packages/rest_framework/serializers.py", line 686, in to_representation Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: self.child.to_representation(item) for item in iterable Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: File "/opt/python/3.6.5/lib/python3.6/site-packages/rest_framework/serializers.py", line 686, in <listcomp> Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: self.child.to_representation(item) for item in iterable Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: File "/opt/python/3.6.5/lib/python3.6/site-packages/rest_framework/serializers.py", line 530, in to_representation Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: ret[field.field_name] = field.to_representation(attribute) Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: File "/opt/python/3.6.5/lib/python3.6/site-packages/rest_framework/relations.py", line 542, in to_representation Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: for value in iterable Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: File "/opt/python/3.6.5/lib/python3.6/site-packages/rest_framework/relations.py", line 542, in <listcomp> Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: for value in iterable Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: File "/opt/python/3.6.5/lib/python3.6/site-packages/pulpcore-3.0.0rc3.dev0-py3.6.egg/pulpcore/app/serializers/task.py", line 25, in to_representation Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: return serializer.data.get('_href') Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: File "/opt/python/3.6.5/lib/python3.6/site-packages/rest_framework/serializers.py", line 563, in data Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: ret = super(Serializer, self).data Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: File "/opt/python/3.6.5/lib/python3.6/site-packages/rest_framework/serializers.py", line 262, in data Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: self._data = self.to_representation(self.instance) Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: File "/opt/python/3.6.5/lib/python3.6/site-packages/rest_framework/serializers.py", line 530, in to_representation Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: ret[field.field_name] = field.to_representation(attribute) Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: File "/opt/python/3.6.5/lib/python3.6/site-packages/pulpcore-3.0.0rc3.dev0-py3.6.egg/pulpcore/app/serializers/fields.py", line 248, in to_representation Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: host.strip('/'), Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: AttributeError: 'NoneType' object has no attribute 'strip' Jun 25 13:53:08 ip-1-76-158-221 gunicorn[147692]: 127.0.0.1 - admin [25/Jun/2019:17:53:08 +0000] "GET /pulp/api/v3/tasks/ HTTP/1.1" 500 27 "-" "HTTPie/0.9.4 From: [email protected] At: 06/25/19 13:46:18To: Bin Li (BLOOMBERG/ 120 PARK ) Cc: [email protected] Subject: Re: [Pulp-list] Pulp 3 internal error The logs are available in journalctl. I usually run 'sudo journalctl -l -f' to monitor the logs. What is the traceback that is emitted when you query this task? A possibly related issue got filed today: https://pulp.plan.io/issues/4945 On Tue, Jun 25, 2019 at 1:35 PM Bin Li (BLOOMBERG/ 120 PARK) <[email protected]> wrote: I suddenly got a internal 500 error when query a task which creates a publication. Restarted all pulp daemons didn't help. Still able to GET repositories, publications, remotes information but query tasks and distributions gave me internal error. Database looks fine. I am able connect and login postgresql as pulp user. How do I troubleshot this issue? used the following to create distribution: http POST http://localhost:24817/pulp/api/v3/distributions/rpm/rpm/<<EOF { "base_path": "bloomberg/systems_third_party_rhel7.2", "name": "systems_third_party_rhel7.2", "publication": "/pulp/api/v3/publications/rpm/rpm/894a8109-001b-42fc-8658-4ac28bc52c0e/" } EOF Below is current status: { "database_connection": { "connected": true }, "missing_workers": [], "online_workers": [ { "_created": "2019-06-20T18:34:01.015187Z", "_href": "/pulp/api/v3/workers/fd7dc86c-3bff-4631-a86d-f49f618d3a79/", "last_heartbeat": "2019-06-25T17:14:50.374396Z", "missing": false, "name": "[email protected]", "online": true }, { "_created": "2019-06-20T18:34:01.347733Z", "_href": "/pulp/api/v3/workers/b4dfbb8a-69f4-4411-9311-ff84a63756c0/", "last_heartbeat": "2019-06-25T17:14:50.575492Z", "missing": false, "name": "reserved-resource-worke...@ip-1-76-158-221.drr1.bcpc.bloomberg.com", "online": true }, { "_created": "2019-06-20T18:34:01.463024Z", "_href": "/pulp/api/v3/workers/d1db2594-52b6-402e-8ef1-7c0a5635c3c4/", "last_heartbeat": "2019-06-25T17:14:50.976399Z", "missing": false, "name": "reserved-resource-worke...@ip-1-76-158-221.drr1.bcpc.bloomberg.com", "online": true } ], "redis_connection": { "connected": true }, "versions": [ { "component": "pulpcore", "version": "3.0.0rc3.dev0" }, { "component": "pulpcore-plugin", "version": "0.1.0rc3.dev0" }, { "component": "pulp_rpm", "version": "3.0.0b4.dev0" } ] } _______________________________________________ Pulp-list mailing list [email protected] https://www.redhat.com/mailman/listinfo/pulp-list
_______________________________________________ Pulp-list mailing list [email protected] https://www.redhat.com/mailman/listinfo/pulp-list
