Re: [Pulp-list] pulpcore-client 3.2 ldap authentication
Django app detected
Working in development environment
BASE_DIR
'/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/pulpcore/app'
DEBUG False
ALLOWED_HOSTS ['*']
MEDIA_ROOT '/var/lib/pulp/'
STATIC_ROOT '/var/lib/pulp/assets/'
DEFAULT_FILE_STORAGE 'pulpcore.app.models.storage.FileSystem'
FILE_UPLOAD_TEMP_DIR '/var/lib/pulp/tmp/'
WORKING_DIRECTORY '/var/lib/pulp/tmp/'
FILE_UPLOAD_HANDLERS ['pulpcore.app.files.HashingFileUploadHandler']
SECRET_KEY '3e$d+861lqv8x6y39p%^0!3(=%jzw6()g!u44%(=u@1_5p42g!'
INSTALLED_APPS ['django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'django_filters',
'drf_yasg',
'rest_framework',
'pulpcore.app',
'pulp_rpm.app.PulpRpmPluginAppConfig',
'pulp_file.app.PulpFilePluginAppConfig']
INSTALLED_PULP_PLUGINS ['pulp_rpm', 'pulp_file']
OPTIONAL_APPS ['crispy_forms', 'django_extensions', 'storages']
MIDDLEWARE ['django.middleware.security.SecurityMiddleware',
'whitenoise.middleware.WhiteNoiseMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware']
AUTHENTICATION_BACKENDS
['pulpcore.app.authentication.PulpNoCreateRemoteUserBackend']
ROOT_URLCONF 'pulpcore.app.urls'
TEMPLATES [{'APP_DIRS': True,
'BACKEND': 'django.template.backends.django.DjangoTemplates',
'DIRS':
['/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/pulpcore/app/templates'],
'OPTIONS': {'context_processors': ['django.template.context_processors.debug',
'django.template.context_processors.request',
'django.contrib.auth.context_processors.auth',
'django.contrib.messages.context_processors.messages']}}]
WSGI_APPLICATION 'pulpcore.app.wsgi.application'
REST_FRAMEWORK {'DEFAULT_AUTHENTICATION_CLASSES':
['rest_framework.authentication.SessionAuthentication',
'pulpcore.app.authentication.PulpRemoteUserAuthentication'],
'DEFAULT_FILTER_BACKENDS':
['django_filters.rest_framework.DjangoFilterBackend'],
'DEFAULT_PAGINATION_CLASS': 'rest_framework.pagination.LimitOffsetPagination',
'DEFAULT_PERMISSION_CLASSES': ['rest_framework.permissions.IsAuthenticated'],
'DEFAULT_VERSIONING_CLASS': 'rest_framework.versioning.URLPathVersioning',
'PAGE_SIZE': 100,
'UPLOADED_FILES_USE_URL': False,
'URL_FIELD_NAME': 'pulp_href'}
AUTH_PASSWORD_VALIDATORS [{'NAME':
'django.contrib.auth.password_validation.UserAttributeSimilarityValidator'},
{'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator'},
{'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator'},
{'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator'}]
LANGUAGE_CODE 'en-us'
TIME_ZONE 'UTC'
USE_I18N ['USE_I18N', True]
USE_L10N True
USE_TZ True
STATIC_URL '/static/'
DATABASES {'default': {'ENGINE': 'django.db.backends.postgresql_psycopg2',
'HOST': 'localhost',
'NAME': 'pulp',
'PASSWORD': 'pulp',
'PORT': 5432,
'USER': 'pulp'}}
LOGGING {'disable_existing_loggers': False,
'formatters': {'simple': {'format': 'pulp: %(name)s:%(levelname)s: '
'%(message)s'}},
'handlers': {'console': {'class': 'logging.StreamHandler',
'formatter': 'simple'}},
'loggers': {'': {'handlers': ['console'], 'level': 'INFO'}},
'version': 1}
CONTENT_PATH_PREFIX '/pulp/content/'
CONTENT_APP_TTL 120
REMOTE_USER_ENVIRON_NAME 'HTTP_REMOTE_USER'
ALLOWED_IMPORT_PATHS []
PROFILE_STAGES_API False
SWAGGER_SETTINGS {'DEFAULT_AUTO_SCHEMA_CLASS':
'pulpcore.app.openapigenerator.PulpAutoSchema',
'DEFAULT_GENERATOR_CLASS':
'pulpcore.app.openapigenerator.PulpOpenAPISchemaGenerator',
'DEFAULT_INFO': 'pulpcore.app.urls.api_info'}
REDOC_SETTINGS {'SPEC_URL':
'/pulp/api/v3/docs/?format=openapi_html=1'}
CONTENT_ORIGIN 'http://myhost.bloomberg.com'
SETTINGS '/etc/pulp/settings.py'
Here is the log. gunicorn seems take the remote_user
Apr 22 09:18:58 ip-1-76-158-49 gunicorn[12150]: pulp: django.request:WARNING:
Forbidden: /pulp/api/v3/remotes/rpm/rpm/
Apr 22 09:18:58 ip-1-76-158-49 gunicorn[12150]: 127.0.0.1 - bli4
[22/Apr/2020:13:18:58 +] "GET /pulp/api/v3/remotes/rpm/rpm/ HTTP/1.0" 403
58 "-" "HTTPie/0.9.4"
Apr 22 09:19:01 ip-1-76-158-49 systemd[1]: Created slice User Slice of root.
Apr 22 09:19:01 ip-1-76-158-49 systemd[1]: Started Session 324743 of user root.
Apr 22 09:19:01 ip-1-76-158-49 systemd[1]: Removed slice User Slice of root.
Apr 22 09:19:05 ip-1-76-158-49 gunicorn[12150]: 127.0.0.1 - bli4
[22/Apr/2020:13:19:05 +] "GET /pulp/api/v3/status/ HTTP/1.0" 200 1178
Re: [Pulp-list] pulpcore-client 3.2 ldap authentication
Could you please share your settings by running the following commands on your Pulp server: export DJANGO_SETTINGS_MODULE=pulpcore.app.settings export PULP_SETTINGS=/etc/pulp/settings.py (or wherever your settings are) dynaconf list Don't forget to obfuscate any settings you don't want to share. On Wed, Apr 22, 2020 at 9:15 AM Bin Li (BLOOMBERG/ 120 PARK) < bli...@bloomberg.net> wrote: > > Thank Dennis. This fixes the issue restarting pulp. With my LDAP > credential, now I can > http -a id:pwd GET localhost/pulp/api/v3/status/ but getting > "Authentication credentials were not provided" for all other uri > /remtes/rpm/rpm/. It looks like pulp is not using external authentication > and still need its own authentication somehow. > > > From: dkli...@redhat.com At: 04/22/20 06:52:35 > To: Bin Li (BLOOMBERG/ 120 PARK ) > Cc: pulp-list@redhat.com > Subject: Re: [Pulp-list] pulpcore-client 3.2 ldap authentication > > You need to replace > > REST_FRAMEWORK['DEFAULT_AUTHENTICATION_CLASSES'] = > > with > > REST_FRAMEWORK__DEFAULT_AUTHENTICATION_CLASSES = > > On Tue, Apr 21, 2020 at 10:09 PM Bin Li (BLOOMBERG/ 120 PARK) < > bli...@bloomberg.net> wrote: > >> This setting actually failed to restart pulp. See errors below. >> >> Apr 21 21:56:27 ip-1-76-158-49 gunicorn[24414]: NameError: name >> 'REST_FRAMEWORK' is not defined >> Apr 21 21:56:27 ip-1-76-158-49 gunicorn[24414]: [2020-04-21 21:56:27 >> -0400] [24417] [INFO] Worker exiting (pid: 24417) >> Apr 21 21:56:27 ip-1-76-158-49 gunicorn[24414]: [2020-04-21 21:56:27 >> -0400] [24414] [INFO] Shutting down: Master >> Apr 21 21:56:27 ip-1-76-158-49 gunicorn[24414]: [2020-04-21 21:56:27 >> -0400] [24414] [INFO] Reason: Worker failed to boot. >> Apr 21 21:56:27 ip-1-76-158-49 systemd[1]: pulpcore-api.service: main >> process exited, code=exited, status=3/NOTIMPLEMENTED >> Apr 21 21:56:27 ip-1-76-158-49 systemd[1]: Unit pulpcore-api.service >> entered failed state. >> Apr 21 21:56:27 ip-1-76-158-49 systemd[1]: pulpcore-api.service failed. >> Apr 21 21:56:27 ip-1-76-158-49 systemd[1]: >> pulpcore-resource-manager.service holdoff time over, scheduling restart. >> >> >> From: Bin Li (BLOOMBERG/ 120 PARK) At: 04/21/20 21:32:49 >> To: dkli...@redhat.com >> Cc: pulp-list@redhat.com >> Subject: Re: [Pulp-list] pulpcore-client 3.2 ldap authentication >> >> Yes, I did >> # pip list |grep dynaconf >> dynaconf 3.0.0rc1 >> >> >> From: dkli...@redhat.com At: 04/21/20 20:01:00 >> To: Bin Li (BLOOMBERG/ 120 PARK ) >> Cc: pulp-list@redhat.com >> Subject: Re: [Pulp-list] pulpcore-client 3.2 ldap authentication >> >> Did you update dynaconf to 3.0.0rc1? There was a bug that caused the >> settings to get merged instead of overwritten. >> >> [0] https://pulp.plan.io/issues/6244 >> [1] https://pypi.org/project/dynaconf/3.0.0rc1/ >> >> On Tue, Apr 21, 2020 at 5:59 PM Bin Li (BLOOMBERG/ 120 PARK) < >> bli...@bloomberg.net> wrote: >> >>> I have followed the setup >>> https://www.nginx.com/blog/nginx-plus-authenticate-users/ to setup >>> nginx LDAP authentication. >>> >>> This command works "http -a admin:password GET >>> localhost/pulp/api/v3/repositories/rpm/rpm/ Cookie:nginxauth=XXX". The >>> Cookie is the base64 encoded ldap username and password. >>> >>> I assume I should follow the below so I don't have to specify admin:pwd >>> >>> https://docs.pulpproject.org/installation/authentication.html#webserver-auth-with-reverse-proxy >>> >>> Adding the below to settings.py doesn't seem to work. >>> REMOTE_USER_ENVIRON_NAME = 'HTTP_REMOTE_USER' >>> AUTHENTICATION_BACKENDS = >>> ['pulpcore.app.authentication.PulpNoCreateRemoteUserBackend'] >>> REST_FRAMEWORK['DEFAULT_AUTHENTICATION_CLASSES'] = ( >>> 'rest_framework.authentication.SessionAuthentication', >>> 'pulpcore.app.authentication.PulpRemoteUserAuthentication' >>> >>> I am a little confused what need to be added for this setup. >>> nginx <---http---> gunicornpulpcore.app.wsgi application >>> >>> Please advise >>> Thanks >>> >>> >>> From: dkli...@redhat.com At: 04/17/20 10:45:31 >>> To: Bin Li (BLOOMBERG/ 120 PARK ) >>> Cc: pulp-list@redhat.com >>> Subject: Re: [Pulp-list] pulpcore-client 3.2 ldap authentication >>> >>> Theoretically you should be able to use pulpcore-client even with LDAP >>> authentication in the web server. However, I have not tested this. I've >>> only helped users that use certificate authentication in the webserver. >>> What error are you seeing on the client side? Do you see any errors in pulp >>> logs? >>> >>> On Fri, Apr 17, 2020 at 10:20 AM Bin Li (BLOOMBERG/ 120 PARK) < >>> bli...@bloomberg.net> wrote: >>> Thanks Dennis. We use pulpcore python client to interact with api. Once we enable ldap on nginx, the below code that pulpcore-client authenticate will not work any more. I am wonder if we are still be able to use pulpcore-client? or we have to rewrite the client code. This sounds too much work for us for now. configuration = pulpcore.Configuration()
Re: [Pulp-list] pulpcore-client 3.2 ldap authentication
Thank Dennis. This fixes the issue restarting pulp. With my LDAP credential, now I can http -a id:pwd GET localhost/pulp/api/v3/status/ but getting "Authentication credentials were not provided" for all other uri /remtes/rpm/rpm/. It looks like pulp is not using external authentication and still need its own authentication somehow. From: dkli...@redhat.com At: 04/22/20 06:52:35To: Bin Li (BLOOMBERG/ 120 PARK ) Cc: pulp-list@redhat.com Subject: Re: [Pulp-list] pulpcore-client 3.2 ldap authentication You need to replace REST_FRAMEWORK['DEFAULT_AUTHENTICATION_CLASSES'] = with REST_FRAMEWORK__DEFAULT_AUTHENTICATION_CLASSES = On Tue, Apr 21, 2020 at 10:09 PM Bin Li (BLOOMBERG/ 120 PARK) wrote: This setting actually failed to restart pulp. See errors below. Apr 21 21:56:27 ip-1-76-158-49 gunicorn[24414]: NameError: name 'REST_FRAMEWORK' is not defined Apr 21 21:56:27 ip-1-76-158-49 gunicorn[24414]: [2020-04-21 21:56:27 -0400] [24417] [INFO] Worker exiting (pid: 24417) Apr 21 21:56:27 ip-1-76-158-49 gunicorn[24414]: [2020-04-21 21:56:27 -0400] [24414] [INFO] Shutting down: Master Apr 21 21:56:27 ip-1-76-158-49 gunicorn[24414]: [2020-04-21 21:56:27 -0400] [24414] [INFO] Reason: Worker failed to boot. Apr 21 21:56:27 ip-1-76-158-49 systemd[1]: pulpcore-api.service: main process exited, code=exited, status=3/NOTIMPLEMENTED Apr 21 21:56:27 ip-1-76-158-49 systemd[1]: Unit pulpcore-api.service entered failed state. Apr 21 21:56:27 ip-1-76-158-49 systemd[1]: pulpcore-api.service failed. Apr 21 21:56:27 ip-1-76-158-49 systemd[1]: pulpcore-resource-manager.service holdoff time over, scheduling restart. From: Bin Li (BLOOMBERG/ 120 PARK) At: 04/21/20 21:32:49To: dkli...@redhat.com Cc: pulp-list@redhat.com Subject: Re: [Pulp-list] pulpcore-client 3.2 ldap authentication Yes, I did # pip list |grep dynaconf dynaconf3.0.0rc1 From: dkli...@redhat.com At: 04/21/20 20:01:00To: Bin Li (BLOOMBERG/ 120 PARK ) Cc: pulp-list@redhat.com Subject: Re: [Pulp-list] pulpcore-client 3.2 ldap authentication Did you update dynaconf to 3.0.0rc1? There was a bug that caused the settings to get merged instead of overwritten. [0] https://pulp.plan.io/issues/6244 [1] https://pypi.org/project/dynaconf/3.0.0rc1/ On Tue, Apr 21, 2020 at 5:59 PM Bin Li (BLOOMBERG/ 120 PARK) wrote: I have followed the setup https://www.nginx.com/blog/nginx-plus-authenticate-users/ to setup nginx LDAP authentication. This command works "http -a admin:password GET localhost/pulp/api/v3/repositories/rpm/rpm/ Cookie:nginxauth=XXX". The Cookie is the base64 encoded ldap username and password. I assume I should follow the below so I don't have to specify admin:pwdhttps://docs.pulpproject.org/installation/authentication.html#webserver-auth-with-reverse-proxy Adding the below to settings.py doesn't seem to work. REMOTE_USER_ENVIRON_NAME = 'HTTP_REMOTE_USER' AUTHENTICATION_BACKENDS = ['pulpcore.app.authentication.PulpNoCreateRemoteUserBackend'] REST_FRAMEWORK['DEFAULT_AUTHENTICATION_CLASSES'] = ( 'rest_framework.authentication.SessionAuthentication', 'pulpcore.app.authentication.PulpRemoteUserAuthentication' I am a little confused what need to be added for this setup. nginx <---http---> gunicornpulpcore.app.wsgi application Please advise Thanks From: dkli...@redhat.com At: 04/17/20 10:45:31To: Bin Li (BLOOMBERG/ 120 PARK ) Cc: pulp-list@redhat.com Subject: Re: [Pulp-list] pulpcore-client 3.2 ldap authentication Theoretically you should be able to use pulpcore-client even with LDAP authentication in the web server. However, I have not tested this. I've only helped users that use certificate authentication in the webserver. What error are you seeing on the client side? Do you see any errors in pulp logs? On Fri, Apr 17, 2020 at 10:20 AM Bin Li (BLOOMBERG/ 120 PARK) wrote: Thanks Dennis. We use pulpcore python client to interact with api. Once we enable ldap on nginx, the below code that pulpcore-client authenticate will not work any more. I am wonder if we are still be able to use pulpcore-client? or we have to rewrite the client code. This sounds too much work for us for now. configuration = pulpcore.Configuration() configuration.host = 'http://localhost' configuration.username = 'admin' configuration.password = 'pwd' rpm_client = pulp_rpm.ApiClient(configuration) From: dkli...@redhat.com At: 04/16/20 08:38:38To: Bin Li (BLOOMBERG/ 120 PARK ) Cc: pulp-list@redhat.com Subject: Re: [Pulp-list] pulpcore-client 3.2 ldap authentication Please be aware that there is a bug in dynaconf 2.2 with how settings are merged[0]. I recommend upgrading it to dynaconf 3.0.0rc1 for best results when configuring authentication backends in pulp. [0] https://pulp.plan.io/issues/6244 [1] https://pypi.org/project/dynaconf/3.0.0rc1/ On Wed, Apr 15, 2020 at 7:02 PM Dennis Kliban wrote: Pulp 3 does not currently support multiple users. We are planning to add
Re: [Pulp-list] pulpcore-client 3.2 ldap authentication
You need to replace REST_FRAMEWORK['DEFAULT_AUTHENTICATION_CLASSES'] = with REST_FRAMEWORK__DEFAULT_AUTHENTICATION_CLASSES = On Tue, Apr 21, 2020 at 10:09 PM Bin Li (BLOOMBERG/ 120 PARK) < bli...@bloomberg.net> wrote: > This setting actually failed to restart pulp. See errors below. > > Apr 21 21:56:27 ip-1-76-158-49 gunicorn[24414]: NameError: name > 'REST_FRAMEWORK' is not defined > Apr 21 21:56:27 ip-1-76-158-49 gunicorn[24414]: [2020-04-21 21:56:27 > -0400] [24417] [INFO] Worker exiting (pid: 24417) > Apr 21 21:56:27 ip-1-76-158-49 gunicorn[24414]: [2020-04-21 21:56:27 > -0400] [24414] [INFO] Shutting down: Master > Apr 21 21:56:27 ip-1-76-158-49 gunicorn[24414]: [2020-04-21 21:56:27 > -0400] [24414] [INFO] Reason: Worker failed to boot. > Apr 21 21:56:27 ip-1-76-158-49 systemd[1]: pulpcore-api.service: main > process exited, code=exited, status=3/NOTIMPLEMENTED > Apr 21 21:56:27 ip-1-76-158-49 systemd[1]: Unit pulpcore-api.service > entered failed state. > Apr 21 21:56:27 ip-1-76-158-49 systemd[1]: pulpcore-api.service failed. > Apr 21 21:56:27 ip-1-76-158-49 systemd[1]: > pulpcore-resource-manager.service holdoff time over, scheduling restart. > > > From: Bin Li (BLOOMBERG/ 120 PARK) At: 04/21/20 21:32:49 > To: dkli...@redhat.com > Cc: pulp-list@redhat.com > Subject: Re: [Pulp-list] pulpcore-client 3.2 ldap authentication > > Yes, I did > # pip list |grep dynaconf > dynaconf 3.0.0rc1 > > > From: dkli...@redhat.com At: 04/21/20 20:01:00 > To: Bin Li (BLOOMBERG/ 120 PARK ) > Cc: pulp-list@redhat.com > Subject: Re: [Pulp-list] pulpcore-client 3.2 ldap authentication > > Did you update dynaconf to 3.0.0rc1? There was a bug that caused the > settings to get merged instead of overwritten. > > [0] https://pulp.plan.io/issues/6244 > [1] https://pypi.org/project/dynaconf/3.0.0rc1/ > > On Tue, Apr 21, 2020 at 5:59 PM Bin Li (BLOOMBERG/ 120 PARK) < > bli...@bloomberg.net> wrote: > >> I have followed the setup >> https://www.nginx.com/blog/nginx-plus-authenticate-users/ to setup nginx >> LDAP authentication. >> >> This command works "http -a admin:password GET >> localhost/pulp/api/v3/repositories/rpm/rpm/ Cookie:nginxauth=XXX". The >> Cookie is the base64 encoded ldap username and password. >> >> I assume I should follow the below so I don't have to specify admin:pwd >> >> https://docs.pulpproject.org/installation/authentication.html#webserver-auth-with-reverse-proxy >> >> Adding the below to settings.py doesn't seem to work. >> REMOTE_USER_ENVIRON_NAME = 'HTTP_REMOTE_USER' >> AUTHENTICATION_BACKENDS = >> ['pulpcore.app.authentication.PulpNoCreateRemoteUserBackend'] >> REST_FRAMEWORK['DEFAULT_AUTHENTICATION_CLASSES'] = ( >> 'rest_framework.authentication.SessionAuthentication', >> 'pulpcore.app.authentication.PulpRemoteUserAuthentication' >> >> I am a little confused what need to be added for this setup. >> nginx <---http---> gunicornpulpcore.app.wsgi application >> >> Please advise >> Thanks >> >> >> From: dkli...@redhat.com At: 04/17/20 10:45:31 >> To: Bin Li (BLOOMBERG/ 120 PARK ) >> Cc: pulp-list@redhat.com >> Subject: Re: [Pulp-list] pulpcore-client 3.2 ldap authentication >> >> Theoretically you should be able to use pulpcore-client even with LDAP >> authentication in the web server. However, I have not tested this. I've >> only helped users that use certificate authentication in the webserver. >> What error are you seeing on the client side? Do you see any errors in pulp >> logs? >> >> On Fri, Apr 17, 2020 at 10:20 AM Bin Li (BLOOMBERG/ 120 PARK) < >> bli...@bloomberg.net> wrote: >> >>> Thanks Dennis. >>> >>> We use pulpcore python client to interact with api. Once we enable ldap >>> on nginx, the below code that pulpcore-client authenticate will not work >>> any more. I am wonder if we are still be able to use pulpcore-client? or we >>> have to rewrite the client code. This sounds too much work for us for now. >>> configuration = pulpcore.Configuration() >>> configuration.host = 'http://localhost' >>> configuration.username = 'admin' >>> configuration.password = 'pwd' >>> rpm_client = pulp_rpm.ApiClient(configuration) >>> >>> From: dkli...@redhat.com At: 04/16/20 08:38:38 >>> To: Bin Li (BLOOMBERG/ 120 PARK ) >>> Cc: pulp-list@redhat.com >>> Subject: Re: [Pulp-list] pulpcore-client 3.2 ldap authentication >>> >>> Please be aware that there is a bug in dynaconf 2.2 with how settings >>> are merged[0]. I recommend upgrading it to dynaconf 3.0.0rc1 for best >>> results when configuring authentication backends in pulp. >>> >>> [0] https://pulp.plan.io/issues/6244 >>> [1] https://pypi.org/project/dynaconf/3.0.0rc1/ >>> >>> >>> On Wed, Apr 15, 2020 at 7:02 PM Dennis Kliban >>> wrote: >>> Pulp 3 does not currently support multiple users. We are planning to add support for RBAC in the near future. However, I don't have a concrete timeline for that. With all that said, you still can configure the web server to perform authentication[0]. In this case Pulp will
