> Hi all,
>
> I have this weird problem: REST API response is not returned to callers
> correctly. It can only be recreated under some situation. Maybe it is a
> caching problem, or incorrect configurations, or bug somewhere (???)
>
> The configuration is Nginx<->uWsgi<->Django Rest Framework (DRF) under
> Ubuntu.
>
> I will summarize the business logic and the problem:
> =========================================
> All REST APIs are protected by Basic Authentication (username/password
> info
> is persisted in Postgres)
> DRF gets the authentication header and does the authentication itself.
> There is a POST REST API "login" ; it returns a response with the user
> information (like username).
> There are other REST APIs, and say one of them is POST API called "XXX".
>
> To recreate the problem:
> ====================
> Client(the tool Postman) calls "login" API with user "A" authentication.
> Response is good with user "A" info; response is 200 (as expected)
> Client calls "XXX" API with user "B". It returns 500 (as expected).
> Client calls "login" again repeatedly with user "A" authentication; now we
> see intermittent problem; sometimes the returned response is about user
> "A", but sometimes it is about user "B". Correct answer should be always
> about user "A". All responses are 200.
>
> My investigation:
> ==============
> When the problem happens, if I look at the Nginx logs, I can see the new
> entries with response status 200 but when I look at the Django logs, I
> don't see the new entries. Based on this observation, I think Django is
> being skipped somehow. Maybe uWsgi is being skipped as well. Maybe Nginx
> is
> caching some response for certain situation. But it seems all is triggered
> by the 500 response.
>
> Any debugging tips is appreciated. I am just looking for new idea on how
> to
> approach the problem.
>
> Thanks in advance.
>
> Steve
>
> Below is configuration details:
> ========================
>
> Ubuntu 16.04 LTS
>
> uWSGI (2.0.14)
>
> Django (1.9.12)
>
> djangorestframework (3.3.3)
>
> nginx version: nginx/1.10.0 (Ubuntu)
>
>
> ===========nginx.conf====================
> user www-data;
> worker_processes auto;
> pid /run/nginx.pid;
>
> events {
> worker_connections 768;
> # multi_accept on;
> }
>
> http {
>
> ##
> # Basic Settings
> ##
>
> sendfile on;
> tcp_nopush on;
> tcp_nodelay on;
> keepalive_timeout 65;
> types_hash_max_size 2048;
> # server_tokens off;
>
> # server_names_hash_bucket_size 64;
> # server_name_in_redirect off;
>
> include /etc/nginx/mime.types;
> default_type application/octet-stream;
>
> ##
> # SSL Settings
> ##
>
> ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
> ssl_prefer_server_ciphers on;
>
> ##
> # Logging Settings
> ##
>
> access_log /var/log/nginx/access.log;
> error_log /var/log/nginx/error.log;
>
> ##
> # Gzip Settings
> ##
>
> gzip on;
> gzip_disable "msie6";
>
> log_format auth
> 'http_authorization= "$http_authorization" '
> 'request= "$request" '
> 'time_local= "$time_local" '
> 'status= "$status" '
> ;
>
> ##
> # Virtual Host Configs
> ##
>
> include /etc/nginx/conf.d/*.conf;
> include /etc/nginx/sites-enabled/*;
> }
>
> ============nginx default file===================
> server {
> listen 80 default_server;
> listen [::]:80 default_server;
>
> root /var/www/html;
>
> # Add index.php to the list if you are using PHP
> index index.html index.htm index.nginx-debian.html;
>
> server_name _;
>
> location / {
> # First attempt to serve request as file, then
> # as directory, then fall back to displaying a 404.
> #try_files $uri $uri/ =404;
> try_files $uri $uri/ /index.html;
> }
> }
>
> ============nginx config file for django================
> # the upstream component nginx needs to connect to
> upstream django {
> # we will be using socket connection
> server unix:///tmp/zlims_uwsgi.sock;
> }
>
> # configuration of the server
> server {
>
> # use this version to receive all requests
> listen 8000 default_server;
>
> # the domain name it will serve for
> # server_name _;
> charset utf-8;
>
> # server root, any location block path will be relative to this root
> root /opt/zlims;
>
> # max upload size
> client_max_body_size 75M; # adjust to taste
>
> # Django media
> location /media {
> #alias /opt/zlims/media; # your Django project's media files -
> amend as required
> }
>
> location /static {
> alias /opt/zlims/be/static/; # your Django project's static files
> -
> amend as required
> }
>
> # Finally, send all non-media requests to the Django server.
> location / {
> uwsgi_pass django;
> #include /opt/zlims/uwsgi_params; # the uwsgi_params file you
> installed
> include uwsgi_params; # the uwsgi_params file you installed
>
> proxy_cache_bypass $cookie_nocache $arg_nocache$arg_comment;
> proxy_cache_bypass $http_pragma $http_authorization;
> proxy_no_cache $cookie_nocache $arg_nocache$arg_comment;
> proxy_no_cache $http_pragma $http_authorization;
>
> access_log /var/log/nginx/auth.log auth;
> }
> }
>
> ========= uwsgi_params file======================
> uwsgi_param QUERY_STRING $query_string;
> uwsgi_param REQUEST_METHOD $request_method;
> uwsgi_param CONTENT_TYPE $content_type;
> uwsgi_param CONTENT_LENGTH $content_length;
>
> uwsgi_param REQUEST_URI $request_uri;
> uwsgi_param PATH_INFO $document_uri;
> uwsgi_param DOCUMENT_ROOT $document_root;
> uwsgi_param SERVER_PROTOCOL $server_protocol;
> uwsgi_param REQUEST_SCHEME $scheme;
> uwsgi_param HTTPS $https if_not_empty;
>
> uwsgi_param REMOTE_ADDR $remote_addr;
> uwsgi_param REMOTE_PORT $remote_port;
> uwsgi_param SERVER_PORT $server_port;
> uwsgi_param SERVER_NAME $server_name;
>
> =============uwsgi vassel .ini file=====================
> [uwsgi]
>
> set-placeholder = base_dir = /opt/zlims/be
>
> chdir =/opt/zlims/be
> module =zlims.wsgi
> home =/opt/zlims/venv
>
> master =true
> processes =10
>
> socket =/tmp/%n.sock
> chmod-socket =666
> vacuum =true
> daemonize =/var/log/zlims/uwsgi_zlims.log
> logdate =%%d/%%m/%%Y %%T
> log-maxsize =10000000
> disable-logging =true # disable request logging
>
> lazy-apps =true
> single-interpreter =true
> enable-threads =true
>
> ========uwsgi service script (for systemd) ===========================
>
> # uwsgi service script
>
> [Unit]
> Description=uWSGI Emperor service
> After=syslog.target
>
> [Service]
> ExecStart=/usr/local/bin/uwsgi --emperor /etc/uwsgi/vassals/
> Restart=always
> KillSignal=SIGQUIT
> Type=notify
> StandardError=syslog
> NotifyAccess=all
>
> [Install]
> WantedBy=multi-user.target
> _______________________________________________
> uWSGI mailing list
> uWSGI@lists.unbit.it
> http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
>
Can you check if in /var/log/zlims/uwsgi_zlims.log there are references to
those "maybe-cached" requests ?
Note: upstream_* directives in nginx, do not work for the uwsgi protocol,
use the uwsgi_ prefixed ones
--
Roberto De Ioris
http://unbit.com
_______________________________________________
uWSGI mailing list
uWSGI@lists.unbit.it
http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
