I've been prototyping Foreman for a while now, and everything has been working perfectly in testing. Now I've built out our production deployment and imported content, and started trying to move hosts to the Foreman and Katello server. When ever I run subscription-manager register, from any host, I get 500 Internal Server Error, and the task hangs. I have to go into the dynaflow console and skip the broken step, and resume the task to clear locks.
I've never had this happen in any of my tests where I've built a number of test servers to test out processes and configurations. All the test systems were able to register clients with no issues at all. The server and client are both fully updated RHEL 7.2 boxes, the server is running foreman-1.12.2-1.el7.noarch and katello-3.1.0-5.el7.noarch. The only other difference between the production and test is that I never imported RHEL content into the test systems. I've spent a week building out the products, repositories, content views, etc. and would really like to recover this without having to start over, but this is obviously a major problem! I've googled the errors, with out finding any thing. Can anyone help me? On the client I run: # subscription-manager register --org="myOrg" --activationkey="rhel-72-x86_64-production" Task 1ac48d97-a570-4a43-9a09-b5cb0be7cf29: RestClient::InternalServerError: 500 Internal Server Error And the production.log contains (abbreviated): 2016-09-13 15:39:15 [app] [I] Started GET "/rhsm/" for XX.XX.XX.XX at 2016-09-13 15:39:15 -0500 2016-09-13 15:39:15 [app] [I] Processing by Katello::Api::V2::RootController#rhsm_resource_list as JSON 2016-09-13 15:39:15 [app] [I] Parameters: {"root"=>{}} 2016-09-13 15:39:15 [app] [I] Rendered /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.1.0.1/app/views/katello/api/v2/root/resource_list.json.rabl within katello/api/v2/layouts/collection (2.4ms) 2016-09-13 15:39:15 [app] [I] Completed 200 OK in 21ms (Views: 3.8ms | ActiveRecord: 0.0ms) 2016-09-13 15:39:15 [app] [I] Started GET "/rhsm/status" for XX.XX.XX.XX at 2016-09-13 15:39:15 -0500 2016-09-13 15:39:15 [app] [I] Processing by Katello::Api::Rhsm::CandlepinProxiesController#server_status as JSON 2016-09-13 15:39:15 [app] [I] Completed 200 OK in 308ms (Views: 0.9ms | ActiveRecord: 0.0ms) 2016-09-13 15:39:16 [app] [I] Started POST "/rhsm/consumers?owner=myOrg&activation_keys=rhel-72-x86_64-production" for XX.XX.XX.XX at 2016-09-13 15:39:16 -0500 2016-09-13 15:39:16 [app] [I] Processing by Katello::Api::Rhsm::CandlepinProxiesController#consumer_activate as JSON 2016-09-13 15:39:16 [app] [I] Parameters: {"facts"=>"[FILTERED]", "installedProducts"=>[{"productName"=>"Red Hat Enterprise Linux Server", "arch"=>"x86_64", "version"=>"7.2", "productId"=>"69"}], "type"=>"system", "name"=>"client.com", "contentTags"=>["rhel-7", "rhel-7-server"], "owner"=>"myOrg", "activation_keys"=>"rhel-72-x86_64-production"} 2016-09-13 15:39:18 [app] [I] Import facts for 'client.example.com' completed. Added: 0, Updated: 1, Deleted 21 facts 2016-09-13 15:39:21 [katello/pulp_rest] [E] RestClient.post "https://foreman.example.com/pulp/api/v2/consumers/", "{\"id\":\"1c796228-efa5-4190-b5de-a0f7605e3074\",\"display_name\":\"client.example.com\"}", "Accept"=>"*/*; q=0.5, application/xml", "Accept-Encoding"=>"gzip, deflate", "Content-Length"=>"100", "accept"=>"application/json", "content_type"=>"application/json" | \n# => 500 InternalServerError | application/json 2551 bytes | 2016-09-13 15:39:21 [foreman-tasks/action] [E] 500 Internal Server Error (RestClient::InternalServerError) | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.8.0/lib/restclient/abstract_response.rb:74:in `return!' | /opt/theforeman/tfm/root/usr/share/gems/gems/runcible-1.7.2/lib/runcible/base.rb:79:in `block in get_response' | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.8.0/lib/restclient/request.rb:493:in `call' <snip> | /opt/rh/sclo-ror42/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/serialized_execution.rb:77:in `block in call_job' | /opt/rh/sclo-ror42/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:333:in `call' | /opt/rh/sclo-ror42/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:333:in `run_task' | /opt/rh/sclo-ror42/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:322:in `block (3 levels) in create_worker' | /opt/rh/sclo-ror42/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:305:in `loop' | /opt/rh/sclo-ror42/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:305:in `block (2 levels) in create_worker' | /opt/rh/sclo-ror42/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:304:in `catch' | /opt/rh/sclo-ror42/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:304:in `block in create_worker' | /opt/theforeman/tfm/root/usr/share/gems/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:in `call' | /opt/theforeman/tfm/root/usr/share/gems/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:in `block in create_with_logging_context' 2016-09-13 15:39:21 [katello/cp_proxy] [E] ForemanTasks::TaskError: Task c837e61c-90c6-4923-81ba-d4970e1249ca: RestClient::InternalServerError: 500 Internal Server Error | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.18/lib/foreman_tasks.rb:40:in `block in sync_task' | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.18/lib/foreman_tasks.rb:39:in `tap' | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.18/lib/foreman_tasks.rb:39:in `sync_task' | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.18/lib/foreman_tasks/triggers.rb:22:in `sync_task' | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.1.0.1/app/controllers/katello/api/rhsm/candlepin_proxies_controller.rb:224:in `consumer_activate' | /opt/rh/sclo-ror42/root/usr/share/gems/gems/actionpack-4.2.5.1/lib/action_controller/metal/implicit_render.rb:4:in `send_action' <snip> | /usr/share/passenger/phusion_passenger/rack/thread_handler_extension.rb:74:in `process_request' | /usr/share/passenger/phusion_passenger/request_handler/thread_handler.rb:141:in `accept_and_process_next_request' | /usr/share/passenger/phusion_passenger/request_handler/thread_handler.rb:109:in `main_loop' | /usr/share/passenger/phusion_passenger/request_handler.rb:455:in `block (3 levels) in start_threads' | /opt/theforeman/tfm/root/usr/share/gems/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:in `call' | /opt/theforeman/tfm/root/usr/share/gems/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:in `block in create_with_logging_context' 2016-09-13 15:39:21 [app] [I] Completed 500 Internal Server Error in 5769ms (Views: 0.4ms | ActiveRecord: 188.9ms) -- You received this message because you are subscribed to the Google Groups "Foreman users" group. To unsubscribe from this group and stop receiving emails from it, send an email to foreman-users+unsubscr...@googlegroups.com. To post to this group, send email to foreman-users@googlegroups.com. Visit this group at https://groups.google.com/group/foreman-users. For more options, visit https://groups.google.com/d/optout.