John, services are typically long running and independent of jobs. They provide a specific function and may take a long time to initialize. They are meant to be reusable across multiple jobs. If your analytics are fast initializing and not meant to be shared you can just use jobs.
We should probably modify the documentation to talk about recommending a secondary broker for services. My recommendation is to download and install the full AMQ runtime for your services. DUCC only bundles essential parts of AMQ distro. You decide about how secure your DUCC should be. We lock DUCC's broker by default to protect it from a malicious user code. If this is not your concern you can use DUCC's broker if you want. Keep in mind that if you use DUCC's broker for services its load will increase and if this is high enough it may effect how DUCC responds. Another benefit of having a secondary broker is that you can keep your services up while DUCC is being shutdown for update or maintenance. This is especially important if your services take hours to initialize. Jerry On Tue, Jul 18, 2017 at 4:07 PM, Osborne, John David (Campus) < ozb...@uab.edu> wrote: > Thanks Jerry! > > I can run it using the insecure option (just tried it - works great!) - > the broker port is closed to all but localhost as another application feeds > requests to DUCC. From a non-security operational point of few though I am > violating your recommendation to use a secondary broker... I hope this > won't get me into too much trouble... > > From the documentation, I had no idea there was a need for a secondary > broker to run services. If I wanted to do this, is this something that can > be set up in the current ducc distribution or would it be better to go > ahead and run my legacy services independently? To do this, I'm presuming > I would instantiate a secondary broker (presumably still 61616) but submit > the deployment descriptor to the DUCC broker (61617) but with the > inputQueue brokerURL pointing to the secondary (61616) broker? > > I was hoping to make a clean migration to DUCC - maybe it would be easier > to get rid of services altogether and just submit jobs? Is this the current > trend or are a lot of folks running secondary brokers and services? > > -John > > > > -----Original Message----- > From: uim...@gmail.com [mailto:uim...@gmail.com] On Behalf Of Jaroslaw > Cwiklik > Sent: Tuesday, July 18, 2017 2:19 PM > To: user@uima.apache.org > Subject: Re: DUCC Security Model for Service Deployment > > Hi, as Lou stated its recommended to use a secondary broker for services > to keep this separate from a DUCC broker. The DUCC broker is protected by > default as it is used for internal communication. > > If you want to run without broker credentials change this setting in > default.ducc.properties > > ducc.broker.configuration = conf/activemq-ducc-unsecure.xml > > Jerry > > > > On Tue, Jul 18, 2017 at 3:05 PM, Osborne, John David (Campus) < > ozb...@uab.edu> wrote: > > > Thanks Lou, > > > > I'm still not totally following. > > > > In UIMA-AS, I could deploy services (as the administrator) using: > > deployAsyncService.sh $SOME_DEPLOYMENT_DESCRIPTOR $BROKER_URL > > > > I thought the equivalent in DUCC was (for example): ducc_services > > --register --process_descriptor $SOME_DEPLOYMENT_DESCRIPTOR > > --classpath $CLASSPATH --autostart true > > > > In the first case, I don't recall worrying about passwords - I presume > > deployAsyncService was privileged and knew any broker credentials. Are > > you saying the command line utility ducc_services is unaware of > > ducc-broker-credentionals.properties and there is no other way for the > > ducc user to pass these credentionals to the broker when > > registering/auto-start services? > > > > Sorry for the confusion, new to DUCC. I can see the > > $DUCC_HOME/resources.private/ducc-broker-credentials.properties - I > > just don't know how to use them when deploying (same thing as > > registering and > > starting?) a service. > > > > -John > > > > > > > > -----Original Message----- > > From: Lou DeGenaro [mailto:lou.degen...@gmail.com] > > Sent: Tuesday, July 18, 2017 1:34 PM > > To: user@uima.apache.org > > Subject: Re: DUCC Security Model for Service Deployment > > > > The DUCC broker is for DUCC. User services should employ a separate > > user broker, not the DUCC broker. > > > > That said, with proper permissions you can discover the DUCC broker > > user and pw in > > $DUCC_HOME/resources.private/ducc-broker-credentials.properties > > > > Lou. > > > > On Tue, Jul 18, 2017 at 2:14 PM, Osborne, John David (Campus) < > > ozb...@uab.edu> wrote: > > > > > Yes - the idea (based on a previous UIMA-AS workflow) was to > > > attached services to the broker and have other application/s call them > as needed. > > > > > > > > > I thought this was still done with DUCC? > > > > > > > > > I see what I *think* is the active security configuration in > > > activemq.xml > > > below: > > > > > > > > > <simpleAuthenticationPlugin anonymousAccessAllowed="false"> > > > <users> > > > <authenticationUser username="${ducc.broker.admin. > > username}" > > > password="${ducc.broker.admin.password}" > > > groups="ducc-admin"/> > > > </users> > > > </simpleAuthenticationPlugin> > > > > > > > > > I'm registering services and running ducc as user ducc. > > > > > > > > > -John > > > > > > > > > ________________________________ > > > From: Lou DeGenaro <lou.degen...@gmail.com> > > > Sent: Tuesday, July 18, 2017 12:19:21 PM > > > To: user@uima.apache.org > > > Subject: Re: DUCC Security Model for Service Deployment > > > > > > DUCC has its own password protected AMQ broker used for daemon > > > communications. Are your services trying to use DUCC's broker? > > > > > > Lou. > > > > > > On Tue, Jul 18, 2017 at 1:02 PM, Osborne, John David (Campus) < > > > ozb...@uab.edu> wrote: > > > > > > > I am having deploying services with DUCC using a single system, > > > > single user setup. I can run test jobs, so I *think* my system is > > > > set up > > > correctly > > > > but I can't register and then start services due to > > > > "SecurityException on Connect". > > > > > > > > > > > > I don't understand how when registering services the credentionals > > > > are passed to DUCC, I know it uses activemq credentionals, but > > > > there is no parameter to pass them on the command line. > > > > > > > > > > > > I am registering the service like this: > > > > > > > > /web/servers/apache-uima-ducc-2.2.0/bin/ducc_services --register > > > > --process_descriptor_DD ../resources/desc/deployment/ > > > > ImportDocumentsDeploymentDescriptorSimple.xml --description "Pull > > > > Documents" --classpath $CLASSPATH --autostart true > > > > > > > > > > > > I have not messed with any of the configuration files > > > > (credentional.properties, etc...) under apache-activemq. > > > > > > > > > > > > My error (actually a warning) is below: > > > > > > > > > > > > Any help appreciated, > > > > > > > > > > > > -John > > > > > > > > > > > > INFO: Controller: MedicsClobsConsumer Trying to Start Listener on > > > > Endpoint: queue://MRN_Document_Pull_Queue Selector: Command=2001 > > Broker: > > > > tcp://localhost:61617 > > > > >>> Service Container Deployed Successfully > > > > DuccAbstractProcessContainer.deploy() <<<<<<<< User Container > > > > deployed .... Deployed Processing Container - Initialization > > > > Successful - Thread 1 > > > > 18 Jul 2017 11:40:22,858 INFO AgentSession - T[1] > > > > notifyAgentWithStatus ... Job Process State Changed - PID:5803. > > > > Process State: Running. JMX > > > > Url:service:jmx:rmi:///jndi/rmi://uimaprapp1.hs.uab.edu:2105/jmxrm > > > > i Dispatched State Update Event to Agent with IP:10.23.142.165 Jul > > > > 18, > > > > 2017 11:40:23 AM org.apache.uima.adapter.jms.activemq. > > > > UimaDefaultMessageListenerContainer onException > > > > WARNING: Service: MedicsClobsConsumer Runtime Exception Jul 18, > > > > 2017 > > > > 11:40:23 AM org.apache.uima.adapter.jms.activemq. > > > > UimaDefaultMessageListenerContainer onException > > > > WARNING: Jms Listener Failed. Endpoint: MRN_Document_Pull_Queue > > > > Managed > > > > By: tcp://localhost:61617 Reason: org.apache.activemq. > > > ConnectionFailedException: > > > > The JMS connection has failed: Force close due to > > > > SecurityException on connect Jul 18, 2017 11:40:23 AM > > > > org.apache.uima.adapter.jms.activemq. > > > > UimaDefaultMessageListenerContainer handleListenerSetupFailure > > > > WARNING: Uima AS Service:MedicsClobsConsumer Listener Unable To > > > > Connect > > > To > > > > Broker: tcp://localhost:61617 Retrying Until Successful ... > > > > > > > > > > > > > > > > > >
