Hi Mohammed, Thank you, that worked.
Regards, --- Gautam On Thu, Jul 26, 2012 at 1:07 AM, Mohammad Islam <[email protected]> wrote: > Hi Gautam, > StoreService is no longer used in Oozie. It is being replace by JPAService. > Your JPAService is configured to use derby. > Please remove those and replace "StoreService" with "JPAService" and then > restart Oozie. > > Regards > Mohammad > > ----- Original Message ----- > From: Gautam Singaraju <[email protected]> > To: [email protected] > Cc: > Sent: Wednesday, July 25, 2012 6:06 PM > Subject: Re: Oozie Mysql connection > > Hi Alejandro, > > Thanks for your suggestion. Hive (mysql metastore) is working with Oozie. > I have the oozie-site.xml without any derby configuration; but mysql > JPA settings. When I start oozie, it creates an derby instance in > data/oozie-db. I log into the oozie's mysql server, and there is no > tables there. > > > From Settings from the oozie web console: > > Name: oozie.service.JPAService.jdbc.driver > Value: org.apache.derby.jdbc.EmbeddedDriver > > Name: oozie.service.JPAService.jdbc.url > Value: jdbc:derby:${oozie.data.dir}/${oozie.db.schema.name}-db;create=true > > > Name: oozie.service.StoreService.jdbc.url > Value: jdbc:mysql://<IP>:3306/oozie > > Name: oozie.service.StoreService.jdbc.driver > Value: com.mysql.jdbc.Driver > > Any suggestions would be appreciated. > --- > Gautam > > > On Thu, Jul 19, 2012 at 8:29 PM, Alejandro Abdelnur <[email protected]> wrote: >> Gautam, >> >> Look at the oozie hive launcher map log, that should give you information >> about what exactly Hive is using. >> >> thx >> >> On Thu, Jul 19, 2012 at 6:04 PM, Gautam Singaraju < >> [email protected]> wrote: >> >>> Oops, that's a stupid question. Thanks, Alejandro. >>> I am still puzzled how my oozie-MySQL config processed a job when I >>> removed the derby conf and only had MySQL config; there were no tables >>> created in MySQL... >>> --- >>> Gautam >>> >>> >>> On Thu, Jul 19, 2012 at 5:47 PM, Alejandro Abdelnur <[email protected]> >>> wrote: >>> > Gautam, >>> > >>> > You are confusing the Oozie server configuration in oozie-site.xml (which >>> > uses mysql) with the hive action metatstore configuration which by >>> default >>> > uses derby. Those are 2 different databases. >>> > >>> > hope this helps >>> > >>> > thx >>> > >>> > On Thu, Jul 19, 2012 at 4:03 PM, Gautam Singaraju < >>> > [email protected]> wrote: >>> > >>> >> Thank you Harsh, for your help. I was able to redo some steps and was >>> >> able to start oozie with the MySQL configuration as described. >>> >> However, something very weird seems to be happening. Once I schedule a >>> >> job, the job completes successfully. But, when I log into MySQL client >>> >> to see the schemas, I dont see anything. Then I noticed that I had the >>> >> following in the configuration: >>> >> >>> >> <property> >>> >> <name>hive.stats.dbclass</name> >>> >> <value>jdbc:derby</value> >>> >> <description>The default database that stores temporary hive >>> >> statistics.</description> >>> >> </property> >>> >> >>> >> <property> >>> >> <name>hive.stats.autogather</name> >>> >> <value>true</value> >>> >> <description>A flag to gather statistics automatically during the >>> >> INSERT OVERWRITE command.</description> >>> >> </property> >>> >> >>> >> <property> >>> >> <name>hive.stats.jdbcdriver</name> >>> >> <value>org.apache.derby.jdbc.EmbeddedDriver</value> >>> >> <description>The JDBC driver for the database that stores temporary >>> >> hive statistics.</description> >>> >> </property> >>> >> >>> >> <property> >>> >> <name>hive.stats.dbconnectionstring</name> >>> >> <value>jdbc:derby:;databaseName=TempStatsStore;create=true</value> >>> >> <description>The default connection string for the database that >>> >> stores temporary hive statistics.</description> >>> >> </property> >>> >> >>> >> >>> >> Could someone explain what is happening here? Does oozie store temp >>> >> jobs in a derby db and flush to mysql at a predetermined interval? >>> >> >>> >> Thank you, >>> >> --- >>> >> Gautam >>> >> >>> >> >>> >> On Thu, Jul 19, 2012 at 3:45 AM, Harsh J <[email protected]> wrote: >>> >> > Hey Gautam, >>> >> > >>> >> > Can you also attach a server side oozie.log snippet after running an >>> >> Oozie >>> >> > job that seems to fail with the error you've sent? >>> >> > >>> >> > I do not think your issue is related to MySQL at this point, but >>> server >>> >> > logs will help determine. Thanks! >>> >> > >>> >> > On Thu, Jul 19, 2012 at 1:09 AM, Gautam Singaraju < >>> >> > [email protected]> wrote: >>> >> > >>> >> >> Hi Oozie community: >>> >> >> >>> >> >> >>> >> >> I am working on getting Oozie working with MySQL; and I have followed >>> >> >> the instructions in a previous post: >>> >> >> >>> >> >> >>> >> >>> http://mail-archives.apache.org/mod_mbox/incubator-oozie-users/201110.mbox/%3CCAC8XGpEFCTEnHZjdbPQoKgWm-jiK0gzY=a_9wdjgzuzpjef...@mail.gmail.com%3E >>> >> >> (I have copied some part of the suggestions at the end of the email.) >>> >> >> >>> >> >> I am using Oozie 3.2 and mysql 5.1. >>> >> >> >>> >> >> Exceptions: >>> >> >> 1. I have installed oozie as linux user: hadoop. >>> >> >> 2. also started it with webconsole. >>> >> >> 3. I am running on AWS. >>> >> >> >>> >> >> When I start oozie, and submit a job, I am getting the following >>> error: >>> >> >> >>> >> >> WARN AuthenticationFilter:381 - Authentication exception: >>> >> >> AuthenticationToken expired >>> >> >> >>> >> >>> org.apache.hadoop.security.authentication.client.AuthenticationException: >>> >> >> AuthenticationToken expired >>> >> >> >>> >> >> Any suggestions would be greatly appreciated. Additionally, Apache >>> >> >> Oozie can benefit from a writeup on how to get oozie and mysql >>> >> >> working. >>> >> >> >>> >> >> Thanks in advance! >>> >> >> <--------------snip> >>> >> >> >>> >> >> 1. Get a jdbc connector jar from here: >>> >> >> http://dev.mysql.com/downloads/connector/j/ >>> >> >> 2. Run the following command (as user oozie): >>> >> >> sudo -u oozie <path-to-oozie-installation/oozie-setup.sh >>> >> >> -jars <path-to-connector-jar> >>> >> >> 3. Create the following DB in your MySQL and grant privs on it: >>> >> >> # mysql -u root -p >>> >> >> create database oozie; >>> >> >> grant all privileges on oozie.* to 'oozie'@'localhost' >>> >> >> identified by 'oozie'; >>> >> >> 4. Replace the following properties in your oozie-site.xml with the >>> >> >> values give below: >>> >> >> >>> >> >> <property> >>> >> >> <name>oozie.service.StoreService.jdbc.driver</name> >>> >> >> <value>com.mysql.jdbc.Driver</value> >>> >> >> <description> >>> >> >> JDBC driver class. >>> >> >> </description> >>> >> >> </property> >>> >> >> >>> >> >> <property> >>> >> >> <name>oozie.service.StoreService.jdbc.url</name> >>> >> >> <value>jdbc:mysql://localhost:3306/oozie</value> >>> >> >> <description> >>> >> >> JDBC URL. >>> >> >> </description> >>> >> >> </property> >>> >> >> >>> >> >> <property> >>> >> >> <name>oozie.service.StoreService.jdbc.username</name> >>> >> >> <value>oozie</value> >>> >> >> <description> >>> >> >> DB user name. >>> >> >> </description> >>> >> >> </property> >>> >> >> >>> >> >> <property> >>> >> >> <name>oozie.service.StoreService.jdbc.password</name> >>> >> >> <value>oozie</value> >>> >> >> <description> >>> >> >> DB user password. >>> >> >> >>> >> >> IMPORTANT: if password is emtpy leave a 1 space string, >>> the >>> >> >> service >>> >> >> trims the value, >>> >> >> if empty Configuration assumes it is NULL. >>> >> >> >>> >> >> IMPORTANT: if the StoreServicePasswordService is active, >>> >> >> it will reset >>> >> >> this value with the value given in >>> >> >> the console. >>> >> >> </description> >>> >> >> </property> >>> >> >> >>> >> >> <-----------------end snip> >>> >> >> >>> >> >> >>> >> >> --- >>> >> >> Gautam >>> >> >> >>> >> > >>> >> > >>> >> > >>> >> > -- >>> >> > Harsh J >>> >> >>> > >>> > >>> > >>> > -- >>> > Alejandro >>> >> >> >> >> -- >> Alejandro >
