[jira] [Commented] (SPARK-14698) CREATE FUNCTION cloud not add function to hive metastore
[ https://issues.apache.org/jira/browse/SPARK-14698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893565#comment-15893565 ] poseidon commented on SPARK-14698: -- [~azeroth2b] I think in spark 1.6.1, author do it on purpose. If this bug fixed, function can store in DB, but can't not loaded again on thrift-server restart. But i can upload the patch anyway. spark-1.6.1\sql\hive\src\main\scala\org\apache\spark\sql\hive\HiveContext.scala private def functionOrMacroDDLPattern(command: String) = Pattern.compile( ".*(create|drop)\\s+(temporary\\s+)(function|macro).+", Pattern.DOTALL).matcher(command) this is the correct regular-expression to lead create function command stored in DB > CREATE FUNCTION cloud not add function to hive metastore > > > Key: SPARK-14698 > URL: https://issues.apache.org/jira/browse/SPARK-14698 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 1.6.1 > Environment: spark1.6.1 >Reporter: poseidon > Labels: easyfix > > build spark 1.6.1 , and run it with 1.2.1 hive version,config mysql as > metastore server. > Start a thrift server , then in beeline , try to CREATE FUNCTION as HIVE SQL > UDF. > find out , can not add this FUNCTION to mysql metastore,but the function > usage goes well. > if you try to add it again , thrift server throw a alread Exist Exception. > [SPARK-10151][SQL] Support invocation of hive macro > add a if condition when runSqlHive, which will exec create function in > hiveexec. caused this problem. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-14698) CREATE FUNCTION cloud not add function to hive metastore
[ https://issues.apache.org/jira/browse/SPARK-14698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15889416#comment-15889416 ] Shawn Lavelle commented on SPARK-14698: --- [~poseidon] Would you be willing (and still able) to upload the patch? > CREATE FUNCTION cloud not add function to hive metastore > > > Key: SPARK-14698 > URL: https://issues.apache.org/jira/browse/SPARK-14698 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 1.6.1 > Environment: spark1.6.1 >Reporter: poseidon > Labels: easyfix > > build spark 1.6.1 , and run it with 1.2.1 hive version,config mysql as > metastore server. > Start a thrift server , then in beeline , try to CREATE FUNCTION as HIVE SQL > UDF. > find out , can not add this FUNCTION to mysql metastore,but the function > usage goes well. > if you try to add it again , thrift server throw a alread Exist Exception. > [SPARK-10151][SQL] Support invocation of hive macro > add a if condition when runSqlHive, which will exec create function in > hiveexec. caused this problem. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-14698) CREATE FUNCTION cloud not add function to hive metastore
[ https://issues.apache.org/jira/browse/SPARK-14698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15291030#comment-15291030 ] Jayadevan M commented on SPARK-14698: - Thanks [~poseidon]. I will post updates > CREATE FUNCTION cloud not add function to hive metastore > > > Key: SPARK-14698 > URL: https://issues.apache.org/jira/browse/SPARK-14698 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 1.6.1 > Environment: spark1.6.1 >Reporter: poseidon > Labels: easyfix > > build spark 1.6.1 , and run it with 1.2.1 hive version,config mysql as > metastore server. > Start a thrift server , then in beeline , try to CREATE FUNCTION as HIVE SQL > UDF. > find out , can not add this FUNCTION to mysql metastore,but the function > usage goes well. > if you try to add it again , thrift server throw a alread Exist Exception. > [SPARK-10151][SQL] Support invocation of hive macro > add a if condition when runSqlHive, which will exec create function in > hiveexec. caused this problem. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-14698) CREATE FUNCTION cloud not add function to hive metastore
[ https://issues.apache.org/jira/browse/SPARK-14698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15291029#comment-15291029 ] Jayadevan M commented on SPARK-14698: - Thanks [~poseidon]. I will post updates > CREATE FUNCTION cloud not add function to hive metastore > > > Key: SPARK-14698 > URL: https://issues.apache.org/jira/browse/SPARK-14698 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 1.6.1 > Environment: spark1.6.1 >Reporter: poseidon > Labels: easyfix > > build spark 1.6.1 , and run it with 1.2.1 hive version,config mysql as > metastore server. > Start a thrift server , then in beeline , try to CREATE FUNCTION as HIVE SQL > UDF. > find out , can not add this FUNCTION to mysql metastore,but the function > usage goes well. > if you try to add it again , thrift server throw a alread Exist Exception. > [SPARK-10151][SQL] Support invocation of hive macro > add a if condition when runSqlHive, which will exec create function in > hiveexec. caused this problem. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-14698) CREATE FUNCTION cloud not add function to hive metastore
[ https://issues.apache.org/jira/browse/SPARK-14698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15275854#comment-15275854 ] poseidon commented on SPARK-14698: -- spark 1.6.1 hive 1.2.1 mysql 5.6 just start thrift-server , and create udf as normal in hive . you can replicate this issue. I've fixed this issue already. add to udf to metastore is just the fist step , you have to fix lookup udf in metastore when parse sql as well . > CREATE FUNCTION cloud not add function to hive metastore > > > Key: SPARK-14698 > URL: https://issues.apache.org/jira/browse/SPARK-14698 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 1.6.1 > Environment: spark1.6.1 >Reporter: poseidon > Labels: easyfix > > build spark 1.6.1 , and run it with 1.2.1 hive version,config mysql as > metastore server. > Start a thrift server , then in beeline , try to CREATE FUNCTION as HIVE SQL > UDF. > find out , can not add this FUNCTION to mysql metastore,but the function > usage goes well. > if you try to add it again , thrift server throw a alread Exist Exception. > [SPARK-10151][SQL] Support invocation of hive macro > add a if condition when runSqlHive, which will exec create function in > hiveexec. caused this problem. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-14698) CREATE FUNCTION cloud not add function to hive metastore
[ https://issues.apache.org/jira/browse/SPARK-14698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15274160#comment-15274160 ] Jayadevan M commented on SPARK-14698: - [~poseidon] I would like to work this issue. Can you give the exact steps to replicate this issue. Thanks Jayadevan M > CREATE FUNCTION cloud not add function to hive metastore > > > Key: SPARK-14698 > URL: https://issues.apache.org/jira/browse/SPARK-14698 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 1.6.1 > Environment: spark1.6.1 >Reporter: poseidon > Labels: easyfix > > build spark 1.6.1 , and run it with 1.2.1 hive version,config mysql as > metastore server. > Start a thrift server , then in beeline , try to CREATE FUNCTION as HIVE SQL > UDF. > find out , can not add this FUNCTION to mysql metastore,but the function > usage goes well. > if you try to add it again , thrift server throw a alread Exist Exception. > [SPARK-10151][SQL] Support invocation of hive macro > add a if condition when runSqlHive, which will exec create function in > hiveexec. caused this problem. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
