[jira] [Created] (KYLIN-3858) defaultCatalog != "defaultCatalog" in query

2019-03-06 Thread Davide Malagoli (JIRA) 
Davide Malagoli created KYLIN-3858:
--

 Summary: defaultCatalog != "defaultCatalog" in query
 Key: KYLIN-3858
 URL: https://issues.apache.org/jira/browse/KYLIN-3858
 Project: Kylin
  Issue Type: Bug
Reporter: Davide Malagoli
 Attachments: exception_text.txt, query-ok.txt, query_ko.txt

I'm using Kylin with a Jaspersoft server, and some times I encounter an 
exception during query.

 

It seems to be related to some query formatting done by the server.

Sometimes the string 'defaultCatalog' is quoted using double quotes, sometimes 
is not quoted.

Everytime is quoted I get an exception.

 

Can we threat it the same way, quoted or not?

Some example attached, you can try it in the "Insight" tab, it is the same.

I also attach the text of the exception thrown



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (KYLIN-3857) add parameter to change sql quoting character for sqoop

2019-03-06 Thread Davide Malagoli (JIRA) 
Davide Malagoli created KYLIN-3857:
--

 Summary: add parameter to change sql quoting character for sqoop
 Key: KYLIN-3857
 URL: https://issues.apache.org/jira/browse/KYLIN-3857
 Project: Kylin
  Issue Type: Bug
Reporter: Davide Malagoli
 Attachments: sqoop-error.log, sqoop-script.sh

I've set up  Kylin to access a SQL server using JDBC and do the ETL step as 
described in

[http://kylin.apache.org/docs/tutorial/setup_jdbc_datasource.html]

but when the "Sqoop To Flat Hive Table" runs it terminates with an error shown 
in the attached log.

It seems that the root cause is the presence of the '`' character in the query, 
which has no meaning for sql server.

 

My problem seems to related to this piece of code

 

public class FlatTableSqlQuoteUtils {

public static final String QUOTE = "`";

/**
 * Quote identifier by default quote `
 * @param identifier
 * @return
 */
 public static String quoteIdentifier(String identifier){
 return QUOTE + identifier + QUOTE;
 }

 

The solution may be to add a parameter to change this character used for 
quoting.

 

Just to be sure I made a copy of the original "sqoop" command to "sqoop.orig" 
and replaced the original with a little script of mine (attached).

This little script removes the "`" character from the original command and the 
call "sqoop.orig" with the cleaned arguments.

It works, but it is still an ugly workaround.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (KYLIN-3807) Error during sample_cube build "Build Dimension Dictionary"

2019-02-09 Thread Davide Malagoli (JIRA) 
Davide Malagoli created KYLIN-3807:
--

 Summary: Error during sample_cube build "Build Dimension 
Dictionary"
 Key: KYLIN-3807
 URL: https://issues.apache.org/jira/browse/KYLIN-3807
 Project: Kylin
  Issue Type: Bug
 Environment: kylin 2.5.2-cdh60
impala 3.0.0-cdh6.0.1
spark version 2.2.0-cdh6.0.1
Reporter: Davide Malagoli


It seems that a duplicate key value is found [null].

But there are no null in that four fields in my table, could it be a wrong 
interpretation of the "timestamp" columns?

 

org.apache.kylin.engine.mr.exception.HadoopShellException: 
java.lang.RuntimeException: Checking snapshot of TableRef[KYLIN_CAL_DT] failed.
 at 
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:103)
 at 
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:50)
 at 
org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:73)
 at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:92)
 at 
org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
 at 
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:164)
 at 
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:70)
 at 
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:164)
 at 
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:113)
 at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
 at java.lang.Thread.run(Thread.java:748)
 Caused by: java.lang.IllegalStateException: The table: KYLIN_CAL_DT Dup key 
found, key=[null], 
value1=[null,null,null,null,null,0,-1,-3,-15,-103,0,-1,-4,-15,-15,41501,228,47,16,6,33,5928,2,8,1364,3,455,114,2012-12-31,2012-09-30,2012-08-31,2012-08-17,16-Aug-2012,Aug
 16th 2012,Fri 
08-16-13,1,0,0,0,2012-06-21,365,92,31,7,2012-12-30,2012-06-30,2012-07-28,2012-08-11,2012-08-12,2012-08-16,Fri
 ,2012M08,Aug-2012,N,2012M08 ,N,Year 2012 - Quarter 03,2012Q03 
,N,33,2012,2012-08-11,2012-08-17,N,Wk.33 - 13,2012-08-11 00:00:00,2012-08-17 
00:00:00,2012W33 ,2012W33 ,08/11/13 - 08/17/13,08/11 - 
08/17,2012,N,2012-08-16,2011-08-16,2012-05-16,2012-02-16,2012-07-16,2012-06-16,2012-08-09,2012-08-02,0,0,0,0,0,0,0,0,8,3,33,3,1,1,1,2005-09-07,USER_X
 ,2012-11-27 00:16:56,USER_X], 
value2=[null,null,null,null,null,0,-3,-10,-47,-328,0,-3,-11,-47,-47,41276,3,3,3,5,1,5896,1,1,1357,1,453,114,2012-12-31,2012-03-31,2012-01-31,2012-01-05,03-Jan-2012,Jan
 3rd 2012,Thu 
01-03-13,1,0,0,0,2012-12-21,365,90,31,5,2012-12-30,2012-12-30,2012-12-30,2012-12-30,2012-12-31,2012-01-03,Thu
 ,2012M01,Jan-2012,N,2012M01 ,N,Year 2012 - Quarter 01,2012Q01 
,N,1,2012,2012-12-30,2012-01-05,N,Wk.01 - 13,2012-01-01 00:00:00,2012-01-05 
00:00:00,2012W01 ,2012W01 ,01/01/13 - 01/05/13,01/01 - 
01/05,2012,N,2012-01-03,2011-01-03,2012-10-03,2012-07-03,2012-12-03,2012-11-03,2012-12-27,2012-12-20,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,2005-09-07,USER_X
 ,2012-11-27 00:16:56,USER_X]
 at org.apache.kylin.dict.lookup.LookupTable.initRow(LookupTable.java:86)
 at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:69)
 at 
org.apache.kylin.dict.lookup.LookupStringTable.init(LookupStringTable.java:80)
 at org.apache.kylin.dict.lookup.LookupTable.(LookupTable.java:57)
 at 
org.apache.kylin.dict.lookup.LookupStringTable.(LookupStringTable.java:66)
 at 
org.apache.kylin.dict.lookup.LookupProviderFactory.getInMemLookupTable(LookupProviderFactory.java:63)
 at org.apache.kylin.cube.CubeManager.getInMemLookupTable(CubeManager.java:481)
 at org.apache.kylin.cube.CubeManager.getLookupTable(CubeManager.java:467)
 at 
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:101)
 ... 11 more

result code:2
 at 
org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:73)
 at 
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:164)
 at 
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:70)
 at 
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:164)
 at 
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:113)
 at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
 at java.lang.Thread.run(Thread.java:748)

 

 

table schema

Query: describe KYLIN_CAL_DT
+--+---++
| name | type | comment |
+--+---++
| cal_dt | timestamp | Date, PK |
| year_beg_dt | timestamp | Y

[jira] [Created] (KYLIN-3806) test Apache Impala Hive compatibility on sample_cube table creation

2019-02-06 Thread Davide Malagoli (JIRA) 
Davide Malagoli created KYLIN-3806:
--

 Summary: test Apache Impala Hive compatibility on sample_cube 
table creation
 Key: KYLIN-3806
 URL: https://issues.apache.org/jira/browse/KYLIN-3806
 Project: Kylin
  Issue Type: Test
  Components: Integration
Reporter: Davide Malagoli
 Attachments: create_sample_tables_impala.sql, sample-impala.sh

Apache Impala declares to be compatible with Hive syntax.

I'll try to test the compatibility running the sample_cube test



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (KYLIN-3761) Kylin for CDH6.1.0

2019-01-08 Thread Davide Malagoli (JIRA) 
Davide Malagoli created KYLIN-3761:
--

 Summary: Kylin for CDH6.1.0
 Key: KYLIN-3761
 URL: https://issues.apache.org/jira/browse/KYLIN-3761
 Project: Kylin
  Issue Type: Wish
Reporter: Davide Malagoli






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)