[jira] [Created] (KYLIN-5043) support floor() or date_trunc() function for Kylin4

2021-07-22 Thread Congling Xia (Jira)
Congling Xia created KYLIN-5043:
---

 Summary: support floor() or date_trunc() function for Kylin4
 Key: KYLIN-5043
 URL: https://issues.apache.org/jira/browse/KYLIN-5043
 Project: Kylin
  Issue Type: Bug
  Components: Query Engine
Affects Versions: v4.0.0-beta
Reporter: Congling Xia
Assignee: Congling Xia
 Attachments: image-2021-07-22-17-48-14-628.png

Dimension tables for date is seldom used in these days. Time function that can 
round datetimes to coarse grained granularities is very useful when writing 
queries in SQL.

Kylin 4 use SparkSQL to execute queries, but query SQL is parsed using Calcite.
 - in Calcite, a FLOOR function can round datetimes down
 - in Spark, the similar function is DATE_TRUNC 

but they cannot be transformed properly for now (fixed in Calcite 1.20, 
CALCITE-3072):
{code:java}
select 
  floor(PART_DT to year), count(1) 
from KYLIN_SALES 
group by 1; {code}

 !image-2021-07-22-17-48-14-628.png|width=1026,height=347!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (KYLIN-5042) floor function return wrong result for date type

2021-07-21 Thread Congling Xia (Jira)
Congling Xia created KYLIN-5042:
---

 Summary: floor function return wrong result for date type
 Key: KYLIN-5042
 URL: https://issues.apache.org/jira/browse/KYLIN-5042
 Project: Kylin
  Issue Type: Bug
Affects Versions: v3.1.2
Reporter: Congling Xia
 Fix For: Future


floor function return wrong result for date type, for example:
{code:java}
select 
  floor(current_date to YEAR), 
  floor(cast('2021-07-01' AS DATE) to YEAR), 
  floor(cast('2021-07-01' AS TIMESTAMP) to YEAR)
{code}
the result is:
{code:java}
EXPR$0  EXPR$1  EXPR$2
1970-01-01  1970-01-01  2021-01-01 00:00:00
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (KYLIN-5029) metrics test may fail because the cache is not refreshed

2021-07-06 Thread Congling Xia (Jira)
Congling Xia created KYLIN-5029:
---

 Summary: metrics test may fail because the cache is not refreshed
 Key: KYLIN-5029
 URL: https://issues.apache.org/jira/browse/KYLIN-5029
 Project: Kylin
  Issue Type: Improvement
  Components: Tools, Build and Test
Reporter: Congling Xia
Assignee: Congling Xia


Tests in _org.apache.kylin.rest.metrics.QueryMetricsTest_ may fail in current 
CI processes. There are some "Thread.sleep(..)" in the test code to wait for 
metrics cache being refreshed. 

After digging into the Hadoop metrics implementation, I find that the default 
refreshing interval is 10 seconds. So test thread should sleep for more than 10 
seconds to may sure metrics are up-to-date. I checked the sleep intervals in 
the test code and they are not greater than 1 (ms). That's why CI processes 
fail accidentally.

It is better to configure metrics refreshing interval to a smaller value.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (KYLIN-5007) queries with limit clause may fail when string dimension is encoded in integer type

2021-06-09 Thread Congling Xia (Jira)
Congling Xia created KYLIN-5007:
---

 Summary: queries with limit clause may fail when string dimension 
is encoded in integer type
 Key: KYLIN-5007
 URL: https://issues.apache.org/jira/browse/KYLIN-5007
 Project: Kylin
  Issue Type: Bug
  Components: Query Engine
Affects Versions: v3.0.2
Reporter: Congling Xia
Assignee: Congling Xia
 Attachments: image-2021-06-10-10-03-54-775.png

Hi, team.

Recently we encounter a problem that queries may fail if there is a LIMIT in 
the SQL. The SQL looks like:

{code}
select gid from some_table group by gid limit 100
{code}

The error message is like the following:
{code:java}
Not sorted! last: source_v1=null,...,gid=276,... fetched: 
source_v1=null,...,gid=100506
{code}
After searching the issues list, we find it is similar with KYLIN-2425, 
KYLIN-3089, and KYLIN-4942. We notice that these problems are not completely 
resolved.

It is an row-key encoding problem, the cube uses integer:4 to encode string 
column _gid_:

!image-2021-06-10-10-03-54-775.png|width=571,height=141!

As [~kangkaisen] mensioned in KYLIN-3089, comparator in 
SortMergedPartitionResultIterator is different from the one in 
SortedIteratorMergerWithLimit. SortedIteratorMergerWithLimit compares tuple of 
dimensions in their origin data type "string" rather than the encoded data type 
"integer" in rowkeys. In the exception above, 276<100506 is false because they 
are compared in "string" type.

It may be resolved by skipping limit pushdown when column type and encoding 
type may produce different comparing results, but it may lead such queries to 
be slower.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (KYLIN-4930) unexpected empty search result in group/user management page

2021-03-10 Thread Congling Xia (Jira)
Congling Xia created KYLIN-4930:
---

 Summary: unexpected empty search result in group/user management 
page
 Key: KYLIN-4930
 URL: https://issues.apache.org/jira/browse/KYLIN-4930
 Project: Kylin
  Issue Type: Bug
  Components: Web 
Reporter: Congling Xia
Assignee: Congling Xia
 Fix For: v3.1.2
 Attachments: image-2021-03-10-22-19-03-858.png, 
image-2021-03-10-22-20-19-633.png

When search a user/group in '/kylin/admin' page, unexpected empty 
page may be shown. For example, when user is navigate page#2 of the user list: 

!image-2021-03-10-22-19-03-858.png|width=581,height=162!

after search is done, the page shows:

!image-2021-03-10-22-20-19-633.png|width=579,height=104!

No user is listed in the page. But obviously, at least one user hits the 
search. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (KYLIN-4929) skip metrics update for simple queries to avoid NPE warnings

2021-03-09 Thread Congling Xia (Jira)
Congling Xia created KYLIN-4929:
---

 Summary: skip metrics update for simple queries to avoid NPE 
warnings
 Key: KYLIN-4929
 URL: https://issues.apache.org/jira/browse/KYLIN-4929
 Project: Kylin
  Issue Type: Improvement
  Components: Metrics
Reporter: Congling Xia
Assignee: Congling Xia
 Fix For: v3.1.2


Users may use simple queries like 'select 1' to check the availability of kylin 
service. No realization is needed for such queries. Metric system will raise 
NullPointerException when trying to get the name of the realization.

It does not cause the query to fail, but prints a lot of annoying warning logs 
with stack-traces.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (KYLIN-4925) Use Spark 3 as build and query engine for Kylin 4

2021-03-04 Thread Congling Xia (Jira)
Congling Xia created KYLIN-4925:
---

 Summary: Use Spark 3 as build and query engine for Kylin 4
 Key: KYLIN-4925
 URL: https://issues.apache.org/jira/browse/KYLIN-4925
 Project: Kylin
  Issue Type: New Feature
  Components: Job Engine, Query Engine
Reporter: Congling Xia
Assignee: Congling Xia
 Fix For: v4.0.0-GA


Spark 3 provides much more developed AQE framework, DPP and the increased 
support for GPUs and Kubernetes. The perspectives of performance increase are 
very promising. 

Currently, we try to run Kylin 4.0 with Spark 3.0 as build & query engine. We'd 
like to share the code changes here.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (KYLIN-4438) null password may cause RuntimeException when starting up

2020-03-29 Thread Congling Xia (Jira)
Congling Xia created KYLIN-4438:
---

 Summary: null password may cause RuntimeException when starting up
 Key: KYLIN-4438
 URL: https://issues.apache.org/jira/browse/KYLIN-4438
 Project: Kylin
  Issue Type: Bug
Reporter: Congling Xia


`PasswordPlaceholderConfigurer` tries to decode potential password values. But 
if no password is provided:
 - any password set to `null`;
 - any password set by a placeholder, and the value is not explictly provided, 
e.g. ${redis.password:}, or ${mysql.password:4stv/RRleOtvie/8SLHmXA==};

it'll raise an exception like the following.
{code:java}
2020-03-17 07:23:07,392 ERROR [localhost-startStop-1] context.ContextLoader:350 
: Context initialization failed
org.springframework.beans.factory.BeanDefinitionStoreException: Invalid bean 
definition with name 'redisConnectionFactory' defined in file 
[/home/admin/apache-kylin-3.0.1-bin-hbase1x/tomcat/webapps/kylin/WEB-INF/classes/applicationContext.xml]:
 Null input buffer; nested exception is java.lang.RuntimeException: Null input 
buffer
at 
org.springframework.beans.factory.config.PlaceholderConfigurerSupport.doProcessProperties(PlaceholderConfigurerSupport.java:223)
at 
org.springframework.beans.factory.config.PropertyPlaceholderConfigurer.processProperties(PropertyPlaceholderConfigurer.java:222)
at 
org.springframework.beans.factory.config.PropertyResourceConfigurer.postProcessBeanFactory(PropertyResourceConfigurer.java:86)
at 
org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanFactoryPostProcessors(PostProcessorRegistrationDelegate.java:281)
at 
org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanFactoryPostProcessors(PostProcessorRegistrationDelegate.java:161)
at 
org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:687)
at 
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:525)
at 
org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:443)
at 
org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:325)
at 
org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:107)
at 
org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:5197)
at 
org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5720)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at 
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:1016)
at 
org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:992)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:639)
at 
org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:1127)
at 
org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:2020)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Null input buffer
at org.apache.kylin.common.util.EncryptUtil.decrypt(EncryptUtil.java:55)
at 
org.apache.kylin.rest.security.PasswordPlaceholderConfigurer.resolvePlaceholder(PasswordPlaceholderConfigurer.java:77)
at 
org.springframework.beans.factory.config.PropertyPlaceholderConfigurer.resolvePlaceholder(PropertyPlaceholderConfigurer.java:162)
at 
org.springframework.beans.factory.config.PropertyPlaceholderConfigurer$PropertyPlaceholderConfigurerResolver.resolvePlaceholder(PropertyPlaceholderConfigurer.java:277)
at 
org.springframework.util.PropertyPlaceholderHelper.parseStringValue(PropertyPlaceholderHelper.java:147)
at 
org.springframework.util.PropertyPlaceholderHelper.replacePlaceholders(PropertyPlaceholderHelper.java:126)
at 
org.springframework.beans.factory.config.PropertyPlaceholderConfigurer$PlaceholderResolvingStringValueResolver.resolveStringValue(PropertyPlaceholderConfigurer.java:258)
at 
org.springframework.beans.factory.config.BeanDefinitionVisitor.resolveStringValue(BeanDefinitionVisitor.java:282)
at 
org.springframework.beans.factory.config.BeanDefinitionVisitor.resolveValue(BeanDefinitionVisitor.java:204)
at 
org.springframework.beans.factory.config.BeanDefinitionVisitor.visitPropertyValues(BeanDefinitionVisitor.java:141)
at 

[jira] [Created] (KYLIN-4435) query cache improvement

2020-03-26 Thread Congling Xia (Jira)
Congling Xia created KYLIN-4435:
---

 Summary: query cache improvement
 Key: KYLIN-4435
 URL: https://issues.apache.org/jira/browse/KYLIN-4435
 Project: Kylin
  Issue Type: Improvement
Reporter: Congling Xia


Hi, there. I try to use RemoteLocalFailOverCacheManager to set a fallback for 
my Redis cache these days. I think some improvements can be made for it:
 * RemoteLocalFailOverCacheManager is designed to set a fallback for a remote 
cache, but the remote cache is restricted to Memcached. It can be generalized 
to support other caches, such as Redis;
 * RemoteLocalFailOverCacheManager can not really disable remote cache since 
the status can be changed by scheduled health check;
 * There's no cache management API for administrators to check and control 
remote cache status.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (KYLIN-4432) duplicated queries with sytax error take unexpect long time when lazy query enabled

2020-03-23 Thread Congling Xia (Jira)
Congling Xia created KYLIN-4432:
---

 Summary: duplicated queries with sytax error take unexpect long 
time when lazy query enabled
 Key: KYLIN-4432
 URL: https://issues.apache.org/jira/browse/KYLIN-4432
 Project: Kylin
  Issue Type: Bug
Reporter: Congling Xia


Hi! Our Kylin server becomes unresponsive recently. All query threads were 
being TIMED_WAITING so that no more thread in the thread pool of Tomcat could 
response new requests. The server logs are full of "Duplicate SQL request"s, 
and many requests retrying for so long time:
{code:java}
$ cat kylin.log.25 | grep 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991 | head -3
2020-03-23 12:15:32,279 INFO [Query 
14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991] service.QueryService:571 : 
Duplicated SQL request is running, waiting...
2020-03-23 12:15:32,379 INFO [Query 
14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991] service.QueryService:571 : 
Duplicated SQL request is running, waiting...
2020-03-23 12:15:32,479 INFO [Query 
14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991] service.QueryService:571 : 
Duplicated SQL request is running, waiting...

$ cat kylin.log.25 | grep 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991 | tail -3
2020-03-23 12:26:56,920 INFO [Query 
14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991] service.QueryService:571 : 
Duplicated SQL request is running, waiting...
2020-03-23 12:26:57,020 INFO [Query 
14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991] service.QueryService:571 : 
Duplicated SQL request is running, waiting...
2020-03-23 12:26:57,121 INFO [Query 
14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991] service.QueryService:571 : 
Duplicated SQL request is running, waiting...
{code}
This can be re-produced easily:
 * enable lazy query (KYLIN-2897) by setting environment variable 
kylin.query.cache-enabled=true and kylin.query.lazy-query-enabled=true
 * send the same query with syntax error many times in several minutes

The first query will quickly response with error message, and the following 
queries will executing for long time and will repeatly output "Duplicated SQL 
request is running, waiting...".

The code in org.apache.kylin.rest.service.QueryService#searchQueryInCache 
indicates duplicated query will wait at most one minute with default setting of 
LazyQueryWaitingTimeoutMilliSeconds but in fact it waits quite longer than that.
{code:java}
while (response.isRunning()) {
// Wait at most one minute
if (System.currentTimeMillis() - response.getLazyQueryStartTime() >= 
getConfig()
.getLazyQueryWaitingTimeoutMilliSeconds()) {
cache.evict(sqlRequest.getCacheKey());
return null;
}
logger.info("Duplicated SQL request is running, waiting...");
try {
Thread.sleep(100L);
} catch (InterruptedException e) {
}
wrapper = cache.get(sqlRequest.getCacheKey());
if (wrapper == null) {
return null;
}
response = (SQLResponse) wrapper.get();
if (response == null) {
return null;
}
}
{code}
After some work-around, I find that the dummy response status remains unchanged 
after execution with exception. So the 2nd query will wait 
LazyQueryWaitingTimeoutMilliSeconds, evict the dummy response created by the 
1st query from cache, and put a new dummy response into the cache. And the 3rd 
query may use the new dummy response's start time to compute timeout, and will 
wait for another LazyQueryWaitingTimeoutMilliSeconds.

Suppose we send _n_ identical bad queries to the server quickly. The last query 
may wait for nearly _(n-1)_ times of LazyQueryWaitingTimeoutMilliSeconds. 
That's why the Web container's thread pool was exhausted – user program uses 
bad queries to check availability of Kylin service and retries many times in a 
short time.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (KYLIN-4424) use static web server to help frontend development

2020-03-11 Thread Congling Xia (Jira)
Congling Xia created KYLIN-4424:
---

 Summary: use static web server to help frontend development
 Key: KYLIN-4424
 URL: https://issues.apache.org/jira/browse/KYLIN-4424
 Project: Kylin
  Issue Type: New Feature
  Components: Tools, Build and Test
Reporter: Congling Xia
Assignee: Congling Xia


To verify frontend code changes, one need to build the source and re-deploy the 
target to Tomcat. It is not so efficient for frontend development. 
We can start a static web server and proxy `/kylin/api/**` to the existing 
Kylin service and  automatically refresh the static web server after source 
code being changed. By doing so, we can make the frontend development easier.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (KYLIN-4423) use spring session to support clustered sessions

2020-03-10 Thread Congling Xia (Jira)
Congling Xia created KYLIN-4423:
---

 Summary: use spring session to support clustered sessions
 Key: KYLIN-4423
 URL: https://issues.apache.org/jira/browse/KYLIN-4423
 Project: Kylin
  Issue Type: New Feature
Reporter: Congling Xia
Assignee: Congling Xia


When routing different APIs to different kylin instances, user may encounter 
permission problems since there are separate sessions.
To share the sessions, one can config tomcat cluster to replicate all sessions 
to all instances. It's a HA solution for Tomcat, and is rarely used in recent 
years. Another idea is to store sessions in shared external storages like 
Redis. 
We choose the latter solution. Spring Session is used to help manage user 
sessions and some work has been done to make it pluggable.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (KYLIN-4403) HTTP 414 when exporting result of long SQLs

2020-03-04 Thread Congling Xia (Jira)
Congling Xia created KYLIN-4403:
---

 Summary: HTTP 414 when exporting result of long SQLs
 Key: KYLIN-4403
 URL: https://issues.apache.org/jira/browse/KYLIN-4403
 Project: Kylin
  Issue Type: Bug
Reporter: Congling Xia


Hi. I have encountered an HTTP-414 (URI too long) when exporting result of long 
SQLs. I find that the export link make GET request to /api/query/format/csv 
which consumes the whole SQL as part of its params. 

Since query result has already been shown on the page, shall we do some 
frontend work to generate the file?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (KYLIN-4373) client side cache control for query API

2020-02-10 Thread Congling Xia (Jira)
Congling Xia created KYLIN-4373:
---

 Summary: client side cache control for query API
 Key: KYLIN-4373
 URL: https://issues.apache.org/jira/browse/KYLIN-4373
 Project: Kylin
  Issue Type: Improvement
Reporter: Congling Xia


When kylin.query.cache-enabled is set to true, query results will be cached for 
a period of time. If a cube updates due to daily build while this result is 
being cached, it will return the out-dated result which is confusing. It seems 
there's no way for client to tell Kylin server to bypass the cache.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (KYLIN-4320) number of replicas of Cuboid files cannot be configured for Spark engine

2019-12-30 Thread Congling Xia (Jira)
Congling Xia created KYLIN-4320:
---

 Summary: number of replicas of Cuboid files cannot be configured 
for Spark engine
 Key: KYLIN-4320
 URL: https://issues.apache.org/jira/browse/KYLIN-4320
 Project: Kylin
  Issue Type: Bug
  Components: Job Engine
Reporter: Congling Xia
 Attachments: cuboid_replications.png

Hi, team. I try to change `dfs.replication` to 3 by adding the following config 
override
{code:java}
kylin.engine.spark-conf.spark.hadoop.dfs.replication=3
{code}
Then, I get a strange result - numbers of replicas of cuboid files varies even 
though they are in the same level.

!cuboid_replications.png!

I guess it is due to the conflicting settings is SparkUtil:
{code:java}
public static void modifySparkHadoopConfiguration(SparkContext sc) throws 
Exception {
sc.hadoopConfiguration().set("dfs.replication", "2"); // cuboid 
intermediate files, replication=2
sc.hadoopConfiguration().set("mapreduce.output.fileoutputformat.compress", 
"true");

sc.hadoopConfiguration().set("mapreduce.output.fileoutputformat.compress.type", 
"BLOCK");

sc.hadoopConfiguration().set("mapreduce.output.fileoutputformat.compress.codec",
 "org.apache.hadoop.io.compress.DefaultCodec"); // or 
org.apache.hadoop.io.compress.SnappyCodec
}
{code}
It may be a bug for Spark property precedence. After checking [Spark 
documents|[http://spark.apache.org/docs/latest/configuration.html#dynamically-loading-spark-properties]],
 it seems that some programmatically set properties may not take effect and it 
is not a recommended way for Spark job configuration.

 

Anyway, cuboid files may survive for weeks until expired or been merged, the 
configuration rewrite in 
`org.apache.kylin.engine.spark.SparkUtil#modifySparkHadoopConfiguration` makes 
those files less reliable.

Is there any way to force cuboid files to remain 3 replicas? or shall we remove 
the code in SparkUtil to make 
kylin.engine.spark-conf.spark.hadoop.dfs.replication work properly?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (KYLIN-4315) use metadata numRows in beeline client for quick row counting

2019-12-26 Thread Congling Xia (Jira)
Congling Xia created KYLIN-4315:
---

 Summary: use metadata numRows in beeline client for quick row 
counting
 Key: KYLIN-4315
 URL: https://issues.apache.org/jira/browse/KYLIN-4315
 Project: Kylin
  Issue Type: Improvement
  Components: Job Engine
Reporter: Congling Xia
Assignee: Congling Xia


Hi, I find that in `BeelineHiveClient`, method `getHiveTableRows` uses "select 
count(*) from " for table row counting. The method is invoked in flat 
intermediate table redistribution step in cube building.

This stats can be loaded in metastore. It costs much less time than scanning 
all rows in Hive table. Since intermediate tables are created and inserted by 
Kylin, statistics will be automatically calculated and stored in metastore when 
`[hive.stats.autogather|https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.stats.autogather]`
 is enabled (which is the default setting for Hive). 

ref Hive wiki for more detail about `numRows` stats: 
[https://cwiki.apache.org/confluence/display/Hive/StatsDev#StatsDev-ExistingTables%E2%80%93ANALYZE]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (KYLIN-4313) add project/cube default settings and necessary config-override

2019-12-24 Thread Congling Xia (Jira)
Congling Xia created KYLIN-4313:
---

 Summary: add project/cube default settings and necessary 
config-override
 Key: KYLIN-4313
 URL: https://issues.apache.org/jira/browse/KYLIN-4313
 Project: Kylin
  Issue Type: Improvement
Reporter: Congling Xia
Assignee: Congling Xia


When creating a project or cube in Kylin, user may need to set many 
configuration and override some like job queue names. 

the problem is:
 * For necessary config-override, their names are difficult to memorise and 
type for users;
 * Some default values are hard-coded in javascripts which cannot be configured 
by a kylin service manager.

It is better to provide these key-value pairs and make them configurable. 
Perhaps, put them in `kylin.properties` and expose them to the front-end by 
`/api/admin/public_config` is a feasible plan.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (KYLIN-4310) "select 1 from tb_name limit 1" seems to scan all segments and dicts

2019-12-20 Thread Congling Xia (Jira)
Congling Xia created KYLIN-4310:
---

 Summary: "select 1 from tb_name limit 1" seems to scan all 
segments and dicts
 Key: KYLIN-4310
 URL: https://issues.apache.org/jira/browse/KYLIN-4310
 Project: Kylin
  Issue Type: Improvement
  Components: Query Engine
Reporter: Congling Xia


Hey! Some users of our kylin system tend to send  SQL like

 
{code:java}
select 1 from  limit 1;
{code}
as a validate query. After checking the log, I found that Kylin seems to scans 
all segments of a cube and load all dictionaries of each columns. More HBase 
scans and HDFS I/O will take place in the process, and may lead to slow 
response.

The dictionaries may cache in and out frequently in `DictionaryManager`. It is 
worse when dictionaries are big files and queries like that execute 
concurrently.

It seems hard work for Kylin to execute that simple SQL...

Any ideas for optimization?

 

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (KYLIN-4304) Project list cannot be correctly sorted by "Create Time"

2019-12-17 Thread Congling Xia (Jira)
Congling Xia created KYLIN-4304:
---

 Summary: Project list cannot be correctly sorted by "Create Time"
 Key: KYLIN-4304
 URL: https://issues.apache.org/jira/browse/KYLIN-4304
 Project: Kylin
  Issue Type: Bug
  Components: Web 
Reporter: Congling Xia
Assignee: Congling Xia
 Fix For: v3.0.0
 Attachments: image-2019-12-17-21-31-52-226.png

In project manage page, after clicking on the "Create Time" table header, 
projects will still be sorted by their name instead of their created time. 

!image-2019-12-17-21-31-52-226.png!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (KYLIN-4240) use kylin SSO without LDAP

2019-11-06 Thread Congling Xia (Jira)
Congling Xia created KYLIN-4240:
---

 Summary: use kylin SSO without LDAP
 Key: KYLIN-4240
 URL: https://issues.apache.org/jira/browse/KYLIN-4240
 Project: Kylin
  Issue Type: Improvement
Reporter: Congling Xia


Recently, we are integrating Kylin with our SSO service based on CAS. In our 
case, SSO provide only authentication but no authorization. Because our LDAP 
service is not used for application specific information management, we plan to 
use the built-in user/group services to manage their authorities (as in testing 
profile).

I am doing some work with CAS authentication to make it work along with form 
login in Kylin. I dont know whether it is a common case that user need to be 
authenticated by an SSO and will be managed for roles/groups in Kylin itself 
when LDAP is not available. I'd like to share it for someone in need.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (KYLIN-4212) Add a user profile page

2019-10-29 Thread Congling Xia (Jira)
Congling Xia created KYLIN-4212:
---

 Summary: Add a user profile page
 Key: KYLIN-4212
 URL: https://issues.apache.org/jira/browse/KYLIN-4212
 Project: Kylin
  Issue Type: Improvement
Reporter: Congling Xia


Kylin's user/group API has been extended since KYLIN-4122. User can get more 
account details and change password by requesting the APIs. But there is no 
webpage for a normal user to view such information or do such changes. 

In this improvement, the following changes are desired: 
 # provide a user profile page to show his/her basic information,  
 # set a button on it to launch the change password dialog for the user



--
This message was sent by Atlassian Jira
(v8.3.4#803005)