@qianhao, 1, I will debug the code if available 2, just now, as observed, after "REFRESH" a cube, even though I click "Reload Metadata" under "Admin" tab on the WEB of the "QUERY" kylin instances, the cube status cannot be syncd with the "ALL" kylin instance(but if restarting the "QUERY" kylin instances, "QUERY" instances can sync the FRESH cube result info), another thing is that it works OK for "BUILD" a cube if just reloading the metadata without restarting the instance, thus,I guess, the above 2 correct URL put methods will not work for "REFERSH" as well
2015-05-07 20:17 GMT+08:00 Zhou, Qianhao <[email protected]>: > Thanks for the research, the root cause should be the wrong URL redirect > > 2015年5月7日 18:59于 dong wang <[email protected]>写道： > @qianzhao, as tested, another discovery is that if I use the URL like: > > http://test.abc.com:8082/kylin/api/cache/cube/lbs_map_active_user_fact/update > , > the cube status of the "QUERY" kylin instance also works SUCCESSFULLY > > thus, the problem should be something wrong with the communication between > "ALL" mode instance and "QUERY" mode instances > > 2015-05-07 18:12 GMT+08:00 dong wang <[email protected]>: > > > @qianhao, @shaofeng, > > > > today, I tried to mockup a PUT request as you mentioned above with the > > following different URLs: > > 1, http://test.abc.com:8082/cube/lbs_map_new_user_fact/update > > > > nothing happened, even more, no log information received in the kylin > > instance log file deployed with port 8082 > > > > 2, http://test.abc.com:8082/kylin/api/cache/all/all/update > > > > with this API, It works OK, the cube status is flushed SUCCESSFULLY, log > > information received as shown below: > > > > [http-bio-8082-exec-7]:[2015-05-07 > > > 17:54:57,469][INFO][org.apache.kylin.rest.controller.CacheController.wipeCache(CacheController.java: > > 64)] - wipe cache type: ALL event:UPDATE name:all > > [http-bio-8082-exec-7]:[2015-05-07 > > > 17:54:57,488][DEBUG][org.apache.kylin.metadata.MetadataManager.reloadAllSourceTable(MetadataManager. > > java:244)] - Reloading SourceTable from folder > > kylin_metadata(key='/table')@kylin_metadata@hbase > > [http-bio-8082-exec-7]:[2015-05-07 > > > 17:54:57,533][DEBUG][org.apache.kylin.metadata.MetadataManager.reloadAllSourceTable(MetadataManager. > > java:253)] - Loaded 8 SourceTable(s) > > [http-bio-8082-exec-7]:[2015-05-07 > > > 17:54:57,534][DEBUG][org.apache.kylin.metadata.MetadataManager.reloadAllSourceTableExd(MetadataManag > > er.java:201)] - Reloading SourceTable exd info from folder > > kylin_metadata(key='/table_exd')@kylin_metadata@hbase > > [http-bio-8082-exec-7]:[2015-05-07 > > > 17:54:57,562][DEBUG][org.apache.kylin.metadata.MetadataManager.reloadAllSourceTableExd(MetadataManag > > er.java:210)] - Loaded 8 SourceTable EXD(s) > > [http-bio-8082-exec-7]:[2015-05-07 > > > 17:54:57,562][DEBUG][org.apache.kylin.metadata.MetadataManager.reloadAllDataModel(MetadataManager.ja > > va:291)] - Reloading DataModel from folder > > kylin_metadata(key='/model_desc')@kylin_metadata@hbase > > [http-bio-8082-exec-7]:[2015-05-07 > > > 17:54:57,599][DEBUG][org.apache.kylin.metadata.MetadataManager.reloadAllDataModel(MetadataManager.ja > > va:305)] - Loaded 8 DataModel(s) > > [http-bio-8082-exec-7]:[2015-05-07 > > > 17:54:57,604][INFO][org.apache.kylin.rest.service.BasicService.resetOLAPDataSources(BasicService.jav > > a:90)] - resetOLAPDataSources is called. > > > > 3, currently, since we deploy 3 kylin instance on the same machine with > > different ports, and I set export KYLIN_HOME=/path_to_kylin at the very > > beginning of the file bin/kylin.sh since we cannot set a global env > > KYLIN_HOME > > > > 2015-05-07 17:50 GMT+08:00 Tao Wong <[email protected]>: > > > >> Cube status on query mode instance still can not change automatically. > >> > >> the all mode node seem to broadcast the query mode nodes,however ,the > >> query > >> node run with error as above and below: > >> > >> Caused by: java.lang.ClassNotFoundException: > >> org.apache.kylin.query.schema.OLAPSchemaFactory > >> 733 at java.net.URLClassLoader$1.run(URLClassLoader.java:366) > >> 734 at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > >> 735 at java.security.AccessController.doPrivileged(Native Method) > >> 736 at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > >> 737 at java.lang.ClassLoader.loadClass(ClassLoader.java:425) > >> 738 at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > >> 739 at java.lang.ClassLoader.loadClass(ClassLoader.java:358) > >> 740 at java.lang.Class.forName0(Native Method) > >> 741 at java.lang.Class.forName(Class.java:191) > >> 742 at > >> org.apache.calcite.model.ModelHandler.visit(ModelHandler.java:209) > >> > >> 2015-05-07 14:17 GMT+08:00 Tao Wong <[email protected]>: > >> > >> > After I build a new segment. the kylin.log on the query mode node. > >> > > >> > kylin.log: > >> > > >> > [http-bio-8082-exec-1]:[2015-05-07 > >> > > >> > 13:53:13,681][DEBUG][org.apache.kylin.rest.service.AdminService.getConfigAsString(AdminService > >> > .java:91)] - Get Kylin Runtime Config > >> > 3261 [http-bio-8082-exec-1]:[2015-05-07 > >> > > >> > 13:56:08,971][INFO][org.apache.kylin.metadata.project.ProjectL2Cache.loadCache(ProjectL2Cache. > >> > java:152)] - Loading L2 project cache for user > >> > 3262 [http-bio-8082-exec-1]:[2015-05-07 > >> > > >> > 13:56:08,974][INFO][org.apache.kylin.rest.controller.TableController.getHiveTables(TableContro > >> > ller.java:74)] - Return all table metadata in 3 seconds > >> > 3263 [http-bio-8082-exec-4]:[2015-05-07 > >> > 13:59:03,657][ERROR][org.apache.kylin.rest.controller.BasicController.handleError(BasicControl > >> > ler.java:52)] - > >> > 3264 java.lang.RuntimeException: Error instantiating > >> > JsonCustomSchema(name=USER) > >> > 3265 at > >> > org.apache.calcite.model.ModelHandler.visit(ModelHandler.java:217) > >> > 3266 at > >> > > >> > org.apache.calcite.model.JsonCustomSchema.accept(JsonCustomSchema.java:36) > >> > 3267 at > >> > org.apache.calcite.model.ModelHandler.visit(ModelHandler.java:144) > >> > 3268 at > >> > org.apache.calcite.model.ModelHandler.<init>(ModelHandler.java:87) > >> > 3269 at > >> > org.apache.calcite.jdbc.Driver$1.onConnectionInit(Driver.java:97) > >> > 3270 at > >> > > >> > org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:145) > >> > 3271 at > java.sql.DriverManager.getConnection(DriverManager.java:571) > >> > 3272 at > java.sql.DriverManager.getConnection(DriverManager.java:187) > >> > 3273 at > >> > > >> > org.springframework.jdbc.datasource.DriverManagerDataSource.getConnectionFromDriverManager(DriverManagerDataSource.java:17 > >> > 3) > >> > 3274 at > >> > > >> > org.springframework.jdbc.datasource.DriverManagerDataSource.getConnectionFromDriver(DriverManagerDataSource.java:164) > >> > 3275 at > >> > > >> > org.springframework.jdbc.datasource.AbstractDriverBasedDataSource.getConnectionFromDriver(AbstractDriverBasedDataSource.ja > >> > va:149) > >> > 3276 at > >> > > >> > org.springframework.jdbc.datasource.AbstractDriverBasedDataSource.getConnection(AbstractDriverBasedDataSource.java:119) > >> > > >> > 2015-05-07 9:15 GMT+08:00 Shi, Shaofeng <[email protected]>: > >> > > >> >> Dong, you can download a build directly from > >> >> https://kylin.incubator.apache.org/download/; > >> >> > >> >> On 5/7/15, 7:26 AM, "周千昊" <[email protected]> wrote: > >> >> > >> >> >Hi, dong > >> >> > #1 you don't have to add "http://"; prefix for the > >> >> kylin.rest.servers > >> >> >config, > >> >> > according to the log, the Broadcaster event has been > >> >> >successfully > >> >> >created, however there maybe some network issues when "ALL" server > >> trying > >> >> >to reach "QUERY" server, can you try to mockup a PUT request like > >> >> >http://test.abc.com:8081/cube/lbs_map_new_user_fact/update to see > if > >> the > >> >> >network is OK? > >> >> > #2 yes you can git clone the "0.7.1-staging" branch to build > the > >> >> >binary > >> >> > > >> >> >dong wang <[email protected]>于2015年5月7日周四 上午12:10写道： > >> >> > > >> >> >> 1, please help to review the above 2 response first > >> >> >> 2, in kylin.properties,currently we set kylin.rest.servers= > >> >> >> test.abc.com:8080 > >> >> >> ,test.abc.com:8081,test.abc.com:8082 and test.abc.com:8080 are > >> the > >> >> >>"ALL" > >> >> >> instance, the other 2 are the "QUERY" kylin instances > >> >> >> should we set with the 'http://' prefix like: > >> kylin.rest.servers= > >> >> >> > >> >> >>http://test.abc.com:8080,http://test.abc.com:8081, > >> >> http://test.abc.com:808 > >> >> >>2 > >> >> >> > >> >> >> 2015-05-06 23:40 GMT+08:00 dong wang <[email protected]>: > >> >> >> > >> >> >> > @shaofeng, another question is that can I git clone the branch > >> >> >> > "0.71-staging"( > >> https://github.com/KylinOLAP/Kylin/tree/0.7.1-staging > >> >> ) > >> >> >>to > >> >> >> > get the latest code to build the binary? > >> >> >> > > >> >> >> > 2015-05-06 23:38 GMT+08:00 dong wang <[email protected]>: > >> >> >> > > >> >> >> >> I double checked one piece of the above log of the "ALL" mode > >> kylin > >> >> >> >> instance: > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> > >> >> > >> >> > >> > >>17:36:21,910][INFO][org.apache.kylin.common.restclient.Broadcaster$1.run( > >> >> >>Broadcaster.java:71)] > >> >> >> >> - new broadcast event:BroadcastEvent{type=cube, name=lbs_ma > >> >> >> >> p_new_user_fact, action=update} > >> >> >> >> 2553 [pool-7-thread-10]:[2015-05-06 > >> >> >> >> > >> >> >> >> > >> >> >> > >> >> > >> >> > >> > >>17:36:21,911][DEBUG][org.apache.kylin.common.persistence.ResourceStore.pu > >> >> >>tResource(ResourceStore.java:171)] > >> >> >> >> - Saving resource /execute_output/f03d3c3c-68 > >> >> >> >> f4-4593-b01d-2778fcaf098b-15 (Store kylin_metadata@hbase) > >> >> >> >> 2554 [pool-7-thread-10]:[2015-05-06 > >> >> >> >> > >> >> >> >> > >> >> >> > >> >> > >> >> > >> > >>17:36:21,921][DEBUG][org.apache.kylin.common.persistence.ResourceStore.pu > >> >> >>tResource(ResourceStore.java:171)] > >> >> >> >> - Saving resource /execute_output/f03d3c3c-68 > >> >> >> >> f4-4593-b01d-2778fcaf098b-15 (Store kylin_metadata@hbase) > >> >> >> >> 2555 [http-bio-8081-exec-5]:[2015-05-06 > >> >> >> >> > >> >> >> >> > >> >> >> > >> >> > >> >> > >> > >>17:36:21,922][INFO][org.apache.kylin.rest.controller.CacheController.wipe > >> >> >>Cache(CacheController.java:63)] > >> >> >> >> - wipe cache type: CUBE event:UPDATE name:lb > >> s_map_new_user_fact > >> >> >> >> > >> >> >> >> > >> >> >> >> from the first line and the last line, it seems that it tries > to > >> >> call > >> >> >> >> sync something related to 'rest', however, when I check the > >> >> >>kylin.log > >> >> >> for > >> >> >> >> each of the "QUERY" mode kylin instances, no related log > reponse > >> to > >> >> >>the > >> >> >> >> rest sync > >> >> >> >> > >> >> >> > > >> >> >> > > >> >> >> > >> >> > >> >> > >> > > >> > > > > >
