Exactly, using the external table. After creating the table, let your upstreaming app output files to the location. And then you can query the data from Hive; If the table is available in Hive, that means Kylin can fetch the data.
2017-11-14 16:13 GMT+08:00 Kumar, Manoj H <[email protected]>: > Do you mean that using external table it can be possible ? Just sample – > Table name would be same as File name & upstream will be loading the file > in that Directory? > > > > create external table parquet_table_name (x INT, y STRING) > > ROW FORMAT SERDE 'parquet.hive.serde.ParquetHiveSerDe' > > STORED AS > > INPUTFORMAT "parquet.hive.DeprecatedParquetInputFormat" > > OUTPUTFORMAT "parquet.hive.DeprecatedParquetOutputFormat" > > LOCATION '/user/cloudera/tinytable'; > > > > Regards, > > Manoj > > > > *From:* ShaoFeng Shi [mailto:[email protected]] > *Sent:* Tuesday, November 14, 2017 12:58 PM > *To:* user > *Subject:* Re: Apache Kylin - Does it take input data from Parquet file > format > > > > Hi Kumar, > > > > You can create a Hive table with Parquet format, and then let upstream > application output the parquet files to this Hive table's HDFS location > (each time can be in a new partition). Then Kylin will be able to fetch the > new data incrementally from this table. > > > > Hive provides the logic layer over the physical model, which is > recommended. > > > > > > 2017-11-14 14:30 GMT+08:00 Kumar, Manoj H <[email protected]>: > > Pls. advise on this – Input source of data – Can it from Parquet File > format instead of Hive table? Upstream system generating the File format as > output. Pls. suggest. This will be important features to have it. > > > > Manoj > > This message is confidential and subject to terms at: http:// > www.jpmorgan.com/emaildisclaimer including on confidentiality, legal > privilege, viruses and monitoring of electronic messages. If you are not > the intended recipient, please delete this message and notify the sender > immediately. Any unauthorized use is strictly prohibited. > > > > > > -- > > Best regards, > > > > Shaofeng Shi 史少锋 > > > > This message is confidential and subject to terms at: http:// > www.jpmorgan.com/emaildisclaimer including on confidentiality, legal > privilege, viruses and monitoring of electronic messages. If you are not > the intended recipient, please delete this message and notify the sender > immediately. Any unauthorized use is strictly prohibited. > -- Best regards, Shaofeng Shi 史少锋
