Hello hive-users, I am reading a hive table with skip.header.line.count set
to 1 in TBLPROPERTIES
In the driver code I do this.
val hiveMetaStoreClient = new HiveMetaStoreClient(new
HiveConf(job.getConfiguration, HiveIngestDriver.getClass))
val hiveTable:Table = hiveMetaStoreClient.getTable("default",
"hiveTableName")
val hiveTableProperties = new Properties()
hiveTableProperties.putAll(hiveTable.getParameters)
logger.info("size: {} getParameters: {}", hiveTable.getParametersSize,
hiveTableProperties.toMap)
val hCatInputFormat = HCatInputFormat.setInput(job.getConfiguration,
"default", "hiveTableName", "day=2017-06-01")
hCatInputFormat.setProperties(hiveTableProperties)
job.setInputFormatClass(classOf[HCatInputFormat])
Log from above shows that skip.header.line.count is set correctly. Even then
HCatInputFormat is unable to apply this as I see the header row in the output.
""
size: 4 getParameters: {last_modified_by=myuser, last_modified_time=1468952183,
transient_lastDdlTime=1468952183, skip.header.line.count=1}""
Any suggestions???
Thanks,-Vinay
