Re: Populating tables using hive and spark

2016-08-22 Thread Nitin Kumar
> So I guess we can call it a bug: > > Hive should detect that the files changed and invalidate its > pre-calculated count. > Optionally, Spark should be nice with Hive and update the the count when > inserting. > > I don't know if this bug has already been reported, and I

Populating tables using hive and spark

2016-08-22 Thread Nitin Kumar
Hi! I've noticed that hive has problems in registering new data records if the same table is written to using both the hive terminal and spark sql. The problem is demonstrated through the commands listed below hive> use