Problem with the windowing function ntile (Exceptions)

[Lars Francke]

Hi,

I'm running a query like this:

CREATE TABLE foo
  STORED AS ORC
AS
SELECT
  id,
  season,
  amount,
  ntile(10)
OVER (
  PARTITION BY season
  ORDER BY amount DESC
)
FROM bar;

On a small enough dataset that works fine but when switching to a
larger sample we're seeing exceptions like this:

"Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Reset on
PersistentByteBasedList not supported"

Looking at the code (without really understanding it) we tried setting:
SET 
hive.ptf.partition.persistence='org.apache.hadoop.hive.ql.exec.PTFPersistence$PartitionedByteBasedList';

because that List supports reset but we are seeing a
ClassNotFoundException so we're doing that wrong.

Next try was setting hive.ptf.partition.persistence.memsize higher
which worked but first of all we don't really understand what all of
that stuff is doing and second of all we fear that it just might break
down again.

Any hints as to what that error really means and how to deal with it
would be greatly appreciated.

Thanks!

Lars