[ https://issues.apache.org/jira/browse/HIVE-860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13904751#comment-13904751 ]
Alan Gates commented on HIVE-860: --------------------------------- Brock, I'm trying to understand if there are any backward compatibility issues here. Are you proposing to change the contents of the hive-exec.jar as distributed with Hive or just as pushed to Hadoop for running a job? If it's the former won't it mean that any project that includes hive-exec.jar in it's pom.xml will have to change its pom to explicitly include all of the extra jars now in the fat jar? > Persistent distributed cache > ---------------------------- > > Key: HIVE-860 > URL: https://issues.apache.org/jira/browse/HIVE-860 > Project: Hive > Issue Type: Improvement > Affects Versions: 0.12.0 > Reporter: Zheng Shao > Assignee: Brock Noland > Fix For: 0.13.0 > > Attachments: HIVE-860.patch, HIVE-860.patch, HIVE-860.patch, > HIVE-860.patch, HIVE-860.patch, HIVE-860.patch > > > DistributedCache is shared across multiple jobs, if the hdfs file name is the > same. > We need to make sure Hive put the same file into the same location every time > and do not overwrite if the file content is the same. > We can achieve 2 different results: > A1. Files added with the same name, timestamp, and md5 in the same session > will have a single copy in distributed cache. > A2. Filed added with the same name, timestamp, and md5 will have a single > copy in distributed cache. > A2 has a bigger benefit in sharing but may raise a question on when Hive > should clean it up in hdfs. -- This message was sent by Atlassian JIRA (v6.1.5#6160)