[
https://issues.apache.org/jira/browse/PIG-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15725747#comment-15725747
]
Adam Szita commented on PIG-4923:
---------------------------------
Thanks for all your comments on this. I've updated the diff both here
([^PIG-4923.2.patch]) and reviewboard.
I tested the build with hadoop 3.0.0-alpha1 jars and there doesn't seem like a
lot of changes needed to make it pass:
{code}
compile:
[echo] *** Building Main Sources ***
[echo] *** To compile with all warnings enabled, supply -Dall.warnings=1
on command line ***
[echo] *** Else, you will only be warned about deprecations ***
[echo] *** Hadoop version used: 2 ; HBase version used: 1 ***
[javac] Compiling 1007 source files to
/Users/szita/shadow/CDH/pig/build/classes
[javac] warning: [options] bootstrap class path not set in conjunction with
-source 1.7
[javac]
/Users/szita/shadow/CDH/pig/src/org/apache/pig/backend/hadoop/datastorage/HDataStorage.java:34:
error: cannot find symbol
[javac] import org.apache.hadoop.hdfs.DistributedFileSystem;
[javac] ^
[javac] symbol: class DistributedFileSystem
[javac] location: package org.apache.hadoop.hdfs
[javac]
/Users/szita/shadow/CDH/pig/src/org/apache/pig/backend/hadoop/executionengine/HExecutionEngine.java:34:
error: cannot find symbol
[javac] import org.apache.hadoop.hdfs.DistributedFileSystem;
[javac] ^
[javac] symbol: class DistributedFileSystem
[javac] location: package org.apache.hadoop.hdfs
[javac]
/Users/szita/shadow/CDH/pig/src/org/apache/pig/backend/hadoop/datastorage/HDataStorage.java:117:
error: cannot find symbol
[javac] if (fs instanceof DistributedFileSystem) {
[javac] ^
[javac] symbol: class DistributedFileSystem
[javac] location: class HDataStorage
[javac]
/Users/szita/shadow/CDH/pig/src/org/apache/pig/backend/hadoop/datastorage/HDataStorage.java:118:
error: cannot find symbol
[javac] DistributedFileSystem dfs = (DistributedFileSystem) fs;
[javac] ^
[javac] symbol: class DistributedFileSystem
[javac] location: class HDataStorage
[javac]
/Users/szita/shadow/CDH/pig/src/org/apache/pig/backend/hadoop/datastorage/HDataStorage.java:118:
error: cannot find symbol
[javac] DistributedFileSystem dfs = (DistributedFileSystem) fs;
[javac] ^
[javac] symbol: class DistributedFileSystem
[javac] location: class HDataStorage
[javac]
/Users/szita/shadow/CDH/pig/src/org/apache/pig/backend/hadoop/executionengine/HExecutionEngine.java:198:
error: cannot find symbol
[javac] new DistributedFileSystem();
[javac] ^
[javac] symbol: class DistributedFileSystem
[javac] location: class HExecutionEngine
[javac] Note: Some input files use or override a deprecated API.
[javac] Note: Recompile with -Xlint:deprecation for details.
[javac] Note: Some input files use unchecked or unsafe operations.
[javac] Note: Recompile with -Xlint:unchecked for details.
[javac] 6 errors
[javac] 1 warning
{code}
Actually I went ahead and applied some small adjustments for Hadoop3 just for a
try, see: [^PIG-4923.2.IncrementalHadoop3.patch]
Are we really sure that we want to keep the shims layer? I think considering
what Hadoop3 will need it might not be a bad idea to remove the additional
complexity of using shims.
Let me know what you think.
> Drop Hadoop 1.x support in Pig 0.17
> -----------------------------------
>
> Key: PIG-4923
> URL: https://issues.apache.org/jira/browse/PIG-4923
> Project: Pig
> Issue Type: Improvement
> Reporter: Daniel Dai
> Assignee: Adam Szita
> Fix For: 0.17.0
>
> Attachments: PIG-4923.1.patch, PIG-4923.2.IncrementalHadoop3.patch,
> PIG-4923.2.patch, ant-contrib-1.0b3.jar
>
>
> To facilitate the future development, we want to get rid of the legacy Hadoop
> 1.x support and reduce the code complexity.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
