[jira] [Commented] (PIG-4160) -forcelocaljars / -j flag when using a remote url for a script
[ https://issues.apache.org/jira/browse/PIG-4160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190819#comment-14190819 ] Thejas M Nair commented on PIG-4160: I think it would be useful to retain mention of the old additional.jars parameter in the web docs, Saying that it is similar to .jars.comma but separated by the OS path charactor and does not allow for scheme to be specified (and that the use of jars.comma is preferred) > -forcelocaljars / -j flag when using a remote url for a script > -- > > Key: PIG-4160 > URL: https://issues.apache.org/jira/browse/PIG-4160 > Project: Pig > Issue Type: Improvement > Components: impl >Affects Versions: 0.14.0 >Reporter: Andrew C. Oliver > Labels: patch > Fix For: 0.14.0 > > Attachments: PIG-4160-2.patch, PIG-4160-3.patch, > forcelocal.trunk.patch, forcelocal.withtests.patch > > Original Estimate: 3h > Remaining Estimate: 3h > > patch adds a -j/forcelocaljars flag which if enabled allows you to do > pig -j -useHCatalog hdfs://myserver:8020/load/scripts/mydir/myscript.pig > thus loading the pig script REMOTELY > while loading the jar files LOCALLY > One does this to avoid a single point of failure but avoid one central > interversion dependent repository for all the jars across all teams/projects. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PIG-4160) -forcelocaljars / -j flag when using a remote url for a script
[ https://issues.apache.org/jira/browse/PIG-4160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190821#comment-14190821 ] Thejas M Nair commented on PIG-4160: I think we should also change the jira title and description to indicate what is being implemented. > -forcelocaljars / -j flag when using a remote url for a script > -- > > Key: PIG-4160 > URL: https://issues.apache.org/jira/browse/PIG-4160 > Project: Pig > Issue Type: Improvement > Components: impl >Affects Versions: 0.14.0 >Reporter: Andrew C. Oliver > Labels: patch > Fix For: 0.14.0 > > Attachments: PIG-4160-2.patch, PIG-4160-3.patch, > forcelocal.trunk.patch, forcelocal.withtests.patch > > Original Estimate: 3h > Remaining Estimate: 3h > > patch adds a -j/forcelocaljars flag which if enabled allows you to do > pig -j -useHCatalog hdfs://myserver:8020/load/scripts/mydir/myscript.pig > thus loading the pig script REMOTELY > while loading the jar files LOCALLY > One does this to avoid a single point of failure but avoid one central > interversion dependent repository for all the jars across all teams/projects. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PIG-4160) -forcelocaljars / -j flag when using a remote url for a script
[ https://issues.apache.org/jira/browse/PIG-4160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190713#comment-14190713 ] Thejas M Nair commented on PIG-4160: Daniel, can you also include the doc edits ? Also the commented line "#HADOOP_OPTS" can be deleted. > -forcelocaljars / -j flag when using a remote url for a script > -- > > Key: PIG-4160 > URL: https://issues.apache.org/jira/browse/PIG-4160 > Project: Pig > Issue Type: Improvement > Components: impl >Affects Versions: 0.14.0 >Reporter: Andrew C. Oliver > Labels: patch > Fix For: 0.14.0 > > Attachments: PIG-4160-2.patch, forcelocal.trunk.patch, > forcelocal.withtests.patch > > Original Estimate: 3h > Remaining Estimate: 3h > > patch adds a -j/forcelocaljars flag which if enabled allows you to do > pig -j -useHCatalog hdfs://myserver:8020/load/scripts/mydir/myscript.pig > thus loading the pig script REMOTELY > while loading the jar files LOCALLY > One does this to avoid a single point of failure but avoid one central > interversion dependent repository for all the jars across all teams/projects. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PIG-4160) -forcelocaljars / -j flag when using a remote url for a script
[ https://issues.apache.org/jira/browse/PIG-4160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172452#comment-14172452 ] Andrew C. Oliver commented on PIG-4160: --- oh you beat me to it, even better :) > -forcelocaljars / -j flag when using a remote url for a script > -- > > Key: PIG-4160 > URL: https://issues.apache.org/jira/browse/PIG-4160 > Project: Pig > Issue Type: Improvement > Components: impl >Affects Versions: 0.14.0 >Reporter: Andrew C. Oliver > Labels: patch > Fix For: 0.14.0 > > Attachments: PIG-4160-2.patch, forcelocal.trunk.patch, > forcelocal.withtests.patch > > Original Estimate: 3h > Remaining Estimate: 3h > > patch adds a -j/forcelocaljars flag which if enabled allows you to do > pig -j -useHCatalog hdfs://myserver:8020/load/scripts/mydir/myscript.pig > thus loading the pig script REMOTELY > while loading the jar files LOCALLY > One does this to avoid a single point of failure but avoid one central > interversion dependent repository for all the jars across all teams/projects. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PIG-4160) -forcelocaljars / -j flag when using a remote url for a script
[ https://issues.apache.org/jira/browse/PIG-4160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172451#comment-14172451 ] Andrew C. Oliver commented on PIG-4160: --- will do. > -forcelocaljars / -j flag when using a remote url for a script > -- > > Key: PIG-4160 > URL: https://issues.apache.org/jira/browse/PIG-4160 > Project: Pig > Issue Type: Improvement > Components: impl >Affects Versions: 0.14.0 >Reporter: Andrew C. Oliver > Labels: patch > Fix For: 0.14.0 > > Attachments: PIG-4160-2.patch, forcelocal.trunk.patch, > forcelocal.withtests.patch > > Original Estimate: 3h > Remaining Estimate: 3h > > patch adds a -j/forcelocaljars flag which if enabled allows you to do > pig -j -useHCatalog hdfs://myserver:8020/load/scripts/mydir/myscript.pig > thus loading the pig script REMOTELY > while loading the jar files LOCALLY > One does this to avoid a single point of failure but avoid one central > interversion dependent repository for all the jars across all teams/projects. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PIG-4160) -forcelocaljars / -j flag when using a remote url for a script
[ https://issues.apache.org/jira/browse/PIG-4160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14147128#comment-14147128 ] Daniel Dai commented on PIG-4160: - hcat/hive jars are known to be locally, so bin/pig will pass file://xxx to Pig to clarify. > -forcelocaljars / -j flag when using a remote url for a script > -- > > Key: PIG-4160 > URL: https://issues.apache.org/jira/browse/PIG-4160 > Project: Pig > Issue Type: Improvement > Components: impl >Affects Versions: 0.14.0 >Reporter: Andrew C. Oliver > Labels: patch > Fix For: 0.14.0 > > Attachments: PIG-4160-2.patch, forcelocal.trunk.patch, > forcelocal.withtests.patch > > Original Estimate: 3h > Remaining Estimate: 3h > > patch adds a -j/forcelocaljars flag which if enabled allows you to do > pig -j -useHCatalog hdfs://myserver:8020/load/scripts/mydir/myscript.pig > thus loading the pig script REMOTELY > while loading the jar files LOCALLY > One does this to avoid a single point of failure but avoid one central > interversion dependent repository for all the jars across all teams/projects. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PIG-4160) -forcelocaljars / -j flag when using a remote url for a script
[ https://issues.apache.org/jira/browse/PIG-4160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131756#comment-14131756 ] Andrew C. Oliver commented on PIG-4160: --- Thanks! where do you want it? Its own Test file or some particular one? TestMain seemed the most obvious to me > -forcelocaljars / -j flag when using a remote url for a script > -- > > Key: PIG-4160 > URL: https://issues.apache.org/jira/browse/PIG-4160 > Project: Pig > Issue Type: Improvement > Components: impl >Affects Versions: 0.14.0 >Reporter: Andrew C. Oliver > Labels: patch > Fix For: 0.14.0 > > Attachments: forcelocal.trunk.patch > > Original Estimate: 3h > Remaining Estimate: 3h > > patch adds a -j/forcelocaljars flag which if enabled allows you to do > pig -j -useHCatalog hdfs://myserver:8020/load/scripts/mydir/myscript.pig > thus loading the pig script REMOTELY > while loading the jar files LOCALLY > One does this to avoid a single point of failure but avoid one central > interversion dependent repository for all the jars across all teams/projects. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PIG-4160) -forcelocaljars / -j flag when using a remote url for a script
[ https://issues.apache.org/jira/browse/PIG-4160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14127264#comment-14127264 ] Daniel Dai commented on PIG-4160: - Take a look of TestPigRunner.scriptsInDfsTest, it uploads a Pig script to dfs and run the remote script. Also TestGrunt.validate, which invoke Main directly in test. > -forcelocaljars / -j flag when using a remote url for a script > -- > > Key: PIG-4160 > URL: https://issues.apache.org/jira/browse/PIG-4160 > Project: Pig > Issue Type: Improvement > Components: impl >Affects Versions: 0.14.0 >Reporter: Andrew C. Oliver > Labels: patch > Fix For: 0.14.0 > > Attachments: forcelocal.trunk.patch > > Original Estimate: 3h > Remaining Estimate: 3h > > patch adds a -j/forcelocaljars flag which if enabled allows you to do > pig -j -useHCatalog hdfs://myserver:8020/load/scripts/mydir/myscript.pig > thus loading the pig script REMOTELY > while loading the jar files LOCALLY > One does this to avoid a single point of failure but avoid one central > interversion dependent repository for all the jars across all teams/projects. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PIG-4160) -forcelocaljars / -j flag when using a remote url for a script
[ https://issues.apache.org/jira/browse/PIG-4160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126830#comment-14126830 ] Andrew C. Oliver commented on PIG-4160: --- Sure, any suggestions on how to go about that? Meaning there aren't currently any unit tests under "test" that actually use hdfs:// I could easily create a pig script that loaded one jar and either resolved it locally or didn't...provided I figure out the hdfs:// part. > -forcelocaljars / -j flag when using a remote url for a script > -- > > Key: PIG-4160 > URL: https://issues.apache.org/jira/browse/PIG-4160 > Project: Pig > Issue Type: Improvement > Components: impl >Affects Versions: 0.14.0 >Reporter: Andrew C. Oliver > Labels: patch > Fix For: 0.14.0 > > Attachments: forcelocal.trunk.patch > > Original Estimate: 3h > Remaining Estimate: 3h > > patch adds a -j/forcelocaljars flag which if enabled allows you to do > pig -j -useHCatalog hdfs://myserver:8020/load/scripts/mydir/myscript.pig > thus loading the pig script REMOTELY > while loading the jar files LOCALLY > One does this to avoid a single point of failure but avoid one central > interversion dependent repository for all the jars across all teams/projects. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PIG-4160) -forcelocaljars / -j flag when using a remote url for a script
[ https://issues.apache.org/jira/browse/PIG-4160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126452#comment-14126452 ] Daniel Dai commented on PIG-4160: - Do you mind adding a unit test? > -forcelocaljars / -j flag when using a remote url for a script > -- > > Key: PIG-4160 > URL: https://issues.apache.org/jira/browse/PIG-4160 > Project: Pig > Issue Type: Improvement > Components: impl >Affects Versions: 0.14.0 >Reporter: Andrew C. Oliver > Labels: patch > Fix For: 0.14.0 > > Attachments: forcelocal.trunk.patch > > Original Estimate: 3h > Remaining Estimate: 3h > > patch adds a -j/forcelocaljars flag which if enabled allows you to do > pig -j -useHCatalog hdfs://myserver:8020/load/scripts/mydir/myscript.pig > thus loading the pig script REMOTELY > while loading the jar files LOCALLY > One does this to avoid a single point of failure but avoid one central > interversion dependent repository for all the jars across all teams/projects. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
