Re: ORDER BY illustrator
I don't know what the issue is, but I can JIRA it for you. What happens when you try the same on Pig 0.10.0? Russell Jurney http://datasyndrome.com On Oct 9, 2012, at 2:10 PM, Allan wrote: > Hi to everybody, > > I was trying to execute this script with Pig (from trunk r1396318) > > A = LOAD 'test01' AS (f1:chararray,f2:int,f3:chararray); > B = order A by f1; > illustrate B; > > And I get this exception: > > == > > java.lang.RuntimeException: ReadOnceBag does not support getMemorySize > operation > at org.apache.pig.data.ReadOnceBag.getMemorySize(ReadOnceBag.java:74) > at org.apache.pig.data.SizeUtil.getPigObjMemSize(SizeUtil.java:61) > at org.apache.pig.data.DefaultTuple.getMemorySize(DefaultTuple.java:180) > at org.apache.pig.pen.util.ExampleTuple.getMemorySize(ExampleTuple.java:97) > at > org.apache.pig.data.DefaultAbstractBag.getMemorySize(DefaultAbstractBag.java:148) > at > org.apache.pig.data.DefaultAbstractBag.markSpillableIfNecessary(DefaultAbstractBag.java:100) > at org.apache.pig.data.DefaultAbstractBag.add(DefaultAbstractBag.java:92) > at org.apache.pig.pen.Illustrator.addData(Illustrator.java:116) > at > org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POPackageLite.illustratorMarkup(POPackageLite.java:227) > at > org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POPackageLite.getNext(POPackageLite.java:182) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.processOnePackageOutput(PigGenericMapReduce.java:422) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.reduce(PigGenericMapReduce.java:413) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.reduce(PigGenericMapReduce.java:257) > at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176) > at > org.apache.pig.pen.LocalMapReduceSimulator.launchPig(LocalMapReduceSimulator.java:235) > at org.apache.pig.pen.ExampleGenerator.getData(ExampleGenerator.java:257) > at org.apache.pig.pen.ExampleGenerator.getData(ExampleGenerator.java:238) > at > org.apache.pig.pen.LineageTrimmingVisitor.init(LineageTrimmingVisitor.java:103) > at > org.apache.pig.pen.LineageTrimmingVisitor.(LineageTrimmingVisitor.java:98) > at > org.apache.pig.pen.ExampleGenerator.getExamples(ExampleGenerator.java:166) > at org.apache.pig.PigServer.getExamples(PigServer.java:1180) > at > org.apache.pig.tools.grunt.GruntParser.processIllustrate(GruntParser.java:738) > at > org.apache.pig.tools.pigscript.parser.PigScriptParser.Illustrate(PigScriptParser.java:626) > at > org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:323) > at > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:193) > at > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:169) > at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69) > at org.apache.pig.Main.run(Main.java:538) > at org.apache.pig.Main.main(Main.java:154) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.util.RunJar.main(RunJar.java:156) > > == > > and at log file: > > Pig Stack Trace > --- > ERROR 2997: Encountered IOException. Exception > > java.io.IOException: Exception >at org.apache.pig.PigServer.getExamples(PigServer.java:1186) >at > org.apache.pig.tools.grunt.GruntParser.processIllustrate(GruntParser.java:738) >at > org.apache.pig.tools.pigscript.parser.PigScriptParser.Illustrate(PigScriptParser.java:626) >at > org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:323) >at > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:193) >at > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:169) >at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69) >at org.apache.pig.Main.run(Main.java:538) >at org.apache.pig.Main.main(Main.java:154) >at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >at java.lang.reflect.Method.invoke(Method.java:597) >at org.apache.hadoop.util.RunJar.main(RunJar.java:156) > Caused by: java.lang.RuntimeException: ReadOnceBag does not support > getMemorySize operation >at > org.apache.pig.data.ReadOnceBag.getMemorySize(ReadOnceBag.java:74) >at org.apache.pig.data.SizeUti
Re: ORDER BY illustrator
Thanks for your reply Russell. On Wed, Oct 10, 2012 at 12:38 AM, Russell Jurney wrote: > Does: illustrate A; work? > > Yes, it's working > Does: C = foreach B generate f1, f2; illustrate C; work? > No, same exception. > Russell Jurney http://datasyndrome.com > > -- Allan Avendaño S. --
[jira] [Commented] (PIG-2910) Make toString() methods on Schema and FieldSchema be readable by Utils.getSchemaFromString()
[
https://issues.apache.org/jira/browse/PIG-2910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472941#comment-13472941
]
Thejas M Nair commented on PIG-2910:
The extra parenthesis added by Schema.toString() are curly braces. I believe
this is because the schema when thought of as a schema of a relation, is a
schema of a bag. I don't think the PIG-2910-2.patch will fix the issue.
But changing the behavior of either Schema.toString() or
Utils.getSchemaFromString() will break backward compatibility. I think we
should just add a new function Utils.getSchemaFromBagSchemaString() and comment
that people should use this one to get schema back from output of
Schema.toString().
The use of input schema of udf, during udf execution is not very common. I
don't think we should serialize it for all udfs.
> Make toString() methods on Schema and FieldSchema be readable by
> Utils.getSchemaFromString()
>
>
> Key: PIG-2910
> URL: https://issues.apache.org/jira/browse/PIG-2910
> Project: Pig
> Issue Type: Bug
> Components: impl, parser
>Affects Versions: 0.9.2, 0.10.0, 0.11, 0.10.1
>Reporter: Russell Jurney
>Assignee: Thejas M Nair
> Labels: newbie
> Fix For: 0.11, 0.10.1
>
> Attachments: PIG-2910-1.patch, PIG-2910-2.patch
>
>
> I want to toString() schemas and send them to the backend via UDFContext. At
> the moment this requires writing your own toString() method that
> Utils.getSchemaFromString() can read. Making a readable schema for the
> backend would be an improvement.
> I spoke with Thejas, who believes this is a bug. The workaround for the
> moment is, for example:
> String schemaString = inputSchema.toString().substring(1,
> inputSchema.toString().length() - 1);
> // Set the input schema for processing
> UDFContext context = UDFContext.getUDFContext();
> Properties udfProp = context.getUDFProperties(this.getClass());
> udfProp.setProperty("horton.json.udf.schema", schemaString);
> ...
> schema = Utils.getSchemaFromString(strSchema);
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Subscription: PIG patch available
Issue Subscription Filter: PIG patch available (36 issues) Subscriber: pigdaily Key Summary PIG-2960Increase the timeout for unit test https://issues.apache.org/jira/browse/PIG-2960 PIG-2959Add a pig.cmd for Pig to run under Windows https://issues.apache.org/jira/browse/PIG-2959 PIG-2958Pig tests do not appear to have a logger attached https://issues.apache.org/jira/browse/PIG-2958 PIG-2957TetsScriptUDF fail due to volume prefix in jar https://issues.apache.org/jira/browse/PIG-2957 PIG-2956Invalid cache specification for some streaming statement https://issues.apache.org/jira/browse/PIG-2956 PIG-2955 Fix bunch of Pig e2e tests on Windows https://issues.apache.org/jira/browse/PIG-2955 PIG-2954 TestParamSubPreproc still depends on "bash" to run https://issues.apache.org/jira/browse/PIG-2954 PIG-2953"which" utility does not exist on Windows https://issues.apache.org/jira/browse/PIG-2953 PIG-2943DevTests, Refactor Windows checks to use new Util.WINDOWS method for code health https://issues.apache.org/jira/browse/PIG-2943 PIG-2942DevTests, TestLoad has a false failure on Windows https://issues.apache.org/jira/browse/PIG-2942 PIG-2940HBaseStorage store fails in secure cluster https://issues.apache.org/jira/browse/PIG-2940 PIG-2931$ signs in the replacement string make parameter substitution fail https://issues.apache.org/jira/browse/PIG-2931 PIG-2928Fix e2e test failures in trunk: FilterBoolean_23/24 https://issues.apache.org/jira/browse/PIG-2928 PIG-2925Extremely long JobConf values should not be added to Streaming environment https://issues.apache.org/jira/browse/PIG-2925 PIG-2908Fix unit tests to work with jdk7 https://issues.apache.org/jira/browse/PIG-2908 PIG-2898Parallel execution of e2e tests https://issues.apache.org/jira/browse/PIG-2898 PIG-2881Add SUBTRACT eval function https://issues.apache.org/jira/browse/PIG-2881 PIG-2873Converting bin/pig shell script to python https://issues.apache.org/jira/browse/PIG-2873 PIG-2834MultiStorage requires unused constructor argument https://issues.apache.org/jira/browse/PIG-2834 PIG-2824Pushing checking number of fields into LoadFunc https://issues.apache.org/jira/browse/PIG-2824 PIG-2801grunt "sh" command should invoke the shell implicitly instead of calling exec directly with the command tokens https://issues.apache.org/jira/browse/PIG-2801 PIG-2798pig streaming tests assume interpreters are auto-resolved https://issues.apache.org/jira/browse/PIG-2798 PIG-2796Local temporary paths are not always valid HDFS path names. https://issues.apache.org/jira/browse/PIG-2796 PIG-2795Fix test cases that generate pig scripts with "load " + pathStr to encode "\" in the path https://issues.apache.org/jira/browse/PIG-2795 PIG-2794Pig test: add utils to simplify testing on Windows https://issues.apache.org/jira/browse/PIG-2794 PIG-2778Add 'matches' operator to predicate pushdown https://issues.apache.org/jira/browse/PIG-2778 PIG-2661Pig uses an extra job for loading data in Pigmix L9 https://issues.apache.org/jira/browse/PIG-2661 PIG-2657Print warning if using wrong jython version https://issues.apache.org/jira/browse/PIG-2657 PIG-2579Support for multiple input schemas in AvroStorage https://issues.apache.org/jira/browse/PIG-2579 PIG-2495Using merge JOIN from a HBaseStorage produces an error https://issues.apache.org/jira/browse/PIG-2495 PIG-2417Streaming UDFs - allow users to easily write UDFs in scripting languages with no JVM implementation. https://issues.apache.org/jira/browse/PIG-2417 PIG-2405svn tags/release-0.9.1: some unit test case failed with open JDK https://issues.apache.org/jira/browse/PIG-2405 PIG-2362Rework Ant build.xml to use macrodef instead of antcall https://issues.apache.org/jira/browse/PIG-2362 PIG-2312NPE when relation and column share the same name and used in Nested Foreach https://issues.apache.org/jira/browse/PIG-2312 PIG-1942script UDF (jython) should utilize the intended output schema to more directly convert Py objects to Pig objects https://issues.apache.org/jira/browse/PIG-1942 PIG-1237Piggybank MutliStorage - specify field to write in output https://issues.apache.org/jira/browse/PIG-1237 You may edit this subscription at: https://issues.apache.org/jira/secure/FilterSubscription!default.jspa?subId=13225&filterId=12322384
[jira] [Commented] (PIG-2957) TetsScriptUDF fail due to volume prefix in jar
[ https://issues.apache.org/jira/browse/PIG-2957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472838#comment-13472838 ] Dmitriy V. Ryaboy commented on PIG-2957: is it safe to assume we are guaranteed to be on windows, and that lopping the volume off is safe, just by checking "scriptPath.charAt(1)==':'" ? Would it be better to isolate logic for figuring out if we are in windows behind some function? > TetsScriptUDF fail due to volume prefix in jar > -- > > Key: PIG-2957 > URL: https://issues.apache.org/jira/browse/PIG-2957 > Project: Pig > Issue Type: Sub-task >Reporter: Daniel Dai >Assignee: Daniel Dai > Fix For: 0.11 > > Attachments: PIG-2957-1.patch > > > testPythonAbsolutePath fail. Stack is: > java.io.IOException: Mkdirs failed to create > C:\tmp\hadoop-Administrator\mapred\local\1_0\taskTracker\Administrator\jobcache\job_20120725074728013_0011\jars\C:\Users\Administrator\pig-monarch > at org.apache.hadoop.util.RunJar.unJar(RunJar.java:47) > at > org.apache.hadoop.mapred.JobLocalizer.localizeJobJarFile(JobLocalizer.java:277) > at > org.apache.hadoop.mapred.JobLocalizer.localizeJobFiles(JobLocalizer.java:377) > at > org.apache.hadoop.mapred.JobLocalizer.localizeJobFiles(JobLocalizer.java:367) > at > org.apache.hadoop.mapred.DefaultTaskController.initializeJob(DefaultTaskController.java:214) > at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1237) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1107) > at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1212) > at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1127) > at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2417) > at java.lang.Thread.run(Thread.java:662) > The reason is we pack the volume prefix into the job.jar. > jar tvf C:\Users\ADMINI~1\AppData\Local\Temp\Job6350 > 669482684441868.jar|grep testPythonAbsolutePath > 98 Wed Jul 25 11:12:58 PDT 2012 C:\Users\Administrator\pig-monarch\testPytho > nAbsolutePath.py -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2958) Pig tests do not appear to have a logger attached
[ https://issues.apache.org/jira/browse/PIG-2958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472824#comment-13472824 ] Dmitriy V. Ryaboy commented on PIG-2958: +1 > Pig tests do not appear to have a logger attached > - > > Key: PIG-2958 > URL: https://issues.apache.org/jira/browse/PIG-2958 > Project: Pig > Issue Type: Sub-task >Reporter: Daniel Dai >Assignee: Daniel Dai > Fix For: 0.11 > > Attachments: PIG-2958-1.patch > > > This causes false failures in TestPigRunner, but also makes debugging > somewhat more difficult than it has to be. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2951) Overflow, Underflow errors
[ https://issues.apache.org/jira/browse/PIG-2951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472819#comment-13472819 ] Dmitriy V. Ryaboy commented on PIG-2951: I don't think killing whole jobs by throwing an exception on a single case of overflow is good. How about we increment a counter instead, and use the appropriate max or min value as a result instead? (eg, mysql in non-strict mode uses max_int if you insert 2*max+int) > Overflow, Underflow errors > -- > > Key: PIG-2951 > URL: https://issues.apache.org/jira/browse/PIG-2951 > Project: Pig > Issue Type: Bug > Components: impl >Affects Versions: 0.0.0, 0.1.0, 0.2.0, 0.3.0, 0.4.0, 0.5.0, 0.6.0, 0.7.0, > 0.8.0, 0.8.1, 0.9.0, 0.9.1, 0.9.2, 0.10.0 >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: pig-2951.patch > > > With very large (or very small) integer values there is a possibility of > overflow (or underflow) errors. Worse thing is instead of failing, this > currently results in incorrect results being returned, thereby leaving user > with no clue that some of the tuples may have wrong value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: ORDER BY illustrator
Does: illustrate A; work? Does: C = foreach B generate f1, f2; illustrate C; work? Russell Jurney http://datasyndrome.com On Oct 9, 2012, at 2:10 PM, Allan wrote: > Hi to everybody, > > I was trying to execute this script with Pig (from trunk r1396318) > > A = LOAD 'test01' AS (f1:chararray,f2:int,f3:chararray); > B = order A by f1; > illustrate B; > > And I get this exception: > > == > > java.lang.RuntimeException: ReadOnceBag does not support getMemorySize > operation > at org.apache.pig.data.ReadOnceBag.getMemorySize(ReadOnceBag.java:74) > at org.apache.pig.data.SizeUtil.getPigObjMemSize(SizeUtil.java:61) > at org.apache.pig.data.DefaultTuple.getMemorySize(DefaultTuple.java:180) > at org.apache.pig.pen.util.ExampleTuple.getMemorySize(ExampleTuple.java:97) > at > org.apache.pig.data.DefaultAbstractBag.getMemorySize(DefaultAbstractBag.java:148) > at > org.apache.pig.data.DefaultAbstractBag.markSpillableIfNecessary(DefaultAbstractBag.java:100) > at org.apache.pig.data.DefaultAbstractBag.add(DefaultAbstractBag.java:92) > at org.apache.pig.pen.Illustrator.addData(Illustrator.java:116) > at > org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POPackageLite.illustratorMarkup(POPackageLite.java:227) > at > org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POPackageLite.getNext(POPackageLite.java:182) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.processOnePackageOutput(PigGenericMapReduce.java:422) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.reduce(PigGenericMapReduce.java:413) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.reduce(PigGenericMapReduce.java:257) > at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176) > at > org.apache.pig.pen.LocalMapReduceSimulator.launchPig(LocalMapReduceSimulator.java:235) > at org.apache.pig.pen.ExampleGenerator.getData(ExampleGenerator.java:257) > at org.apache.pig.pen.ExampleGenerator.getData(ExampleGenerator.java:238) > at > org.apache.pig.pen.LineageTrimmingVisitor.init(LineageTrimmingVisitor.java:103) > at > org.apache.pig.pen.LineageTrimmingVisitor.(LineageTrimmingVisitor.java:98) > at > org.apache.pig.pen.ExampleGenerator.getExamples(ExampleGenerator.java:166) > at org.apache.pig.PigServer.getExamples(PigServer.java:1180) > at > org.apache.pig.tools.grunt.GruntParser.processIllustrate(GruntParser.java:738) > at > org.apache.pig.tools.pigscript.parser.PigScriptParser.Illustrate(PigScriptParser.java:626) > at > org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:323) > at > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:193) > at > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:169) > at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69) > at org.apache.pig.Main.run(Main.java:538) > at org.apache.pig.Main.main(Main.java:154) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.util.RunJar.main(RunJar.java:156) > > == > > and at log file: > > Pig Stack Trace > --- > ERROR 2997: Encountered IOException. Exception > > java.io.IOException: Exception >at org.apache.pig.PigServer.getExamples(PigServer.java:1186) >at > org.apache.pig.tools.grunt.GruntParser.processIllustrate(GruntParser.java:738) >at > org.apache.pig.tools.pigscript.parser.PigScriptParser.Illustrate(PigScriptParser.java:626) >at > org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:323) >at > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:193) >at > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:169) >at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69) >at org.apache.pig.Main.run(Main.java:538) >at org.apache.pig.Main.main(Main.java:154) >at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >at java.lang.reflect.Method.invoke(Method.java:597) >at org.apache.hadoop.util.RunJar.main(RunJar.java:156) > Caused by: java.lang.RuntimeException: ReadOnceBag does not support > getMemorySize operation >at > org.apache.pig.data.ReadOnceBag.getMemorySize(ReadOnceBag.java:74) >at org.apache.pig.data.SizeUtil.getPigObjMemSize(SizeUt
[jira] [Commented] (PIG-2959) Add a pig.cmd for Pig to run under Windows
[ https://issues.apache.org/jira/browse/PIG-2959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472782#comment-13472782 ] Dmitriy V. Ryaboy commented on PIG-2959: how are we going to keep this in sync with the bash script? > Add a pig.cmd for Pig to run under Windows > -- > > Key: PIG-2959 > URL: https://issues.apache.org/jira/browse/PIG-2959 > Project: Pig > Issue Type: Sub-task >Reporter: Daniel Dai >Assignee: Daniel Dai > Fix For: 0.11 > > Attachments: pig.cmd > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2960) Increase the timeout for unit test
[ https://issues.apache.org/jira/browse/PIG-2960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472781#comment-13472781 ] Dmitriy V. Ryaboy commented on PIG-2960: +1 but i really wish we could just speed up the tests :-/ > Increase the timeout for unit test > -- > > Key: PIG-2960 > URL: https://issues.apache.org/jira/browse/PIG-2960 > Project: Pig > Issue Type: Sub-task >Reporter: Daniel Dai >Assignee: Daniel Dai > Fix For: 0.11 > > Attachments: PIG-2960-1.patch > > > Unit test under Windows is slower, some of unit tests cannot finish in time. > Increase timeout. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2941) Ivy resolvers in pig don't have consistent chaining and don't have a kitchen sink option for novices
[ https://issues.apache.org/jira/browse/PIG-2941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gianmarco De Francisci Morales updated PIG-2941: Assignee: John Gordon Status: Open (was: Patch Available) Can you regenerate the patch with --no-prefix option and without the email headers? Also, can you explain why maven2 is both on internal and external resolver? Canceling patch for the moment. > Ivy resolvers in pig don't have consistent chaining and don't have a kitchen > sink option for novices > > > Key: PIG-2941 > URL: https://issues.apache.org/jira/browse/PIG-2941 > Project: Pig > Issue Type: Bug > Components: build >Affects Versions: 0.10.0 >Reporter: John Gordon >Assignee: John Gordon > Fix For: 0.10.0 > > Attachments: > 0001-IvySettings.xml-refactor-to-simplify-resolution.patch > > > The Ivy resolvers in Pig are split into default, external, and internal -- > and they are all actually distinct. There isn't a resolver that rolls over > all three, and fallbacks aren't in place. Ideally, these resolver should > chain right through with the default following a best practice fallback for > novices. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2950) Fix tiny documentation error in BagToString builtin.
[ https://issues.apache.org/jira/browse/PIG-2950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Reisman updated PIG-2950: - Attachment: PIG-2950-2.patch Small fix: as long as we're repairing docs, lets use proper sentence structure, etc. > Fix tiny documentation error in BagToString builtin. > > > Key: PIG-2950 > URL: https://issues.apache.org/jira/browse/PIG-2950 > Project: Pig > Issue Type: Bug > Components: documentation >Reporter: Eli Reisman >Assignee: Eli Reisman >Priority: Trivial > Labels: newbie > Attachments: PIG-2950-1.patch, PIG-2950-2.patch > > > The default delimiter character for BagToString is not '-' as listed in the > docs but actually '_' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2910) Make toString() methods on Schema and FieldSchema be readable by Utils.getSchemaFromString()
[
https://issues.apache.org/jira/browse/PIG-2910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eli Reisman updated PIG-2910:
-
Attachment: PIG-2910-2.patch
This just removes the obligatory '(' and ')' in toString() call, was this more
what you had in mind? It passes 'ant compile-test'
I might throw up another version of the first patch too. If this is a common
workflow (passing the schema through Properties) it might be nice to have?
> Make toString() methods on Schema and FieldSchema be readable by
> Utils.getSchemaFromString()
>
>
> Key: PIG-2910
> URL: https://issues.apache.org/jira/browse/PIG-2910
> Project: Pig
> Issue Type: Bug
> Components: impl, parser
>Affects Versions: 0.9.2, 0.10.0, 0.11, 0.10.1
>Reporter: Russell Jurney
>Assignee: Thejas M Nair
> Labels: newbie
> Fix For: 0.11, 0.10.1
>
> Attachments: PIG-2910-1.patch, PIG-2910-2.patch
>
>
> I want to toString() schemas and send them to the backend via UDFContext. At
> the moment this requires writing your own toString() method that
> Utils.getSchemaFromString() can read. Making a readable schema for the
> backend would be an improvement.
> I spoke with Thejas, who believes this is a bug. The workaround for the
> moment is, for example:
> String schemaString = inputSchema.toString().substring(1,
> inputSchema.toString().length() - 1);
> // Set the input schema for processing
> UDFContext context = UDFContext.getUDFContext();
> Properties udfProp = context.getUDFProperties(this.getClass());
> udfProp.setProperty("horton.json.udf.schema", schemaString);
> ...
> schema = Utils.getSchemaFromString(strSchema);
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
