Re: Pig 0.13.0 release

2014-02-12 Thread Dmitriy Ryaboy
So I think we agree that we should branch 0.13 at this point, right?

(and possibly look at releasing an incremental bump to 12 or 10? I'm not
sure what should be included there but I support the general idea).


On Thu, Feb 6, 2014 at 2:22 PM, Koji Noguchi  wrote:

> > Releasing 0.13 and 0.10.1 is totally independent in my opinion
> >
> I should have referenced my previous request on including 0.10.1 on the
> top release page.
>http://www.mail-archive.com/dev@pig.apache.org/msg20629.html
>
> > By minor I meant 0.13
> > 0.10.1 is a bug fix release.
> > as in Major.Minor.BugFix
> >
> I see. Then I should have said,
>   "but I'd like to request we make BugFix releases more often."
>
> Thanks for correcting my mistake.
>
> Koji
>
>
> On Feb 6, 2014, at 5:05 PM, Julien Le Dem  wrote:
>
> > Releasing 0.13 and 0.10.1 is totally independent in my opinion.
> > It just takes the time of a committer that needs the release to happen
> to do it.
> >
> > By minor I meant 0.13
> > 0.10.1 is a bug fix release.
> > as in Major.Minor.BugFix
> >
> > Our Major version is still 0
> >
> > On Feb 6, 2014, at 1:43 PM, Koji Noguchi wrote:
> >
> >>> To add to the discussion, I think we should release more often, based
> on time elapsed rather than volume of change.
> >>>
> >> I don't have preference on the frequency, but I'd like to request we
> make minor releases more often.
> >>
> >> At this moment, stable pig release (to me) is still 0.10.1.
> >> 0.11.1 and 0.12.0 both have regression bug PIG-3492 that caused
> multiple production pig scripts in our clusters to fail randomly.
> >> (unless user is disabling ColumnMapKeyPrune)
> >>
> >> If releasing 0.13 means 0.10.1 gets kicked out from the front release
> list, I'd like to see minor release on 0.11 or 0.12 first.
> >>
> >> Koji
> >>
> >>
> >>
> >>
> >> On Feb 6, 2014, at 4:25 PM, Cheolsoo Park 
> >> wrote:
> >>
> >>> +1 to 0.13 release. Why not if someone is volunteering?
> >>>
> >>> On Thu, Feb 6, 2014 at 4:06 PM, Julien Le Dem 
> wrote:
> >>>
>  To add to the discussion, I think we should release more often, based
> on
>  time elapsed rather than volume of change.
>  The more often we release, the easier it is to release.
>  Also that makes it easier for contributors to use their own
> contributions
>  in official releases.
>  It is also probably a good idea to have a clean starting point before
>  merging the Tez branch
> 
>  That said, I think those changes by themselves are enough to warrant a
>  minor release.
> 
>  Julien
> 
>  On Feb 6, 2014, at 12:24 PM, Dmitriy Ryaboy wrote:
> 
> > Major updates since we release 12 that are currently in trunk:
> >
> > - lazy output (don't generate empty part files)
> > - jar caching optimization
> > - automatic local mode for small job (big wall-clock wins for
> long-tail
> > jobs)
> > - improved support for BigInteger, BigDecimal
> > - hbase loader improvements
> > - debug mode that leaves temp files around for examination (!)
> > - fixes to a few nasty bugs (PIG-3641)
> > - pluggable execution engine allowing work like Tez and Spork
> > - .. and more
> >
> > I'd say this justifies a release.
> >
> > D
> >
> >
> > On Wed, Feb 5, 2014 at 3:55 PM, Aniket Mokashi 
>  wrote:
> >
> >> List I mentioned is pending tasks before we can make a release.
> >>
> >> A complete list of contributions can be seen at -
> >> http://svn.apache.org/viewvc/pig/trunk/CHANGES.txt?view=markup.
> >>
> >> Some of the things that make it a good candidate for a release-
> >> - PIG-3419 (has several backwards incompatible api changes)
> >> - PIG-2672
> >> - PIG-3642
> >> - PIG-3463
> >> - PIG-3511
> >> - PIG-3657
> >>
> >> Thanks,
> >> Aniket
> >>
> >>
> >>
> >> On Wed, Feb 5, 2014 at 3:23 PM, Olga Natkovich <
> onatkov...@yahoo.com>
> >> wrote:
> >>
> >>> Just going by the list that Aniket provided, I don't really see
> enough
> >> for
> >>> a full release. Two mentioned JIRAs are doc updates and one is a
> bug
>  fix
> >>> that was ported into Pig 12.
> >>>
> >>>
> >>>
> >>> On Wednesday, February 5, 2014 3:13 PM, Aniket Mokashi <
> >>> aniket...@gmail.com> wrote:
> >>>
> >>> Hi All,
> >>>
> >>> A good number of improvements and bug fixes have gone into trunk
> >> recently.
> >>> I'd like to know if we can roll out a Pig 0.13 release around
>  mid-March?
> >>>
> >>> I am aware that we are planning to merge tez branch into trunk
> soon.
> >>> However, making a release before tez branch is merged will be
> good. Any
> >>> objections?
> >>>
> >>> Following are few jiras we need to wrap up before 0.13 release-
> >>> PIG-3591
> >>> PIG-3740
> >>> PIG-3745
> >>> PIG-3347
> >>> PIG-3731
> >>> Any other?
> >>>
> >>> Th

[jira] Subscription: PIG patch available

2014-02-12 Thread jira
Issue Subscription
Filter: PIG patch available (12 issues)

Subscriber: pigdaily

Key Summary
PIG-3761Outer join fail on tez
https://issues.apache.org/jira/browse/PIG-3761
PIG-3759CombinerOptimizer should not optimize cogroup case in tez
https://issues.apache.org/jira/browse/PIG-3759
PIG-3758Fix desc order by in Tez
https://issues.apache.org/jira/browse/PIG-3758
PIG-3737Bundle dependent jars in distribution in %PIG_HOME%/lib folder
https://issues.apache.org/jira/browse/PIG-3737
PIG-3735UDF to data cleanse the dirty data with expected pattern
https://issues.apache.org/jira/browse/PIG-3735
PIG-3668COR built-in function when atleast one of the coefficient values is 
NaN
https://issues.apache.org/jira/browse/PIG-3668
PIG-3635Fix e2e tests for Hadoop 2.X on Windows
https://issues.apache.org/jira/browse/PIG-3635
PIG-3613UDF for SimilarityMatching between strings with matching scores
https://issues.apache.org/jira/browse/PIG-3613
PIG-3587add functionality for rolling over dates
https://issues.apache.org/jira/browse/PIG-3587
PIG-3456Reduce threadlocal conf access in backend for each record
https://issues.apache.org/jira/browse/PIG-3456
PIG-3441Allow Pig to use default resources from Configuration objects
https://issues.apache.org/jira/browse/PIG-3441
PIG-3373XMLLoader returns non-matching nodes when a tag name spans through 
the block boundary
https://issues.apache.org/jira/browse/PIG-3373

You may edit this subscription at:
https://issues.apache.org/jira/secure/FilterSubscription!default.jspa?subId=13225&filterId=12322384


Re: GSOC 2014

2014-02-12 Thread Russell Jurney
Suggestion: ON ERROR: https://wiki.apache.org/pig/PigErrorHandlingInScripts


On Wed, Feb 12, 2014 at 1:26 PM, Aniket Mokashi  wrote:

> I have a small prototype of https://issues.apache.org/jira/browse/PIG-2784
> .
> I am available for mentoring this year.
>
> ~Aniket
>
>
> On Wed, Feb 12, 2014 at 11:58 AM, Julien Le Dem  wrote:
>
> > I created a jira and added GSOC2014 label to PIG-2599
> >
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%2012310730%20AND%20labels%20%3D%20gsoc2014
> >
> > On Feb 12, 2014, at 11:21 AM, Daniel Dai wrote:
> >
> > > We need to create Jira and label it to make a GSOC candidate project.
> > > We have PIG-2599 for mavenize, can we create Jira for others? Who will
> > > be available for mentoring this year?
> > >
> > > On Tue, Feb 11, 2014 at 11:56 AM, Rohini Palaniswamy
> > >  wrote:
> > >> We can come up with a set of tasks on Tez - Dynamic reducer
> > configuration
> > >> for example.
> > >>
> > >> - Rohini
> > >>
> > >> On Tue, Feb 11, 2014 at 11:49 AM, Julien Le Dem 
> > wrote:
> > >>
> > >>> Some project ideas:
> > >>>  - mavenize Pig (I know... but still, I think it should be done)
> > >>>  - Compile physical operators to bytecode. I have a prototype, that
> > could
> > >>> be made real by a student:
> > >>>
> > https://github.com/julienledem/pig/compare/trunk...compile_physical_plan
> > >>>
> > >>> Julien
> > >>>
> > >>> On Feb 11, 2014, at 9:42 AM, Daniel Dai wrote:
> > >>>
> >  Any committer interested in mentoring? Any project ideas? We need to
> >  make project description ready by 2/24.
> > 
> >  Thanks,
> >  Daniel
> > 
> >  --
> >  CONFIDENTIALITY NOTICE
> >  NOTICE: This message is intended for the use of the individual or
> > entity
> > >>> to
> >  which it is addressed and may contain information that is
> > confidential,
> >  privileged and exempt from disclosure under applicable law. If the
> > reader
> >  of this message is not the intended recipient, you are hereby
> notified
> > >>> that
> >  any printing, copying, dissemination, distribution, disclosure or
> >  forwarding of this communication is strictly prohibited. If you have
> >  received this communication in error, please contact the sender
> > >>> immediately
> >  and delete it from your system. Thank You.
> > >>>
> > >>>
> > >
> > > --
> > > CONFIDENTIALITY NOTICE
> > > NOTICE: This message is intended for the use of the individual or
> entity
> > to
> > > which it is addressed and may contain information that is confidential,
> > > privileged and exempt from disclosure under applicable law. If the
> reader
> > > of this message is not the intended recipient, you are hereby notified
> > that
> > > any printing, copying, dissemination, distribution, disclosure or
> > > forwarding of this communication is strictly prohibited. If you have
> > > received this communication in error, please contact the sender
> > immediately
> > > and delete it from your system. Thank You.
> >
> >
>
>
> --
> "...:::Aniket:::... Quetzalco@tl"
>



-- 
Russell Jurney twitter.com/rjurney russell.jur...@gmail.com datasyndrome.com


[jira] [Updated] (PIG-2784) Framework for dynamic query optimization

2014-02-12 Thread Aniket Mokashi (JIRA)

 [ 
https://issues.apache.org/jira/browse/PIG-2784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aniket Mokashi updated PIG-2784:


Labels: GSOC2014  (was: )

> Framework for dynamic query optimization
> 
>
> Key: PIG-2784
> URL: https://issues.apache.org/jira/browse/PIG-2784
> Project: Pig
>  Issue Type: New Feature
>Reporter: Jie Li
>Assignee: Aniket Mokashi
>  Labels: GSOC2014
>
> We need a framework to implement dynamic query optimization, i.e. changing 
> the query plan at runtime. Currently we support estimating the number of 
> reducers dynamically, which works well as the first step but was not 
> perfectly implemented. In near future, we'll support more dynamic 
> optimization, like [removing sample job for 
> order-by|https://issues.apache.org/jira/browse/PIG-483], [removing limit 
> job|https://issues.apache.org/jira/browse/PIG-2675], dynamically detecting 
> skew and using skew-join, etc.
> Currently estimating #reducer is implemented in JobControlCompiler after 
> MRCompiler compiles all the MapReduceOperators and generate the complete 
> MRPlan. One place (discussed with Thejas) to implement the framework is at 
> the MRCompiler, where the MRPlan'll be generated at batches and adjusted 
> dynamically. 
> Any comment?



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)


Re: GSOC 2014

2014-02-12 Thread Aniket Mokashi
I have a small prototype of https://issues.apache.org/jira/browse/PIG-2784.
I am available for mentoring this year.

~Aniket


On Wed, Feb 12, 2014 at 11:58 AM, Julien Le Dem  wrote:

> I created a jira and added GSOC2014 label to PIG-2599
>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%2012310730%20AND%20labels%20%3D%20gsoc2014
>
> On Feb 12, 2014, at 11:21 AM, Daniel Dai wrote:
>
> > We need to create Jira and label it to make a GSOC candidate project.
> > We have PIG-2599 for mavenize, can we create Jira for others? Who will
> > be available for mentoring this year?
> >
> > On Tue, Feb 11, 2014 at 11:56 AM, Rohini Palaniswamy
> >  wrote:
> >> We can come up with a set of tasks on Tez - Dynamic reducer
> configuration
> >> for example.
> >>
> >> - Rohini
> >>
> >> On Tue, Feb 11, 2014 at 11:49 AM, Julien Le Dem 
> wrote:
> >>
> >>> Some project ideas:
> >>>  - mavenize Pig (I know... but still, I think it should be done)
> >>>  - Compile physical operators to bytecode. I have a prototype, that
> could
> >>> be made real by a student:
> >>>
> https://github.com/julienledem/pig/compare/trunk...compile_physical_plan
> >>>
> >>> Julien
> >>>
> >>> On Feb 11, 2014, at 9:42 AM, Daniel Dai wrote:
> >>>
>  Any committer interested in mentoring? Any project ideas? We need to
>  make project description ready by 2/24.
> 
>  Thanks,
>  Daniel
> 
>  --
>  CONFIDENTIALITY NOTICE
>  NOTICE: This message is intended for the use of the individual or
> entity
> >>> to
>  which it is addressed and may contain information that is
> confidential,
>  privileged and exempt from disclosure under applicable law. If the
> reader
>  of this message is not the intended recipient, you are hereby notified
> >>> that
>  any printing, copying, dissemination, distribution, disclosure or
>  forwarding of this communication is strictly prohibited. If you have
>  received this communication in error, please contact the sender
> >>> immediately
>  and delete it from your system. Thank You.
> >>>
> >>>
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
>
>


-- 
"...:::Aniket:::... Quetzalco@tl"


[jira] [Updated] (PIG-2599) Mavenize Pig

2014-02-12 Thread Julien Le Dem (JIRA)

 [ 
https://issues.apache.org/jira/browse/PIG-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julien Le Dem updated PIG-2599:
---

Labels: gsoc2014  (was: gsoc2013)

> Mavenize Pig
> 
>
> Key: PIG-2599
> URL: https://issues.apache.org/jira/browse/PIG-2599
> Project: Pig
>  Issue Type: New Feature
>  Components: build
>Reporter: Daniel Dai
>  Labels: gsoc2014
> Fix For: 0.13.0
>
> Attachments: maven-pig.1.zip
>
>
> Switch Pig build system from ant to maven.
> This is a candidate project for Google summer of code 2013. More information 
> about the program can be found at 
> https://cwiki.apache.org/confluence/display/PIG/GSoc2013



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)


Re: GSOC 2014

2014-02-12 Thread Julien Le Dem
I created a jira and added GSOC2014 label to PIG-2599
https://issues.apache.org/jira/issues/?jql=project%20%3D%2012310730%20AND%20labels%20%3D%20gsoc2014

On Feb 12, 2014, at 11:21 AM, Daniel Dai wrote:

> We need to create Jira and label it to make a GSOC candidate project.
> We have PIG-2599 for mavenize, can we create Jira for others? Who will
> be available for mentoring this year?
> 
> On Tue, Feb 11, 2014 at 11:56 AM, Rohini Palaniswamy
>  wrote:
>> We can come up with a set of tasks on Tez - Dynamic reducer configuration
>> for example.
>> 
>> - Rohini
>> 
>> On Tue, Feb 11, 2014 at 11:49 AM, Julien Le Dem  wrote:
>> 
>>> Some project ideas:
>>>  - mavenize Pig (I know... but still, I think it should be done)
>>>  - Compile physical operators to bytecode. I have a prototype, that could
>>> be made real by a student:
>>> https://github.com/julienledem/pig/compare/trunk...compile_physical_plan
>>> 
>>> Julien
>>> 
>>> On Feb 11, 2014, at 9:42 AM, Daniel Dai wrote:
>>> 
 Any committer interested in mentoring? Any project ideas? We need to
 make project description ready by 2/24.
 
 Thanks,
 Daniel
 
 --
 CONFIDENTIALITY NOTICE
 NOTICE: This message is intended for the use of the individual or entity
>>> to
 which it is addressed and may contain information that is confidential,
 privileged and exempt from disclosure under applicable law. If the reader
 of this message is not the intended recipient, you are hereby notified
>>> that
 any printing, copying, dissemination, distribution, disclosure or
 forwarding of this communication is strictly prohibited. If you have
 received this communication in error, please contact the sender
>>> immediately
 and delete it from your system. Thank You.
>>> 
>>> 
> 
> -- 
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to 
> which it is addressed and may contain information that is confidential, 
> privileged and exempt from disclosure under applicable law. If the reader 
> of this message is not the intended recipient, you are hereby notified that 
> any printing, copying, dissemination, distribution, disclosure or 
> forwarding of this communication is strictly prohibited. If you have 
> received this communication in error, please contact the sender immediately 
> and delete it from your system. Thank You.



[jira] [Created] (PIG-3764) Compile physical operators to bytecode

2014-02-12 Thread Julien Le Dem (JIRA)
Julien Le Dem created PIG-3764:
--

 Summary: Compile physical operators to bytecode
 Key: PIG-3764
 URL: https://issues.apache.org/jira/browse/PIG-3764
 Project: Pig
  Issue Type: Improvement
  Components: impl
Reporter: Julien Le Dem


I started a prototype here:
https://github.com/julienledem/pig/compare/trunk...compile_physical_plan

The current physical plan is relatively inefficient at evaluating expressions.
In the context of a better execution engine (Tez, Spark, ...), compiling 
expressions to bytecode would be a significant speedup.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)


[jira] [Assigned] (PIG-3446) Umbrella jira for Pig on Tez

2014-02-12 Thread Mark Wagner (JIRA)

 [ 
https://issues.apache.org/jira/browse/PIG-3446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Wagner reassigned PIG-3446:


Assignee: Mark Wagner  (was: Cheolsoo Park)

> Umbrella jira for Pig on Tez
> 
>
> Key: PIG-3446
> URL: https://issues.apache.org/jira/browse/PIG-3446
> Project: Pig
>  Issue Type: New Feature
>  Components: tez
>Affects Versions: tez-branch
>Reporter: Cheolsoo Park
>Assignee: Mark Wagner
> Fix For: tez-branch
>
>
> This is a umbrella jira for Pig on Tez. More detailed subtasks will be added.
> More information can be found on the following wiki page:
> https://cwiki.apache.org/confluence/display/PIG/Pig+on+Tez
> How to set up your development environment- 
> # Check out [Tez trunk|https://github.com/apache/incubator-tez].
> # Install protobuf 2.5.0.
> # Build Tez with Hadoop 2.2.0.(By default, it builds against Hadoop trunk, 
> which is 3.0.0.)
> # Install Tez jars on local maven repository with "mvn install -DskipTests".
> # Check out [Pig Tez branch|https://github.com/apache/pig/tree/tez].
> # Build Pig running "ant jar-withouthadoop".
> # Set up a single-node (or multi-node) Hadoop 2.2 cluster.
> # Install Tez following the instructions on the [Tez 
> homepage|http://tez.incubator.apache.org/install.html].
> # Run Pig with "-x tez" option.
> How to run Tez tests-
> * unit test
> {code}
> ant test-tez
> {code}
> By default, exectype is tez, and hadoopversion is 23 in tez branch. But you 
> can run unit tests in mr mode as follows:
> {code}
> ant test -Dexectype=mr -Dhadoopversion=20
> {code}
> * e2e tests
> {code}
> ant -Dharness.old.pig=$PIG_HOME -Dharness.hadoop.home=$HADOOP_HOME 
> -Dharness.cluster.conf=$HADOOP_CONF -Dharness.cluster.bin=$HADOOP_BIN 
> test-e2e-tez -Dhadoopversion=23
> {code}



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)


[jira] [Assigned] (PIG-3446) Umbrella jira for Pig on Tez

2014-02-12 Thread Mark Wagner (JIRA)

 [ 
https://issues.apache.org/jira/browse/PIG-3446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Wagner reassigned PIG-3446:


Assignee: Cheolsoo Park  (was: Mark Wagner)

> Umbrella jira for Pig on Tez
> 
>
> Key: PIG-3446
> URL: https://issues.apache.org/jira/browse/PIG-3446
> Project: Pig
>  Issue Type: New Feature
>  Components: tez
>Affects Versions: tez-branch
>Reporter: Cheolsoo Park
>Assignee: Cheolsoo Park
> Fix For: tez-branch
>
>
> This is a umbrella jira for Pig on Tez. More detailed subtasks will be added.
> More information can be found on the following wiki page:
> https://cwiki.apache.org/confluence/display/PIG/Pig+on+Tez
> How to set up your development environment- 
> # Check out [Tez trunk|https://github.com/apache/incubator-tez].
> # Install protobuf 2.5.0.
> # Build Tez with Hadoop 2.2.0.(By default, it builds against Hadoop trunk, 
> which is 3.0.0.)
> # Install Tez jars on local maven repository with "mvn install -DskipTests".
> # Check out [Pig Tez branch|https://github.com/apache/pig/tree/tez].
> # Build Pig running "ant jar-withouthadoop".
> # Set up a single-node (or multi-node) Hadoop 2.2 cluster.
> # Install Tez following the instructions on the [Tez 
> homepage|http://tez.incubator.apache.org/install.html].
> # Run Pig with "-x tez" option.
> How to run Tez tests-
> * unit test
> {code}
> ant test-tez
> {code}
> By default, exectype is tez, and hadoopversion is 23 in tez branch. But you 
> can run unit tests in mr mode as follows:
> {code}
> ant test -Dexectype=mr -Dhadoopversion=20
> {code}
> * e2e tests
> {code}
> ant -Dharness.old.pig=$PIG_HOME -Dharness.hadoop.home=$HADOOP_HOME 
> -Dharness.cluster.conf=$HADOOP_CONF -Dharness.cluster.bin=$HADOOP_BIN 
> test-e2e-tez -Dhadoopversion=23
> {code}



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)


[jira] [Updated] (PIG-3762) Properties aren't propagated to edges or vertices in Tez

2014-02-12 Thread Mark Wagner (JIRA)

 [ 
https://issues.apache.org/jira/browse/PIG-3762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Wagner updated PIG-3762:
-

Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Properties aren't propagated to edges or vertices in Tez
> 
>
> Key: PIG-3762
> URL: https://issues.apache.org/jira/browse/PIG-3762
> Project: Pig
>  Issue Type: Sub-task
>Reporter: Mark Wagner
>Assignee: Mark Wagner
> Attachments: PIG-3762.1.patch, PIG-3762.2.patch
>
>
> In TezDagBuilder, we are creating new Confs for the edges and vertices 
> without considering properties that have been set. The effect of this is that 
> you can't set tuning parameters, e.g. sort buffer size.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)


[jira] [Updated] (PIG-3762) Properties aren't propagated to edges or vertices in Tez

2014-02-12 Thread Mark Wagner (JIRA)

 [ 
https://issues.apache.org/jira/browse/PIG-3762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Wagner updated PIG-3762:
-

Attachment: PIG-3762.2.patch

This is the final patch that was committed. Thanks to Rohini for the review!

> Properties aren't propagated to edges or vertices in Tez
> 
>
> Key: PIG-3762
> URL: https://issues.apache.org/jira/browse/PIG-3762
> Project: Pig
>  Issue Type: Sub-task
>Reporter: Mark Wagner
>Assignee: Mark Wagner
> Attachments: PIG-3762.1.patch, PIG-3762.2.patch
>
>
> In TezDagBuilder, we are creating new Confs for the edges and vertices 
> without considering properties that have been set. The effect of this is that 
> you can't set tuning parameters, e.g. sort buffer size.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)


Re: GSOC 2014

2014-02-12 Thread Daniel Dai
We need to create Jira and label it to make a GSOC candidate project.
We have PIG-2599 for mavenize, can we create Jira for others? Who will
be available for mentoring this year?

On Tue, Feb 11, 2014 at 11:56 AM, Rohini Palaniswamy
 wrote:
> We can come up with a set of tasks on Tez - Dynamic reducer configuration
> for example.
>
> - Rohini
>
> On Tue, Feb 11, 2014 at 11:49 AM, Julien Le Dem  wrote:
>
>> Some project ideas:
>>   - mavenize Pig (I know... but still, I think it should be done)
>>   - Compile physical operators to bytecode. I have a prototype, that could
>> be made real by a student:
>> https://github.com/julienledem/pig/compare/trunk...compile_physical_plan
>>
>> Julien
>>
>> On Feb 11, 2014, at 9:42 AM, Daniel Dai wrote:
>>
>> > Any committer interested in mentoring? Any project ideas? We need to
>> > make project description ready by 2/24.
>> >
>> > Thanks,
>> > Daniel
>> >
>> > --
>> > CONFIDENTIALITY NOTICE
>> > NOTICE: This message is intended for the use of the individual or entity
>> to
>> > which it is addressed and may contain information that is confidential,
>> > privileged and exempt from disclosure under applicable law. If the reader
>> > of this message is not the intended recipient, you are hereby notified
>> that
>> > any printing, copying, dissemination, distribution, disclosure or
>> > forwarding of this communication is strictly prohibited. If you have
>> > received this communication in error, please contact the sender
>> immediately
>> > and delete it from your system. Thank You.
>>
>>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.


[jira] [Commented] (PIG-3757) Make scalar work

2014-02-12 Thread Daniel Dai (JIRA)

[ 
https://issues.apache.org/jira/browse/PIG-3757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13899315#comment-13899315
 ] 

Daniel Dai commented on PIG-3757:
-

Please hold on reviewing. I probably want to fix in a different way.

> Make scalar work
> 
>
> Key: PIG-3757
> URL: https://issues.apache.org/jira/browse/PIG-3757
> Project: Pig
>  Issue Type: Sub-task
>  Components: tez
>Reporter: Daniel Dai
>Assignee: Daniel Dai
> Fix For: tez-branch
>
> Attachments: PIG-3757-1.patch, PIG-3757-2.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.1.5#6160)