Re: What is the limit to the number of properties set in the configuration object

2015-06-16 Thread Steve Loughran
they also get sent over the wire with things like job submissions, so can make things slower. in my little grumpy project, https://github.com/steveloughran/grumpy , I actually stuck the groovy scripts into the config files as strings, so they'd be submitted as jobs; the mapper & reducer would

Re: What is the limit to the number of properties set in the configuration object

2015-06-15 Thread Colin P. McCabe
Much like zombo.com, the only limit is yourself. But huge Configuration objects are going to be really inefficient, so I would look elsewhere for storing lots of data. best, Colin On Fri, Jun 12, 2015 at 7:30 PM, Sitaraman Vilayannur wrote: > Thanks Allen, what is the total size limit? > Sitara

Re: What is the limit to the number of properties set in the configuration object

2015-06-12 Thread Sitaraman Vilayannur
Thanks Allen, what is the total size limit? Sitaraman On Fri, Jun 12, 2015 at 10:53 PM, Allen Wittenauer wrote: > > On Jun 12, 2015, at 12:37 AM, Sitaraman Vilayannur < > vrsitaramanietfli...@gmail.com> wrote: > > > Hi, > > What is the limit on the number of properties that can be set using >

Re: What is the limit to the number of properties set in the configuration object

2015-06-12 Thread Allen Wittenauer
On Jun 12, 2015, at 12:37 AM, Sitaraman Vilayannur wrote: > Hi, > What is the limit on the number of properties that can be set using > set(String s1, String s2) on the Configuration object for hadoop? > Is this limit configurable if so what is the maximum that can be set? It's a "tot

What is the limit to the number of properties set in the configuration object

2015-06-12 Thread Sitaraman Vilayannur
Hi, What is the limit on the number of properties that can be set using set(String s1, String s2) on the Configuration object for hadoop? Is this limit configurable if so what is the maximum that can be set? Thanks Sitaraman