Re: ASF Board Meeting Summary - December 17, 2014

2014-12-17 Thread Flavio Pompermaier
Congratulations!! On Dec 17, 2014 10:07 PM, "Fabian Hueske" wrote: > Great news! :-) > > 2014-12-17 21:54 GMT+01:00 Robert Metzger : > > > > Thank you for putting it to the mailing list! > > > > I'm super excited. > > > > On Wed, Dec 17, 2014 at 9:49 PM, Alan Gates > wrote: > > > > > > Congratul

Re: [VOTE] Graduate Flink from the Incubator

2014-12-05 Thread Flavio Pompermaier
So you have my +1 :) On Fri, Dec 5, 2014 at 11:46 AM, Kostas Tzoumas wrote: > Hi everyone, > > It seems that everyone is excited about graduating, so I am calling for a > community vote to graduate Apache Flink (incubating) to a top-level project > (TLP). If this VOTE proceeds successfully, I wi

Re: InputFormat API and current scanned row count

2014-12-02 Thread Flavio Pompermaier
> > Do you want to count the number of produced tuples for monitoring the > progress or do you see a different use case? > > 2014-11-28 9:37 GMT+01:00 Flavio Pompermaier : > > > Hi guys, > > > > I was debugging an inputFormat and I discovered that there's no

InputFormat API and current scanned row count

2014-11-28 Thread Flavio Pompermaier
Hi guys, I was debugging an inputFormat and I discovered that there's no way to understand how many records have been processed in a split. So I added a counter in my input format incremented every nextRecord..do you think adding something to similar like "public int getProcessedRecordsCount()" to

Re: Heartbeat lost

2014-11-18 Thread Flavio Pompermaier
Have you evaluated to adopt reactor instead of akka? On Nov 18, 2014 10:57 AM, "Stephan Ewen" wrote: > Yes, that sounds like a good idea. > > I have experienced that occasionally before, under high parallelism and > algorithms where the task manager got long garbage collection stalls... > > The d

Re: HBase 0.98 addon for Flink 0.8

2014-11-14 Thread Flavio Pompermaier
probably I can't test that before Tuesday..I think we can talk about that at the Berlin meeting if you cannot debug it :/ On Nov 14, 2014 11:41 PM, "Stephan Ewen" wrote: > Hi! > > I still believe it is an issue of the class loader. To test that, can you > remove the HBase jar file from the Flink

Re: HBase 0.98 addon for Flink 0.8

2014-11-14 Thread Flavio Pompermaier
2014 9:26 PM, "Fabian Hueske" wrote: > What exactly is required to configure the TableInputFormat? > Would it be easier and more flexible to just set the hostname of the HBase > master, the table name, etc, directly as strings in the InputFormat? > > 2014-11-14

Re: HBase 0.98 addon for Flink 0.8

2014-11-14 Thread Flavio Pompermaier
our job for execution? > > 2014-11-14 13:58 GMT+01:00 Flavio Pompermaier : > > > The strange thing us that everything works if I create HTable outside > > configure().. > > On Nov 14, 2014 10:32 AM, "Stephan Ewen" wrote: > > > > > I think that this

Re: HBase 0.98 addon for Flink 0.8

2014-11-14 Thread Flavio Pompermaier
the configuration yourself and give that configuration > to HBase? > > Stephan > Am 13.11.2014 22:06 schrieb "Flavio Pompermaier" : > > > The only config files available are within the submitted jar. Things > works > > in eclipse using local environment while fai

Re: HBase 0.98 addon for Flink 0.8

2014-11-13 Thread Flavio Pompermaier
ob jar file? Or is simply no config at all > available when the configure method is called? > > > > > > > -- > Fabian Hueske > Phone: +49 170 5549438 > Email: fhue...@gmail.com > Web: http://www.user.tu-berlin.de/fabian.hueske > > > >

Re: HBase 0.98 addon for Flink 0.8

2014-11-13 Thread Flavio Pompermaier
is it bundled with > your job.jar file? > > > > > > -- > Fabian Hueske > Phone: +49 170 5549438 > Email: fhue...@gmail.com > Web: http://www.user.tu-berlin.de/fabian.hueske > > > > > > From: Flavio Pompermaier >

Re: HBase 0.98 addon for Flink 0.8

2014-11-13 Thread Flavio Pompermaier
Any help with this? :( On Thu, Nov 13, 2014 at 2:06 PM, Flavio Pompermaier wrote: > We definitely discovered that instantiating HTable and Scan in configure() > method of TableInputFormat causes problem in distributed environment! > If you look at my implementation at > https:

Re: HBase 0.98 addon for Flink 0.8

2014-11-13 Thread Flavio Pompermaier
/addons/hbase/TableInputFormat.java you can see that Scan and HTable were made transient and recreated within configure but this causes HBaseConfiguration.create() to fail searching for classpath files...could you help us understanding why? On Wed, Nov 12, 2014 at 8:10 PM, Flavio Pompermaier wrote

Re: HBase 0.98 addon for Flink 0.8

2014-11-12 Thread Flavio Pompermaier
classpath. Maybe is the hbase-site.xml > file not accessible through the classpath when running on the cluster? > > On Wed, Nov 12, 2014 at 7:40 PM, Flavio Pompermaier > wrote: > > > Today we tried tp execute a job on the cluster instead of on local > executor > > and

Re: HBase 0.98 addon for Flink 0.8

2014-11-12 Thread Flavio Pompermaier
rrently working on that and said he expects this to be done by > end of the week. > > Cheers, Fabian > > > 2014-11-07 12:49 GMT+01:00 Flavio Pompermaier : > > > I fixed also the profile for Cloudera CDH5.1.3. You can build it with the > > command: > > mvn c

Re: [jira] [Created] (FLINK-1228) Add REST Interface to JobManager

2014-11-10 Thread Flavio Pompermaier
That would be very useful! On Mon, Nov 10, 2014 at 3:25 PM, Arvid Heise (JIRA) wrote: > Arvid Heise created FLINK-1228: > -- > > Summary: Add REST Interface to JobManager > Key: FLINK-1228 > URL: https://issues.apache

Re: Flink on Tez

2014-11-09 Thread Flavio Pompermaier
than Flink-on-Tez. > > On Saturday, November 8, 2014, Flavio Pompermaier > wrote: > > > Sorry for the newbie question but for me it's not clear when users > > should switch from flink runtime to tez, > > they seems to do the same thing right? > > > > On

Re: Flink on Tez

2014-11-08 Thread Flavio Pompermaier
Sorry for the newbie question but for me it's not clear when users should switch from flink runtime to tez, they seems to do the same thing right? On Sat, Nov 8, 2014 at 6:02 PM, Aljoscha Krettek wrote: > Yes, but the beauty of it is that Flink is designed in such a way that > we can switch unde

Re: HBase 0.98 addon for Flink 0.8

2014-11-07 Thread Flavio Pompermaier
-cdh5.1.3-incubating) Best, Flavio On Fri, Nov 7, 2014 at 12:44 PM, Flavio Pompermaier wrote: > I've just updated the code on my fork (synch with current master and > applied improvements coming from comments on related PR). > I still have to understand how to write results back to a

Re: HBase 0.98 addon for Flink 0.8

2014-11-07 Thread Flavio Pompermaier
I've just updated the code on my fork (synch with current master and applied improvements coming from comments on related PR). I still have to understand how to write results back to an HBase Sink/OutputFormat... On Mon, Nov 3, 2014 at 12:05 PM, Flavio Pompermaier wrote: > Thanks

Re: Blog post for release 0.7.0

2014-11-04 Thread Flavio Pompermaier
Now it works..probably there was a network error.. On Tue, Nov 4, 2014 at 1:30 PM, Flavio Pompermaier wrote: > This link is not working > On Nov 4, 2014 1:07 PM, "Kostas Tzoumas" wrote: > >> I updated the blog with a blog post for the 0.7.0 release: >> http://

Re: Blog post for release 0.7.0

2014-11-04 Thread Flavio Pompermaier
This link is not working On Nov 4, 2014 1:07 PM, "Kostas Tzoumas" wrote: > I updated the blog with a blog post for the 0.7.0 release: > http://flink.incubator.apache.org/news/2014/11/04/release-0.7.0.html > > Feel free to share! > > Let me know if you contributed and your name is missing. >

Re: HBase 0.98 addon for Flink 0.8

2014-11-03 Thread Flavio Pompermaier
bout the storage location of the region. During execution, > input splits are assigned to InputFormats that can do local reads. > > Best, Fabian > > 2014-11-03 11:13 GMT+01:00 Stephan Ewen : > > > Hi! > > > > The way of passing parameters through the configuration i

Re: HBase 0.98 addon for Flink 0.8

2014-11-03 Thread Flavio Pompermaier
3, 2014 at 10:11 AM, Flavio Pompermaier > wrote: > > > That is one class I removed because it was using the deprecated API > > GenericDataSink..I can restore them but the it will be a good idea to > > remove those warning (also because from what I understood the R

Re: HBase 0.98 addon for Flink 0.8

2014-11-03 Thread Flavio Pompermaier
: > I'm not familiar with the HBase connector code, but are you maybe looking > for the GenericTableOutputFormat? > > 2014-11-03 9:44 GMT+01:00 Flavio Pompermaier : > > > | was trying to modify the example setting hbaseDs.output(new > > HBaseOutputFormat()); but I can&#

Re: HBase 0.98 addon for Flink 0.8

2014-11-03 Thread Flavio Pompermaier
| was trying to modify the example setting hbaseDs.output(new HBaseOutputFormat()); but I can't see any HBaseOutputFormat class..maybe we shall use another class? On Mon, Nov 3, 2014 at 9:39 AM, Flavio Pompermaier wrote: > Maybe that's something I could add to the HBase example a

Re: HBase 0.98 addon for Flink 0.8

2014-11-03 Thread Flavio Pompermaier
ot working (where it says For a complete example program, have a look at KMeans Algorithm). Best, Flavio On Mon, Nov 3, 2014 at 9:12 AM, Flavio Pompermaier wrote: > Ah ok, perfect! That was the reason why I removed it :) > > On Mon, Nov 3, 2014 at 9:10 AM, Stephan Ewen wrote: > >

Re: HBase 0.98 addon for Flink 0.8

2014-11-03 Thread Flavio Pompermaier
Ah ok, perfect! That was the reason why I removed it :) On Mon, Nov 3, 2014 at 9:10 AM, Stephan Ewen wrote: > You do not really need a HBase data sink. You can call "DataSet.output(new > HBaseOutputFormat())" > > Stephan > Am 02.11.2014 23:05 schrieb "Flavio Pomp

Re: HBase 0.98 addon for Flink 0.8

2014-11-02 Thread Flavio Pompermaier
Just one last thing..I removed the HbaseDataSink because I think it was using the old APIs..can someone help me in updating that class? On Sun, Nov 2, 2014 at 10:55 AM, Flavio Pompermaier wrote: > Indeed this time the build has been successful :) > > On Sun, Nov 2, 2014 at 10:29 A

Re: HBase 0.98 addon for Flink 0.8

2014-11-02 Thread Flavio Pompermaier
th HBase. I do also not have a HBase setup available > here. > Maybe somebody else of the community who was involved with a previous > version of the HBase connector could comment on your question. > > Best, Fabian > > 2014-11-02 9:57 GMT+01:00 Flavio Pompermaier : > > > A

HBase 0.98 addon for Flink 0.8

2014-11-02 Thread Flavio Pompermaier
As suggestes by Fabian I moved the discussion on this mailing list. I think that what is still to be discussed is how to retrigger the build on Travis (I don't have an account) and if the PR can be integrated. Maybe what I can do is to move the HBase example in the test package (right now I left