Re: How to compile Flink against vendor-specific Hadoop libs?

Hi, the builds using maven are controlled by maven properties and build profiles. There are two profiles, one for hadoop1 and one for hadoop2. When the hadoop2 profile is enabled, the system will also build the YARN client. The hadoop1 profile is used by default. To enable the hadoop2 profile, set

[jira] [Created] (FLINK-1004) Add MapR and Hortonworks repositories to main pom file.

Robert Metzger created FLINK-1004: - Summary: Add MapR and Hortonworks repositories to main pom file. Key: FLINK-1004 URL: https://issues.apache.org/jira/browse/FLINK-1004 Project: Flink Issue

Intentional delta-iteration constraint or bug?

Hi, I know a similar problem to this was raised earlier last month from the archive (http://mail-archives.apache.org/mod_mbox/flink-dev/201406.mbox/browser). However, I am unable to see if this was ever solved. I am encountering the same problem "In the given plan, the solution set delta doe

[jira] [Created] (FLINK-1003) Spread out scheduling strategy

Till Rohrmann created FLINK-1003: Summary: Spread out scheduling strategy Key: FLINK-1003 URL: https://issues.apache.org/jira/browse/FLINK-1003 Project: Flink Issue Type: Improvement

Re: Future of the current runtime?

That is right, we think of Tez as an optional additional backend for the time being.

How to compile Flink against vendor-specific Hadoop libs?

Hi, I would like to compile Flink against a specific Hadoop distribution. What is the best way to do this? I saw there is already a special build profile defined for Cloudera. Would it make sense to introduce such a build profile for Hortonworks and MapR as well? Bests, Daniel

Re: Hardware Requirements

Hey Sebastian, did you already try to increase the number of buffers in accordance to Stephan's suggestion? The current defaults for the number and size of network buffers are 2048 and 32768 bytes, resulting in 64 MB of memory for the network buffers. Out of curiosity: on how many machines are

Re: Future of the current runtime?

Hi Daniel, To the best of my knowledge the Tez-based runtime will only be an option and the current one stays as default. Cheers, Marton On Sun, Jul 6, 2014 at 1:21 PM, Daniel Warneke wrote: > Hi guys, > > now that there are ongoing efforts to port Flink to Tez, what is the > future of the c

[jira] [Created] (FLINK-1002) Add CSVReader support to ignore multiple lines at the beginning of a document

Bastian Köcher created FLINK-1002: - Summary: Add CSVReader support to ignore multiple lines at the beginning of a document Key: FLINK-1002 URL: https://issues.apache.org/jira/browse/FLINK-1002 Project

Future of the current runtime?

Hi guys, now that there are ongoing efforts to port Flink to Tez, what is the future of the current runtime? In particular, there are quite a few Jira tickets related to bugs or improvements of the current runtime. Some call for major changes and require considerable work. Does it still make sens

Re: Draft for Project Website

I pushed the md files to the svn repo: https://svn.apache.org/repos/asf/incubator/flink The md files of the documentation are in the main git repostiory. The build.sh script from the svn assembles the md files from the different locations in the correct way. On Sun, Jul 6, 2014 at 2:32 AM, Henry