http://git-wip-us.apache.org/repos/asf/flink-web/blob/24f3ba5a/content/news/2014/02/18/amazon-elastic-mapreduce-cloud-yarn.html ---------------------------------------------------------------------- diff --git a/content/news/2014/02/18/amazon-elastic-mapreduce-cloud-yarn.html b/content/news/2014/02/18/amazon-elastic-mapreduce-cloud-yarn.html deleted file mode 100644 index 4616924..0000000 --- a/content/news/2014/02/18/amazon-elastic-mapreduce-cloud-yarn.html +++ /dev/null @@ -1,390 +0,0 @@ -<!DOCTYPE html> -<html lang="en"> - <head> - <meta charset="utf-8"> - <meta http-equiv="X-UA-Compatible" content="IE=edge"> - <meta name="viewport" content="width=device-width, initial-scale=1"> - <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags --> - <title>Apache Flink: Use Stratosphere with Amazon Elastic MapReduce</title> - <link rel="shortcut icon" href="/favicon.ico" type="image/x-icon"> - <link rel="icon" href="/favicon.ico" type="image/x-icon"> - - <!-- Bootstrap --> - <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.4/css/bootstrap.min.css";> - <link rel="stylesheet" href="/css/flink.css"> - <link rel="stylesheet" href="/css/syntax.css"> - - <!-- Blog RSS feed --> - <link href="/blog/feed.xml" rel="alternate" type="application/rss+xml" title="Apache Flink Blog: RSS feed" /> - - <!-- jQuery (necessary for Bootstrap's JavaScript plugins) --> - <!-- We need to load Jquery in the header for custom google analytics event tracking--> - <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.2/jquery.min.js";></script> - - <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries --> - <!-- WARNING: Respond.js doesn't work if you view the page via file:// --> - <!--[if lt IE 9]> - <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js";></script> - <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js";></script> - <![endif]--> - </head> - <body> - - - <!-- Top navbar. --> - <nav class="navbar navbar-default navbar-fixed-top"> - <div class="container"> - <!-- The logo. --> - <div class="navbar-header"> - <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#bs-example-navbar-collapse-1"> - <span class="icon-bar"></span> - <span class="icon-bar"></span> - <span class="icon-bar"></span> - </button> - <div class="navbar-logo"> - <a href="/"> - <img alt="Apache Flink" src="/img/navbar-brand-logo.jpg" width="78px" height="40px"> - </a> - </div> - </div><!-- /.navbar-header --> - - <!-- The navigation links. --> - <div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1"> - <ul class="nav navbar-nav"> - - <!-- Overview --> - <li><a href="/index.html">Overview</a></li> - - <!-- Features --> - <li><a href="/features.html">Features</a></li> - - <!-- Downloads --> - <li><a href="/downloads.html">Downloads</a></li> - - <!-- FAQ --> - <li><a href="/faq.html">FAQ</a></li> - - - <!-- Quickstart --> - <li class="dropdown"> - <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false"><small><span class="glyphicon glyphicon-new-window"></span></small> Quickstart <span class="caret"></span></a> - <ul class="dropdown-menu" role="menu"> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-release-1.0/quickstart/setup_quickstart.html";>Setup</a></li> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-release-1.0/quickstart/run_example_quickstart.html";>Example: Wikipedia Edit Stream</a></li> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-release-1.0/quickstart/java_api_quickstart.html";>Java API</a></li> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-release-1.0/quickstart/scala_api_quickstart.html";>Scala API</a></li> - </ul> - </li> - - <!-- Documentation --> - <li class="dropdown"> - <a href="" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false"><small><span class="glyphicon glyphicon-new-window"></span></small> Documentation <span class="caret"></span></a> - <ul class="dropdown-menu" role="menu"> - <!-- Latest stable release --> - <li role="presentation" class="dropdown-header"><strong>Latest Release</strong> (Stable)</li> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-release-1.0";>1.0 Documentation</a></li> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-release-1.0/api/java"; class="active">1.0 Javadocs</a></li> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-release-1.0/api/scala/index.html"; class="active">1.0 ScalaDocs</a></li> - - <!-- Snapshot docs --> - <li class="divider"></li> - <li role="presentation" class="dropdown-header"><strong>Snapshot</strong> (Development)</li> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-master";>1.1 Documentation</a></li> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-master/api/java"; class="active">1.1 Javadocs</a></li> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-master/api/scala/index.html"; class="active">1.1 ScalaDocs</a></li> - - <!-- Wiki --> - <li class="divider"></li> - <li><a href="/visualizer/"><small><span class="glyphicon glyphicon-new-window"></span></small> Plan Visualizer</a></li> - <li><a href="https://cwiki.apache.org/confluence/display/FLINK/Apache+Flink+Home";><small><span class="glyphicon glyphicon-new-window"></span></small> Wiki</a></li> - </ul> - </li> - - </ul> - - <ul class="nav navbar-nav navbar-right"> - <!-- Blog --> - <li class=" active hidden-md hidden-sm"><a href="/blog/">Blog</a></li> - - <li class="dropdown hidden-md hidden-sm"> - <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">Community <span class="caret"></span></a> - <ul class="dropdown-menu" role="menu"> - <!-- Community --> - <li role="presentation" class="dropdown-header"><strong>Community</strong></li> - <li><a href="/community.html#mailing-lists">Mailing Lists</a></li> - <li><a href="/community.html#irc">IRC</a></li> - <li><a href="/community.html#stack-overflow">Stack Overflow</a></li> - <li><a href="/community.html#issue-tracker">Issue Tracker</a></li> - <li><a href="/community.html#third-party-packages">Third Party Packages</a></li> - <li><a href="/community.html#source-code">Source Code</a></li> - <li><a href="/community.html#people">People</a></li> - <li><a href="https://cwiki.apache.org/confluence/display/FLINK/Powered+by+Flink";><small><span class="glyphicon glyphicon-new-window"></span></small> Powered by Flink</a></li> - - <!-- Contribute --> - <li class="divider"></li> - <li role="presentation" class="dropdown-header"><strong>Contribute</strong></li> - <li><a href="/how-to-contribute.html">How to Contribute</a></li> - <li><a href="/contribute-code.html">Contribute Code</a></li> - <li><a href="/contribute-documentation.html">Contribute Documentation</a></li> - <li><a href="/improve-website.html">Improve the Website</a></li> - </ul> - </li> - - <li class="dropdown hidden-md hidden-sm"> - <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">Project <span class="caret"></span></a> - <ul class="dropdown-menu" role="menu"> - <!-- Project --> - <li role="presentation" class="dropdown-header"><strong>Project</strong></li> - <li><a href="/slides.html">Slides</a></li> - <li><a href="/material.html">Material</a></li> - <li><a href="https://twitter.com/apacheflink";><small><span class="glyphicon glyphicon-new-window"></span></small> Twitter</a></li> - <li><a href="https://github.com/apache/flink";><small><span class="glyphicon glyphicon-new-window"></span></small> GitHub</a></li> - <li><a href="https://cwiki.apache.org/confluence/display/FLINK/Apache+Flink+Home";><small><span class="glyphicon glyphicon-new-window"></span></small> Wiki</a></li> - </ul> - </li> - </ul> - </div><!-- /.navbar-collapse --> - </div><!-- /.container --> - </nav> - - - <!-- Main content. --> - <div class="container"> - - -<div class="row"> - <div class="col-sm-8 col-sm-offset-2"> - <div class="row"> - <h1>Use Stratosphere with Amazon Elastic MapReduce</h1> - - <article> - <p>18 Feb 2014 by Robert Metzger (<a href="https://twitter.com/rmetzger_";>@rmetzger_</a>)</p> - -<div class="lead">Get started with Stratosphere within 10 minutes using Amazon Elastic MapReduce.</div> - -<p>This step-by-step tutorial will guide you through the setup of Stratosphere using Amazon Elastic MapReduce.</p> - -<h3 id="background">Background</h3> -<p><a href="http://aws.amazon.com/elasticmapreduce/";>Amazon Elastic MapReduce</a> (Amazon EMR) is part of Amazon Web services. EMR allows to create Hadoop clusters that analyze data stored in Amazon S3 (AWSâ cloud storage). Stratosphere runs on top of Hadoop using the <a href="http://hadoop.apache.org/docs/r2.2.0/hadoop-project-dist/hadoop-common/releasenotes.html";>recently</a> released cluster resource manager <a href="http://hadoop.apache.org/docs/current2/hadoop-yarn/hadoop-yarn-site/YARN.html";>YARN</a>. YARN allows to use many different data analysis tools in your cluster side by side. Tools that run with YARN are, for example <a href="https://giraph.apache.org/";>Apache Giraph</a>, <a href="http://spark.incubator.apache.org/";>Spark</a> or <a href="http://hortonworks.com/blog/introducing-hoya-hbase-on-yarn/";>HBase</a>. Stratosphere also <a href="/docs/0.4/setup/yarn.html">runs on YARN</a> and thatâs the approach for this tutorial.</p> - -<h3 id="step-login-to-aws-and-prepare-secure-access">1. Step: Login to AWS and prepare secure access</h3> - -<ul> - <li>Log in to the <a href="https://console.aws.amazon.com/console/home";>AWS Console</a></li> -</ul> - -<p>You need to have SSH keys to access the Hadoop master node. If you do not have keys for your computer, generate them:</p> - -<div class="row" style="padding-top:15px"> - <div class="col-md-6"> -<a data-lightbox="example-1" href="/img/blog/emr-security.png"><img class="img-responsive" src="/img/blog/emr-security.png" /></a> - </div> - <div class="col-md-6"> - <ul> - <li>Select <a href="https://console.aws.amazon.com/ec2/v2/home";>EC2</a> and click on "Key Pairs" in the "NETWORK & SECURITY" section.</li> - <li>Click on "Create Key Pair" and give it a name</li> - <li>After pressing "Yes" it will download a .pem file.</li> - <li>Change the permissions of the .pem file</li> - -<div class="highlight"><pre><code class="language-bash" data-lang="bash">chmod og-rwx ~/work-laptop.pem</code></pre></div> - - </ul> - </div> -</div> - -<h3 id="step-create-your-hadoop-cluster-in-the-cloud">2. Step: Create your Hadoop Cluster in the cloud</h3> - -<ul> - <li>Select <a href="https://console.aws.amazon.com/elasticmapreduce/vnext/";>Elastic MapReduce</a> from the AWS console</li> - <li>Click the blue âCreate clusterâ button.</li> -</ul> - -<div class="row" style="padding-top:15px"> - <div class="col-md-6"> -<a data-lightbox="example-1" href="/img/blog/emr-hadoopversion.png"><img class="img-responsive" src="/img/blog/emr-hadoopversion.png" /></a> - </div> - <div class="col-md-6"> - <ul> - <li>Choose a Cluster name</li> - <li>You can let the other settings remain unchanged (termination protection, logging, debugging)</li> - <li>For the Hadoop distribution, it is very important to choose one with YARN support. We use <b>3.0.3 (Hadoop 2.2.0)</b> (the minor version might change over time)</li> - <li>Remove all applications to be installed (unless you want to use them)</li> - <li>Choose the instance types you want to start. Stratosphere runs fine with m1.large instances. Core and Task instances both run Stratosphere, but only core instances contain HDFS data nodes.</li> - <li>Choose the <b>EC2 key pair</b> you've created in the previous step!</li> - </ul> - </div> -</div> - -<ul> - <li>Thats it! You can now press the âCreate clusterâ button at the end of the form to boot it!</li> -</ul> - -<h3 id="step-launch-stratosphere">3. Step: Launch Stratosphere</h3> - -<p>You might need to wait a few minutes until Amazon started your cluster. (You can monitor the progress of the instances in EC2). Use the refresh button in the top right corner.</p> - -<p>You see that the master is up if the field <b>Master public DNS</b> contains a value (first line), connect to it using SSH.</p> - -<div class="highlight"><pre><code class="language-bash" data-lang="bash">ssh hadoop@<your master public DNS> -i <path to your .pem> -<span class="c"># for my example, it looks like this:</span> -ssh had...@ec2-54-213-61-105.us-west-2.compute.amazonaws.com -i ~/Downloads/work-laptop.pem</code></pre></div> - -<p>(Windows users have to follow <a href="http://docs.aws.amazon.com/ElasticMapReduce/latest/DeveloperGuide/emr-connect-master-node-ssh.html";>these instructions</a> to SSH into the machine running the master.) </br></br> -Once connected to the master, download and start Stratosphere for YARN: </p> -<ul> - <li>Download and extract Stratosphere-YARN</li> - -<div class="highlight"><pre><code class="language-bash" data-lang="bash">wget http://stratosphere-bin.s3-website-us-east-1.amazonaws.com/stratosphere-dist-0.5-SNAPSHOT-yarn.tar.gz -<span class="c"># extract it</span> -tar xvzf stratosphere-dist-0.5-SNAPSHOT-yarn.tar.gz</code></pre></div> - - <li>Start Stratosphere in the cluster using Hadoop YARN</li> - - -<div class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">cd </span>stratosphere-yarn-0.5-SNAPSHOT/ -./bin/yarn-session.sh -n <span class="m">4</span> -jm <span class="m">1024</span> -tm 3000</code></pre></div> - - -The arguments have the following meaning - <ul> - <li><code>-n</code> number of TaskManagers (=workers). This number must not exeed the number of task instances</li> - <li><code>-jm</code> memory (heapspace) for the JobManager</li> - <li><code>-tm</code> memory for the TaskManagers</li> - </ul> -</ul> - -<p>Once the output has changed from </p> - -<div class="highlight"><pre><code class="language-bash" data-lang="bash">JobManager is now running on N/A:6123</code></pre></div> - -<p>to </p> - -<div class="highlight"><pre><code class="language-bash" data-lang="bash">JobManager is now running on ip-172-31-13-68.us-west-2.compute.internal:6123</code></pre></div> - -<p>Stratosphere has started the JobManager. It will take a few seconds until the TaskManagers (workers) have connected to the JobManager. To see how many TaskManagers connected, you have to access the JobManagerâs web interface. Follow the steps below to do that â¦</p> - -<h3> 4. Step: Launch a Stratosphere Job</h3> - -<p>This step shows how to submit and monitor a Stratosphere Job in the Amazon Cloud.</p> - -<ul> -<li> Open an additional terminal and connect again to the master of your cluster. </li> - -We recommend to create a SOCKS-proxy with your SSH that allows you to easily connect into the cluster. (If you've already a VPN setup with EC2, you can probably use that as well.) - - -<div class="highlight"><pre><code class="language-bash" data-lang="bash">ssh -D localhost:2001 hadoop@<your master dns name> -i <your pem file></code></pre></div> - - -Notice the <code>-D localhost:2001</code> argument: It opens a SOCKS proxy on your computer allowing any application to use it to communicate through the proxy via an SSH tunnel to the master node. This allows you to access all services in your EMR cluster, such as the HDFS NameNode or the YARN web interface. - -<li>Configure a browser to use the SOCKS proxy. Open a browser with SOCKS proxy support (such as Firefox). Ideally, do not use your primary browser for this, since ALL traffic will be routed through Amazon.</li> - -<div class="row" style="padding-top:15px"> - <div class="col-md-6"> -<a data-lightbox="example-1" href="/img/blog/emr-firefoxsettings.png"><img class="img-responsive" src="/img/blog/emr-firefoxsettings.png" /></a> - </div> - <div class="col-md-6"> - <ul> - <li>To configure the SOCKS proxy with Firefox, click on "Edit", "Preferences", choose the "Advanced" tab and press the "Settings ..." button.</li> - <li>Enter the details of the SOCKS proxy <b>localhost:2001</b>. Choose SOCKS v4.</li> - <li>Close the settings, your browser is now talking to the master node of your cluster</li> - </ul> - </div> -</div> - -</ul> - -<p>Since youâre connected to the master now, you can open several web interfaces: <br /> -<b>YARN Resource Manager</b>: <code>http://<masterIPAddress>:9026/</code> <br /> -<b>HDFS NameNode</b>: <code>http://<masterIPAddress>:9101/</code></p> - -<p>You find the <code>masterIPAddress</code> by entering <code>ifconfig</code> into the terminal:</p> - -<div class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="o">[</span>hadoop@ip-172-31-38-95 ~<span class="o">]</span><span class="nv">$ </span>ifconfig -eth0 Link encap:Ethernet HWaddr 02:CF:8E:CB:28:B2 - inet addr:172.31.38.95 Bcast:172.31.47.255 Mask:255.255.240.0 - inet6 addr: fe80::cf:8eff:fecb:28b2/64 Scope:Link - RX bytes:166314967 <span class="o">(</span>158.6 MiB<span class="o">)</span> TX bytes:89319246 <span class="o">(</span>85.1 MiB<span class="o">)</span></code></pre></div> - -<p><strong>Optional:</strong> If you want to use the hostnames within your Firefox (that also makes the NameNode links work), you have to enable DNS resolution over the SOCKS proxy. Open the Firefox config <code>about:config</code> and set <code>network.proxy.socks_remote_dns</code> to <code>true</code>.</p> - -<p>The YARN ResourceManager also allows you to connect to <b>Stratosphereâs JobManager web interface</b>. Click the <b>ApplicationMaster</b> link in the âTracking UIâ column.</p> - -<p>To run the Wordcount example, you have to upload some sample data.</p> - -<div class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="c"># download a text</span> -wget http://www.gnu.org/licenses/gpl.txt -<span class="c"># upload it to HDFS:</span> -hadoop fs -copyFromLocal gpl.txt /input</code></pre></div> - -<p>To run a Job, enter the following command into the masterâs command line:</p> - -<div class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="c"># optional: go to the extracted directory</span> -<span class="nb">cd </span>stratosphere-yarn-0.5-SNAPSHOT/ -<span class="c"># run the wordcount example</span> -./bin/stratosphere run -w -j examples/stratosphere-java-examples-0.5-SNAPSHOT-WordCount.jar -a <span class="m">16</span> hdfs:///input hdfs:///output</code></pre></div> - -<p>Make sure that the number of TaskManagerâs have connected to the JobManager.</p> - -<p>Lets go through the command in detail:</p> - -<ul> - <li><code>./bin/stratosphere</code> is the standard launcher for Stratosphere jobs from the command line</li> - <li>The <code>-w</code> flag stands for âwaitâ. It is a very useful to track the progress of the job.</li> - <li><code>-j examples/stratosphere-java-examples-0.5-SNAPSHOT-WordCount.jar</code> the <code>-j</code> command sets the jar file containing the job. If you have you own application, place your Jar-file here.</li> - <li><code>-a 16 hdfs:///input hdfs:///output</code> the <code>-a</code> command specifies the Job-specific arguments. In this case, the wordcount expects the following input <code><numSubStasks> <input> <output></code>.</li> -</ul> - -<p>You can monitor the progress of your job in the JobManager webinterface. Once the job has finished (which should be the case after less than 10 seconds), you can analyze it there. -Inspect the result in HDFS using:</p> - -<div class="highlight"><pre><code class="language-bash" data-lang="bash">hadoop fs -tail /output</code></pre></div> - -<p>If you want to shut down the whole cluster in the cloud, use Amazonâs webinterface and click on âTerminate clusterâ. If you just want to stop the YARN session, press CTRL+C in the terminal. The Stratosphere instances will be killed by YARN.</p> - - </article> - </div> - - <div class="row"> - <div id="disqus_thread"></div> - <script type="text/javascript"> - /* * * CONFIGURATION VARIABLES: EDIT BEFORE PASTING INTO YOUR WEBPAGE * * */ - var disqus_shortname = 'stratosphere-eu'; // required: replace example with your forum shortname - - /* * * DON'T EDIT BELOW THIS LINE * * */ - (function() { - var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true; - dsq.src = '//' + disqus_shortname + '.disqus.com/embed.js'; - (document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq); - })(); - </script> - </div> - </div> -</div> - - <hr /> - <div class="footer text-center"> - <p>Copyright © 2014-2015 <a href="http://apache.org";>The Apache Software Foundation</a>. All Rights Reserved.</p> - <p>Apache Flink, Apache, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> - <p><a href="/privacy-policy.html">Privacy Policy</a> · <a href="/blog/feed.xml">RSS feed</a></p> - </div> - - </div><!-- /.container --> - - <!-- Include all compiled plugins (below), or include individual files as needed --> - <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.4/js/bootstrap.min.js";></script> - <script src="/js/codetabs.js"></script> - - <!-- Google Analytics --> - <script> - (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ - (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), - m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) - })(window,document,'script','//www.google-analytics.com/analytics.js','ga'); - - ga('create', 'UA-52545728-1', 'auto'); - ga('send', 'pageview'); - </script> - </body> -</html>
http://git-wip-us.apache.org/repos/asf/flink-web/blob/24f3ba5a/content/news/2014/02/24/stratosphere-google-summer-of-code-2014.html ---------------------------------------------------------------------- diff --git a/content/news/2014/02/24/stratosphere-google-summer-of-code-2014.html b/content/news/2014/02/24/stratosphere-google-summer-of-code-2014.html deleted file mode 100644 index 142fe2e..0000000 --- a/content/news/2014/02/24/stratosphere-google-summer-of-code-2014.html +++ /dev/null @@ -1,217 +0,0 @@ -<!DOCTYPE html> -<html lang="en"> - <head> - <meta charset="utf-8"> - <meta http-equiv="X-UA-Compatible" content="IE=edge"> - <meta name="viewport" content="width=device-width, initial-scale=1"> - <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags --> - <title>Apache Flink: Stratosphere got accepted for Google Summer of Code 2014</title> - <link rel="shortcut icon" href="/favicon.ico" type="image/x-icon"> - <link rel="icon" href="/favicon.ico" type="image/x-icon"> - - <!-- Bootstrap --> - <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.4/css/bootstrap.min.css";> - <link rel="stylesheet" href="/css/flink.css"> - <link rel="stylesheet" href="/css/syntax.css"> - - <!-- Blog RSS feed --> - <link href="/blog/feed.xml" rel="alternate" type="application/rss+xml" title="Apache Flink Blog: RSS feed" /> - - <!-- jQuery (necessary for Bootstrap's JavaScript plugins) --> - <!-- We need to load Jquery in the header for custom google analytics event tracking--> - <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.2/jquery.min.js";></script> - - <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries --> - <!-- WARNING: Respond.js doesn't work if you view the page via file:// --> - <!--[if lt IE 9]> - <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js";></script> - <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js";></script> - <![endif]--> - </head> - <body> - - - <!-- Top navbar. --> - <nav class="navbar navbar-default navbar-fixed-top"> - <div class="container"> - <!-- The logo. --> - <div class="navbar-header"> - <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#bs-example-navbar-collapse-1"> - <span class="icon-bar"></span> - <span class="icon-bar"></span> - <span class="icon-bar"></span> - </button> - <div class="navbar-logo"> - <a href="/"> - <img alt="Apache Flink" src="/img/navbar-brand-logo.jpg" width="78px" height="40px"> - </a> - </div> - </div><!-- /.navbar-header --> - - <!-- The navigation links. --> - <div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1"> - <ul class="nav navbar-nav"> - - <!-- Overview --> - <li><a href="/index.html">Overview</a></li> - - <!-- Features --> - <li><a href="/features.html">Features</a></li> - - <!-- Downloads --> - <li><a href="/downloads.html">Downloads</a></li> - - <!-- FAQ --> - <li><a href="/faq.html">FAQ</a></li> - - - <!-- Quickstart --> - <li class="dropdown"> - <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false"><small><span class="glyphicon glyphicon-new-window"></span></small> Quickstart <span class="caret"></span></a> - <ul class="dropdown-menu" role="menu"> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-release-1.0/quickstart/setup_quickstart.html";>Setup</a></li> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-release-1.0/quickstart/run_example_quickstart.html";>Example: Wikipedia Edit Stream</a></li> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-release-1.0/quickstart/java_api_quickstart.html";>Java API</a></li> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-release-1.0/quickstart/scala_api_quickstart.html";>Scala API</a></li> - </ul> - </li> - - <!-- Documentation --> - <li class="dropdown"> - <a href="" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false"><small><span class="glyphicon glyphicon-new-window"></span></small> Documentation <span class="caret"></span></a> - <ul class="dropdown-menu" role="menu"> - <!-- Latest stable release --> - <li role="presentation" class="dropdown-header"><strong>Latest Release</strong> (Stable)</li> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-release-1.0";>1.0 Documentation</a></li> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-release-1.0/api/java"; class="active">1.0 Javadocs</a></li> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-release-1.0/api/scala/index.html"; class="active">1.0 ScalaDocs</a></li> - - <!-- Snapshot docs --> - <li class="divider"></li> - <li role="presentation" class="dropdown-header"><strong>Snapshot</strong> (Development)</li> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-master";>1.1 Documentation</a></li> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-master/api/java"; class="active">1.1 Javadocs</a></li> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-master/api/scala/index.html"; class="active">1.1 ScalaDocs</a></li> - - <!-- Wiki --> - <li class="divider"></li> - <li><a href="/visualizer/"><small><span class="glyphicon glyphicon-new-window"></span></small> Plan Visualizer</a></li> - <li><a href="https://cwiki.apache.org/confluence/display/FLINK/Apache+Flink+Home";><small><span class="glyphicon glyphicon-new-window"></span></small> Wiki</a></li> - </ul> - </li> - - </ul> - - <ul class="nav navbar-nav navbar-right"> - <!-- Blog --> - <li class=" active hidden-md hidden-sm"><a href="/blog/">Blog</a></li> - - <li class="dropdown hidden-md hidden-sm"> - <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">Community <span class="caret"></span></a> - <ul class="dropdown-menu" role="menu"> - <!-- Community --> - <li role="presentation" class="dropdown-header"><strong>Community</strong></li> - <li><a href="/community.html#mailing-lists">Mailing Lists</a></li> - <li><a href="/community.html#irc">IRC</a></li> - <li><a href="/community.html#stack-overflow">Stack Overflow</a></li> - <li><a href="/community.html#issue-tracker">Issue Tracker</a></li> - <li><a href="/community.html#third-party-packages">Third Party Packages</a></li> - <li><a href="/community.html#source-code">Source Code</a></li> - <li><a href="/community.html#people">People</a></li> - <li><a href="https://cwiki.apache.org/confluence/display/FLINK/Powered+by+Flink";><small><span class="glyphicon glyphicon-new-window"></span></small> Powered by Flink</a></li> - - <!-- Contribute --> - <li class="divider"></li> - <li role="presentation" class="dropdown-header"><strong>Contribute</strong></li> - <li><a href="/how-to-contribute.html">How to Contribute</a></li> - <li><a href="/contribute-code.html">Contribute Code</a></li> - <li><a href="/contribute-documentation.html">Contribute Documentation</a></li> - <li><a href="/improve-website.html">Improve the Website</a></li> - </ul> - </li> - - <li class="dropdown hidden-md hidden-sm"> - <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">Project <span class="caret"></span></a> - <ul class="dropdown-menu" role="menu"> - <!-- Project --> - <li role="presentation" class="dropdown-header"><strong>Project</strong></li> - <li><a href="/slides.html">Slides</a></li> - <li><a href="/material.html">Material</a></li> - <li><a href="https://twitter.com/apacheflink";><small><span class="glyphicon glyphicon-new-window"></span></small> Twitter</a></li> - <li><a href="https://github.com/apache/flink";><small><span class="glyphicon glyphicon-new-window"></span></small> GitHub</a></li> - <li><a href="https://cwiki.apache.org/confluence/display/FLINK/Apache+Flink+Home";><small><span class="glyphicon glyphicon-new-window"></span></small> Wiki</a></li> - </ul> - </li> - </ul> - </div><!-- /.navbar-collapse --> - </div><!-- /.container --> - </nav> - - - <!-- Main content. --> - <div class="container"> - - -<div class="row"> - <div class="col-sm-8 col-sm-offset-2"> - <div class="row"> - <h1>Stratosphere got accepted for Google Summer of Code 2014</h1> - - <article> - <p>24 Feb 2014</p> - -<div class="lead">Students: Apply now for exciting summer projects in the Big Data / Analytics field</div> - -<p>We are pleased to announce that Stratosphere got accepted to <a href="http://www.google-melange.com/gsoc/homepage/google/gsoc2014";>Google Summer of Code 2014</a> as a mentoring organization. This means that we will host a bunch of students to conduct projects within Stratosphere over the summer. <a href="http://en.flossmanuals.net/GSoCStudentGuide/";>Read more on the GSoC manual for students</a> and the <a href="http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page";>official FAQ</a>. Students can improve their coding skills, learn to work with open-source projects, improve their CV and get a nice paycheck from Google.</p> - -<p>If you are an interested student, check out our <a href="https://github.com/stratosphere/stratosphere/wiki/Google-Summer-of-Code-2014";>idea list</a> in the wiki. It contains different projects with varying ranges of difficulty and requirement profiles. Students can also suggest their own projects.</p> - -<p>We welcome students to sign up at our <a href="https://groups.google.com/forum/#!forum/stratosphere-dev";>developer mailing list</a> to discuss their ideas. -Applying students can use our wiki (create a new page) to create a project proposal. We are happy to have a look at it.</p> - - - </article> - </div> - - <div class="row"> - <div id="disqus_thread"></div> - <script type="text/javascript"> - /* * * CONFIGURATION VARIABLES: EDIT BEFORE PASTING INTO YOUR WEBPAGE * * */ - var disqus_shortname = 'stratosphere-eu'; // required: replace example with your forum shortname - - /* * * DON'T EDIT BELOW THIS LINE * * */ - (function() { - var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true; - dsq.src = '//' + disqus_shortname + '.disqus.com/embed.js'; - (document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq); - })(); - </script> - </div> - </div> -</div> - - <hr /> - <div class="footer text-center"> - <p>Copyright © 2014-2015 <a href="http://apache.org";>The Apache Software Foundation</a>. All Rights Reserved.</p> - <p>Apache Flink, Apache, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> - <p><a href="/privacy-policy.html">Privacy Policy</a> · <a href="/blog/feed.xml">RSS feed</a></p> - </div> - - </div><!-- /.container --> - - <!-- Include all compiled plugins (below), or include individual files as needed --> - <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.4/js/bootstrap.min.js";></script> - <script src="/js/codetabs.js"></script> - - <!-- Google Analytics --> - <script> - (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ - (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), - m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) - })(window,document,'script','//www.google-analytics.com/analytics.js','ga'); - - ga('create', 'UA-52545728-1', 'auto'); - ga('send', 'pageview'); - </script> - </body> -</html> http://git-wip-us.apache.org/repos/asf/flink-web/blob/24f3ba5a/content/news/2014/04/16/stratosphere-goes-apache-incubator.html ---------------------------------------------------------------------- diff --git a/content/news/2014/04/16/stratosphere-goes-apache-incubator.html b/content/news/2014/04/16/stratosphere-goes-apache-incubator.html deleted file mode 100644 index d844f2b..0000000 --- a/content/news/2014/04/16/stratosphere-goes-apache-incubator.html +++ /dev/null @@ -1,212 +0,0 @@ -<!DOCTYPE html> -<html lang="en"> - <head> - <meta charset="utf-8"> - <meta http-equiv="X-UA-Compatible" content="IE=edge"> - <meta name="viewport" content="width=device-width, initial-scale=1"> - <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags --> - <title>Apache Flink: Stratosphere accepted as Apache Incubator Project</title> - <link rel="shortcut icon" href="/favicon.ico" type="image/x-icon"> - <link rel="icon" href="/favicon.ico" type="image/x-icon"> - - <!-- Bootstrap --> - <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.4/css/bootstrap.min.css";> - <link rel="stylesheet" href="/css/flink.css"> - <link rel="stylesheet" href="/css/syntax.css"> - - <!-- Blog RSS feed --> - <link href="/blog/feed.xml" rel="alternate" type="application/rss+xml" title="Apache Flink Blog: RSS feed" /> - - <!-- jQuery (necessary for Bootstrap's JavaScript plugins) --> - <!-- We need to load Jquery in the header for custom google analytics event tracking--> - <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.2/jquery.min.js";></script> - - <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries --> - <!-- WARNING: Respond.js doesn't work if you view the page via file:// --> - <!--[if lt IE 9]> - <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js";></script> - <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js";></script> - <![endif]--> - </head> - <body> - - - <!-- Top navbar. --> - <nav class="navbar navbar-default navbar-fixed-top"> - <div class="container"> - <!-- The logo. --> - <div class="navbar-header"> - <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#bs-example-navbar-collapse-1"> - <span class="icon-bar"></span> - <span class="icon-bar"></span> - <span class="icon-bar"></span> - </button> - <div class="navbar-logo"> - <a href="/"> - <img alt="Apache Flink" src="/img/navbar-brand-logo.jpg" width="78px" height="40px"> - </a> - </div> - </div><!-- /.navbar-header --> - - <!-- The navigation links. --> - <div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1"> - <ul class="nav navbar-nav"> - - <!-- Overview --> - <li><a href="/index.html">Overview</a></li> - - <!-- Features --> - <li><a href="/features.html">Features</a></li> - - <!-- Downloads --> - <li><a href="/downloads.html">Downloads</a></li> - - <!-- FAQ --> - <li><a href="/faq.html">FAQ</a></li> - - - <!-- Quickstart --> - <li class="dropdown"> - <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false"><small><span class="glyphicon glyphicon-new-window"></span></small> Quickstart <span class="caret"></span></a> - <ul class="dropdown-menu" role="menu"> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-release-1.0/quickstart/setup_quickstart.html";>Setup</a></li> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-release-1.0/quickstart/run_example_quickstart.html";>Example: Wikipedia Edit Stream</a></li> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-release-1.0/quickstart/java_api_quickstart.html";>Java API</a></li> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-release-1.0/quickstart/scala_api_quickstart.html";>Scala API</a></li> - </ul> - </li> - - <!-- Documentation --> - <li class="dropdown"> - <a href="" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false"><small><span class="glyphicon glyphicon-new-window"></span></small> Documentation <span class="caret"></span></a> - <ul class="dropdown-menu" role="menu"> - <!-- Latest stable release --> - <li role="presentation" class="dropdown-header"><strong>Latest Release</strong> (Stable)</li> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-release-1.0";>1.0 Documentation</a></li> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-release-1.0/api/java"; class="active">1.0 Javadocs</a></li> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-release-1.0/api/scala/index.html"; class="active">1.0 ScalaDocs</a></li> - - <!-- Snapshot docs --> - <li class="divider"></li> - <li role="presentation" class="dropdown-header"><strong>Snapshot</strong> (Development)</li> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-master";>1.1 Documentation</a></li> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-master/api/java"; class="active">1.1 Javadocs</a></li> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-master/api/scala/index.html"; class="active">1.1 ScalaDocs</a></li> - - <!-- Wiki --> - <li class="divider"></li> - <li><a href="/visualizer/"><small><span class="glyphicon glyphicon-new-window"></span></small> Plan Visualizer</a></li> - <li><a href="https://cwiki.apache.org/confluence/display/FLINK/Apache+Flink+Home";><small><span class="glyphicon glyphicon-new-window"></span></small> Wiki</a></li> - </ul> - </li> - - </ul> - - <ul class="nav navbar-nav navbar-right"> - <!-- Blog --> - <li class=" active hidden-md hidden-sm"><a href="/blog/">Blog</a></li> - - <li class="dropdown hidden-md hidden-sm"> - <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">Community <span class="caret"></span></a> - <ul class="dropdown-menu" role="menu"> - <!-- Community --> - <li role="presentation" class="dropdown-header"><strong>Community</strong></li> - <li><a href="/community.html#mailing-lists">Mailing Lists</a></li> - <li><a href="/community.html#irc">IRC</a></li> - <li><a href="/community.html#stack-overflow">Stack Overflow</a></li> - <li><a href="/community.html#issue-tracker">Issue Tracker</a></li> - <li><a href="/community.html#third-party-packages">Third Party Packages</a></li> - <li><a href="/community.html#source-code">Source Code</a></li> - <li><a href="/community.html#people">People</a></li> - <li><a href="https://cwiki.apache.org/confluence/display/FLINK/Powered+by+Flink";><small><span class="glyphicon glyphicon-new-window"></span></small> Powered by Flink</a></li> - - <!-- Contribute --> - <li class="divider"></li> - <li role="presentation" class="dropdown-header"><strong>Contribute</strong></li> - <li><a href="/how-to-contribute.html">How to Contribute</a></li> - <li><a href="/contribute-code.html">Contribute Code</a></li> - <li><a href="/contribute-documentation.html">Contribute Documentation</a></li> - <li><a href="/improve-website.html">Improve the Website</a></li> - </ul> - </li> - - <li class="dropdown hidden-md hidden-sm"> - <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">Project <span class="caret"></span></a> - <ul class="dropdown-menu" role="menu"> - <!-- Project --> - <li role="presentation" class="dropdown-header"><strong>Project</strong></li> - <li><a href="/slides.html">Slides</a></li> - <li><a href="/material.html">Material</a></li> - <li><a href="https://twitter.com/apacheflink";><small><span class="glyphicon glyphicon-new-window"></span></small> Twitter</a></li> - <li><a href="https://github.com/apache/flink";><small><span class="glyphicon glyphicon-new-window"></span></small> GitHub</a></li> - <li><a href="https://cwiki.apache.org/confluence/display/FLINK/Apache+Flink+Home";><small><span class="glyphicon glyphicon-new-window"></span></small> Wiki</a></li> - </ul> - </li> - </ul> - </div><!-- /.navbar-collapse --> - </div><!-- /.container --> - </nav> - - - <!-- Main content. --> - <div class="container"> - - -<div class="row"> - <div class="col-sm-8 col-sm-offset-2"> - <div class="row"> - <h1>Stratosphere accepted as Apache Incubator Project</h1> - - <article> - <p>16 Apr 2014</p> - -<p>We are happy to announce that Stratosphere has been accepted as a project for the <a href="https://incubator.apache.org/";>Apache Incubator</a>. The <a href="https://wiki.apache.org/incubator/StratosphereProposal";>proposal</a> has been accepted by the Incubator PMC members earlier this week. The Apache Incubator is the first step in the process of giving a project to the <a href="http://apache.org";>Apache Software Foundation</a>. While under incubation, the project will move to the Apache infrastructure and adopt the community-driven development principles of the Apache Foundation. Projects can graduate from incubation to become top-level projects if they show activity, a healthy community dynamic, and releases.</p> - -<p>We are glad to have Alan Gates as champion on board, as well as a set of great mentors, including Sean Owen, Ted Dunning, Owen OâMalley, Henry Saputra, and Ashutosh Chauhan. We are confident that we will make this a great open source effort.</p> - - - </article> - </div> - - <div class="row"> - <div id="disqus_thread"></div> - <script type="text/javascript"> - /* * * CONFIGURATION VARIABLES: EDIT BEFORE PASTING INTO YOUR WEBPAGE * * */ - var disqus_shortname = 'stratosphere-eu'; // required: replace example with your forum shortname - - /* * * DON'T EDIT BELOW THIS LINE * * */ - (function() { - var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true; - dsq.src = '//' + disqus_shortname + '.disqus.com/embed.js'; - (document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq); - })(); - </script> - </div> - </div> -</div> - - <hr /> - <div class="footer text-center"> - <p>Copyright © 2014-2015 <a href="http://apache.org";>The Apache Software Foundation</a>. All Rights Reserved.</p> - <p>Apache Flink, Apache, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> - <p><a href="/privacy-policy.html">Privacy Policy</a> · <a href="/blog/feed.xml">RSS feed</a></p> - </div> - - </div><!-- /.container --> - - <!-- Include all compiled plugins (below), or include individual files as needed --> - <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.4/js/bootstrap.min.js";></script> - <script src="/js/codetabs.js"></script> - - <!-- Google Analytics --> - <script> - (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ - (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), - m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) - })(window,document,'script','//www.google-analytics.com/analytics.js','ga'); - - ga('create', 'UA-52545728-1', 'auto'); - ga('send', 'pageview'); - </script> - </body> -</html> http://git-wip-us.apache.org/repos/asf/flink-web/blob/24f3ba5a/content/news/2014/05/31/release-0.5.html ---------------------------------------------------------------------- diff --git a/content/news/2014/05/31/release-0.5.html b/content/news/2014/05/31/release-0.5.html deleted file mode 100644 index 06cbc72..0000000 --- a/content/news/2014/05/31/release-0.5.html +++ /dev/null @@ -1,289 +0,0 @@ -<!DOCTYPE html> -<html lang="en"> - <head> - <meta charset="utf-8"> - <meta http-equiv="X-UA-Compatible" content="IE=edge"> - <meta name="viewport" content="width=device-width, initial-scale=1"> - <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags --> - <title>Apache Flink: Stratosphere version 0.5 available</title> - <link rel="shortcut icon" href="/favicon.ico" type="image/x-icon"> - <link rel="icon" href="/favicon.ico" type="image/x-icon"> - - <!-- Bootstrap --> - <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.4/css/bootstrap.min.css";> - <link rel="stylesheet" href="/css/flink.css"> - <link rel="stylesheet" href="/css/syntax.css"> - - <!-- Blog RSS feed --> - <link href="/blog/feed.xml" rel="alternate" type="application/rss+xml" title="Apache Flink Blog: RSS feed" /> - - <!-- jQuery (necessary for Bootstrap's JavaScript plugins) --> - <!-- We need to load Jquery in the header for custom google analytics event tracking--> - <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.2/jquery.min.js";></script> - - <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries --> - <!-- WARNING: Respond.js doesn't work if you view the page via file:// --> - <!--[if lt IE 9]> - <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js";></script> - <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js";></script> - <![endif]--> - </head> - <body> - - - <!-- Top navbar. --> - <nav class="navbar navbar-default navbar-fixed-top"> - <div class="container"> - <!-- The logo. --> - <div class="navbar-header"> - <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#bs-example-navbar-collapse-1"> - <span class="icon-bar"></span> - <span class="icon-bar"></span> - <span class="icon-bar"></span> - </button> - <div class="navbar-logo"> - <a href="/"> - <img alt="Apache Flink" src="/img/navbar-brand-logo.jpg" width="78px" height="40px"> - </a> - </div> - </div><!-- /.navbar-header --> - - <!-- The navigation links. --> - <div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1"> - <ul class="nav navbar-nav"> - - <!-- Overview --> - <li><a href="/index.html">Overview</a></li> - - <!-- Features --> - <li><a href="/features.html">Features</a></li> - - <!-- Downloads --> - <li><a href="/downloads.html">Downloads</a></li> - - <!-- FAQ --> - <li><a href="/faq.html">FAQ</a></li> - - - <!-- Quickstart --> - <li class="dropdown"> - <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false"><small><span class="glyphicon glyphicon-new-window"></span></small> Quickstart <span class="caret"></span></a> - <ul class="dropdown-menu" role="menu"> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-release-1.0/quickstart/setup_quickstart.html";>Setup</a></li> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-release-1.0/quickstart/run_example_quickstart.html";>Example: Wikipedia Edit Stream</a></li> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-release-1.0/quickstart/java_api_quickstart.html";>Java API</a></li> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-release-1.0/quickstart/scala_api_quickstart.html";>Scala API</a></li> - </ul> - </li> - - <!-- Documentation --> - <li class="dropdown"> - <a href="" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false"><small><span class="glyphicon glyphicon-new-window"></span></small> Documentation <span class="caret"></span></a> - <ul class="dropdown-menu" role="menu"> - <!-- Latest stable release --> - <li role="presentation" class="dropdown-header"><strong>Latest Release</strong> (Stable)</li> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-release-1.0";>1.0 Documentation</a></li> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-release-1.0/api/java"; class="active">1.0 Javadocs</a></li> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-release-1.0/api/scala/index.html"; class="active">1.0 ScalaDocs</a></li> - - <!-- Snapshot docs --> - <li class="divider"></li> - <li role="presentation" class="dropdown-header"><strong>Snapshot</strong> (Development)</li> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-master";>1.1 Documentation</a></li> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-master/api/java"; class="active">1.1 Javadocs</a></li> - <li><a href="http://ci.apache.org/projects/flink/flink-docs-master/api/scala/index.html"; class="active">1.1 ScalaDocs</a></li> - - <!-- Wiki --> - <li class="divider"></li> - <li><a href="/visualizer/"><small><span class="glyphicon glyphicon-new-window"></span></small> Plan Visualizer</a></li> - <li><a href="https://cwiki.apache.org/confluence/display/FLINK/Apache+Flink+Home";><small><span class="glyphicon glyphicon-new-window"></span></small> Wiki</a></li> - </ul> - </li> - - </ul> - - <ul class="nav navbar-nav navbar-right"> - <!-- Blog --> - <li class=" active hidden-md hidden-sm"><a href="/blog/">Blog</a></li> - - <li class="dropdown hidden-md hidden-sm"> - <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">Community <span class="caret"></span></a> - <ul class="dropdown-menu" role="menu"> - <!-- Community --> - <li role="presentation" class="dropdown-header"><strong>Community</strong></li> - <li><a href="/community.html#mailing-lists">Mailing Lists</a></li> - <li><a href="/community.html#irc">IRC</a></li> - <li><a href="/community.html#stack-overflow">Stack Overflow</a></li> - <li><a href="/community.html#issue-tracker">Issue Tracker</a></li> - <li><a href="/community.html#third-party-packages">Third Party Packages</a></li> - <li><a href="/community.html#source-code">Source Code</a></li> - <li><a href="/community.html#people">People</a></li> - <li><a href="https://cwiki.apache.org/confluence/display/FLINK/Powered+by+Flink";><small><span class="glyphicon glyphicon-new-window"></span></small> Powered by Flink</a></li> - - <!-- Contribute --> - <li class="divider"></li> - <li role="presentation" class="dropdown-header"><strong>Contribute</strong></li> - <li><a href="/how-to-contribute.html">How to Contribute</a></li> - <li><a href="/contribute-code.html">Contribute Code</a></li> - <li><a href="/contribute-documentation.html">Contribute Documentation</a></li> - <li><a href="/improve-website.html">Improve the Website</a></li> - </ul> - </li> - - <li class="dropdown hidden-md hidden-sm"> - <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">Project <span class="caret"></span></a> - <ul class="dropdown-menu" role="menu"> - <!-- Project --> - <li role="presentation" class="dropdown-header"><strong>Project</strong></li> - <li><a href="/slides.html">Slides</a></li> - <li><a href="/material.html">Material</a></li> - <li><a href="https://twitter.com/apacheflink";><small><span class="glyphicon glyphicon-new-window"></span></small> Twitter</a></li> - <li><a href="https://github.com/apache/flink";><small><span class="glyphicon glyphicon-new-window"></span></small> GitHub</a></li> - <li><a href="https://cwiki.apache.org/confluence/display/FLINK/Apache+Flink+Home";><small><span class="glyphicon glyphicon-new-window"></span></small> Wiki</a></li> - </ul> - </li> - </ul> - </div><!-- /.navbar-collapse --> - </div><!-- /.container --> - </nav> - - - <!-- Main content. --> - <div class="container"> - - -<div class="row"> - <div class="col-sm-8 col-sm-offset-2"> - <div class="row"> - <h1>Stratosphere version 0.5 available</h1> - - <article> - <p>31 May 2014</p> - -<p>We are happy to announce a new major Stratosphere release, version 0.5. This release adds many new features and improves the interoperability, stability, and performance of the system. The major theme of the release is the completely new Java API that makes it easy to write powerful distributed programs.</p> - -<p>The release can be downloaded from the [Stratosphere website] (http://stratosphere.eu/downloads/) and from [GitHub] (https://github.com/stratosphere/stratosphere/releases/tag/release-0.5). All components are available as Apache Maven dependencies, making it simple to include Stratosphere in other projects. The website provides <a href="http://stratosphere.eu/docs/0.5/";>extensive documentation</a> of the system and the new features.</p> - -<h2 id="shortlist-of-new-features">Shortlist of new Features</h2> - -<p>Below is a short list of the most important additions to the Stratosphere system.</p> - -<h4 id="new-java-api">New Java API</h4> -<p>This release introduces a completely new <strong>data set-centric Java API</strong>. This programming model significantly eases the development of Stratosphere programs, supports flexible use of regular Java classes as data types, and adds many new built-in operators to simplify the writing of powerful programs. The result are programs that need less code, are more readable, interoperate better with existing code, and execute faster.</p> - -<p>Take a look at the <a href="http://stratosphere.eu/docs/0.5/programming_guides/examples_java.html";>examples</a> to get a feel for the API.</p> - -<h4 id="general-api-improvements">General API Improvements</h4> -<p><strong>Broadcast Variables:</strong> Publish a data set to all instances of another operator. This is handy if the your operator depends on the result of a computation, e.g., filter all values smaller than the average.</p> - -<p><strong>Distributed Cache:</strong> Make (local and HDFS) files locally available on each machine processing a task.</p> - -<p><strong>Iteration Termination Improvements</strong> Iterative algorithms can now terminate based on intermediate data sets, not only through aggregated statistics.</p> - -<p><strong>Collection data sources and sinks:</strong> Speed-up the development and testing of Stratosphere programs by reading data from regular Java collections and inserting back into them.</p> - -<p><strong>JDBC data sources and sinks:</strong> Read data from and write data to relational databases using a JDBC driver.</p> - -<p><strong>Hadoop input format and output format support:</strong> Read and write data with any Hadoop input or output format.</p> - -<p><strong>Support for Avro encoded data:</strong> Read data that has been materialized using Avro.</p> - -<p><strong>Deflate Files:</strong> Stratosphere now transparently reads <code>.deflate</code> compressed files.</p> - -<h4 id="runtime-and-optimizer-improvements">Runtime and Optimizer Improvements</h4> - -<p><strong>DAG Runtime Streaming:</strong> Detection and resolution of streaming data flow deadlocks in the data flow optimizer.</p> - -<p><strong>Intermediate results across iteration boundaries:</strong> Intermediate results computed outside iterative parts can be used inside iterative parts of the program.</p> - -<p><strong>Stability fixes:</strong> Various stability fixes in both optimizer and runtime.</p> - -<h4 id="setup--tooling">Setup & Tooling</h4> - -<p><strong>Improved YARN support:</strong> Many improvements based on user-feedback: Packaging, Permissions, Error handling.</p> - -<p><strong>Java 8 compatibility</strong></p> - -<h2 id="contributors">Contributors</h2> - -<p>In total, 26 people have contributed to Stratosphere since the last release. Thank you for making this project possible!</p> - -<ul> - <li>Alexander Alexandrov</li> - <li>Jesus Camacho</li> - <li>Ufuk Celebi</li> - <li>Mikhail Erofeev</li> - <li>Stephan Ewen</li> - <li>Alexandr Ferodov</li> - <li>Filip Haase</li> - <li>Jonathan Hasenberg</li> - <li>Markus Holzemer</li> - <li>Fabian Hueske</li> - <li>Vasia Kalavri</li> - <li>Aljoscha Krettek</li> - <li>Rajika Kumarasiri</li> - <li>Sebastian Kunert</li> - <li>Aaron Lam</li> - <li>Robert Metzger</li> - <li>Faisal Moeen</li> - <li>Martin Neumann</li> - <li>Mingliang Qi</li> - <li>Till Rohrmann</li> - <li>Chesnay Schepler</li> - <li>Vyachislav Soludev</li> - <li>Tuan Trieu</li> - <li>Artem Tsikiridis</li> - <li>Timo Walther</li> - <li>Robert Waury</li> -</ul> - -<h2 id="stratosphere-is-going-apache">Stratosphere is going Apache</h2> - -<p>The Stratosphere project has been accepted to the Apache Incubator and will continue its work under the umbrella of the Apache Software Foundation. Due to a name conflict, we are switching the name of the project. We will make future releases of Stratosphere through the Apache foundation under a new name.</p> - - - </article> - </div> - - <div class="row"> - <div id="disqus_thread"></div> - <script type="text/javascript"> - /* * * CONFIGURATION VARIABLES: EDIT BEFORE PASTING INTO YOUR WEBPAGE * * */ - var disqus_shortname = 'stratosphere-eu'; // required: replace example with your forum shortname - - /* * * DON'T EDIT BELOW THIS LINE * * */ - (function() { - var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true; - dsq.src = '//' + disqus_shortname + '.disqus.com/embed.js'; - (document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq); - })(); - </script> - </div> - </div> -</div> - - <hr /> - <div class="footer text-center"> - <p>Copyright © 2014-2015 <a href="http://apache.org";>The Apache Software Foundation</a>. All Rights Reserved.</p> - <p>Apache Flink, Apache, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> - <p><a href="/privacy-policy.html">Privacy Policy</a> · <a href="/blog/feed.xml">RSS feed</a></p> - </div> - - </div><!-- /.container --> - - <!-- Include all compiled plugins (below), or include individual files as needed --> - <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.4/js/bootstrap.min.js";></script> - <script src="/js/codetabs.js"></script> - - <!-- Google Analytics --> - <script> - (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ - (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), - m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) - })(window,document,'script','//www.google-analytics.com/analytics.js','ga'); - - ga('create', 'UA-52545728-1', 'auto'); - ga('send', 'pageview'); - </script> - </body> -</html> http://git-wip-us.apache.org/repos/asf/flink-web/blob/24f3ba5a/content/news/2014/11/18/hadoop-compatibility.html ---------------------------------------------------------------------- diff --git a/content/news/2014/11/18/hadoop-compatibility.html b/content/news/2014/11/18/hadoop-compatibility.html index 2700d0f..beea975 100644 --- a/content/news/2014/11/18/hadoop-compatibility.html +++ b/content/news/2014/11/18/hadoop-compatibility.html @@ -197,25 +197,25 @@ <span class="c1">// Setup Hadoopâs TextInputFormat</span> <span class="n">HadoopInputFormat</span><span class="o"><</span><span class="n">LongWritable</span><span class="o">,</span> <span class="n">Text</span><span class="o">></span> <span class="n">hadoopInputFormat</span> <span class="o">=</span> <span class="k">new</span> <span class="n">HadoopInputFormat</span><span class="o"><</span><span class="n">LongWritable</span><span class="o">,</span> <span class="n">Text</span><span class="o">>(</span> - <span class="k">new</span> <span class="nf">TextInputFormat</span><span class="o">(),</span> <span class="n">LongWritable</span><span class="o">.</span><span class="na">class</span><span class="o">,</span> <span class="n">Text</span><span class="o">.</span><span class="na">class</span><span class="o">,</span> <span class="k">new</span> <span class="nf">JobConf</span><span class="o">());</span> - <span class="n">TextInputFormat</span><span class="o">.</span><span class="na">addInputPath</span><span class="o">(</span><span class="n">hadoopInputFormat</span><span class="o">.</span><span class="na">getJobConf</span><span class="o">(),</span> <span class="k">new</span> <span class="nf">Path</span><span class="o">(</span><span class="n">inputPath</span><span class="o">));</span> + <span class="k">new</span> <span class="n">TextInputFormat</span><span class="o">(),</span> <span class="n">LongWritable</span><span class="o">.</span><span class="na">class</span><span class="o">,</span> <span class="n">Text</span><span class="o">.</span><span class="na">class</span><span class="o">,</span> <span class="k">new</span> <span class="n">JobConf</span><span class="o">());</span> + <span class="n">TextInputFormat</span><span class="o">.</span><span class="na">addInputPath</span><span class="o">(</span><span class="n">hadoopInputFormat</span><span class="o">.</span><span class="na">getJobConf</span><span class="o">(),</span> <span class="k">new</span> <span class="n">Path</span><span class="o">(</span><span class="n">inputPath</span><span class="o">));</span> <span class="c1">// Read a DataSet with the Hadoop InputFormat</span> <span class="n">DataSet</span><span class="o"><</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">LongWritable</span><span class="o">,</span> <span class="n">Text</span><span class="o">>></span> <span class="n">text</span> <span class="o">=</span> <span class="n">env</span><span class="o">.</span><span class="na">createInput</span><span class="o">(</span><span class="n">hadoopInputFormat</span><span class="o">);</span> <span class="n">DataSet</span><span class="o"><</span><span class="n">Tuple2</span><span class="o"><</span><span class="n">Text</span><span class="o">,</span> <span class="n">LongWritable</span><span class="o">>></span> <span class="n">words</span> <span class="o">=</span> <span class="n">text</span> <span class="c1">// Wrap Tokenizer Mapper function</span> - <span class="o">.</span><span class="na">flatMap</span><span class="o">(</span><span class="k">new</span> <span class="n">HadoopMapFunction</span><span class="o"><</span><span class="n">LongWritable</span><span class="o">,</span> <span class="n">Text</span><span class="o">,</span> <span class="n">Text</span><span class="o">,</span> <span class="n">LongWritable</span><span class="o">>(</span><span class="k">new</span> <span class="nf">Tokenizer</span><span class="o">()))</span> + <span class="o">.</span><span class="na">flatMap</span><span class="o">(</span><span class="k">new</span> <span class="n">HadoopMapFunction</span><span class="o"><</span><span class="n">LongWritable</span><span class="o">,</span> <span class="n">Text</span><span class="o">,</span> <span class="n">Text</span><span class="o">,</span> <span class="n">LongWritable</span><span class="o">>(</span><span class="k">new</span> <span class="n">Tokenizer</span><span class="o">()))</span> <span class="o">.</span><span class="na">groupBy</span><span class="o">(</span><span class="mi">0</span><span class="o">)</span> <span class="c1">// Wrap Counter Reducer function (used as Reducer and Combiner)</span> <span class="o">.</span><span class="na">reduceGroup</span><span class="o">(</span><span class="k">new</span> <span class="n">HadoopReduceCombineFunction</span><span class="o"><</span><span class="n">Text</span><span class="o">,</span> <span class="n">LongWritable</span><span class="o">,</span> <span class="n">Text</span><span class="o">,</span> <span class="n">LongWritable</span><span class="o">>(</span> - <span class="k">new</span> <span class="nf">Counter</span><span class="o">(),</span> <span class="k">new</span> <span class="nf">Counter</span><span class="o">()));</span> + <span class="k">new</span> <span class="n">Counter</span><span class="o">(),</span> <span class="k">new</span> <span class="n">Counter</span><span class="o">()));</span> <span class="c1">// Setup Hadoopâs TextOutputFormat</span> <span class="n">HadoopOutputFormat</span><span class="o"><</span><span class="n">Text</span><span class="o">,</span> <span class="n">LongWritable</span><span class="o">></span> <span class="n">hadoopOutputFormat</span> <span class="o">=</span> <span class="k">new</span> <span class="n">HadoopOutputFormat</span><span class="o"><</span><span class="n">Text</span><span class="o">,</span> <span class="n">LongWritable</span><span class="o">>(</span> - <span class="k">new</span> <span class="n">TextOutputFormat</span><span class="o"><</span><span class="n">Text</span><span class="o">,</span> <span class="n">LongWritable</span><span class="o">>(),</span> <span class="k">new</span> <span class="nf">JobConf</span><span class="o">());</span> + <span class="k">new</span> <span class="n">TextOutputFormat</span><span class="o"><</span><span class="n">Text</span><span class="o">,</span> <span class="n">LongWritable</span><span class="o">>(),</span> <span class="k">new</span> <span class="n">JobConf</span><span class="o">());</span> <span class="n">hadoopOutputFormat</span><span class="o">.</span><span class="na">getJobConf</span><span class="o">().</span><span class="na">set</span><span class="o">(</span><span class="s">"mapred.textoutputformat.separator"</span><span class="o">,</span> <span class="s">" "</span><span class="o">);</span> - <span class="n">TextOutputFormat</span><span class="o">.</span><span class="na">setOutputPath</span><span class="o">(</span><span class="n">hadoopOutputFormat</span><span class="o">.</span><span class="na">getJobConf</span><span class="o">(),</span> <span class="k">new</span> <span class="nf">Path</span><span class="o">(</span><span class="n">outputPath</span><span class="o">));</span> + <span class="n">TextOutputFormat</span><span class="o">.</span><span class="na">setOutputPath</span><span class="o">(</span><span class="n">hadoopOutputFormat</span><span class="o">.</span><span class="na">getJobConf</span><span class="o">(),</span> <span class="k">new</span> <span class="n">Path</span><span class="o">(</span><span class="n">outputPath</span><span class="o">));</span> <span class="c1">// Output & Execute</span> <span class="n">words</span><span class="o">.</span><span class="na">output</span><span class="o">(</span><span class="n">hadoopOutputFormat</span><span class="o">);</span> @@ -238,7 +238,7 @@ <p>Flink lets you reuse a lot of the code you wrote for Hadoop MapReduce, including all data types, all Input- and OutputFormats, and Mapper and Reducers of the mapred-API. Hadoop functions can be used within Flink programs and mixed with all other Flink functions. Due to Flinkâs pipelined execution, Hadoop functions can arbitrarily be assembled without data exchange via HDFS. Moreover, the Flink community is currently working on a dedicated Hadoop Job operation to supporting the execution of Hadoop jobs as a whole.</p> -<p>If you want to use Flinkâs Hadoop compatibility package checkout our <a href="http://ci.apache.org/projects/flink/flink-docs-release-0.7/hadoop_compatibility.html";>documentation</a>.</p> +<p>If you want to use Flinkâs Hadoop compatibility package checkout our <a href="https://ci.apache.org/projects/flink/flink-docs-master/apis/batch/hadoop_compatibility.html";>documentation</a>.</p> </article> </div> http://git-wip-us.apache.org/repos/asf/flink-web/blob/24f3ba5a/content/news/2015/02/04/january-in-flink.html ---------------------------------------------------------------------- diff --git a/content/news/2015/02/04/january-in-flink.html b/content/news/2015/02/04/january-in-flink.html index c3faa75..7febbcf 100644 --- a/content/news/2015/02/04/january-in-flink.html +++ b/content/news/2015/02/04/january-in-flink.html @@ -170,17 +170,13 @@ <p>The community has published a <a href="https://cwiki.apache.org/confluence/display/FLINK/Flink+Roadmap";>roadmap for 2015</a> on the Flink wiki. Check it out to see what is coming up in Flink, and pick up an issue to contribute!</p> -<h3 id="scaling-als">Scaling ALS</h3> - -<p>Flink committers employed at <a href="http://data-artisans.com";>data Artisans</a> published a <a href="http://data-artisans.com/computing-recommendations-with-flink.html";>blog post</a> on how they scaled matrix factorization with Flink and Google Compute Engine to matrices with 28 billion elements.</p> - <h3 id="articles-in-the-press">Articles in the press</h3> <p>The Apache Software Foundation <a href="https://blogs.apache.org/foundation/entry/the_apache_software_foundation_announces69";>announced</a> Flink as a Top-Level Project. The announcement was picked up by the media, e.g., <a href="http://sdtimes.com/inside-apache-software-foundations-newest-top-level-project-apache-flink/?utm_content=11232092&utm_medium=social&utm_source=twitter";>here</a>, <a href="http://www.datanami.com/2015/01/12/apache-flink-takes-route-distributed-data-processing/";>here</a>, and <a href="http://i-programmer.info/news/197-data-mining/8176-flink-reaches-top-level-status.html";>here</a>.</p> <h3 id="hadoop-summit">Hadoop Summit</h3> -<p>A submitted abstract on Flink Streaming <a href="http://2015.hadoopsummit.org/amsterdam-blog/announcing-the-community-vote-session-winners-for-the-2015-hadoop-summit-europe/";>won the community</a> vote at âThe Future of Hadoopâ track.</p> +<p>A submitted abstract on Flink Streaming won the community vote at âThe Future of Hadoopâ track.</p> <h3 id="meetups-and-talks">Meetups and talks</h3>
