This is an automated email from the ASF dual-hosted git repository.
jincheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git
The following commit(s) were added to refs/heads/master by this push:
new f059b19 [FLINK-12962][python] Allows pyflink to be pip installed.
f059b19 is described below
commit f059b1981b8d2a62acfd2fac3456b8ca3a5bd18f
Author: Wei Zhong <weizhong0...@gmail.com>
AuthorDate: Tue Jun 25 09:51:05 2019 +0800
[FLINK-12962][python] Allows pyflink to be pip installed.
This closes #8863
---
docs/flinkDev/building.md | 15 ++
docs/flinkDev/building.zh.md | 15 ++
flink-dist/src/main/flink-bin/bin/config.sh | 5 +-
.../src/main/flink-bin/bin/find-flink-home.sh | 11 +-
flink-dist/src/main/flink-bin/bin/pyflink-shell.sh | 5 +-
flink-python/{pyflink/version.py => MANIFEST.in} | 14 +-
flink-python/dev/pip_test_code.py | 53 ++++++
.../{pyflink/version.py => dev/run_pip_test.sh} | 5 +-
flink-python/pyflink/find_flink_home.py | 31 ++-
flink-python/pyflink/shell.py | 12 +-
flink-python/pyflink/version.py | 6 +-
flink-python/{pyflink/version.py => setup.cfg} | 6 +-
flink-python/setup.py | 210 ++++++++++++++++++---
flink-python/tox.ini | 3 +
tools/releasing/update_branch_version.sh | 6 +
15 files changed, 355 insertions(+), 42 deletions(-)
diff --git a/docs/flinkDev/building.md b/docs/flinkDev/building.md
index 521bc9f..c78bb3b 100644
--- a/docs/flinkDev/building.md
+++ b/docs/flinkDev/building.md
@@ -58,6 +58,21 @@ mvn clean install -DskipTests -Dfast
The default build adds a Flink-specific JAR for Hadoop 2, to allow using Flink
with HDFS and YARN.
+## Build PyFlink
+
+If you want to build a PyFlink package that can be used for pip installation,
you need to build Flink jars first, as described in [Build Flink](##Build
Flink).
+Then go to the root directory of flink source code and run this command to
build the sdist package and wheel package:
+
+{% highlight bash %}
+cd flink-python; python3 setup.py sdist bdist_wheel
+{% endhighlight %}
+
+The sdist and wheel package will be found under `./flink-python/dist/`. Either
of them could be used for pip installation, such as:
+
+{% highlight bash %}
+pip install dist/*.tar.gz
+{% endhighlight %}
+
## Dependency Shading
Flink [shades away](https://maven.apache.org/plugins/maven-shade-plugin/) some
of the libraries it uses, in order to avoid version clashes with user programs
that use different versions of these libraries. Among the shaded libraries are
*Google Guava*, *Asm*, *Apache Curator*, *Apache HTTP Components*, *Netty*, and
others.
diff --git a/docs/flinkDev/building.zh.md b/docs/flinkDev/building.zh.md
index c48de50..aa70511 100644
--- a/docs/flinkDev/building.zh.md
+++ b/docs/flinkDev/building.zh.md
@@ -58,6 +58,21 @@ mvn clean install -DskipTests -Dfast
The default build adds a Flink-specific JAR for Hadoop 2, to allow using Flink
with HDFS and YARN.
+## 构建PyFlink
+
+如果您想构建一个可用于pip安装的PyFlink包，您需要先构建Flink的Jar包，如[构建Flink](##Build Flink)中所述。
+之后，进入Flink源码根目录，并执行以下命令，构建PyFlink的源码发布包和wheel包：
+
+{% highlight bash %}
+cd flink-python; python3 setup.py sdist bdist_wheel
+{% endhighlight %}
+
+构建好的源码发布包和wheel包位于`./flink-python/dist/`目录下。它们均可使用pip安装,比如:
+
+{% highlight bash %}
+pip install dist/*.tar.gz
+{% endhighlight %}
+
## Dependency Shading
Flink [shades away](https://maven.apache.org/plugins/maven-shade-plugin/) some
of the libraries it uses, in order to avoid version clashes with user programs
that use different versions of these libraries. Among the shaded libraries are
*Google Guava*, *Asm*, *Apache Curator*, *Apache HTTP Components*, *Netty*, and
others.
diff --git a/flink-dist/src/main/flink-bin/bin/config.sh
b/flink-dist/src/main/flink-bin/bin/config.sh
index 79fa6ad..4dc57df 100755
--- a/flink-dist/src/main/flink-bin/bin/config.sh
+++ b/flink-dist/src/main/flink-bin/bin/config.sh
@@ -296,7 +296,10 @@ bin=`dirname "$target"`
SYMLINK_RESOLVED_BIN=`cd "$bin"; pwd -P`
# Define the main directory of the flink installation
-FLINK_HOME=`dirname "$SYMLINK_RESOLVED_BIN"`
+# If config.sh is called by pyflink-shell.sh in python bin directory(pip
installed), then do not need to set the FLINK_HOME here.
+if [ -z "$_FLINK_HOME_DETERMINED" ]; then
+ FLINK_HOME=`dirname "$SYMLINK_RESOLVED_BIN"`
+fi
FLINK_LIB_DIR=$FLINK_HOME/lib
FLINK_PLUGINS_DIR=$FLINK_HOME/plugins
FLINK_OPT_DIR=$FLINK_HOME/opt
diff --git a/flink-python/pyflink/version.py
b/flink-dist/src/main/flink-bin/bin/find-flink-home.sh
old mode 100644
new mode 100755
similarity index 72%
copy from flink-python/pyflink/version.py
copy to flink-dist/src/main/flink-bin/bin/find-flink-home.sh
index ca27a42..e0fe95f
--- a/flink-python/pyflink/version.py
+++ b/flink-dist/src/main/flink-bin/bin/find-flink-home.sh
@@ -1,3 +1,4 @@
+#!/usr/bin/env bash
################################################################################
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
@@ -16,4 +17,12 @@
# limitations under the License.
################################################################################
-__version__ = "0.1.0"
+CURRENT_DIR="$( cd "$(dirname "$0")" ; pwd -P )"
+FIND_FLINK_HOME_PYTHON_SCRIPT="$CURRENT_DIR/find_flink_home.py"
+
+if [ ! -f "$FIND_FLINK_HOME_PYTHON_SCRIPT" ]; then
+ export FLINK_HOME="$( cd "$CURRENT_DIR"/.. ; pwd -P )"
+else
+ PYFLINK_PYTHON="${PYFLINK_PYTHON:-"python"}"
+ export FLINK_HOME=$("$FIND_FLINK_HOME_PYTHON_SCRIPT")
+fi
diff --git a/flink-dist/src/main/flink-bin/bin/pyflink-shell.sh
b/flink-dist/src/main/flink-bin/bin/pyflink-shell.sh
index 577952b..6bbc942 100755
--- a/flink-dist/src/main/flink-bin/bin/pyflink-shell.sh
+++ b/flink-dist/src/main/flink-bin/bin/pyflink-shell.sh
@@ -18,8 +18,11 @@
################################################################################
bin=`dirname "$0"`
bin=`cd "$bin"; pwd`
+. "$bin"/find-flink-home.sh
-. "$bin"/config.sh
+_FLINK_HOME_DETERMINED=1
+
+. "$FLINK_HOME"/bin/config.sh
FLINK_CLASSPATH=`constructFlinkClassPath`
PYTHON_JAR_PATH=`echo "$FLINK_OPT_DIR"/flink-python*java-binding.jar`
diff --git a/flink-python/pyflink/version.py b/flink-python/MANIFEST.in
similarity index 75%
copy from flink-python/pyflink/version.py
copy to flink-python/MANIFEST.in
index ca27a42..a6e3bf5 100644
--- a/flink-python/pyflink/version.py
+++ b/flink-python/MANIFEST.in
@@ -16,4 +16,16 @@
# limitations under the License.
################################################################################
-__version__ = "0.1.0"
+global-exclude *.py[cod] __pycache__ .DS_Store
+graft deps/lib
+recursive-include deps/opt *
+recursive-include deps/plugins *
+graft deps/bin
+graft deps/conf
+graft deps/log
+recursive-include deps/examples *.py
+graft deps/licenses
+include README.md
+include pyflink/LICENSE
+include pyflink/NOTICE
+include pyflink/README.txt
diff --git a/flink-python/dev/pip_test_code.py
b/flink-python/dev/pip_test_code.py
new file mode 100644
index 0000000..c9a4798
--- /dev/null
+++ b/flink-python/dev/pip_test_code.py
@@ -0,0 +1,53 @@
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+# test pyflink shell environment
+from pyflink.shell import bt_env, FileSystem, OldCsv, DataTypes, Schema
+
+import tempfile
+import os
+import shutil
+
+sink_path = tempfile.gettempdir() + '/batch.csv'
+if os.path.exists(sink_path):
+ if os.path.isfile(sink_path):
+ os.remove(sink_path)
+ else:
+ shutil.rmtree(sink_path)
+bt_env.exec_env().set_parallelism(1)
+t = bt_env.from_elements([(1, 'hi', 'hello'), (2, 'hi', 'hello')], ['a', 'b',
'c'])
+bt_env.connect(FileSystem().path(sink_path)) \
+ .with_format(OldCsv()
+ .field_delimiter(',')
+ .field("a", DataTypes.BIGINT())
+ .field("b", DataTypes.STRING())
+ .field("c", DataTypes.STRING())) \
+ .with_schema(Schema()
+ .field("a", DataTypes.BIGINT())
+ .field("b", DataTypes.STRING())
+ .field("c", DataTypes.STRING())) \
+ .register_table_sink("batch_sink")
+
+t.select("a + 1, b, c").insert_into("batch_sink")
+
+bt_env.exec_env().execute()
+
+with open(sink_path, 'r') as f:
+ lines = f.read()
+ assert lines == '2,hi,hello\n' + '3,hi,hello\n'
+
+print('pip_test_code.py success!')
diff --git a/flink-python/pyflink/version.py b/flink-python/dev/run_pip_test.sh
old mode 100644
new mode 100755
similarity index 93%
copy from flink-python/pyflink/version.py
copy to flink-python/dev/run_pip_test.sh
index ca27a42..0b674aa
--- a/flink-python/pyflink/version.py
+++ b/flink-python/dev/run_pip_test.sh
@@ -1,3 +1,4 @@
+#!/usr/bin/env bash
################################################################################
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
@@ -16,4 +17,6 @@
# limitations under the License.
################################################################################
-__version__ = "0.1.0"
+cd "$( dirname "$0" )"
+
+python pip_test_code.py
diff --git a/flink-python/pyflink/find_flink_home.py
b/flink-python/pyflink/find_flink_home.py
index 0491368..b75cc29 100644
--- a/flink-python/pyflink/find_flink_home.py
+++ b/flink-python/pyflink/find_flink_home.py
@@ -1,4 +1,5 @@
-################################################################################
+#!/usr/bin/env python
+#################################################################################
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
@@ -15,12 +16,21 @@
# See the License for the specific language governing permissions and
# limitations under the License.
################################################################################
-
+import glob
import logging
import os
import sys
+def _is_flink_home(path):
+ pyflink_file = path + "/bin/pyflink-gateway-server.sh"
+ flink_dist_jar_file = path + "/lib/flink-dist*.jar"
+ if os.path.isfile(pyflink_file) and len(glob.glob(flink_dist_jar_file)) >
0:
+ return True
+ else:
+ return False
+
+
def _find_flink_home():
"""
Find the FLINK_HOME.
@@ -30,12 +40,23 @@ def _find_flink_home():
return os.environ['FLINK_HOME']
else:
try:
- flink_root_dir =
os.path.abspath(os.path.dirname(os.path.abspath(__file__)) + "/../../")
+ current_dir =
os.path.abspath(os.path.dirname(os.path.abspath(__file__)))
+ flink_root_dir = os.path.abspath(current_dir + "/../../")
build_target = flink_root_dir + "/build-target"
- pyflink_file = build_target + "/bin/pyflink-gateway-server.sh"
- if os.path.isfile(pyflink_file):
+ if _is_flink_home(build_target):
os.environ['FLINK_HOME'] = build_target
return build_target
+
+ if sys.version < "3":
+ import imp
+ module_home = imp.find_module("pyflink")[1]
+ else:
+ from importlib.util import find_spec
+ module_home = os.path.dirname(find_spec("pyflink").origin)
+
+ if _is_flink_home(module_home):
+ os.environ['FLINK_HOME'] = module_home
+ return module_home
except Exception:
pass
logging.error("Could not find valid FLINK_HOME(Flink distribution
directory) "
diff --git a/flink-python/pyflink/shell.py b/flink-python/pyflink/shell.py
index 9822c30..2ea3a6b 100644
--- a/flink-python/pyflink/shell.py
+++ b/flink-python/pyflink/shell.py
@@ -1,3 +1,4 @@
+#!/usr/bin/env python
################################################################################
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
@@ -15,7 +16,9 @@
# See the License for the specific language governing permissions and
# limitations under the License.
################################################################################
+import codecs
import platform
+import sys
from pyflink.dataset import ExecutionEnvironment
from pyflink.datastream import StreamExecutionEnvironment
@@ -24,6 +27,11 @@ from pyflink.table.catalog import *
from pyflink.table.descriptors import *
from pyflink.table.window import *
+if sys.version > '3':
+ utf8_out = open(sys.stdout.fileno(), mode='w', encoding='utf8',
buffering=1)
+else:
+ utf8_out = codecs.getwriter("utf-8")(sys.stdout)
+
print("Using Python version %s (%s, %s)" % (
platform.python_version(),
platform.python_build()[0],
@@ -131,8 +139,8 @@ NOTE: Use the prebound Table Environment to implement batch
or streaming Table p
* t.select("a + 1, b, c").insert_into("stream_sink")
*
* st_env.exec_env().execute()
- '''
-print(welcome_msg)
+'''
+utf8_out.write(welcome_msg)
bt_env =
BatchTableEnvironment.create(ExecutionEnvironment.get_execution_environment())
diff --git a/flink-python/pyflink/version.py b/flink-python/pyflink/version.py
index ca27a42..1d1fb46 100644
--- a/flink-python/pyflink/version.py
+++ b/flink-python/pyflink/version.py
@@ -16,4 +16,8 @@
# limitations under the License.
################################################################################
-__version__ = "0.1.0"
+"""
+The pyflink version will be consistent with the flink version and follow the
PEP440.
+.. seealso:: https://www.python.org/dev/peps/pep-0440
+"""
+__version__ = "1.9.dev0"
diff --git a/flink-python/pyflink/version.py b/flink-python/setup.cfg
similarity index 93%
copy from flink-python/pyflink/version.py
copy to flink-python/setup.cfg
index ca27a42..138d2bd 100644
--- a/flink-python/pyflink/version.py
+++ b/flink-python/setup.cfg
@@ -16,4 +16,8 @@
# limitations under the License.
################################################################################
-__version__ = "0.1.0"
+[bdist_wheel]
+universal = 1
+
+[metadata]
+description-file = README.md
diff --git a/flink-python/setup.py b/flink-python/setup.py
index 906ebf9..ec7153e 100644
--- a/flink-python/setup.py
+++ b/flink-python/setup.py
@@ -20,6 +20,8 @@ from __future__ import print_function
import io
import os
import sys
+from shutil import copytree, copy, rmtree
+
from setuptools import setup
if sys.version_info < (2, 7):
@@ -42,31 +44,183 @@ VERSION = __version__ # noqa
with io.open(os.path.join(this_directory, 'README.md'), 'r', encoding='utf-8')
as f:
long_description = f.read()
-setup(
- name='pyflink',
- version=VERSION,
- packages=['pyflink',
- 'pyflink.table',
- 'pyflink.util',
- 'pyflink.datastream',
- 'pyflink.dataset',
- 'pyflink.common'],
- url='http://flink.apache.org',
- license='http://www.apache.org/licenses/LICENSE-2.0',
- author='Flink Developers',
- author_email='d...@flink.apache.org',
- install_requires=['py4j==0.10.8.1', 'python-dateutil'],
- tests_require=['pytest==4.4.1'],
- description='Apache Flink Python API',
- long_description=long_description,
- long_description_content_type='text/markdown',
- classifiers=[
- 'Development Status :: 1 - Planning',
- 'License :: OSI Approved :: Apache Software License',
- 'Programming Language :: Python :: 2.7',
- 'Programming Language :: Python :: 3.3',
- 'Programming Language :: Python :: 3.4',
- 'Programming Language :: Python :: 3.5',
- 'Programming Language :: Python :: 3.6',
- 'Programming Language :: Python :: 3.7']
-)
+TEMP_PATH = "deps"
+
+LIB_TEMP_PATH = os.path.join(TEMP_PATH, "lib")
+OPT_TEMP_PATH = os.path.join(TEMP_PATH, "opt")
+CONF_TEMP_PATH = os.path.join(TEMP_PATH, "conf")
+LOG_TEMP_PATH = os.path.join(TEMP_PATH, "log")
+EXAMPLES_TEMP_PATH = os.path.join(TEMP_PATH, "examples")
+LICENSES_TEMP_PATH = os.path.join(TEMP_PATH, "licenses")
+PLUGINS_TEMP_PATH = os.path.join(TEMP_PATH, "plugins")
+SCRIPTS_TEMP_PATH = os.path.join(TEMP_PATH, "bin")
+
+LICENSE_FILE_TEMP_PATH = os.path.join("pyflink", "LICENSE")
+NOTICE_FILE_TEMP_PATH = os.path.join("pyflink", "NOTICE")
+README_FILE_TEMP_PATH = os.path.join("pyflink", "README.txt")
+
+in_flink_source =
os.path.isfile("../flink-java/src/main/java/org/apache/flink/api/java/"
+ "ExecutionEnvironment.java")
+
+try:
+ if in_flink_source:
+
+ try:
+ os.mkdir(TEMP_PATH)
+ except:
+ print("Temp path for symlink to parent already exists
{0}".format(TEMP_PATH),
+ file=sys.stderr)
+ sys.exit(-1)
+
+ FLINK_HOME = os.path.abspath("../build-target")
+
+ incorrect_invocation_message = """
+If you are installing pyflink from flink source, you must first build Flink and
+run sdist.
+
+ To build Flink with maven you can run:
+ mvn -DskipTests clean package
+ Building the source dist is done in the flink-python directory:
+ cd flink-python
+ python setup.py sdist
+ pip install dist/*.tar.gz"""
+
+ LIB_PATH = os.path.join(FLINK_HOME, "lib")
+ OPT_PATH = os.path.join(FLINK_HOME, "opt")
+ CONF_PATH = os.path.join(FLINK_HOME, "conf")
+ EXAMPLES_PATH = os.path.join(FLINK_HOME, "examples")
+ LICENSES_PATH = os.path.join(FLINK_HOME, "licenses")
+ PLUGINS_PATH = os.path.join(FLINK_HOME, "plugins")
+ SCRIPTS_PATH = os.path.join(FLINK_HOME, "bin")
+
+ LICENSE_FILE_PATH = os.path.join(FLINK_HOME, "LICENSE")
+ NOTICE_FILE_PATH = os.path.join(FLINK_HOME, "NOTICE")
+ README_FILE_PATH = os.path.join(FLINK_HOME, "README.txt")
+
+ if not os.path.isdir(LIB_PATH):
+ print(incorrect_invocation_message, file=sys.stderr)
+ sys.exit(-1)
+
+ if getattr(os, "symlink", None) is not None:
+ os.symlink(LIB_PATH, LIB_TEMP_PATH)
+ os.symlink(OPT_PATH, OPT_TEMP_PATH)
+ os.symlink(CONF_PATH, CONF_TEMP_PATH)
+ os.symlink(EXAMPLES_PATH, EXAMPLES_TEMP_PATH)
+ os.symlink(LICENSES_PATH, LICENSES_TEMP_PATH)
+ os.symlink(PLUGINS_PATH, PLUGINS_TEMP_PATH)
+ os.symlink(SCRIPTS_PATH, SCRIPTS_TEMP_PATH)
+ os.symlink(LICENSE_FILE_PATH, LICENSE_FILE_TEMP_PATH)
+ os.symlink(NOTICE_FILE_PATH, NOTICE_FILE_TEMP_PATH)
+ os.symlink(README_FILE_PATH, README_FILE_TEMP_PATH)
+ else:
+ copytree(LIB_PATH, LIB_TEMP_PATH)
+ copytree(OPT_PATH, OPT_TEMP_PATH)
+ copytree(CONF_PATH, CONF_TEMP_PATH)
+ copytree(EXAMPLES_PATH, EXAMPLES_TEMP_PATH)
+ copytree(LICENSES_PATH, LICENSES_TEMP_PATH)
+ copytree(PLUGINS_PATH, PLUGINS_TEMP_PATH)
+ copytree(SCRIPTS_PATH, SCRIPTS_TEMP_PATH)
+ copy(LICENSE_FILE_PATH, LICENSE_FILE_TEMP_PATH)
+ copy(NOTICE_FILE_PATH, NOTICE_FILE_TEMP_PATH)
+ copy(README_FILE_PATH, README_FILE_TEMP_PATH)
+ os.mkdir(LOG_TEMP_PATH)
+ with open(os.path.join(LOG_TEMP_PATH, "empty.txt"), 'w') as f:
+ f.write("This file is used to force setuptools to include the log
directory. "
+ "You can delete it at any time after installation.")
+ else:
+ if not os.path.isdir(LIB_TEMP_PATH) or not
os.path.isdir(OPT_TEMP_PATH) \
+ or not os.path.isdir(SCRIPTS_TEMP_PATH):
+ print("The flink core files are not found. Please make sure your
installation package "
+ "is complete, or do this in the flink-python directory of
the flink source "
+ "directory.")
+ sys.exit(-1)
+
+ script_names = ["pyflink-shell.sh", "find-flink-home.sh"]
+ scripts = [os.path.join(SCRIPTS_TEMP_PATH, script) for script in
script_names]
+ scripts.append("pyflink/find_flink_home.py")
+
+ setup(
+ name='pyflink',
+ version=VERSION,
+ packages=['pyflink',
+ 'pyflink.table',
+ 'pyflink.util',
+ 'pyflink.datastream',
+ 'pyflink.dataset',
+ 'pyflink.common',
+ 'pyflink.lib',
+ 'pyflink.opt',
+ 'pyflink.conf',
+ 'pyflink.log',
+ 'pyflink.examples',
+ 'pyflink.licenses',
+ 'pyflink.plugins',
+ 'pyflink.bin'],
+ include_package_data=True,
+ package_dir={
+ 'pyflink.lib': TEMP_PATH + '/lib',
+ 'pyflink.opt': TEMP_PATH + '/opt',
+ 'pyflink.conf': TEMP_PATH + '/conf',
+ 'pyflink.log': TEMP_PATH + '/log',
+ 'pyflink.examples': TEMP_PATH + '/examples',
+ 'pyflink.licenses': TEMP_PATH + '/licenses',
+ 'pyflink.plugins': TEMP_PATH + '/plugins',
+ 'pyflink.bin': TEMP_PATH + '/bin'
+ },
+ package_data={
+ 'pyflink': ['LICENSE', 'NOTICE', 'README.txt'],
+ 'pyflink.lib': ['*.jar'],
+ 'pyflink.opt': ['*', '*/*'],
+ 'pyflink.conf': ['*'],
+ 'pyflink.log': ['*'],
+ 'pyflink.examples': ['*.py', '*/*.py'],
+ 'pyflink.licenses': ['*'],
+ 'pyflink.plugins': ['*', '*/*'],
+ 'pyflink.bin': ['*']
+ },
+ scripts=scripts,
+ url='http://flink.apache.org',
+ license='http://www.apache.org/licenses/LICENSE-2.0',
+ author='Flink Developers',
+ author_email='d...@flink.apache.org',
+ install_requires=['py4j==0.10.8.1', 'python-dateutil'],
+ tests_require=['pytest==4.4.1'],
+ description='Apache Flink Python API',
+ long_description=long_description,
+ long_description_content_type='text/markdown',
+ classifiers=[
+ 'Development Status :: 1 - Planning',
+ 'License :: OSI Approved :: Apache Software License',
+ 'Programming Language :: Python :: 2.7',
+ 'Programming Language :: Python :: 3.3',
+ 'Programming Language :: Python :: 3.4',
+ 'Programming Language :: Python :: 3.5',
+ 'Programming Language :: Python :: 3.6',
+ 'Programming Language :: Python :: 3.7']
+ )
+finally:
+ if in_flink_source:
+ if getattr(os, "symlink", None) is not None:
+ os.remove(LIB_TEMP_PATH)
+ os.remove(OPT_TEMP_PATH)
+ os.remove(CONF_TEMP_PATH)
+ os.remove(EXAMPLES_TEMP_PATH)
+ os.remove(LICENSES_TEMP_PATH)
+ os.remove(PLUGINS_TEMP_PATH)
+ os.remove(SCRIPTS_TEMP_PATH)
+ os.remove(LICENSE_FILE_TEMP_PATH)
+ os.remove(NOTICE_FILE_TEMP_PATH)
+ os.remove(README_FILE_TEMP_PATH)
+ else:
+ rmtree(LIB_TEMP_PATH)
+ rmtree(OPT_TEMP_PATH)
+ rmtree(CONF_TEMP_PATH)
+ rmtree(EXAMPLES_TEMP_PATH)
+ rmtree(LICENSES_TEMP_PATH)
+ rmtree(PLUGINS_TEMP_PATH)
+ rmtree(SCRIPTS_TEMP_PATH)
+ os.remove(LICENSE_FILE_TEMP_PATH)
+ os.remove(NOTICE_FILE_TEMP_PATH)
+ os.remove(README_FILE_TEMP_PATH)
+ rmtree(LOG_TEMP_PATH)
+ os.rmdir(TEMP_PATH)
diff --git a/flink-python/tox.ini b/flink-python/tox.ini
index dbfc4e9..8c56f8f 100644
--- a/flink-python/tox.ini
+++ b/flink-python/tox.ini
@@ -24,12 +24,15 @@
envlist = py27, py33, py34, py35, py36, py37
[testenv]
+whitelist_externals=
+ /bin/bash
deps =
pytest
commands =
python --version
python setup.py install --force
pytest
+ bash ./dev/run_pip_test.sh
[flake8]
# We follow PEP 8 (https://www.python.org/dev/peps/pep-0008/) with one
exception: lines can be
diff --git a/tools/releasing/update_branch_version.sh
b/tools/releasing/update_branch_version.sh
index 6f00093..bc0a0c4 100755
--- a/tools/releasing/update_branch_version.sh
+++ b/tools/releasing/update_branch_version.sh
@@ -57,6 +57,12 @@ perl -pi -e "s#^version: .*#version: \"${NEW_VERSION}\"#"
_config.yml
perl -pi -e "s#^version_title: .*#version_title: \"${NEW_VERSION}\"#"
_config.yml
cd ..
+#change version of pyflink
+cd flink-python/pyflink
+perl -pi -e "s#^__version__ = \".*\"#__version__ = \"${NEW_VERSION}\"#"
version.py
+perl -pi -e "s#-SNAPSHOT#\\.dev0#" version.py
+cd ../..
+
git commit -am "Update version to $NEW_VERSION"
echo "Don't forget to push the change."
