[jira] [Comment Edited] (FLINK-16026) Travis failed due to python setup
[ https://issues.apache.org/jira/browse/FLINK-16026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036656#comment-17036656 ] Dian Fu edited comment on FLINK-16026 at 2/14/20 2:53 AM: -- [~chesnay] Thanks a lot for your suggestions. Appreciated! I have investigated this issue with [~sunjincheng121] offline and want to share our thoughts as following: - It's a good idea to limit the versions of the direct dependencies and I think we have already done that in PyFlink. - It may be not a good idea to limit the versions of the transitive dependencies to a different version declared by the direct dependency: -- We should trust the direct dependency, e.g. if apache-beam declares that it support avro-python3<2, we should trust that it does support all versions of avro-python3 which are less than 2. -- We should not change the range of versions of the transitive dependencies declared by the direct dependency, e.g. if apache-beam declares that it supports avro-python3<2, it may not be a good idea to change the version limit to avro-python3<1.9.1 in PyFlink. This will introduce a lot of issues: --- We have to upgrade the versions of the transitive dependencies periodically if new packages have been released --- For released package of PyFlink, there is no way to upgrade the versions of the transitive dependencies any more -- If there are exceptions for the above cases, we could address them case by case(just like this JIRA does). For example, regarding to the issue of this JIRA, it's because that avro community has released an error package. We could just disable it for the time being. This is a trade off, however, I guess most Python projects handle this kind of issues in this way. What's your thoughts? was (Author: dian.fu): [~chesnay] Thanks a lot for your suggestions. Appreciated! I have investigated this issue with [~sunjincheng121] offline and want to share our thoughts as following: - It's a good idea to limit the versions of the direct dependencies and I think we have already done that in PyFlink. - It may be not a good idea to limit the versions of the transitive dependencies to a different version declared by the direct dependency: -- We should trust the direct dependency, e.g. if apache-beam declares that it support avro-python3<2, we should trust that it does support all versions of avro-python3 which are less than 2. -- We should not change the range of versions of the transitive dependencies declared by the direct dependency, e.g. if apache-beam declares that it supports avro-python3<2, it may not be a good idea to change the version limit to avro-python3<1.9.1 in PyFlink. This will introduce a lot of issues: --- We have to upgrade the versions of the transitive dependencies periodically if new packages have been released --- For released package of PyFlink, there is no way to upgrade the versions of the transitive dependencies any more -- If there are exceptions for the above cases, we could address them case by case. For example, regarding to the issue of this JIRA, it's because that avro community has released an error package. We could just disable it for the time being. This is a trade off, however, I guess most Python projects handle this kind of issues in this way. What's your thoughts? > Travis failed due to python setup > - > > Key: FLINK-16026 > URL: https://issues.apache.org/jira/browse/FLINK-16026 > Project: Flink > Issue Type: Bug > Components: API / Python >Reporter: Jingsong Lee >Assignee: Huang Xingbo >Priority: Critical > Labels: pull-request-available > Fix For: 1.10.1, 1.11.0 > > Time Spent: 20m > Remaining Estimate: 0h > > [https://api.travis-ci.com/v3/job/286671652/log.txt] > [https://api.travis-ci.org/v3/job/649754603/log.txt] > [https://api.travis-ci.com/v3/job/286409130/log.txt] > Collecting avro-python3<2.0.0,>=1.8.1; python_version >= "3.0" (from > apache-beam==2.19.0->apache-flink==1.11.dev0) Using cached > https://files.pythonhosted.org/packages/31/21/d98e2515e5ca0337d7e747e8065227ee77faf5c817bbb74391899613178a/avro-python3-1.9.2.tar.gz > Complete output from command python setup.py egg_info: Traceback (most > recent call last): File "", line 1, in File > "/tmp/pip-install-d6uvsl_b/avro-python3/setup.py", line 41, in > import pycodestyle ModuleNotFoundError: No module named 'pycodestyle' > Command "python setup.py egg_info" > failed with error code 1 in /tmp/pip-install-d6uvsl_b/avro-python3/ You are > using pip version 10.0.1, however version 20.0.2 is available. You should > consider upgrading via the 'pip install --upgrade pip' command. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (FLINK-16026) Travis failed due to python setup
[ https://issues.apache.org/jira/browse/FLINK-16026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036656#comment-17036656 ] Dian Fu edited comment on FLINK-16026 at 2/14/20 2:52 AM: -- [~chesnay] Thanks a lot for your suggestions. Appreciated! I have investigated this issue with [~sunjincheng121] offline and want to share our thoughts as following: - It's a good idea to limit the versions of the direct dependencies and I think we have already done that in PyFlink. - It may be not a good idea to limit the versions of the transitive dependencies to a different version declared by the direct dependency: -- We should trust the direct dependency, e.g. if apache-beam declares that it support avro-python3<2, we should trust that it does support all versions of avro-python3 which are less than 2. -- We should not change the range of versions of the transitive dependencies declared by the direct dependency, e.g. if apache-beam declares that it supports avro-python3<2, it may not be a good idea to change the version limit to avro-python3<1.9.1 in PyFlink. This will introduce a lot of issues: --- We have to upgrade the versions of the transitive dependencies periodically if new packages have been released --- For released package of PyFlink, there is no way to upgrade the versions of the transitive dependencies any more -- If there are exceptions for the above cases, we could address them case by case. For example, regarding to the issue of this JIRA, it's because that avro community has released an error package. We could just disable it for the time being. This is a trade off, however, I guess most Python projects handle this kind of issues in this way. What's your thoughts? was (Author: dian.fu): [~chesnay] Thanks a lot for your suggestions. Appreciated! We have investigated this issue and want to share our thoughts as following: - It's a good idea to limit the versions of the direct dependencies and I think we have already done that in PyFlink. - It may be not a good idea to limit the versions of the transitive dependencies to a different version declared by the direct dependency: -- We should trust the direct dependency, e.g. if apache-beam declares that it support avro-python3<2, we should trust that it does support all versions of avro-python3 which are less than 2. -- We should not change the range of versions of the transitive dependencies declared by the direct dependency, e.g. if apache-beam declares that it supports avro-python3<2, it may not be a good idea to change the version limit to avro-python3<1.9.1 in PyFlink. This will introduce a lot of issues: --- We have to upgrade the versions of the transitive dependencies periodically if new packages have been released --- For released package of PyFlink, there is no way to upgrade the versions of the transitive dependencies any more -- If there are exceptions for the above cases, we could address them case by case. For example, regarding to the issue of this JIRA, it's because that avro community has released an error package. We could just disable it for the time being. This is a trade off, however, I guess most Python projects handle this kind of issues in this way. What's your thoughts? > Travis failed due to python setup > - > > Key: FLINK-16026 > URL: https://issues.apache.org/jira/browse/FLINK-16026 > Project: Flink > Issue Type: Bug > Components: API / Python >Reporter: Jingsong Lee >Assignee: Huang Xingbo >Priority: Critical > Labels: pull-request-available > Fix For: 1.10.1, 1.11.0 > > Time Spent: 20m > Remaining Estimate: 0h > > [https://api.travis-ci.com/v3/job/286671652/log.txt] > [https://api.travis-ci.org/v3/job/649754603/log.txt] > [https://api.travis-ci.com/v3/job/286409130/log.txt] > Collecting avro-python3<2.0.0,>=1.8.1; python_version >= "3.0" (from > apache-beam==2.19.0->apache-flink==1.11.dev0) Using cached > https://files.pythonhosted.org/packages/31/21/d98e2515e5ca0337d7e747e8065227ee77faf5c817bbb74391899613178a/avro-python3-1.9.2.tar.gz > Complete output from command python setup.py egg_info: Traceback (most > recent call last): File "", line 1, in File > "/tmp/pip-install-d6uvsl_b/avro-python3/setup.py", line 41, in > import pycodestyle ModuleNotFoundError: No module named 'pycodestyle' > Command "python setup.py egg_info" > failed with error code 1 in /tmp/pip-install-d6uvsl_b/avro-python3/ You are > using pip version 10.0.1, however version 20.0.2 is available. You should > consider upgrading via the 'pip install --upgrade pip' command. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (FLINK-16026) Travis failed due to python setup
[ https://issues.apache.org/jira/browse/FLINK-16026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036656#comment-17036656 ] Dian Fu edited comment on FLINK-16026 at 2/14/20 2:51 AM: -- [~chesnay] Thanks a lot for your suggestions. Appreciated! We have investigated this issue and want to share our thoughts as following: - It's a good idea to limit the versions of the direct dependencies and I think we have already done that in PyFlink. - It may be not a good idea to limit the versions of the transitive dependencies to a different version declared by the direct dependency: -- We should trust the direct dependency, e.g. if apache-beam declares that it support avro-python3<2, we should trust that it does support all versions of avro-python3 which are less than 2. -- We should not change the range of versions of the transitive dependencies declared by the direct dependency, e.g. if apache-beam declares that it supports avro-python3<2, it may not be a good idea to change the version limit to avro-python3<1.9.1 in PyFlink. This will introduce a lot of issues: --- We have to upgrade the versions of the transitive dependencies periodically if new packages have been released --- For released package of PyFlink, there is no way to upgrade the versions of the transitive dependencies any more -- If there are exceptions for the above cases, we could address them case by case. For example, regarding to the issue of this JIRA, it's because that avro community has released an error package. We could just disable it for the time being. This is a trade off, however, I guess most Python projects handle this kind of issues in this way. What's your thoughts? was (Author: dian.fu): [~chesnay] Thanks a lot for your suggestions. Appreciated! We have investigated this issue and want to share our thoughts as following: - It's a good idea to limit the versions of the direct dependencies and I think we have already done that in PyFlink. - It may be not a good idea to limit the versions of the transitive dependencies to a different version declared by the direct dependency: 1. We should trust the direct dependency, e.g. if apache-beam declares that it support avro-python3<2, we should trust that it does support all versions of avro-python3 which are less than 2. 2. We should not change the range of versions of the transitive dependencies declared by the direct dependency, e.g. if apache-beam declares that it supports avro-python3<2, it may not be a good idea to change the version limit to avro-python3<1.9.1 in PyFlink. This will introduce a lot of issues: 1) We have to upgrade the versions of the transitive dependencies periodically if new packages have been released 2) For released package of PyFlink, there is no way to upgrade the versions of the transitive dependencies any more 3. If there are exceptions for the above cases, we could address them case by case. For example, regarding to the issue of this JIRA, it's because that avro community has released an error package. We could just disable it for the time being. This is a trade off, however, I guess most Python projects handle this kind of issues in this way. What's your thoughts? > Travis failed due to python setup > - > > Key: FLINK-16026 > URL: https://issues.apache.org/jira/browse/FLINK-16026 > Project: Flink > Issue Type: Bug > Components: API / Python >Reporter: Jingsong Lee >Assignee: Huang Xingbo >Priority: Critical > Labels: pull-request-available > Fix For: 1.10.1, 1.11.0 > > Time Spent: 20m > Remaining Estimate: 0h > > [https://api.travis-ci.com/v3/job/286671652/log.txt] > [https://api.travis-ci.org/v3/job/649754603/log.txt] > [https://api.travis-ci.com/v3/job/286409130/log.txt] > Collecting avro-python3<2.0.0,>=1.8.1; python_version >= "3.0" (from > apache-beam==2.19.0->apache-flink==1.11.dev0) Using cached > https://files.pythonhosted.org/packages/31/21/d98e2515e5ca0337d7e747e8065227ee77faf5c817bbb74391899613178a/avro-python3-1.9.2.tar.gz > Complete output from command python setup.py egg_info: Traceback (most > recent call last): File "", line 1, in File > "/tmp/pip-install-d6uvsl_b/avro-python3/setup.py", line 41, in > import pycodestyle ModuleNotFoundError: No module named 'pycodestyle' > Command "python setup.py egg_info" > failed with error code 1 in /tmp/pip-install-d6uvsl_b/avro-python3/ You are > using pip version 10.0.1, however version 20.0.2 is available. You should > consider upgrading via the 'pip install --upgrade pip' command. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (FLINK-16026) Travis failed due to python setup
[ https://issues.apache.org/jira/browse/FLINK-16026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036088#comment-17036088 ] sunjincheng edited comment on FLINK-16026 at 2/13/20 9:54 AM: -- Thanks for the reminder [~chesnay]. We have offline discussed about initializing all current dependencies to a fixed version before this patch. The current patch is a quick fix for Travis. Will feedback here when have the final conclusion. was (Author: sunjincheng121): Thanks for the reminder [~chesnay]. We have discussed about initializing all current dependencies to a fixed version before this patch. The current patch is a quick fix for Travis. Will feedback here when have the final conclusion. > Travis failed due to python setup > - > > Key: FLINK-16026 > URL: https://issues.apache.org/jira/browse/FLINK-16026 > Project: Flink > Issue Type: Bug > Components: API / Python >Reporter: Jingsong Lee >Assignee: Huang Xingbo >Priority: Critical > Labels: pull-request-available > Fix For: 1.10.1, 1.11.0 > > Time Spent: 20m > Remaining Estimate: 0h > > [https://api.travis-ci.com/v3/job/286671652/log.txt] > [https://api.travis-ci.org/v3/job/649754603/log.txt] > [https://api.travis-ci.com/v3/job/286409130/log.txt] > Collecting avro-python3<2.0.0,>=1.8.1; python_version >= "3.0" (from > apache-beam==2.19.0->apache-flink==1.11.dev0) Using cached > https://files.pythonhosted.org/packages/31/21/d98e2515e5ca0337d7e747e8065227ee77faf5c817bbb74391899613178a/avro-python3-1.9.2.tar.gz > Complete output from command python setup.py egg_info: Traceback (most > recent call last): File "", line 1, in File > "/tmp/pip-install-d6uvsl_b/avro-python3/setup.py", line 41, in > import pycodestyle ModuleNotFoundError: No module named 'pycodestyle' > Command "python setup.py egg_info" > failed with error code 1 in /tmp/pip-install-d6uvsl_b/avro-python3/ You are > using pip version 10.0.1, however version 20.0.2 is available. You should > consider upgrading via the 'pip install --upgrade pip' command. -- This message was sent by Atlassian Jira (v8.3.4#803005)
