Repository: beam Updated Branches: refs/heads/master 655227ab0 -> 9c2a6e771
Add futurize to pylint script Break down futurize pylint Fix the pylint script checking futurize Consists of: Quote the futurize options shell scripting Jenkins seems to be having weird behaviour with futurize_filtered=, temporary debugging commit Try quoting the count line. Do the style fixes from shellcheck on pylint, parallilize futurize command & simplify the grep on output Fall through on grep exit code of 1 which happens when everything is filtered out & change count check to 0 Remove some debugging from pylint Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/dd6a7899 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/dd6a7899 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/dd6a7899 Branch: refs/heads/master Commit: dd6a78992fb66e49ff53ae69114f0d682be0888f Parents: 22f9263 Author: Holden Karau <hol...@us.ibm.com> Authored: Fri Sep 1 19:53:42 2017 -0700 Committer: Robert Bradshaw <rober...@gmail.com> Committed: Thu Oct 12 15:50:07 2017 -0700 ---------------------------------------------------------------------- sdks/python/run_pylint.sh | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/beam/blob/dd6a7899/sdks/python/run_pylint.sh ---------------------------------------------------------------------- diff --git a/sdks/python/run_pylint.sh b/sdks/python/run_pylint.sh index df71e44..d53bb14 100755 --- a/sdks/python/run_pylint.sh +++ b/sdks/python/run_pylint.sh @@ -33,7 +33,7 @@ usage(){ echo "Usage: $0 [MODULE|--help] # The default MODULE is $MODULE"; } if test $# -gt 0; then case "$@" in --help) usage; exit 1;; - *) MODULE="$@";; + *) MODULE="$*";; esac fi @@ -59,15 +59,16 @@ done echo "Skipping lint for generated files: $FILES_TO_IGNORE" echo "Running pylint for module $MODULE:" -pylint $MODULE --ignore-patterns="$FILES_TO_IGNORE" +pylint "$MODULE" --ignore-patterns="$FILES_TO_IGNORE" echo "Running pycodestyle for module $MODULE:" -pycodestyle $MODULE --exclude="$FILES_TO_IGNORE" +pycodestyle "$MODULE" --exclude="$FILES_TO_IGNORE" echo "Running isort for module $MODULE:" # Skip files where isort is behaving weirdly ISORT_EXCLUDED=( "apiclient.py" "avroio_test.py" "datastore_wordcount.py" + "datastoreio_test.py" "iobase_test.py" "fast_coders_test.py" "slow_coders_test.py" @@ -79,6 +80,15 @@ done for file in "${EXCLUDED_GENERATED_FILES[@]}"; do SKIP_PARAM="$SKIP_PARAM --skip $(basename $file)" done -pushd $MODULE +pushd "$MODULE" isort -p apache_beam -w 120 -y -c -ot -cs -sl ${SKIP_PARAM} popd +echo "Checking for files requiring stage 1 refactoring from futurize" +futurize_results=$(futurize -j 8 --stage1 apache_beam 2>&1 |grep Refactored) +futurize_filtered=$(echo "$futurize_results" |grep -v 'pb2\|typehints.py\|trivial_inference.py' || echo "") +count=${#futurize_filtered} +if [ "$count" != "0" ]; then + echo "Some of the changes require futurize stage 1 changes." + echo "$futurize_filtered" + exit 1 +fi
