Fokko commented on pull request #29180: URL: https://github.com/apache/spark/pull/29180#issuecomment-662383147
@zero323 The annotations aren't random at all. This PR fixes all the outstanding violations on the current master: ``` fokkodriesprong@Fan python % mypy --ignore-missing-imports . pyspark/storagelevel.py:52: error: "Type[StorageLevel]" has no attribute "DISK_ONLY" pyspark/storagelevel.py:53: error: "Type[StorageLevel]" has no attribute "DISK_ONLY_2" pyspark/storagelevel.py:54: error: "Type[StorageLevel]" has no attribute "MEMORY_ONLY" pyspark/storagelevel.py:55: error: "Type[StorageLevel]" has no attribute "MEMORY_ONLY_2" pyspark/storagelevel.py:56: error: "Type[StorageLevel]" has no attribute "MEMORY_AND_DISK" pyspark/storagelevel.py:57: error: "Type[StorageLevel]" has no attribute "MEMORY_AND_DISK_2" pyspark/storagelevel.py:58: error: "Type[StorageLevel]" has no attribute "OFF_HEAP" setup.py:31: error: Name '__version__' is not defined setup.py:52: error: Incompatible types in assignment (expression has type "str", variable has type "List[str]") setup.py:55: error: Incompatible types in assignment (expression has type "str", variable has type "List[str]") setup.py:120: error: Argument 1 to "symlink" has incompatible type "List[str]"; expected "Union[bytes, str, _PathLike[Any]]" setup.py:128: error: Argument 1 to "copytree" has incompatible type "List[str]"; expected "Union[str, _PathLike[str]]" pyspark/cloudpickle/cloudpickle.py:59: error: Module 'pickle' has no attribute '_getattribute' pyspark/cloudpickle/cloudpickle.py:66: error: Incompatible types in assignment (expression has type "None", variable has type Module) pyspark/cloudpickle/cloudpickle.py:66: error: Incompatible types in assignment (expression has type "None", variable has type "_SpecialForm") pyspark/cloudpickle/cloudpickle.py:71: error: Incompatible types in assignment (expression has type "None", variable has type "_SpecialForm") pyspark/cloudpickle/cloudpickle.py:74: error: Module 'types' has no attribute 'CellType' pyspark/cloudpickle/cloudpickle.py:93: error: Need type annotation for '_DYNAMIC_CLASS_TRACKER_BY_CLASS' pyspark/cloudpickle/cloudpickle.py:94: error: Need type annotation for '_DYNAMIC_CLASS_TRACKER_BY_ID' pyspark/cloudpickle/cloudpickle.py:104: error: Need type annotation for '_extract_code_globals_cache' pyspark/cloudpickle/cloudpickle_fast.py:77: error: All conditional function variants must have identical signatures pyspark/cloudpickle/cloudpickle_fast.py:89: error: All conditional function variants must have identical signatures pyspark/cloudpickle/cloudpickle_fast.py:462: error: Invalid index type "Type[TextIOWrapper]" for "Dict[Type[classmethod], Callable[[Any], Any]]"; expected type "Type[classmethod]" pyspark/cloudpickle/cloudpickle_fast.py:463: error: Invalid index type "Type[Logger]" for "Dict[Type[classmethod], Callable[[Any], Any]]"; expected type "Type[classmethod]" pyspark/cloudpickle/cloudpickle_fast.py:464: error: Invalid index type "Type[RootLogger]" for "Dict[Type[classmethod], Callable[[Any], Any]]"; expected type "Type[classmethod]" pyspark/cloudpickle/cloudpickle_fast.py:465: error: Invalid index type "Type[memoryview]" for "Dict[Type[classmethod], Callable[[Any], Any]]"; expected type "Type[classmethod]" pyspark/cloudpickle/cloudpickle_fast.py:466: error: Invalid index type "Type[property]" for "Dict[Type[classmethod], Callable[[Any], Any]]"; expected type "Type[classmethod]" pyspark/cloudpickle/cloudpickle_fast.py:467: error: Invalid index type "Type[staticmethod]" for "Dict[Type[classmethod], Callable[[Any], Any]]"; expected type "Type[classmethod]" pyspark/cloudpickle/cloudpickle_fast.py:469: error: Invalid index type "Type[CodeType]" for "Dict[Type[classmethod], Callable[[Any], Any]]"; expected type "Type[classmethod]" pyspark/cloudpickle/cloudpickle_fast.py:470: error: Invalid index type "Type[GetSetDescriptorType]" for "Dict[Type[classmethod], Callable[[Any], Any]]"; expected type "Type[classmethod]" pyspark/cloudpickle/cloudpickle_fast.py:471: error: Invalid index type "Type[Module]" for "Dict[Type[classmethod], Callable[[Any], Any]]"; expected type "Type[classmethod]" pyspark/cloudpickle/cloudpickle_fast.py:472: error: Invalid index type "Type[MethodType]" for "Dict[Type[classmethod], Callable[[Any], Any]]"; expected type "Type[classmethod]" pyspark/cloudpickle/cloudpickle_fast.py:473: error: Invalid index type "Type[MappingProxyType[Any, Any]]" for "Dict[Type[classmethod], Callable[[Any], Any]]"; expected type "Type[classmethod]" pyspark/cloudpickle/cloudpickle_fast.py:474: error: Invalid index type "Type[WeakSet[Any]]" for "Dict[Type[classmethod], Callable[[Any], Any]]"; expected type "Type[classmethod]" pyspark/cloudpickle/cloudpickle_fast.py:475: error: Invalid index type "object" for "Dict[Type[classmethod], Callable[[Any], Any]]"; expected type "Type[classmethod]" pyspark/cloudpickle/cloudpickle_fast.py:477: error: Module has no attribute "dispatch_table" pyspark/cloudpickle/cloudpickle_fast.py:643: error: All conditional function variants must have identical signatures pyspark/util.py:23: error: Need type annotation for '__all__' (hint: "__all__: List[<type>] = ...") pyspark/heapq3.py:872: error: Module '_heapq' has no attribute '_heapreplace_max'; maybe "heapreplace"? pyspark/heapq3.py:872: error: Name '_heapreplace_max' already defined on line 470 pyspark/heapq3.py:876: error: Module '_heapq' has no attribute '_heapify_max' pyspark/heapq3.py:876: error: Name '_heapify_max' already defined on line 477 pyspark/heapq3.py:880: error: Module '_heapq' has no attribute '_heappop_max' pyspark/heapq3.py:880: error: Name '_heappop_max' already defined on line 460 pyspark/serializers.py:345: error: Need type annotation for '__cls' (hint: "__cls: Dict[<type>, <type>] = ...") pyspark/accumulators.py:104: error: Need type annotation for '_accumulatorRegistry' (hint: "_accumulatorRegistry: Dict[<type>, <type>] = ...") pyspark/broadcast.py:34: error: Need type annotation for '_broadcastRegistry' (hint: "_broadcastRegistry: Dict[<type>, <type>] = ...") pyspark/sql/types.py:97: error: Need type annotation for '_instances' (hint: "_instances: Dict[<type>, <type>] = ...") pyspark/sql/types.py:742: error: "type" has no attribute "typeName" pyspark/sql/types.py:743: error: "type" has no attribute "typeName" pyspark/sql/types.py:971: error: Incompatible types in assignment (expression has type "Type[StringType]", target has type "Type[FractionalType]") pyspark/sql/streaming.py:45: error: Decorated property not supported pyspark/sql/streaming.py:56: error: Decorated property not supported pyspark/sql/streaming.py:64: error: Decorated property not supported pyspark/sql/streaming.py:74: error: Decorated property not supported pyspark/sql/streaming.py:101: error: Decorated property not supported pyspark/sql/streaming.py:109: error: Decorated property not supported pyspark/sql/streaming.py:118: error: Decorated property not supported pyspark/sql/streaming.py:202: error: Decorated property not supported pyspark/sql/session.py:79: error: Need type annotation for '_options' (hint: "_options: Dict[<type>, <type>] = ...") pyspark/sql/session.py:280: error: Decorated property not supported pyspark/sql/session.py:286: error: Decorated property not supported pyspark/sql/session.py:292: error: Decorated property not supported pyspark/sql/session.py:305: error: Decorated property not supported pyspark/sql/session.py:318: error: Decorated property not supported pyspark/sql/session.py:647: error: Decorated property not supported pyspark/sql/session.py:658: error: Decorated property not supported pyspark/sql/session.py:671: error: Decorated property not supported pyspark/sql/dataframe.py:79: error: Decorated property not supported pyspark/sql/dataframe.py:89: error: Decorated property not supported pyspark/sql/dataframe.py:96: error: Decorated property not supported pyspark/sql/dataframe.py:212: error: Decorated property not supported pyspark/sql/dataframe.py:223: error: Decorated property not supported pyspark/sql/dataframe.py:236: error: Decorated property not supported pyspark/sql/dataframe.py:380: error: Decorated property not supported pyspark/sql/dataframe.py:694: error: Decorated property not supported pyspark/sql/dataframe.py:1002: error: Decorated property not supported pyspark/sql/dataframe.py:1012: error: Decorated property not supported pyspark/sql/context.py:157: error: Decorated property not supported pyspark/sql/context.py:434: error: Decorated property not supported pyspark/sql/context.py:445: error: Decorated property not supported pyspark/sql/context.py:462: error: Decorated property not supported pyspark/ml/linalg/__init__.py:434: error: Argument 1 to "_delegate" has incompatible type "str"; expected "DenseVector" pyspark/ml/linalg/__init__.py:435: error: Argument 1 to "_delegate" has incompatible type "str"; expected "DenseVector" pyspark/ml/linalg/__init__.py:436: error: Argument 1 to "_delegate" has incompatible type "str"; expected "DenseVector" pyspark/ml/linalg/__init__.py:437: error: Argument 1 to "_delegate" has incompatible type "str"; expected "DenseVector" pyspark/ml/linalg/__init__.py:438: error: Argument 1 to "_delegate" has incompatible type "str"; expected "DenseVector" pyspark/ml/linalg/__init__.py:439: error: Argument 1 to "_delegate" has incompatible type "str"; expected "DenseVector" pyspark/ml/linalg/__init__.py:440: error: Argument 1 to "_delegate" has incompatible type "str"; expected "DenseVector" pyspark/ml/linalg/__init__.py:441: error: Argument 1 to "_delegate" has incompatible type "str"; expected "DenseVector" pyspark/ml/linalg/__init__.py:442: error: Argument 1 to "_delegate" has incompatible type "str"; expected "DenseVector" pyspark/ml/linalg/__init__.py:443: error: Argument 1 to "_delegate" has incompatible type "str"; expected "DenseVector" pyspark/ml/linalg/__init__.py:444: error: Argument 1 to "_delegate" has incompatible type "str"; expected "DenseVector" pyspark/ml/linalg/__init__.py:445: error: Argument 1 to "_delegate" has incompatible type "str"; expected "DenseVector" pyspark/testing/sqlutils.py:150: error: Incompatible types in assignment (expression has type "PythonOnlyUDT", base class "ExamplePoint" defined the type as "ExamplePointUDT") pyspark/mllib/evaluation.py:69: error: Decorated property not supported pyspark/mllib/evaluation.py:78: error: Decorated property not supported pyspark/mllib/evaluation.py:136: error: Decorated property not supported pyspark/mllib/evaluation.py:145: error: Decorated property not supported pyspark/mllib/evaluation.py:154: error: Decorated property not supported pyspark/mllib/evaluation.py:163: error: Decorated property not supported pyspark/mllib/evaluation.py:172: error: Decorated property not supported pyspark/mllib/evaluation.py:315: error: Decorated property not supported pyspark/mllib/evaluation.py:324: error: Decorated property not supported pyspark/mllib/evaluation.py:333: error: Decorated property not supported pyspark/mllib/evaluation.py:341: error: Decorated property not supported pyspark/mllib/evaluation.py:350: error: Decorated property not supported pyspark/mllib/evaluation.py:436: error: Decorated property not supported pyspark/mllib/evaluation.py:562: error: Decorated property not supported pyspark/mllib/evaluation.py:571: error: Decorated property not supported pyspark/mllib/evaluation.py:580: error: Decorated property not supported pyspark/mllib/evaluation.py:589: error: Decorated property not supported pyspark/mllib/evaluation.py:597: error: Decorated property not supported pyspark/mllib/evaluation.py:606: error: Decorated property not supported pyspark/ml/util.py:564: error: Decorated property not supported pyspark/ml/util.py:573: error: Decorated property not supported pyspark/ml/image.py:212: error: Cannot assign to a method pyspark/ml/base.py:311: error: Decorated property not supported pyspark/ml/wrapper.py:391: error: Decorated property not supported pyspark/ml/tuning.py:29: error: Module 'pyspark.sql.functions' has no attribute 'col' pyspark/ml/tuning.py:29: error: Module 'pyspark.sql.functions' has no attribute 'lit' pyspark/ml/tree.py:32: error: Decorated property not supported pyspark/ml/tree.py:38: error: Decorated property not supported pyspark/ml/tree.py:44: error: Decorated property not supported pyspark/ml/tree.py:169: error: Decorated property not supported pyspark/ml/tree.py:175: error: Decorated property not supported pyspark/ml/tree.py:181: error: Decorated property not supported pyspark/ml/tree.py:187: error: Decorated property not supported pyspark/ml/tree.py:193: error: Decorated property not supported pyspark/ml/stat.py:25: error: Module 'pyspark.sql.functions' has no attribute 'lit' pyspark/ml/recommendation.py:499: error: Decorated property not supported pyspark/ml/recommendation.py:505: error: Decorated property not supported pyspark/ml/recommendation.py:514: error: Decorated property not supported pyspark/ml/fpm.py:116: error: Decorated property not supported pyspark/ml/fpm.py:126: error: Decorated property not supported pyspark/ml/feature.py:870: error: Decorated property not supported pyspark/ml/feature.py:1406: error: Decorated property not supported pyspark/ml/feature.py:1414: error: Decorated property not supported pyspark/ml/feature.py:1422: error: Decorated property not supported pyspark/ml/feature.py:1689: error: Decorated property not supported pyspark/ml/feature.py:1876: error: Decorated property not supported pyspark/ml/feature.py:2158: error: Decorated property not supported pyspark/ml/feature.py:2166: error: Decorated property not supported pyspark/ml/feature.py:2550: error: Decorated property not supported pyspark/ml/feature.py:3061: error: Decorated property not supported pyspark/ml/feature.py:3069: error: Decorated property not supported pyspark/ml/feature.py:3425: error: Decorated property not supported pyspark/ml/feature.py:3433: error: Decorated property not supported pyspark/ml/feature.py:3702: error: Decorated property not supported pyspark/ml/feature.py:4286: error: Decorated property not supported pyspark/ml/feature.py:4294: error: Decorated property not supported pyspark/ml/feature.py:4801: error: Decorated property not supported pyspark/ml/feature.py:4810: error: Decorated property not supported pyspark/ml/feature.py:5194: error: Decorated property not supported pyspark/ml/feature.py:5728: error: Decorated property not supported pyspark/ml/regression.py:319: error: Decorated property not supported pyspark/ml/regression.py:327: error: Decorated property not supported pyspark/ml/regression.py:335: error: Decorated property not supported pyspark/ml/regression.py:343: error: Decorated property not supported pyspark/ml/regression.py:379: error: Decorated property not supported pyspark/ml/regression.py:387: error: Decorated property not supported pyspark/ml/regression.py:396: error: Decorated property not supported pyspark/ml/regression.py:405: error: Decorated property not supported pyspark/ml/regression.py:414: error: Decorated property not supported pyspark/ml/regression.py:430: error: Decorated property not supported pyspark/ml/regression.py:444: error: Decorated property not supported pyspark/ml/regression.py:458: error: Decorated property not supported pyspark/ml/regression.py:471: error: Decorated property not supported pyspark/ml/regression.py:486: error: Decorated property not supported pyspark/ml/regression.py:500: error: Decorated property not supported pyspark/ml/regression.py:508: error: Decorated property not supported pyspark/ml/regression.py:516: error: Decorated property not supported pyspark/ml/regression.py:524: error: Decorated property not supported pyspark/ml/regression.py:533: error: Decorated property not supported pyspark/ml/regression.py:547: error: Decorated property not supported pyspark/ml/regression.py:561: error: Decorated property not supported pyspark/ml/regression.py:585: error: Decorated property not supported pyspark/ml/regression.py:597: error: Decorated property not supported pyspark/ml/regression.py:777: error: Decorated property not supported pyspark/ml/regression.py:785: error: Decorated property not supported pyspark/ml/regression.py:794: error: Decorated property not supported pyspark/ml/regression.py:1034: error: Decorated property not supported pyspark/ml/regression.py:1286: error: Decorated property not supported pyspark/ml/regression.py:1292: error: Decorated property not supported pyspark/ml/regression.py:1585: error: Decorated property not supported pyspark/ml/regression.py:1600: error: Decorated property not supported pyspark/ml/regression.py:1843: error: Decorated property not supported pyspark/ml/regression.py:1851: error: Decorated property not supported pyspark/ml/regression.py:1859: error: Decorated property not supported pyspark/ml/regression.py:2176: error: Decorated property not supported pyspark/ml/regression.py:2184: error: Decorated property not supported pyspark/ml/regression.py:2192: error: Decorated property not supported pyspark/ml/regression.py:2229: error: Decorated property not supported pyspark/ml/regression.py:2237: error: Decorated property not supported pyspark/ml/regression.py:2246: error: Decorated property not supported pyspark/ml/regression.py:2254: error: Decorated property not supported pyspark/ml/regression.py:2262: error: Decorated property not supported pyspark/ml/regression.py:2270: error: Decorated property not supported pyspark/ml/regression.py:2278: error: Decorated property not supported pyspark/ml/regression.py:2296: error: Decorated property not supported pyspark/ml/regression.py:2304: error: Decorated property not supported pyspark/ml/regression.py:2312: error: Decorated property not supported pyspark/ml/regression.py:2323: error: Decorated property not supported pyspark/ml/regression.py:2340: error: Decorated property not supported pyspark/ml/regression.py:2348: error: Decorated property not supported pyspark/ml/regression.py:2356: error: Decorated property not supported pyspark/ml/regression.py:2367: error: Decorated property not supported pyspark/ml/regression.py:2378: error: Decorated property not supported pyspark/ml/regression.py:2624: error: Decorated property not supported pyspark/ml/regression.py:2632: error: Decorated property not supported pyspark/ml/regression.py:2640: error: Decorated property not supported pyspark/ml/clustering.py:42: error: Decorated property not supported pyspark/ml/clustering.py:50: error: Decorated property not supported pyspark/ml/clustering.py:58: error: Decorated property not supported pyspark/ml/clustering.py:66: error: Decorated property not supported pyspark/ml/clustering.py:74: error: Decorated property not supported pyspark/ml/clustering.py:82: error: Decorated property not supported pyspark/ml/clustering.py:90: error: Decorated property not supported pyspark/ml/clustering.py:153: error: Decorated property not supported pyspark/ml/clustering.py:163: error: Decorated property not supported pyspark/ml/clustering.py:176: error: Decorated property not supported pyspark/ml/clustering.py:186: error: Decorated property not supported pyspark/ml/clustering.py:445: error: Decorated property not supported pyspark/ml/clustering.py:453: error: Decorated property not supported pyspark/ml/clustering.py:461: error: Decorated property not supported pyspark/ml/clustering.py:477: error: Decorated property not supported pyspark/ml/clustering.py:560: error: Decorated property not supported pyspark/ml/clustering.py:824: error: Decorated property not supported pyspark/ml/clustering.py:1017: error: Decorated property not supported pyspark/ml/classification.py:101: error: Decorated property not supported pyspark/ml/classification.py:208: error: Decorated property not supported pyspark/ml/classification.py:256: error: Decorated property not supported pyspark/ml/classification.py:264: error: Decorated property not supported pyspark/ml/classification.py:272: error: Decorated property not supported pyspark/ml/classification.py:281: error: Decorated property not supported pyspark/ml/classification.py:290: error: Decorated property not supported pyspark/ml/classification.py:304: error: Decorated property not supported pyspark/ml/classification.py:312: error: Decorated property not supported pyspark/ml/classification.py:320: error: Decorated property not supported pyspark/ml/classification.py:328: error: Decorated property not supported pyspark/ml/classification.py:343: error: Decorated property not supported pyspark/ml/classification.py:353: error: Decorated property not supported pyspark/ml/classification.py:362: error: Decorated property not supported pyspark/ml/classification.py:370: error: Decorated property not supported pyspark/ml/classification.py:379: error: Decorated property not supported pyspark/ml/classification.py:403: error: Decorated property not supported pyspark/ml/classification.py:413: error: Decorated property not supported pyspark/ml/classification.py:430: error: Decorated property not supported pyspark/ml/classification.py:439: error: Decorated property not supported pyspark/ml/classification.py:452: error: Decorated property not supported pyspark/ml/classification.py:461: error: Decorated property not supported pyspark/ml/classification.py:471: error: Decorated property not supported pyspark/ml/classification.py:480: error: Decorated property not supported pyspark/ml/classification.py:490: error: Decorated property not supported pyspark/ml/classification.py:709: error: Decorated property not supported pyspark/ml/classification.py:717: error: Decorated property not supported pyspark/ml/classification.py:1182: error: Decorated property not supported pyspark/ml/classification.py:1191: error: Decorated property not supported pyspark/ml/classification.py:1200: error: Decorated property not supported pyspark/ml/classification.py:1208: error: Decorated property not supported pyspark/ml/classification.py:1216: error: Decorated property not supported pyspark/ml/classification.py:1259: error: Decorated property not supported pyspark/ml/classification.py:1268: error: Decorated property not supported pyspark/ml/classification.py:1525: error: Decorated property not supported pyspark/ml/classification.py:1786: error: Decorated property not supported pyspark/ml/classification.py:1801: error: Decorated property not supported pyspark/ml/classification.py:1807: error: Decorated property not supported pyspark/ml/classification.py:2181: error: Decorated property not supported pyspark/ml/classification.py:2196: error: Decorated property not supported pyspark/ml/classification.py:2399: error: Decorated property not supported pyspark/ml/classification.py:2407: error: Decorated property not supported pyspark/ml/classification.py:2415: error: Decorated property not supported pyspark/ml/classification.py:2632: error: Decorated property not supported pyspark/ml/classification.py:3253: error: Decorated property not supported pyspark/ml/classification.py:3261: error: Decorated property not supported pyspark/ml/classification.py:3269: error: Decorated property not supported pyspark/sql/tests/test_udf.py:362: error: Argument 2 to "skipIf" has incompatible type "Optional[str]"; expected "str" pyspark/sql/tests/test_udf.py:379: error: Argument 2 to "skipIf" has incompatible type "Optional[str]"; expected "str" pyspark/sql/tests/test_udf.py:569: error: Argument 2 to "skipIf" has incompatible type "Optional[str]"; expected "str" pyspark/sql/tests/test_types.py:28: error: Module 'pyspark.sql.functions' has no attribute 'col' pyspark/sql/tests/test_streaming.py:23: error: Module 'pyspark.sql.functions' has no attribute 'lit' pyspark/sql/tests/test_serde.py:24: error: Module 'pyspark.sql.functions' has no attribute 'lit' pyspark/sql/tests/test_readwriter.py:22: error: Module 'pyspark.sql.functions' has no attribute 'col' pyspark/sql/tests/test_pandas_udf_window.py:21: error: Module 'pyspark.sql.functions' has no attribute 'col' pyspark/sql/tests/test_pandas_udf_window.py:21: error: Module 'pyspark.sql.functions' has no attribute 'lit' pyspark/sql/tests/test_pandas_udf_window.py:21: error: Module 'pyspark.sql.functions' has no attribute 'mean' pyspark/sql/tests/test_pandas_udf_window.py:21: error: Module 'pyspark.sql.functions' has no attribute 'min' pyspark/sql/tests/test_pandas_udf_window.py:21: error: Module 'pyspark.sql.functions' has no attribute 'max' pyspark/sql/tests/test_pandas_udf_window.py:21: error: Module 'pyspark.sql.functions' has no attribute 'rank' pyspark/sql/tests/test_pandas_udf_window.py:34: error: Argument 2 to "skipIf" has incompatible type "Optional[str]"; expected "str" pyspark/sql/tests/test_pandas_udf_typehints.py:21: error: Module 'pyspark.sql.functions' has no attribute 'mean' pyspark/sql/tests/test_pandas_udf_typehints.py:21: error: Module 'pyspark.sql.functions' has no attribute 'lit' pyspark/sql/tests/test_pandas_udf_typehints.py:37: error: Argument 2 to "skipIf" has incompatible type "Optional[str]"; expected "str" pyspark/sql/tests/test_pandas_udf_scalar.py:30: error: Module 'pyspark.sql.functions' has no attribute 'col' pyspark/sql/tests/test_pandas_udf_scalar.py:30: error: Module 'pyspark.sql.functions' has no attribute 'lit' pyspark/sql/tests/test_pandas_udf_scalar.py:30: error: Module 'pyspark.sql.functions' has no attribute 'sum' pyspark/sql/tests/test_pandas_udf_scalar.py:49: error: Argument 2 to "skipIf" has incompatible type "Optional[str]"; expected "str" pyspark/sql/tests/test_pandas_udf_scalar.py:1098: error: Argument 2 to "skipIf" has incompatible type "Optional[str]"; expected "str" pyspark/sql/tests/test_pandas_udf_grouped_agg.py:22: error: Module 'pyspark.sql.functions' has no attribute 'col' pyspark/sql/tests/test_pandas_udf_grouped_agg.py:22: error: Module 'pyspark.sql.functions' has no attribute 'lit' pyspark/sql/tests/test_pandas_udf_grouped_agg.py:22: error: Module 'pyspark.sql.functions' has no attribute 'mean' pyspark/sql/tests/test_pandas_udf_grouped_agg.py:22: error: Module 'pyspark.sql.functions' has no attribute 'sum' pyspark/sql/tests/test_pandas_udf_grouped_agg.py:37: error: Argument 2 to "skipIf" has incompatible type "Optional[str]"; expected "str" pyspark/sql/tests/test_pandas_udf.py:31: error: Argument 2 to "skipIf" has incompatible type "Optional[str]"; expected "str" pyspark/sql/tests/test_pandas_map.py:32: error: Argument 2 to "skipIf" has incompatible type "Optional[str]"; expected "str" pyspark/sql/tests/test_pandas_grouped_map.py:26: error: Module 'pyspark.sql.functions' has no attribute 'col' pyspark/sql/tests/test_pandas_grouped_map.py:26: error: Module 'pyspark.sql.functions' has no attribute 'lit' pyspark/sql/tests/test_pandas_grouped_map.py:26: error: Module 'pyspark.sql.functions' has no attribute 'sum' pyspark/sql/tests/test_pandas_grouped_map.py:43: error: Argument 2 to "skipIf" has incompatible type "Optional[str]"; expected "str" pyspark/sql/tests/test_pandas_cogrouped_map.py:21: error: Module 'pyspark.sql.functions' has no attribute 'col' pyspark/sql/tests/test_pandas_cogrouped_map.py:21: error: Module 'pyspark.sql.functions' has no attribute 'lit' pyspark/sql/tests/test_pandas_cogrouped_map.py:21: error: Module 'pyspark.sql.functions' has no attribute 'sum' pyspark/sql/tests/test_pandas_cogrouped_map.py:37: error: Argument 2 to "skipIf" has incompatible type "Optional[str]"; expected "str" pyspark/sql/tests/test_functions.py:24: error: Module 'pyspark.sql.functions' has no attribute 'col' pyspark/sql/tests/test_functions.py:24: error: Module 'pyspark.sql.functions' has no attribute 'lit' pyspark/sql/tests/test_dataframe.py:522: error: Argument 2 to "skipIf" has incompatible type "Optional[str]"; expected "str" pyspark/sql/tests/test_dataframe.py:534: error: Argument 2 to "skipIf" has incompatible type "Optional[str]"; expected "str" pyspark/sql/tests/test_dataframe.py:547: error: Argument 2 to "skipIf" has incompatible type "Optional[str]"; expected "str" pyspark/sql/tests/test_dataframe.py:573: error: Argument 2 to "skipIf" has incompatible type "Optional[str]"; expected "str" pyspark/sql/tests/test_dataframe.py:585: error: Argument 2 to "skipIf" has incompatible type "Optional[str]"; expected "str" pyspark/sql/tests/test_dataframe.py:605: error: Argument 2 to "skipIf" has incompatible type "Optional[str]"; expected "str" pyspark/sql/tests/test_dataframe.py:633: error: Argument 2 to "skipIf" has incompatible type "Optional[str]"; expected "str" pyspark/sql/tests/test_dataframe.py:661: error: Argument 2 to "skipIf" has incompatible type "Optional[str]"; expected "str" pyspark/sql/tests/test_dataframe.py:689: error: Argument 2 to "skipIf" has incompatible type "Optional[str]"; expected "str" pyspark/sql/tests/test_dataframe.py:893: error: Argument 2 to "skipIf" has incompatible type "Optional[str]"; expected "str" pyspark/sql/tests/test_arrow.py:43: error: Argument 2 to "skipIf" has incompatible type "Optional[str]"; expected "str" pyspark/sql/tests/test_arrow.py:460: error: Argument 2 to "skipIf" has incompatible type "Optional[str]"; expected "str" pyspark/mllib/linalg/__init__.py:479: error: Argument 1 to "_delegate" has incompatible type "str"; expected "DenseVector" pyspark/mllib/linalg/__init__.py:480: error: Argument 1 to "_delegate" has incompatible type "str"; expected "DenseVector" pyspark/mllib/linalg/__init__.py:481: error: Argument 1 to "_delegate" has incompatible type "str"; expected "DenseVector" pyspark/mllib/linalg/__init__.py:482: error: Argument 1 to "_delegate" has incompatible type "str"; expected "DenseVector" pyspark/mllib/linalg/__init__.py:483: error: Argument 1 to "_delegate" has incompatible type "str"; expected "DenseVector" pyspark/mllib/linalg/__init__.py:484: error: Argument 1 to "_delegate" has incompatible type "str"; expected "DenseVector" pyspark/mllib/linalg/__init__.py:485: error: Argument 1 to "_delegate" has incompatible type "str"; expected "DenseVector" pyspark/mllib/linalg/__init__.py:486: error: Argument 1 to "_delegate" has incompatible type "str"; expected "DenseVector" pyspark/mllib/linalg/__init__.py:487: error: Argument 1 to "_delegate" has incompatible type "str"; expected "DenseVector" pyspark/mllib/linalg/__init__.py:488: error: Argument 1 to "_delegate" has incompatible type "str"; expected "DenseVector" pyspark/mllib/linalg/__init__.py:489: error: Argument 1 to "_delegate" has incompatible type "str"; expected "DenseVector" pyspark/mllib/linalg/__init__.py:490: error: Argument 1 to "_delegate" has incompatible type "str"; expected "DenseVector" pyspark/mllib/linalg/__init__.py:1339: error: Decorated property not supported pyspark/mllib/linalg/__init__.py:1348: error: Decorated property not supported pyspark/mllib/regression.py:84: error: Decorated property not supported pyspark/mllib/regression.py:90: error: Decorated property not supported pyspark/mllib/classification.py:58: error: Decorated property not supported pyspark/mllib/classification.py:177: error: Decorated property not supported pyspark/mllib/classification.py:185: error: Decorated property not supported pyspark/testing/streamingutils.py:50: error: Incompatible types in assignment (expression has type "None", variable has type "str") pyspark/mllib/recommendation.py:200: error: Decorated property not supported pyspark/mllib/feature.py:161: error: Decorated property not supported pyspark/mllib/feature.py:169: error: Decorated property not supported pyspark/mllib/feature.py:177: error: Decorated property not supported pyspark/mllib/feature.py:185: error: Decorated property not supported pyspark/mllib/clustering.py:62: error: Decorated property not supported pyspark/mllib/clustering.py:69: error: Decorated property not supported pyspark/mllib/clustering.py:223: error: Decorated property not supported pyspark/mllib/clustering.py:229: error: Decorated property not supported pyspark/mllib/clustering.py:417: error: Decorated property not supported pyspark/mllib/clustering.py:426: error: Decorated property not supported pyspark/mllib/clustering.py:437: error: Decorated property not supported pyspark/mllib/clustering.py:596: error: Decorated property not supported pyspark/mllib/clustering.py:733: error: Decorated property not supported pyspark/mllib/linalg/distributed.py:400: error: Decorated property not supported pyspark/mllib/linalg/distributed.py:417: error: Decorated property not supported pyspark/mllib/linalg/distributed.py:425: error: Decorated property not supported Found 364 errors in 54 files (checked 186 source files) ``` And it enables the mypy check so we can see where it will fail in the future. This allows us to integrates the types file by file. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
