Hi Team, I am running the pyspark tests in Spark version and it failed with P*rovider org.apache.spark.sql.sources.FakeSourceOne not found.*
Spark Version: 3.4.0/3.5.0 Python Version: 3.8.10 OS: Ubuntu 20.04 *Steps: * # /opt/data/spark/build/sbt -Phive clean package # /opt/data/spark/build/sbt test:compile # pip3 install -r /opt/data/spark/dev/requirements.txt # /opt/data/spark/python/run-tests --python-executables=python3 *Exception:* ====================================================================== ERROR [15.081s]: test_read_images (pyspark.ml.tests.test_image.ImageFileFormatTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/opt/data/spark/python/pyspark/ml/tests/test_image.py", line 29, in test_read_images self.spark.read.format("image") File "/opt/data/spark/python/pyspark/sql/readwriter.py", line 300, in load return self._df(self._jreader.load(path)) File "/opt/data/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/java_gateway.py" , line 1322, in __call__ return_value = get_return_value( File "/opt/data/spark/python/pyspark/errors/exceptions/captured.py", line 176, in deco return f(*a, **kw) File "/opt/data/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/protocol.py", line 326, in get_return_value raise Py4JJavaError( py4j.protocol.Py4JJavaError: An error occurred while calling o33.load. : java.util.ServiceConfigurationError: org.apache.spark.sql.sources.DataSourceRegister: Provider org.apache.spark.sql.sources.FakeSourceOne not found at java.util.ServiceLoader.fail(ServiceLoader.java:239) at java.util.ServiceLoader.access$300(ServiceLoader.java:185) at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:372) at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) at java.util.ServiceLoader$1.next(ServiceLoader.java:480) at scala.collection.convert.Wrappers$JIteratorWrapper .next(Wrappers.scala:46) at scala.collection.Iterator.foreach(Iterator.scala:943) at scala.collection.Iterator.foreach$(Iterator.scala:943) at scala.collection.AbstractIterator.foreach(Iterator.scala:1431) at scala.collection.IterableLike.foreach(IterableLike.scala:74) at scala.collection.IterableLike.foreach$(IterableLike.scala:73) at scala.collection.AbstractIterable.foreach(Iterable.scala:56) at scala.collection.TraversableLike.filterImpl(TraversableLike.scala:303) at scala.collection.TraversableLike.filterImpl$(TraversableLike.scala:297) at scala.collection.AbstractTraversable.filterImpl(Traversable.scala:108) at scala.collection.TraversableLike.filter(TraversableLike.scala:395) at scala.collection.TraversableLike.filter$(TraversableLike.scala:395) at scala.collection.AbstractTraversable.filter(Traversable.scala:108) at org.apache.spark.sql.execution.datasources.DataSource$ .lookupDataSource(DataSource.scala:629) at org.apache.spark.sql.execution.datasources.DataSource$ .lookupDataSourceV2(DataSource.scala:697) at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:208) at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:186) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244) at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:374) at py4j.Gateway.invoke(Gateway.java:282) at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) at py4j.commands.CallCommand.execute(CallCommand.java:79) at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182) at py4j.ClientServerConnection.run(ClientServerConnection.java:106) at java.lang.Thread.run(Thread.java:750) Could someone help me how to proceed further? -- Thanks and Regards *Ranga Reddy* *------------------------------* *Bangalore, Karnataka, India* *Mobile : +91-9986183183 | Email: rangareddy.av...@gmail.com <rangareddy.av...@gmail.com>*