Hi Mich,
FYI we're using spark
operator(https://github.com/GoogleCloudPlatform/spark-on-k8s-operator) to build
stateful structured streaming on k8s for a year. Haven't test it using
non-operator way.
Besides that, the main contributor of the spark operator, Yinan Li, has been
inactive for
try explain codegen on your DF and then pardee the string
On Fri, 7 Apr, 2023, 3:53 pm Chenghao Lyu, wrote:
> Hi,
>
> The detailed stage page shows the involved WholeStageCodegen Ids in its
> DAG visualization from the Spark UI when running a SparkSQL. (e.g., under
> the link
>
Because the name of the directory cannot be an object, it has to be a
string to create partitioned dirs like "date=2023-04-10"
On Tue, 11 Apr, 2023, 8:27 pm Charles vinodh, wrote:
>
> Hi Team,
>
> We are running into the below error when we are trying to run a simple
> query a partitioned table
Hi Team,
We are running into the below error when we are trying to run a simple
query a partitioned table in Spark.
*MetaException(message:Filtering is supported only on partition keys
of type string)
*
Our the partition column has been to type *date *instead of string and
query is a very
