[jira] [Commented] (CARBONDATA-3970) Carbondata 2.0.1 MV ERROR CarbonInternalMetastore$: Adding/Modifying tableProperties operation failed
[
https://issues.apache.org/jira/browse/CARBONDATA-3970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17379648#comment-17379648
]
Indhumathi commented on CARBONDATA-3970:
HI, from the below exception, looks like issue with the configurations related
to 'spark.sql.extensions=org.apache.spark.sql.CarbonExtensions' .
20/08/26 01:04:50 ERROR CarbonInternalMetastore$: Adding/Modifying
tableProperties operation failed: org.apache.spark.sql.hive.HiveExternalCatalog
cannot be cast to
org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener.
Closing this JIRA, for further clarifications, please contact through slack
channel.
> Carbondata 2.0.1 MV ERROR CarbonInternalMetastore$: Adding/Modifying
> tableProperties operation failed
> --
>
> Key: CARBONDATA-3970
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3970
> Project: CarbonData
> Issue Type: Bug
> Components: data-query, hive-integration
>Affects Versions: 2.0.1
> Environment: CarbonData 2.0.1 with Spark 2.4.5
>Reporter: Sushant Sammanwar
>Priority: Major
>
> Hi ,
>
> I am facing issues with materialized views - the query is not hitting the
> view in the explain plan .I would really appreciate if you could help me on
> this.
> Below are the details :
> I am using Spark shell to connect to Carbon 2.0.1 using spark 2.4.5
> Underlying table has data loaded.
> I think problem is while create materialized view as i am getting a error
> related to metastore.
>
>
> scala> carbon.sql("create MATERIALIZED VIEW agg_sales_mv as select country,
> sex,sum(quantity),avg(price) from sales group by country,sex").show()
> 20/08/26 01:04:41 AUDIT audit: \{"time":"August 26, 2020 1:04:41 AM
> IST","username":"root","opName":"CREATE MATERIALIZED
> VIEW","opId":"16462372696035311","opStatus":"START"}
> 20/08/26 01:04:45 AUDIT audit: \{"time":"August 26, 2020 1:04:45 AM
> IST","username":"root","opName":"CREATE
> TABLE","opId":"16462377160819798","opStatus":"START"}
> 20/08/26 01:04:46 AUDIT audit: \{"time":"August 26, 2020 1:04:46 AM
> IST","username":"root","opName":"CREATE
> TABLE","opId":"16462377696791275","opStatus":"START"}
> 20/08/26 01:04:48 AUDIT audit: \{"time":"August 26, 2020 1:04:48 AM
> IST","username":"root","opName":"CREATE
> TABLE","opId":"16462377696791275","opStatus":"SUCCESS","opTime":"2326
> ms","table":"NA","extraInfo":{}}
> 20/08/26 01:04:48 AUDIT audit: \{"time":"August 26, 2020 1:04:48 AM
> IST","username":"root","opName":"CREATE
> TABLE","opId":"16462377160819798","opStatus":"SUCCESS","opTime":"2955
> ms","table":"default.agg_sales_mv","extraInfo":{"local_dictionary_threshold":"1","bad_record_path":"","table_blocksize":"1024","local_dictionary_enable":"true","flat_folder":"false","external":"false","sort_columns":"","comment":"","carbon.column.compressor":"snappy","mv_related_tables":"sales"}}
> 20/08/26 01:04:50 ERROR CarbonInternalMetastore$: Adding/Modifying
> tableProperties operation failed:
> org.apache.spark.sql.hive.HiveExternalCatalog cannot be cast to
> org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener
> 20/08/26 01:04:50 ERROR CarbonInternalMetastore$: Adding/Modifying
> tableProperties operation failed:
> org.apache.spark.sql.hive.HiveExternalCatalog cannot be cast to
> org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener
> 20/08/26 01:04:51 AUDIT audit: \{"time":"August 26, 2020 1:04:51 AM
> IST","username":"root","opName":"CREATE MATERIALIZED
> VIEW","opId":"16462372696035311","opStatus":"SUCCESS","opTime":"10551
> ms","table":"NA","extraInfo":{"mvName":"agg_sales_mv"}}
> ++
> ||
> ++
> ++
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CARBONDATA-3970) Carbondata 2.0.1 MV ERROR CarbonInternalMetastore$: Adding/Modifying tableProperties operation failed
[
https://issues.apache.org/jira/browse/CARBONDATA-3970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17218172#comment-17218172
]
Akash R Nilugal commented on CARBONDATA-3970:
-
[~sushantsam] please join our slack channel. It will be easy to discuss all the
issues , as JIRA does not notify all the users.
https://join.slack.com/t/carbondataworkspace/shared_invite/zt-g8sv1g92-pr3GTvjrW5H9DVvNl6H2dg
> Carbondata 2.0.1 MV ERROR CarbonInternalMetastore$: Adding/Modifying
> tableProperties operation failed
> --
>
> Key: CARBONDATA-3970
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3970
> Project: CarbonData
> Issue Type: Bug
> Components: data-query, hive-integration
>Affects Versions: 2.0.1
> Environment: CarbonData 2.0.1 with Spark 2.4.5
>Reporter: Sushant Sammanwar
>Priority: Major
>
> Hi ,
>
> I am facing issues with materialized views - the query is not hitting the
> view in the explain plan .I would really appreciate if you could help me on
> this.
> Below are the details :
> I am using Spark shell to connect to Carbon 2.0.1 using spark 2.4.5
> Underlying table has data loaded.
> I think problem is while create materialized view as i am getting a error
> related to metastore.
>
>
> scala> carbon.sql("create MATERIALIZED VIEW agg_sales_mv as select country,
> sex,sum(quantity),avg(price) from sales group by country,sex").show()
> 20/08/26 01:04:41 AUDIT audit: \{"time":"August 26, 2020 1:04:41 AM
> IST","username":"root","opName":"CREATE MATERIALIZED
> VIEW","opId":"16462372696035311","opStatus":"START"}
> 20/08/26 01:04:45 AUDIT audit: \{"time":"August 26, 2020 1:04:45 AM
> IST","username":"root","opName":"CREATE
> TABLE","opId":"16462377160819798","opStatus":"START"}
> 20/08/26 01:04:46 AUDIT audit: \{"time":"August 26, 2020 1:04:46 AM
> IST","username":"root","opName":"CREATE
> TABLE","opId":"16462377696791275","opStatus":"START"}
> 20/08/26 01:04:48 AUDIT audit: \{"time":"August 26, 2020 1:04:48 AM
> IST","username":"root","opName":"CREATE
> TABLE","opId":"16462377696791275","opStatus":"SUCCESS","opTime":"2326
> ms","table":"NA","extraInfo":{}}
> 20/08/26 01:04:48 AUDIT audit: \{"time":"August 26, 2020 1:04:48 AM
> IST","username":"root","opName":"CREATE
> TABLE","opId":"16462377160819798","opStatus":"SUCCESS","opTime":"2955
> ms","table":"default.agg_sales_mv","extraInfo":{"local_dictionary_threshold":"1","bad_record_path":"","table_blocksize":"1024","local_dictionary_enable":"true","flat_folder":"false","external":"false","sort_columns":"","comment":"","carbon.column.compressor":"snappy","mv_related_tables":"sales"}}
> 20/08/26 01:04:50 ERROR CarbonInternalMetastore$: Adding/Modifying
> tableProperties operation failed:
> org.apache.spark.sql.hive.HiveExternalCatalog cannot be cast to
> org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener
> 20/08/26 01:04:50 ERROR CarbonInternalMetastore$: Adding/Modifying
> tableProperties operation failed:
> org.apache.spark.sql.hive.HiveExternalCatalog cannot be cast to
> org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener
> 20/08/26 01:04:51 AUDIT audit: \{"time":"August 26, 2020 1:04:51 AM
> IST","username":"root","opName":"CREATE MATERIALIZED
> VIEW","opId":"16462372696035311","opStatus":"SUCCESS","opTime":"10551
> ms","table":"NA","extraInfo":{"mvName":"agg_sales_mv"}}
> ++
> ||
> ++
> ++
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CARBONDATA-3970) Carbondata 2.0.1 MV ERROR CarbonInternalMetastore$: Adding/Modifying tableProperties operation failed
[
https://issues.apache.org/jira/browse/CARBONDATA-3970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17218169#comment-17218169
]
Akash R Nilugal commented on CARBONDATA-3970:
-
I think the issue with the carbon configurations. [~sushantsam] have you
configured carbonExtensions and using sparksession itself for the queries ?
Because we support and integrated SparkExtensions. Please can you tell us what
is your configurations for the integration.
> Carbondata 2.0.1 MV ERROR CarbonInternalMetastore$: Adding/Modifying
> tableProperties operation failed
> --
>
> Key: CARBONDATA-3970
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3970
> Project: CarbonData
> Issue Type: Bug
> Components: data-query, hive-integration
>Affects Versions: 2.0.1
> Environment: CarbonData 2.0.1 with Spark 2.4.5
>Reporter: Sushant Sammanwar
>Priority: Major
>
> Hi ,
>
> I am facing issues with materialized views - the query is not hitting the
> view in the explain plan .I would really appreciate if you could help me on
> this.
> Below are the details :
> I am using Spark shell to connect to Carbon 2.0.1 using spark 2.4.5
> Underlying table has data loaded.
> I think problem is while create materialized view as i am getting a error
> related to metastore.
>
>
> scala> carbon.sql("create MATERIALIZED VIEW agg_sales_mv as select country,
> sex,sum(quantity),avg(price) from sales group by country,sex").show()
> 20/08/26 01:04:41 AUDIT audit: \{"time":"August 26, 2020 1:04:41 AM
> IST","username":"root","opName":"CREATE MATERIALIZED
> VIEW","opId":"16462372696035311","opStatus":"START"}
> 20/08/26 01:04:45 AUDIT audit: \{"time":"August 26, 2020 1:04:45 AM
> IST","username":"root","opName":"CREATE
> TABLE","opId":"16462377160819798","opStatus":"START"}
> 20/08/26 01:04:46 AUDIT audit: \{"time":"August 26, 2020 1:04:46 AM
> IST","username":"root","opName":"CREATE
> TABLE","opId":"16462377696791275","opStatus":"START"}
> 20/08/26 01:04:48 AUDIT audit: \{"time":"August 26, 2020 1:04:48 AM
> IST","username":"root","opName":"CREATE
> TABLE","opId":"16462377696791275","opStatus":"SUCCESS","opTime":"2326
> ms","table":"NA","extraInfo":{}}
> 20/08/26 01:04:48 AUDIT audit: \{"time":"August 26, 2020 1:04:48 AM
> IST","username":"root","opName":"CREATE
> TABLE","opId":"16462377160819798","opStatus":"SUCCESS","opTime":"2955
> ms","table":"default.agg_sales_mv","extraInfo":{"local_dictionary_threshold":"1","bad_record_path":"","table_blocksize":"1024","local_dictionary_enable":"true","flat_folder":"false","external":"false","sort_columns":"","comment":"","carbon.column.compressor":"snappy","mv_related_tables":"sales"}}
> 20/08/26 01:04:50 ERROR CarbonInternalMetastore$: Adding/Modifying
> tableProperties operation failed:
> org.apache.spark.sql.hive.HiveExternalCatalog cannot be cast to
> org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener
> 20/08/26 01:04:50 ERROR CarbonInternalMetastore$: Adding/Modifying
> tableProperties operation failed:
> org.apache.spark.sql.hive.HiveExternalCatalog cannot be cast to
> org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener
> 20/08/26 01:04:51 AUDIT audit: \{"time":"August 26, 2020 1:04:51 AM
> IST","username":"root","opName":"CREATE MATERIALIZED
> VIEW","opId":"16462372696035311","opStatus":"SUCCESS","opTime":"10551
> ms","table":"NA","extraInfo":{"mvName":"agg_sales_mv"}}
> ++
> ||
> ++
> ++
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CARBONDATA-3970) Carbondata 2.0.1 MV ERROR CarbonInternalMetastore$: Adding/Modifying tableProperties operation failed
[
https://issues.apache.org/jira/browse/CARBONDATA-3970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17209365#comment-17209365
]
SHREELEKHYA GAMPA commented on CARBONDATA-3970:
---
Hi [~sushantsam], Could you please provide the spark configurations set
particularly related to metastore, and complete stacktrace of error.
> Carbondata 2.0.1 MV ERROR CarbonInternalMetastore$: Adding/Modifying
> tableProperties operation failed
> --
>
> Key: CARBONDATA-3970
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3970
> Project: CarbonData
> Issue Type: Bug
> Components: data-query, hive-integration
>Affects Versions: 2.0.1
> Environment: CarbonData 2.0.1 with Spark 2.4.5
>Reporter: Sushant Sammanwar
>Priority: Major
>
> Hi ,
>
> I am facing issues with materialized views - the query is not hitting the
> view in the explain plan .I would really appreciate if you could help me on
> this.
> Below are the details :
> I am using Spark shell to connect to Carbon 2.0.1 using spark 2.4.5
> Underlying table has data loaded.
> I think problem is while create materialized view as i am getting a error
> related to metastore.
>
>
> scala> carbon.sql("create MATERIALIZED VIEW agg_sales_mv as select country,
> sex,sum(quantity),avg(price) from sales group by country,sex").show()
> 20/08/26 01:04:41 AUDIT audit: \{"time":"August 26, 2020 1:04:41 AM
> IST","username":"root","opName":"CREATE MATERIALIZED
> VIEW","opId":"16462372696035311","opStatus":"START"}
> 20/08/26 01:04:45 AUDIT audit: \{"time":"August 26, 2020 1:04:45 AM
> IST","username":"root","opName":"CREATE
> TABLE","opId":"16462377160819798","opStatus":"START"}
> 20/08/26 01:04:46 AUDIT audit: \{"time":"August 26, 2020 1:04:46 AM
> IST","username":"root","opName":"CREATE
> TABLE","opId":"16462377696791275","opStatus":"START"}
> 20/08/26 01:04:48 AUDIT audit: \{"time":"August 26, 2020 1:04:48 AM
> IST","username":"root","opName":"CREATE
> TABLE","opId":"16462377696791275","opStatus":"SUCCESS","opTime":"2326
> ms","table":"NA","extraInfo":{}}
> 20/08/26 01:04:48 AUDIT audit: \{"time":"August 26, 2020 1:04:48 AM
> IST","username":"root","opName":"CREATE
> TABLE","opId":"16462377160819798","opStatus":"SUCCESS","opTime":"2955
> ms","table":"default.agg_sales_mv","extraInfo":{"local_dictionary_threshold":"1","bad_record_path":"","table_blocksize":"1024","local_dictionary_enable":"true","flat_folder":"false","external":"false","sort_columns":"","comment":"","carbon.column.compressor":"snappy","mv_related_tables":"sales"}}
> 20/08/26 01:04:50 ERROR CarbonInternalMetastore$: Adding/Modifying
> tableProperties operation failed:
> org.apache.spark.sql.hive.HiveExternalCatalog cannot be cast to
> org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener
> 20/08/26 01:04:50 ERROR CarbonInternalMetastore$: Adding/Modifying
> tableProperties operation failed:
> org.apache.spark.sql.hive.HiveExternalCatalog cannot be cast to
> org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener
> 20/08/26 01:04:51 AUDIT audit: \{"time":"August 26, 2020 1:04:51 AM
> IST","username":"root","opName":"CREATE MATERIALIZED
> VIEW","opId":"16462372696035311","opStatus":"SUCCESS","opTime":"10551
> ms","table":"NA","extraInfo":{"mvName":"agg_sales_mv"}}
> ++
> ||
> ++
> ++
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
