Re: [DISCUSSION] Support JOIN query with spatial index

+1 -- Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

Re: [Discussion]Presto Queries leveraging Secondary Index

Hi all.! As discussed in the community meeting held on last week of Feb 2021, we already have plan to make Secondary Index as a Coarse Grain Datamap in the future. And It would be more appropriate for this requirement to implement Secondary Index as the CG Datamap. Presto query can leverage

Re: [DISCUSSION] Describe complex columns

+1 Nice idea. It improves the readibility. Regards, Venu -- Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

Re: [DISCUSSION] Display the segment ID when carbondata load is successful

+1 Good idea. Agree with you. Regards, Venu -- Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

Re: Improve carbondata CDC performance

Hi Akash, +1 Just ad few queries regd. query to filter only necessary target files/blocks. - SELECT target.filepath FROM targetTable,srcTable WHERE srcTable.value BETWEEN targetTable.min AND targetTable.max; 1. This range will be one per the extended blocklet. right ? I

[Discussion]Presto Queries leveraging Secondary Index

Hi all.! At present Carbon table queries with Presto engine do not make use of indexes(SI, Bloom etc) in query processing. Exploring feasible approaches without query plan rewrite to make use of secondary indexes(if any available) similar to that of existing datamap. * Option 1: * Presto get

Re: [DISCUSSION] Geo spatial index algorithm improvement and UDFs enhancement

+1 Good to see Geo enhancements. Just few points/queries : 1. Util UDFs seem to take origin latitude and grid size as argments as well. Shall we inherit them from table specified in the query during query processing? Probably can avoid invalid/inconsistent origin latitude and grid size values

Re: [ANN] Indhumathi as new Apache CarbonData committer

Congratulations Indhumathi.! Regards, Venu -- Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

Re: [Discussion]Query Regarding Task launch mechanism for data load operations

Hi Vishal, Thank you for the response. Configuring `load_min_size_inmb` has helped to control the number of tasks to launch in case of load from csv and could eventually reduce the carnondata files along with. But in case of insert into table select from flow `loadDataFrame()`, problem didn't

[Discussion]Query Regarding Task launch mechanism for data load operations

Hi all.! For the past few days, was trying to tune the perfomance parameters to improve the data loading speed(Carbon with Spark) for a new customer. During the course of tuning, have noticed a few points and stuck there as it seem to be base behavior. Could you guys help me to understand the

Re: [DISCUSSION] Presto+Carbon transactional and Non-transactional Write Support

+1 Good to enhance Carbon with Presto engine support. Regards, Venu -- Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

Re: [Discussion] Implement delete and update feature in carbondata SDK.

+1 Have a small query regaridng this update API - public CarbonSDKUID update(String path, String column, String value, String updColumn, String updValue); I believe column argument is column to be matched for the given value argument. so it is matchColumn & matchValue. Upon match we update

[Discussion]Do we still need to support carbon.merge.index.in.segment property ?

Dear Community.! Have recently encountered a problem of Segment directory and segment file in metadata directory are not created for partitioned table when 'carbon.merge.index.in.segment' property is set to 'false'. And actual index files which were present in respective partition's '.tmp'

[Discussion]Float and Double compatibility issue with external segments to Carbon

Hi Community! Need a help!! Was trying to create a carbon table with float field. I understand that internally it stores float as double in its schema. When we try to add an externa segment with float field, add segment/queries on ext segment would not be successful due to datatype mismatch.

Re: [DISCUSSION]Support for Geospatial indexing

@Ajantha Agreed. Have updated the design doc as suggested. -- Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

Re: [DISCUSSION]Support for Geospatial indexing

Hi all, I've refreshed the design document in jira. Have incorporated changes to table properties and fixed review comments. Please find the latest design doc at https://issues.apache.org/jira/browse/CARBONDATA-3548 Request review and let me know your opinion. Thanks, Venu Reddy -- Sent

Re: [DISCUSSION]Support for Geospatial indexing

@xuchuanyin With this new property, we can create a non-schema column internally and can generate the customized value to it upon each row add from the existing schema columns values(i.e., from source column values). Note that source columns are specified with property. Since the

Re: [DISCUSSION]Support for Geospatial indexing

@Indhumathi Please find my reply inline 1. Does geospatial index supports on streaming table?. If so, will there be any impact on generating geoIndex on steaming data? => Yes. We can support for steaming tables as well. But we shall restrict it for now and enhance in the future. 2. Does it have

Re: [DISCUSSION]Support for Geospatial indexing

1. Would table with geospatial location column be allowed to be updated with non-geospatial data and vice verca . Or would it according to the existing behavior and any unsupported data in type/column would be treated as bad records ? => Location columns cannot be allowed with invalid datatypes.