[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4110: [WIP]Secondary Index as a coarse grain datamap
CarbonDataQA2 commented on pull request #4110: URL: https://github.com/apache/carbondata/pull/4110#issuecomment-809125024 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/3348/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [VOTE] Apache CarbonData 2.1.1(RC2) release
+1(binding) Regards Liang -- Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
[GitHub] [carbondata] ShreelekhyaG opened a new pull request #4113: [WIP] Describe complex columns
ShreelekhyaG opened a new pull request #4113: URL: https://github.com/apache/carbondata/pull/4113 ### Why is this PR needed? Currently describe formatted displays the column information of a table and some additional information. When complex types such as ARRAY, STRUCT, and MAP types are present in the table, column definition can be long and it’s difficult to read in a nested format. ### What changes were proposed in this PR? For complex types available, the DESCRIBE output can be formatted to avoid long lines for multiple fields. We can pass the complex field name to the command and visualize its structure as if were a table. DDL Commands: ``` DESCRIBE COLUMN fieldname ON [db_name.]table_name; DESCRIBE short [db_name.]table_name; ``` ### Does this PR introduce any user interface change? - Yes ### Is any new testcase added? - Yes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] VenuReddy2103 commented on pull request #4110: [WIP]Secondary Index as a coarse grain datamap
VenuReddy2103 commented on pull request #4110: URL: https://github.com/apache/carbondata/pull/4110#issuecomment-809203807 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4113: [WIP] Describe complex columns
CarbonDataQA2 commented on pull request #4113: URL: https://github.com/apache/carbondata/pull/4113#issuecomment-809254008 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/3349/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4113: [WIP] Describe complex columns
CarbonDataQA2 commented on pull request #4113: URL: https://github.com/apache/carbondata/pull/4113#issuecomment-809254470 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/5100/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4110: [WIP]Secondary Index as a coarse grain datamap
CarbonDataQA2 commented on pull request #4110: URL: https://github.com/apache/carbondata/pull/4110#issuecomment-809273056 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/3350/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4110: [WIP]Secondary Index as a coarse grain datamap
CarbonDataQA2 commented on pull request #4110: URL: https://github.com/apache/carbondata/pull/4110#issuecomment-809275416 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/5101/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [Discussion]Presto Queries leveraging Secondary Index
+1 for the design On Tue, Mar 23, 2021 at 10:37 AM VenuReddy wrote: > Hi all.! > > As discussed in the community meeting held on last week of Feb 2021, we > already have plan to make Secondary Index as a Coarse Grain Datamap in the > future. And It would be more appropriate for this requirement to implement > Secondary Index as the CG Datamap. Presto query can leverage secondary > index > in the pruning through the datamap interface. Spark queries can still > continue to make use of secondary indexes with existing approach of query > plan modification. > > Have added the detailed design in the below doc. > > > https://docs.google.com/document/d/1VZlRYqydjzBXmZcFLQ4Ty-lK8RQlYVDoEfIId7vOaxk/edit?usp=sharing > > Please review it and let me know your suggestions/inputs. > > Thanks, > Venu Reddy > > > > -- > Sent from: > http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/ >
Re: Improve carbondata CDC performance
+1, agree with ravi's suggestion On Thu, Mar 11, 2021 at 7:53 PM Ravindra Pesala wrote: > +1 > Instead of doing the cartesian join, we can broadcast the sorted min/max > with file paths and do the binary search inside the map function. > > Thank you > > On Wed, 24 Feb 2021 at 13:02, akashrn5 wrote: > > > Hi Venu, > > > > Thanks for your review. > > > > I have replied the same in the document. > > you are right > > > > 1. its taken care to group by extended blocklets on split path and get > the > > min-max on block level > > 2. we need to do group by on the file path to avoid the duplicates from > > dataframe output. I have updated the same in the doc please have a look. > > > > Thanks, > > Akash R > > > > > > > > -- > > Sent from: > > http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/ > > > > > -- > Thanks & Regards, > Ravi >
Re: [Discussion]Presto Queries leveraging Secondary Index
+1 for design. Please find my comments. 1. About updating IndexStatus.ENABLED property, Need to consider compatibility scenarios as well. 2. Can update the query behavior when carbon.enable.distributed.index and carbon.disable.index.server.fallback is enabled. Regards, Indhumathi M -- Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
Re: Improve carbondata CDC performance
+1 Regards, Indhumathi M -- Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
Re: [Discussion]Presto Queries leveraging Secondary Index
+1 Thanks, Ajantha On Mon, Mar 29, 2021 at 5:58 PM Indhumathi wrote: > +1 for design. > > Please find my comments. > > 1. About updating IndexStatus.ENABLED property, Need to consider > compatibility scenarios as well. > 2. Can update the query behavior when carbon.enable.distributed.index > and carbon.disable.index.server.fallback is enabled. > > > Regards, > Indhumathi M > > > > > -- > Sent from: > http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/ >
Re: [VOTE] Apache CarbonData 2.1.1(RC2) release
Hi all, PMC vote has passed for Apache Carbondata 2.1.1 release, the result is as below: +1(binding): 5（Kunal Kapoor, David CaiQiang, Kumar Vishal, Ravindra Pesala, Liang Chen) +1(non-binding) : 2 (Akash, Indhumathi) Thanks all for your vote. On Mon, Mar 29, 2021 at 12:57 PM Liang Chen wrote: > +1(binding) > > Regards > Liang > > > > -- > Sent from: > http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/ >
[GitHub] [carbondata] dependabot[bot] commented on pull request #3456: Bump solr.version from 6.3.0 to 8.3.0 in /datamap/lucene
dependabot[bot] commented on pull request #3456: URL: https://github.com/apache/carbondata/pull/3456#issuecomment-809411078 Dependabot tried to update this pull request, but something went wrong. We're looking into it, but in the meantime you can retry the update by commenting `@dependabot rebase`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] dependabot[bot] commented on pull request #3447: Bump dep.jackson.version from 2.6.5 to 2.10.1 in /store/sdk
dependabot[bot] commented on pull request #3447: URL: https://github.com/apache/carbondata/pull/3447#issuecomment-809411166 Dependabot tried to update this pull request, but something went wrong. We're looking into it, but in the meantime you can retry the update by commenting `@dependabot rebase`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [Discussion]Presto Queries leveraging Secondary Index
Hi, +1 for the feature and the design. I have give some comments on the design doc for handling some missing scenarios and small changes. Can you please update the design doc. As not so major comments except one or two, can go ahead with feature and parallelly can update comments. Thanks Regards, Akash R -- Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
Re: [GitHub] [carbondata] dependabot[bot] commented on pull request #3456: Bump solr.version from 6.3.0 to 8.3.0 in /datamap/lucene
Hi - pls unsubscribe me from below email. Thnx Sent from my iPad > On Mar 29, 2021, at 7:13 AM, GitBox wrote: > > > dependabot[bot] commented on pull request #3456: > URL: https://github.com/apache/carbondata/pull/3456#issuecomment-809411078 > > > Dependabot tried to update this pull request, but something went wrong. > We're looking into it, but in the meantime you can retry the update by > commenting `@dependabot rebase`. > > > -- > This is an automated message from the Apache Git Service. > To respond to the message, please log on to GitHub and use the > URL above to go to the specific comment. > > For queries about this service, please contact Infrastructure at: > us...@infra.apache.org > >
Re: [DISCUSSION] Support JOIN query with spatial index
+1 On Mon, Mar 22, 2021 at 4:07 PM Indhumathi wrote: > Hi community, > > Currently, carbon supports IN_POLYGON and IN_POLYGON_LIST udf's, > where user has to manually provide the polygon points(series of latitude > and longitude pair), to query carbon table based on spatial index. > > This feature will support JOIN tables based on IN_POLYGON udf > filter, where polygon data exists in a table. > > Please find below link of design doc. Please check and give > your inputs/suggestions. > > > https://docs.google.com/document/d/11PnotaAiEJQK_QvKsHznDy1I9tO4idflW32LstwcLhc/edit#heading=h.yh6qp815dh3p > > > Thanks & Regards, > Indhumathi M > > > > -- > Sent from: > http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/ >
Re: [DISCUSSION] Support alter schema for complex types
+1 On Fri, Mar 26, 2021 at 6:19 PM akshay_nuthala wrote: > No, these and other nested level operations will be taken care in the next > phase. > > > > -- > Sent from: > http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/ >
Re: [DISCUSSION] Support alter schema for complex types
Hi Akshay, The mail description and document content are not matching. For single-level struct also document says cannot support. So, please list down all the work that need to be done in points and then divide which is supported in phase1 and which is supported in phase 2 clearly in the summary section of the document. Also in the query flow, after adding the column, for previously loaded segments what will be the output NULL or empty complex type ? you can refer hive behavior for this. Hope schema evolution (column drift) also intact with complex column support. Thanks, Ajantha On Tue, Mar 30, 2021 at 11:18 AM Kunal Kapoor wrote: > +1 > > On Fri, Mar 26, 2021 at 6:19 PM akshay_nuthala > wrote: > > > No, these and other nested level operations will be taken care in the > next > > phase. > > > > > > > > -- > > Sent from: > > http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/ > > >
