Re: [DISCUSSION] Support Avro Datatype Conversion to Carbon
Hi Indhumathi, Thanks for initiating the discussion. Please list the internal storage types in carbon how you are planning to store and retrieve these types in query. -- Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
Re: [DISCUSSION] Support Standard Spark's FileFormat interface in Carbondata
+ 1 for Carbon FileFormat. Thanks for the design details. Regards, Kanaka -- Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
Re: CarbonWriterBuild issue
+1 for the proposal to clear SDK APIs. Thanks Ajantha for initiating the code changes. For schema input for writer creation, I also feel we should unify to all writer creation methods to Builder. API looks cleaner if we provide just build() without out any more arguments. "withTableProperties(Map options) vs sortBy(..),withBlockSize(...),etc" - I think both of these methods can serve for different purposes. withTableProperties(Map options) can be used by customer apps which takes property input directly by end users who is familiar with carbon create table syntax. Individual methods can be used by customers app code to avoid problems like wrong spells or wrong names. "public CarbonWriterBuilder isTransactionalTable(boolean isTransactionalTable)" -- I think we can remove if we are not clear on the usecase at this moment and to avoid confusions -- Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
Re: [VOTE] Apache CarbonData 1.5.1(RC2) release
+1 I think CARBONDATA-3116 is not introduced in 1.5.1 & As Ravindra mentioned, If currently it is not intended for end users, we can discuss and optimize configuration in next version About the apache commons logging in TableDataMap.java#L78, as we verified our local cluster required messages still shown in log file. So, we can fix together with other classes in next version. Regards, Kanaka -- Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
Re: [VOTE] Apache CarbonData 1.5.4(RC1) release
+1 -- Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
Re: [Discussion] Blocklet DataMap caching in driver
Hi Manish, Thanks for proposing configured columns for min max cache. This will help customers who has large data but only few columns are used for filter condition. +1 for the solution 1. Regards, Kanaka On Fri, Jun 22, 2018 at 11:39 AM, manishgupta88 wrote: > Thanks Ravi for the feedback. I completely agree with you that we need to > develop the second solution ASAP. Please find my response below for your > queries. > > 1. what if the query comes on noncached columns, will it start read from > disk in driver side for minmax ? > - If query is on a non-cached column then all the blocks will be selected > and min/max pruning will be done in each executor. In driver side there > will > not be any read as it is a single process and it will increase the pruning > time if for every query min/max values are read from disk. So I feel it is > better to read in distributed way using the executors. > > 2. Are we planning to cache blocklet level information or block level > information in driver side for cached columns? > - We will provide an option to user to cache at Block or Blocklet level. It > will be configurable at table level and default caching will be at Block > level. I will cover this part in detail in the design document. > > 3. What is the impact if we automatically chose cached columns from the > user query instead of letting the user configure them? > - Every query can have different filter columns. So if we choose > automatically then for every different column it will read from disk and > load into cache. This can be more cumbersome and query time can vary > unexpectedly which may not be justifiable. So I feel it is better to let > user to decide which columns to be cached. > > Let me know for any more clarifications. > > Regards > Manish Gupta > > > > -- > Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556. > n5.nabble.com/ >
Re: [VOTE] Apache CarbonData 1.4.1(RC2) release
+1 Many enhancements.. -Regards Kanaka On Fri, 10 Aug 2018, 08:14 Ravindra Pesala, wrote: > Hi > > > I submit the Apache CarbonData 1.4.1 (RC2) for your vote. > > > 1.Release Notes: > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12320220&version=12343148 > > Some key features and improvements in this release: > >1. Supported Local dictionary to improve IO and query performance. >2. Improved and stabilized Bloom filter datamap. >3. Supported left outer join MV datamap(Alpha feature) >4. Supported driver min max caching for specified columns and >segregate block and blocklet cache. >5. Support Flat folder structure in carbon to maintain the same folder >structure as Hive. >6. Supported S3 read and write on carbondata files >7. Support projection push down for struct data type. >8. Improved complex datatypes compression and performance through >adaptive encoding. >9. Many Bug fixes and stabilized carbondata. > > > 2. The tag to be voted upon : apache-carbondata-1.4.1.rc2(commit: > a17db2439aa51f6db7da293215f9732ffb200bd9) > > > https://github.com/apache/carbondata/releases/tag/apache-carbondata-1.4.1-rc2 > > > 3. The artifacts to be voted on are located here: > > https://dist.apache.org/repos/dist/dev/carbondata/1.4.1-rc2/ > > > 4. A staged Maven repository is available for review at: > > https://repository.apache.org/content/repositories/orgapachecarbondata-1032 > > > 5. Release artifacts are signed with the following key: > > *https://people.apache.org/keys/committer/ravipesala.asc > < > https://link.getmailspring.com/link/1524823736.local-38e60b2f-d8f4-v1.2.1-7e744...@getmailspring.com/9?redirect=https%3A%2F%2Fpeople.apache.org%2Fkeys%2Fcommitter%2Fravipesala.asc&recipient=ZGV2QGNhcmJvbmRhdGEuYXBhY2hlLm9yZw%3D%3D > >* > > > Please vote on releasing this package as Apache CarbonData 1.4.1, The vote > > will be open for the next 72 hours and passes if a majority of > > at least three +1 PMC votes are cast. > > > [ ] +1 Release this package as Apache CarbonData 1.4.1 > > [ ] 0 I don't feel strongly about it, but I'm okay with the release > > [ ] -1 Do not release this package because... > > > Regards, > Ravindra. >
Re: [ANNOUNCE] Bo Xu as new Apache CarbonData committer
Congrats Xubo. -Regards Kanaka On Sat 8 Dec, 2018, 09:41 Raghunandan S Congrats xubo. Welcome on board > > On Sat, 8 Dec 2018, 8:37 am Liang Chen, wrote: > > > Hi all > > > > We are pleased to announce that the PMC has invited Bo Xu as new > > Apache CarbonData > > committer, and the invite has been accepted! > > > > Congrats to Bo Xu and welcome aboard. > > > > Regards > > Apache CarbonData PMC > > >
Re: [ANNOUNCE] Kunal Kapoor as new PMC for Apache CarbonData
Congratulations Kunal !!! -Regards Kanaka On Sun, 29 Mar, 2020, 12:37 Liang Chen, wrote: > Hi > > > We are pleased to announce that Kunal Kapoor as new PMC for Apache > CarbonData. > > > Congrats to Kunal Kapoor! > > > Apache CarbonData PMC >
