Re: [ANNOUNCE] Ajantha as new Apache CarbonData committer

2019-10-03 Thread Mohammad Shahid Khan
Congrats Ajantha On Thu, 3 Oct, 2019, 11:48 PM Ajantha Bhat, wrote: > Thank you all. > > On Thu, 3 Oct, 2019, 6:47 PM Kunal Kapoor, > wrote: > > > Congratulations ajantha > > > > On Thu, Oct 3, 2019, 5:30 PM Liang Chen wrote: > > > > > Hi > > > > > > > > > We are pleased to announce that the P

Re: [ANNOUNCE] Akash as new Apache CarbonData committer

2019-04-25 Thread Mohammad Shahid Khan
Congrats Akash Regards, Mohammad Shahid Khan On Thu 25 Apr, 2019, 7:51 PM Liang Chen, wrote: > Hi all > > We are pleased to announce that the PMC has invited Akash as new > Apache CarbonData > committer, and the invite has been accepted! > > Congrats to Akash and welcom

Re: [DISCUSS]Query performance Multiple and possible memory leaks

2019-01-16 Thread Mohammad Shahid Khan
@xuchuanyin Count star on carbon rely over the carbon index and carbonindexmerge so it will load the index files into the driver LRU cache. @litao If there is no other query and executing the same sql again then increasing memory footprint from 14g to 20g really looks like a memory leak. -Regards

Re: [ANNOUNCE] Chuanyin Xu as new PMC for Apache CarbonData

2019-01-02 Thread Mohammad Shahid Khan
Congrats Regards, Shahid On Wed 2 Jan, 2019, 5:49 AM Liang Chen Hi > > We are pleased to announce that Chuanyin Xu as new PMC for Apache > CarbonData > . > > Congrats to Chuanyin Xu! > > Apache CarbonData PMC >

Re: refresh table [CARBON-1.3.1]

2018-04-03 Thread Mohammad Shahid Khan
ow. *https://github.com/apache/carbondata/blob/master/docs/data-management-on-carbondata.md#refresh-table <https://github.com/apache/carbondata/blob/master/docs/data-management-on-carbondata.md#refresh-table>* Regards, Mohammad Shahid Khan On Tue, Apr 3, 2018 at 5:56 PM, sunerhan1...

Re: Storing Data Frame as CarbonData Table

2018-04-03 Thread Mohammad Shahid Khan
Hi Michael, Hope below details will help you. 1. How should I configure carbon to get performance ? Please refer below link to optimize data loading performance in Carbon. *https://github.com/apache/carbondata/blob/master/docs/useful-tips-on-carbondata.md#configuration-for-optimizing-data-loading

Re: Storing Data Frame as CarbonData Table

2018-04-03 Thread Mohammad Shahid Khan
Hi Michael, Hope below details will help you. 1. How should I configure carbon to get performance ? Please refer below link to optimize data loading performance in Carbon. *https://github.com/apache/carbondata/blob/master/docs/useful-tips-on-carbondata.md#configuration-for-optimizing-data-loading-

Problem with with writing the loadStartTime in "dd-MM-yyyy HH:mm:ss:SSS" format

2017-12-15 Thread Mohammad Shahid Khan
dateToStr = parser.parse(factTimeStamp); return dateToStr.getTime(); } catch (ParseException e) { LOGGER .error("Cannot convert" + factTimeStamp + " to Time/Long type value" + e.getMessage()); parser = new SimpleDateFormat(CarbonCommonConstants.CARBON_TIMESTAMP); try { // if the load is in progress, factTimeStamp will be null, so use current time if (null == factTimeStamp) { return System.currentTimeMillis(); } dateToStr = parser.parse(factTimeStamp); return dateToStr.getTime(); } catch (ParseException e1) { LOGGER.error( "Cannot convert" + factTimeStamp + " to Time/Long type value" + e1.getMessage()); return 0; } } } } Regards, Mohammad Shahid Khan

Re: DDL for CarbonData table backup and recovery (new feature)

2017-11-26 Thread Mohammad Shahid Khan
Hi Ravindra & Likun I am freezing the design and going to start the code. Please revert me if any issues. --Regards, Shahid On Fri, Nov 24, 2017 at 12:40 PM, mohdshahidkhan < mohdshahidkhan1...@gmail.com> wrote: > *Please update solution: > Instead of passing the dbLocation, the database name

DDL for CarbonData table backup and recovery (new feature)

2017-11-23 Thread Mohammad Shahid Khan
Hi Dev, *Please find initial solution.* *CarbonData table backup and recovery* *Background* Customer has created one CarbonData table which is already loaded very huge data, and now they install another cluster which want to use the same data as this table and don’t want load again, because lo

Re: [DISCUSSION] Support Database Location Configuration whileCreating Database

2017-10-31 Thread Mohammad Shahid Khan
similar to hive.. eg., CREATE TABLE [IF NOT EXISTS] [db_name.]table_name [LOCATION hdfs_path] --- Regards, Naresh P R On Thu, Oct 12, 2017 at 6:06 PM, Mohammad Shahid Khan < mohdshahidkhan1...@gmail.com> wrote: > Hi Dev, > > Please find updated design documents for "*Support

Re: [DISCUSSION] Support Database Location Configuration whileCreating Database

2017-10-12 Thread Mohammad Shahid Khan
Hi Dev, Please find updated design documents for "*Support Database Location Configuration while Creating Database*" Changes: Carbon will follow the same approach as hive is following. The table path should be formed from database location or fixed Carbon store location and table name as given be

Re: [DISCUSSION] Support Database Location Configuration while Creating Database

2017-10-05 Thread Mohammad Shahid Khan
nly. We should achieve that > inside carbon, it should only know the upper layer is > specified a table location to write table data. > > All database concept and commands should be managed by > upper layer. This is not conflicting with your requirement. > > Regards, > Jacky &

Re: [DISCUSSION] Support Database Location Configuration while Creating Database

2017-10-05 Thread Mohammad Shahid Khan
on HDFS and on S3 in same context, which also break the table > level abstraction. > > Regards, > Jacky > > > 在 2017年10月3日,下午10:36,Mohammad Shahid Khan > 写道: > > > > Hi Dev, > > Please find the design document for Support Database Location > Configuration while Creating Database. > > > > Regards, > > Shahid > > > > > >

[DISCUSSION] Support Database Location Configuration while Creating Database

2017-10-03 Thread Mohammad Shahid Khan
Hi Dev, Please find the design document for Support Database Location Configuration while Creating Database. Regards, Shahid Support Database Location.docx Description: MS-Word 2007 document

[jira] [Created] (CARBONDATA-1283) Carbon should continue with the default value if wrong value is set for the configurable parameter.

2017-07-10 Thread Mohammad Shahid Khan (JIRA)
Mohammad Shahid Khan created CARBONDATA-1283: Summary: Carbon should continue with the default value if wrong value is set for the configurable parameter. Key: CARBONDATA-1283 URL: https

[jira] [Created] (CARBONDATA-1258) CarbonData should not allow loading Date Type values violating the boundary condition ("0001-01-01" through "9999-12-31")

2017-07-03 Thread Mohammad Shahid Khan (JIRA)
Mohammad Shahid Khan created CARBONDATA-1258: Summary: CarbonData should not allow loading Date Type values violating the boundary condition ("0001-01-01" through "-12-31")

[jira] [Created] (CARBONDATA-1252) Add BAD_RECORD_PATH options in Load options in the Carbon Help doc

2017-06-29 Thread Mohammad Shahid Khan (JIRA)
Mohammad Shahid Khan created CARBONDATA-1252: Summary: Add BAD_RECORD_PATH options in Load options in the Carbon Help doc Key: CARBONDATA-1252 URL: https://issues.apache.org/jira/browse/CARBONDATA

[jira] [Created] (CARBONDATA-1249) Wrong order of columns i n redirected csv for bad records

2017-06-29 Thread Mohammad Shahid Khan (JIRA)
Mohammad Shahid Khan created CARBONDATA-1249: Summary: Wrong order of columns i n redirected csv for bad records Key: CARBONDATA-1249 URL: https://issues.apache.org/jira/browse/CARBONDATA-1249

[jira] [Created] (CARBONDATA-1239) Add validation for set command parameters

2017-06-27 Thread Mohammad Shahid Khan (JIRA)
Mohammad Shahid Khan created CARBONDATA-1239: Summary: Add validation for set command parameters Key: CARBONDATA-1239 URL: https://issues.apache.org/jira/browse/CARBONDATA-1239 Project

[jira] [Created] (CARBONDATA-1218) In case of data-load failure the BadRecordsLogger.badRecordEntry map holding the task Status is not removing the task Entry.

2017-06-22 Thread Mohammad Shahid Khan (JIRA)
Mohammad Shahid Khan created CARBONDATA-1218: Summary: In case of data-load failure the BadRecordsLogger.badRecordEntry map holding the task Status is not removing the task Entry. Key: CARBONDATA-1218

[jira] [Created] (CARBONDATA-1210) Exception should be thrown on bad record write failure to log file or csv file.

2017-06-21 Thread Mohammad Shahid Khan (JIRA)
Mohammad Shahid Khan created CARBONDATA-1210: Summary: Exception should be thrown on bad record write failure to log file or csv file. Key: CARBONDATA-1210 URL: https://issues.apache.org/jira/browse

[jira] [Created] (CARBONDATA-1207) Resource leak problem in CarbonDictionaryWriter

2017-06-21 Thread Mohammad Shahid Khan (JIRA)
Mohammad Shahid Khan created CARBONDATA-1207: Summary: Resource leak problem in CarbonDictionaryWriter Key: CARBONDATA-1207 URL: https://issues.apache.org/jira/browse/CARBONDATA-1207

[jira] [Created] (CARBONDATA-1101) Avoid widening between wrapper classes

2017-05-28 Thread Mohammad Shahid Khan (JIRA)
Mohammad Shahid Khan created CARBONDATA-1101: Summary: Avoid widening between wrapper classes Key: CARBONDATA-1101 URL: https://issues.apache.org/jira/browse/CARBONDATA-1101 Project

Re: I want to be a contribute in Apache carbon data

2017-05-22 Thread Mohammad Shahid Khan
Please follow below link https://github.com/apache/carbondata/blob/master/docs/How-to-contribute-to-Apache-CarbonData.md Regards, Shahid On 22 May 2017 16:34, "Kushal Sah" wrote: > Hi Team, > Please add me to the Apache community where I can contribute. > > Thanks > Kushal sah >

[jira] [Created] (CARBONDATA-1077) ColumnDict and ALL_DICTIONARY_PATH must be used with SINGLE_PASS='true'

2017-05-22 Thread Mohammad Shahid Khan (JIRA)
Mohammad Shahid Khan created CARBONDATA-1077: Summary: ColumnDict and ALL_DICTIONARY_PATH must be used with SINGLE_PASS='true' Key: CARBONDATA-1077 URL: https://issues.apache.org/j

Re: [ANNOUNCE] Ravindra as new Apache CarbonData PMC

2017-05-21 Thread Mohammad Shahid Khan
Congratulations Ravindra 😊 On Sun, May 21, 2017 at 2:59 PM, manish gupta wrote: > Congratulations Ravindra..:) > > Regards > Manish Gupta > > On Sat, May 20, 2017 at 2:28 PM, Naresh P R > wrote: > > > Congrats Ravindra. > > --- > > Regards, > > Naresh P R > > > > On May 19, 2017 4:56 PM, "Liang

[jira] [Created] (CARBONDATA-1061) If AL_DICTIONARY_PATH is used in load option then by SINGLE_PASS must be used.

2017-05-17 Thread Mohammad Shahid Khan (JIRA)
Mohammad Shahid Khan created CARBONDATA-1061: Summary: If AL_DICTIONARY_PATH is used in load option then by SINGLE_PASS must be used. Key: CARBONDATA-1061 URL: https://issues.apache.org/jira/browse

[jira] [Created] (CARBONDATA-1049) avoid logging data into log file

2017-05-12 Thread Mohammad Shahid Khan (JIRA)
Mohammad Shahid Khan created CARBONDATA-1049: Summary: avoid logging data into log file Key: CARBONDATA-1049 URL: https://issues.apache.org/jira/browse/CARBONDATA-1049 Project: CarbonData

[jira] [Created] (CARBONDATA-1027) insert into/data load failing for numeric dictionary included column having null value

2017-05-05 Thread Mohammad Shahid Khan (JIRA)
Mohammad Shahid Khan created CARBONDATA-1027: Summary: insert into/data load failing for numeric dictionary included column having null value Key: CARBONDATA-1027 URL: https://issues.apache.org/jira

Re: Use dev@carbondata.apache.org to test new mailing list without incubator

2017-05-01 Thread Mohammad Shahid Khan
ACK On Wed, Apr 26, 2017 at 11:47 PM, Vimal Das Kammath < vimaldas.kamm...@gmail.com> wrote: > ACK > > On Tue, Apr 25, 2017 at 10:14 PM, Henry Saputra > wrote: > > > ACK > > On Tue, Apr 25, 2017 at 5:54 AM Liang Chen > > wrote: > > > > > > > > > > >

[jira] [Created] (CARBONDATA-991) add test case for validation the bad record log file creation at the configured bad store location

2017-04-26 Thread Mohammad Shahid Khan (JIRA)
Mohammad Shahid Khan created CARBONDATA-991: --- Summary: add test case for validation the bad record log file creation at the configured bad store location Key: CARBONDATA-991 URL: https

[jira] [Created] (CARBONDATA-965) dataload fail message is not correct when there is no good data to load

2017-04-20 Thread Mohammad Shahid Khan (JIRA)
Mohammad Shahid Khan created CARBONDATA-965: --- Summary: dataload fail message is not correct when there is no good data to load Key: CARBONDATA-965 URL: https://issues.apache.org/jira/browse/CARBONDATA