Re: CarbonWriterBuild issue
+1 I feel it is better to remove the transactional flag from sdk API as it is redundant currently. we better support it in a better way in future. -- Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
Re: CarbonWriterBuild issue
Also now we that we support Hadoop conf, we don't require below API. we can remove them from CarbonWriterBuilder. *setAccessKeysetAccessKeysetSecretKeysetSecretKeysetEndPointsetEndPoint* Thanks, AB On Thu, Sep 20, 2018 at 11:16 PM Ajantha Bhat wrote: > > @xuchuanyin: > > yes, method signatures will be like you specified. > > @kanaka: I still think we should keep only table properties Map as we > validate "wrong_spells and names". More options will create more confusion. > So, just keeping table properties Map can simplify configurations. End > user can form a map and pass. Just like existing withLoadOptions map > > Any other suggestions are welcome > > Thanks, > AB > . > > On Thu, Sep 20, 2018 at 10:55 PM kanaka > wrote: > >> +1 for the proposal to clear SDK APIs. >> Thanks Ajantha for initiating the code changes. >> >> For schema input for writer creation, I also feel we should unify to all >> writer creation methods to Builder. API looks cleaner if we provide just >> build() without out any more arguments. >> >> >> "withTableProperties(Map options) vs >> sortBy(..),withBlockSize(...),etc" >> - I think both of these methods can serve for different purposes. >> withTableProperties(Map options) can be used by customer >> apps which takes property input directly by end users who is familiar with >> carbon create table syntax. >> Individual methods can be used by customers app code to avoid problems >> like >> wrong spells or wrong names. >> >> "public CarbonWriterBuilder isTransactionalTable(boolean >> isTransactionalTable)" >> -- I think we can remove if we are not clear on the usecase at this moment >> and to avoid confusions >> >> >> >> >> >> >> -- >> Sent from: >> http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/ >> >
Re: CarbonWriterBuild issue
Yeah, it actually belongs to 'Builder Pattern'. We should simplify this before they are widely used. -- Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
CarbonWriterBuild issue
Hi Dev, I observed that we have added many buildXXXWriter in CarbonWriterBuilder in SDK, but all of them is just different parameter combination. I think it is better to add those parameter by withXXX function, which complies to Builder pattern. Otherwise it is hard to maintain if we keep adding these parameter and new build functions. Any suggestion on this? public CarbonWriter buildWriterForCSVInput(Schema schema, Configuration configuration) public CarbonWriter buildThreadSafeWriterForCSVInput(Schema schema, short numOfThreads, Configuration configuration) public CarbonWriter buildWriterForAvroInput(org.apache.avro.Schema avroSchema, Configuration configuration) public CarbonWriter buildThreadSafeWriterForAvroInput(org.apache.avro.Schema avroSchema, short numOfThreads, Configuration configuration) public JsonCarbonWriter buildWriterForJsonInput(Schema carbonSchema, Configuration configuration) public JsonCarbonWriter buildThreadSafeWriterForJsonInput(Schema carbonSchema, short numOfThreads, Configuration configuration) Regards, Jacky
