Re: Trafodion 2.0.1 Sandbox
Hi there All- Has anyone had a chance to test this out? I know it works for me :) but does it work for everyone else? Thanks a bunch! Amanda On Wed, Jun 29, 2016 at 11:29 AM, Amanda Moran <amanda.mo...@esgyn.com> wrote: > Hi there All- > > I have created a Trafodion 2.0.1 VirtualBox VM with HDP 2.3 > > > https://www.dropbox.com/s/ij70vomlt1123d8/trafodion_2.0.1.ova?dl=0 > > *Instructions * > > >1. Download sandbox (this will take a bit of time :)) >2. Import Sandbox into VirtualBox >3. *Under "Settings-> Network" you will need to toggle these settings >to match your set up, so that you have an internet connect on the VM and a >way to access the VM via putty/winscp etc. This will take some >experimenting on your part. HDP has issues starting up if this isn't set >correcting.* >4. Start VM >5. Select "other" at log in: root/traf123 or log in as trafodion >/traf123 >6. Open Firefox -> Open Ambari (localhost:8080) (admin/admin) -> Start >All services >7. Once everything is started open a terminal -> hbase shell -> status >(If no errors come up then everything started correctly... *do not try >to start Trafodion if HDFS/HBase* has not come up). >8. sudo su trafodion >9. sqstart >10. sqlci -> create table -> get table > > Might be needed: http://trafodion.apache.org/port-assignment.html > > Thanks! Let me know if you have any issues or feedback :) > > -- > Thanks, > > Amanda Moran > -- Thanks, Amanda Moran
Re: 答复: HDFS/HBase/Zookeeper Settings set by Installer
Thanks a lot for the help Ming (and thanks for the kind words Pierre!). I will take a look at testing without adding the coprocessor settings (but I won't remove from the installer for now). Thanks! Amanda On Tue, May 3, 2016 at 12:42 AM, Liu, Ming (Ming) <ming@esgyn.cn> wrote: > Yes, this is very important topic. > > > > I can confirm to remove two of them : > > Name: hbase.bulkload.staging.dir > > Value: /hbase-staging > > Why: > > Still Needed: NO > > > > Name: hbase.regionserver.region.transactional.tlog > > Value: true > > Why: > > Still Needed: NO > > > > We need more test for : > > > > Name: hbase_coprocessor_region_classes > > Value: > "org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionObserver,org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionEndpoint,org.apache.hadoop.hbase.coprocessor.AggregateImplementation" > > Why: Not fully tested to remove it. > > Still needed: Yes > > > > There is already a fix that can get rid of this settings. But we need more > test. That is, remove this from installer and see if any issues. We tested > it briefly and it works well, but never get it full QAed. Trafodion now > should add coprocessor at runtime if installer not doing this. > > > > Another one I have comment: > > > > Name: hbase.hregion.impl > > Value: > org.apache.hadoop.hbase.regionserver.transactional.TransactionalRegion > > Why: Trafodion coprocessor need to invoke a private API of HBase Region, > so have to overwrite this class now. When Trafodion move to HBase 1.2, it > is possible to get rid of it, since HBase R1.2 public one required method > for us. See https://issues.apache.org/jira/browse/HBASE-15076?filter=-2 > > Still Needed: Yes. > > > > > > IMHO, all other settings should not be mandatory, but kind of > performance/stability tunings. However, I am not the author of those > settings, so not very sure. It is better others to comment more. > > > > Yes, thanks Amanda to bring this up, and I hope you can keep pushing this > topic. > > I feel some settings there is too old that nobody knows the reason or just > forget, I think testing is a solution: remove it, and test to see if any > issue. But if someone knows that one is MUST-TO-KEEP or DELETE-IT, it is > better to reply. > > > > Thanks, > > Ming > > > > *发件人:* Pierre Smits [mailto:pierre.sm...@gmail.com] > *发送时间:* 2016年5月3日 15:20 > *收件人:* user@trafodion.incubator.apache.org > *主题:* Re: HDFS/HBase/Zookeeper Settings set by Installer > > > > Hi Amanda, > > > > Thanks for bringing this up. Especially the 'why' aspect. This will be > good input for the documentation. > > > > Best regards, > > > Pierre Smits > > > *ORRTIZ.COM <http://www.orrtiz.com>* > > OFBiz based solutions & services > > > *OFBiz Extensions Marketplace* > > http://oem.ofbizci.net/oci-2/ > > > > On Mon, May 2, 2016 at 10:54 PM, Amanda Moran <amanda.mo...@esgyn.com> > wrote: > > Hi there All- > > > > I have been looking over the HDFS/HBase/Zookeeper settings that get set in > the installer and I am wondering if they are all still needed. > > > > If you have requested a setting in the past, could you please add a > description of why it is needed (and if it is still needed)? > > > > Thanks a bunch! > > > > **Note: I know this would have looked much better in a spreadsheet but > I want to make sure everyone can see... and email is the best for that! > > > > *HDFS Settings* > > > > Name: namenode_java_heapsize > > Value: 1GB (or 1073741824 bytes) > > Why: > > Still needed: > > > > Name: secondary_namenode_java_heapsize > > Value: 1 GB (or 1073741824 bytes) > > Why: > > Still needed: > > > > Name: dfs_namenode_acls_enabled > > Value: true > > Why: > > Still needed: > > > > *HBase Master Settings* > > > > =HBase Master Config Safety Valve= > > > > Name: hbase_master_distributed_log_splitting > > Value: false > > Why: > > Still needed: > > > > Name: hbase_snapshot_master_timeoutMillis > > Value: 60 > > Why: > > Still needed: > > > > *HBase Region Server Settings * > > > > Name: hbase_coprocessor_region_classes > > Value: > "org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionObserver,org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionEndpoint,org.apache.hadoop.hbase.coprocessor.AggregateImplementation" > > Why: > > Still needed: >
HDFS/HBase/Zookeeper Settings set by Installer
Hi there All- I have been looking over the HDFS/HBase/Zookeeper settings that get set in the installer and I am wondering if they are all still needed. If you have requested a setting in the past, could you please add a description of why it is needed (and if it is still needed)? Thanks a bunch! **Note: I know this would have looked much better in a spreadsheet but I want to make sure everyone can see... and email is the best for that! *HDFS Settings* Name: namenode_java_heapsize Value: 1GB (or 1073741824 bytes) Why: Still needed: Name: secondary_namenode_java_heapsize Value: 1 GB (or 1073741824 bytes) Why: Still needed: Name: dfs_namenode_acls_enabled Value: true Why: Still needed: *HBase Master Settings* =HBase Master Config Safety Valve= Name: hbase_master_distributed_log_splitting Value: false Why: Still needed: Name: hbase_snapshot_master_timeoutMillis Value: 60 Why: Still needed: *HBase Region Server Settings * Name: hbase_coprocessor_region_classes Value: "org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionObserver,org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionEndpoint,org.apache.hadoop.hbase.coprocessor.AggregateImplementation" Why: Still needed: Name: hbase_regionserver_lease_period Value: 60 Why: Still Needed: =HBase RegionServer Config Safety Valve= Name: hbase.hregion.impl Value: org.apache.hadoop.hbase.regionserver.transactional.TransactionalRegion Why: Still Needed: Name: hbase.regionserver.region.split.policy Value: org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy Why: Still Needed: Name: hbase.snapshot.enabled Value: true Why: Still Needed: Name: hbase.bulkload.staging.dir Value: /hbase-staging Why: Still Needed: Name: hbase.regionserver.region.transactional.tlog Value: true Why: Still Needed: Name: hbase.snapshot.region.timeout Value: 60 Why: Still Needed: *Zookeeper Settings* Name: maxClientCnxns Value: 0 Why: Still Needed: -- Thanks, Amanda Moran
Re: installation prerequisites
Yes, you could build the code yourself and then create a tar ball to install. You can also use a daily build just for testing/installation that we have here...this is not a released version just something to help. http://downloads.trafodion.org/ Thanks! On Fri, Apr 1, 2016 at 12:58 PM, Sharu Gupta <gupta.sh...@gmail.com> wrote: > Thank you for your reply. > I went to http://trafodion.apache.org/download.html to get started. There > is no mention the version number on the link. > > Can the daily build be cloned from git? > > Sharu > > > On Fri, Apr 1, 2016 at 2:27 PM, Amanda Moran <amanda.mo...@esgyn.com> > wrote: > >> HI there- >> >> Which version of Trafodion are you going to try to install? 1.3 release >> of latest daily build? >> >> Trafodion 1.3 release does not work on 5.4 or 5.5 >> >> The daily build should now work on 5.4 (we just merged in that change >> yesterday ..) >> >> Thanks! >> >> Amanda >> >> On Fri, Apr 1, 2016 at 12:19 PM, Sharu Gupta <gupta.sh...@gmail.com> >> wrote: >> >>> Hello, >>> >>> I am a new user on your mailing list. I was reading the instructions to >>> download and install Trafodion. >>> >>> The instructions say that either Cloudera CDH 5.3 or CDH 5.2 are >>> required. However, I can only find CDH versions 5.4 or higher on >>> http://www.cloudera.com/downloads/quickstart_vms/5-5.html >>> >>> Will it be OK to use hadoop distribution by CDH 5.4 or higher? >>> >>> Thank you, >>> Sharu >>> >> >> >> >> -- >> Thanks, >> >> Amanda Moran >> > > -- Thanks, Amanda Moran
Re: installation prerequisites
HI there- Which version of Trafodion are you going to try to install? 1.3 release of latest daily build? Trafodion 1.3 release does not work on 5.4 or 5.5 The daily build should now work on 5.4 (we just merged in that change yesterday ..) Thanks! Amanda On Fri, Apr 1, 2016 at 12:19 PM, Sharu Gupta <gupta.sh...@gmail.com> wrote: > Hello, > > I am a new user on your mailing list. I was reading the instructions to > download and install Trafodion. > > The instructions say that either Cloudera CDH 5.3 or CDH 5.2 are required. > However, I can only find CDH versions 5.4 or higher on > http://www.cloudera.com/downloads/quickstart_vms/5-5.html > > Will it be OK to use hadoop distribution by CDH 5.4 or higher? > > Thank you, > Sharu > -- Thanks, Amanda Moran
Re: Trafodion 1.3 Sandbox
Sorry all! Forgot that in the steps! admin:admin This is the default for HDP. Thanks! Sent from my iPhone > On Mar 27, 2016, at 7:21 AM, Pierre Smits <pierre.sm...@gmail.com> wrote: > > Hi Amanda, > > The steps up to #5 worked like a breeze. I used root/traf123 to login. > > #6: Open the browser and open Ambari to start all services didn't work. > Starting this requires a userId and password too. So I couldn't continue. > > Best regards, > > Pierre Smits > > ORRTIZ.COM > OFBiz based solutions & services > > OFBiz Extensions Marketplace > http://oem.ofbizci.net/oci-2/ > >> On Fri, Mar 25, 2016 at 9:48 PM, Amanda Moran <amanda.mo...@esgyn.com> wrote: >> Hi there All- >> >> I have created a Trafodion 1.3 VirtualBox VM with HDP 2.2. >> >> https://www.dropbox.com/s/5i2fk4nbpltw11m/trafodion_1.3.ova?dl=0 >> >> Instructions >> >> Download sandbox (this will take a bit of time :)) >> Import Sandbox into VirtualBox >> Under "Settings-> Network" you will need to toggle these settings to match >> your set up, so that you have an internet connect on the VM and a way to >> access the VM via putty/winscp etc. This will take some experimenting on >> your part. HDP has issues starting up if this isn't set correcting. >> Start VM >> Select "other" at log in: root/traf123 or log in as trafodion/traf123 >> Open Firefox -> Open Ambari (localhost:8080) -> Start All services >> Once everything is started open a terminal -> hbase shell -> status (If no >> errors come up then everything started correctly... do not try to start >> Trafodion if HDFS/HBase has not come up). >> sudo su trafodion >> sqstart >> sqlci -> create table -> get table >> >> Thanks! Let me know if you have any issues or feedback :) >> >> -- >> Thanks, >> >> Amanda Moran >
Re: Trafodion 1.3 Sandbox
Hi Pierre! You are welcome! Let me know if you have any issues with it. Thanks! Amanda On Fri, Mar 25, 2016 at 2:02 PM, Pierre Smits <pierre.sm...@gmail.com> wrote: > Hi Amanda, > > Thank you form making that available. I will download and test it shortly. > > Best regards, > > Pierre > > > Op vrijdag 25 maart 2016 heeft Amanda Moran <amanda.mo...@esgyn.com> het > volgende geschreven: > >> Hi there All- >> >> I have created a Trafodion 1.3 VirtualBox VM with HDP 2.2. >> >> https://www.dropbox.com/s/5i2fk4nbpltw11m/trafodion_1.3.ova?dl=0 >> >> *Instructions * >> >> >>1. Download sandbox (this will take a bit of time :)) >>2. Import Sandbox into VirtualBox >>3. Under "Settings-> Network" you will need to toggle these settings >>to match your set up, so that you have an internet connect on the VM and a >>way to access the VM via putty/winscp etc. This will take some >>experimenting on your part. HDP has issues starting up if this isn't set >>correcting. >>4. Start VM >>5. Select "other" at log in: root/traf123 or log in as >>trafodion/traf123 >>6. Open Firefox -> Open Ambari (localhost:8080) -> Start All services >>7. Once everything is started open a terminal -> hbase shell -> >>status (If no errors come up then everything started correctly... do not >>try to start Trafodion if HDFS/HBase has not come up). >>8. sudo su trafodion >>9. sqstart >>10. sqlci -> create table -> get table >> >> >> Thanks! Let me know if you have any issues or feedback :) >> >> -- >> Thanks, >> >> Amanda Moran >> > > > -- > Pierre Smits > > ORRTIZ.COM <http://www.orrtiz.com> > OFBiz based solutions & services > > OFBiz Extensions Marketplace > http://oem.ofbizci.net/oci-2/ > > -- Thanks, Amanda Moran
Anyway to start Trafodion without sqstart
Hi there- Is there a way to start up Trafodion not by using sqstart...? I would like to be able to start up/stop each node individually. Thanks! -- Thanks, Amanda Moran
Re: 答复: A failed Trafodion installation can lead to the hbase:meta table staying in the FAILED_OPEN state.
lassNotFoundException: Class
> org.apache.hadoop.hbase.regionserver.transactional.TransactionalRegion not
> found
> at
>
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2018)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2110)
> ... 11 more
> 2016-03-07 16:45:27,250 INFO
> org.apache.hadoop.hbase.coordination.ZkOpenRegionCoordination: Opening of
> region {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '',
> ENDKEY => ''} failed, transitioning from OPENING to FAILED_OPEN in ZK,
> expecting version 115
>
> After consulting with our installer expert, the issue was in fact that the
> needed files had not been copied to each node. At that point one option
> would be to re-install the previous build or at least undo the changes made
> to point to the new build. I did not try that and I'll leave that fallback
> option as a separate topic.
>
> Instead, I took the path to see if I could get HBase to successfully
> come up without getting the new Trafodion installation properly completed.
> To do that there are two HBase properties that have to be reset:
>
> . hbase.coprocessor.region.classes
> . hbase.hregion.impl
>
> I actually deleted all of the properties listed under the hbase-site.xml
> that showed as non-default values by Cloudera Manager but I assume only the
> hbase.hregion.impl property had to be removed. Remember to save the
> configuration and remove both sets of properties. I forgot to do both of
> those and each time the restart hit the same basic error.
>
> Once the configuration is properly updated the restart will be
> successful and after the hbase:meta table can be opened by the Region
> Server, all the other regions will also be able to be opened. However,
> without Trafodion running I would assume none of the Trafodion tables
> should be acted upon.
> This exercise was to prove HBase could be restarted and running so that
> when the Trafodion installation was started it would have a viable
> Cloudera/HBase/HDFS environment to act on.
>
>
--
Thanks,
Amanda Moran
Re: perl-DBD-SQLite*
Okay perfect. This is exactly what I wanted to know.
Thanks to Eason and Dennis for actually digging into the code!!
Appreciate it!
On Wed, Mar 9, 2016 at 5:36 AM, Zhang, Yi (Eason) <yi.zh...@esgyn.cn> wrote:
> Hi,
>
> In my VM environment there’s only perl-DBD-SQLite installed, both ‘sqgen’
> and Trafodion can work well, so I think the package perl-DBD-SQLite2 is not
> needed.
>
> Thanks,
> Eason
>
>
> From: "D. Markt"
> Reply-To: "user@trafodion.incubator.apache.org"
> Date: Wednesday, March 9, 2016 at 16:03
> To: "user@trafodion.incubator.apache.org"
> Subject: RE: perl-DBD-SQLite*
>
> Hi,
>
>
>
> I’m not a yum expert, but could the ‘*’ actually be part of a glob
> expression? For example on a CentOS 6.5 node I did:
>
>
>
> yum list installed "perl-DBD-SQLite*"
>
> …
>
> perl-DBD-SQLite.x86_64
> 1.27-3.el6@base
>
> perl-DBD-SQLite2.x86_64
> 0.33-12.el6 @epel
>
>
>
> Then again without the ‘*’:
>
>
>
> yum list installed "perl-DBD-SQLite"
>
> …
>
> perl-DBD-SQLite.x86_64
> 1.27-3.el6@base
>
>
>
> So the ‘*’ was likely used to install both packages, but from an
> Internet search it looks like there is a SQLite2 but one site indicates it
> has been deprecated and that SQLite is actually version 3 of the software
> (and that was circa 2009). I did find several files under my somewhat
> dated source tree that have the common Perl suffix of “.pl”. Many of those
> files are under the local_hadoop directory so aren’t Trafodion files given
> a quick look. Looking for the SQLite term that one site mentioned I see:
>
>
>
> ./core/sqf/sql/scripts/gensq.pl:use DBI;
>
> ./core/sqf/sql/scripts/gensq.pl:$DBH =
> DBI->connect("dbi:SQLite:dbname=sqconfig.db","","",$dbargs);
>
>
>
> So I appears this is used by sqgen, and it doesn’t specifically ask for
> SQLite2. Looking at the sqconfig.db from a today’s install I see:
>
>
>
> more sqconfig.db
>
> SQLite format 3
>
> …
>
>
>
> So if you install SQLite on RHEL 7.1, and see the generated sqconfig.db
> file has the same “SQLite format 3” (or higher), which I’m sure it will,
> then that should be enough. Of course it’s always possible some other
> component is using SQLite but my recollection was sqgen was the one
> component that was using the package. If other Perl scripts are using
> SQLite2 then they should probably be enhanced to use the newer version
> anyway.
>
>
>
> Regards,
>
> Dennis
>
>
>
> *From:* Amanda Moran [mailto:amanda.mo...@esgyn.com
> <amanda.mo...@esgyn.com>]
> *Sent:* Tuesday, March 08, 2016 6:03 PM
> *To:* user@trafodion.incubator.apache.org
> *Subject:* Re: perl-DBD-SQLite*
>
>
>
> Yes, I totally agree with the way to test.
>
>
>
> I am waiting on some changes to be merged and I will do just that.
>
>
>
> Just was hoping, if anyone had any words of wisdom before then!
>
>
>
> Thanks!
>
>
>
> On Tue, Mar 8, 2016 at 3:59 PM, Carol Pearson <carol.pearson...@gmail.com>
> wrote:
>
> Hi Amanda,
>
>
>
> At one point, I know we used SQLite for some internal configuration
> information, but I've lost track of whether or not we still do. Otherwise,
> SQLite would be needed for a dependency, and at that point and we'd have to
> track that one down to see what's really needed.
>
>
>
> If we don't install the full set, does the install complete and does
> Trafodion start? No guarantees that we don't have a problem if it installs
> and starts because the dependency could be later in the execution path.
> But if install/start fails, at least that tells us that the dependency
> matters and points us to at least one place *where* something cares.
>
>
>
> -Carol P.
>
>
> ---
>
> Email:carol.pearson...@gmail.com
>
> Twitter: @CarolP222
>
> -------
>
>
>
> On Tue, Mar 8, 2016 at 2:55 PM, Amanda Moran <amanda.mo...@esgyn.com>
> wrote:
>
> Hi there All-
>
>
>
> In the current installer we try to install this package: perl-DBD-SQLite*
> (note the *), on RHEL 6 and Centos 6 this has worked fine.
>
>
>
> I am testing the installer on RHEL 7.1 and it is not able to
> install perl-DBD-SQLite* only perl-DBD-SQLite.
>
>
>
> Is just installing perl-DBD-SQLite going to be an issue?
>
>
>
> Thanks!
>
>
>
> --
>
> Thanks,
>
>
>
> Amanda Moran
>
>
>
>
>
>
>
> --
>
> Thanks,
>
>
>
> Amanda Moran
>
--
Thanks,
Amanda Moran
Re: Trafodion Logo is LIVE!
Looks good! I think we made a good choice, the logo is cute! On Wed, Feb 24, 2016 at 11:38 AM, Gunnar Tapper <tapper.gun...@gmail.com> wrote: > Please checkout the web site and the documentation PDF files. > > Also, we've updated the Download page with an easy-to-use installation > procedure and created a Quick Start page to make it easy to try out > Trafodion. > > Please take a look at your earliest convenience. > > -- > Thanks, > > Gunnar > *If you think you can you can, if you think you can't you're right.* > -- Thanks, Amanda Moran
Re: 答复: Logo Proposal
+1 to 4g or 13. I will say I think 12 is hard to read... I don't think I have seen any votes for it but I think some fonts with all capitals would make Trafodion hard to read. Thanks! On Mon, Feb 15, 2016 at 5:43 PM, Venkat Muthuswamy < venkat.muthusw...@esgyn.com> wrote: > Hi Ming, > > > > HBase already has taken the Whale logo (Orca) J > > > > Venkat > > > > *From:* Liu, Ming (Ming) [mailto:ming@esgyn.cn] > *Sent:* Friday, February 12, 2016 6:35 PM > *To:* user@trafodion.incubator.apache.org > *Subject:* 答复: 答复: Logo Proposal > > > > Thanks Gunnar, > > > > I can see it now. I like dragon very much, but hope it is not too popular. > > I think we also think about whale before? > > > > The meaning behind whale is: it is the biggest animal on the earth , and > real J > > > > And Hadoop projects seem to use cute animals (Hive, Hadoop). Maybe we can > draw a baby dragon? > > > > And most people like sky blue? I used to have black or golden dragon as my > favorite J > > > > My 2 cents. > > > > Overall, I like this logo, it is already good enough for me. It shows to > me how powerful Trafodion is. I know it is hard to get consensus for this > kind of design. > > > > Thanks, > > Ming > > > > *发件人**:* Gunnar Tapper [mailto:tapper.gun...@gmail.com > <tapper.gun...@gmail.com>] > *发送时间:* 2016年2月13日 9:43 > *收件人:* user@trafodion.incubator.apache.org > *主题:* Re: 答复: Logo Proposal > > > > Trying an attachment... > > > > On Fri, Feb 12, 2016 at 6:26 PM, Liu, Ming (Ming) <ming@esgyn.cn> > wrote: > > Hi, Gunnar, > > > > Is it possible to put the logo as attachment? If they are not too big. I > saw QiFan once has an attachment of a logo in one of his post. Some people > cannot access Google drive . > > > > Thanks, > > Ming > > > > *发件人**:* Gunnar Tapper [mailto:tapper.gun...@gmail.com] > *发送时间**:* 2016年2月13日 4:31 > *收件人**:* user@trafodion.incubator.apache.org > *主题**:* Re: Logo Proposal > > > > Hi, > > > > Based on the input from several people in this thread, I'd like to propose > the following draft as a candidate: > https://drive.google.com/open?id=0BxlwNhWxn8iTQV85eDlyc0RxWGc > > > > I'm hoping that Ken, Sandhya, or someone else can make a real version for > consideration rather than my PowerPoint hack. It needs a bit of work on the > gray hue (s;lightly darker IMO), size of dragon, and alignment but that's > better done in a real designer program. The font was downloaded from Google > Fonts: Telex Regular. > > > > As mentioned, the draft fits into the overall website well: > https://drive.google.com/open?id=0BxlwNhWxn8iTczNMLTRjYUVXSkE and I'm > sure it'll meet the other criteria, too. > > > > Thanks, > > > > Gunnar > > > > > > > > > > > > On Fri, Feb 12, 2016 at 12:39 PM, Carol Pearson < > carol.pearson...@gmail.com> wrote: > > We also need our logo to look good on our customers' and users' and > ecosystem denizens' websites so they're eager to show off that Trafodion > fits into the puzzle ;-) . > > > > I don't think we're quite converged and ready for a real vote, and to be > honest, I hate to call for a 3 day vote on this over a long weekend in the > US and at the end of a holiday week in China since that's where most of our > traffic comes from. > > > > Further discussion over the weekend and final vote on Tuesday? > > > > -Carol P. > > > --- > > Email:carol.pearson...@gmail.com > > Twitter: @CarolP222 > > --- > > > > On Fri, Feb 12, 2016 at 9:34 AM, Venkat Muthuswamy < > venkat.muthusw...@esgyn.com> wrote: > > Having Apache in our logo, brings a lot of credibility and shows we are > serious. > > Let’s also keep in mind that the logo/text should look good on the > website, presentation slides and any brochures if you will. > > Certainly we need multiple colors to bring attention to the product name. > > > > Venkat > > > > *From:* Christophe LeRouzo [mailto:c...@esgyn.com] > *Sent:* Friday, February 12, 2016 7:59 AM > *To:* user@trafodion.incubator.apache.org > *Cc:* Rohit Jain <rohit.j...@esgyn.com>; Gunnar Tapper < > tapper.gun...@gmail.com> > *Subject:* RE: Logo Proposal > > > > It seems we might need multiple votes: > > - One color or multiple colors? > > - Thin font or thick font? > > -
Re: traf_*mods in trafodion installation
I agree with that :) Thanks Qifan! On Tue, Feb 9, 2016 at 8:43 AM, Qifan Chen <qifan.c...@esgyn.com> wrote: > Hi Amanda, > > I know the topic that I brought up is a little bit different :-). > > But the objectives of both thoughts are pretty much in line with the > improvement of install experiences. > > Thank you. > > --Qifan > > On Tue, Feb 9, 2016 at 10:36 AM, Amanda Moran <amanda.mo...@esgyn.com> > wrote: > >> Thanks for the response Qifan... but I am not sure we are talking about >> the same thing. >> >> I am talking about the traf__mods98 script that is run as >> apart of the Trafodion Installation scripts. It is used to setup HBase >> settings, copy hbase-trx*.jar file, and restart HBase, HDFS, and Zookeeper. >> >> I would like to know what are the smallest number of settings that need >> to be set in advance for Trafodion to install/start. >> >> Examples of things set: >> hbase-site hbase.master.distributed.log.splitting false >> >> hbase-site hbase.coprocessor.region.classes >> "org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionObserver,org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionEndpoint,org.apache.hadoop.hbase.coprocessor.AggregateImplementation" >> >> hbase-site hbase.hregion.impl >> org.apache.hadoop.hbase.regionserver.transactional.TransactionalRegion >> >> hbase-site hbase.regionserver.region.split.policy >> org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy >> >> hbase-site hbase.snapshot.enabled true >> >> hbase-site hbase.bulkload.staging.dir /hbase-staging >> >> hbase-site hbase.regionserver.region.transactional.tlog true >> >> hbase-site hbase.snapshot.master.timeoutMillis 60 >> >> hbase-site hbase.snapshot.region.timeout 60 >> >> hdfs-site dfs.namenode.acls.enabled true >> >> hbase-site hbase.client.scanner.timeout.period 60 >> >> Thanks all! >> >> On Thu, Feb 4, 2016 at 5:40 AM, Qifan Chen <qifan.c...@esgyn.com> wrote: >> >>> Hi Amanda, >>> >>> I like your idea. >>> >>> On Trafodion's modification of config files for HBase, HDFS, and >>> Zookeeper, I wonder if it is possible to re-difine the port range for an >>> install, without going through all the install steps. This has the >>> practical value that due to port conflict, an install could not start >>> successfully, and one can easily switch to a different port range to retry >>> with this capability. >>> >>> In addition, some of the install steps, such as TPCDS data set >>> generation, and conversion to UTF8, could be made parallel to shorten the >>> install duration. They are done one table at a time in serial mode now. >>> >>> TPCDS table data is generated through the command "dsdgen -scale ..." >>> which could take the option "-parallel". >>> Data conversion to UTF8 is done through the iconv utility. >>> >>> Regards, --Qifan >>> >>> >>> >>> >>> >>> >>> >>> On Wed, Feb 3, 2016 at 10:34 PM, Amanda Moran <amanda.mo...@esgyn.com> >>> wrote: >>> >>>> HI there All- >>>> >>>> The traf_*_mods scripts are used during installation to set HBase, >>>> HDFS, and Zookeeper settings needed by Trafodion. >>>> >>>> The script takes a long time to run (slowing down installation by a >>>> lot) because it needs to restart all these services for the changes to be >>>> picked up. >>>> >>>> My question is: What if I wanted to set the settings in advance of >>>> running the installer. What would be the absolute minimum set of settings >>>> that I need to get installed? I could set other settings later with a >>>> script or by hand. >>>> >>>> Thanks! >>>> -- >>>> Thanks, >>>> >>>> Amanda Moran >>>> >>> >>> >>> >>> -- >>> Regards, --Qifan >>> >>> >> >> >> -- >> Thanks, >> >> Amanda Moran >> > > > > -- > Regards, --Qifan > > -- Thanks, Amanda Moran
Re: traf_*mods in trafodion installation
Thanks for the response Qifan... but I am not sure we are talking about the same thing. I am talking about the traf__mods98 script that is run as apart of the Trafodion Installation scripts. It is used to setup HBase settings, copy hbase-trx*.jar file, and restart HBase, HDFS, and Zookeeper. I would like to know what are the smallest number of settings that need to be set in advance for Trafodion to install/start. Examples of things set: hbase-site hbase.master.distributed.log.splitting false hbase-site hbase.coprocessor.region.classes "org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionObserver,org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionEndpoint,org.apache.hadoop.hbase.coprocessor.AggregateImplementation" hbase-site hbase.hregion.impl org.apache.hadoop.hbase.regionserver.transactional.TransactionalRegion hbase-site hbase.regionserver.region.split.policy org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy hbase-site hbase.snapshot.enabled true hbase-site hbase.bulkload.staging.dir /hbase-staging hbase-site hbase.regionserver.region.transactional.tlog true hbase-site hbase.snapshot.master.timeoutMillis 60 hbase-site hbase.snapshot.region.timeout 60 hdfs-site dfs.namenode.acls.enabled true hbase-site hbase.client.scanner.timeout.period 60 Thanks all! On Thu, Feb 4, 2016 at 5:40 AM, Qifan Chen <qifan.c...@esgyn.com> wrote: > Hi Amanda, > > I like your idea. > > On Trafodion's modification of config files for HBase, HDFS, and > Zookeeper, I wonder if it is possible to re-difine the port range for an > install, without going through all the install steps. This has the > practical value that due to port conflict, an install could not start > successfully, and one can easily switch to a different port range to retry > with this capability. > > In addition, some of the install steps, such as TPCDS data set generation, > and conversion to UTF8, could be made parallel to shorten the install > duration. They are done one table at a time in serial mode now. > > TPCDS table data is generated through the command "dsdgen -scale ..." > which could take the option "-parallel". > Data conversion to UTF8 is done through the iconv utility. > > Regards, --Qifan > > > > > > > > On Wed, Feb 3, 2016 at 10:34 PM, Amanda Moran <amanda.mo...@esgyn.com> > wrote: > >> HI there All- >> >> The traf_*_mods scripts are used during installation to set HBase, HDFS, >> and Zookeeper settings needed by Trafodion. >> >> The script takes a long time to run (slowing down installation by a lot) >> because it needs to restart all these services for the changes to be picked >> up. >> >> My question is: What if I wanted to set the settings in advance of >> running the installer. What would be the absolute minimum set of settings >> that I need to get installed? I could set other settings later with a >> script or by hand. >> >> Thanks! >> -- >> Thanks, >> >> Amanda Moran >> > > > > -- > Regards, --Qifan > > -- Thanks, Amanda Moran
Re: Trafodion Release 2.0 Hadoop/HBase Versions?
The core needs to change based on the CDH or HDP version (hbase-trx*.jar to be specific). Other folks speak up on this one... I am no expert :) On Fri, Feb 5, 2016 at 9:27 AM, Gunnar Tapper <tapper.gun...@gmail.com> wrote: > +1 > > The Trafodion project needs to do everything it can to make it easy for > users to adopt the great software. Is there any special work that needs to > be done beyond maintaining the installer and running tests? > > Thanks, > > Gunnar > > On Fri, Feb 5, 2016 at 9:56 AM, Carol Pearson <carol.pearson...@gmail.com> > wrote: > >> Hi Atanu, >> >> Thanks! I think that we need to make it as easy as possible for our >> users to try Trafodion and to use Trafodion. I don't think most of our >> non-developer users will be using Trafodion on a plain Apache platform, >> especially if they're trying it in a commercial environment. Hortonworks >> and Cloudera have free community editions. >> >> As we're trying to increase our user base and grow our community, it >> seems to me that we want to make it easier for folks to try and to use >> Trafodion, rather than harder. So making sure we have basic functionality >> that works on the most common distros seems to me to be a basic requirement >> for success. >> >> -Carol P. >> >> --- >> Email:carol.pearson...@gmail.com >> Twitter: @CarolP222 >> --- >> >> On Thu, Feb 4, 2016 at 11:25 PM, Atanu Mishra <atanumis...@gmail.com> >> wrote: >> >>> Hi Carol, >>> >>> Great question! Once Trafodion supports Apache Hadoop/HBase, I question >>> whether there is a reason to continue to support the Cloudera and >>> HortonWorks distros. >>> >>> Thanks, >>> Atanu >>> >>> - >>> >>> *From:* Carol Pearson [mailto:carol.pearson...@gmail.com] >>> *Sent:* Thursday, February 4, 2016 4:53 PM >>> *To:* user@trafodion.incubator.apache.org >>> *Subject:* Trafodion Release 2.0 Hadoop/HBase Versions? >>> >>> >>> >>> Hi Trafodion Fans, >>> >>> >>> >>> I'm trying to make sure I'm clear on the targeted runtime environments >>> for our next release, which I think we're calling Trafodion 2.0. I've seen >>> that it needs to support working with Apache Hadoop, since the 1.3 release >>> supported only the Cloudera and Hadoop distros. >>> >>> >>> >>> *Trafodion 2.0* >>> >>> >>> >>> From what's been discussed on the Dev list and here, I think the target >>> is: >>> >>> >>> >>> - Linux Versions CentOS 6.5-6.7 (RHEL 6.5-6.7 - untested? but should >>> support) >>> >>> - SUSE? >>> >>> >>> >>> Hadoop/VersionHBase Version >>> >>> = >>> >>> Cloudera CDH 5.4.x 1.1 >>> >>> Hortonworks HDP 2.31.1 >>> >>> Apache Hadoop 2.7.11.1 >>> >>> >>> >>> >>> >>> Are these the targets that folks are working on Trafodion 2.0 to >>> support? Are they the right ones? (For example, should we be looking at >>> Apache Hadoop 2.6.3?) >>> >>> >>> >>> Are there other versions of software in the Hadoop ecosystem whose >>> versions should be specified as targets, especially for distro-less vanilla >>> Apache Hadoop? >>> >>> >>> >>> I think that once we release, we should identify the specific versions >>> that folks have tested with (which might not be all versions and all >>> distros). I think we should also list versions for software that we >>> believe should work but have not been tested. As I was trying to understand >>> which releases worked with what, I came across a great chart in the the >>> HBase reference book in section 4.1: >>> https://hbase.apache.org/book.html#basic.prerequisites then scroll down >>> to see the Hadoop versions. It would be great to develop that chart for >>> Trafodion as 2.0 goes forward. >>> >>> >>> >>> Thanks for any light people can shed on this! >>> >>> -Carol P. >>> >>> >>> >>> PS: For reference, here's my understanding of what Trafodion 1.3 >>> supported: >>> >>> I've put together the following chart based on the notes from the >>> Trafodion website for release 1.3 and my understanding of Trafodion 2.0: >>> >>> >>> >>> *Trafodion 1.3: (Current release)* >>> >>> >>> >>> - Linux Versions: CentOS 6.5-6.7, (RHEL 6.5-6.7 - untested? but >>> should support) >>> >>> >>> >>> Hadoop/VersionHBase Version >>> >>> = >>> >>> Cloudera CDH 5.3.x 0.98 >>> >>> Hortonworks HDP 2.20.98 >>> >>> >>> >>> >>> >>> >>> --- >>> >>> Email:carol.pearson...@gmail.com >>> >>> Twitter: @CarolP222 >>> >>> --- >>> >>> Atanu Mishra >>> >> >> > > > -- > Thanks, > > Gunnar > *If you think you can you can, if you think you can't you're right.* > -- Thanks, Amanda Moran
traf_*mods in trafodion installation
HI there All- The traf_*_mods scripts are used during installation to set HBase, HDFS, and Zookeeper settings needed by Trafodion. The script takes a long time to run (slowing down installation by a lot) because it needs to restart all these services for the changes to be picked up. My question is: What if I wanted to set the settings in advance of running the installer. What would be the absolute minimum set of settings that I need to get installed? I could set other settings later with a script or by hand. Thanks! -- Thanks, Amanda Moran
Re: Trafodion 1.3 and Hortonworks
+1 to Steve. Thanks. On Wed, Feb 3, 2016 at 9:31 AM, Steve Varnau <steve.var...@esgyn.com> wrote: > The daily test automation has been running on both HDP and CDH distros for > a long time. That is only one aspect of the QA testing, but Trafodion has > been developed, regression tested, and supported on both distros. > > > > --Steve > > > > *From:* Jin, Jian (Seth) [mailto:jian@esgyn.cn] > *Sent:* Tuesday, February 2, 2016 6:52 PM > > *To:* user@trafodion.incubator.apache.org; Ma, Sheng-Chen (Aven) < > shengchen...@esgyn.cn> > *Subject:* RE: Trafodion 1.3 and Hortonworks > > > > Just got Aven comment: we didn’t test on HDP previously. Now it comes > question, do we need to support it? > > If yes, we need to plan to thoroughly test on HDP and make it work. > > > > Br, > > > > Seth > > > > *From:* Jin, Jian (Seth) [mailto:jian@esgyn.cn <jian@esgyn.cn>] > *Sent:* 2016年2月3日 10:37 > *To:* user@trafodion.incubator.apache.org; Ma, Sheng-Chen (Aven) < > shengchen...@esgyn.cn> > *Subject:* RE: Trafodion 1.3 and Hortonworks > > > > Hi all, > > > > Do we support HDP for Trafodion 1.3? My understanding shall be only > vanilla Hbase and CDH Hbase 0.98.10. > > Aven worked for this we may need get his comment. > > > > Br, > > > > Seth > > > > *From:* Amanda Moran [mailto:amanda.mo...@esgyn.com > <amanda.mo...@esgyn.com>] > *Sent:* 2016年2月3日 5:07 > *To:* user@trafodion.incubator.apache.org > *Subject:* Re: Trafodion 1.3 and Hortonworks > > > > HDP 2.2 > > > > On Tue, Feb 2, 2016 at 1:06 PM, Gunnar Tapper <tapper.gun...@gmail.com> > wrote: > > Hi Amanda, > > > > Thanks for the CDH clarification. > > > > What about HDP? > > > > Thanks, > > > > Gunnar > > > > On Tue, Feb 2, 2016 at 2:05 PM, Amanda Moran <amanda.mo...@esgyn.com> > wrote: > > Trafodion support CDH 5.2 and 5.3 both on HBase 0.98. > > > > On Tue, Feb 2, 2016 at 1:04 PM, Gunnar Tapper <tapper.gun...@gmail.com> > wrote: > > Hi, > > > > It's my understanding that Trafodion supports both CDH 5.4.4 and CDH 5.4.5 > and, therefore, HBase 1.0. > > > > What about HDP 2.2? It uses HBase 0.98.4. > > > > *working on the 1.3 Provisioning Guide* > > > > -- > > Thanks, > > > > Gunnar > > *If you think you can you can, if you think you can't you're right.* > > > > > > -- > > Thanks, > > > > Amanda Moran > > > > > > -- > > Thanks, > > > > Gunnar > > *If you think you can you can, if you think you can't you're right.* > > > > > > -- > > Thanks, > > > > Amanda Moran > -- Thanks, Amanda Moran
Re: How to Update the Apache Trafodion Website...
There will no longer be Trafodion installation instructions on the main site? http://trafodion.apache.org/install.html This link is going away? On Tue, Feb 2, 2016 at 10:33 AM, Gunnar Tapper <tapper.gun...@gmail.com> wrote: > Hi, > > Don't bother. It'll be replaced with the upcoming Trafodion Provision > Guide. > > But, in general, all these how-to instructions are now documented in the > Trafodion Contributor Guide: > https://cwiki.apache.org/confluence/display/TRAFODION/Trafodion+Contributor+Guide > > Gunnar > > On Tue, Feb 2, 2016 at 11:30 AM, Amanda Moran <amanda.mo...@esgyn.com> > wrote: > >> Hi there All- >> >> I would like to update the installation page on the Apache Trafodion >> Website. It's too complicated right now, and doesn't flow correctly. >> >> Can we please have the steps to that in bullet points here? A link to the >> page on the website would also be fine. >> >> Thanks. >> >> -- >> Thanks, >> >> Amanda Moran >> > > > > -- > Thanks, > > Gunnar > *If you think you can you can, if you think you can't you're right.* > -- Thanks, Amanda Moran
Re: Trafodion 1.3 and Hortonworks
HDP 2.2 On Tue, Feb 2, 2016 at 1:06 PM, Gunnar Tapper <tapper.gun...@gmail.com> wrote: > Hi Amanda, > > Thanks for the CDH clarification. > > What about HDP? > > Thanks, > > Gunnar > > On Tue, Feb 2, 2016 at 2:05 PM, Amanda Moran <amanda.mo...@esgyn.com> > wrote: > >> Trafodion support CDH 5.2 and 5.3 both on HBase 0.98. >> >> On Tue, Feb 2, 2016 at 1:04 PM, Gunnar Tapper <tapper.gun...@gmail.com> >> wrote: >> >>> Hi, >>> >>> It's my understanding that Trafodion supports both CDH 5.4.4 and CDH >>> 5.4.5 and, therefore, HBase 1.0. >>> >>> What about HDP 2.2? It uses HBase 0.98.4. >>> >>> *working on the 1.3 Provisioning Guide* >>> >>> -- >>> Thanks, >>> >>> Gunnar >>> *If you think you can you can, if you think you can't you're right.* >>> >> >> >> >> -- >> Thanks, >> >> Amanda Moran >> > > > > -- > Thanks, > > Gunnar > *If you think you can you can, if you think you can't you're right.* > -- Thanks, Amanda Moran
Re: Trafodion 1.3 and Hortonworks
Trafodion support CDH 5.2 and 5.3 both on HBase 0.98. On Tue, Feb 2, 2016 at 1:04 PM, Gunnar Tapper <tapper.gun...@gmail.com> wrote: > Hi, > > It's my understanding that Trafodion supports both CDH 5.4.4 and CDH 5.4.5 > and, therefore, HBase 1.0. > > What about HDP 2.2? It uses HBase 0.98.4. > > *working on the 1.3 Provisioning Guide* > > -- > Thanks, > > Gunnar > *If you think you can you can, if you think you can't you're right.* > -- Thanks, Amanda Moran
Re: MRv1 vs. MRv2
I have done many installs without Yarn or MapReduce installed at all. Trafodion runs fine :) On Tue, Feb 2, 2016 at 2:39 PM, Hans Zeller <hans.zel...@esgyn.com> wrote: > No, it is not required in the build environment. > > Hans > > On Tue, Feb 2, 2016 at 1:54 PM, Gunnar Tapper <tapper.gun...@gmail.com> > wrote: > >> Does this mean that MRv1 is now required in the build environment? >> >> On Tue, Feb 2, 2016 at 2:12 PM, Hans Zeller <hans.zel...@esgyn.com> >> wrote: >> >>> Hi, >>> >>> That decision would be made by Hive, not Trafodion. For people who use >>> install_local_hadoop, we recently changed that setup to use local >>> MapReduce, not YARN, see >>> https://issues.apache.org/jira/browse/TRAFODION-1781. >>> >>> Hans >>> >>> On Tue, Feb 2, 2016 at 12:58 PM, Gunnar Tapper <tapper.gun...@gmail.com> >>> wrote: >>> >>>> Hi Suresh: >>>> >>>> Thanks for the information. >>>> >>>> Given from what you write, it seems that YARN with MRv2 is required for >>>> full functionality. >>>> >>>> MRv1 is a separate install in current distributions, which is why I am >>>> asking about it. How does Trafodion decide to run the MapReduce job as MRv1 >>>> vs. MRv2 if both are installed? >>>> >>>> Thanks, >>>> >>>> Gunnar >>>> >>>> >>>> >>>> On Tue, Feb 2, 2016 at 1:50 PM, Suresh Subbiah < >>>> suresh.subbia...@gmail.com> wrote: >>>> >>>>> Hi, >>>>> >>>>> I don't think Trafodion requires YARN for most activity. >>>>> >>>>> For Hive table access, Trafodion uses Hive metadata access Java API >>>>> and libhdfs to actually scan the data file. Therefore YARN is not needed >>>>> for Hive access. >>>>> YARN is not needed for native Hbase or Trafodion table access too. >>>>> YARN is needed for backup/restore, since the HBase exportSnapshot >>>>> class Trafodion calls, used MapReduce to copy large snapshot files to/from >>>>> the backup location. >>>>> YARN is also needed for developer regressions as some vanilla Hive >>>>> commands are executed during the regression run. >>>>> For the last 2 lines I think both MRv1 and MRv2 is supported. >>>>> >>>>> Thanks >>>>> Suresh >>>>> >>>>> >>>>> On Tue, Feb 2, 2016 at 2:36 PM, Gunnar Tapper <tapper.gun...@gmail.com >>>>> > wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> Does Trafodion require YARN with MRv2 or is MRv1 supported, too? >>>>>> >>>>>> -- >>>>>> Thanks, >>>>>> >>>>>> Gunnar >>>>>> *If you think you can you can, if you think you can't you're right.* >>>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> Thanks, >>>> >>>> Gunnar >>>> *If you think you can you can, if you think you can't you're right.* >>>> >>> >>> >> >> >> -- >> Thanks, >> >> Gunnar >> *If you think you can you can, if you think you can't you're right.* >> > > -- Thanks, Amanda Moran
