Re: Can I configure multiple M/Rs and normal processes to one workflow?

2015-02-04 Thread Ted Yu
bq. Can Oozie handle this workflow? I think so. Better confirm on oozie mailing list. Cheers On Wed, Feb 4, 2015 at 2:30 PM, 임정택 wrote: > This cluster is in service for manipulating OLTP (HBase), so I'm finding > simpler solution which may not required to modify cluster. > > Can Oozie handle t

Re: Can I configure multiple M/Rs and normal processes to one workflow?

2015-02-04 Thread 임정택
This cluster is in service for manipulating OLTP (HBase), so I'm finding simpler solution which may not required to modify cluster. Can Oozie handle this workflow? On 2015년 2월 5일 (목) at 오전 5:03 Ted Yu wrote: > Have you considered using Apache Phoenix ? > That way all your data is stored in one p

Re: Can I configure multiple M/Rs and normal processes to one workflow?

2015-02-04 Thread daemeon reiydelle
I see this frequently as long runninng output phase to relational db's. So your experience is reasonable. Sometimes it is possible to partition the mysequel table, but if you need agreggates over the whole, you are sort of stuck. (Good luck, may your business case never require you to run a singl

Re: Can I configure multiple M/Rs and normal processes to one workflow?

2015-02-04 Thread 임정택
Yes, it takes more than 10 hours in per CP. And we don't have enough resource to run all regions concurrently, it needs about one day to complete. On 2015년 2월 5일 (목) at 오전 4:51 daemeon reiydelle wrote: > Null map step (at a guess?), 3 step reduce. No problem. Suspect 3 may be > rather long runnin

Re: Can I configure multiple M/Rs and normal processes to one workflow?

2015-02-04 Thread Ted Yu
Have you considered using Apache Phoenix ? That way all your data is stored in one place. See http://phoenix.apache.org/ Cheers On Tue, Feb 3, 2015 at 6:44 PM, 임정택 wrote: > Hello all. > > We're periodically scan HBase tables to aggregate statistic information, > and store it to MySQL. > > We h

Re: Can I configure multiple M/Rs and normal processes to one workflow?

2015-02-04 Thread daemeon reiydelle
Null map step (at a guess?), 3 step reduce. No problem. Suspect 3 may be rather long running? *...* *“Life should not be a journey to the grave with the intention of arriving safely in apretty and well preserved body, but rather to skid in broadside in a cloud of smoke,thoroughly used

Can I configure multiple M/Rs and normal processes to one workflow?

2015-02-03 Thread 임정택
Hello all. We're periodically scan HBase tables to aggregate statistic information, and store it to MySQL. We have 3 kinds of CP (kind of data source), each has one Channel and one Article table. (Channel : Article is 1:N relation.) All CPs table schema are different a bit, so in order to aggreg