Re: Can I configure multiple M/Rs and normal processes to one workflow?

2015-02-04 Thread Ted Yu
Have you considered using Apache Phoenix ? That way all your data is stored in one place. See http://phoenix.apache.org/ Cheers On Tue, Feb 3, 2015 at 6:44 PM, 임정택 kabh...@gmail.com wrote: Hello all. We're periodically scan HBase tables to aggregate statistic information, and store it to

Re: Can I configure multiple M/Rs and normal processes to one workflow?

2015-02-04 Thread daemeon reiydelle
Null map step (at a guess?), 3 step reduce. No problem. Suspect 3 may be rather long running? *...* *“Life should not be a journey to the grave with the intention of arriving safely in apretty and well preserved body, but rather to skid in broadside in a cloud of smoke,thoroughly used

Re: Can I configure multiple M/Rs and normal processes to one workflow?

2015-02-04 Thread 임정택
Yes, it takes more than 10 hours in per CP. And we don't have enough resource to run all regions concurrently, it needs about one day to complete. On 2015년 2월 5일 (목) at 오전 4:51 daemeon reiydelle daeme...@gmail.com wrote: Null map step (at a guess?), 3 step reduce. No problem. Suspect 3 may be

Re: Can I configure multiple M/Rs and normal processes to one workflow?

2015-02-04 Thread daemeon reiydelle
I see this frequently as long runninng output phase to relational db's. So your experience is reasonable. Sometimes it is possible to partition the mysequel table, but if you need agreggates over the whole, you are sort of stuck. (Good luck, may your business case never require you to run a

Re: Can I configure multiple M/Rs and normal processes to one workflow?

2015-02-04 Thread Ted Yu
bq. Can Oozie handle this workflow? I think so. Better confirm on oozie mailing list. Cheers On Wed, Feb 4, 2015 at 2:30 PM, 임정택 kabh...@gmail.com wrote: This cluster is in service for manipulating OLTP (HBase), so I'm finding simpler solution which may not required to modify cluster. Can

Re: Can I configure multiple M/Rs and normal processes to one workflow?

2015-02-04 Thread 임정택
This cluster is in service for manipulating OLTP (HBase), so I'm finding simpler solution which may not required to modify cluster. Can Oozie handle this workflow? On 2015년 2월 5일 (목) at 오전 5:03 Ted Yu yuzhih...@gmail.com wrote: Have you considered using Apache Phoenix ? That way all your data

Can I configure multiple M/Rs and normal processes to one workflow?

2015-02-03 Thread 임정택
Hello all. We're periodically scan HBase tables to aggregate statistic information, and store it to MySQL. We have 3 kinds of CP (kind of data source), each has one Channel and one Article table. (Channel : Article is 1:N relation.) All CPs table schema are different a bit, so in order to