Fwd: Run Eagle on Spark Streaming 报错

[覃兆坤]

-- Forwarded message --
From: 覃兆坤 <qinzhao...@gmail.com>
Date: 2016-05-28 16:39 GMT+08:00
Subject: Re: Run Eagle on Spark Streaming 报错
To: yuming wang <wgy...@gmail.com>


你好：
  非常感谢关注Eagle，感谢您提出的Bug，我会尽快修复，谢谢。。。

2016-05-27 17:21 GMT+08:00 yuming wang <wgy...@gmail.com>:

> hi,
>
> 我在看你的Run Eagle on Spark Streaming
> <https://github.com/apache/incubator-eagle/pull/191>
> ，自带的cassandraQueryLog可以运行，但是如果增加HbaseResourceSensitivityDataJoinExecutor就会报错:
> Exception in thread "main" java.lang.IllegalArgumentException: Cannot
> compile unknown HbaseResourceSensitivityDataJoinExecutor to a DStream
> at
> org.apache.eagle.datastream.sparkstreaming.DStreamFactory$.createInputDStream(DStreamFactory.scala:95)
> at
> org.apache.eagle.datastream.sparkstreaming.SparkStreamingCompiler.buildTopology(SparkStreamingCompiler.scala:51)
> at
> org.apache.eagle.datastream.sparkstreaming.SparkStreamingExecutionEnvironment.execute(SparkStreamingExecutionEnvironment.scala:24)
> at
> org.apache.eagle.datastream.core.StreamContextAdapter$class.submit(ExecutionEnvironment.scala:25)
>
> 这个该怎么处理呢？
>
> 我的代码如下：
>
> object TestSparkStreamingWithAlertDSLUseHBase extends App {
>   val env = 
> ExecutionEnvironments.get[SparkStreamingExecutionEnvironment](args)
>   val streamName = "hbaseAuditLogStream"
>   val streamExecutorId = "hbaseAuditLogExecutor"
>   env.config.set("dataSourceConfig.deserializerClass", 
> classOf[HbaseAuditLogKafkaDeserializer].getCanonicalName)
>   env.fromKafka().parallelism(1).nameAs(streamName).!(Seq(streamName), 
> streamExecutorId)
> *.flatMap(new HbaseResourceSensitivityDataJoinExecutor())*
>   env.execute()
> }
>
> 谢谢
>
>
>

Re: Eagle Docker's root password

[覃兆坤]

Hi yuming Wang:
If your docker run in Linux, the root's password is your Linux root's
password.
If your docker run in boot2Docker (Mac or Window), the default username
and password are docker and tcuser

   Just a little suggestion, if your want to run eagle in docker, the min
configuration of your device is 8G memory and 6 core processor.

2016-04-15 23:02 GMT+08:00 yuming wang :

>
> hi
>
> What is the Eagle Docker's root password?
>
>
> ​
>
> Thanks
>

Re: [Welcome] New Contributors in Apache Eagle

[覃兆坤]

Thanks, it is my pleasure, I will continues to make my contributions to
Apache Eagle

2016-01-14 14:37 GMT+08:00 Krishna, Murali(AWF) :

> Thank you Arun and Edward :)
> Looking forward to contribute more.
>
> - Murali Krishna
>
> -Original Message-
> From: Edward Zhang [mailto:yonzhang2...@apache.org]
> Sent: Thursday, January 14, 2016 12:06 PM
> To: dev@eagle.incubator.apache.org
> Subject: Re: [Welcome] New Contributors in Apache Eagle
>
> Welcome Murali and Zhaokun. I know you have done a lot :-)
>
> thanks
> Edward
>
> On Wed, Jan 13, 2016 at 10:27 PM, arunkarthick m 
> wrote:
>
> > Welcome Murali and Zhaokun Qin. Thanks for your contributions.
> >
> > Looking forward to be working with you.
> >
> > Regards,
> > Arun
> >
> > On Wed, Jan 13, 2016 at 9:29 PM, Michael Wu  wrote:
> >
> > > It's my pleasure, team. Thanks for your words. 
> > >
> > > On Wed, Jan 13, 2016 at 7:04 PM, Hao Chen  wrote:
> > >
> > > > Welcome Zhaokun Qin as well, who has made great contributions for
> > > > eagle docker image.
> > > >
> > > > Regards,
> > > > Hao
> > > >
> > > > On Wed, Jan 13, 2016 at 4:17 PM, Prasad Mujumdar
> > > > 
> > > > wrote:
> > > >
> > > > >   Thanks!
> > > > > Looking forward to work with the community to make better
> > > contributions!
> > > > >
> > > > > thanks
> > > > > Prasad
> > > > >
> > > > >
> > > > > On Tue, Jan 12, 2016 at 3:45 PM, Arun Manoharan <
> > > > arunmanoha...@apache.org>
> > > > > wrote:
> > > > >
> > > > > > Hi Everyone,
> > > > > >
> > > > > > In the last 2 months we have had some increasing participation
> > > > > > and
> > > > active
> > > > > > discussions from the community on various topics.
> > > > > >
> > > > > > I would like to thank Prasad Mujumdar, Ralph Su, Michael Wu,
> > > > > > Daniel
> > > > Zhou
> > > > > > for their contributions to the project.
> > > > > >
> > > > > > Thanks,
> > > > > > Arun
> > > > > >
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > Arun Karthick
> > 201-850-9399
> >
>

Re: [bug]eagle docker in mac

[覃兆坤]

Hi Edward:
Docker in Window ox is running well after I increased processor number
in my virtualbox. Thank you for finding this bug. Storm related port has
been mapped (8744), the hdfs related ports will be mapped soon.

Thanks

2015-12-25 15:15 GMT+08:00 覃兆坤 <qinzhao...@gmail.com>:

> I am very glad to hear eagle docker can be run in Mac OX. I reset VM 's
> memory (about 48G), but it still exists problem (Hive metastore stall
> error), I will continue to follow eagle docker in Windows and map more
> ports to host. Thanks, Edward.
>
> 2015-12-25 13:42 GMT+08:00 Zhang, Edward (GDI Hadoop) <yonzh...@ebay.com>:
>
>> I found the reason for hanging is we should assign enough memory/cpu to vm
>> in mac OX, otherwise hadoop related processes like hbase is not able to be
>> started.
>>
>> we should suggest user assign 6G memory and >4 cpus for VM in mac ox.
>>
>> Also I would suggest we export hdfs console port, storm related port to
>> host machine.
>>
>> Anything else is very smooth!
>>
>> Thanks
>> Edward
>>
>> On 12/23/15, 22:47, "覃兆坤" <qinzhao...@gmail.com> wrote:
>>
>> >In your terminal, use command: "docker exec -it sandbox bash" to go into
>> >container.
>> >
>> >2015-12-24 14:37 GMT+08:00 Zhang, Edward (GDI Hadoop) <yonzh...@ebay.com
>> >:
>> >
>> >> In Ambari, not all components are correctly started up, so I manually
>> >> started region server. But hive, kafka, storm may still are not live.
>> >>
>> >> But looks I can’t attach myself into docker container, so I could not
>> >>know
>> >> why it is failing starting eagle service.
>> >>
>> >> Are you able to attach to ambari container?
>> >>
>> >> On 12/23/15, 22:34, "覃兆坤" <qinzhao...@gmail.com> wrote:
>> >>
>> >> >Hi:
>> >> >I am working to deploy eagle docker in Windows/Mac. I discover a
>> >> >problem in Windows. The Hive Metastore component can not start. Maybe
>> >>the
>> >> >same situation happens in Mac. Now you can access ambari UI, so can
>> you
>> >> >check whether the ambari UI shows all the components were installed
>> >> >successfully(hdfs,hbase,hive,kafka,storm). If the installation
>> >>failed
>> >> >in Hive Metastore start, I think the same problem in Windows and Mac.
>> I
>> >> >will try to follow this.
>> >> >
>> >> >Thanks
>> >> >Zhaokun
>> >> >
>> >> >2015-12-24 14:12 GMT+08:00 Chen, Hao <hao.ch...@ebay.com>:
>> >> >
>> >> >> I think it should because docker runtime under MacOSX/Windows is
>> >>using
>> >> >> delegate architecture but not like native LXC container under Linux
>> >>OS,
>> >> >> which may cause some problem like network/ip, etc.
>> >> >>
>> >> >> Docker Proxy Client under Windows/MacOSX  -> VirtualBox VM[Docker
>> >> >> Instances Inside VM]
>> >> >>
>> >> >> Zhaokun is helping investigate the problem.
>> >> >>
>> >> >> Thanks,
>> >> >> Hao
>> >> >>
>> >> >>
>> >> >> On 12/24/15, 1:20 PM, "Zhang, Edward (GDI Hadoop)"
>> >><yonzh...@ebay.com>
>> >> >> wrote:
>> >> >>
>> >> >> >Hi,
>> >> >> >
>> >> >> >Based on https://hub.docker.com/r/apacheeagle/sandbox/ I can’t run
>> >> >> docker well in mac, there are multiple issues:
>> >> >> >
>> >> >> >1) the 2nd docker run hangs forever in starting eagle service.
>> >> >> >
>> >> >> >2) the 1st docker run can start up ambari, and I can access ambari
>> >>UI,
>> >> >> but I can’t access other ports for example 60010 which is for hbase
>> >> >>master
>> >> >> info port
>> >> >> >
>> >> >> >Thanks
>> >> >> >Edward
>> >> >>
>> >>
>> >>
>>
>>
>

Re: [bug]eagle docker in mac

[覃兆坤]

I am very glad to hear eagle docker can be run in Mac OX. I reset VM 's
memory (about 48G), but it still exists problem (Hive metastore stall
error), I will continue to follow eagle docker in Windows and map more
ports to host. Thanks, Edward.

2015-12-25 13:42 GMT+08:00 Zhang, Edward (GDI Hadoop) <yonzh...@ebay.com>:

> I found the reason for hanging is we should assign enough memory/cpu to vm
> in mac OX, otherwise hadoop related processes like hbase is not able to be
> started.
>
> we should suggest user assign 6G memory and >4 cpus for VM in mac ox.
>
> Also I would suggest we export hdfs console port, storm related port to
> host machine.
>
> Anything else is very smooth!
>
> Thanks
> Edward
>
> On 12/23/15, 22:47, "覃兆坤" <qinzhao...@gmail.com> wrote:
>
> >In your terminal, use command: "docker exec -it sandbox bash" to go into
> >container.
> >
> >2015-12-24 14:37 GMT+08:00 Zhang, Edward (GDI Hadoop) <yonzh...@ebay.com
> >:
> >
> >> In Ambari, not all components are correctly started up, so I manually
> >> started region server. But hive, kafka, storm may still are not live.
> >>
> >> But looks I can’t attach myself into docker container, so I could not
> >>know
> >> why it is failing starting eagle service.
> >>
> >> Are you able to attach to ambari container?
> >>
> >> On 12/23/15, 22:34, "覃兆坤" <qinzhao...@gmail.com> wrote:
> >>
> >> >Hi:
> >> >I am working to deploy eagle docker in Windows/Mac. I discover a
> >> >problem in Windows. The Hive Metastore component can not start. Maybe
> >>the
> >> >same situation happens in Mac. Now you can access ambari UI, so can you
> >> >check whether the ambari UI shows all the components were installed
> >> >successfully(hdfs,hbase,hive,kafka,storm). If the installation
> >>failed
> >> >in Hive Metastore start, I think the same problem in Windows and Mac. I
> >> >will try to follow this.
> >> >
> >> >Thanks
> >> >Zhaokun
> >> >
> >> >2015-12-24 14:12 GMT+08:00 Chen, Hao <hao.ch...@ebay.com>:
> >> >
> >> >> I think it should because docker runtime under MacOSX/Windows is
> >>using
> >> >> delegate architecture but not like native LXC container under Linux
> >>OS,
> >> >> which may cause some problem like network/ip, etc.
> >> >>
> >> >> Docker Proxy Client under Windows/MacOSX  -> VirtualBox VM[Docker
> >> >> Instances Inside VM]
> >> >>
> >> >> Zhaokun is helping investigate the problem.
> >> >>
> >> >> Thanks,
> >> >> Hao
> >> >>
> >> >>
> >> >> On 12/24/15, 1:20 PM, "Zhang, Edward (GDI Hadoop)"
> >><yonzh...@ebay.com>
> >> >> wrote:
> >> >>
> >> >> >Hi,
> >> >> >
> >> >> >Based on https://hub.docker.com/r/apacheeagle/sandbox/ I can’t run
> >> >> docker well in mac, there are multiple issues:
> >> >> >
> >> >> >1) the 2nd docker run hangs forever in starting eagle service.
> >> >> >
> >> >> >2) the 1st docker run can start up ambari, and I can access ambari
> >>UI,
> >> >> but I can’t access other ports for example 60010 which is for hbase
> >> >>master
> >> >> info port
> >> >> >
> >> >> >Thanks
> >> >> >Edward
> >> >>
> >>
> >>
>
>