Re: Please add me to mesos contributors

[Vinod Kone]

Looks like you are set?

On Mon, May 25, 2015 at 5:07 PM, Shuai Lin  wrote:

> ping, anyone can help?
>
> On Sun, May 24, 2015 at 10:19 AM, Shuai Lin 
> wrote:
>
> > Hi all,
> >
> > I would like to contribute to the following mesos issues:
> >
> > MESOS-1806
> > MESOS-2724
> >
> > Could someone add me to "contributors" group? My jira username is lins05.
> > Thanks!
> >
> > Best Regards,
> > Shuai
> >
> >
>

[GitHub] mesos pull request: Powered by - added Allegro Group.

[davelester]

Github user davelester commented on the pull request:

https://github.com/apache/mesos/pull/40#issuecomment-105639071
  
Thanks for this contribution, I just added Allegro to the Powered By Mesos 
list by applying this pull request: https://github.com/apache/mesos/pull/43

Is it safe to assume that Allegro and the Allegro Group are the same 
company? Or is this just a coincidence?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Please add me to mesos contributors

[Shuai Lin]

ping, anyone can help?

On Sun, May 24, 2015 at 10:19 AM, Shuai Lin  wrote:

> Hi all,
>
> I would like to contribute to the following mesos issues:
>
> MESOS-1806
> MESOS-2724
>
> Could someone add me to "contributors" group? My jira username is lins05.
> Thanks!
>
> Best Regards,
> Shuai
>
>

[GitHub] mesos pull request:

[intoeleven]

Github user intoeleven commented on the pull request:


https://github.com/apache/mesos/commit/9d54ad4ee039af12dec05c4ef390645fc38479e7#commitcomment-11352719
  
Hi, I tried to install the latest mesos(0.22.1) on AWS by the steps from 
mesos documentation 
http://mesos.apache.org/documentation/latest/ec2-scripts/

 // I can launch spark cluster on AWS in eu-west-1a with same key and key-id
When I setted up my security key and key id ready to launch mesos cluster, 
it is constantly failed at "Invalid availability zone" error. 

// the reason I tried launch cluster on eu-west-1 because my key is 
generated at eu-west-1
I also have tried eu-west-1, eu-west-1b and eu-west-1c, neither of ways can 
succeeded. 


I have printed out the opts below the Launch slaves(the script is stopped 
at here), is it normal?
opts:
{'ami': 'ami-4517dc2c', 'ft': 1, 'branch': 'master', 'zone': 'eu-west-1a', 
'resume': False, 'ebs_vol_size': 0, 'master_instance_type': '', 
'identity_file': 'awskey.pem', 'proxy_port': None, 'instance_type': 'm1.large', 
'key_pair': 'awskey', 'swap': 1024, 'slaves': 1, 'download': 'none', 
'spot_price': None, 'os': 'amazon64', 'wait': 60}


Do you have idea how to fix it?  Any helps would be appreciated!
Best regards,


 ./mesos-ec2 -k awskey -i awskey.pem --zone=eu-west-1a launch test01-mesos
Setting up security groups...
Checking for running cluster...
Launching instances...
ERROR:boto:400 Bad Request
ERROR:boto:
InvalidParameterValueInvalid 
availability zone: 
[eu-west-1a]273c2517-bb07-4b79-a481-bbe7d898341a
Traceback (most recent call last):
  File "./mesos_ec2.py", line 571, in 
main()
  File "./mesos_ec2.py", line 478, in main
conn, opts, cluster_name)
  File "./mesos_ec2.py", line 261, in launch_cluster
block_device_map = block_map)
  File "/usr/local/lib/python2.7/dist-packages/boto/ec2/image.py", line 
329, in run
tenancy=tenancy, dry_run=dry_run)
  File "/usr/local/lib/python2.7/dist-packages/boto/ec2/connection.py", 
line 973, in run_instances
verb='POST')
  File "/usr/local/lib/python2.7/dist-packages/boto/connection.py", line 
1208, in get_object
raise self.ResponseError(response.status, response.reason, body)
boto.exception.EC2ResponseError: EC2ResponseError: 400 Bad Request

InvalidParameterValueInvalid 
availability zone: 
[eu-west-1a]273c2517-bb07-4b79-a481-bbe7d898341a



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] mesos pull request:

[intoeleven]

Github user intoeleven commented on the pull request:


https://github.com/apache/mesos/commit/fe09d7de7f84de82f447d4321fe89912c06e402c#commitcomment-11352979
  
I can launch the cluster by default "us-east-1b" , but I changed default as 
"eu-west-1a" in mesos_ec2.py, it still can't launch and the same problem as the 
above.

Do you have any good idea to fix it?

Regards



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] mesos pull request:

[intoeleven]

Github user intoeleven commented on the pull request:


https://github.com/apache/mesos/commit/fe09d7de7f84de82f447d4321fe89912c06e402c#commitcomment-11352747
  
Hi, I tried to install the latest mesos(0.22.1) on AWS by the steps from 
mesos documentation 
http://mesos.apache.org/documentation/latest/ec2-scripts/

// I can launch spark cluster on AWS in eu-west-1a with same key and key-id
When I setted up my security key and key id ready to launch mesos cluster, 
it is constantly failed at "Invalid availability zone" error.

// the reason I tried launch cluster on eu-west-1 because my key is 
generated on eu-west-1
I also have tried eu-west-1, eu-west-1b and eu-west-1c, neither of ways can 
succeeded.

I have printed out the opts below the Launch slaves(the launch script is 
stopped at here), is it normal?
opts:
{'ami': 'ami-4517dc2c', 'ft': 1, 'branch': 'master', 'zone': 'eu-west-1a', 
'resume': False, 'ebs_vol_size': 0, 'master_instance_type': '', 
'identity_file': 'awskey.pem', 'proxy_port': None, 'instance_type': 'm1.large', 
'key_pair': 'awskey', 'swap': 1024, 'slaves': 1, 'download': 'none', 
'spot_price': None, 'os': 'amazon64', 'wait': 60}

Do you have idea how to fix it? Any helps would be appreciated!
Best regards,

./mesos-ec2 -k awskey -i awskey.pem --zone=eu-west-1a launch test01-mesos
Setting up security groups...
Checking for running cluster...
Launching instances...
ERROR:boto:400 Bad Request
ERROR:boto:
InvalidParameterValueInvalid availability zone: 
[eu-west-1a]273c2517-bb07-4b79-a481-bbe7d898341a
Traceback (most recent call last):
File "./mesos_ec2.py", line 571, in 
main()
File "./mesos_ec2.py", line 478, in main
conn, opts, cluster_name)
File "./mesos_ec2.py", line 261, in launch_cluster
block_device_map = block_map)
File "/usr/local/lib/python2.7/dist-packages/boto/ec2/image.py", line 329, 
in run
tenancy=tenancy, dry_run=dry_run)
File "/usr/local/lib/python2.7/dist-packages/boto/ec2/connection.py", line 
973, in run_instances
verb='POST')
File "/usr/local/lib/python2.7/dist-packages/boto/connection.py", line 
1208, in get_object
raise self.ResponseError(response.status, response.reason, body)
boto.exception.EC2ResponseError: EC2ResponseError: 400 Bad Request

InvalidParameterValueInvalid availability zone: 
[eu-west-1a]273c2517-bb07-4b79-a481-bbe7d898341a


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] mesos pull request: Add Allegro to the list of adopters

[davelester]

Github user davelester commented on the pull request:

https://github.com/apache/mesos/pull/43#issuecomment-105638062
  
Thanks for this contribution, I just pushed this to master. Could you 
please close this PR?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Please add me to jira group

[Timothy Chen]

Hi Shuai,

Sorry about that, just added you now.

Thanks,

Tim

On Tue, May 26, 2015 at 6:32 PM, Shuai Lin  wrote:
> Hi list,
>
> I asked for joining the mesos "contributors" jira group last week, but
> seems no one replied so far:
>
> https://www.mail-archive.com/dev%40mesos.apache.org/msg32459.html
>
> So could anyone please do that at your convenience? My jira username is
> lins05.
>
> Best Regards,
> Shuai Lin

Please add me to jira group

[Shuai Lin]

Hi list,

I asked for joining the mesos "contributors" jira group last week, but
seems no one replied so far:

https://www.mail-archive.com/dev%40mesos.apache.org/msg32459.html

So could anyone please do that at your convenience? My jira username is
lins05.

Best Regards,
Shuai Lin

Re: Custom Mesos Allocation module

[Alex Rukletsov]

Dariia,

what Mesos version are you using? The ability to write and and use custom
allocator modules without hacking on Mesos source code has just landed in
Mesos (means will be available in 0.23 release) with docs being updated
right now (here  is the RR request if
you really want to be on the bleeding edge).

However, if you're on an older version of Mesos, you can still write your
own module and replace / adjust the line you refer to in master/main.cpp to
support your custom allocator.

On Mon, May 25, 2015 at 1:18 PM, Dariia Zhyrova 
wrote:

> Hello,
>
> I need to write custom Allocation module, but I don't see a possibility to
> load it dynamically using --modules command line argument.
> This is the allocation module creation in master/main:
>
> allocator::Allocator* allocator = new
> allocator::HierarchicalDRFAllocator();
>
> (so it is just hardcoded)
> Other modules are loaded dynamically like this:
>
> if (flags.modules.isSome()) {
> Try result = ModuleManager::load(flags.modules.get());
> if (result.isError()) {
>   EXIT(1) << "Error loading modules: " << result.error();
> }
>   }
> ...
> // Create anonymous modules.
>   foreach (const string& name, ModuleManager::find()) {
> Try create = ModuleManager::create(name);
> if (create.isError()) {
>   EXIT(1) << "Failed to create anonymous module named '" << name <<
> "'";
> }
> ...
> Authenticator* authenticator;
>   // TODO(tillt): Allow multiple authenticators to be loaded and enable
>   // the authenticatee to select the appropriate one. See MESOS-1939.
>   if (authenticatorNames[0] == DEFAULT_AUTHENTICATOR) {
> LOG(INFO) << "Using default CRAM-MD5 authenticator";
> authenticator = new cram_md5::CRAMMD5Authenticator();
>   } else {
> Try module =
>   modules::ModuleManager::create(authenticatorNames[0]);
> if (module.isError()) {
>   EXIT(1) << "Could not create authenticator module '"
>   << authenticatorNames[0] << "': " << module.error();
> }
> ...
> if (ModuleManager::contains(type)) {
>   Try isolator = ModuleManager::create(type);
>   if (isolator.isError()) {
> return Error(
> "Could not create isolator " + type + ": " + isolator.error());
>   }
>
> This is a list of supported modules which can be loaded dynamically:
>  kindToVersion["Anonymous"] = MESOS_VERSION;
>   kindToVersion["Authenticatee"] = MESOS_VERSION;
>   kindToVersion["Authenticator"] = MESOS_VERSION;
>   kindToVersion["Hook"] = MESOS_VERSION;
>   kindToVersion["Isolator"] = MESOS_VERSION;
>   kindToVersion["TestModule"] = MESOS_VERSION;
>
> If allocation module is absent here, it will not pass such check:
>
> Try ModuleManager::verifyModule(
> const string& moduleName,
> const ModuleBase* moduleBase)
> {
> ...
> if (!kindToVersion.contains(moduleBase->kind)) {
> return Error("Unknown module kind: " + stringify(moduleBase->kind));
>   }
> ...
> }
>
> Mesos documentation also does not mention Allocation module as dynamically
> loadable:  http://mesos.apache.org/documentation/latest/modules/
>
> Could you please help me, is there a possibility to use custom allocation
> module with Mesos?
>
> Best regards,
> Dariia
>

Re: Custom Mesos Allocation module

[Niklas Nielsen]

Hi Dariia,

Allocator modules was first added to the master May 14th (git sha
614efe18cf3c7b02a2f3c8269c8eb76cbbe4ddd4), which version of the source are
you looking into?

You should be able to configure your allocator module as usual (
https://github.com/apache/mesos/blob/master/docs/modules.md) and select it
through the new --allocator flag.

Let us know if it still doesn't work/make sense.

Cheers,
Niklas

On 25 May 2015 at 04:18, Dariia Zhyrova  wrote:

> Hello,
>
> I need to write custom Allocation module, but I don't see a possibility to
> load it dynamically using --modules command line argument.
> This is the allocation module creation in master/main:
>
> allocator::Allocator* allocator = new
> allocator::HierarchicalDRFAllocator();
>
> (so it is just hardcoded)
> Other modules are loaded dynamically like this:
>
> if (flags.modules.isSome()) {
> Try result = ModuleManager::load(flags.modules.get());
> if (result.isError()) {
>   EXIT(1) << "Error loading modules: " << result.error();
> }
>   }
> ...
> // Create anonymous modules.
>   foreach (const string& name, ModuleManager::find()) {
> Try create = ModuleManager::create(name);
> if (create.isError()) {
>   EXIT(1) << "Failed to create anonymous module named '" << name <<
> "'";
> }
> ...
> Authenticator* authenticator;
>   // TODO(tillt): Allow multiple authenticators to be loaded and enable
>   // the authenticatee to select the appropriate one. See MESOS-1939.
>   if (authenticatorNames[0] == DEFAULT_AUTHENTICATOR) {
> LOG(INFO) << "Using default CRAM-MD5 authenticator";
> authenticator = new cram_md5::CRAMMD5Authenticator();
>   } else {
> Try module =
>   modules::ModuleManager::create(authenticatorNames[0]);
> if (module.isError()) {
>   EXIT(1) << "Could not create authenticator module '"
>   << authenticatorNames[0] << "': " << module.error();
> }
> ...
> if (ModuleManager::contains(type)) {
>   Try isolator = ModuleManager::create(type);
>   if (isolator.isError()) {
> return Error(
> "Could not create isolator " + type + ": " + isolator.error());
>   }
>
> This is a list of supported modules which can be loaded dynamically:
>  kindToVersion["Anonymous"] = MESOS_VERSION;
>   kindToVersion["Authenticatee"] = MESOS_VERSION;
>   kindToVersion["Authenticator"] = MESOS_VERSION;
>   kindToVersion["Hook"] = MESOS_VERSION;
>   kindToVersion["Isolator"] = MESOS_VERSION;
>   kindToVersion["TestModule"] = MESOS_VERSION;
>
> If allocation module is absent here, it will not pass such check:
>
> Try ModuleManager::verifyModule(
> const string& moduleName,
> const ModuleBase* moduleBase)
> {
> ...
> if (!kindToVersion.contains(moduleBase->kind)) {
> return Error("Unknown module kind: " + stringify(moduleBase->kind));
>   }
> ...
> }
>
> Mesos documentation also does not mention Allocation module as dynamically
> loadable:  http://mesos.apache.org/documentation/latest/modules/
>
> Could you please help me, is there a possibility to use custom allocation
> module with Mesos?
>
> Best regards,
> Dariia
>

Re: Not able to connect to mesos from different machine

[Alex Rukletsov]

Alberto,

What may be happening in your case is that Master is not able to talk to
your scheduler. When responding to a scheduler, Mesos Master doesn't use
the IP from which a request came from, but rather an IP set in the
"Libprocess-from" field instead. That's exactly what you specify in
LIBPROCESS_IP env var prior starting your scheduler. Could you please
double check the it set up correctly and that IP is reachable for Mesos
Master?

In case you are not able to solve the problem, please provide scheduler and
Master logs together with master, zookeeper, and scheduler configurations.


On Mon, May 25, 2015 at 6:30 PM, Alberto Rodriguez 
wrote:

> Hi all,
>
> I managed to get a mesos cluster up & running on a Ubuntu VM. I've
> been also able to run and connect a spark-shell from this machine and
> it works properly.
>
> Unfortunately, I'm trying to connect from the host machine where the
> VM is running to launch spark jobs and I can not.
>
> See below the spark console output:
>
> Using Scala version 2.10.4 (Java HotSpot(TM) 64-Bit Server VM, Java
> 1.7.0_75)
> Type in expressions to have them evaluated.
> Type :help for more information.
> 15/05/25 18:13:00 INFO SecurityManager: Changing view acls to: arodriguez
> 15/05/25 18:13:00 INFO SecurityManager: Changing modify acls to: arodriguez
> 15/05/25 18:13:00 INFO SecurityManager: SecurityManager:
> authentication disabled; ui acls disabled; users with view
> permissions: Set(arodriguez); users with modify permissions:
> Set(arodriguez)
> 15/05/25 18:13:01 INFO Slf4jLogger: Slf4jLogger started
> 15/05/25 18:13:01 INFO Remoting: Starting remoting
> 15/05/25 18:13:01 INFO Remoting: Remoting started; listening on
> addresses :[akka.tcp://sparkDriver@localhost.localdomain:47229]
> 15/05/25 18:13:01 INFO Utils: Successfully started service
> 'sparkDriver' on port 47229.
> 15/05/25 18:13:01 INFO SparkEnv: Registering MapOutputTracker
> 15/05/25 18:13:01 INFO SparkEnv: Registering BlockManagerMaster
> 15/05/25 18:13:01 INFO DiskBlockManager: Created local directory at
> /tmp/spark-local-20150525181301-7fa8
> 15/05/25 18:13:01 INFO MemoryStore: MemoryStore started with capacity
> 265.4 MB
> 15/05/25 18:13:01 WARN NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where
> applicable
> 15/05/25 18:13:01 INFO HttpFileServer: HTTP File server directory is
> /tmp/spark-1249c23f-adc8-4fcd-a044-b65a80f40e16
> 15/05/25 18:13:01 INFO HttpServer: Starting HTTP Server
> 15/05/25 18:13:01 INFO Utils: Successfully started service 'HTTP file
> server' on port 51659.
> 15/05/25 18:13:01 INFO Utils: Successfully started service 'SparkUI'
> on port 4040.
> 15/05/25 18:13:01 INFO SparkUI: Started SparkUI at
> http://localhost.localdomain:4040
> WARNING: Logging before InitGoogleLogging() is written to STDERR
> W0525 18:13:01.749449 10908 sched.cpp:1323]
> **
> Scheduler driver bound to loopback interface! Cannot communicate with
> remote master(s). You might want to set 'LIBPROCESS_IP' environment
> variable to use a routable IP address.
> **
> 2015-05-25 18:13:01,749:10746(0x7fd4b1ffb700):ZOO_INFO@log_env@712:
> Client environment:zookeeper.version=zookeeper C client 3.4.6
> 2015-05-25 18:13:01,749:10746(0x7fd4b1ffb700):ZOO_INFO@log_env@716:
> Client environment:host.name=localhost.localdomain
> 2015-05-25 18:13:01,749:10746(0x7fd4b1ffb700):ZOO_INFO@log_env@723:
> Client environment:os.name=Linux
> 2015-05-25 18:13:01,749:10746(0x7fd4b1ffb700):ZOO_INFO@log_env@724:
> Client environment:os.arch=3.19.7-200.fc21.x86_64
> 2015-05-25 18:13:01,749:10746(0x7fd4b1ffb700):ZOO_INFO@log_env@725:
> Client environment:os.version=#1 SMP Thu May 7 22:00:21 UTC 2015
> 2015-05-25 18:13:01,749:10746(0x7fd4b1ffb700):ZOO_INFO@log_env@733:
> Client environment:user.name=arodriguez
> I0525 18:13:01.749791 10908 sched.cpp:157] Version: 0.22.1
> 2015-05-25 18:13:01,749:10746(0x7fd4b1ffb700):ZOO_INFO@log_env@741:
> Client environment:user.home=/home/arodriguez
> 2015-05-25 18:13:01,749:10746(0x7fd4b1ffb700):ZOO_INFO@log_env@753:
> Client
> environment:user.dir=/home/arodriguez/dev/spark-1.2.0-bin-hadoop2.4/bin
> 2015-05-25 18:13:01,749:10746(0x7fd4b1ffb700):ZOO_INFO@zookeeper_init@786:
> Initiating client connection, host=10.141.141.10:2181
> sessionTimeout=1 watcher=0x7fd4c2f0d5b0 sessionId=0
> sessionPasswd= context=0x7fd3d40063c0 flags=0
> 2015-05-25 18:13:01,750:10746(0x7fd4ab7fe700):ZOO_INFO@check_events@1705:
> initiated connection to server [10.141.141.10:2181]
> 2015-05-25 18:13:01,752:10746(0x7fd4ab7fe700):ZOO_INFO@check_events@1752:
> session establishment complete on server [10.141.141.10:2181],
> sessionId=0x14d8babef360022, negotiated timeout=1
> I0525 18:13:01.752760 10913 group.cpp:313] Group process
> (group(1)@127.0.0.1:48557) connected to ZooKeeper
> I0525 18:13:01.752787 10913 group.cpp:790] Syncing group operations:
> queue size

Re: Docker containers not removed

[Olivier Sallou]

On 05/26/2015 03:44 PM, Olivier Sallou wrote:
> Hi,
> I could make a test scrip to submit tasks in Docker containers.
>
> My tasks ends in FINISHED state, and everything goes fine.
>
> The point is the container is not removed (can be seen with a docker ps
> -a), though documentation states:
>
> " 6. On container exit or containerizer destroy, stop and remove the
> docker container."
>
> even after a few minutes, they are still present.
>
> am i missing something?
I just found somewhere in config:

--docker_remove_delay=VALUE


default being 6 hrs. I will wait and check !
>
> Thanks
>
> Olivier
>
>

-- 
Olivier Sallou
IRISA / University of Rennes 1
Campus de Beaulieu, 35000 RENNES - FRANCE
Tel: 02.99.84.71.95

gpg key id: 4096R/326D8438  (keyring.debian.org)
Key fingerprint = 5FB4 6F83 D3B9 5204 6335  D26D 78DC 68DB 326D 8438

Docker containers not removed

[Olivier Sallou]

Hi,
I could make a test scrip to submit tasks in Docker containers.

My tasks ends in FINISHED state, and everything goes fine.

The point is the container is not removed (can be seen with a docker ps
-a), though documentation states:

" 6. On container exit or containerizer destroy, stop and remove the
docker container."

even after a few minutes, they are still present.

am i missing something?

Thanks

Olivier


-- 


gpg key id: 4096R/326D8438  (keyring.debian.org)
Key fingerprint = 5FB4 6F83 D3B9 5204 6335  D26D 78DC 68DB 326D 8438

Re: Please add me to jira

[baotiao]

Thank you guy.


陈宗志
 
Blog:baotiao.github.io
 
 
 

> On May 26, 2015, at 17:54, Till Toenshoff  wrote:
> 
> I have added you to the list of Mesos contributors - you should now be able 
> to assign the ticket in question to yourself.
> 
>> On May 25, 2015, at 5:54 AM, baotiao  wrote:
>> 
>> Hi guys
>> 
>> I would like to solve the issue 2588, I have commented in the issue, so 
>> could you add me to jira
>> my jira username is  baotiao
>> 
>> Thank you
>> 
>> 
>> 
>> 陈宗志
>> 
>> Blog:baotiao.github.io
>> 
>> 
>> 
>> 
> 

Re: Please add me to jira

[Till Toenshoff]

I have added you to the list of Mesos contributors - you should now be able to 
assign the ticket in question to yourself.

> On May 25, 2015, at 5:54 AM, baotiao  wrote:
> 
> Hi guys
> 
> I would like to solve the issue 2588, I have commented in the issue, so could 
> you add me to jira
> my jira username is  baotiao
> 
> Thank you
> 
> 
> 
> 陈宗志
> 
> Blog:baotiao.github.io
> 
> 
> 
>