[ https://issues.apache.org/jira/browse/KAFKA-1207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15839085#comment-15839085 ]
postmas...@inn.ru commented on KAFKA-1207: ------------------------------------------ Delivery is delayed to these recipients or groups: e...@inn.ru<mailto:e...@inn.ru> Subject: [jira] [Commented] (KAFKA-1207) Launch Kafka from within Apache Mesos This message hasn't been delivered yet. Delivery will continue to be attempted. The server will keep trying to deliver this message for the next 1 days, 19 hours and 51 minutes. You'll be notified if the message can't be delivered by that time. Diagnostic information for administrators: Generating server: lc-exch-02.inn.local Receiving server: inn.ru (109.105.153.25) e...@inn.ru Server at inn.ru (109.105.153.25) returned '400 4.4.7 Message delayed' 1/26/2017 2:40:38 AM - Server at inn.ru (109.105.153.25) returned '441 4.4.1 Error communicating with target host: "Failed to connect. Winsock error code: 10060, Win32 error code: 10060." Last endpoint attempted was 109.105.153.25:25' Original message headers: Received: from lc-exch-04.inn.local (10.64.37.99) by lc-exch-02.inn.local (10.64.37.98) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Thu, 26 Jan 2017 01:42:33 +0300 Received: from lc-asp-02.inn.ru (10.64.37.105) by lc-exch-04.inn.local (10.64.37.100) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32 via Frontend Transport; Thu, 26 Jan 2017 01:42:33 +0300 Received-SPF: None (no SPF record) identity=mailfrom; client-ip=209.188.14.142; helo=spamd3-us-west.apache.org; envelope-from=j...@apache.org; receiver=e...@inn.ru X-Envelope-From: <j...@apache.org> Received: from spamd3-us-west.apache.org (pnap-us-west-generic-nat.apache.org [209.188.14.142]) by lc-asp-02.inn.ru (Postfix) with ESMTP id DF310400C3 for <e...@inn.ru>; Wed, 25 Jan 2017 23:42:31 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 58296184907 for <e...@inn.ru>; Wed, 25 Jan 2017 22:42:32 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -1.999 X-Spam-Level: X-Spam-Status: No, score=-1.999 tagged_above=-999 required=6.31 tests=[KAM_LAZY_DOMAIN_SECURITY=1, RP_MATCHES_RCVD=-2.999] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id sgLAgLoTwb-D for <e...@inn.ru>; Wed, 25 Jan 2017 22:42:30 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id DF02C5FB79 for <e...@inn.ru>; Wed, 25 Jan 2017 22:42:29 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id C5B02E0436 for <e...@inn.ru>; Wed, 25 Jan 2017 22:42:27 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 9720225291 for <e...@inn.ru>; Wed, 25 Jan 2017 22:42:26 +0000 (UTC) Date: Wed, 25 Jan 2017 22:42:26 +0000 From: "postmas...@inn.ru (JIRA)" <j...@apache.org> To: <e...@inn.ru> Message-ID: <jira.12689059.1389811935000.205946.1485384146...@atlassian.jira> In-Reply-To: <jira.12689059.1389811935...@atlassian.jira> References: <jira.12689059.1389811935...@atlassian.jira> <jira.12689059.1389811935...@jira-lw-us.apache.org> Subject: [jira] [Commented] (KAFKA-1207) Launch Kafka from within Apache Mesos MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 X-inn-MailScanner-ESVA-Information: Please contact for more information X-inn-MailScanner-ESVA-ID: DF310400C3.A715A X-inn-MailScanner-ESVA: Found to be clean X-inn-MailScanner-ESVA-From: j...@apache.org X-inn-MailScanner-ESVA-Watermark: 1485988952.29701@ZBSmOSv0ll0VSfEqepc3Wg Return-Path: j...@apache.org X-OrganizationHeadersPreserved: lc-exch-02.inn.local X-CrossPremisesHeadersFilteredByDsnGenerator: lc-exch-02.inn.local > Launch Kafka from within Apache Mesos > ------------------------------------- > > Key: KAFKA-1207 > URL: https://issues.apache.org/jira/browse/KAFKA-1207 > Project: Kafka > Issue Type: Bug > Reporter: Joe Stein > Labels: mesos > Attachments: KAFKA-1207_2014-01-19_00:04:58.patch, > KAFKA-1207_2014-01-19_00:48:49.patch, KAFKA-1207.patch > > > There are a few components to this. > 1) The Framework: This is going to be responsible for starting up and > managing the fail over of brokers within the mesos cluster. This will have > to get some Kafka focused paramaters for launching new replica brokers, > moving topics and partitions around based on what is happening in the grid > through time. > 2) The Scheduler: This is what is going to ask for resources for Kafka > brokers (new ones, replacement ones, commissioned ones) and other operations > such as stopping tasks (decommissioning brokers). I think this should also > expose a user interface (or at least a rest api) for producers and consumers > so we can have producers and consumers run inside of the mesos cluster if > folks want (just add the jar) > 3) The Executor : This is the task launcher. It launches tasks kills them > off. > 4) Sharing data between Scheduler and Executor: I looked at the a few > implementations of this. I like parts of the Storm implementation but think > using the environment variable > ExectorInfo.CommandInfo.Enviornment.Variables[] is the best shot. We can > have a command line bin/kafka-mesos-scheduler-start.sh that would build the > contrib project if not already built and support conf/server.properties to > start. > The Framework and operating Scheduler would run in on an administrative node. > I am probably going to hook Apache Curator into it so it can do it's own > failure to a another follower. Running more than 2 should be sufficient as > long as it can bring back it's state (e.g. from zk). I think we can add this > in after once everything is working. > Additional detail can be found on the Wiki page > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=38570672 -- This message was sent by Atlassian JIRA (v6.3.4#6332)