Implement a Client/Server RPC communication

Hi guys, I am trying to do some changes in the YARN code. Through other things, I am trying to instantiate a Server and a Client which communicate via RPC. To do that, I did the following: 1. I created a proto file where I describe the messages and the service 2. I create a server class with im

Non blocking RPC service

Hi all, This might be a bit out of topic, but I don't know where to ask. I am trying to create a non blocking RPC service in Yarn and Tez. However, I observed that all the communication is blocking. I was trying to implement a non blocking service inheriting from .Interface instead of .Blocki

Re: Non blocking RPC service

yet though.  Hopefully for a lot more. Planning on using this in Tez as well at some point, for task to AM communication. Otherwise there's alternates like grpc, thrift. HTH On Thu, Mar 2, 2017 at 1:33 PM, Robert Grandl wrote: > Hi all, > This might be a bit out of topic, but I don'

YARN container resize

Hi guys, I am wondering, what is the status of container resizing support as of now in Yarn? I saw a bunch of JIRAs such as: https://issues.apache.org/jira/browse/YARN-1197 https://issues.apache.org/jira/browse/YARN-3866 https://issues.apache.org/jira/browse/YARN-1448 but it seems this effort is

Run Distributed TensorFlow on YARN

Hi all, I am wondering if there is any stable support to run distributed TensorFlow atop YARN at the moment.  I found this blog post from Hortonworks. It seems this it is possible starting YARN 3.1.0.https://hortonworks.com/blog/distributed-tensorflow-assembly-hadoop-yarn/ Also I found some m

Re: Run Distributed TensorFlow on YARN

(TensorFlow on YARN) which is a platform built for this purpose. Jonathan From: Sunil G Sent: Tuesday, November 6, 2018 10:05:14 PM To: Robert Grandl Cc: yarn-dev@hadoop.apache.org; yarn-dev-h...@hadoop.apache.org; General Subject: Re: Run Distributed TensorFlow on YAR

[jira] [Created] (YARN-3819) Collect network usage on the node

Robert Grandl created YARN-3819: --- Summary: Collect network usage on the node Key: YARN-3819 URL: https://issues.apache.org/jira/browse/YARN-3819 Project: Hadoop YARN Issue Type: New Feature

[jira] [Created] (YARN-3820) Collect disks usages on the node

Robert Grandl created YARN-3820: --- Summary: Collect disks usages on the node Key: YARN-3820 URL: https://issues.apache.org/jira/browse/YARN-3820 Project: Hadoop YARN Issue Type: New Feature

[jira] [Created] (YARN-4093) Encapsulate additional information through AM to RM heartbeat

Robert Grandl created YARN-4093: --- Summary: Encapsulate additional information through AM to RM heartbeat Key: YARN-4093 URL: https://issues.apache.org/jira/browse/YARN-4093 Project: Hadoop YARN

[jira] [Created] (YARN-2965) Enhance Node Managers to monitor and report the resource usage on the machines

Robert Grandl created YARN-2965: --- Summary: Enhance Node Managers to monitor and report the resource usage on the machines Key: YARN-2965 URL: https://issues.apache.org/jira/browse/YARN-2965 Project

[jira] [Created] (YARN-2966) Extend ask request to include additional fields

Robert Grandl created YARN-2966: --- Summary: Extend ask request to include additional fields Key: YARN-2966 URL: https://issues.apache.org/jira/browse/YARN-2966 Project: Hadoop YARN Issue Type

[jira] [Created] (YARN-2967) New task matching logic at the RM

Robert Grandl created YARN-2967: --- Summary: New task matching logic at the RM Key: YARN-2967 URL: https://issues.apache.org/jira/browse/YARN-2967 Project: Hadoop YARN Issue Type: Sub-task

[jira] [Created] (YARN-1443) blacklist is not propagated from AM to RM

Robert Grandl created YARN-1443: --- Summary: blacklist is not propagated from AM to RM Key: YARN-1443 URL: https://issues.apache.org/jira/browse/YARN-1443 Project: Hadoop YARN Issue Type: Bug

[jira] [Created] (YARN-1444) RM crashes when force reducer to be node local

Robert Grandl created YARN-1444: --- Summary: RM crashes when force reducer to be node local Key: YARN-1444 URL: https://issues.apache.org/jira/browse/YARN-1444 Project: Hadoop YARN Issue Type

[jira] [Created] (YARN-2745) YARN new pluggable scheduler which does multi-resource packing

Robert Grandl created YARN-2745: --- Summary: YARN new pluggable scheduler which does multi-resource packing Key: YARN-2745 URL: https://issues.apache.org/jira/browse/YARN-2745 Project: Hadoop YARN