surahman commented on pull request #3741:
URL: https://github.com/apache/incubator-heron/pull/3741#issuecomment-986259908
Please double check your commands and script. I see a trailing `"` in the
command below.
> ```
> heron submit heron /opt/src.pex - $2 --verbose \
> --config-property heron.kubernetes.manager.limits.cpu=300m \
> --config-property heron.kubernetes.manager.limits.memory=300Mi \
> --config-property heron.kubernetes.manager.requests.cpu=20m \
> --config-property heron.kubernetes.manager.requests.memory=100Mi"
> ```
Here is my run on `acking` with your settings and everything is inorder:
```bash
~/bin/heron submit kubernetes ~/.heron/examples/heron-api-examples.jar \
org.apache.heron.examples.api.AckingTopology acking \
--verbose \
--config-property heron.kubernetes.manager.limits.cpu=300m \
--config-property heron.kubernetes.manager.limits.memory=300Mi \
--config-property heron.kubernetes.manager.requests.cpu=20m \
--config-property heron.kubernetes.manager.requests.memory=100Mi
```
<details><summary>Manager StatefulSet</summary>
```yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
creationTimestamp: "2021-12-05T16:12:40Z"
generation: 1
labels:
app: heron
topology: acking
name: acking-manager
namespace: default
resourceVersion: "858"
uid: 6be69dad-2943-4813-9c26-7ed5e185a0e1
spec:
podManagementPolicy: Parallel
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
app: heron
topology: acking
serviceName: acking
template:
metadata:
annotations:
prometheus.io/port: "8080"
prometheus.io/scrape: "true"
creationTimestamp: null
labels:
app: heron
topology: acking
spec:
containers:
- command:
- sh
- -c
- './heron-core/bin/heron-downloader-config kubernetes &&
./heron-core/bin/heron-downloader
distributedlog://zookeeper:2181/heronbkdl/acking-saad-tag-0--4532610184000198972.tar.gz
. && SHARD_ID=${POD_NAME##*-} && echo shardId=${SHARD_ID} &&
./heron-core/bin/heron-executor
--topology-name=acking
--topology-id=acking2d4d1f63-90db-435d-9e2b-6be7f5bfc0ee
--topology-defn-file=acking.defn
--state-manager-connection=zookeeper:2181
--state-manager-root=/heron
--state-manager-config-file=./heron-conf/statemgr.yaml
--tmanager-binary=./heron-core/bin/heron-tmanager
--stmgr-binary=./heron-core/bin/heron-stmgr
--metrics-manager-classpath=./heron-core/lib/metricsmgr/*
--instance-jvm-opts="LVhYOitIZWFwRHVtcE9uT3V0T2ZNZW1vcnlFcnJvcg(61)(61)"
--classpath=heron-api-examples.jar
--heron-internals-config-file=./heron-conf/heron_internals.yaml
--override-config-file=./heron-conf/override.yaml
--component-ram-map=exclaim1:1073741824,word:1073741824
--component-jvm-opts="" --pkg-type=jar
--topology-binary-file=heron-api-examples.jar
--heron-java-home=$JAVA_HOME
--heron-shell-binary=./heron-core/bin/heron-shell
--cluster=kubernetes --role=saad --environment=default
--instance-classpath=./heron-core/lib/instance/*
--metrics-sinks-config-file=./heron-conf/metrics_sinks.yaml
--scheduler-classpath=./heron-core/lib/scheduler/*:./heron-core/lib/packing/*:./heron-core/lib/statemgr/*
--python-instance-binary=./heron-core/bin/heron-python-instance
--cpp-instance-binary=./heron-core/bin/heron-cpp-instance
--metricscache-manager-classpath=./heron-core/lib/metricscachemgr/*
--metricscache-manager-mode=disabled
--is-stateful=false
--checkpoint-manager-classpath=./heron-core/lib/ckptmgr/*:./heron-core/lib/statefulstorage/*:
--stateful-config-file=./heron-conf/stateful.yaml
--checkpoint-manager-ram=1073741824
--health-manager-mode=disabled
--health-manager-classpath=./heron-core/lib/healthmgr/*
--shard=$SHARD_ID --server-port=6001
--tmanager-controller-port=6002 --tmanager-stats-port=6003
--shell-port=6004 --metrics-manager-port=6005
--scheduler-port=6006 --metricscache-manager-server-port=6007
--metricscache-manager-stats-port=6008
--checkpoint-manager-port=6009'
env:
- name: HOST
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: status.podIP
- name: POD_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.name
image: apache/heron:testbuild
imagePullPolicy: IfNotPresent
name: manager
ports:
- containerPort: 6003
name: tmanager-stats
protocol: TCP
- containerPort: 6007
name: metrics-cache-m
protocol: TCP
- containerPort: 6004
name: shell-port
protocol: TCP
- containerPort: 6001
name: server
protocol: TCP
- containerPort: 6002
name: tmanager-ctl
protocol: TCP
- containerPort: 6009
name: ckptmgr
protocol: TCP
- containerPort: 6006
name: scheduler
protocol: TCP
- containerPort: 6005
name: metrics-mgr
protocol: TCP
- containerPort: 6008
name: metrics-cache-s
protocol: TCP
resources:
limits:
cpu: 300m
memory: 300Mi
requests:
cpu: 20m
memory: 100Mi
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
terminationGracePeriodSeconds: 0
tolerations:
- effect: NoExecute
key: node.kubernetes.io/not-ready
operator: Exists
tolerationSeconds: 10
- effect: NoExecute
key: node.kubernetes.io/unreachable
operator: Exists
tolerationSeconds: 10
updateStrategy:
rollingUpdate:
partition: 0
type: RollingUpdate
status:
availableReplicas: 1
collisionCount: 0
currentReplicas: 1
currentRevision: acking-manager-f5f4784
observedGeneration: 1
readyReplicas: 1
replicas: 1
updateRevision: acking-manager-f5f4784
updatedReplicas: 1
```
</details>
<details><summary>Executor StatefulSet</summary>
```yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
creationTimestamp: "2021-12-05T16:12:40Z"
generation: 1
labels:
app: heron
topology: acking
name: acking-executors
namespace: default
resourceVersion: "862"
uid: afe8b301-953d-49fa-af05-37289f9cf721
spec:
podManagementPolicy: Parallel
replicas: 2
revisionHistoryLimit: 10
selector:
matchLabels:
app: heron
topology: acking
serviceName: acking
template:
metadata:
annotations:
prometheus.io/port: "8080"
prometheus.io/scrape: "true"
creationTimestamp: null
labels:
app: heron
topology: acking
spec:
containers:
- command:
- sh
- -c
- './heron-core/bin/heron-downloader-config kubernetes &&
./heron-core/bin/heron-downloader
distributedlog://zookeeper:2181/heronbkdl/acking-saad-tag-0--4532610184000198972.tar.gz
. && SHARD_ID=$((${POD_NAME##*-} + 1)) && echo shardId=${SHARD_ID}
&& ./heron-core/bin/heron-executor
--topology-name=acking
--topology-id=acking2d4d1f63-90db-435d-9e2b-6be7f5bfc0ee
--topology-defn-file=acking.defn
--state-manager-connection=zookeeper:2181
--state-manager-root=/heron
--state-manager-config-file=./heron-conf/statemgr.yaml
--tmanager-binary=./heron-core/bin/heron-tmanager
--stmgr-binary=./heron-core/bin/heron-stmgr
--metrics-manager-classpath=./heron-core/lib/metricsmgr/*
--instance-jvm-opts="LVhYOitIZWFwRHVtcE9uT3V0T2ZNZW1vcnlFcnJvcg(61)(61)"
--classpath=heron-api-examples.jar
--heron-internals-config-file=./heron-conf/heron_internals.yaml
--override-config-file=./heron-conf/override.yaml
--component-ram-map=exclaim1:1073741824,word:1073741824
--component-jvm-opts="" --pkg-type=jar
--topology-binary-file=heron-api-examples.jar
--heron-java-home=$JAVA_HOME
--heron-shell-binary=./heron-core/bin/heron-shell
--cluster=kubernetes --role=saad --environment=default
--instance-classpath=./heron-core/lib/instance/*
--metrics-sinks-config-file=./heron-conf/metrics_sinks.yaml
--scheduler-classpath=./heron-core/lib/scheduler/*:./heron-core/lib/packing/*:./heron-core/lib/statemgr/*
--python-instance-binary=./heron-core/bin/heron-python-instance
--cpp-instance-binary=./heron-core/bin/heron-cpp-instance
--metricscache-manager-classpath=./heron-core/lib/metricscachemgr/*
--metricscache-manager-mode=disabled
--is-stateful=false
--checkpoint-manager-classpath=./heron-core/lib/ckptmgr/*:./heron-core/lib/statefulstorage/*:
--stateful-config-file=./heron-conf/stateful.yaml
--checkpoint-manager-ram=1073741824
--health-manager-mode=disabled
--health-manager-classpath=./heron-core/lib/healthmgr/*
--shard=$SHARD_ID --server-port=6001
--tmanager-controller-port=6002 --tmanager-stats-port=6003
--shell-port=6004 --metrics-manager-port=6005
--scheduler-port=6006 --metricscache-manager-server-port=6007
--metricscache-manager-stats-port=6008
--checkpoint-manager-port=6009'
env:
- name: HOST
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: status.podIP
- name: POD_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.name
image: apache/heron:testbuild
imagePullPolicy: IfNotPresent
name: executor
ports:
- containerPort: 6003
name: tmanager-stats
protocol: TCP
- containerPort: 6007
name: metrics-cache-m
protocol: TCP
- containerPort: 6004
name: shell-port
protocol: TCP
- containerPort: 6001
name: server
protocol: TCP
- containerPort: 6002
name: tmanager-ctl
protocol: TCP
- containerPort: 6009
name: ckptmgr
protocol: TCP
- containerPort: 6006
name: scheduler
protocol: TCP
- containerPort: 6005
name: metrics-mgr
protocol: TCP
- containerPort: 6008
name: metrics-cache-s
protocol: TCP
resources:
limits:
cpu: "3"
memory: 4Gi
requests:
cpu: "3"
memory: 4Gi
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
terminationGracePeriodSeconds: 0
tolerations:
- effect: NoExecute
key: node.kubernetes.io/not-ready
operator: Exists
tolerationSeconds: 10
- effect: NoExecute
key: node.kubernetes.io/unreachable
operator: Exists
tolerationSeconds: 10
updateStrategy:
rollingUpdate:
partition: 0
type: RollingUpdate
status:
availableReplicas: 2
collisionCount: 0
currentReplicas: 2
currentRevision: acking-executors-548c6dbd6c
observedGeneration: 1
readyReplicas: 2
replicas: 2
updateRevision: acking-executors-548c6dbd6c
updatedReplicas: 2
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
