This is an automated email from the ASF dual-hosted git repository. zhifgli pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-uniffle-website.git
commit a3963d9da26db3922dd8c86b7d297887470de81d Author: frankzfli <[email protected]> AuthorDate: Tue Oct 25 20:55:25 2022 +0800 Add metrics docs --- docs/01-intro.md | 17 ----------------- docs/02-client-guide.md | 17 ----------------- .../00-coordinator-guide.md | 17 ----------------- .../01-server-guide.md | 17 ----------------- .../02-hardware-provisioning.md | 17 ----------------- .../00-uniffle-operator-design.md | 17 ----------------- .../01-install.md | 18 ------------------ .../02-examples.md | 18 ------------------ docs/05-metrics.md | 22 ++++++++++++++++++++-- 9 files changed, 20 insertions(+), 140 deletions(-) diff --git a/docs/01-intro.md b/docs/01-intro.md index 2ae9ca2..3f2b737 100644 --- a/docs/01-intro.md +++ b/docs/01-intro.md @@ -1,20 +1,3 @@ -<!-- - ~ Licensed to the Apache Software Foundation (ASF) under one or more - ~ contributor license agreements. See the NOTICE file distributed with - ~ this work for additional information regarding copyright ownership. - ~ The ASF licenses this file to You under the Apache License, Version 2.0 - ~ (the "License"); you may not use this file except in compliance with - ~ the License. You may obtain a copy of the License at - ~ - ~ http://www.apache.org/licenses/LICENSE-2.0 - ~ - ~ Unless required by applicable law or agreed to in writing, software - ~ distributed under the License is distributed on an "AS IS" BASIS, - ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - ~ See the License for the specific language governing permissions and - ~ limitations under the License. - --> - --- title: intro sidebar_position: 1 diff --git a/docs/02-client-guide.md b/docs/02-client-guide.md index dce7120..4bb2946 100644 --- a/docs/02-client-guide.md +++ b/docs/02-client-guide.md @@ -1,20 +1,3 @@ -<!-- - ~ Licensed to the Apache Software Foundation (ASF) under one or more - ~ contributor license agreements. See the NOTICE file distributed with - ~ this work for additional information regarding copyright ownership. - ~ The ASF licenses this file to You under the Apache License, Version 2.0 - ~ (the "License"); you may not use this file except in compliance with - ~ the License. You may obtain a copy of the License at - ~ - ~ http://www.apache.org/licenses/LICENSE-2.0 - ~ - ~ Unless required by applicable law or agreed to in writing, software - ~ distributed under the License is distributed on an "AS IS" BASIS, - ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - ~ See the License for the specific language governing permissions and - ~ limitations under the License. - --> - # Uniffle Shuffle Client Guide Uniffle is designed as a unified shuffle engine for multiple computing frameworks, including Apache Spark and Apache Hadoop. diff --git a/docs/03-deploy-uniffle-cluster/00-coordinator-guide.md b/docs/03-Deploy/00-coordinator-guide.md similarity index 87% rename from docs/03-deploy-uniffle-cluster/00-coordinator-guide.md rename to docs/03-Deploy/00-coordinator-guide.md index 0afb99c..0274885 100644 --- a/docs/03-deploy-uniffle-cluster/00-coordinator-guide.md +++ b/docs/03-Deploy/00-coordinator-guide.md @@ -1,20 +1,3 @@ -<!-- - ~ Licensed to the Apache Software Foundation (ASF) under one or more - ~ contributor license agreements. See the NOTICE file distributed with - ~ this work for additional information regarding copyright ownership. - ~ The ASF licenses this file to You under the Apache License, Version 2.0 - ~ (the "License"); you may not use this file except in compliance with - ~ the License. You may obtain a copy of the License at - ~ - ~ http://www.apache.org/licenses/LICENSE-2.0 - ~ - ~ Unless required by applicable law or agreed to in writing, software - ~ distributed under the License is distributed on an "AS IS" BASIS, - ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - ~ See the License for the specific language governing permissions and - ~ limitations under the License. - --> - # Uniffle Coordinator Guide Uniffle is a unified remote shuffle service for compute engines, the role of coordinator is responsibility for diff --git a/docs/03-deploy-uniffle-cluster/01-server-guide.md b/docs/03-Deploy/01-server-guide.md similarity index 79% rename from docs/03-deploy-uniffle-cluster/01-server-guide.md rename to docs/03-Deploy/01-server-guide.md index a11c339..6a00a00 100644 --- a/docs/03-deploy-uniffle-cluster/01-server-guide.md +++ b/docs/03-Deploy/01-server-guide.md @@ -1,20 +1,3 @@ -<!-- - ~ Licensed to the Apache Software Foundation (ASF) under one or more - ~ contributor license agreements. See the NOTICE file distributed with - ~ this work for additional information regarding copyright ownership. - ~ The ASF licenses this file to You under the Apache License, Version 2.0 - ~ (the "License"); you may not use this file except in compliance with - ~ the License. You may obtain a copy of the License at - ~ - ~ http://www.apache.org/licenses/LICENSE-2.0 - ~ - ~ Unless required by applicable law or agreed to in writing, software - ~ distributed under the License is distributed on an "AS IS" BASIS, - ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - ~ See the License for the specific language governing permissions and - ~ limitations under the License. - --> - # Uniffle Shuffle Server Guide ## Deploy diff --git a/docs/03-deploy-uniffle-cluster/02-hardware-provisioning.md b/docs/03-Deploy/02-hardware-provisioning.md similarity index 56% rename from docs/03-deploy-uniffle-cluster/02-hardware-provisioning.md rename to docs/03-Deploy/02-hardware-provisioning.md index 66b2951..d7182a2 100644 --- a/docs/03-deploy-uniffle-cluster/02-hardware-provisioning.md +++ b/docs/03-Deploy/02-hardware-provisioning.md @@ -1,20 +1,3 @@ -<!-- - ~ Licensed to the Apache Software Foundation (ASF) under one or more - ~ contributor license agreements. See the NOTICE file distributed with - ~ this work for additional information regarding copyright ownership. - ~ The ASF licenses this file to You under the Apache License, Version 2.0 - ~ (the "License"); you may not use this file except in compliance with - ~ the License. You may obtain a copy of the License at - ~ - ~ http://www.apache.org/licenses/LICENSE-2.0 - ~ - ~ Unless required by applicable law or agreed to in writing, software - ~ distributed under the License is distributed on an "AS IS" BASIS, - ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - ~ See the License for the specific language governing permissions and - ~ limitations under the License. - --> - # HARDWARE PROVISIONING A common question received by Uniffle users is how to configure hardware for it. While the right hardware will depend on the situation, we make the following recommendations. diff --git a/docs/04-how-to-deploy-on-k8s/00-uniffle-operator-design.md b/docs/04-k8s/00-uniffle-operator-design.md similarity index 87% rename from docs/04-how-to-deploy-on-k8s/00-uniffle-operator-design.md rename to docs/04-k8s/00-uniffle-operator-design.md index 13c81da..fb57447 100644 --- a/docs/04-how-to-deploy-on-k8s/00-uniffle-operator-design.md +++ b/docs/04-k8s/00-uniffle-operator-design.md @@ -1,20 +1,3 @@ -<!-- - ~ Licensed to the Apache Software Foundation (ASF) under one or more - ~ contributor license agreements. See the NOTICE file distributed with - ~ this work for additional information regarding copyright ownership. - ~ The ASF licenses this file to You under the Apache License, Version 2.0 - ~ (the "License"); you may not use this file except in compliance with - ~ the License. You may obtain a copy of the License at - ~ - ~ http://www.apache.org/licenses/LICENSE-2.0 - ~ - ~ Unless required by applicable law or agreed to in writing, software - ~ distributed under the License is distributed on an "AS IS" BASIS, - ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - ~ See the License for the specific language governing permissions and - ~ limitations under the License. - --> - # Uniffle Operator Design ## Summary diff --git a/docs/04-how-to-deploy-on-k8s/01-install.md b/docs/04-k8s/01-install.md similarity index 63% rename from docs/04-how-to-deploy-on-k8s/01-install.md rename to docs/04-k8s/01-install.md index 82b8930..b6c4fb4 100644 --- a/docs/04-how-to-deploy-on-k8s/01-install.md +++ b/docs/04-k8s/01-install.md @@ -1,21 +1,3 @@ -<!-- - ~ Licensed to the Apache Software Foundation (ASF) under one or more - ~ contributor license agreements. See the NOTICE file distributed with - ~ this work for additional information regarding copyright ownership. - ~ The ASF licenses this file to You under the Apache License, Version 2.0 - ~ (the "License"); you may not use this file except in compliance with - ~ the License. You may obtain a copy of the License at - ~ - ~ http://www.apache.org/licenses/LICENSE-2.0 - ~ - ~ Unless required by applicable law or agreed to in writing, software - ~ distributed under the License is distributed on an "AS IS" BASIS, - ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - ~ See the License for the specific language governing permissions and - ~ limitations under the License. - --> - - # Installation This section shows us how to install operator in our cluster. diff --git a/docs/04-how-to-deploy-on-k8s/02-examples.md b/docs/04-k8s/02-examples.md similarity index 55% rename from docs/04-how-to-deploy-on-k8s/02-examples.md rename to docs/04-k8s/02-examples.md index f8514c0..d18939c 100644 --- a/docs/04-how-to-deploy-on-k8s/02-examples.md +++ b/docs/04-k8s/02-examples.md @@ -1,21 +1,3 @@ -<!-- - ~ Licensed to the Apache Software Foundation (ASF) under one or more - ~ contributor license agreements. See the NOTICE file distributed with - ~ this work for additional information regarding copyright ownership. - ~ The ASF licenses this file to You under the Apache License, Version 2.0 - ~ (the "License"); you may not use this file except in compliance with - ~ the License. You may obtain a copy of the License at - ~ - ~ http://www.apache.org/licenses/LICENSE-2.0 - ~ - ~ Unless required by applicable law or agreed to in writing, software - ~ distributed under the License is distributed on an "AS IS" BASIS, - ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - ~ See the License for the specific language governing permissions and - ~ limitations under the License. - --> - - # Examples We need to create configMap first which saves coordinators, shuffleServers and log4j's configuration(we can refer diff --git a/docs/05-metrics.md b/docs/05-metrics.md index aac7ffe..f152f76 100644 --- a/docs/05-metrics.md +++ b/docs/05-metrics.md @@ -1,3 +1,21 @@ -# Import Metrics +# Metrics System -for test \ No newline at end of file +In Uniffle, both coordinator and shuffle server have provided many runtime metrics, which help to maintain the Uniffle cluster in production. + +We list a [script](https://github.com/apache/incubator-uniffle/blob/master/bin/get-metrics.py) to get these metrics via http protocol. + +``` +python get-metrics.py [serverListFile] [jettyPort] [metricName] +``` + +Extract server ip from line in the serverListFile, which contains lines of server node and the format is $IP-$PORT. +The ip and port are server's rpc ip and port respectively, the port is useless here so we only extract the +ip and assemble the url with jetty http port to get metrics and extract the value of the metric specified by metricName. + +See all available [server runtime metrics](https://github.com/apache/incubator-uniffle/blob/master/server/src/main/java/org/apache/uniffle/server/ShuffleServerMetrics.java). + +See all available [server grpc metrics](https://github.com/apache/incubator-uniffle/blob/master/server/src/main/java/org/apache/uniffle/server/ShuffleServerGrpcMetrics.java). + +See all available [coordinator runtime metrics](https://github.com/apache/incubator-uniffle/blob/master/coordinator/src/main/java/org/apache/uniffle/coordinator/CoordinatorMetrics.java). + +See all available [coordinator grpc metrics](https://github.com/apache/incubator-uniffle/blob/master/coordinator/src/main/java/org/apache/uniffle/coordinator/CoordinatorMetrics.java). \ No newline at end of file
