[GitHub] flink pull request #5209: [FLINK-7711] Port JarListHandler to WebMonitorEndp...

2018-02-17 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/flink/pull/5209


---


[GitHub] flink pull request #5209: [FLINK-7711] Port JarListHandler to WebMonitorEndp...

2018-01-10 Thread GJL
Github user GJL commented on a diff in the pull request:

https://github.com/apache/flink/pull/5209#discussion_r160638625
  
--- Diff: 
flink-runtime/src/main/java/org/apache/flink/runtime/util/JarWithProgramUtils.java
 ---
@@ -0,0 +1,416 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.util;
+
+import org.apache.flink.api.common.Program;
+import org.apache.flink.api.common.ProgramDescription;
+import org.apache.flink.util.InstantiationUtil;
+
+import java.io.BufferedInputStream;
+import java.io.File;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.lang.reflect.Method;
+import java.lang.reflect.Modifier;
+import java.net.MalformedURLException;
+import java.net.URISyntaxException;
+import java.net.URL;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.Enumeration;
+import java.util.List;
+import java.util.Random;
+import java.util.jar.Attributes;
+import java.util.jar.JarEntry;
+import java.util.jar.JarFile;
+import java.util.jar.Manifest;
+
+/**
+ * Jar file validation tool class for {@link Program}.
+ */
+public class JarWithProgramUtils {
--- End diff --

I can see that you had to copy a lot of code from `PackagedProgram` because 
`flink-runtime` does not have a dependency on `flink-clients`. Also, 
`JobWithJars` had to be moved to the `flink-runtime` module. I think we should 
try to avoid duplicating the code in `PackagedProgram`. After talking to 
@tillrohrmann offline, we think that we could achieve this by putting the new 
`JarUploadHandler` into the `flink-runtime-web` module. This module already has 
a dependency to `flink-clients` and `flink-runtime`. In `WebMonitorEndpoint`, 
the jar handlers could be dynamically loaded according to whether 
`flink-runtime-web` is in the classpath or not; similar to the 
`StaticFileServerHandler` (see  `WebMonitorUtils.tryLoadWebContent()`). What do 
you think?


---


[GitHub] flink pull request #5209: [FLINK-7711] Port JarListHandler to WebMonitorEndp...

2017-12-27 Thread zjureel
GitHub user zjureel opened a pull request:

https://github.com/apache/flink/pull/5209

[FLINK-7711] Port JarListHandler to WebMonitorEndpoint

## What is the purpose of the change

Port JarListHandler to WebMonitorEndpoint

## Brief change log

  - *Create JarWithProgramUtils from PackageProgram to check jar file*
  - *Create new JarListHandler and JarListInfo*
  - *Add uploadDir to RestHandlerConfiguration*


## Verifying this change

This change added tests and can be verified as follows:

  - *Added test case JarListInfoTest*

## Does this pull request potentially affect one of the following parts:

  - Dependencies (does it add or upgrade a dependency): (no)
  - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (no)
  - The serializers: (no)
  - The runtime per-record code paths (performance sensitive): (no)
  - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
  - The S3 file system connector: (no)

## Documentation

  - Does this pull request introduce a new feature? (no)
  - If yes, how is the feature documented? (not applicable / docs / 
JavaDocs / not documented)


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/zjureel/flink FLINK-7711

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/5209.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5209


commit 145f153530914d0ee8066686a1fb382c6ddf5af5
Author: zjureel 
Date:   2017-12-27T07:34:57Z

[FLINK-7711] Create program validation class in runtime project for 
JarListHandler

commit c8f30345a93e6640d4b91d887c5caa4184068782
Author: zjureel 
Date:   2017-12-27T07:35:27Z

[FLINK-7711] Port JarListHander in WebMonitorEndpoint

commit 48ee1d1ca4c0bc9cf6a6520b0135e9abdbcc3458
Author: zjureel 
Date:   2017-12-28T01:50:10Z

[FLINK-7711] Add test case for JarListInfo




---