I don't know why we delete it either, but my guess is that at one point
this was a temporary directory that we properly cleaned up, and later
allowed it to be configurable.
Currently, this directory must be in the local file system. We could
change it to also allow non-local paths, which should be fairly easy to
do. Add a condition that the directory is only cleaned up when it wasn't
explicitly configured and we should've covered everything.
In regards to the HistoryServer, i mean of course could we include the
user-jar, but I'm wondering about the user-cases for it.
It's not like you could "just download it and give it to a JobManager";
you would be missing any additional libraries put under /lib, as well as
the original configuration of the cluster. It would also require extra
steps (and tools) to identify which dependencies are required.
And there are downsides to this, like a significant increase in the size
of the job archives (along with a duplication of jars for every job
submission), and the archiving also becomes more complex.
On 17.05.2017 15:52, Timo Walther wrote:
Hey Mauro,
I'm not aware of any reason for that. I loop in Chesnay, maybe he
knows why. @Chesnay wouldn't it be helpful to also archive the jars
using the HistoryServer?
Timo
Am 17.05.17 um 12:31 schrieb Mauro Cortellazzi:
Hi Flink comunity,
is there a particular reason to delete the jobmanager.web.upload.dir
content when the JobManager restart? Could it be interesting have the
jar previously uploaded when JM restart? Could the jars be saved to
HDFS for HA mode?
Thank you for your support.
Mauro
