Hi Sumeet,
The archive files will be uploaded to the blob server. This is the same no
matter specifying the archives via command line option `—pyArchives` or via
`add_python_archive`.
> And when I try to programmatically do this by calling add_python_archive(),
> the job gets submitted but
I'm using a standalone deployment on Kubernetes for this use case. Does the
archive get uploaded to the cluster via the :8081 REST/WebUI port or via
some other port like 6123/RPC or 6124/BLOB-SERVER? I'm wondering if not
exposing those ports on the local machine might prevent the archive from
Hi Sumeet,
Probably there is an issue with uploading the archive while submitting the job.
The commands and API usage look good to me.
Dian could you please confirm that?
Regards,
Roman
On Fri, Jun 11, 2021 at 9:04 AM Sumeet Malhotra
wrote:
>
> Thank you Roman. Yes, that's what I am going to
Thank you Roman. Yes, that's what I am going to do.
But I'm running into another issue... when I specify the *--pyArchives*
option on the command line, the job never gets submitted and is stuck
forever. And when I try to programmatically do this by calling
*add_python_archive()*, the job gets
Hi,
I think the second option is what you need. The documentation says
only zip format is supported.
Alternatively, you could upload the files to S3 or other DFS and
access from TMs and re-upload when needed.
[1]
Hi,
I'm using UDTFs in PyFlink, that depend upon a few resource files (JSON
schema files actually). The path of this file can be passed into the UDTF,
but essentially this path needs to exist on the Task Manager node where the
task executes. What's the best way to upload these resource files? As
