Re: Bug : PutGCSObject : Version NIFI 1.23.2

[e-sociaux]

Hi Pierre,

 

But it seems weird because when I put the wrong "storage api url : https://storage-euwest1.p.googleapis.com/storage/v1/"

 

We got the wrong url but has the correct hostname

 

Not Found POST https://storage-euwest1.p.googleapis.com/upload/storage/v1/storage/v1/b/xxx/o?name=test/minh.txt&uploadType=resumable

 

Envoyé: mercredi 8 novembre 2023 à 16:29
De: "Pierre Villard" <pierre.villard...@gmail.com>
À: users@nifi.apache.org
Objet: Re: Bug : PutGCSObject : Version NIFI 1.23.2

IIRC, but not in a position to confirm right now, the right API to set would be https://storage-euwest1.p.googleapis.com but your host should still be able to resolve googleapis.com (I believe this can work through proxy as well).

 

Le mer. 8 nov. 2023 à 16:16, <e-soci...@gmx.fr> a écrit :

Hi 

 

I have checked the DNS entry for "storage-euwest1.p.googleapis.com" it seems OK

From the VM nifi, in the terminal, I could reach the port 443 it works fine.

 

I check with the devops google team also.

 

Thanks

 

Envoyé: mercredi 8 novembre 2023 à 15:40
De: "Joe Witt" <joe.w...@gmail.com>
À: users@nifi.apache.org
Objet: Re: Bug : PutGCSObject : Version NIFI 1.23.2

Ok sounds good. I do think this is the correct path to be taking then.

 

I think though there might be a dns configuration needed.  Im not quite sure but I am aware of users that rely on this value heavily.

 

Thanks

 

On Wed, Nov 8, 2023 at 7:34 AM <e-soci...@gmx.fr> wrote:

Hello Joe

 

Until now we leave the "storage api url" empty by default, but we use "proxy configuration service" because we are behind the corporate network.

 

Since some month ago we have configured "private connection" with google.

So I have setup "storage api url" with the private url and leave empty "proxy configuration service"

 

Regards 

 

 

Envoyé: mercredi 8 novembre 2023 à 14:31
De: "Joe Witt" <joe.w...@gmail.com>
À: users@nifi.apache.org
Objet: Re: Bug : PutGCSObject : Version NIFI 1.23.2

Minh

 

Do you need to override that value?  Did Google specify a different URL for you or you have a private connection?

 

If you dont set that value what happens?

 

Thanks

 

On Wed, Nov 8, 2023 at 4:57 AM <e-soci...@gmx.fr> wrote:

Hello,

 

I'm trying to user the overwriting storage url parameter in the nifi version 1.23.2.

 

Do you know what is the correct parameter to set for the "Storage API URL" in the processor PutGCSObject ?

 

If I setup "https://storage-euwest1.p.googleapis.com/storage/v1/"

It got this error :

 

PutGCSObject[id=aea7e07c-018b-1000-ffff] Failure completing upload flowfile=test/minh.txt bucket=xxx key=test/minh.txt reason=404 Not FoundPOST https://storage-euwest1.p.googleapis.com/upload/storage/v1/storage/v1/b/xxx/o?name=test/minh.txt&uploadType=resumableNot Found: com.google.cloud.storage.StorageException: 404 Not Found

 


If I setup "https://storage-euwest1.p.googleapis.com"

It got this error :




PutGCSObject[id=aea7e07c-018b-1000-ffff] Failure completing upload flowfile=test/minh.txt bucket=xxx key=test/minh.txt reason=www.googleapis.com: com.google.cloud.storage.StorageException: www.googleapis.com- Caused by: java.net.UnknownHostException: www.googleapis.com



 

Thanks for help

 

Minh