[jira] [Commented] (JCLOUDS-1615) Using Azure SAS Token unable to upload some parts of the file (blockid is "Not a valid base64 string")
[
https://issues.apache.org/jira/browse/JCLOUDS-1615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605198#comment-17605198
]
Christian Jung commented on JCLOUDS-1615:
-
I added a PR #155, but, as i have no experience with github, it might not be
linked correctly to this issue.
> Using Azure SAS Token unable to upload some parts of the file (blockid is
> "Not a valid base64 string")
> --
>
> Key: JCLOUDS-1615
> URL: https://issues.apache.org/jira/browse/JCLOUDS-1615
> Project: jclouds
> Issue Type: Bug
> Components: jclouds-blobstore
>Affects Versions: 2.5.0
>Reporter: Amber Liu
>Assignee: Andrew Gaul
>Priority: Major
> Labels: azureblob
>
> I'm trying to [put
> block|https://docs.microsoft.com/en-us/rest/api/storageservices/put-block#request]
> with Azure SAS Token.
> However, it failed in some "specific" parts.
> Here is the error message:
> {code:java}
> org.jclouds.azure.storage.AzureStorageResponseException: command
> [method=org.jclouds.azureblob.AzureBlobClient.public abstract void
> org.jclouds.azureblob.AzureBlobClient.putBlock(java.lang.String,java.lang.String,java.lang.String,org.jclouds.io.Payload)[azure-auto-123,
> sub2/100MBFile, AAV+QA==, [content=true, contentMetadata=[cacheControl=null,
> contentDisposition=null, contentEncoding=null, contentLanguage=null,
> contentLength=1048576, contentMD5=null, contentType=application/unknown,
> expires=null], written=false, isSensitive=false]], request=PUT
> https://casbazure.blob.core.windows.net/azure-auto-123/sub2/100MBFile?comp=block=AAV%2BQA%3D%3D
> HTTP/1.1] failed with code 400, error:
> AzureError{requestId='5b9a2f41-301e-008d-4e5b-97e57d00',
> code='InvalidQueryParameterValue', message='Value for one of the query
> parameters specified in the request URI is invalid.
> RequestId:5b9a2f41-301e-008d-4e5b-97e57d00
> Time:2022-07-14T08:24:41.1097266Z', context='{QueryParameterValue=AAV QA==,
> QueryParameterName=blockid, Reason=Not a valid base64 string.}'} {code}
> As you can see, the *blockid* becomes "AAV QA==" which is a invalid base64
> string.
>
> *My Observation*
> I checked the failed parts and they all have "+" in their *blockid.*
> For example, from part 1 to part 500, the following parts have "+" in their
> blockid :
>
> ||Part Number||blockid||
> |248|+A==|
> |249|+Q==|
> |250|+g==|
> |251|+w==|
>
> I used the same code in
> [here|https://github.com/apache/jclouds/blob/0b89ee0825d45de1193090cdd5efc5f1135fa200/providers/azureblob/src/main/java/org/jclouds/azureblob/blobstore/AzureBlobStore.java#L444-L450]
> to generate *blockid*
> {code:java}
> String blockId = BaseEncoding.base64().encode(Ints.toByteArray(partNumber));
> {code}
>
> And my version of *guava* is 27.1-jre which is used in jclouds 2.5.0
> {code:java}
>
> com.google.guava
> guava
> 27.1-jre
> {code}
>
> By the way, it works fine with *Azure account name* and {*}account key{*}.
> I can successfully upload all parts.
> (I think the *blockid* still has "+" but I don't know why it works with
> *Azure account name* and {*}account key{*})
>
> Please help me to fix this problem. Thank you!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (JCLOUDS-1615) Using Azure SAS Token unable to upload some parts of the file (blockid is "Not a valid base64 string")
[
https://issues.apache.org/jira/browse/JCLOUDS-1615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17603758#comment-17603758
]
Elvan commented on JCLOUDS-1615:
[~gaul] , How do you retrieve the Azure SAS Token?
> Using Azure SAS Token unable to upload some parts of the file (blockid is
> "Not a valid base64 string")
> --
>
> Key: JCLOUDS-1615
> URL: https://issues.apache.org/jira/browse/JCLOUDS-1615
> Project: jclouds
> Issue Type: Bug
> Components: jclouds-blobstore
>Affects Versions: 2.5.0
>Reporter: Amber Liu
>Assignee: Andrew Gaul
>Priority: Major
> Labels: azureblob
>
> I'm trying to [put
> block|https://docs.microsoft.com/en-us/rest/api/storageservices/put-block#request]
> with Azure SAS Token.
> However, it failed in some "specific" parts.
> Here is the error message:
> {code:java}
> org.jclouds.azure.storage.AzureStorageResponseException: command
> [method=org.jclouds.azureblob.AzureBlobClient.public abstract void
> org.jclouds.azureblob.AzureBlobClient.putBlock(java.lang.String,java.lang.String,java.lang.String,org.jclouds.io.Payload)[azure-auto-123,
> sub2/100MBFile, AAV+QA==, [content=true, contentMetadata=[cacheControl=null,
> contentDisposition=null, contentEncoding=null, contentLanguage=null,
> contentLength=1048576, contentMD5=null, contentType=application/unknown,
> expires=null], written=false, isSensitive=false]], request=PUT
> https://casbazure.blob.core.windows.net/azure-auto-123/sub2/100MBFile?comp=block=AAV%2BQA%3D%3D
> HTTP/1.1] failed with code 400, error:
> AzureError{requestId='5b9a2f41-301e-008d-4e5b-97e57d00',
> code='InvalidQueryParameterValue', message='Value for one of the query
> parameters specified in the request URI is invalid.
> RequestId:5b9a2f41-301e-008d-4e5b-97e57d00
> Time:2022-07-14T08:24:41.1097266Z', context='{QueryParameterValue=AAV QA==,
> QueryParameterName=blockid, Reason=Not a valid base64 string.}'} {code}
> As you can see, the *blockid* becomes "AAV QA==" which is a invalid base64
> string.
>
> *My Observation*
> I checked the failed parts and they all have "+" in their *blockid.*
> For example, from part 1 to part 500, the following parts have "+" in their
> blockid :
>
> ||Part Number||blockid||
> |248|+A==|
> |249|+Q==|
> |250|+g==|
> |251|+w==|
>
> I used the same code in
> [here|https://github.com/apache/jclouds/blob/0b89ee0825d45de1193090cdd5efc5f1135fa200/providers/azureblob/src/main/java/org/jclouds/azureblob/blobstore/AzureBlobStore.java#L444-L450]
> to generate *blockid*
> {code:java}
> String blockId = BaseEncoding.base64().encode(Ints.toByteArray(partNumber));
> {code}
>
> And my version of *guava* is 27.1-jre which is used in jclouds 2.5.0
> {code:java}
>
> com.google.guava
> guava
> 27.1-jre
> {code}
>
> By the way, it works fine with *Azure account name* and {*}account key{*}.
> I can successfully upload all parts.
> (I think the *blockid* still has "+" but I don't know why it works with
> *Azure account name* and {*}account key{*})
>
> Please help me to fix this problem. Thank you!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (JCLOUDS-1615) Using Azure SAS Token unable to upload some parts of the file (blockid is "Not a valid base64 string")
[
https://issues.apache.org/jira/browse/JCLOUDS-1615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17603582#comment-17603582
]
Christian Jung commented on JCLOUDS-1615:
-
Surely there is already some test covering multipart. (Where?)
So if this test just uploads a big payload, the current issue could be
reproduced by simply increasing the payload size.
Otherwise, i have no better proposal than extracting some method
{code:java}
static String makeBlockId(int partNumber)
{code}
and do unit tests on it.
This would be better than just changing the code, but not much.
> Using Azure SAS Token unable to upload some parts of the file (blockid is
> "Not a valid base64 string")
> --
>
> Key: JCLOUDS-1615
> URL: https://issues.apache.org/jira/browse/JCLOUDS-1615
> Project: jclouds
> Issue Type: Bug
> Components: jclouds-blobstore
>Affects Versions: 2.5.0
>Reporter: Amber Liu
>Assignee: Andrew Gaul
>Priority: Major
> Labels: azureblob
>
> I'm trying to [put
> block|https://docs.microsoft.com/en-us/rest/api/storageservices/put-block#request]
> with Azure SAS Token.
> However, it failed in some "specific" parts.
> Here is the error message:
> {code:java}
> org.jclouds.azure.storage.AzureStorageResponseException: command
> [method=org.jclouds.azureblob.AzureBlobClient.public abstract void
> org.jclouds.azureblob.AzureBlobClient.putBlock(java.lang.String,java.lang.String,java.lang.String,org.jclouds.io.Payload)[azure-auto-123,
> sub2/100MBFile, AAV+QA==, [content=true, contentMetadata=[cacheControl=null,
> contentDisposition=null, contentEncoding=null, contentLanguage=null,
> contentLength=1048576, contentMD5=null, contentType=application/unknown,
> expires=null], written=false, isSensitive=false]], request=PUT
> https://casbazure.blob.core.windows.net/azure-auto-123/sub2/100MBFile?comp=block=AAV%2BQA%3D%3D
> HTTP/1.1] failed with code 400, error:
> AzureError{requestId='5b9a2f41-301e-008d-4e5b-97e57d00',
> code='InvalidQueryParameterValue', message='Value for one of the query
> parameters specified in the request URI is invalid.
> RequestId:5b9a2f41-301e-008d-4e5b-97e57d00
> Time:2022-07-14T08:24:41.1097266Z', context='{QueryParameterValue=AAV QA==,
> QueryParameterName=blockid, Reason=Not a valid base64 string.}'} {code}
> As you can see, the *blockid* becomes "AAV QA==" which is a invalid base64
> string.
>
> *My Observation*
> I checked the failed parts and they all have "+" in their *blockid.*
> For example, from part 1 to part 500, the following parts have "+" in their
> blockid :
>
> ||Part Number||blockid||
> |248|+A==|
> |249|+Q==|
> |250|+g==|
> |251|+w==|
>
> I used the same code in
> [here|https://github.com/apache/jclouds/blob/0b89ee0825d45de1193090cdd5efc5f1135fa200/providers/azureblob/src/main/java/org/jclouds/azureblob/blobstore/AzureBlobStore.java#L444-L450]
> to generate *blockid*
> {code:java}
> String blockId = BaseEncoding.base64().encode(Ints.toByteArray(partNumber));
> {code}
>
> And my version of *guava* is 27.1-jre which is used in jclouds 2.5.0
> {code:java}
>
> com.google.guava
> guava
> 27.1-jre
> {code}
>
> By the way, it works fine with *Azure account name* and {*}account key{*}.
> I can successfully upload all parts.
> (I think the *blockid* still has "+" but I don't know why it works with
> *Azure account name* and {*}account key{*})
>
> Please help me to fix this problem. Thank you!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (JCLOUDS-1615) Using Azure SAS Token unable to upload some parts of the file (blockid is "Not a valid base64 string")
[
https://issues.apache.org/jira/browse/JCLOUDS-1615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17603570#comment-17603570
]
Andrew Gaul commented on JCLOUDS-1615:
--
Pull requests welcome, particularly those with additional tests.
> Using Azure SAS Token unable to upload some parts of the file (blockid is
> "Not a valid base64 string")
> --
>
> Key: JCLOUDS-1615
> URL: https://issues.apache.org/jira/browse/JCLOUDS-1615
> Project: jclouds
> Issue Type: Bug
> Components: jclouds-blobstore
>Affects Versions: 2.5.0
>Reporter: Amber Liu
>Assignee: Andrew Gaul
>Priority: Major
> Labels: azureblob
>
> I'm trying to [put
> block|https://docs.microsoft.com/en-us/rest/api/storageservices/put-block#request]
> with Azure SAS Token.
> However, it failed in some "specific" parts.
> Here is the error message:
> {code:java}
> org.jclouds.azure.storage.AzureStorageResponseException: command
> [method=org.jclouds.azureblob.AzureBlobClient.public abstract void
> org.jclouds.azureblob.AzureBlobClient.putBlock(java.lang.String,java.lang.String,java.lang.String,org.jclouds.io.Payload)[azure-auto-123,
> sub2/100MBFile, AAV+QA==, [content=true, contentMetadata=[cacheControl=null,
> contentDisposition=null, contentEncoding=null, contentLanguage=null,
> contentLength=1048576, contentMD5=null, contentType=application/unknown,
> expires=null], written=false, isSensitive=false]], request=PUT
> https://casbazure.blob.core.windows.net/azure-auto-123/sub2/100MBFile?comp=block=AAV%2BQA%3D%3D
> HTTP/1.1] failed with code 400, error:
> AzureError{requestId='5b9a2f41-301e-008d-4e5b-97e57d00',
> code='InvalidQueryParameterValue', message='Value for one of the query
> parameters specified in the request URI is invalid.
> RequestId:5b9a2f41-301e-008d-4e5b-97e57d00
> Time:2022-07-14T08:24:41.1097266Z', context='{QueryParameterValue=AAV QA==,
> QueryParameterName=blockid, Reason=Not a valid base64 string.}'} {code}
> As you can see, the *blockid* becomes "AAV QA==" which is a invalid base64
> string.
>
> *My Observation*
> I checked the failed parts and they all have "+" in their *blockid.*
> For example, from part 1 to part 500, the following parts have "+" in their
> blockid :
>
> ||Part Number||blockid||
> |248|+A==|
> |249|+Q==|
> |250|+g==|
> |251|+w==|
>
> I used the same code in
> [here|https://github.com/apache/jclouds/blob/0b89ee0825d45de1193090cdd5efc5f1135fa200/providers/azureblob/src/main/java/org/jclouds/azureblob/blobstore/AzureBlobStore.java#L444-L450]
> to generate *blockid*
> {code:java}
> String blockId = BaseEncoding.base64().encode(Ints.toByteArray(partNumber));
> {code}
>
> And my version of *guava* is 27.1-jre which is used in jclouds 2.5.0
> {code:java}
>
> com.google.guava
> guava
> 27.1-jre
> {code}
>
> By the way, it works fine with *Azure account name* and {*}account key{*}.
> I can successfully upload all parts.
> (I think the *blockid* still has "+" but I don't know why it works with
> *Azure account name* and {*}account key{*})
>
> Please help me to fix this problem. Thank you!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (JCLOUDS-1615) Using Azure SAS Token unable to upload some parts of the file (blockid is "Not a valid base64 string")
[
https://issues.apache.org/jira/browse/JCLOUDS-1615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17603567#comment-17603567
]
Christian Jung commented on JCLOUDS-1615:
-
I also encountered this issue.
Maybe a simple solution would be to use {{BaseEncoding.base64Url()}} in [the
line encoding the
blockid|https://github.com/apache/jclouds/blob/57a9e7b7ccb900a8d0d8808098fd5beaf75d6332/providers/azureblob/src/main/java/org/jclouds/azureblob/blobstore/AzureBlobStore.java#L437].
The same code is also duplicated in
[{{uploadMultipartPart()}}|https://github.com/apache/jclouds/blob/57a9e7b7ccb900a8d0d8808098fd5beaf75d6332/providers/azureblob/src/main/java/org/jclouds/azureblob/blobstore/AzureBlobStore.java#L445].
> Using Azure SAS Token unable to upload some parts of the file (blockid is
> "Not a valid base64 string")
> --
>
> Key: JCLOUDS-1615
> URL: https://issues.apache.org/jira/browse/JCLOUDS-1615
> Project: jclouds
> Issue Type: Bug
> Components: jclouds-blobstore
>Affects Versions: 2.5.0
>Reporter: Amber Liu
>Assignee: Andrew Gaul
>Priority: Major
> Labels: azureblob
>
> I'm trying to [put
> block|https://docs.microsoft.com/en-us/rest/api/storageservices/put-block#request]
> with Azure SAS Token.
> However, it failed in some "specific" parts.
> Here is the error message:
> {code:java}
> org.jclouds.azure.storage.AzureStorageResponseException: command
> [method=org.jclouds.azureblob.AzureBlobClient.public abstract void
> org.jclouds.azureblob.AzureBlobClient.putBlock(java.lang.String,java.lang.String,java.lang.String,org.jclouds.io.Payload)[azure-auto-123,
> sub2/100MBFile, AAV+QA==, [content=true, contentMetadata=[cacheControl=null,
> contentDisposition=null, contentEncoding=null, contentLanguage=null,
> contentLength=1048576, contentMD5=null, contentType=application/unknown,
> expires=null], written=false, isSensitive=false]], request=PUT
> https://casbazure.blob.core.windows.net/azure-auto-123/sub2/100MBFile?comp=block=AAV%2BQA%3D%3D
> HTTP/1.1] failed with code 400, error:
> AzureError{requestId='5b9a2f41-301e-008d-4e5b-97e57d00',
> code='InvalidQueryParameterValue', message='Value for one of the query
> parameters specified in the request URI is invalid.
> RequestId:5b9a2f41-301e-008d-4e5b-97e57d00
> Time:2022-07-14T08:24:41.1097266Z', context='{QueryParameterValue=AAV QA==,
> QueryParameterName=blockid, Reason=Not a valid base64 string.}'} {code}
> As you can see, the *blockid* becomes "AAV QA==" which is a invalid base64
> string.
>
> *My Observation*
> I checked the failed parts and they all have "+" in their *blockid.*
> For example, from part 1 to part 500, the following parts have "+" in their
> blockid :
>
> ||Part Number||blockid||
> |248|+A==|
> |249|+Q==|
> |250|+g==|
> |251|+w==|
>
> I used the same code in
> [here|https://github.com/apache/jclouds/blob/0b89ee0825d45de1193090cdd5efc5f1135fa200/providers/azureblob/src/main/java/org/jclouds/azureblob/blobstore/AzureBlobStore.java#L444-L450]
> to generate *blockid*
> {code:java}
> String blockId = BaseEncoding.base64().encode(Ints.toByteArray(partNumber));
> {code}
>
> And my version of *guava* is 27.1-jre which is used in jclouds 2.5.0
> {code:java}
>
> com.google.guava
> guava
> 27.1-jre
> {code}
>
> By the way, it works fine with *Azure account name* and {*}account key{*}.
> I can successfully upload all parts.
> (I think the *blockid* still has "+" but I don't know why it works with
> *Azure account name* and {*}account key{*})
>
> Please help me to fix this problem. Thank you!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
