[jira] [Commented] (JCLOUDS-1615) Using Azure SAS Token unable to upload some parts of the file (blockid is "Not a valid base64 string")
[ https://issues.apache.org/jira/browse/JCLOUDS-1615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603567#comment-17603567 ] Christian Jung commented on JCLOUDS-1615: - I also encountered this issue. Maybe a simple solution would be to use {{BaseEncoding.base64Url()}} in [the line encoding the blockid|https://github.com/apache/jclouds/blob/57a9e7b7ccb900a8d0d8808098fd5beaf75d6332/providers/azureblob/src/main/java/org/jclouds/azureblob/blobstore/AzureBlobStore.java#L437]. The same code is also duplicated in [{{uploadMultipartPart()}}|https://github.com/apache/jclouds/blob/57a9e7b7ccb900a8d0d8808098fd5beaf75d6332/providers/azureblob/src/main/java/org/jclouds/azureblob/blobstore/AzureBlobStore.java#L445]. > Using Azure SAS Token unable to upload some parts of the file (blockid is > "Not a valid base64 string") > -- > > Key: JCLOUDS-1615 > URL: https://issues.apache.org/jira/browse/JCLOUDS-1615 > Project: jclouds > Issue Type: Bug > Components: jclouds-blobstore >Affects Versions: 2.5.0 >Reporter: Amber Liu >Assignee: Andrew Gaul >Priority: Major > Labels: azureblob > > I'm trying to [put > block|https://docs.microsoft.com/en-us/rest/api/storageservices/put-block#request] > with Azure SAS Token. > However, it failed in some "specific" parts. > Here is the error message: > {code:java} > org.jclouds.azure.storage.AzureStorageResponseException: command > [method=org.jclouds.azureblob.AzureBlobClient.public abstract void > org.jclouds.azureblob.AzureBlobClient.putBlock(java.lang.String,java.lang.String,java.lang.String,org.jclouds.io.Payload)[azure-auto-123, > sub2/100MBFile, AAV+QA==, [content=true, contentMetadata=[cacheControl=null, > contentDisposition=null, contentEncoding=null, contentLanguage=null, > contentLength=1048576, contentMD5=null, contentType=application/unknown, > expires=null], written=false, isSensitive=false]], request=PUT > https://casbazure.blob.core.windows.net/azure-auto-123/sub2/100MBFile?comp=block&blockid=AAV%2BQA%3D%3D > HTTP/1.1] failed with code 400, error: > AzureError{requestId='5b9a2f41-301e-008d-4e5b-97e57d00', > code='InvalidQueryParameterValue', message='Value for one of the query > parameters specified in the request URI is invalid. > RequestId:5b9a2f41-301e-008d-4e5b-97e57d00 > Time:2022-07-14T08:24:41.1097266Z', context='{QueryParameterValue=AAV QA==, > QueryParameterName=blockid, Reason=Not a valid base64 string.}'} {code} > As you can see, the *blockid* becomes "AAV QA==" which is a invalid base64 > string. > > *My Observation* > I checked the failed parts and they all have "+" in their *blockid.* > For example, from part 1 to part 500, the following parts have "+" in their > blockid : > > ||Part Number||blockid|| > |248|+A==| > |249|+Q==| > |250|+g==| > |251|+w==| > > I used the same code in > [here|https://github.com/apache/jclouds/blob/0b89ee0825d45de1193090cdd5efc5f1135fa200/providers/azureblob/src/main/java/org/jclouds/azureblob/blobstore/AzureBlobStore.java#L444-L450] > to generate *blockid* > {code:java} > String blockId = BaseEncoding.base64().encode(Ints.toByteArray(partNumber)); > {code} > > And my version of *guava* is 27.1-jre which is used in jclouds 2.5.0 > {code:java} > > com.google.guava > guava > 27.1-jre > {code} > > By the way, it works fine with *Azure account name* and {*}account key{*}. > I can successfully upload all parts. > (I think the *blockid* still has "+" but I don't know why it works with > *Azure account name* and {*}account key{*}) > > Please help me to fix this problem. Thank you! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (JCLOUDS-1615) Using Azure SAS Token unable to upload some parts of the file (blockid is "Not a valid base64 string")
[ https://issues.apache.org/jira/browse/JCLOUDS-1615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603570#comment-17603570 ] Andrew Gaul commented on JCLOUDS-1615: -- Pull requests welcome, particularly those with additional tests. > Using Azure SAS Token unable to upload some parts of the file (blockid is > "Not a valid base64 string") > -- > > Key: JCLOUDS-1615 > URL: https://issues.apache.org/jira/browse/JCLOUDS-1615 > Project: jclouds > Issue Type: Bug > Components: jclouds-blobstore >Affects Versions: 2.5.0 >Reporter: Amber Liu >Assignee: Andrew Gaul >Priority: Major > Labels: azureblob > > I'm trying to [put > block|https://docs.microsoft.com/en-us/rest/api/storageservices/put-block#request] > with Azure SAS Token. > However, it failed in some "specific" parts. > Here is the error message: > {code:java} > org.jclouds.azure.storage.AzureStorageResponseException: command > [method=org.jclouds.azureblob.AzureBlobClient.public abstract void > org.jclouds.azureblob.AzureBlobClient.putBlock(java.lang.String,java.lang.String,java.lang.String,org.jclouds.io.Payload)[azure-auto-123, > sub2/100MBFile, AAV+QA==, [content=true, contentMetadata=[cacheControl=null, > contentDisposition=null, contentEncoding=null, contentLanguage=null, > contentLength=1048576, contentMD5=null, contentType=application/unknown, > expires=null], written=false, isSensitive=false]], request=PUT > https://casbazure.blob.core.windows.net/azure-auto-123/sub2/100MBFile?comp=block&blockid=AAV%2BQA%3D%3D > HTTP/1.1] failed with code 400, error: > AzureError{requestId='5b9a2f41-301e-008d-4e5b-97e57d00', > code='InvalidQueryParameterValue', message='Value for one of the query > parameters specified in the request URI is invalid. > RequestId:5b9a2f41-301e-008d-4e5b-97e57d00 > Time:2022-07-14T08:24:41.1097266Z', context='{QueryParameterValue=AAV QA==, > QueryParameterName=blockid, Reason=Not a valid base64 string.}'} {code} > As you can see, the *blockid* becomes "AAV QA==" which is a invalid base64 > string. > > *My Observation* > I checked the failed parts and they all have "+" in their *blockid.* > For example, from part 1 to part 500, the following parts have "+" in their > blockid : > > ||Part Number||blockid|| > |248|+A==| > |249|+Q==| > |250|+g==| > |251|+w==| > > I used the same code in > [here|https://github.com/apache/jclouds/blob/0b89ee0825d45de1193090cdd5efc5f1135fa200/providers/azureblob/src/main/java/org/jclouds/azureblob/blobstore/AzureBlobStore.java#L444-L450] > to generate *blockid* > {code:java} > String blockId = BaseEncoding.base64().encode(Ints.toByteArray(partNumber)); > {code} > > And my version of *guava* is 27.1-jre which is used in jclouds 2.5.0 > {code:java} > > com.google.guava > guava > 27.1-jre > {code} > > By the way, it works fine with *Azure account name* and {*}account key{*}. > I can successfully upload all parts. > (I think the *blockid* still has "+" but I don't know why it works with > *Azure account name* and {*}account key{*}) > > Please help me to fix this problem. Thank you! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (JCLOUDS-1615) Using Azure SAS Token unable to upload some parts of the file (blockid is "Not a valid base64 string")
[ https://issues.apache.org/jira/browse/JCLOUDS-1615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603582#comment-17603582 ] Christian Jung commented on JCLOUDS-1615: - Surely there is already some test covering multipart. (Where?) So if this test just uploads a big payload, the current issue could be reproduced by simply increasing the payload size. Otherwise, i have no better proposal than extracting some method {code:java} static String makeBlockId(int partNumber) {code} and do unit tests on it. This would be better than just changing the code, but not much. > Using Azure SAS Token unable to upload some parts of the file (blockid is > "Not a valid base64 string") > -- > > Key: JCLOUDS-1615 > URL: https://issues.apache.org/jira/browse/JCLOUDS-1615 > Project: jclouds > Issue Type: Bug > Components: jclouds-blobstore >Affects Versions: 2.5.0 >Reporter: Amber Liu >Assignee: Andrew Gaul >Priority: Major > Labels: azureblob > > I'm trying to [put > block|https://docs.microsoft.com/en-us/rest/api/storageservices/put-block#request] > with Azure SAS Token. > However, it failed in some "specific" parts. > Here is the error message: > {code:java} > org.jclouds.azure.storage.AzureStorageResponseException: command > [method=org.jclouds.azureblob.AzureBlobClient.public abstract void > org.jclouds.azureblob.AzureBlobClient.putBlock(java.lang.String,java.lang.String,java.lang.String,org.jclouds.io.Payload)[azure-auto-123, > sub2/100MBFile, AAV+QA==, [content=true, contentMetadata=[cacheControl=null, > contentDisposition=null, contentEncoding=null, contentLanguage=null, > contentLength=1048576, contentMD5=null, contentType=application/unknown, > expires=null], written=false, isSensitive=false]], request=PUT > https://casbazure.blob.core.windows.net/azure-auto-123/sub2/100MBFile?comp=block&blockid=AAV%2BQA%3D%3D > HTTP/1.1] failed with code 400, error: > AzureError{requestId='5b9a2f41-301e-008d-4e5b-97e57d00', > code='InvalidQueryParameterValue', message='Value for one of the query > parameters specified in the request URI is invalid. > RequestId:5b9a2f41-301e-008d-4e5b-97e57d00 > Time:2022-07-14T08:24:41.1097266Z', context='{QueryParameterValue=AAV QA==, > QueryParameterName=blockid, Reason=Not a valid base64 string.}'} {code} > As you can see, the *blockid* becomes "AAV QA==" which is a invalid base64 > string. > > *My Observation* > I checked the failed parts and they all have "+" in their *blockid.* > For example, from part 1 to part 500, the following parts have "+" in their > blockid : > > ||Part Number||blockid|| > |248|+A==| > |249|+Q==| > |250|+g==| > |251|+w==| > > I used the same code in > [here|https://github.com/apache/jclouds/blob/0b89ee0825d45de1193090cdd5efc5f1135fa200/providers/azureblob/src/main/java/org/jclouds/azureblob/blobstore/AzureBlobStore.java#L444-L450] > to generate *blockid* > {code:java} > String blockId = BaseEncoding.base64().encode(Ints.toByteArray(partNumber)); > {code} > > And my version of *guava* is 27.1-jre which is used in jclouds 2.5.0 > {code:java} > > com.google.guava > guava > 27.1-jre > {code} > > By the way, it works fine with *Azure account name* and {*}account key{*}. > I can successfully upload all parts. > (I think the *blockid* still has "+" but I don't know why it works with > *Azure account name* and {*}account key{*}) > > Please help me to fix this problem. Thank you! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (JCLOUDS-1615) Using Azure SAS Token unable to upload some parts of the file (blockid is "Not a valid base64 string")
[ https://issues.apache.org/jira/browse/JCLOUDS-1615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603758#comment-17603758 ] Elvan commented on JCLOUDS-1615: [~gaul] , How do you retrieve the Azure SAS Token? > Using Azure SAS Token unable to upload some parts of the file (blockid is > "Not a valid base64 string") > -- > > Key: JCLOUDS-1615 > URL: https://issues.apache.org/jira/browse/JCLOUDS-1615 > Project: jclouds > Issue Type: Bug > Components: jclouds-blobstore >Affects Versions: 2.5.0 >Reporter: Amber Liu >Assignee: Andrew Gaul >Priority: Major > Labels: azureblob > > I'm trying to [put > block|https://docs.microsoft.com/en-us/rest/api/storageservices/put-block#request] > with Azure SAS Token. > However, it failed in some "specific" parts. > Here is the error message: > {code:java} > org.jclouds.azure.storage.AzureStorageResponseException: command > [method=org.jclouds.azureblob.AzureBlobClient.public abstract void > org.jclouds.azureblob.AzureBlobClient.putBlock(java.lang.String,java.lang.String,java.lang.String,org.jclouds.io.Payload)[azure-auto-123, > sub2/100MBFile, AAV+QA==, [content=true, contentMetadata=[cacheControl=null, > contentDisposition=null, contentEncoding=null, contentLanguage=null, > contentLength=1048576, contentMD5=null, contentType=application/unknown, > expires=null], written=false, isSensitive=false]], request=PUT > https://casbazure.blob.core.windows.net/azure-auto-123/sub2/100MBFile?comp=block&blockid=AAV%2BQA%3D%3D > HTTP/1.1] failed with code 400, error: > AzureError{requestId='5b9a2f41-301e-008d-4e5b-97e57d00', > code='InvalidQueryParameterValue', message='Value for one of the query > parameters specified in the request URI is invalid. > RequestId:5b9a2f41-301e-008d-4e5b-97e57d00 > Time:2022-07-14T08:24:41.1097266Z', context='{QueryParameterValue=AAV QA==, > QueryParameterName=blockid, Reason=Not a valid base64 string.}'} {code} > As you can see, the *blockid* becomes "AAV QA==" which is a invalid base64 > string. > > *My Observation* > I checked the failed parts and they all have "+" in their *blockid.* > For example, from part 1 to part 500, the following parts have "+" in their > blockid : > > ||Part Number||blockid|| > |248|+A==| > |249|+Q==| > |250|+g==| > |251|+w==| > > I used the same code in > [here|https://github.com/apache/jclouds/blob/0b89ee0825d45de1193090cdd5efc5f1135fa200/providers/azureblob/src/main/java/org/jclouds/azureblob/blobstore/AzureBlobStore.java#L444-L450] > to generate *blockid* > {code:java} > String blockId = BaseEncoding.base64().encode(Ints.toByteArray(partNumber)); > {code} > > And my version of *guava* is 27.1-jre which is used in jclouds 2.5.0 > {code:java} > > com.google.guava > guava > 27.1-jre > {code} > > By the way, it works fine with *Azure account name* and {*}account key{*}. > I can successfully upload all parts. > (I think the *blockid* still has "+" but I don't know why it works with > *Azure account name* and {*}account key{*}) > > Please help me to fix this problem. Thank you! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (JCLOUDS-1615) Using Azure SAS Token unable to upload some parts of the file (blockid is "Not a valid base64 string")
[ https://issues.apache.org/jira/browse/JCLOUDS-1615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17605198#comment-17605198 ] Christian Jung commented on JCLOUDS-1615: - I added a PR #155, but, as i have no experience with github, it might not be linked correctly to this issue. > Using Azure SAS Token unable to upload some parts of the file (blockid is > "Not a valid base64 string") > -- > > Key: JCLOUDS-1615 > URL: https://issues.apache.org/jira/browse/JCLOUDS-1615 > Project: jclouds > Issue Type: Bug > Components: jclouds-blobstore >Affects Versions: 2.5.0 >Reporter: Amber Liu >Assignee: Andrew Gaul >Priority: Major > Labels: azureblob > > I'm trying to [put > block|https://docs.microsoft.com/en-us/rest/api/storageservices/put-block#request] > with Azure SAS Token. > However, it failed in some "specific" parts. > Here is the error message: > {code:java} > org.jclouds.azure.storage.AzureStorageResponseException: command > [method=org.jclouds.azureblob.AzureBlobClient.public abstract void > org.jclouds.azureblob.AzureBlobClient.putBlock(java.lang.String,java.lang.String,java.lang.String,org.jclouds.io.Payload)[azure-auto-123, > sub2/100MBFile, AAV+QA==, [content=true, contentMetadata=[cacheControl=null, > contentDisposition=null, contentEncoding=null, contentLanguage=null, > contentLength=1048576, contentMD5=null, contentType=application/unknown, > expires=null], written=false, isSensitive=false]], request=PUT > https://casbazure.blob.core.windows.net/azure-auto-123/sub2/100MBFile?comp=block&blockid=AAV%2BQA%3D%3D > HTTP/1.1] failed with code 400, error: > AzureError{requestId='5b9a2f41-301e-008d-4e5b-97e57d00', > code='InvalidQueryParameterValue', message='Value for one of the query > parameters specified in the request URI is invalid. > RequestId:5b9a2f41-301e-008d-4e5b-97e57d00 > Time:2022-07-14T08:24:41.1097266Z', context='{QueryParameterValue=AAV QA==, > QueryParameterName=blockid, Reason=Not a valid base64 string.}'} {code} > As you can see, the *blockid* becomes "AAV QA==" which is a invalid base64 > string. > > *My Observation* > I checked the failed parts and they all have "+" in their *blockid.* > For example, from part 1 to part 500, the following parts have "+" in their > blockid : > > ||Part Number||blockid|| > |248|+A==| > |249|+Q==| > |250|+g==| > |251|+w==| > > I used the same code in > [here|https://github.com/apache/jclouds/blob/0b89ee0825d45de1193090cdd5efc5f1135fa200/providers/azureblob/src/main/java/org/jclouds/azureblob/blobstore/AzureBlobStore.java#L444-L450] > to generate *blockid* > {code:java} > String blockId = BaseEncoding.base64().encode(Ints.toByteArray(partNumber)); > {code} > > And my version of *guava* is 27.1-jre which is used in jclouds 2.5.0 > {code:java} > > com.google.guava > guava > 27.1-jre > {code} > > By the way, it works fine with *Azure account name* and {*}account key{*}. > I can successfully upload all parts. > (I think the *blockid* still has "+" but I don't know why it works with > *Azure account name* and {*}account key{*}) > > Please help me to fix this problem. Thank you! -- This message was sent by Atlassian Jira (v8.20.10#820010)