Re: [PR] Pass original message down through conversion for storage write api [beam]

reuvenlax commented on PR #31106: URL: https://github.com/apache/beam/pull/31106#issuecomment-2123223998 If I understand this correctly, we are now propagating both ElementT and StorageApiWritePayload - correct? Doesn't this double the amount of data being processed? -- This is an

Re: [PR] Pass original message down through conversion for storage write api [beam]

johnjcasey commented on code in PR #31106: URL: https://github.com/apache/beam/pull/31106#discussion_r1608769452 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiLoads.java: ## @@ -52,16 +52,18 @@ /** This {@link PTransform}

Re: [PR] Pass original message down through conversion for storage write api [beam]

johnjcasey commented on code in PR #31106: URL: https://github.com/apache/beam/pull/31106#discussion_r1608768600 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiLoads.java: ## @@ -52,16 +52,18 @@ /** This {@link PTransform}

Re: [PR] Pass original message down through conversion for storage write api [beam]

johnjcasey commented on code in PR #31106: URL: https://github.com/apache/beam/pull/31106#discussion_r1608768007 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiLoads.java: ## @@ -52,16 +52,18 @@ /** This {@link PTransform}

Re: [PR] Pass original message down through conversion for storage write api [beam]

scwhittle commented on code in PR #31106: URL: https://github.com/apache/beam/pull/31106#discussion_r1607961310 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiLoads.java: ## @@ -52,16 +52,18 @@ /** This {@link PTransform}

Re: [PR] Pass original message down through conversion for storage write api [beam]

johnjcasey commented on code in PR #31106: URL: https://github.com/apache/beam/pull/31106#discussion_r1606900416 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiLoads.java: ## @@ -52,16 +52,18 @@ /** This {@link PTransform}

Re: [PR] Pass original message down through conversion for storage write api [beam]

kennknowles commented on code in PR #31106: URL: https://github.com/apache/beam/pull/31106#discussion_r1605439707 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiLoads.java: ## @@ -52,16 +52,18 @@ /** This {@link PTransform}

Re: [PR] Pass original message down through conversion for storage write api [beam]

chamikaramj commented on code in PR #31106: URL: https://github.com/apache/beam/pull/31106#discussion_r1605250329 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiLoads.java: ## @@ -52,16 +52,18 @@ /** This {@link PTransform}

Re: [PR] Pass original message down through conversion for storage write api [beam]

chamikaramj commented on code in PR #31106: URL: https://github.com/apache/beam/pull/31106#discussion_r1604360288 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiLoads.java: ## @@ -52,16 +52,18 @@ /** This {@link PTransform}

Re: [PR] Pass original message down through conversion for storage write api [beam]

chamikaramj commented on code in PR #31106: URL: https://github.com/apache/beam/pull/31106#discussion_r1604360288 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiLoads.java: ## @@ -52,16 +52,18 @@ /** This {@link PTransform}

Re: [PR] Pass original message down through conversion for storage write api [beam]

ahmedabu98 commented on code in PR #31106: URL: https://github.com/apache/beam/pull/31106#discussion_r1602137894 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiLoads.java: ## @@ -52,16 +52,18 @@ /** This {@link PTransform}

Re: [PR] Pass original message down through conversion for storage write api [beam]

johnjcasey commented on code in PR #31106: URL: https://github.com/apache/beam/pull/31106#discussion_r1594510451 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiLoads.java: ## @@ -52,16 +52,18 @@ /** This {@link PTransform}

Re: [PR] Pass original message down through conversion for storage write api [beam]

chamikaramj commented on code in PR #31106: URL: https://github.com/apache/beam/pull/31106#discussion_r1594322815 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiLoads.java: ## @@ -52,16 +52,18 @@ /** This {@link PTransform}

Re: [PR] Pass original message down through conversion for storage write api [beam]

johnjcasey commented on code in PR #31106: URL: https://github.com/apache/beam/pull/31106#discussion_r1594308718 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiLoads.java: ## @@ -52,16 +52,18 @@ /** This {@link PTransform}

Re: [PR] Pass original message down through conversion for storage write api [beam]

johnjcasey commented on code in PR #31106: URL: https://github.com/apache/beam/pull/31106#discussion_r1594306136 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiLoads.java: ## @@ -52,16 +52,18 @@ /** This {@link PTransform}

Re: [PR] Pass original message down through conversion for storage write api [beam]

chamikaramj commented on code in PR #31106: URL: https://github.com/apache/beam/pull/31106#discussion_r1594280411 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiLoads.java: ## @@ -52,16 +52,18 @@ /** This {@link PTransform}

Re: [PR] Pass original message down through conversion for storage write api [beam]

johnjcasey commented on code in PR #31106: URL: https://github.com/apache/beam/pull/31106#discussion_r1594176649 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiLoads.java: ## @@ -52,16 +52,18 @@ /** This {@link PTransform}

Re: [PR] Pass original message down through conversion for storage write api [beam]

johnjcasey commented on code in PR #31106: URL: https://github.com/apache/beam/pull/31106#discussion_r1594162564 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiLoads.java: ## @@ -52,16 +52,18 @@ /** This {@link PTransform}

Re: [PR] Pass original message down through conversion for storage write api [beam]

johnjcasey commented on code in PR #31106: URL: https://github.com/apache/beam/pull/31106#discussion_r1594161262 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiLoads.java: ## @@ -52,16 +52,18 @@ /** This {@link PTransform}

Re: [PR] Pass original message down through conversion for storage write api [beam]

liferoad commented on code in PR #31106: URL: https://github.com/apache/beam/pull/31106#discussion_r1594068005 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiLoads.java: ## @@ -52,16 +52,18 @@ /** This {@link PTransform}

Re: [PR] Pass original message down through conversion for storage write api [beam]

johnjcasey commented on code in PR #31106: URL: https://github.com/apache/beam/pull/31106#discussion_r1593993849 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiLoads.java: ## @@ -52,16 +52,18 @@ /** This {@link PTransform}

Re: [PR] Pass original message down through conversion for storage write api [beam]

chamikaramj commented on code in PR #31106: URL: https://github.com/apache/beam/pull/31106#discussion_r1591462040 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiLoads.java: ## @@ -52,16 +52,18 @@ /** This {@link PTransform}

Re: [PR] Pass original message down through conversion for storage write api [beam]

johnjcasey commented on code in PR #31106: URL: https://github.com/apache/beam/pull/31106#discussion_r1591459919 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiLoads.java: ## @@ -52,16 +52,18 @@ /** This {@link PTransform}

Re: [PR] Pass original message down through conversion for storage write api [beam]

chamikaramj commented on code in PR #31106: URL: https://github.com/apache/beam/pull/31106#discussion_r1591260926 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiLoads.java: ## @@ -52,16 +52,18 @@ /** This {@link PTransform}

Re: [PR] Pass original message down through conversion for storage write api [beam]

ahmedabu98 commented on code in PR #31106: URL: https://github.com/apache/beam/pull/31106#discussion_r1591186536 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiConvertMessages.java: ## @@ -177,11 +189,16 @@ public void

Re: [PR] Pass original message down through conversion for storage write api [beam]

liferoad commented on code in PR #31106: URL: https://github.com/apache/beam/pull/31106#discussion_r1591008594 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiLoads.java: ## @@ -52,16 +52,18 @@ /** This {@link PTransform}

Re: [PR] Pass original message down through conversion for storage write api [beam]

johnjcasey commented on code in PR #31106: URL: https://github.com/apache/beam/pull/31106#discussion_r1590989576 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiLoads.java: ## @@ -52,16 +52,18 @@ /** This {@link PTransform}

Re: [PR] Pass original message down through conversion for storage write api [beam]

chamikaramj commented on code in PR #31106: URL: https://github.com/apache/beam/pull/31106#discussion_r1590610612 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiLoads.java: ## @@ -52,16 +52,18 @@ /** This {@link PTransform}

Re: [PR] Pass original message down through conversion for storage write api [beam]

johnjcasey commented on code in PR #31106: URL: https://github.com/apache/beam/pull/31106#discussion_r1587977083 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiLoads.java: ## @@ -52,16 +52,18 @@ /** This {@link PTransform}

Re: [PR] Pass original message down through conversion for storage write api [beam]

Abacn commented on PR #31106: URL: https://github.com/apache/beam/pull/31106#issuecomment-2090745715 Also going to run some load test to see if it has performance implications -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] Pass original message down through conversion for storage write api [beam]

Abacn commented on code in PR #31106: URL: https://github.com/apache/beam/pull/31106#discussion_r1587789379 ## sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiSinkFailedRowsIT.java: ## @@ -26,13 +26,15 @@ import

Re: [PR] Pass original message down through conversion for storage write api [beam]

johnjcasey commented on PR #31106: URL: https://github.com/apache/beam/pull/31106#issuecomment-2088916288 @Abacn @ahmedabu98 could you take a look at this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Pass original message down through conversion for storage write api [beam]

github-actions[bot] commented on PR #31106: URL: https://github.com/apache/beam/pull/31106#issuecomment-2079627506 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @robertwb for label java. R: @Abacn for label io.

Re: [PR] Pass original message down through conversion for storage write api [beam]

github-actions[bot] commented on PR #31106: URL: https://github.com/apache/beam/pull/31106#issuecomment-2077919954 Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers` -- This is an automated

[PR] Pass original message down through conversion for storage write api [beam]

johnjcasey opened a new pull request, #31106: URL: https://github.com/apache/beam/pull/31106 **Please** add a meaningful description for your change here Thank you for your contribution! Follow this checklist to help us incorporate your contribution