[GitHub] incubator-beam pull request #1501: [BEAM-1084] Update Apitools to version 0....
Github user sb2nov closed the pull request at: https://github.com/apache/incubator-beam/pull/1501 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1617: [BEAM-964] json avro flag migration
GitHub user sb2nov opened a pull request: https://github.com/apache/incubator-beam/pull/1617 [BEAM-964] json avro flag migration Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [x] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [x] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- R: @chamikaramj PTAL You can merge this pull request into a Git repository by running: $ git pull https://github.com/sb2nov/incubator-beam BEAM-964-json-avro-flag-migration Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1617.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1617 commit 0a558c7171d6e4452d88ecffd16a024a19cbfc42 Author: Sourabh Bajaj Date: 2016-12-14T19:44:46Z Update the BQ export flat from Json to Avro --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1485: [BEAM-1087] Fix inconsistent pickle from ...
Github user sb2nov closed the pull request at: https://github.com/apache/incubator-beam/pull/1485 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1514: Rollback for release
GitHub user sb2nov opened a pull request: https://github.com/apache/incubator-beam/pull/1514 Rollback for release Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [x] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- R: @chamikaramj @robertwb PTAL You can merge this pull request into a Git repository by running: $ git pull https://github.com/sb2nov/incubator-beam BEAM_rollback Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1514.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1514 commit 4cb59130d87cf1c98c552f813b69ee717b1bd11a Author: Sourabh Bajaj Date: 2016-12-06T01:53:19Z Rollback for release --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1510: [BEAM-964] Change export format to AVRO f...
GitHub user sb2nov opened a pull request: https://github.com/apache/incubator-beam/pull/1510 [BEAM-964] Change export format to AVRO for BQ Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [x] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- R: @chamikaramj PTAL You can merge this pull request into a Git repository by running: $ git pull https://github.com/sb2nov/incubator-beam BEAM-json-avro-bq-dataflow Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1510.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1510 commit 390fbfddf444df410f7cb61329496f6f24a0532c Author: Sourabh Bajaj Date: 2016-12-05T22:04:54Z Change export format to AVRO for BQ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1501: [BEAM-1084] Update Apitools to version 0....
GitHub user sb2nov opened a pull request: https://github.com/apache/incubator-beam/pull/1501 [BEAM-1084] Update Apitools to version 0.5.6 Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [x] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- R: @chamikaramj You can merge this pull request into a Git repository by running: $ git pull https://github.com/sb2nov/incubator-beam BEAM-bump-apitools-version Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1501.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1501 commit c576d3c769df2b89f0aaadbe96a2197825838813 Author: Sourabh Bajaj Date: 2016-12-03T17:02:32Z Update Apitools to version 0.5.6 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1499: [BEAM-1082] Clean up all internal referen...
GitHub user sb2nov opened a pull request: https://github.com/apache/incubator-beam/pull/1499 [BEAM-1082] Clean up all internal references to use_legacy_sql Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [x] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- R: @chamikaramj PTAL You can merge this pull request into a Git repository by running: $ git pull https://github.com/sb2nov/incubator-beam BEAM-consistent-flag-bw-java-py-2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1499.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1499 commit 53ffed690ad316fb79942f9801c2a375e1a8b354 Author: Sourabh Bajaj Date: 2016-12-03T16:36:44Z Clean up all internal references to use_legacy_sql --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1497: [BEAM-1082] Make the legacy SQL flag cons...
Github user sb2nov closed the pull request at: https://github.com/apache/incubator-beam/pull/1497 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1497: Make the legacy SQL flag consistent betwe...
GitHub user sb2nov opened a pull request: https://github.com/apache/incubator-beam/pull/1497 Make the legacy SQL flag consistent between Java and Python Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- R: @chamikaramj @aaltay PTAL You can merge this pull request into a Git repository by running: $ git pull https://github.com/sb2nov/incubator-beam BEAM-consistent-flag-bw-java-py Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1497.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1497 commit cfc0ba009316671c883b1f383395d362b2dee796 Author: Sourabh Bajaj Date: 2016-12-03T00:45:19Z Make the legacy SQL flag consistent between Java and Python --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1431: [BEAM-1079] Improve validation speed for ...
Github user sb2nov closed the pull request at: https://github.com/apache/incubator-beam/pull/1431 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1485: WIP - Beam test main session pickle
GitHub user sb2nov opened a pull request: https://github.com/apache/incubator-beam/pull/1485 WIP - Beam test main session pickle Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- DO NOT LAND - WIP You can merge this pull request into a Git repository by running: $ git pull https://github.com/sb2nov/incubator-beam BEAM-test-main-session-pickle Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1485.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1485 commit b0334e47ac01b63761105ce634bce742ac8141a2 Author: Sourabh Bajaj Date: 2016-12-01T22:48:17Z Test commit for dill issue commit ae89d57393942b208cc45bda43ccf34a0d7cb343 Author: Sourabh Bajaj Date: 2016-12-01T22:49:09Z T2 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1472: Add snippet for standard sql
Github user sb2nov closed the pull request at: https://github.com/apache/incubator-beam/pull/1472 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1468: [BEAM] Add table schema from json parser
Github user sb2nov closed the pull request at: https://github.com/apache/incubator-beam/pull/1468 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1467: Improve size estimation speed for file sa...
Github user sb2nov closed the pull request at: https://github.com/apache/incubator-beam/pull/1467 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1472: Add snippet for standard sql
GitHub user sb2nov opened a pull request: https://github.com/apache/incubator-beam/pull/1472 Add snippet for standard sql Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- R: @hadarhg PTAL You can merge this pull request into a Git repository by running: $ git pull https://github.com/sb2nov/incubator-beam BEAM-update-bq-snippets Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1472.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1472 commit abe0c5785823cef21b5bfc3935ca4ab2c5827a5a Author: Sourabh Bajaj Date: 2016-11-30T22:59:15Z Add snippet for standard sql --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1468: [BEAM] Add table schema from json parser
GitHub user sb2nov opened a pull request: https://github.com/apache/incubator-beam/pull/1468 [BEAM] Add table schema from json parser Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- R: @chamikaramj PTAL You can merge this pull request into a Git repository by running: $ git pull https://github.com/sb2nov/incubator-beam BEAM-add-table-schema-from-json-parser Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1468.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1468 commit 8620095eb2024a52427249386b412e318d676e44 Author: Sourabh Bajaj Date: 2016-11-30T21:48:28Z Parse table schema from JSON --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1467: Improve size estimation speed for file sa...
GitHub user sb2nov opened a pull request: https://github.com/apache/incubator-beam/pull/1467 Improve size estimation speed for file samples Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- R: @chamikaramj PTAL You can merge this pull request into a Git repository by running: $ git pull https://github.com/sb2nov/incubator-beam BEAM-remove-full-size-reading-sample Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1467.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1467 commit f1cbcee35170c2f93e606a3da0c42e145085f308 Author: Sourabh Bajaj Date: 2016-11-30T20:18:04Z Improve size estimation speed for file samples --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1421: Improve the GCS stat using batchAPI
Github user sb2nov closed the pull request at: https://github.com/apache/incubator-beam/pull/1421 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1404: Improve the speed of getting file sizes u...
Github user sb2nov closed the pull request at: https://github.com/apache/incubator-beam/pull/1404 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1431: Improve validation speed for large number...
GitHub user sb2nov opened a pull request: https://github.com/apache/incubator-beam/pull/1431 Improve validation speed for large number of files Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- Do not need to list all files in GCS for validation. Add limit field to fileIO You can merge this pull request into a Git repository by running: $ git pull https://github.com/sb2nov/incubator-beam BEAM-improve-validation-speed Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1431.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1431 commit fd713d46af0b2230d8becc0ff7a0c4fd3f727f62 Author: Sourabh Bajaj Date: 2016-11-23T22:49:51Z Do not need to list all files in GCS for validation. Add limit field to fileIO --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1421: Improve the GCS stat using batchAPI
GitHub user sb2nov opened a pull request: https://github.com/apache/incubator-beam/pull/1421 Improve the GCS stat using batchAPI Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/sb2nov/incubator-beam BEAM-stat-files-using-batch-api Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1421.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1421 commit 6c59b7311946d56cac08dfc156bbeea2c254f021 Author: Sourabh Bajaj Date: 2016-11-22T22:36:33Z Improve the GCS stat using batchAPI --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1404: Improve the speed of getting file sizes
GitHub user sb2nov opened a pull request: https://github.com/apache/incubator-beam/pull/1404 Improve the speed of getting file sizes Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [x] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- R: @chamikaramj PTAL You can merge this pull request into a Git repository by running: $ git pull https://github.com/sb2nov/incubator-beam BEAM-stat-files-using-list-api Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1404.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1404 commit 972391ac1d23a0879d1d17383cca0b7eda23d26b Author: Sourabh Bajaj Date: 2016-11-21T23:50:21Z Improve the speed of getting file sizes --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1396: Add fields retry
Github user sb2nov closed the pull request at: https://github.com/apache/incubator-beam/pull/1396 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1396: Add fields retry
GitHub user sb2nov opened a pull request: https://github.com/apache/incubator-beam/pull/1396 Add fields retry Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [x] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- Allow changing all the fields from FuzzedExponentialIntervals in the retry decorator R: @chamikaramj PTAL You can merge this pull request into a Git repository by running: $ git pull https://github.com/sb2nov/incubator-beam BEAM-add-fields-retry Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1396.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1396 commit 68dc37be6dc0f59f6c36a9bb52025d921fb01229 Author: Sourabh Bajaj Date: 2016-11-21T19:28:56Z Add missing fields to the retry decorator --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1242: [BEAM-873] Support For Biq Query 2.0 Sql
Github user sb2nov closed the pull request at: https://github.com/apache/incubator-beam/pull/1242 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1220: [BEAM-852] Add validation to file based s...
Github user sb2nov closed the pull request at: https://github.com/apache/incubator-beam/pull/1220 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1250: [BEAM-875] Consistency of time types acro...
Github user sb2nov closed the pull request at: https://github.com/apache/incubator-beam/pull/1250 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1304: [BEAM-964] Pass export format flag for BQ
Github user sb2nov closed the pull request at: https://github.com/apache/incubator-beam/pull/1304 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1365: [BEAM-985] Fix shared state across retry ...
Github user sb2nov closed the pull request at: https://github.com/apache/incubator-beam/pull/1365 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1365: Fix shared state across retry decorated f...
GitHub user sb2nov opened a pull request: https://github.com/apache/incubator-beam/pull/1365 Fix shared state across retry decorated functions Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [x] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [x] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- The fix was fairly straight forward and just needed to move the generator inside the wrapped function. R: @aaltay @chamikaramj PTAL You can merge this pull request into a Git repository by running: $ git pull https://github.com/sb2nov/incubator-beam BEAM-985-fix-shared-state-in-retries Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1365.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1365 commit 5f865234dfbad47ae19a86959f92ccae7789c540 Author: Sourabh Bajaj Date: 2016-11-15T23:04:37Z Fix shared state across retry decorated functions --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1304: Pass export format flag for BQ
GitHub user sb2nov opened a pull request: https://github.com/apache/incubator-beam/pull/1304 Pass export format flag for BQ Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [x] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- Pass the export format flag so that we can change formats in the future. R: @chamikaramj PTAL You can merge this pull request into a Git repository by running: $ git pull https://github.com/sb2nov/incubator-beam sb_big_query_avro Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1304.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1304 commit 139e874ab02915d1451c7775f1e7e958ba27c65c Author: Sourabh Bajaj Date: 2016-11-08T02:15:17Z Allow for passing format so that we can migrate to BQ Avro export later --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1268: Allow an option to not flatten record typ...
GitHub user sb2nov opened a pull request: https://github.com/apache/incubator-beam/pull/1268 Allow an option to not flatten record types in BQ Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [x] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [x] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- - Add a flag to BQ source so that we can allow for record type to not be flattened - Change the BQReader so that DirectRunner parses the dictionaries correctly - Add tests to make sure that dict parsing is working correctly Future work: - This has a lot of conflicts with #1242 so will be updated after that has been merged into master. - Write an integration test so that can check the cleanup of temporary datasets correctly, for now verified this manually R: @chamikaramj PTAL You can merge this pull request into a Git repository by running: $ git pull https://github.com/sb2nov/incubator-beam BEAM-877-disallow-record-flattening Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1268.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1268 commit 73cab2dceace94f41f8a78494bc0ba1615ba7a16 Author: Sourabh Bajaj Date: 2016-11-03T00:42:06Z Allow an option to not flatten records in BQ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1250: [BEAM-875] Make the BQ input for dataflow...
GitHub user sb2nov opened a pull request: https://github.com/apache/incubator-beam/pull/1250 [BEAM-875] Make the BQ input for dataflow runner and local runner is identical Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [x] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [x] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- Time related datatypes were not all supported for the local runner as well as timestamps were being serialized different so changing the local runner to match the big query export environment now. R: @chamikaramj You can merge this pull request into a Git repository by running: $ git pull https://github.com/sb2nov/incubator-beam BEAM-875-datetime-inconsistency Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1250.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1250 commit 28ab78d87393d7b7d57980e2dfdccdc366d213b2 Author: Sourabh Bajaj Date: 2016-11-01T20:06:10Z Make the BQ input for dataflow runner and local runner is identical for time related types --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1242: Temp
GitHub user sb2nov opened a pull request: https://github.com/apache/incubator-beam/pull/1242 Temp Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [x] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [x] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- Added support for passing a flag that allows the user to toggle between the two SQL formats supported by BQ. We set the default to True to keep backward compatibility here. Tested by running queries in both the formats. R: @chamikaramj You can merge this pull request into a Git repository by running: $ git pull https://github.com/sb2nov/incubator-beam BEAM-873-support_for_biq_query_2_sql Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1242.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1242 commit 43e3a8a7057bcd7c635114c4b3d6e5548cc70687 Author: Sourabh Bajaj Date: 2016-10-28T20:59:23Z Temp --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1220: [BEAM-852] Add validation to file based s...
GitHub user sb2nov opened a pull request: https://github.com/apache/incubator-beam/pull/1220 [BEAM-852] Add validation to file based sources during create time Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [x] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [x] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- Adding validation to the fileSourceIO so that we can inform the user if the files don't exist at the create time. There is a flag to bypass verification incase your local system doesn't have the same access as the runner etc. PTAL @chamikaramj You can merge this pull request into a Git repository by running: $ git pull https://github.com/sb2nov/incubator-beam BEAM-852-validate_file_source_at_create Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1220.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1220 commit 5e866687c3ef8f3e67838e868a3deaeb9555e460 Author: Sourabh Bajaj Date: 2016-10-28T17:18:51Z [BEAM-852] Add validation to file based sources during create time --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
