[GitHub] incubator-beam pull request #965: Removed unnecessary throttling of rename p...
Github user mdvorsky closed the pull request at: https://github.com/apache/incubator-beam/pull/965 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #752: Beam Python GCSIO write performance improv...
GitHub user mdvorsky opened a pull request: https://github.com/apache/incubator-beam/pull/752 Beam Python GCSIO write performance improvements. Increased the GCS buffer size from 1MB to 8MB and introduced a 128kB buffer for the pipe. Increasing the GCS buffer even more could improve the performance, but for the price of spending a lot of memory on buffers (which can be a problem if many GCS files are being written to concurrently). Increasing the pipe buffer further appears to have no benefit. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mdvorsky/incubator-beam fast_gcsio Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/752.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #752 commit a1f1fa06ee8683273182548e7eb2d6612040d2bf Author: Marian Dvorsky <mari...@google.com> Date: 2016-07-28T20:02:15Z Increased the GCS buffer size from 1MB to 8MB and introduced a 128kB buffer for the pipe. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #625: Made checksum_output optional in bigshuffl...
Github user mdvorsky closed the pull request at: https://github.com/apache/incubator-beam/pull/625 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #652: Added cy_combiners.py to the list of Cytho...
GitHub user mdvorsky opened a pull request: https://github.com/apache/incubator-beam/pull/652 Added cy_combiners.py to the list of Cythonized Python SDK files. R: @robertwb You can merge this pull request into a Git repository by running: $ git pull https://github.com/mdvorsky/incubator-beam cythonize Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/652.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #652 commit 1a8ef4fc3facca2a2545f6a20572118e8ba3d45a Author: Marian Dvorsky <mari...@google.com> Date: 2016-07-14T00:16:58Z Added cy_combiners.py to the list of Cythonized Python SDK files. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #625: Made checksum_output optional in bigshuffl...
GitHub user mdvorsky opened a pull request: https://github.com/apache/incubator-beam/pull/625 Made checksum_output optional in bigshuffle.py. Hi @chamikaramj, can you please take a look? You can merge this pull request into a Git repository by running: $ git pull https://github.com/mdvorsky/incubator-beam python-bigshuffle Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/625.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #625 commit 735185017380603a3c7c5a199c3fbeace3a3dc73 Author: Marian Dvorsky <mari...@google.com> Date: 2016-07-11T18:47:01Z Made checksum_output optional in bigshuffle.py. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
