Github user chamikaramj closed the pull request at:
https://github.com/apache/incubator-beam/pull/1385
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
GitHub user chamikaramj opened a pull request:
https://github.com/apache/incubator-beam/pull/1385
Fixes couple of issues of FileBasedSource.
(1) Updates code so that a user-specified coder properly gets set to splits.
(2) Currently each SingleFileSource takes a reference
GitHub user chamikaramj opened a pull request:
https://github.com/apache/incubator-beam/pull/1267
Fixes two bugs in avroio_test 'test_corrupted_file'.
(1) Updates the test to perform corruption properly (setting 'A' and 'B').
(2) Removes an invalid usage of bytearray().
You can
Github user chamikaramj closed the pull request at:
https://github.com/apache/incubator-beam/pull/1235
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
GitHub user chamikaramj opened a pull request:
https://github.com/apache/incubator-beam/pull/1235
[BEAM-700] Improvements related to size estimation.
Updates FileBasedSource so that size estimation of glob patterns that
expand into a large number of files is done using sampling
Github user chamikaramj closed the pull request at:
https://github.com/apache/incubator-beam/pull/1058
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
GitHub user chamikaramj opened a pull request:
https://github.com/apache/incubator-beam/pull/1058
Fixes a bug in avroio_test.py
Fixes a bug in avroio_test.py where we open a binary file without 'b' mode.
Without this, file can get corrupted in Windows and the test becomes flaky
Github user chamikaramj closed the pull request at:
https://github.com/apache/incubator-beam/pull/881
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
GitHub user chamikaramj opened a pull request:
https://github.com/apache/incubator-beam/pull/1002
[BEAM-614] Updates FileBasedSource to support CompressionType.AUTO.
Updates FileBasedSource to support CompressionType.AUTO.
Fixes some tests that were not properly being
GitHub user chamikaramj opened a pull request:
https://github.com/apache/incubator-beam/pull/987
Adds __all__ tags to source modules.
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
- [ ] Make sure the PR title
Github user chamikaramj closed the pull request at:
https://github.com/apache/incubator-beam/pull/978
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user chamikaramj closed the pull request at:
https://github.com/apache/incubator-beam/pull/975
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
GitHub user chamikaramj opened a pull request:
https://github.com/apache/incubator-beam/pull/979
[BEAM-643] Updates lint configurations to ignore generated files.
Adds ability to ignore certain generated files when running pylint and pep8.
You can merge this pull request
GitHub user chamikaramj opened a pull request:
https://github.com/apache/incubator-beam/pull/978
[BEAM-643] Updates Dataflow API client.
Updates Cloud Dataflow API client files to the latest version.
You can merge this pull request into a Git repository by running:
$ git pull
GitHub user chamikaramj opened a pull request:
https://github.com/apache/incubator-beam/pull/975
[BEAM-643] Adds support for specifying a custom service account.
Adds support for specifying a custom service account when using
DataflowPipelineRunner.
Updates Dataflow API
Github user chamikaramj closed the pull request at:
https://github.com/apache/incubator-beam/pull/920
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
GitHub user chamikaramj opened a pull request:
https://github.com/apache/incubator-beam/pull/920
[BEAM-553] Adds a text source for Python SDK.
Current text source (fileio.TextFileSource) is specific to Dataflow runner.
This adds a runner independent TextSource that is based
GitHub user chamikaramj opened a pull request:
https://github.com/apache/incubator-beam/pull/890
Updates SourceTestBase concurrent splitting test to share thread pool
Updates SourceTestBase concurrent splitting test to share thread pool
across runs.
Without this, runs
GitHub user chamikaramj opened a pull request:
https://github.com/apache/incubator-beam/pull/881
Updates sources to report consumed and remaining number of split points.
Adds several methods to the RangeTracker interface to support this. Please
see comments for details
Github user chamikaramj closed the pull request at:
https://github.com/apache/incubator-beam/pull/866
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
GitHub user chamikaramj opened a pull request:
https://github.com/apache/incubator-beam/pull/779
[BEAM-522] Fixes GcsIO.exists() to properly handle files that do not exist
Currently this invocation fails for non existing files instead of returning
false.
Updates
GitHub user chamikaramj opened a pull request:
https://github.com/apache/incubator-beam/pull/765
[BEAM-502] Updates JSON to/from Python object conversion to handle
null/None values.
Updates Python object to JSON conversion to handle 'None' values.
Updates JSON to Python
GitHub user chamikaramj opened a pull request:
https://github.com/apache/incubator-beam/pull/763
[BEAM-499] Deletes some code that is not used by SDK.
Some code in apiclient.py is not used by Python SDK.
Deleting unused code and corresponding tests.
You can merge this pull
Github user chamikaramj closed the pull request at:
https://github.com/apache/incubator-beam/pull/672
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user chamikaramj closed the pull request at:
https://github.com/apache/incubator-beam/pull/667
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user chamikaramj closed the pull request at:
https://github.com/apache/incubator-beam/pull/670
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
GitHub user chamikaramj opened a pull request:
https://github.com/apache/incubator-beam/pull/672
[BEAM-360] Adds a PTransform for Avro source and updates snippets.
Wrapping a custom source as a 'PTransform' is better than directly using
the source using 'df.Read' since
GitHub user chamikaramj opened a pull request:
https://github.com/apache/incubator-beam/pull/670
Clarifies that 'TextFileSource' only supports UTF-8 and ASCII encodings.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com
GitHub user chamikaramj opened a pull request:
https://github.com/apache/incubator-beam/pull/667
[BEAM-455] Adds a test harnesses and utilities framework for sources.
Helper functions and test harnesses for checking correctness of source
(``iobase.BoundedSource``) and range tracker
Github user chamikaramj closed the pull request at:
https://github.com/apache/incubator-beam/pull/599
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
GitHub user chamikaramj reopened a pull request:
https://github.com/apache/incubator-beam/pull/599
[BEAM-360] Some updates related to dynamic work rebalancing of custom
sources.
Adds a class 'iobase.BoundedSourceSplit' to represent dynamic work
rebalancing results of custom
GitHub user chamikaramj opened a pull request:
https://github.com/apache/incubator-beam/pull/599
[BEAM-360] Some updates related to dynamic work rebalancing of custom
sources.
Adds a class 'iobase.BoundedSourceSplit' to represent dynamic work
rebalancing results of custom sources
GitHub user chamikaramj opened a pull request:
https://github.com/apache/incubator-beam/pull/565
[BEAM-393] Adds more code snippets for Python SDK
Adds code snippets related to following.
(1) Creating and using a new custom source
(2) Creating and using a new custom
Github user chamikaramj closed the pull request at:
https://github.com/apache/incubator-beam/pull/507
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
GitHub user chamikaramj opened a pull request:
https://github.com/apache/incubator-beam/pull/507
[BEAM-360] Implements a framework for developing Python SDK sources for new
file types
Module 'filebasedsource' provides a framework for creating sources for new
file types
35 matches
Mail list logo
