[GitHub] orc pull request #108: Core Docs: Add a more advanced example
GitHub user Citrullin opened a pull request: https://github.com/apache/orc/pull/108 Core Docs: Add a more advanced example I added a more complex example to the docs. This example explains the handling of maps and also, indirectly, arrays. You can merge this pull request into a Git repository by running: $ git pull https://github.com/Citrullin/orc master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/orc/pull/108.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #108 commit c1083247a917f18289761bcf13f902555b5cbda8 Author: Philipp Blum Date: 2017-04-10T16:26:06Z fix wrong batch.size counter commit b4c75761115d3fc48f53ebed51bb64a6e8a204c8 Author: Philipp Blum Date: 2017-04-10T16:29:06Z Rename ListColumnVector to MapColumnVector commit 04dead08250fba0a80b438959b933eb61420c8d0 Author: Philipp Blum Date: 2017-04-13T23:58:00Z Add an advanced map example to core docs commit 602974b4d223f6408243d275d5f5a15a93355b20 Author: Philipp Blum Date: 2017-04-14T00:07:12Z Add batch writing at the end of the simple example commit 5633e8af37de9573b91bd17f77972fe7e4fb9c14 Author: Philipp Blum Date: 2017-04-14T00:11:17Z Remove wrong Simple Example batch++ commit cfc165a36e2df82968c544a246d509f1ad7d36f7 Author: Philipp Blum Date: 2017-04-14T06:40:29Z reset also the batchSizeCounter in the advanced example --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] orc pull request #107: Doc: An advanced map writing example
Github user Citrullin closed the pull request at: https://github.com/apache/orc/pull/107 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] orc pull request #107: Doc: An advanced map writing example
GitHub user Citrullin opened a pull request: https://github.com/apache/orc/pull/107 Doc: An advanced map writing example I added a more complex example to the docs. This example explains the handling of maps and also, indirectly, arrays. You can merge this pull request into a Git repository by running: $ git pull https://github.com/Citrullin/orc master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/orc/pull/107.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #107 commit c1083247a917f18289761bcf13f902555b5cbda8 Author: Philipp Blum Date: 2017-04-10T16:26:06Z fix wrong batch.size counter commit b4c75761115d3fc48f53ebed51bb64a6e8a204c8 Author: Philipp Blum Date: 2017-04-10T16:29:06Z Rename ListColumnVector to MapColumnVector commit 04dead08250fba0a80b438959b933eb61420c8d0 Author: Philipp Blum Date: 2017-04-13T23:58:00Z Add an advanced map example to core docs commit 602974b4d223f6408243d275d5f5a15a93355b20 Author: Philipp Blum Date: 2017-04-14T00:07:12Z Add batch writing at the end of the simple example commit 5633e8af37de9573b91bd17f77972fe7e4fb9c14 Author: Philipp Blum Date: 2017-04-14T00:11:17Z Remove wrong Simple Example batch++ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] orc issue #104: Core documentation fixes
Github user Citrullin commented on the issue: https://github.com/apache/orc/pull/104 Thanks @omalley, I forgot that is a post increment. So, yes, works fine :) I only have to do it a bit differently in scala. :) Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] orc pull request #104: Core documentation fixes
Github user Citrullin commented on a diff in the pull request:
https://github.com/apache/orc/pull/104#discussion_r111481491
--- Diff: site/_docs/core-java.md ---
@@ -233,14 +233,15 @@ VectorizedRowBatch batch = schema.createRowBatch();
LongColumnVector x = (LongColumnVector) batch.cols[0];
LongColumnVector y = (LongColumnVector) batch.cols[1];
for(int r=0; r < 1; ++r) {
- int row = batch.size++;
+ int row = batch.size;
x.vector[row] = r;
y.vector[row] = r * 3;
// If the batch is full, write it out and start over.
if (batch.size == batch.getMaxSize()) {
writer.addRowBatch(batch);
batch.reset();
}
+ batch.size++;
}
writer.close();
--- End diff --
That's correct. I already fixed it in my code. Had also this issue when the
batch smaller than the max size.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] orc pull request #104: Core documentation fixes
Github user Citrullin commented on a diff in the pull request:
https://github.com/apache/orc/pull/104#discussion_r111478051
--- Diff: site/_docs/core-java.md ---
@@ -233,14 +233,15 @@ VectorizedRowBatch batch = schema.createRowBatch();
LongColumnVector x = (LongColumnVector) batch.cols[0];
LongColumnVector y = (LongColumnVector) batch.cols[1];
for(int r=0; r < 1; ++r) {
- int row = batch.size++;
+ int row = batch.size;
x.vector[row] = r;
--- End diff --
Hi, I'm not that good in Java. I'm more familiar with Scala. But you take a
look into the Library I wrote. I created two branches where I changed only the
position of the up counting.
In example-1 I count both batch.size and rowBatchSize up before I add a row
to VectorizedBatch.
[See more
here](https://github.com/Citrullin/scalaOrcWriter/blob/ORC-168-wrong-example-1/src/main/scala/citrullin/orcwriter/OrcWriter.scala#L77)
In example 2 I count only batch.size up before I add the row to the batch.
rowBatchSize will up counted after a row is written.
[See more
here](https://github.com/Citrullin/scalaOrcWriter/blob/ORC-168-wrong-example-2/src/main/scala/citrullin/orcwriter/OrcWriter.scala#L77)
The working example is in the dev branch.
[More
here](https://github.com/Citrullin/scalaOrcWriter/blob/dev/src/main/scala/citrullin/orcwriter/OrcWriter.scala#L77)
You can run the Implicit ComplexMap example to see the differences.
[Here is the
source](https://github.com/Citrullin/scalaOrcWriter/blob/dev/src/main/scala/citrullin/orcwriter/examples/orcwriterimplicitapi/WriteComplexMap.scala)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] orc pull request #104: Core documentation fixes
GitHub user Citrullin opened a pull request: https://github.com/apache/orc/pull/104 Core documentation fixes There's an issue with the size count and the MapColumnVector is named ListColumnVector Also wrote a jira ticket for it: https://issues.apache.org/jira/browse/ORC-168 You can merge this pull request into a Git repository by running: $ git pull https://github.com/Citrullin/orc master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/orc/pull/104.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #104 commit c1083247a917f18289761bcf13f902555b5cbda8 Author: Philipp Blum Date: 2017-04-10T16:26:06Z fix wrong batch.size counter commit b4c75761115d3fc48f53ebed51bb64a6e8a204c8 Author: Philipp Blum Date: 2017-04-10T16:29:06Z Rename ListColumnVector to MapColumnVector --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
