It's good for you, Ryan, because there are many alternatives.
FYI, Apache Spark 3.0.0 is using Apache Hive 2.3.7.
And, everything is running in a local mode on the single container.
> Would the entire thing run within the same container and then I leverage
the Spark APIs from that in local mode?
dongjoon-hyun commented on pull request #519:
URL: https://github.com/apache/orc/pull/519#issuecomment-659162096
Hi, @omalley . Could you review this PR when you have some time?
This is an automated message from the Apache Gi
Dongjoon Hyun created ORC-647:
-
Summary: Add macOS 10.15 test to Travis CI
Key: ORC-647
URL: https://issues.apache.org/jira/browse/ORC-647
Project: ORC
Issue Type: Test
Components: buil
dongjoon-hyun opened a new pull request #519:
URL: https://github.com/apache/orc/pull/519
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
Ryan,
It's possible there are some changes that would cause that code not to
compile for Hive 2, but I have done some work with porting similar
processors to Hive 2 and as I recall it was mostly API-type breaking
changes and not so much from the behavior side of things, more of a
Maven and Java-pa
Great, thanks Matt! Looking at this code now and feel this will really help
me a lot. Anything you think would break using this logic for Hive 2.3.5?
On Wed, Jul 15, 2020 at 5:04 PM Matt Burgess wrote:
> Ryan,
>
> In Apache NiFi we have a ConvertAvroToOrc processor [1], you may find
> code there
Ryan,
In Apache NiFi we have a ConvertAvroToOrc processor [1], you may find
code there that you can use in your Java program (take a look at line
212 and down). We had to create our own OrcFileWriter because the one
in Apache ORC writes to a FileSystem where we needed to write to our
own FlowFile
Great response, thanks for the info. Question on the Spark approach, as I
have been thinking about this, but don't have a lot of knowledge in the
area.
Let's say I develop my application, let's call it AvroToORC.java which has
a dependency on Spark. Would the entire thing run within the same conta
Hi Ryan.
If you need to build from the scratch, you may want to see a standalone
converter example in Apache ORC repository.
-
https://github.com/apache/orc/blob/master/java/tools/src/java/org/apache/orc/tools/convert/ConvertTool.java
Although it doesn't support Avro, there are CsvReader and
dongjoon-hyun edited a comment on pull request #517:
URL: https://github.com/apache/orc/pull/517#issuecomment-658275200
@nikitamikhaylov . Apache ORC has a dockerized testing environment like the
following. When I ran this PR on clang9 on Ubuntu 20.04, this didn't pass the
test. So, I'm wo
I'm writing a standalone Java process and interested in converting the
consumed Avro messages to ORC. I've seen a plethora of examples of writing
to ORC, but the conversion to ORC from Avro is what I can't seem to find a
lot of examples of.
This is just a standard Java process running inside of a
11 matches
Mail list logo