Re: Companies using Apache Avro

[Vikas Saxena]

I have used Avro as the storage format for the Network probes data in my
current company  Optus (Telco) in Australia.
The probes data is in hexadecimal form and its decoded and stored in Avro
with snappy compression.
The solution was developed in 2014 and is being used till date.
We also used it to transfer files between various systems/platforms through
smtp
At that point of time, avro was a perfect choice because:
1) being a row based format its best for transactional accuracy
2) Great compression with snappy hence savings on storage.
3) Supported most of the complex datatypes such as structure, enums, arrays
etc so the structure was 100% aligned to the source but was in a human
readable form using hive (through AvroSerde)

Thanks,
Vikas

On Wed, Jan 27, 2021 at 8:27 PM Lee Hambley  wrote:

> Thanks for the extensive response! I think a lot of what you are saying
>> is very spot on!
>>
>
> I hope it's useful, if and when your paper is published, I'd love to give
> it a read.
>
>
>> I'm working on a paper surveying ~13 binary schema-less and
>> schema-driven serialization formats (including Avro) that can handle any
>> data structure that JSON can represent. Therefore, I was particularly
>> interested on why you wanted to convert JSON Schema to Avro IDL.
>>
>
> So GraphQL's IDL (interface definition language) isn't quite a JSON
> Schema, but the responses are often represented as JSON, but it's not JSON
> Schema per-se.
>
> For use the use-case is very different, even if Avro and GraphQL's IDLs
> could _almost_ be losslessly interchanged at some level, they both have a
> decent type system, they both allow definition of RPC services, one is a
> great candidate for public APIs (GraphQL has "directives", and really nice
> annotation and documentation generation tools), and Avro is ideal for our
> internal APIs. An Avro payload for us runs ~12-30 bytes, where JSON would
> be at least 2-3x the size (we send a lot of very small messages, very
> similar ones, so JSON reserializing the keys every time would kill us). So
> Avro gives us something nothing JSON oriented can. Also, we use Avro as our
> archival format using the
> https://avro.apache.org/docs/current/spec.html#Object+Container+Files
> which I believe is also sort-of unique.
>
>
>> Is JSON Schema the ubiquitous "contract" language that you are using in
>> your company, so you want to keep it as the source of truth while also
>> being able to work with Avro?
>>
>
> It's just public vs. private (or, internal) APIs, and being deliberate
> about storing those IDL files in separate repositories and training teams
> to get into the habit of planning and co-designing changes to these
> ubiquitous contracts before they need to do implementation work, since
> changing the contract affects everyone. (be that some "near realtime" RPC
> service that is in the hot path of customer requests  on the web API, or
> whether that's offline processing by our BI teams who are running reports
> based on the archived data from the datawarehouse)
>
> The company just went through explosive growth, so, whilst we
> adopted/inherited Avro as part of adopting JVM/Akka stuff for some parts of
> the infra, the pivot to nominate these IDLs as the point at which teams
> have to synchronize and coordinate is still something we are building out.
>
>
>> On Tue, Jan 26, 2021 at 04:38:18PM +0100, Lee Hambley wrote:
>> > I would say that in general, being around the industry for 15 or so
>> years
>> > now, that there has been a definite uptake in these binary protocols.
>> >
>> > If I had to speculate, I'd that that outside a few niches, the ASN.1 and
>> > similar protocols never *really* took-off outside telecoms, which is
>> > regrettable because they are really fantastic protocols (they are used
>> > extensively in certificates, DER/PEM are in the ASN.1 family of things,
>> SSL
>> > certs are all ASN.1 encoded, usually, etc.)
>> >
>> > These days seems like everyone has some "big data" pipe, and having
>> > Hadoop/Spark/etc has become the must-have thing in most SMEs, so you
>> > inherit some of these things by "accident".
>> >
>> > I personally come from the event-sourcing, CQRS, domain-driven-design
>> > circles, here having a ubiquitous language "contract", preferably a
>> > bullet-proof one with good change management tooling is something that
>> you
>> > explicitly go looking for. In that sphere you come across msgpack,
>> > capnproto, protobufs, thrift, etc which all offer insane performance,
>> very
>> > compact payloads, but Avro is unique in offering something like a schema
>> > registry and concrete guarantees about rolling coordinating deploys with
>> > between producers and consumers (note: I _think_ protobufs got something
>> > like a schema registry now, but I never used it)
>> >
>> > Another increasingly good option for this in the "SDL" (schema
>> definition
>> > language) spec space is GraphQL which isn't a _binary_ packing format,
>> but
>> > does offer a standalone 

Re: Companies using Apache Avro

[Niels Basjes]

Hi,

I work at bol.com (biggest online retailer in the Netherlands/Belgium area)
and we heavily use Avro.

Here is a talk about why we use Avro:
https://www.youtube.com/watch?v=QOdhaEHbSZM

Niels Basjes

On Mon, Jan 25, 2021 at 9:28 PM Juan Cruz Viotti  wrote:

> Hey there!
>
> Do you know where can I find a list of relatively well-known companies
> that make use of Apache Avro? I'm trying to collect a small list for
> research purposes and my search is not yielding many results apart from
> Facebook.
>
> Thanks in advance,
>
> --
> Juan Cruz Viotti
> Software Engineer
> https://www.jviotti.com
>


-- 
Best regards / Met vriendelijke groeten,

Niels Basjes

Re: Companies using Apache Avro

[Oscar Westra van Holthe - Kind]

On wed 27 jan. 2021 11:20, Ismaël Mejía  wrote:

> A good reference  and comparison between formats and the advantages of
> Avro that you can refer on your paper is on Martin Kleppmann book:
>
> https://www.oreilly.com/library/view/designing-data-intensive-applications/9781491903063/ch04.html


This is an excellent reference (and primer) on many distributed data
subjects. It lists many more references, in case you wouldneed them.



Kind regards,
Oscar

Re: Companies using Apache Avro

[Ismaël Mejía]

Avro is basically used by most Big Data streaming systems: Kafka,
Pulsar, Flink, Beam, etc, so most companies use it. The reason why
Avro is used is not only structuring data with Schemas and the compact
binary representation, but people also use Avro because its support of
versioning/evolution.

You can find more details about this on Confluent's Avro documentation:
https://docs.confluent.io/platform/current/schema-registry/avro.html

A good reference  and comparison between formats and the advantages of
Avro that you can refer on your paper is on Martin Kleppmann book:
https://www.oreilly.com/library/view/designing-data-intensive-applications/9781491903063/ch04.html

On Wed, Jan 27, 2021 at 10:27 AM Lee Hambley  wrote:
>>
>> Thanks for the extensive response! I think a lot of what you are saying
>> is very spot on!
>
>
> I hope it's useful, if and when your paper is published, I'd love to give it 
> a read.
>
>>
>> I'm working on a paper surveying ~13 binary schema-less and
>> schema-driven serialization formats (including Avro) that can handle any
>> data structure that JSON can represent. Therefore, I was particularly
>> interested on why you wanted to convert JSON Schema to Avro IDL.
>
>
> So GraphQL's IDL (interface definition language) isn't quite a JSON Schema, 
> but the responses are often represented as JSON, but it's not JSON Schema 
> per-se.
>
> For use the use-case is very different, even if Avro and GraphQL's IDLs could 
> _almost_ be losslessly interchanged at some level, they both have a decent 
> type system, they both allow definition of RPC services, one is a great 
> candidate for public APIs (GraphQL has "directives", and really nice 
> annotation and documentation generation tools), and Avro is ideal for our 
> internal APIs. An Avro payload for us runs ~12-30 bytes, where JSON would be 
> at least 2-3x the size (we send a lot of very small messages, very similar 
> ones, so JSON reserializing the keys every time would kill us). So Avro gives 
> us something nothing JSON oriented can. Also, we use Avro as our archival 
> format using the 
> https://avro.apache.org/docs/current/spec.html#Object+Container+Files which I 
> believe is also sort-of unique.
>
>>
>> Is JSON Schema the ubiquitous "contract" language that you are using in
>> your company, so you want to keep it as the source of truth while also
>> being able to work with Avro?
>
>
> It's just public vs. private (or, internal) APIs, and being deliberate about 
> storing those IDL files in separate repositories and training teams to get 
> into the habit of planning and co-designing changes to these ubiquitous 
> contracts before they need to do implementation work, since changing the 
> contract affects everyone. (be that some "near realtime" RPC service that is 
> in the hot path of customer requests  on the web API, or whether that's 
> offline processing by our BI teams who are running reports based on the 
> archived data from the datawarehouse)
>
> The company just went through explosive growth, so, whilst we 
> adopted/inherited Avro as part of adopting JVM/Akka stuff for some parts of 
> the infra, the pivot to nominate these IDLs as the point at which teams have 
> to synchronize and coordinate is still something we are building out.
>
>>
>> On Tue, Jan 26, 2021 at 04:38:18PM +0100, Lee Hambley wrote:
>> > I would say that in general, being around the industry for 15 or so years
>> > now, that there has been a definite uptake in these binary protocols.
>> >
>> > If I had to speculate, I'd that that outside a few niches, the ASN.1 and
>> > similar protocols never *really* took-off outside telecoms, which is
>> > regrettable because they are really fantastic protocols (they are used
>> > extensively in certificates, DER/PEM are in the ASN.1 family of things, SSL
>> > certs are all ASN.1 encoded, usually, etc.)
>> >
>> > These days seems like everyone has some "big data" pipe, and having
>> > Hadoop/Spark/etc has become the must-have thing in most SMEs, so you
>> > inherit some of these things by "accident".
>> >
>> > I personally come from the event-sourcing, CQRS, domain-driven-design
>> > circles, here having a ubiquitous language "contract", preferably a
>> > bullet-proof one with good change management tooling is something that you
>> > explicitly go looking for. In that sphere you come across msgpack,
>> > capnproto, protobufs, thrift, etc which all offer insane performance, very
>> > compact payloads, but Avro is unique in offering something like a schema
>> > registry and concrete guarantees about rolling coordinating deploys with
>> > between producers and consumers (note: I _think_ protobufs got something
>> > like a schema registry now, but I never used it)
>> >
>> > Another increasingly good option for this in the "SDL" (schema definition
>> > language) spec space is GraphQL which isn't a _binary_ packing format, but
>> > does offer a standalone schema definition language for defining service
>> > contracts. 

Re: Companies using Apache Avro

[Lee Hambley]

>
> Thanks for the extensive response! I think a lot of what you are saying
> is very spot on!
>

I hope it's useful, if and when your paper is published, I'd love to give
it a read.


> I'm working on a paper surveying ~13 binary schema-less and
> schema-driven serialization formats (including Avro) that can handle any
> data structure that JSON can represent. Therefore, I was particularly
> interested on why you wanted to convert JSON Schema to Avro IDL.
>

So GraphQL's IDL (interface definition language) isn't quite a JSON Schema,
but the responses are often represented as JSON, but it's not JSON Schema
per-se.

For use the use-case is very different, even if Avro and GraphQL's IDLs
could _almost_ be losslessly interchanged at some level, they both have a
decent type system, they both allow definition of RPC services, one is a
great candidate for public APIs (GraphQL has "directives", and really nice
annotation and documentation generation tools), and Avro is ideal for our
internal APIs. An Avro payload for us runs ~12-30 bytes, where JSON would
be at least 2-3x the size (we send a lot of very small messages, very
similar ones, so JSON reserializing the keys every time would kill us). So
Avro gives us something nothing JSON oriented can. Also, we use Avro as our
archival format using the
https://avro.apache.org/docs/current/spec.html#Object+Container+Files which
I believe is also sort-of unique.


> Is JSON Schema the ubiquitous "contract" language that you are using in
> your company, so you want to keep it as the source of truth while also
> being able to work with Avro?
>

It's just public vs. private (or, internal) APIs, and being deliberate
about storing those IDL files in separate repositories and training teams
to get into the habit of planning and co-designing changes to these
ubiquitous contracts before they need to do implementation work, since
changing the contract affects everyone. (be that some "near realtime" RPC
service that is in the hot path of customer requests  on the web API, or
whether that's offline processing by our BI teams who are running reports
based on the archived data from the datawarehouse)

The company just went through explosive growth, so, whilst we
adopted/inherited Avro as part of adopting JVM/Akka stuff for some parts of
the infra, the pivot to nominate these IDLs as the point at which teams
have to synchronize and coordinate is still something we are building out.


> On Tue, Jan 26, 2021 at 04:38:18PM +0100, Lee Hambley wrote:
> > I would say that in general, being around the industry for 15 or so years
> > now, that there has been a definite uptake in these binary protocols.
> >
> > If I had to speculate, I'd that that outside a few niches, the ASN.1 and
> > similar protocols never *really* took-off outside telecoms, which is
> > regrettable because they are really fantastic protocols (they are used
> > extensively in certificates, DER/PEM are in the ASN.1 family of things,
> SSL
> > certs are all ASN.1 encoded, usually, etc.)
> >
> > These days seems like everyone has some "big data" pipe, and having
> > Hadoop/Spark/etc has become the must-have thing in most SMEs, so you
> > inherit some of these things by "accident".
> >
> > I personally come from the event-sourcing, CQRS, domain-driven-design
> > circles, here having a ubiquitous language "contract", preferably a
> > bullet-proof one with good change management tooling is something that
> you
> > explicitly go looking for. In that sphere you come across msgpack,
> > capnproto, protobufs, thrift, etc which all offer insane performance,
> very
> > compact payloads, but Avro is unique in offering something like a schema
> > registry and concrete guarantees about rolling coordinating deploys with
> > between producers and consumers (note: I _think_ protobufs got something
> > like a schema registry now, but I never used it)
> >
> > Another increasingly good option for this in the "SDL" (schema definition
> > language) spec space is GraphQL which isn't a _binary_ packing format,
> but
> > does offer a standalone schema definition language for defining service
> > contracts. Whilst Avro does account for RPC protocols
> > , I
> > haven't really seen that used so much in the wild, but maybe that's just
> my
> > "bubble" speaking. GraphQL doesn't *really* have the schema migration
> tools
> > that Avro has, but at least when dealing with GraphQL payloads, most
> > language implementations give you the underlying syntax tree for the
> > payload, so it's a bit easier to see what clients are requesting and what
> > fields need various levels of scrutiny before being changed.
> >
> > Anyway, probably nothing of this is really interesting to your paper,
> but I
> > never miss a good opportunity to share unsolicited opinions :D
> >
> > Lee Hambley
> > http://lee.hambley.name/
> > +49 (0) 170 298 5667
> >
> >
> > On Tue, 26 Jan 2021 at 16:27, Juan Cruz 

Re: Companies using Apache Avro

[Zoltan Farkas]

I think LinkedIn a fairly well known user…
They have several blogs on how they use avro, and some work like: 
https://github.com/linkedin/avro-util 

—Z

> On Jan 25, 2021, at 3:27 PM, Juan Cruz Viotti  wrote:
> 
> Hey there!
> 
> Do you know where can I find a list of relatively well-known companies
> that make use of Apache Avro? I'm trying to collect a small list for
> research purposes and my search is not yielding many results apart from
> Facebook.
> 
> Thanks in advance,
> 
> -- 
> Juan Cruz Viotti
> Software Engineer
> https://www.jviotti.com

Re: Companies using Apache Avro

[Juan Cruz Viotti]

Hi Lee,

Thanks for the extensive response! I think a lot of what you are saying
is very spot on! 

I'm working on a paper surveying ~13 binary schema-less and
schema-driven serialization formats (including Avro) that can handle any
data structure that JSON can represent. Therefore, I was particularly
interested on why you wanted to convert JSON Schema to Avro IDL. 

Is JSON Schema the ubiquitous "contract" language that you are using in
your company, so you want to keep it as the source of truth while also
being able to work with Avro?

On Tue, Jan 26, 2021 at 04:38:18PM +0100, Lee Hambley wrote:
> I would say that in general, being around the industry for 15 or so years
> now, that there has been a definite uptake in these binary protocols.
> 
> If I had to speculate, I'd that that outside a few niches, the ASN.1 and
> similar protocols never *really* took-off outside telecoms, which is
> regrettable because they are really fantastic protocols (they are used
> extensively in certificates, DER/PEM are in the ASN.1 family of things, SSL
> certs are all ASN.1 encoded, usually, etc.)
> 
> These days seems like everyone has some "big data" pipe, and having
> Hadoop/Spark/etc has become the must-have thing in most SMEs, so you
> inherit some of these things by "accident".
> 
> I personally come from the event-sourcing, CQRS, domain-driven-design
> circles, here having a ubiquitous language "contract", preferably a
> bullet-proof one with good change management tooling is something that you
> explicitly go looking for. In that sphere you come across msgpack,
> capnproto, protobufs, thrift, etc which all offer insane performance, very
> compact payloads, but Avro is unique in offering something like a schema
> registry and concrete guarantees about rolling coordinating deploys with
> between producers and consumers (note: I _think_ protobufs got something
> like a schema registry now, but I never used it)
> 
> Another increasingly good option for this in the "SDL" (schema definition
> language) spec space is GraphQL which isn't a _binary_ packing format, but
> does offer a standalone schema definition language for defining service
> contracts. Whilst Avro does account for RPC protocols
> , I
> haven't really seen that used so much in the wild, but maybe that's just my
> "bubble" speaking. GraphQL doesn't *really* have the schema migration tools
> that Avro has, but at least when dealing with GraphQL payloads, most
> language implementations give you the underlying syntax tree for the
> payload, so it's a bit easier to see what clients are requesting and what
> fields need various levels of scrutiny before being changed.
> 
> Anyway, probably nothing of this is really interesting to your paper, but I
> never miss a good opportunity to share unsolicited opinions :D
> 
> Lee Hambley
> http://lee.hambley.name/
> +49 (0) 170 298 5667
> 
> 
> On Tue, 26 Jan 2021 at 16:27, Juan Cruz Viotti  wrote:
> 
> > > I don't mean to make light of your question, just to point out that I
> > > don't think many companies are proudly announcing to the world that
> > > they use Avro... why would they?
> >
> > Indeed, I totally agree. I'm writing a research paper involving Apache
> > Avro and just wanted to enrich the historical sections a bit with some
> > industry usage information!
> >
> > On Mon, Jan 25, 2021 at 10:40:31PM +0100, Lee Hambley wrote:
> > > I work for two companies using Avro (contractor, I won't name them) but I
> > > don't know what good it serves anyone knowing that we use them. Would you
> > > ask the same question about JSON, or XML, or whether we use nginx or
> > > apache?
> > >
> > > Avro is one of about 5 components in the distributed messaging
> > > architectures, and aside that is is very nicely designed (I believe the
> > > schema versioning and rigorously documented canonical forms are an almost
> > > unique point of attraction)
> > >
> > > I don't mean to make light of your question, just to point out that I
> > don't
> > > think many companies are proudly announcing to the world that they use
> > > Avro... why would they?
> > >
> > > Lee Hambley
> > > http://lee.hambley.name/
> > > +49 (0) 170 298 5667
> > >
> > >
> > > On Mon, 25 Jan 2021 at 22:30, M. Manna  wrote:
> > >
> > > >
> > > > I believe Confluent and Imply are the two companies I know of.
> > > >
> > > >
> > > > On Mon, 25 Jan 2021 at 20:28, Juan Cruz Viotti  wrote:
> > > >
> > > >> Hey there!
> > > >>
> > > >> Do you know where can I find a list of relatively well-known companies
> > > >> that make use of Apache Avro? I'm trying to collect a small list for
> > > >> research purposes and my search is not yielding many results apart
> > from
> > > >> Facebook.
> > > >>
> > > >> Thanks in advance,
> > > >>
> > > >> --
> > > >> Juan Cruz Viotti
> > > >> Software Engineer
> > > >> https://www.jviotti.com
> > > >>
> > > >
> >
> > --
> > Juan Cruz Viotti
> > Software Engineer
> > 

Re: Companies using Apache Avro

[Lee Hambley]

I would say that in general, being around the industry for 15 or so years
now, that there has been a definite uptake in these binary protocols.

If I had to speculate, I'd that that outside a few niches, the ASN.1 and
similar protocols never *really* took-off outside telecoms, which is
regrettable because they are really fantastic protocols (they are used
extensively in certificates, DER/PEM are in the ASN.1 family of things, SSL
certs are all ASN.1 encoded, usually, etc.)

These days seems like everyone has some "big data" pipe, and having
Hadoop/Spark/etc has become the must-have thing in most SMEs, so you
inherit some of these things by "accident".

I personally come from the event-sourcing, CQRS, domain-driven-design
circles, here having a ubiquitous language "contract", preferably a
bullet-proof one with good change management tooling is something that you
explicitly go looking for. In that sphere you come across msgpack,
capnproto, protobufs, thrift, etc which all offer insane performance, very
compact payloads, but Avro is unique in offering something like a schema
registry and concrete guarantees about rolling coordinating deploys with
between producers and consumers (note: I _think_ protobufs got something
like a schema registry now, but I never used it)

Another increasingly good option for this in the "SDL" (schema definition
language) spec space is GraphQL which isn't a _binary_ packing format, but
does offer a standalone schema definition language for defining service
contracts. Whilst Avro does account for RPC protocols
, I
haven't really seen that used so much in the wild, but maybe that's just my
"bubble" speaking. GraphQL doesn't *really* have the schema migration tools
that Avro has, but at least when dealing with GraphQL payloads, most
language implementations give you the underlying syntax tree for the
payload, so it's a bit easier to see what clients are requesting and what
fields need various levels of scrutiny before being changed.

Anyway, probably nothing of this is really interesting to your paper, but I
never miss a good opportunity to share unsolicited opinions :D

Lee Hambley
http://lee.hambley.name/
+49 (0) 170 298 5667


On Tue, 26 Jan 2021 at 16:27, Juan Cruz Viotti  wrote:

> > I don't mean to make light of your question, just to point out that I
> > don't think many companies are proudly announcing to the world that
> > they use Avro... why would they?
>
> Indeed, I totally agree. I'm writing a research paper involving Apache
> Avro and just wanted to enrich the historical sections a bit with some
> industry usage information!
>
> On Mon, Jan 25, 2021 at 10:40:31PM +0100, Lee Hambley wrote:
> > I work for two companies using Avro (contractor, I won't name them) but I
> > don't know what good it serves anyone knowing that we use them. Would you
> > ask the same question about JSON, or XML, or whether we use nginx or
> > apache?
> >
> > Avro is one of about 5 components in the distributed messaging
> > architectures, and aside that is is very nicely designed (I believe the
> > schema versioning and rigorously documented canonical forms are an almost
> > unique point of attraction)
> >
> > I don't mean to make light of your question, just to point out that I
> don't
> > think many companies are proudly announcing to the world that they use
> > Avro... why would they?
> >
> > Lee Hambley
> > http://lee.hambley.name/
> > +49 (0) 170 298 5667
> >
> >
> > On Mon, 25 Jan 2021 at 22:30, M. Manna  wrote:
> >
> > >
> > > I believe Confluent and Imply are the two companies I know of.
> > >
> > >
> > > On Mon, 25 Jan 2021 at 20:28, Juan Cruz Viotti  wrote:
> > >
> > >> Hey there!
> > >>
> > >> Do you know where can I find a list of relatively well-known companies
> > >> that make use of Apache Avro? I'm trying to collect a small list for
> > >> research purposes and my search is not yielding many results apart
> from
> > >> Facebook.
> > >>
> > >> Thanks in advance,
> > >>
> > >> --
> > >> Juan Cruz Viotti
> > >> Software Engineer
> > >> https://www.jviotti.com
> > >>
> > >
>
> --
> Juan Cruz Viotti
> Software Engineer
> https://www.jviotti.com
>

Re: Companies using Apache Avro

[Juan Cruz Viotti]

> I don't mean to make light of your question, just to point out that I
> don't think many companies are proudly announcing to the world that
> they use Avro... why would they?

Indeed, I totally agree. I'm writing a research paper involving Apache
Avro and just wanted to enrich the historical sections a bit with some
industry usage information!

On Mon, Jan 25, 2021 at 10:40:31PM +0100, Lee Hambley wrote:
> I work for two companies using Avro (contractor, I won't name them) but I
> don't know what good it serves anyone knowing that we use them. Would you
> ask the same question about JSON, or XML, or whether we use nginx or
> apache?
> 
> Avro is one of about 5 components in the distributed messaging
> architectures, and aside that is is very nicely designed (I believe the
> schema versioning and rigorously documented canonical forms are an almost
> unique point of attraction)
> 
> I don't mean to make light of your question, just to point out that I don't
> think many companies are proudly announcing to the world that they use
> Avro... why would they?
> 
> Lee Hambley
> http://lee.hambley.name/
> +49 (0) 170 298 5667
> 
> 
> On Mon, 25 Jan 2021 at 22:30, M. Manna  wrote:
> 
> >
> > I believe Confluent and Imply are the two companies I know of.
> >
> >
> > On Mon, 25 Jan 2021 at 20:28, Juan Cruz Viotti  wrote:
> >
> >> Hey there!
> >>
> >> Do you know where can I find a list of relatively well-known companies
> >> that make use of Apache Avro? I'm trying to collect a small list for
> >> research purposes and my search is not yielding many results apart from
> >> Facebook.
> >>
> >> Thanks in advance,
> >>
> >> --
> >> Juan Cruz Viotti
> >> Software Engineer
> >> https://www.jviotti.com
> >>
> >

-- 
Juan Cruz Viotti
Software Engineer
https://www.jviotti.com

Re: Companies using Apache Avro

[Vance Duncan]

I worked for Blue Cross Blue Shield of Florida (AKA Florida Blue) and they
were using it heavily for their new event-driven enterprise data platform,
which uses Kafka as the event bus. I developed a number of tools for the
platform, including a spreadsheet-based management utility for defining
record types and attributes. The tool generated Avro schemas, JSON schemas,
and JSON test data for JSON, all from the same spreadsheet definition.

On Mon, Jan 25, 2021 at 3:28 PM Juan Cruz Viotti  wrote:

> Hey there!
>
> Do you know where can I find a list of relatively well-known companies
> that make use of Apache Avro? I'm trying to collect a small list for
> research purposes and my search is not yielding many results apart from
> Facebook.
>
> Thanks in advance,
>
> --
> Juan Cruz Viotti
> Software Engineer
> https://www.jviotti.com
>
-- 
Regards,

Vance Duncan
mailto:dunca...@gmail.com
http://www.linkedin.com/in/VanceDuncan
(904) 553-5582

Re: Companies using Apache Avro

[Jeremy Custenborder]

There are not a lot of companies that directly use avro. They will
most likely use a system that uses avro.

If you're looking for companies, then you're looking for systems that
the companies make or use thinks like:

Apache Spark
Apache Hadoop
Apache Kafka
Apache Hive
Apache Drill
Snowflake
Presto
Google Big table

Here is a link of the usages of avro. You'll see a ton of things.
https://mvnrepository.com/artifact/org.apache.avro/avro/usages

On Mon, Jan 25, 2021 at 3:40 PM Lee Hambley  wrote:
>
> I work for two companies using Avro (contractor, I won't name them) but I 
> don't know what good it serves anyone knowing that we use them. Would you ask 
> the same question about JSON, or XML, or whether we use nginx or apache?
>
> Avro is one of about 5 components in the distributed messaging architectures, 
> and aside that is is very nicely designed (I believe the schema versioning 
> and rigorously documented canonical forms are an almost unique point of 
> attraction)
>
> I don't mean to make light of your question, just to point out that I don't 
> think many companies are proudly announcing to the world that they use 
> Avro... why would they?
>
> Lee Hambley
> http://lee.hambley.name/
> +49 (0) 170 298 5667
>
>
> On Mon, 25 Jan 2021 at 22:30, M. Manna  wrote:
>>
>>
>> I believe Confluent and Imply are the two companies I know of.
>>
>>
>> On Mon, 25 Jan 2021 at 20:28, Juan Cruz Viotti  wrote:
>>>
>>> Hey there!
>>>
>>> Do you know where can I find a list of relatively well-known companies
>>> that make use of Apache Avro? I'm trying to collect a small list for
>>> research purposes and my search is not yielding many results apart from
>>> Facebook.
>>>
>>> Thanks in advance,
>>>
>>> --
>>> Juan Cruz Viotti
>>> Software Engineer
>>> https://www.jviotti.com

Re: Companies using Apache Avro

[Lee Hambley]

I work for two companies using Avro (contractor, I won't name them) but I
don't know what good it serves anyone knowing that we use them. Would you
ask the same question about JSON, or XML, or whether we use nginx or
apache?

Avro is one of about 5 components in the distributed messaging
architectures, and aside that is is very nicely designed (I believe the
schema versioning and rigorously documented canonical forms are an almost
unique point of attraction)

I don't mean to make light of your question, just to point out that I don't
think many companies are proudly announcing to the world that they use
Avro... why would they?

Lee Hambley
http://lee.hambley.name/
+49 (0) 170 298 5667


On Mon, 25 Jan 2021 at 22:30, M. Manna  wrote:

>
> I believe Confluent and Imply are the two companies I know of.
>
>
> On Mon, 25 Jan 2021 at 20:28, Juan Cruz Viotti  wrote:
>
>> Hey there!
>>
>> Do you know where can I find a list of relatively well-known companies
>> that make use of Apache Avro? I'm trying to collect a small list for
>> research purposes and my search is not yielding many results apart from
>> Facebook.
>>
>> Thanks in advance,
>>
>> --
>> Juan Cruz Viotti
>> Software Engineer
>> https://www.jviotti.com
>>
>

Re: Companies using Apache Avro

[M. Manna]

I believe Confluent and Imply are the two companies I know of.


On Mon, 25 Jan 2021 at 20:28, Juan Cruz Viotti  wrote:

> Hey there!
>
> Do you know where can I find a list of relatively well-known companies
> that make use of Apache Avro? I'm trying to collect a small list for
> research purposes and my search is not yielding many results apart from
> Facebook.
>
> Thanks in advance,
>
> --
> Juan Cruz Viotti
> Software Engineer
> https://www.jviotti.com
>

Companies using Apache Avro

[Juan Cruz Viotti]

Hey there!

Do you know where can I find a list of relatively well-known companies
that make use of Apache Avro? I'm trying to collect a small list for
research purposes and my search is not yielding many results apart from
Facebook.

Thanks in advance,

-- 
Juan Cruz Viotti
Software Engineer
https://www.jviotti.com