[
https://issues.apache.org/jira/browse/MINIFICPP-954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16893988#comment-16893988
]
Mr TheSegfault edited comment on MINIFICPP-954 at 7/26/19 5:04 PM:
-------------------------------------------------------------------
[~bakaid] I like some of the concepts you present but prior CVEs were why we
moved away from protobuf initially in community discussions. I think
offloading the testing to protobuf maintainers is a reason to have Protobuf but
I think both will need to exist. We also considered flatbuffers and capn proto
– would love to see that added to this discussion.
I think this ultimately would be either adding protobuf/flatbuffers/capn as a
selectable line format. We have a very simple line format that isn't going to
change ( this is because our line formats are designed to use the RESTFul API
for the complicated stuff ). The line format is simply for sending a very tiny
subset of information. We also want to use the same code for nanofi, where I
think our consumers are more likely to appreciate not including protobuf. As a
result we can add this as an extension ( in fact, I love the idea ) –
especially since we can mitigate concerns of others by having it be
configurable in the agent ( and compile time for nanofi ECUs ).
If we went down the route of cap'n proto we can rely on the embedded RPC and
use that as a separate C2 protocol. I have used Cap'n Proto, for example, and
we could easily define those RPC calls to lessen the load, but I would likely
change this ticket to exploring alternative protocols/wire formats and then
making a decision as an addition to what we have now as opposed to a
replacement.
We will be forced to maintain it but we have no choice either as we have users
who specifically don't want protobuf and we have released it so we are forced
to maintain it and improve its testing.
Keep in mind that whomever tackles this will need to balance maintaining what
we already have ( not replacing ) so we then increase what we must test. We can
rely on capn proto or protobuf's tests, but then we still need to perform our
test in the venn diagram of testing. Would that effort be better spent
improving tests for our work since we already need to do that and must maintain
it?
I'll leave that question up to the implementor – but it's certainly not an
affront against doing this work, more a word of caution. I'm glad we decided
against protobuf at the time, though – would have made that decision again. It
was a calculated risk with many thoughts in mind – but would love to see
something else like protobuf/flatbuffers/capn .
Aside from this comment I would love the community to weigh in and make a
choice. I've made my points known and have no intention of implementing this
myself due to other technical debt I feel is more important so thoughts on what
to do are up to the implementation seeker.
was (Author: phrocker):
[~bakaid] I like some of the concepts you present but prior CVEs were why we
moved away from protobuf initially in community discussions. I think
offloading the testing to protobuf maintainers is a reason to have Protobuf but
I think both will need to exist. We also considered flatbuffers and capn proto.
I think this ultimately would be either adding protobuf/flatbuffers/capn as a
selectable line format. We have a very simple line format that isn't going to
change ( this is because our line formats are designed to use the RESTFul API
for the complicated stuff ). The line format is simply for sending a very tiny
subset of information. We also want to use the same code for nanofi, where I
think our consumers are more likely to appreciate not including protobuf. As a
result we can add this as an extension ( in fact, I love the idea ) –
especially since we can mitigate concerns of others by having it be
configurable in the agent ( and compile time for nanofi ECUs ).
If we went down the route of cap'n proto we can rely on the embedded RPC and
use that as a separate C2 protocol. I have used Cap'n Proto, for example, and
we could easily define those RPC calls to lessen the load, but I would likely
change this ticket to exploring alternative protocols/wire formats and then
making a decision as an addition to what we have now as opposed to a
replacement.
We will be forced to maintain it but we have no choice either as we have users
who specifically don't want protobuf and we have released it so we are forced
to maintain it and improve its testing.
Keep in mind that whomever tackles this will need to balance maintaining what
we already have ( not replacing ) so we then increase what we must test. We can
rely on capn proto or protobuf's tests, but then we still need to perform our
test in the venn diagram of testing. Would that effort be better spent
improving tests for our work since we already need to do that and must maintain
it?
I'll leave that question up to the implementor – but it's certainly not an
affront against doing this work, more a word of caution. I'm glad we decided
against protobuf at the time, though – would have made that decision again. It
was a calculated risk with many thoughts in mind – but would love to see
something else like protobuf/flatbuffers/capn .
> Consider moving the C2 line protocol to Google Protobuf
> -------------------------------------------------------
>
> Key: MINIFICPP-954
> URL: https://issues.apache.org/jira/browse/MINIFICPP-954
> Project: Apache NiFi MiNiFi C++
> Issue Type: Improvement
> Reporter: Daniel Bakai
> Priority: Major
>
> * Very compact serialized format
> * Easy protocol description
> * Support for generating parsers for many languages from the proto file
> * Has good versioning support
> * Thoroughly tested, fuzzed
> * Less chance of introducing security vulnerabilities with hand-written
> parsers
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
