[jira] [Commented] (THRIFT-3752) nil collections are serialized as empty collections
[
https://issues.apache.org/jira/browse/THRIFT-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879232#comment-15879232
]
Jens Geyer commented on THRIFT-3752:
I remember having a somewhat lengthy discussion about that during one of the
last big rewrites of the Thrift Go bindings with the author(s) of that patch.
Not quite sure what the main reason was, but at the end we decided to implement
it this way. That being said, I would be happy if we come up with a better
solution that is more aligned to the usually expected behaviour, there's
absolutely no question about that. This is Go - so it should be possible,
shouldn't it ;-)?
> nil collections are serialized as empty collections
> ---
>
> Key: THRIFT-3752
> URL: https://issues.apache.org/jira/browse/THRIFT-3752
> Project: Thrift
> Issue Type: Bug
> Components: Go - Compiler
>Reporter: John Sirois
>Assignee: John Sirois
>
> See discussion here: https://reviews.apache.org/r/45193/
> This is likely related to THRIFT-3700.
> In short, for this struct:
> {noformat}
> struct TaskQuery {
> 4: optional set taskIds
> }
> {noformat}
> The following go struct is generated:
> {noformat}
> type TaskQuery struct {
> TaskIds map[string]bool `thrift:"taskIds,4" json:"taskIds"`
> }
> {noformat}
> This is all well and good, since {{TaskQuery{}.TaskIds == nil}}; ie the
> {{TaskIds}} collection field is - by default - unset.
> The problem is in the serialization for the field, which wipes out the unset
> ({{nil}}) vs empty collection distinction at the wire level:
> {noformat}
> func (p *TaskQuery) writeField4(oprot thrift.TProtocol) (err error) {
> if err := oprot.WriteFieldBegin("taskIds", thrift.SET, 4); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T write field begin
> error 4:taskIds: ", p), err)
> }
> if err := oprot.WriteSetBegin(thrift.STRING, len(p.TaskIds)); err !=
> nil {
> return thrift.PrependError("error writing set begin: ", err)
> }
> for v, _ := range p.TaskIds {
> if err := oprot.WriteString(string(v)); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T. (0) field
> write error: ", p), err)
> }
> }
> if err := oprot.WriteSetEnd(); err != nil {
> return thrift.PrependError("error writing set end: ", err)
> }
> if err := oprot.WriteFieldEnd(); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T write field end
> error 4:taskIds: ", p), err)
> }
> return err
> }
> {noformat}
> So on the receiving end of the wire, a {{nil}} collection is turned into an
> empty collection and so unset-ness cannot be distinguished from set but empty.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (THRIFT-3752) nil collections are serialized as empty collections
[
https://issues.apache.org/jira/browse/THRIFT-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877173#comment-15877173
]
John Sirois commented on THRIFT-3752:
-
[~jking], I don't think so. The compiler still does the {{IsSetXXX,
WriteFieldBegin}} sequence, and the problem noted in this issue is this encodes
a different notion of "unset requiredness" than the java compiler. The issue
was specifically cross-lang incompatibility. So unless the java compiler has
been changed to have the same notion of "unset requiredness" in the intervening
time (aka if the cross-tests are enabled, complete and including all
languages), this is probably still an issue.
> nil collections are serialized as empty collections
> ---
>
> Key: THRIFT-3752
> URL: https://issues.apache.org/jira/browse/THRIFT-3752
> Project: Thrift
> Issue Type: Bug
> Components: Go - Compiler
>Reporter: John Sirois
>Assignee: John Sirois
>
> See discussion here: https://reviews.apache.org/r/45193/
> This is likely related to THRIFT-3700.
> In short, for this struct:
> {noformat}
> struct TaskQuery {
> 4: optional set taskIds
> }
> {noformat}
> The following go struct is generated:
> {noformat}
> type TaskQuery struct {
> TaskIds map[string]bool `thrift:"taskIds,4" json:"taskIds"`
> }
> {noformat}
> This is all well and good, since {{TaskQuery{}.TaskIds == nil}}; ie the
> {{TaskIds}} collection field is - by default - unset.
> The problem is in the serialization for the field, which wipes out the unset
> ({{nil}}) vs empty collection distinction at the wire level:
> {noformat}
> func (p *TaskQuery) writeField4(oprot thrift.TProtocol) (err error) {
> if err := oprot.WriteFieldBegin("taskIds", thrift.SET, 4); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T write field begin
> error 4:taskIds: ", p), err)
> }
> if err := oprot.WriteSetBegin(thrift.STRING, len(p.TaskIds)); err !=
> nil {
> return thrift.PrependError("error writing set begin: ", err)
> }
> for v, _ := range p.TaskIds {
> if err := oprot.WriteString(string(v)); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T. (0) field
> write error: ", p), err)
> }
> }
> if err := oprot.WriteSetEnd(); err != nil {
> return thrift.PrependError("error writing set end: ", err)
> }
> if err := oprot.WriteFieldEnd(); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T write field end
> error 4:taskIds: ", p), err)
> }
> return err
> }
> {noformat}
> So on the receiving end of the wire, a {{nil}} collection is turned into an
> empty collection and so unset-ness cannot be distinguished from set but empty.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (THRIFT-3752) nil collections are serialized as empty collections
[
https://issues.apache.org/jira/browse/THRIFT-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877116#comment-15877116
]
James E. King, III commented on THRIFT-3752:
Is it possible THRIFT-4011 helped to resolve this issue?
> nil collections are serialized as empty collections
> ---
>
> Key: THRIFT-3752
> URL: https://issues.apache.org/jira/browse/THRIFT-3752
> Project: Thrift
> Issue Type: Bug
> Components: Go - Compiler
>Reporter: John Sirois
>Assignee: John Sirois
>
> See discussion here: https://reviews.apache.org/r/45193/
> This is likely related to THRIFT-3700.
> In short, for this struct:
> {noformat}
> struct TaskQuery {
> 4: optional set taskIds
> }
> {noformat}
> The following go struct is generated:
> {noformat}
> type TaskQuery struct {
> TaskIds map[string]bool `thrift:"taskIds,4" json:"taskIds"`
> }
> {noformat}
> This is all well and good, since {{TaskQuery{}.TaskIds == nil}}; ie the
> {{TaskIds}} collection field is - by default - unset.
> The problem is in the serialization for the field, which wipes out the unset
> ({{nil}}) vs empty collection distinction at the wire level:
> {noformat}
> func (p *TaskQuery) writeField4(oprot thrift.TProtocol) (err error) {
> if err := oprot.WriteFieldBegin("taskIds", thrift.SET, 4); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T write field begin
> error 4:taskIds: ", p), err)
> }
> if err := oprot.WriteSetBegin(thrift.STRING, len(p.TaskIds)); err !=
> nil {
> return thrift.PrependError("error writing set begin: ", err)
> }
> for v, _ := range p.TaskIds {
> if err := oprot.WriteString(string(v)); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T. (0) field
> write error: ", p), err)
> }
> }
> if err := oprot.WriteSetEnd(); err != nil {
> return thrift.PrependError("error writing set end: ", err)
> }
> if err := oprot.WriteFieldEnd(); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T write field end
> error 4:taskIds: ", p), err)
> }
> return err
> }
> {noformat}
> So on the receiving end of the wire, a {{nil}} collection is turned into an
> empty collection and so unset-ness cannot be distinguished from set but empty.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (THRIFT-3752) nil collections are serialized as empty collections
[
https://issues.apache.org/jira/browse/THRIFT-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324969#comment-15324969
]
Connor Gorman commented on THRIFT-3752:
---
Hey [~jsirois],
How is this going? I can jump in and help if you'd like. Considering this is
for Aurora, watch out for https://issues.apache.org/jira/browse/THRIFT-3851.
Making a patch for it now
> nil collections are serialized as empty collections
> ---
>
> Key: THRIFT-3752
> URL: https://issues.apache.org/jira/browse/THRIFT-3752
> Project: Thrift
> Issue Type: Bug
> Components: Go - Compiler
>Reporter: John Sirois
>Assignee: John Sirois
>
> See discussion here: https://reviews.apache.org/r/45193/
> This is likely related to THRIFT-3700.
> In short, for this struct:
> {noformat}
> struct TaskQuery {
> 4: optional set taskIds
> }
> {noformat}
> The following go struct is generated:
> {noformat}
> type TaskQuery struct {
> TaskIds map[string]bool `thrift:"taskIds,4" json:"taskIds"`
> }
> {noformat}
> This is all well and good, since {{TaskQuery{}.TaskIds == nil}}; ie the
> {{TaskIds}} collection field is - by default - unset.
> The problem is in the serialization for the field, which wipes out the unset
> ({{nil}}) vs empty collection distinction at the wire level:
> {noformat}
> func (p *TaskQuery) writeField4(oprot thrift.TProtocol) (err error) {
> if err := oprot.WriteFieldBegin("taskIds", thrift.SET, 4); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T write field begin
> error 4:taskIds: ", p), err)
> }
> if err := oprot.WriteSetBegin(thrift.STRING, len(p.TaskIds)); err !=
> nil {
> return thrift.PrependError("error writing set begin: ", err)
> }
> for v, _ := range p.TaskIds {
> if err := oprot.WriteString(string(v)); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T. (0) field
> write error: ", p), err)
> }
> }
> if err := oprot.WriteSetEnd(); err != nil {
> return thrift.PrependError("error writing set end: ", err)
> }
> if err := oprot.WriteFieldEnd(); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T write field end
> error 4:taskIds: ", p), err)
> }
> return err
> }
> {noformat}
> So on the receiving end of the wire, a {{nil}} collection is turned into an
> empty collection and so unset-ness cannot be distinguished from set but empty.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (THRIFT-3752) nil collections are serialized as empty collections
[
https://issues.apache.org/jira/browse/THRIFT-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15218753#comment-15218753
]
Jens Geyer commented on THRIFT-3752:
Correct, thanks. The same behaviour is used by some other languages as well.
I'll add that information to the docs.
> nil collections are serialized as empty collections
> ---
>
> Key: THRIFT-3752
> URL: https://issues.apache.org/jira/browse/THRIFT-3752
> Project: Thrift
> Issue Type: Bug
> Components: Go - Compiler
>Reporter: John Sirois
>Assignee: John Sirois
>
> See discussion here: https://reviews.apache.org/r/45193/
> This is likely related to THRIFT-3700.
> In short, for this struct:
> {noformat}
> struct TaskQuery {
> 4: optional set taskIds
> }
> {noformat}
> The following go struct is generated:
> {noformat}
> type TaskQuery struct {
> TaskIds map[string]bool `thrift:"taskIds,4" json:"taskIds"`
> }
> {noformat}
> This is all well and good, since {{TaskQuery{}.TaskIds == nil}}; ie the
> {{TaskIds}} collection field is - by default - unset.
> The problem is in the serialization for the field, which wipes out the unset
> ({{nil}}) vs empty collection distinction at the wire level:
> {noformat}
> func (p *TaskQuery) writeField4(oprot thrift.TProtocol) (err error) {
> if err := oprot.WriteFieldBegin("taskIds", thrift.SET, 4); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T write field begin
> error 4:taskIds: ", p), err)
> }
> if err := oprot.WriteSetBegin(thrift.STRING, len(p.TaskIds)); err !=
> nil {
> return thrift.PrependError("error writing set begin: ", err)
> }
> for v, _ := range p.TaskIds {
> if err := oprot.WriteString(string(v)); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T. (0) field
> write error: ", p), err)
> }
> }
> if err := oprot.WriteSetEnd(); err != nil {
> return thrift.PrependError("error writing set end: ", err)
> }
> if err := oprot.WriteFieldEnd(); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T write field end
> error 4:taskIds: ", p), err)
> }
> return err
> }
> {noformat}
> So on the receiving end of the wire, a {{nil}} collection is turned into an
> empty collection and so unset-ness cannot be distinguished from set but empty.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (THRIFT-3752) nil collections are serialized as empty collections
[
https://issues.apache.org/jira/browse/THRIFT-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15213669#comment-15213669
]
John Sirois commented on THRIFT-3752:
-
Actually [~jensg] - I can't make sense of the spec you added for this case. If
I look at the java generated serialization code for a no-requiredness set
I find:
{noformat}
if (struct.instanceIds != null) {
oprot.writeFieldBegin(INSTANCE_IDS_FIELD_DESC);
{
oprot.writeSetBegin(new
org.apache.thrift.protocol.TSet(org.apache.thrift.protocol.TType.I32,
struct.instanceIds.size()));
for (int _iter143 : struct.instanceIds)
{
oprot.writeI32(_iter143);
}
oprot.writeSetEnd();
}
oprot.writeFieldEnd();
}
{noformat}
Your spec in THRIFT-3756 says "Write: Like required, the fields are always
written.". In the java case though, the field is only written when non-null -
which is a good thing, since it allows the read-side of the wire to correctly
interpret this field as unset.
> nil collections are serialized as empty collections
> ---
>
> Key: THRIFT-3752
> URL: https://issues.apache.org/jira/browse/THRIFT-3752
> Project: Thrift
> Issue Type: Bug
> Components: Go - Compiler
>Reporter: John Sirois
>Assignee: John Sirois
>
> See discussion here: https://reviews.apache.org/r/45193/
> This is likely related to THRIFT-3700.
> In short, for this struct:
> {noformat}
> struct TaskQuery {
> 4: optional set taskIds
> }
> {noformat}
> The following go struct is generated:
> {noformat}
> type TaskQuery struct {
> TaskIds map[string]bool `thrift:"taskIds,4" json:"taskIds"`
> }
> {noformat}
> This is all well and good, since {{TaskQuery{}.TaskIds == nil}}; ie the
> {{TaskIds}} collection field is - by default - unset.
> The problem is in the serialization for the field, which wipes out the unset
> ({{nil}}) vs empty collection distinction at the wire level:
> {noformat}
> func (p *TaskQuery) writeField4(oprot thrift.TProtocol) (err error) {
> if err := oprot.WriteFieldBegin("taskIds", thrift.SET, 4); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T write field begin
> error 4:taskIds: ", p), err)
> }
> if err := oprot.WriteSetBegin(thrift.STRING, len(p.TaskIds)); err !=
> nil {
> return thrift.PrependError("error writing set begin: ", err)
> }
> for v, _ := range p.TaskIds {
> if err := oprot.WriteString(string(v)); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T. (0) field
> write error: ", p), err)
> }
> }
> if err := oprot.WriteSetEnd(); err != nil {
> return thrift.PrependError("error writing set end: ", err)
> }
> if err := oprot.WriteFieldEnd(); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T write field end
> error 4:taskIds: ", p), err)
> }
> return err
> }
> {noformat}
> So on the receiving end of the wire, a {{nil}} collection is turned into an
> empty collection and so unset-ness cannot be distinguished from set but empty.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (THRIFT-3752) nil collections are serialized as empty collections
[
https://issues.apache.org/jira/browse/THRIFT-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15213646#comment-15213646
]
John Sirois commented on THRIFT-3752:
-
Noting the not-quite-yet-minted docs are here:
https://github.com/Jens-G/thrift/blob/THRIFT-3756/doc/specs/idl.md
I'll proceed with a fix that conforms to those docs.
> nil collections are serialized as empty collections
> ---
>
> Key: THRIFT-3752
> URL: https://issues.apache.org/jira/browse/THRIFT-3752
> Project: Thrift
> Issue Type: Bug
> Components: Go - Compiler
>Reporter: John Sirois
>Assignee: John Sirois
>
> See discussion here: https://reviews.apache.org/r/45193/
> This is likely related to THRIFT-3700.
> In short, for this struct:
> {noformat}
> struct TaskQuery {
> 4: optional set taskIds
> }
> {noformat}
> The following go struct is generated:
> {noformat}
> type TaskQuery struct {
> TaskIds map[string]bool `thrift:"taskIds,4" json:"taskIds"`
> }
> {noformat}
> This is all well and good, since {{TaskQuery{}.TaskIds == nil}}; ie the
> {{TaskIds}} collection field is - by default - unset.
> The problem is in the serialization for the field, which wipes out the unset
> ({{nil}}) vs empty collection distinction at the wire level:
> {noformat}
> func (p *TaskQuery) writeField4(oprot thrift.TProtocol) (err error) {
> if err := oprot.WriteFieldBegin("taskIds", thrift.SET, 4); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T write field begin
> error 4:taskIds: ", p), err)
> }
> if err := oprot.WriteSetBegin(thrift.STRING, len(p.TaskIds)); err !=
> nil {
> return thrift.PrependError("error writing set begin: ", err)
> }
> for v, _ := range p.TaskIds {
> if err := oprot.WriteString(string(v)); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T. (0) field
> write error: ", p), err)
> }
> }
> if err := oprot.WriteSetEnd(); err != nil {
> return thrift.PrependError("error writing set end: ", err)
> }
> if err := oprot.WriteFieldEnd(); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T write field end
> error 4:taskIds: ", p), err)
> }
> return err
> }
> {noformat}
> So on the receiving end of the wire, a {{nil}} collection is turned into an
> empty collection and so unset-ness cannot be distinguished from set but empty.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (THRIFT-3752) nil collections are serialized as empty collections
[
https://issues.apache.org/jira/browse/THRIFT-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15209376#comment-15209376
]
John Sirois commented on THRIFT-3752:
-
OK - great. Thanks [~jensg]. After hitting this incompat in Go clients of the
java Aurora scheduler, my 1st searches after not finding thrift docs [1][2]
landed me here which made me suspect the non-comformance of thrift compiler
backend verticals was more widespread. I'll pretend its limited to only the Go
backend until I have evidence otherwise!
[1] https://twitter.github.io/scrooge/Semantics.html
[2] http://lionet.livejournal.com/66899.html
> nil collections are serialized as empty collections
> ---
>
> Key: THRIFT-3752
> URL: https://issues.apache.org/jira/browse/THRIFT-3752
> Project: Thrift
> Issue Type: Bug
> Components: Go - Compiler
>Reporter: John Sirois
>Assignee: John Sirois
>
> See discussion here: https://reviews.apache.org/r/45193/
> This is likely related to THRIFT-3700.
> In short, for this struct:
> {noformat}
> struct TaskQuery {
> 4: optional set taskIds
> }
> {noformat}
> The following go struct is generated:
> {noformat}
> type TaskQuery struct {
> TaskIds map[string]bool `thrift:"taskIds,4" json:"taskIds"`
> }
> {noformat}
> This is all well and good, since {{TaskQuery{}.TaskIds == nil}}; ie the
> {{TaskIds}} collection field is - by default - unset.
> The problem is in the serialization for the field, which wipes out the unset
> ({{nil}}) vs empty collection distinction at the wire level:
> {noformat}
> func (p *TaskQuery) writeField4(oprot thrift.TProtocol) (err error) {
> if err := oprot.WriteFieldBegin("taskIds", thrift.SET, 4); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T write field begin
> error 4:taskIds: ", p), err)
> }
> if err := oprot.WriteSetBegin(thrift.STRING, len(p.TaskIds)); err !=
> nil {
> return thrift.PrependError("error writing set begin: ", err)
> }
> for v, _ := range p.TaskIds {
> if err := oprot.WriteString(string(v)); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T. (0) field
> write error: ", p), err)
> }
> }
> if err := oprot.WriteSetEnd(); err != nil {
> return thrift.PrependError("error writing set end: ", err)
> }
> if err := oprot.WriteFieldEnd(); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T write field end
> error 4:taskIds: ", p), err)
> }
> return err
> }
> {noformat}
> So on the receiving end of the wire, a {{nil}} collection is turned into an
> empty collection and so unset-ness cannot be distinguished from set but empty.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (THRIFT-3752) nil collections are serialized as empty collections
[
https://issues.apache.org/jira/browse/THRIFT-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15209239#comment-15209239
]
Jens Geyer commented on THRIFT-3752:
I don't know what you mean but the semantics of required, optional and default
requiredness are quite clear. I agree on the docs part, at least some more
details should be added to the web site.
{quote}
The problem is in the serialization for the field, which wipes out the unset
(nil) vs empty collection distinction at the wire level:
{quote}
If it is wrong (and after looking at your examples I very much tend to agree to
that), then the logical conclusion is to fix it.
> nil collections are serialized as empty collections
> ---
>
> Key: THRIFT-3752
> URL: https://issues.apache.org/jira/browse/THRIFT-3752
> Project: Thrift
> Issue Type: Bug
> Components: Go - Compiler
>Reporter: John Sirois
>Assignee: John Sirois
>
> See discussion here: https://reviews.apache.org/r/45193/
> This is likely related to THRIFT-3700.
> In short, for this struct:
> {noformat}
> struct TaskQuery {
> 4: optional set taskIds
> }
> {noformat}
> The following go struct is generated:
> {noformat}
> type TaskQuery struct {
> TaskIds map[string]bool `thrift:"taskIds,4" json:"taskIds"`
> }
> {noformat}
> This is all well and good, since {{TaskQuery{}.TaskIds == nil}}; ie the
> {{TaskIds}} collection field is - by default - unset.
> The problem is in the serialization for the field, which wipes out the unset
> ({{nil}}) vs empty collection distinction at the wire level:
> {noformat}
> func (p *TaskQuery) writeField4(oprot thrift.TProtocol) (err error) {
> if err := oprot.WriteFieldBegin("taskIds", thrift.SET, 4); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T write field begin
> error 4:taskIds: ", p), err)
> }
> if err := oprot.WriteSetBegin(thrift.STRING, len(p.TaskIds)); err !=
> nil {
> return thrift.PrependError("error writing set begin: ", err)
> }
> for v, _ := range p.TaskIds {
> if err := oprot.WriteString(string(v)); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T. (0) field
> write error: ", p), err)
> }
> }
> if err := oprot.WriteSetEnd(); err != nil {
> return thrift.PrependError("error writing set end: ", err)
> }
> if err := oprot.WriteFieldEnd(); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T write field end
> error 4:taskIds: ", p), err)
> }
> return err
> }
> {noformat}
> So on the receiving end of the wire, a {{nil}} collection is turned into an
> empty collection and so unset-ness cannot be distinguished from set but empty.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (THRIFT-3752) nil collections are serialized as empty collections
[
https://issues.apache.org/jira/browse/THRIFT-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15208762#comment-15208762
]
John Sirois commented on THRIFT-3752:
-
Hrm, so this may not be a thrift compiler bug per-se. Marking
{{TaskQuery.taskIds}} as optional yields:
{noformat}
type TaskQuery struct {
TaskIds map[string]bool `thrift:"taskIds,4"
json:"taskIds,omitempty"`
}
func (p *TaskQuery) IsSetTaskIds() bool {
return p.TaskIds != nil
}
func (p *TaskQuery) writeField4(oprot thrift.TProtocol) (err error) {
if p.IsSetTaskIds() {
if err := oprot.WriteFieldBegin("taskIds", thrift.SET, 4); err != nil {
return thrift.PrependError(fmt.Sprintf("%T write field begin error
4:taskIds: ", p), err)
}
if err := oprot.WriteSetBegin(thrift.STRING, len(p.TaskIds)); err !=
nil {
return thrift.PrependError("error writing set begin: ", err)
}
for v, _ := range p.TaskIds {
if err := oprot.WriteString(string(v)); err != nil {
return thrift.PrependError(fmt.Sprintf("%T. (0) field write
error: ", p), err)
}
}
if err := oprot.WriteSetEnd(); err != nil {
return thrift.PrependError("error writing set end: ", err)
}
if err := oprot.WriteFieldEnd(); err != nil {
return thrift.PrependError(fmt.Sprintf("%T write field end error
4:taskIds: ", p), err)
}
}
return err
}
{noformat}
The {{IsSetTaskIds}} method is new and its use to skip serialization for an
unset {{taskIds}} field is what we want.
In other words, the thirft compiler for Go has a different notion of unset
requiredness than the java compiler (The TaskQuery receiver in this case is the
Aurora scheduler - written in java). The perils of thrift's global lack of
specificity as to how to handle optional vs required vs is well
documented and the real issue here :/.
[~jfarrell] Has there been any discussion of fixing this, declaring bankruptcy,
or picking some other path to deal with incompatible behavior across the
various language backends wrt requiredness?
> nil collections are serialized as empty collections
> ---
>
> Key: THRIFT-3752
> URL: https://issues.apache.org/jira/browse/THRIFT-3752
> Project: Thrift
> Issue Type: Bug
> Components: Go - Compiler
>Reporter: John Sirois
>Assignee: John Sirois
>
> See discussion here: https://reviews.apache.org/r/45193/
> This is likely related to THRIFT-3700.
> In short, for this struct:
> {noformat}
> struct TaskQuery {
> 4: optional set taskIds
> }
> {noformat}
> The following go struct is generated:
> {noformat}
> type TaskQuery struct {
> TaskIds map[string]bool `thrift:"taskIds,4" json:"taskIds"`
> }
> {noformat}
> This is all well and good, since {{TaskQuery{}.TaskIds == nil}}; ie the
> {{TaskIds}} collection field is - by default - unset.
> The problem is in the serialization for the field, which wipes out the unset
> ({{nil}}) vs empty collection distinction at the wire level:
> {noformat}
> func (p *TaskQuery) writeField4(oprot thrift.TProtocol) (err error) {
> if err := oprot.WriteFieldBegin("taskIds", thrift.SET, 4); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T write field begin
> error 4:taskIds: ", p), err)
> }
> if err := oprot.WriteSetBegin(thrift.STRING, len(p.TaskIds)); err !=
> nil {
> return thrift.PrependError("error writing set begin: ", err)
> }
> for v, _ := range p.TaskIds {
> if err := oprot.WriteString(string(v)); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T. (0) field
> write error: ", p), err)
> }
> }
> if err := oprot.WriteSetEnd(); err != nil {
> return thrift.PrependError("error writing set end: ", err)
> }
> if err := oprot.WriteFieldEnd(); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T write field end
> error 4:taskIds: ", p), err)
> }
> return err
> }
> {noformat}
> So on the receiving end of the wire, a {{nil}} collection is turned into an
> empty collection and so unset-ness cannot be distinguished from set but empty.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
