[jira] [Created] (THRIFT-3752) nil collections are serialized as empty collections
John Sirois created THRIFT-3752: --- Summary: nil collections are serialized as empty collections Key: THRIFT-3752 URL: https://issues.apache.org/jira/browse/THRIFT-3752 Project: Thrift Issue Type: Bug Components: Go - Compiler Reporter: John Sirois See discussion here: https://reviews.apache.org/r/45193/ This is likely related to THRIFT-3700. In short, for this struct: {noformat} struct TaskQuery { 4: optional set taskIds } {noformat} The following go struct is generated: {noformat} type TaskQuery struct { TaskIds map[string]bool `thrift:"taskIds,4" json:"taskIds"` } {noformat} This is all well and good, since {{TaskQuery{}.TaskIds == nil}}; ie the {{TaskIds}} collection field is - by default - unset. The problem is in the serialization for the field, which wipes out the unset ({{nil}}) vs empty collection distinction at the wire level: {noformat} func (p *TaskQuery) writeField4(oprot thrift.TProtocol) (err error) { if err := oprot.WriteFieldBegin("taskIds", thrift.SET, 4); err != nil { return thrift.PrependError(fmt.Sprintf("%T write field begin error 4:taskIds: ", p), err) } if err := oprot.WriteSetBegin(thrift.STRING, len(p.TaskIds)); err != nil { return thrift.PrependError("error writing set begin: ", err) } for v, _ := range p.TaskIds { if err := oprot.WriteString(string(v)); err != nil { return thrift.PrependError(fmt.Sprintf("%T. (0) field write error: ", p), err) } } if err := oprot.WriteSetEnd(); err != nil { return thrift.PrependError("error writing set end: ", err) } if err := oprot.WriteFieldEnd(); err != nil { return thrift.PrependError(fmt.Sprintf("%T write field end error 4:taskIds: ", p), err) } return err } {noformat} So on the receiving end of the wire, a {{nil}} collection is turned into an empty collection and so unset-ness cannot be distinguished from set but empty. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (THRIFT-3752) nil collections are serialized as empty collections
[ https://issues.apache.org/jira/browse/THRIFT-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sirois reassigned THRIFT-3752: --- Assignee: John Sirois > nil collections are serialized as empty collections > --- > > Key: THRIFT-3752 > URL: https://issues.apache.org/jira/browse/THRIFT-3752 > Project: Thrift > Issue Type: Bug > Components: Go - Compiler >Reporter: John Sirois >Assignee: John Sirois > > See discussion here: https://reviews.apache.org/r/45193/ > This is likely related to THRIFT-3700. > In short, for this struct: > {noformat} > struct TaskQuery { > 4: optional set taskIds > } > {noformat} > The following go struct is generated: > {noformat} > type TaskQuery struct { > TaskIds map[string]bool `thrift:"taskIds,4" json:"taskIds"` > } > {noformat} > This is all well and good, since {{TaskQuery{}.TaskIds == nil}}; ie the > {{TaskIds}} collection field is - by default - unset. > The problem is in the serialization for the field, which wipes out the unset > ({{nil}}) vs empty collection distinction at the wire level: > {noformat} > func (p *TaskQuery) writeField4(oprot thrift.TProtocol) (err error) { > if err := oprot.WriteFieldBegin("taskIds", thrift.SET, 4); err != nil { > return thrift.PrependError(fmt.Sprintf("%T write field begin > error 4:taskIds: ", p), err) > } > if err := oprot.WriteSetBegin(thrift.STRING, len(p.TaskIds)); err != > nil { > return thrift.PrependError("error writing set begin: ", err) > } > for v, _ := range p.TaskIds { > if err := oprot.WriteString(string(v)); err != nil { > return thrift.PrependError(fmt.Sprintf("%T. (0) field > write error: ", p), err) > } > } > if err := oprot.WriteSetEnd(); err != nil { > return thrift.PrependError("error writing set end: ", err) > } > if err := oprot.WriteFieldEnd(); err != nil { > return thrift.PrependError(fmt.Sprintf("%T write field end > error 4:taskIds: ", p), err) > } > return err > } > {noformat} > So on the receiving end of the wire, a {{nil}} collection is turned into an > empty collection and so unset-ness cannot be distinguished from set but empty. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3700) Go Map has wrong default value when optional
[ https://issues.apache.org/jira/browse/THRIFT-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208543#comment-15208543 ] John Sirois commented on THRIFT-3700: - [~connorgorman] - I filed THRIFT-3752 today which is very similar - down to the underlying thrift-using project in question! My bug may be dup to yours, but the specific problem I found was on the serialization side of things. If you have tie to read that bug report and comment on the mergeability of these two issues, I'd be grateful. I'm working on a fix for THRIFT-3752 currently and wouldn't mind knocking off a fix for this issue too. > Go Map has wrong default value when optional > > > Key: THRIFT-3700 > URL: https://issues.apache.org/jira/browse/THRIFT-3700 > Project: Thrift > Issue Type: Bug > Components: Go - Compiler >Affects Versions: 0.9.3 >Reporter: Connor Gorman > > Template: > {code} > struct TaskQuery { > 4: optional set taskIds > } > {code} > renders in Golang as : > {code} > TaskIds map[string]bool `thrift:"taskIds,4" json:"taskIds,omitempty"` > {code} > In Golang, the default value for a map is simply an empty map and not nil, > but the optional check > {code} > func (p *TaskQuery) IsSetTaskIds() bool { > return p.TaskIds != nil > } > {code} > checks against nil instead if an empty map -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3752) nil collections are serialized as empty collections
[ https://issues.apache.org/jira/browse/THRIFT-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208762#comment-15208762 ] John Sirois commented on THRIFT-3752: - Hrm, so this may not be a thrift compiler bug per-se. Marking {{TaskQuery.taskIds}} as optional yields: {noformat} type TaskQuery struct { TaskIds map[string]bool `thrift:"taskIds,4" json:"taskIds,omitempty"` } func (p *TaskQuery) IsSetTaskIds() bool { return p.TaskIds != nil } func (p *TaskQuery) writeField4(oprot thrift.TProtocol) (err error) { if p.IsSetTaskIds() { if err := oprot.WriteFieldBegin("taskIds", thrift.SET, 4); err != nil { return thrift.PrependError(fmt.Sprintf("%T write field begin error 4:taskIds: ", p), err) } if err := oprot.WriteSetBegin(thrift.STRING, len(p.TaskIds)); err != nil { return thrift.PrependError("error writing set begin: ", err) } for v, _ := range p.TaskIds { if err := oprot.WriteString(string(v)); err != nil { return thrift.PrependError(fmt.Sprintf("%T. (0) field write error: ", p), err) } } if err := oprot.WriteSetEnd(); err != nil { return thrift.PrependError("error writing set end: ", err) } if err := oprot.WriteFieldEnd(); err != nil { return thrift.PrependError(fmt.Sprintf("%T write field end error 4:taskIds: ", p), err) } } return err } {noformat} The {{IsSetTaskIds}} method is new and its use to skip serialization for an unset {{taskIds}} field is what we want. In other words, the thirft compiler for Go has a different notion of unset requiredness than the java compiler (The TaskQuery receiver in this case is the Aurora scheduler - written in java). The perils of thrift's global lack of specificity as to how to handle optional vs required vs is well documented and the real issue here :/. [~jfarrell] Has there been any discussion of fixing this, declaring bankruptcy, or picking some other path to deal with incompatible behavior across the various language backends wrt requiredness? > nil collections are serialized as empty collections > --- > > Key: THRIFT-3752 > URL: https://issues.apache.org/jira/browse/THRIFT-3752 > Project: Thrift > Issue Type: Bug > Components: Go - Compiler >Reporter: John Sirois >Assignee: John Sirois > > See discussion here: https://reviews.apache.org/r/45193/ > This is likely related to THRIFT-3700. > In short, for this struct: > {noformat} > struct TaskQuery { > 4: optional set taskIds > } > {noformat} > The following go struct is generated: > {noformat} > type TaskQuery struct { > TaskIds map[string]bool `thrift:"taskIds,4" json:"taskIds"` > } > {noformat} > This is all well and good, since {{TaskQuery{}.TaskIds == nil}}; ie the > {{TaskIds}} collection field is - by default - unset. > The problem is in the serialization for the field, which wipes out the unset > ({{nil}}) vs empty collection distinction at the wire level: > {noformat} > func (p *TaskQuery) writeField4(oprot thrift.TProtocol) (err error) { > if err := oprot.WriteFieldBegin("taskIds", thrift.SET, 4); err != nil { > return thrift.PrependError(fmt.Sprintf("%T write field begin > error 4:taskIds: ", p), err) > } > if err := oprot.WriteSetBegin(thrift.STRING, len(p.TaskIds)); err != > nil { > return thrift.PrependError("error writing set begin: ", err) > } > for v, _ := range p.TaskIds { > if err := oprot.WriteString(string(v)); err != nil { > return thrift.PrependError(fmt.Sprintf("%T. (0) field > write error: ", p), err) > } > } > if err := oprot.WriteSetEnd(); err != nil { > return thrift.PrependError("error writing set end: ", err) > } > if err := oprot.WriteFieldEnd(); err != nil { > return thrift.PrependError(fmt.Sprintf("%T write field end > error 4:taskIds: ", p), err) > } > return err > } > {noformat} > So on the receiving end of the wire, a {{nil}} collection is turned into an > empty collection and so unset-ness cannot be distinguished from set but empty. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (THRIFT-3752) nil collections are serialized as empty collections
[ https://issues.apache.org/jira/browse/THRIFT-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208762#comment-15208762 ] John Sirois edited comment on THRIFT-3752 at 3/23/16 4:55 PM: -- Hrm, so this may not be a thrift compiler bug per-se. Marking {{TaskQuery.taskIds}} as optional yields: {noformat} type TaskQuery struct { TaskIds map[string]bool `thrift:"taskIds,4" json:"taskIds,omitempty"` } func (p *TaskQuery) IsSetTaskIds() bool { return p.TaskIds != nil } func (p *TaskQuery) writeField4(oprot thrift.TProtocol) (err error) { if p.IsSetTaskIds() { if err := oprot.WriteFieldBegin("taskIds", thrift.SET, 4); err != nil { return thrift.PrependError(fmt.Sprintf("%T write field begin error 4:taskIds: ", p), err) } if err := oprot.WriteSetBegin(thrift.STRING, len(p.TaskIds)); err != nil { return thrift.PrependError("error writing set begin: ", err) } for v, _ := range p.TaskIds { if err := oprot.WriteString(string(v)); err != nil { return thrift.PrependError(fmt.Sprintf("%T. (0) field write error: ", p), err) } } if err := oprot.WriteSetEnd(); err != nil { return thrift.PrependError("error writing set end: ", err) } if err := oprot.WriteFieldEnd(); err != nil { return thrift.PrependError(fmt.Sprintf("%T write field end error 4:taskIds: ", p), err) } } return err } {noformat} The {{IsSetTaskIds}} method is new and its used to skip serialization for an unset {{taskIds}} field is what we want. In other words, the thirft compiler for Go has a different notion of unset requiredness than the java compiler (The TaskQuery receiver in this case is the Aurora scheduler - written in java). The perils of thrift's global lack of specificity as to how to handle optional vs required vs is well documented and the real issue here :/. [~jfarrell] Has there been any discussion of fixing this, declaring bankruptcy, or picking some other path to deal with incompatible behavior across the various language backends wrt requiredness? was (Author: jsirois): Hrm, so this may not be a thrift compiler bug per-se. Marking {{TaskQuery.taskIds}} as optional yields: {noformat} type TaskQuery struct { TaskIds map[string]bool `thrift:"taskIds,4" json:"taskIds,omitempty"` } func (p *TaskQuery) IsSetTaskIds() bool { return p.TaskIds != nil } func (p *TaskQuery) writeField4(oprot thrift.TProtocol) (err error) { if p.IsSetTaskIds() { if err := oprot.WriteFieldBegin("taskIds", thrift.SET, 4); err != nil { return thrift.PrependError(fmt.Sprintf("%T write field begin error 4:taskIds: ", p), err) } if err := oprot.WriteSetBegin(thrift.STRING, len(p.TaskIds)); err != nil { return thrift.PrependError("error writing set begin: ", err) } for v, _ := range p.TaskIds { if err := oprot.WriteString(string(v)); err != nil { return thrift.PrependError(fmt.Sprintf("%T. (0) field write error: ", p), err) } } if err := oprot.WriteSetEnd(); err != nil { return thrift.PrependError("error writing set end: ", err) } if err := oprot.WriteFieldEnd(); err != nil { return thrift.PrependError(fmt.Sprintf("%T write field end error 4:taskIds: ", p), err) } } return err } {noformat} The {{IsSetTaskIds}} method is new and its use to skip serialization for an unset {{taskIds}} field is what we want. In other words, the thirft compiler for Go has a different notion of unset requiredness than the java compiler (The TaskQuery receiver in this case is the Aurora scheduler - written in java). The perils of thrift's global lack of specificity as to how to handle optional vs required vs is well documented and the real issue here :/. [~jfarrell] Has there been any discussion of fixing this, declaring bankruptcy, or picking some other path to deal with incompatible behavior across the various language backends wrt requiredness? > nil collections are serialized as empty collections > --- > > Key: THRIFT-3752 > URL: https://issues.apache.org/jira/browse/THRIFT-3752 > Project: Thrift > Issue Type: Bug > Components: Go - Compiler >Reporter: John Sirois >Assignee: John Sirois > > See discussion here: https://reviews.apache.org/r/45193/ > This is likely related to THRIFT-3700. > In short, for this struct: > {noformat} > struct TaskQuery { > 4: optional set taskIds > } > {noformat} > The following go struct is generated: > {noformat} > type TaskQuery struct { > TaskIds map[string]bool `thrift:"taskI
[jira] [Commented] (THRIFT-3752) nil collections are serialized as empty collections
[ https://issues.apache.org/jira/browse/THRIFT-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15209376#comment-15209376 ] John Sirois commented on THRIFT-3752: - OK - great. Thanks [~jensg]. After hitting this incompat in Go clients of the java Aurora scheduler, my 1st searches after not finding thrift docs [1][2] landed me here which made me suspect the non-comformance of thrift compiler backend verticals was more widespread. I'll pretend its limited to only the Go backend until I have evidence otherwise! [1] https://twitter.github.io/scrooge/Semantics.html [2] http://lionet.livejournal.com/66899.html > nil collections are serialized as empty collections > --- > > Key: THRIFT-3752 > URL: https://issues.apache.org/jira/browse/THRIFT-3752 > Project: Thrift > Issue Type: Bug > Components: Go - Compiler >Reporter: John Sirois >Assignee: John Sirois > > See discussion here: https://reviews.apache.org/r/45193/ > This is likely related to THRIFT-3700. > In short, for this struct: > {noformat} > struct TaskQuery { > 4: optional set taskIds > } > {noformat} > The following go struct is generated: > {noformat} > type TaskQuery struct { > TaskIds map[string]bool `thrift:"taskIds,4" json:"taskIds"` > } > {noformat} > This is all well and good, since {{TaskQuery{}.TaskIds == nil}}; ie the > {{TaskIds}} collection field is - by default - unset. > The problem is in the serialization for the field, which wipes out the unset > ({{nil}}) vs empty collection distinction at the wire level: > {noformat} > func (p *TaskQuery) writeField4(oprot thrift.TProtocol) (err error) { > if err := oprot.WriteFieldBegin("taskIds", thrift.SET, 4); err != nil { > return thrift.PrependError(fmt.Sprintf("%T write field begin > error 4:taskIds: ", p), err) > } > if err := oprot.WriteSetBegin(thrift.STRING, len(p.TaskIds)); err != > nil { > return thrift.PrependError("error writing set begin: ", err) > } > for v, _ := range p.TaskIds { > if err := oprot.WriteString(string(v)); err != nil { > return thrift.PrependError(fmt.Sprintf("%T. (0) field > write error: ", p), err) > } > } > if err := oprot.WriteSetEnd(); err != nil { > return thrift.PrependError("error writing set end: ", err) > } > if err := oprot.WriteFieldEnd(); err != nil { > return thrift.PrependError(fmt.Sprintf("%T write field end > error 4:taskIds: ", p), err) > } > return err > } > {noformat} > So on the receiving end of the wire, a {{nil}} collection is turned into an > empty collection and so unset-ness cannot be distinguished from set but empty. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3752) nil collections are serialized as empty collections
[ https://issues.apache.org/jira/browse/THRIFT-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15213646#comment-15213646 ] John Sirois commented on THRIFT-3752: - Noting the not-quite-yet-minted docs are here: https://github.com/Jens-G/thrift/blob/THRIFT-3756/doc/specs/idl.md I'll proceed with a fix that conforms to those docs. > nil collections are serialized as empty collections > --- > > Key: THRIFT-3752 > URL: https://issues.apache.org/jira/browse/THRIFT-3752 > Project: Thrift > Issue Type: Bug > Components: Go - Compiler >Reporter: John Sirois >Assignee: John Sirois > > See discussion here: https://reviews.apache.org/r/45193/ > This is likely related to THRIFT-3700. > In short, for this struct: > {noformat} > struct TaskQuery { > 4: optional set taskIds > } > {noformat} > The following go struct is generated: > {noformat} > type TaskQuery struct { > TaskIds map[string]bool `thrift:"taskIds,4" json:"taskIds"` > } > {noformat} > This is all well and good, since {{TaskQuery{}.TaskIds == nil}}; ie the > {{TaskIds}} collection field is - by default - unset. > The problem is in the serialization for the field, which wipes out the unset > ({{nil}}) vs empty collection distinction at the wire level: > {noformat} > func (p *TaskQuery) writeField4(oprot thrift.TProtocol) (err error) { > if err := oprot.WriteFieldBegin("taskIds", thrift.SET, 4); err != nil { > return thrift.PrependError(fmt.Sprintf("%T write field begin > error 4:taskIds: ", p), err) > } > if err := oprot.WriteSetBegin(thrift.STRING, len(p.TaskIds)); err != > nil { > return thrift.PrependError("error writing set begin: ", err) > } > for v, _ := range p.TaskIds { > if err := oprot.WriteString(string(v)); err != nil { > return thrift.PrependError(fmt.Sprintf("%T. (0) field > write error: ", p), err) > } > } > if err := oprot.WriteSetEnd(); err != nil { > return thrift.PrependError("error writing set end: ", err) > } > if err := oprot.WriteFieldEnd(); err != nil { > return thrift.PrependError(fmt.Sprintf("%T write field end > error 4:taskIds: ", p), err) > } > return err > } > {noformat} > So on the receiving end of the wire, a {{nil}} collection is turned into an > empty collection and so unset-ness cannot be distinguished from set but empty. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3752) nil collections are serialized as empty collections
[ https://issues.apache.org/jira/browse/THRIFT-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15213669#comment-15213669 ] John Sirois commented on THRIFT-3752: - Actually [~jensg] - I can't make sense of the spec you added for this case. If I look at the java generated serialization code for a no-requiredness set I find: {noformat} if (struct.instanceIds != null) { oprot.writeFieldBegin(INSTANCE_IDS_FIELD_DESC); { oprot.writeSetBegin(new org.apache.thrift.protocol.TSet(org.apache.thrift.protocol.TType.I32, struct.instanceIds.size())); for (int _iter143 : struct.instanceIds) { oprot.writeI32(_iter143); } oprot.writeSetEnd(); } oprot.writeFieldEnd(); } {noformat} Your spec in THRIFT-3756 says "Write: Like required, the fields are always written.". In the java case though, the field is only written when non-null - which is a good thing, since it allows the read-side of the wire to correctly interpret this field as unset. > nil collections are serialized as empty collections > --- > > Key: THRIFT-3752 > URL: https://issues.apache.org/jira/browse/THRIFT-3752 > Project: Thrift > Issue Type: Bug > Components: Go - Compiler >Reporter: John Sirois >Assignee: John Sirois > > See discussion here: https://reviews.apache.org/r/45193/ > This is likely related to THRIFT-3700. > In short, for this struct: > {noformat} > struct TaskQuery { > 4: optional set taskIds > } > {noformat} > The following go struct is generated: > {noformat} > type TaskQuery struct { > TaskIds map[string]bool `thrift:"taskIds,4" json:"taskIds"` > } > {noformat} > This is all well and good, since {{TaskQuery{}.TaskIds == nil}}; ie the > {{TaskIds}} collection field is - by default - unset. > The problem is in the serialization for the field, which wipes out the unset > ({{nil}}) vs empty collection distinction at the wire level: > {noformat} > func (p *TaskQuery) writeField4(oprot thrift.TProtocol) (err error) { > if err := oprot.WriteFieldBegin("taskIds", thrift.SET, 4); err != nil { > return thrift.PrependError(fmt.Sprintf("%T write field begin > error 4:taskIds: ", p), err) > } > if err := oprot.WriteSetBegin(thrift.STRING, len(p.TaskIds)); err != > nil { > return thrift.PrependError("error writing set begin: ", err) > } > for v, _ := range p.TaskIds { > if err := oprot.WriteString(string(v)); err != nil { > return thrift.PrependError(fmt.Sprintf("%T. (0) field > write error: ", p), err) > } > } > if err := oprot.WriteSetEnd(); err != nil { > return thrift.PrependError("error writing set end: ", err) > } > if err := oprot.WriteFieldEnd(); err != nil { > return thrift.PrependError(fmt.Sprintf("%T write field end > error 4:taskIds: ", p), err) > } > return err > } > {noformat} > So on the receiving end of the wire, a {{nil}} collection is turned into an > empty collection and so unset-ness cannot be distinguished from set but empty. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-4029) Accelerated protocols do not build from thrift-py 0.10.0 on PyPI
[ https://issues.apache.org/jira/browse/THRIFT-4029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15822950#comment-15822950 ] John Sirois commented on THRIFT-4029: - This really is the behavior you want - mutable artifacts cause no end of debugging grief. It's really a good prompt to release 0.10.1 - difficulty or ease of this aside. Having one good and 1 bad version of the same artifact seems like it's also asking for debugging trouble and confusion for users. > Accelerated protocols do not build from thrift-py 0.10.0 on PyPI > > > Key: THRIFT-4029 > URL: https://issues.apache.org/jira/browse/THRIFT-4029 > Project: Thrift > Issue Type: Bug > Components: Python - Library >Affects Versions: 0.10.0 >Reporter: Chandler May >Assignee: Jake Farrell > > The thrift 0.10.0 distribution on PyPI does not include extension headers and > a C++ (template) file, preventing the accelerated protocols from being built. > {{pip install}} reports a brief error: > {code} > Running thrift-0.10.0/setup.py -q bdist_egg --dist-dir > /tmp/easy_install-bVU8VN/thrift-0.10.0/egg-dist-tmp-ghJzGL > cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for > C/ObjC but not for C++ > src/ext/module.cpp:21:19: fatal error: types.h: No such file or directory > #include "types.h" >^ > compilation terminated. > () > > An error occurred while trying to compile with the C extension enabled > Attempting to build without the extension now > > () > {code} > The list of files that is not included in the distribution is as follows. In > addition to the headers and source file mentioned above, there's a Windows > compatibility header missing and a couple of test files missing. It looks > like there was a file extension filter (accidentally) applied to the C++ > files at least: > {code} > src/ext/binary.h > src/ext/compact.h > src/ext/endian.h > src/ext/protocol.h > src/ext/protocol.tcc > src/ext/types.h > compat/win32/stdint.h > test/_import_local_thrift.py > test/thrift_json.py > {code} > [~jfarrell] were you who made the release? Is it possible to hotfix? My > team has been waiting on the release of the accelerated compact protocol for > a while, this discovery is saddening for us. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3752) nil collections are serialized as empty collections
[ https://issues.apache.org/jira/browse/THRIFT-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15877173#comment-15877173 ] John Sirois commented on THRIFT-3752: - [~jking], I don't think so. The compiler still does the {{IsSetXXX, WriteFieldBegin}} sequence, and the problem noted in this issue is this encodes a different notion of "unset requiredness" than the java compiler. The issue was specifically cross-lang incompatibility. So unless the java compiler has been changed to have the same notion of "unset requiredness" in the intervening time (aka if the cross-tests are enabled, complete and including all languages), this is probably still an issue. > nil collections are serialized as empty collections > --- > > Key: THRIFT-3752 > URL: https://issues.apache.org/jira/browse/THRIFT-3752 > Project: Thrift > Issue Type: Bug > Components: Go - Compiler >Reporter: John Sirois >Assignee: John Sirois > > See discussion here: https://reviews.apache.org/r/45193/ > This is likely related to THRIFT-3700. > In short, for this struct: > {noformat} > struct TaskQuery { > 4: optional set taskIds > } > {noformat} > The following go struct is generated: > {noformat} > type TaskQuery struct { > TaskIds map[string]bool `thrift:"taskIds,4" json:"taskIds"` > } > {noformat} > This is all well and good, since {{TaskQuery{}.TaskIds == nil}}; ie the > {{TaskIds}} collection field is - by default - unset. > The problem is in the serialization for the field, which wipes out the unset > ({{nil}}) vs empty collection distinction at the wire level: > {noformat} > func (p *TaskQuery) writeField4(oprot thrift.TProtocol) (err error) { > if err := oprot.WriteFieldBegin("taskIds", thrift.SET, 4); err != nil { > return thrift.PrependError(fmt.Sprintf("%T write field begin > error 4:taskIds: ", p), err) > } > if err := oprot.WriteSetBegin(thrift.STRING, len(p.TaskIds)); err != > nil { > return thrift.PrependError("error writing set begin: ", err) > } > for v, _ := range p.TaskIds { > if err := oprot.WriteString(string(v)); err != nil { > return thrift.PrependError(fmt.Sprintf("%T. (0) field > write error: ", p), err) > } > } > if err := oprot.WriteSetEnd(); err != nil { > return thrift.PrependError("error writing set end: ", err) > } > if err := oprot.WriteFieldEnd(); err != nil { > return thrift.PrependError(fmt.Sprintf("%T write field end > error 4:taskIds: ", p), err) > } > return err > } > {noformat} > So on the receiving end of the wire, a {{nil}} collection is turned into an > empty collection and so unset-ness cannot be distinguished from set but empty. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (THRIFT-2621) Add basic default hash code method to python
[ https://issues.apache.org/jira/browse/THRIFT-2621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15005212#comment-15005212 ] John Sirois commented on THRIFT-2621: - Maps and lists (and sets) are common struct and service method parameters and it's not uncommon for these container types to contain structs. When this contained structs themselves have maps, lit's or sets, thinges go boom. Is there any way to work around blowing up when reading these types off the wire? > Add basic default hash code method to python > > > Key: THRIFT-2621 > URL: https://issues.apache.org/jira/browse/THRIFT-2621 > Project: Thrift > Issue Type: Improvement > Components: Python - Compiler >Reporter: Jens Geyer >Assignee: Kishor Patil > Fix For: 0.9.2 > > Attachments: > THRIFT-2621_Add_basic_default_hash_code_method_to_python.patch > > > GitHub user kishorvpatil opened a pull request: > https://github.com/apache/thrift/pull/156 > Add basic default hash code method to python, > It doesn't work for maps/lists. This based off Nathan Matz's patch for > storm branch 6d4b008512cd9fcd5a6bb598d09f35eaf2dc7412 > > > You can merge this pull request into a Git repository by running: > $ git pull https://github.com/kishorvpatil/thrift thrift_patch > Alternatively you can review and apply these changes as the patch at: > https://github.com/apache/thrift/pull/156.patch > To close this pull request, make a commit to your master/trunk branch > with (at least) the following in the commit message: > This closes #156 > > > commit f22d578579ab6a78fd07a99a59c2d4af2bb8d3f5 > Author: Kishor Patil > Date: 2014-07-10T20:44:41Z > Adding default hash function to python > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3388) hash doesn't work on set/list
[ https://issues.apache.org/jira/browse/THRIFT-3388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15005591#comment-15005591 ] John Sirois commented on THRIFT-3388: - [~jensg] One appropriate fix seems like a revert. THRIFT-2621 ~lifted a patch striaght from [here|https://github.com/nathanmarz/thrift/commit/6d4b008512cd9fcd5a6bb598d09f35eaf2dc7412] that was appropriate in its original context but it does not seem appropriate to ship with the official Apache thrift. It breaks very common use cases in thrift, namely using map, set, list. It would help to understand [~kishorvpatil]'s use case for THRIFT-2621, since a revert would break presumably break his work. My guess is wanting to avoid writing custom wrappers or keys for thrift structs for use in unit test comparisons and as members of application-level dicts and sets. Increasing the fix complexity, but being sensitive to [~kishorvpatil]'s use case would mean adding deep hash/eq as an option, there is precedent for this in {{java:hashcode}}. Interested in your opinions in particular since I'm not familiar with the thrift community or its protocols on things like this. > hash doesn't work on set/list > - > > Key: THRIFT-3388 > URL: https://issues.apache.org/jira/browse/THRIFT-3388 > Project: Thrift > Issue Type: Bug > Components: Python - Compiler >Affects Versions: 0.9.3 > Environment: ubuntu 14.04 python 2.6.9 >Reporter: Michi Mutsuzaki > > THRIFT-2621 added hash function to thrift generated classes, but certain > types like set or list cannot be hashed. One solution is to convert them to > immutable types (fronzenset, tuple) before calling hash() on them. > {noformat} > $ thrift -version > Thrift version 0.9.3 > $ cat test.thrift > struct Test { > 1: required set test > } > $ thrift --gen py -out . test.thrift > $ cat test.py > from test.ttypes import Test > from thrift.protocol import TBinaryProtocol > from thrift.transport import TTransport > # serialize > t = Test(test=set(["a"])) > tout = TTransport.TMemoryBuffer() > pout = TBinaryProtocol.TBinaryProtocol(tout) > t.write(pout) > # deserialize > tin = TTransport.TMemoryBuffer(tout.getvalue()) > pin = TBinaryProtocol.TBinaryProtocol(tin) > t2 = Test() > t2.read(pin) > # put the deserialized object to a set > a = set([t2]) > $ python test.py > Traceback (most recent call last): > File "test.py", line 18, in > a = set([t2]) > File "/tmp/test/test/ttypes.py", line 81, in __hash__ > value = (value * 31) ^ hash(self.test) > TypeError: unhashable type: 'set' > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3388) hash doesn't work on set/list
[ https://issues.apache.org/jira/browse/THRIFT-3388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15005609#comment-15005609 ] John Sirois commented on THRIFT-3388: - My thought is that THRIFT-2621 makes thrift python un-useable under 0.9.2+, so I'd classify it not as "does not solve the whole issue", but instead as it breaks thrift python for almost all real world cases. As such I favor a revert under the premise that ~no-one can actually be using thrift python support under 0.9.2+ because of this break in the deserialization pipeline inside thrift itself. Although confident of this, the patch was accepted and so I was hoping to get [~kishorvpatil]'s thoughts before posting a revert patch as a proposal. > hash doesn't work on set/list > - > > Key: THRIFT-3388 > URL: https://issues.apache.org/jira/browse/THRIFT-3388 > Project: Thrift > Issue Type: Bug > Components: Python - Compiler >Affects Versions: 0.9.3 > Environment: ubuntu 14.04 python 2.6.9 >Reporter: Michi Mutsuzaki > > THRIFT-2621 added hash function to thrift generated classes, but certain > types like set or list cannot be hashed. One solution is to convert them to > immutable types (fronzenset, tuple) before calling hash() on them. > {noformat} > $ thrift -version > Thrift version 0.9.3 > $ cat test.thrift > struct Test { > 1: required set test > } > $ thrift --gen py -out . test.thrift > $ cat test.py > from test.ttypes import Test > from thrift.protocol import TBinaryProtocol > from thrift.transport import TTransport > # serialize > t = Test(test=set(["a"])) > tout = TTransport.TMemoryBuffer() > pout = TBinaryProtocol.TBinaryProtocol(tout) > t.write(pout) > # deserialize > tin = TTransport.TMemoryBuffer(tout.getvalue()) > pin = TBinaryProtocol.TBinaryProtocol(tin) > t2 = Test() > t2.read(pin) > # put the deserialized object to a set > a = set([t2]) > $ python test.py > Traceback (most recent call last): > File "test.py", line 18, in > a = set([t2]) > File "/tmp/test/test/ttypes.py", line 81, in __hash__ > value = (value * 31) ^ hash(self.test) > TypeError: unhashable type: 'set' > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3388) hash doesn't work on set/list
[ https://issues.apache.org/jira/browse/THRIFT-3388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15009596#comment-15009596 ] John Sirois commented on THRIFT-3388: - Thank you [~jensg]! Does 1.0 have a vague timeline for release or totally unknown? > hash doesn't work on set/list > - > > Key: THRIFT-3388 > URL: https://issues.apache.org/jira/browse/THRIFT-3388 > Project: Thrift > Issue Type: Bug > Components: Python - Compiler >Affects Versions: 0.9.3 > Environment: ubuntu 14.04 python 2.6.9 >Reporter: Michi Mutsuzaki >Assignee: Jens Geyer > Fix For: 1.0 > > Attachments: THRIFT-3388-hash-doesn-t-work-on-set-list.patch > > > THRIFT-2621 added hash function to thrift generated classes, but certain > types like set or list cannot be hashed. One solution is to convert them to > immutable types (fronzenset, tuple) before calling hash() on them. > {noformat} > $ thrift -version > Thrift version 0.9.3 > $ cat test.thrift > struct Test { > 1: required set test > } > $ thrift --gen py -out . test.thrift > $ cat test.py > from test.ttypes import Test > from thrift.protocol import TBinaryProtocol > from thrift.transport import TTransport > # serialize > t = Test(test=set(["a"])) > tout = TTransport.TMemoryBuffer() > pout = TBinaryProtocol.TBinaryProtocol(tout) > t.write(pout) > # deserialize > tin = TTransport.TMemoryBuffer(tout.getvalue()) > pin = TBinaryProtocol.TBinaryProtocol(tin) > t2 = Test() > t2.read(pin) > # put the deserialized object to a set > a = set([t2]) > $ python test.py > Traceback (most recent call last): > File "test.py", line 18, in > a = set([t2]) > File "/tmp/test/test/ttypes.py", line 81, in __hash__ > value = (value * 31) ^ hash(self.test) > TypeError: unhashable type: 'set' > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (THRIFT-3583) Add support for generating immutable java stubs for structs and unions
John Sirois created THRIFT-3583: --- Summary: Add support for generating immutable java stubs for structs and unions Key: THRIFT-3583 URL: https://issues.apache.org/jira/browse/THRIFT-3583 Project: Thrift Issue Type: New Feature Components: Java - Compiler, Java - Library Reporter: John Sirois In certain domains and depending on consuming project coding style, having thrift entities (structs and unions) be immutable by default is advantageous to both reasoning about code safety and correctness as well as certain performance optimizations. An example of such a domain/project is found in Apache Aurora which uses thrift to define its core data model, RPC API and as its medium for stable storage serialization. Aurora must ensure thrift structs used as keys in sets and maps are immutable and it must be able to perform bulk calculations against these objects efficiently. Both requirements are hindered by the thrift java generated code which is fundamentally mutable. Today Aurora works around this by generating immutable wrappers with a custom python script, but ideally it could just say {{thrift --gen java:immutable}} or {{thrift --gen java_immut}} and opt in to immutable generated entities. An experiment writing an immutable code generator using the Facebook swift library is discussed [here|http://markmail.org/message/a6sdqcelgokw6mwz] and implemented [here|https://reviews.apache.org/r/42748]. Performance improvements for the Aurora use cases are documented [here|https://goo.gl/gR8zgu]. The concrete proposal is to add a new mode of java codegen (I think a new gen lang, like {{java_immut}} will probably be needed vs another option to the existing {{java}} gen lang) that generates immutable thrift entities. This is harder than it sounds given the goal of leveraging the existing java support lib as much as possible since the existing supporting thrift java lib assumes mutable entities at its core with wide ripple. This starts in [TBase|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TBase.java#L37] and flows from there up through the [deserialization stack|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TDeserializer.java#L70-L86]. Although not insurmountable, some thought will need to be given to a restructuring of these mutable-assuming APIs into read and write parts that the existing interfaces can be re-composed on top of to make way for the immutable option. The alternative is to write a whole new support lib like {{javame}} has done - not a 1st choice! -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (THRIFT-3583) Add support for generating immutable java stubs for structs and unions
[ https://issues.apache.org/jira/browse/THRIFT-3583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sirois updated THRIFT-3583: Description: In certain domains and depending on consuming project coding style, having thrift entities (structs and unions) be immutable by default is advantageous to both reasoning about code safety and correctness as well as certain performance optimizations. An example of such a domain/project is found in Apache Aurora which uses thrift to define its core data model, RPC API and as its medium for stable storage serialization. Aurora must ensure thrift structs used as keys in sets and maps are immutable and it must be able to perform bulk calculations against these objects efficiently. Both requirements are hindered by the thrift java generated code which is fundamentally mutable. Today Aurora works around this by generating immutable wrappers with a custom python script, but ideally it could just say {{thrift --gen java:immutable}} or {{thrift --gen java_immut}} and opt in to immutable generated entities. An experiment writing an immutable code generator using the Facebook [swift|https://github.com/facebook/swift] library is discussed [here|http://markmail.org/message/a6sdqcelgokw6mwz] and implemented [here|https://reviews.apache.org/r/42748]. Performance improvements for the Aurora use cases are documented [here|https://goo.gl/gR8zgu]. The concrete proposal is to add a new mode of java codegen (I think a new gen lang, like {{java_immut}} will probably be needed vs another option to the existing {{java}} gen lang) that generates immutable thrift entities. This is harder than it sounds given the goal of leveraging the existing java support lib as much as possible since the existing supporting thrift java lib assumes mutable entities at its core with wide ripple. This starts in [TBase|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TBase.java#L37] and flows from there up through the [deserialization stack|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TDeserializer.java#L70-L86]. Although not insurmountable, some thought will need to be given to a restructuring of these mutable-assuming APIs into read and write parts that the existing interfaces can be re-composed on top of to make way for the immutable option. The alternative is to write a whole new support lib like {{javame}} has done - not a 1st choice! was: In certain domains and depending on consuming project coding style, having thrift entities (structs and unions) be immutable by default is advantageous to both reasoning about code safety and correctness as well as certain performance optimizations. An example of such a domain/project is found in Apache Aurora which uses thrift to define its core data model, RPC API and as its medium for stable storage serialization. Aurora must ensure thrift structs used as keys in sets and maps are immutable and it must be able to perform bulk calculations against these objects efficiently. Both requirements are hindered by the thrift java generated code which is fundamentally mutable. Today Aurora works around this by generating immutable wrappers with a custom python script, but ideally it could just say {{thrift --gen java:immutable}} or {{thrift --gen java_immut}} and opt in to immutable generated entities. An experiment writing an immutable code generator using the Facebook swift library is discussed [here|http://markmail.org/message/a6sdqcelgokw6mwz] and implemented [here|https://reviews.apache.org/r/42748]. Performance improvements for the Aurora use cases are documented [here|https://goo.gl/gR8zgu]. The concrete proposal is to add a new mode of java codegen (I think a new gen lang, like {{java_immut}} will probably be needed vs another option to the existing {{java}} gen lang) that generates immutable thrift entities. This is harder than it sounds given the goal of leveraging the existing java support lib as much as possible since the existing supporting thrift java lib assumes mutable entities at its core with wide ripple. This starts in [TBase|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TBase.java#L37] and flows from there up through the [deserialization stack|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TDeserializer.java#L70-L86]. Although not insurmountable, some thought will need to be given to a restructuring of these mutable-assuming APIs into read and write parts that the existing interfaces can be re-composed on top of to make way for the immutable option. The alternative is to write a whole new support lib like {{javame}} has done - not a 1st choice! > Add support for generating immutable java stubs for structs and unions > --
[jira] [Commented] (THRIFT-3583) Add support for generating immutable java stubs for structs and unions
[ https://issues.apache.org/jira/browse/THRIFT-3583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15121734#comment-15121734 ] John Sirois commented on THRIFT-3583: - Noting THRIFT-576 which looks to attack immutability by having 2-state objects with a mutable mode, and a frozen mode triggered with a {{freeze()}} method fwict. > Add support for generating immutable java stubs for structs and unions > -- > > Key: THRIFT-3583 > URL: https://issues.apache.org/jira/browse/THRIFT-3583 > Project: Thrift > Issue Type: New Feature > Components: Java - Compiler, Java - Library >Reporter: John Sirois > > In certain domains and depending on consuming project coding style, having > thrift entities (structs and unions) be immutable by default is advantageous > to both reasoning about code safety and correctness as well as certain > performance optimizations. An example of such a domain/project is found in > Apache Aurora which uses thrift to define its core data model, RPC API and as > its medium for stable storage serialization. Aurora must ensure thrift > structs used as keys in sets and maps are immutable and it must be able to > perform bulk calculations against these objects efficiently. Both > requirements are hindered by the thrift java generated code which is > fundamentally mutable. Today Aurora works around this by generating > immutable wrappers with a custom python script, but ideally it could just say > {{thrift --gen java:immutable}} or {{thrift --gen java_immut}} and opt in to > immutable generated entities. An experiment writing an immutable code > generator using the Facebook [swift|https://github.com/facebook/swift] > library is discussed [here|http://markmail.org/message/a6sdqcelgokw6mwz] and > implemented [here|https://reviews.apache.org/r/42748]. Performance > improvements for the Aurora use cases are documented > [here|https://goo.gl/gR8zgu]. > The concrete proposal is to add a new mode of java codegen (I think a new gen > lang, like {{java_immut}} will probably be needed vs another option to the > existing {{java}} gen lang) that generates immutable thrift entities. This > is harder than it sounds given the goal of leveraging the existing java > support lib as much as possible since the existing supporting thrift java lib > assumes mutable entities at its core with wide ripple. This starts in > [TBase|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TBase.java#L37] > and flows from there up through the [deserialization > stack|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TDeserializer.java#L70-L86]. > Although not insurmountable, some thought will need to be given to a > restructuring of these mutable-assuming APIs into read and write parts that > the existing interfaces can be re-composed on top of to make way for the > immutable option. The alternative is to write a whole new support lib like > {{javame}} has done - not a 1st choice! > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3583) Add support for generating immutable java stubs for structs and unions
[ https://issues.apache.org/jira/browse/THRIFT-3583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15121757#comment-15121757 ] John Sirois commented on THRIFT-3583: - Another class of complications comes from collections. Ideally we could adopt the unofficial offcial java std lib here and use guava {{Immutable\{List,Set,Map\}}}; however, that leads to all sorts of jar hell problems for consumers. If we implement our own immutable collections, they'll both perform worse ~by definition - its a specialized lib - and present an impedance mismatch to the guava "stdlib" users out there. In a similar vein, is @Nullable/Optional treatment. Ideally we'd support modern java coding style here and have nullable fields be represented with Optional (or at least offfer that as a codegen option), but that requires raising the floor for compatibility to java 1.8 for the immutable java codegen option. > Add support for generating immutable java stubs for structs and unions > -- > > Key: THRIFT-3583 > URL: https://issues.apache.org/jira/browse/THRIFT-3583 > Project: Thrift > Issue Type: New Feature > Components: Java - Compiler, Java - Library >Reporter: John Sirois > > In certain domains and depending on consuming project coding style, having > thrift entities (structs and unions) be immutable by default is advantageous > to both reasoning about code safety and correctness as well as certain > performance optimizations. An example of such a domain/project is found in > Apache Aurora which uses thrift to define its core data model, RPC API and as > its medium for stable storage serialization. Aurora must ensure thrift > structs used as keys in sets and maps are immutable and it must be able to > perform bulk calculations against these objects efficiently. Both > requirements are hindered by the thrift java generated code which is > fundamentally mutable. Today Aurora works around this by generating > immutable wrappers with a custom python script, but ideally it could just say > {{thrift --gen java:immutable}} or {{thrift --gen java_immut}} and opt in to > immutable generated entities. An experiment writing an immutable code > generator using the Facebook [swift|https://github.com/facebook/swift] > library is discussed [here|http://markmail.org/message/a6sdqcelgokw6mwz] and > implemented [here|https://reviews.apache.org/r/42748]. Performance > improvements for the Aurora use cases are documented > [here|https://goo.gl/gR8zgu]. > The concrete proposal is to add a new mode of java codegen (I think a new gen > lang, like {{java_immut}} will probably be needed vs another option to the > existing {{java}} gen lang) that generates immutable thrift entities. This > is harder than it sounds given the goal of leveraging the existing java > support lib as much as possible since the existing supporting thrift java lib > assumes mutable entities at its core with wide ripple. This starts in > [TBase|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TBase.java#L37] > and flows from there up through the [deserialization > stack|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TDeserializer.java#L70-L86]. > Although not insurmountable, some thought will need to be given to a > restructuring of these mutable-assuming APIs into read and write parts that > the existing interfaces can be re-composed on top of to make way for the > immutable option. The alternative is to write a whole new support lib like > {{javame}} has done - not a 1st choice! > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3583) Add support for generating immutable java stubs for structs and unions
[ https://issues.apache.org/jira/browse/THRIFT-3583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15121911#comment-15121911 ] John Sirois commented on THRIFT-3583: - Started a discussion here to vet the idea and the path to take: http://markmail.org/message/mlxyyauyvlvuxjsf > Add support for generating immutable java stubs for structs and unions > -- > > Key: THRIFT-3583 > URL: https://issues.apache.org/jira/browse/THRIFT-3583 > Project: Thrift > Issue Type: New Feature > Components: Java - Compiler, Java - Library >Reporter: John Sirois > > In certain domains and depending on consuming project coding style, having > thrift entities (structs and unions) be immutable by default is advantageous > to both reasoning about code safety and correctness as well as certain > performance optimizations. An example of such a domain/project is found in > Apache Aurora which uses thrift to define its core data model, RPC API and as > its medium for stable storage serialization. Aurora must ensure thrift > structs used as keys in sets and maps are immutable and it must be able to > perform bulk calculations against these objects efficiently. Both > requirements are hindered by the thrift java generated code which is > fundamentally mutable. Today Aurora works around this by generating > immutable wrappers with a custom python script, but ideally it could just say > {{thrift --gen java:immutable}} or {{thrift --gen java_immut}} and opt in to > immutable generated entities. An experiment writing an immutable code > generator using the Facebook [swift|https://github.com/facebook/swift] > library is discussed [here|http://markmail.org/message/a6sdqcelgokw6mwz] and > implemented [here|https://reviews.apache.org/r/42748]. Performance > improvements for the Aurora use cases are documented > [here|https://goo.gl/gR8zgu]. > The concrete proposal is to add a new mode of java codegen (I think a new gen > lang, like {{java_immut}} will probably be needed vs another option to the > existing {{java}} gen lang) that generates immutable thrift entities. This > is harder than it sounds given the goal of leveraging the existing java > support lib as much as possible since the existing supporting thrift java lib > assumes mutable entities at its core with wide ripple. This starts in > [TBase|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TBase.java#L37] > and flows from there up through the [deserialization > stack|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TDeserializer.java#L70-L86]. > Although not insurmountable, some thought will need to be given to a > restructuring of these mutable-assuming APIs into read and write parts that > the existing interfaces can be re-composed on top of to make way for the > immutable option. The alternative is to write a whole new support lib like > {{javame}} has done - not a 1st choice! > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-2157) generated code would cause ClassCastException
[ https://issues.apache.org/jira/browse/THRIFT-2157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15131098#comment-15131098 ] John Sirois commented on THRIFT-2157: - [~jensg] or [~mbreslow] Is there more coming in this fix? As things stand the ClassCastException will still be thrown fwict so the issue should be re-opened. My reasoning: >From a fresh master: {noformat} $ git log -1 commit d094e79de7e0bd61320f006c83c0de669363bce8 Author: Nobuaki Sukegawa Date: Mon Feb 1 21:47:49 2016 +0900 THRIFT-3592 Add basic test client This closes #830 {noformat} The cast to TBase is still present in the compiler generated code; yet TApplicationException is only a TSerializable now after this fix, not a TBase: {noformat} $ git grep -A1 "(org.apache.thrift.TBase)" compiler/cpp/src/generate/t_java_generator.cc compiler/cpp/src/generate/t_java_generator.cc:indent(f_service_) << "msg = (org.apache.thrift.TBase)new " compiler/cpp/src/generate/t_java_generator.cc- "org.apache.thrift.TApplicationException(org.apache.thrift." $ git grep "class TApplicationException" -- lib/java/ lib/java/src/org/apache/thrift/TApplicationException.java:public class TApplicationException extends TException implements TSerializable { $ git grep "class TException" -- lib/java/ lib/java/src/org/apache/thrift/TException.java:public class TException extends Exception { $ git grep "interface TSerializable" -- lib/java/ lib/java/src/org/apache/thrift/TSerializable.java:public interface TSerializable { {noformat} So [~mbreslow]'s change sets the stage for the full fix but does not complete it fwict. To complete this fix a change to {{compiler/cpp/src/generate/t_java_generator.cc}} code generation for {{AsyncProcessFunction.getResultHandler}}'s {{onError}} implementation is still needed as well as a fix of the signature of {{AsyncProcessFunction.sendResponse}}. A minimal finishing of the fix: {noformat} $ git diff --full-index diff --git a/compiler/cpp/src/generate/t_java_generator.cc b/compiler/cpp/src/generate/t_java_generator.cc index 7c610fb8f025e8609d01cf8bc55f498d20076848..d93fd8a28a5d9eaac46215cc51a69f5a5ec17cd2 100644 --- a/compiler/cpp/src/generate/t_java_generator.cc +++ b/compiler/cpp/src/generate/t_java_generator.cc @@ -3301,7 +3301,7 @@ void t_java_generator::generate_process_async_function(t_service* tservice, t_fu if (!tfunction->is_oneway()) { indent(f_service_) << "byte msgType = org.apache.thrift.protocol.TMessageType.REPLY;" << endl; -indent(f_service_) << "org.apache.thrift.TBase msg;" << endl; +indent(f_service_) << "org.apache.thrift.TSerializable msg;" << endl; indent(f_service_) << resultname << " result = new " << resultname << "();" << endl; t_struct* xs = tfunction->get_xceptions(); @@ -3327,7 +3327,7 @@ void t_java_generator::generate_process_async_function(t_service* tservice, t_fu indent(f_service_) << "{" << endl; indent_up(); indent(f_service_) << "msgType = org.apache.thrift.protocol.TMessageType.EXCEPTION;" << endl; -indent(f_service_) << "msg = (org.apache.thrift.TBase)new " +indent(f_service_) << "msg = new " "org.apache.thrift.TApplicationException(org.apache.thrift." "TApplicationException.INTERNAL_ERROR, e.getMessage());" << endl; indent_down(); diff --git a/lib/java/src/org/apache/thrift/AsyncProcessFunction.java b/lib/java/src/org/apache/thrift/AsyncProcessFunction.java index 799e02d58ace7a7fc29e70168f4d785a0b61599f..8537e32518844b682a555e0dcbfd417086c31597 100644 --- a/lib/java/src/org/apache/thrift/AsyncProcessFunction.java +++ b/lib/java/src/org/apache/thrift/AsyncProcessFunction.java @@ -43,7 +43,7 @@ public abstract class AsyncProcessFunction { return methodName; } -public void sendResponse(final AbstractNonblockingServer.AsyncFrameBuffer fb, final TBase result, final byte type, final int seqid) throws TException { +public void sendResponse(final AbstractNonblockingServer.AsyncFrameBuffer fb, final TSerializable result, final byte type, final int seqid) throws TException { TProtocol oprot = fb.getOutputProtocol(); oprot.writeMessageBegin(new TMessage(getMethodName(), type, seqid)); {noformat} That passes {{make check}} (well, go tests hang, but that also occurs on clean master :/, see [\[1\]|#1] below), but that said, it seems to me a unit test should be required to close out this issue as truly resolved. I'm happy to work up this missing patch with a unit test if everyone concurs the fix is incomplete. {anchor:1} \[1\] master go hang {noformat} make check ... /home/jsirois/.rvm/gems/ruby-2.1.4@global/bin/bundle exec /home/jsirois/.rvm/rubies/ruby-2.1.4/bin/ruby -I. test_suite.rb Loaded suite test_suite Started Finished in 0.0015732 seconds. --
[jira] [Commented] (THRIFT-3583) Add support for generating immutable java stubs for structs and unions
[ https://issues.apache.org/jira/browse/THRIFT-3583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15131140#comment-15131140 ] John Sirois commented on THRIFT-3583: - Ran into this 1/2-way refactoring of TBase when rebasing my branch :/ : THRIFT-2157 I'll get that finished so my refactor can properly take it into account. > Add support for generating immutable java stubs for structs and unions > -- > > Key: THRIFT-3583 > URL: https://issues.apache.org/jira/browse/THRIFT-3583 > Project: Thrift > Issue Type: New Feature > Components: Java - Compiler, Java - Library >Reporter: John Sirois > > In certain domains and depending on consuming project coding style, having > thrift entities (structs and unions) be immutable by default is advantageous > to both reasoning about code safety and correctness as well as certain > performance optimizations. An example of such a domain/project is found in > Apache Aurora which uses thrift to define its core data model, RPC API and as > its medium for stable storage serialization. Aurora must ensure thrift > structs used as keys in sets and maps are immutable and it must be able to > perform bulk calculations against these objects efficiently. Both > requirements are hindered by the thrift java generated code which is > fundamentally mutable. Today Aurora works around this by generating > immutable wrappers with a custom python script, but ideally it could just say > {{thrift --gen java:immutable}} or {{thrift --gen java_immut}} and opt in to > immutable generated entities. An experiment writing an immutable code > generator using the Facebook [swift|https://github.com/facebook/swift] > library is discussed [here|http://markmail.org/message/a6sdqcelgokw6mwz] and > implemented [here|https://reviews.apache.org/r/42748]. Performance > improvements for the Aurora use cases are documented > [here|https://goo.gl/gR8zgu]. > The concrete proposal is to add a new mode of java codegen (I think a new gen > lang, like {{java_immut}} will probably be needed vs another option to the > existing {{java}} gen lang) that generates immutable thrift entities. This > is harder than it sounds given the goal of leveraging the existing java > support lib as much as possible since the existing supporting thrift java lib > assumes mutable entities at its core with wide ripple. This starts in > [TBase|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TBase.java#L37] > and flows from there up through the [deserialization > stack|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TDeserializer.java#L70-L86]. > Although not insurmountable, some thought will need to be given to a > restructuring of these mutable-assuming APIs into read and write parts that > the existing interfaces can be re-composed on top of to make way for the > immutable option. The alternative is to write a whole new support lib like > {{javame}} has done - not a 1st choice! > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3583) Add support for generating immutable java stubs for structs and unions
[ https://issues.apache.org/jira/browse/THRIFT-3583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15131143#comment-15131143 ] John Sirois commented on THRIFT-3583: - Adding THRIFT-2157 as a blocker even though its not really one, just treating that way to clear a clean path for this work. > Add support for generating immutable java stubs for structs and unions > -- > > Key: THRIFT-3583 > URL: https://issues.apache.org/jira/browse/THRIFT-3583 > Project: Thrift > Issue Type: New Feature > Components: Java - Compiler, Java - Library >Reporter: John Sirois > > In certain domains and depending on consuming project coding style, having > thrift entities (structs and unions) be immutable by default is advantageous > to both reasoning about code safety and correctness as well as certain > performance optimizations. An example of such a domain/project is found in > Apache Aurora which uses thrift to define its core data model, RPC API and as > its medium for stable storage serialization. Aurora must ensure thrift > structs used as keys in sets and maps are immutable and it must be able to > perform bulk calculations against these objects efficiently. Both > requirements are hindered by the thrift java generated code which is > fundamentally mutable. Today Aurora works around this by generating > immutable wrappers with a custom python script, but ideally it could just say > {{thrift --gen java:immutable}} or {{thrift --gen java_immut}} and opt in to > immutable generated entities. An experiment writing an immutable code > generator using the Facebook [swift|https://github.com/facebook/swift] > library is discussed [here|http://markmail.org/message/a6sdqcelgokw6mwz] and > implemented [here|https://reviews.apache.org/r/42748]. Performance > improvements for the Aurora use cases are documented > [here|https://goo.gl/gR8zgu]. > The concrete proposal is to add a new mode of java codegen (I think a new gen > lang, like {{java_immut}} will probably be needed vs another option to the > existing {{java}} gen lang) that generates immutable thrift entities. This > is harder than it sounds given the goal of leveraging the existing java > support lib as much as possible since the existing supporting thrift java lib > assumes mutable entities at its core with wide ripple. This starts in > [TBase|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TBase.java#L37] > and flows from there up through the [deserialization > stack|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TDeserializer.java#L70-L86]. > Although not insurmountable, some thought will need to be given to a > restructuring of these mutable-assuming APIs into read and write parts that > the existing interfaces can be re-composed on top of to make way for the > immutable option. The alternative is to write a whole new support lib like > {{javame}} has done - not a 1st choice! > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (THRIFT-3597) `make check` hangs in go tests
John Sirois created THRIFT-3597: --- Summary: `make check` hangs in go tests Key: THRIFT-3597 URL: https://issues.apache.org/jira/browse/THRIFT-3597 Project: Thrift Issue Type: Bug Components: Go - Library Reporter: John Sirois I noticed this in other work and bisected to THRIFT-3251. Looks like: {noformat} git reset --hard f8ca05528e04a24b9f843c82c6600e4de5e42291 make check ... Your bundle is complete! Use `bundle show [gemname]` to see where a bundled gem is installed. /home/jsirois/.rvm/gems/ruby-2.1.4@global/bin/bundle exec /home/jsirois/.rvm/rubies/ruby-2.1.4/bin/ruby -I. test_suite.rb Loaded suite test_suite Started Finished in 0.001784889 seconds. -- 12 tests, 25 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications 100% passed -- 6723.11 tests/s, 14006.47 assertions/s make[2]: Leaving directory '/home/jsirois/dev/3rdparty/jsirois-thrift/test/rb' Making check in go make[2]: Entering directory '/home/jsirois/dev/3rdparty/jsirois-thrift/test/go' Makefile:649: warning: overriding recipe for target 'check' Makefile:498: warning: ignoring old recipe for target 'check' mkdir -p src/gen ../../compiler/cpp/thrift -out src/gen --gen go:thrift_import=thrift ThriftTest.thrift [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/go/ThriftTest.thrift:83] The "byte" type is a compatibility alias for "i8". Use "i8" to emphasize the signedness of this type. [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/go/ThriftTest.thrift:44] No generator named 'noexist' could be found! [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/go/ThriftTest.thrift:46] cpp generator does not accept 'noexist' as sub-namespace! ../../compiler/cpp/thrift -out src/gen --gen go:thrift_import=thrift ../StressTest.thrift [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/StressTest.thrift:27] The "byte" type is a compatibility alias for "i8". Use "i8" to emphasize the signedness of this type. [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/StressTest.thrift:31] Consider using the more efficient "binary" type instead of "list". [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/StressTest.thrift:31] Consider using the more efficient "binary" type instead of "list". ln -nfs ../../../lib/go/thrift src/thrift GOPATH=`pwd` /usr/bin/go get github.com/golang/mock/gomock touch gopath GOPATH=`pwd` /usr/bin/go test -v common/... === RUN TestAllConnection SIGQUIT: quit PC=0x45f9b9 m=0 goroutine 0 [idle]: runtime.epollwait(0x4, 0x7fff2e2496b0, 0x0080, 0x0, 0x, 0x0, 0x0, 0x0, 0x0, 0x0, ...) /usr/lib/go/src/runtime/sys_linux_amd64.s:420 +0x19 runtime.netpoll(0xa93501, 0x0) /usr/lib/go/src/runtime/netpoll_epoll.go:68 +0x94 runtime.findrunnable(0xc82001c000, 0x0) /usr/lib/go/src/runtime/proc1.go:1520 +0x598 runtime.schedule() /usr/lib/go/src/runtime/proc1.go:1647 +0x267 runtime.park_m(0xc820001680) /usr/lib/go/src/runtime/proc1.go:1706 +0x18b runtime.mcall(0x7fff2e249dc0) /usr/lib/go/src/runtime/asm_amd64.s:204 +0x5b goroutine 1 [chan receive]: testing.RunTests(0x93d3c0, 0xa8a5b0, 0x1, 0x1, 0x1) /usr/lib/go/src/testing/testing.go:562 +0x8ad testing.(*M).Run(0xc820051ef8, 0x0) /usr/lib/go/src/testing/testing.go:494 +0x70 main.main() common/_test/_testmain.go:54 +0x116 goroutine 17 [syscall, locked to thread]: runtime.goexit() /usr/lib/go/src/runtime/asm_amd64.s:1721 +0x1 goroutine 5 [IO wait]: net.runtime_pollWait(0x7f1b6ce4afa0, 0x72, 0xc820010250) /usr/lib/go/src/runtime/netpoll.go:157 +0x60 net.(*pollDesc).Wait(0xc82005c370, 0x72, 0x0, 0x0) /usr/lib/go/src/net/fd_poll_runtime.go:73 +0x3a net.(*pollDesc).WaitRead(0xc82005c370, 0x0, 0x0) /usr/lib/go/src/net/fd_poll_runtime.go:78 +0x36 net.(*netFD).accept(0xc82005c310, 0x0, 0x7f1b6ce4b060, 0xc82000a9c0) /usr/lib/go/src/net/fd_unix.go:408 +0x27c net.(*TCPListener).AcceptTCP(0xc820030048, 0x, 0x0, 0x0) /usr/lib/go/src/net/tcpsock_posix.go:254 +0x4d net.(*TCPListener).Accept(0xc820030048, 0x0, 0x0, 0x0, 0x0) /usr/lib/go/src/net/tcpsock_posix.go:264 +0x3d thrift.(*TServerSocket).Accept(0xc820014640, 0x0, 0x0, 0x0, 0x0) /home/jsirois/dev/3rdparty/jsirois-thrift/test/go/src/thrift/se
[jira] [Commented] (THRIFT-3597) `make check` hangs in go tests
[ https://issues.apache.org/jira/browse/THRIFT-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15131412#comment-15131412 ] John Sirois commented on THRIFT-3597: - I can't find how to self-assign but I have a patch and will post shortly. > `make check` hangs in go tests > -- > > Key: THRIFT-3597 > URL: https://issues.apache.org/jira/browse/THRIFT-3597 > Project: Thrift > Issue Type: Bug > Components: Go - Library >Reporter: John Sirois > > I noticed this in other work and bisected to THRIFT-3251. Looks like: > {noformat} > git reset --hard f8ca05528e04a24b9f843c82c6600e4de5e42291 > make check > ... > Your bundle is complete! > Use `bundle show [gemname]` to see where a bundled gem is installed. > /home/jsirois/.rvm/gems/ruby-2.1.4@global/bin/bundle exec > /home/jsirois/.rvm/rubies/ruby-2.1.4/bin/ruby -I. test_suite.rb > Loaded suite test_suite > Started > > Finished in 0.001784889 seconds. > -- > 12 tests, 25 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 > notifications > 100% passed > -- > 6723.11 tests/s, 14006.47 assertions/s > make[2]: Leaving directory '/home/jsirois/dev/3rdparty/jsirois-thrift/test/rb' > Making check in go > make[2]: Entering directory > '/home/jsirois/dev/3rdparty/jsirois-thrift/test/go' > Makefile:649: warning: overriding recipe for target 'check' > Makefile:498: warning: ignoring old recipe for target 'check' > mkdir -p src/gen > ../../compiler/cpp/thrift -out src/gen --gen go:thrift_import=thrift > ThriftTest.thrift > [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/go/ThriftTest.thrift:83] > The "byte" type is a compatibility alias for "i8". Use "i8" to emphasize the > signedness of this type. > [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/go/ThriftTest.thrift:44] > No generator named 'noexist' could be found! > [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/go/ThriftTest.thrift:46] > cpp generator does not accept 'noexist' as sub-namespace! > ../../compiler/cpp/thrift -out src/gen --gen go:thrift_import=thrift > ../StressTest.thrift > [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/StressTest.thrift:27] > The "byte" type is a compatibility alias for "i8". Use "i8" to emphasize the > signedness of this type. > [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/StressTest.thrift:31] > Consider using the more efficient "binary" type instead of "list". > [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/StressTest.thrift:31] > Consider using the more efficient "binary" type instead of "list". > ln -nfs ../../../lib/go/thrift src/thrift > GOPATH=`pwd` /usr/bin/go get github.com/golang/mock/gomock > touch gopath > GOPATH=`pwd` /usr/bin/go test -v common/... > === RUN TestAllConnection > SIGQUIT: quit > PC=0x45f9b9 m=0 > goroutine 0 [idle]: > runtime.epollwait(0x4, 0x7fff2e2496b0, 0x0080, 0x0, 0x, > 0x0, 0x0, 0x0, 0x0, 0x0, ...) > /usr/lib/go/src/runtime/sys_linux_amd64.s:420 +0x19 > runtime.netpoll(0xa93501, 0x0) > /usr/lib/go/src/runtime/netpoll_epoll.go:68 +0x94 > runtime.findrunnable(0xc82001c000, 0x0) > /usr/lib/go/src/runtime/proc1.go:1520 +0x598 > runtime.schedule() > /usr/lib/go/src/runtime/proc1.go:1647 +0x267 > runtime.park_m(0xc820001680) > /usr/lib/go/src/runtime/proc1.go:1706 +0x18b > runtime.mcall(0x7fff2e249dc0) > /usr/lib/go/src/runtime/asm_amd64.s:204 +0x5b > goroutine 1 [chan receive]: > testing.RunTests(0x93d3c0, 0xa8a5b0, 0x1, 0x1, 0x1) > /usr/lib/go/src/testing/testing.go:562 +0x8ad > testing.(*M).Run(0xc820051ef8, 0x0) > /usr/lib/go/src/testing/testing.go:494 +0x70 > main.main() > common/_test/_testmain.go:54 +0x116 > goroutine 17 [syscall, locked to thread]: > runtime.goexit() > /usr/lib/go/src/runtime/asm_amd64.s:1721 +0x1 > goroutine 5 [IO wait]: > net.runtime_pollWait(0x7f1b6ce4afa0, 0x72, 0xc820010250) > /usr/lib/go/src/runtime/netpoll.go:157 +0x60 > net.(*pollDesc).Wait(0xc82005c370, 0x72, 0x0, 0x0) > /usr/lib/go/src/net/fd_poll_runtime.go:73 +0x3a > net.(*pollDesc).WaitRead(0xc82005c370, 0x0, 0x0) > /usr/lib/go/src/net/fd_poll_runtime.go:78 +0x36 > net.(*netFD).accept(0xc82005c310, 0x0, 0x7f1b6ce4b060, 0xc82000a9c0) > /
[jira] [Commented] (THRIFT-3583) Add support for generating immutable java stubs for structs and unions
[ https://issues.apache.org/jira/browse/THRIFT-3583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15131416#comment-15131416 ] John Sirois commented on THRIFT-3583: - THRIFT-3597 doesn't really block, but hit this one too and will also fix. > Add support for generating immutable java stubs for structs and unions > -- > > Key: THRIFT-3583 > URL: https://issues.apache.org/jira/browse/THRIFT-3583 > Project: Thrift > Issue Type: New Feature > Components: Java - Compiler, Java - Library >Reporter: John Sirois > > In certain domains and depending on consuming project coding style, having > thrift entities (structs and unions) be immutable by default is advantageous > to both reasoning about code safety and correctness as well as certain > performance optimizations. An example of such a domain/project is found in > Apache Aurora which uses thrift to define its core data model, RPC API and as > its medium for stable storage serialization. Aurora must ensure thrift > structs used as keys in sets and maps are immutable and it must be able to > perform bulk calculations against these objects efficiently. Both > requirements are hindered by the thrift java generated code which is > fundamentally mutable. Today Aurora works around this by generating > immutable wrappers with a custom python script, but ideally it could just say > {{thrift --gen java:immutable}} or {{thrift --gen java_immut}} and opt in to > immutable generated entities. An experiment writing an immutable code > generator using the Facebook [swift|https://github.com/facebook/swift] > library is discussed [here|http://markmail.org/message/a6sdqcelgokw6mwz] and > implemented [here|https://reviews.apache.org/r/42748]. Performance > improvements for the Aurora use cases are documented > [here|https://goo.gl/gR8zgu]. > The concrete proposal is to add a new mode of java codegen (I think a new gen > lang, like {{java_immut}} will probably be needed vs another option to the > existing {{java}} gen lang) that generates immutable thrift entities. This > is harder than it sounds given the goal of leveraging the existing java > support lib as much as possible since the existing supporting thrift java lib > assumes mutable entities at its core with wide ripple. This starts in > [TBase|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TBase.java#L37] > and flows from there up through the [deserialization > stack|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TDeserializer.java#L70-L86]. > Although not insurmountable, some thought will need to be given to a > restructuring of these mutable-assuming APIs into read and write parts that > the existing interfaces can be re-composed on top of to make way for the > immutable option. The alternative is to write a whole new support lib like > {{javame}} has done - not a 1st choice! > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3583) Add support for generating immutable java stubs for structs and unions
[ https://issues.apache.org/jira/browse/THRIFT-3583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15131417#comment-15131417 ] John Sirois commented on THRIFT-3583: - [~jfarrell] any hints on how I self-assign this issue? Does that take magic committer powers or just better eyes? > Add support for generating immutable java stubs for structs and unions > -- > > Key: THRIFT-3583 > URL: https://issues.apache.org/jira/browse/THRIFT-3583 > Project: Thrift > Issue Type: New Feature > Components: Java - Compiler, Java - Library >Reporter: John Sirois > > In certain domains and depending on consuming project coding style, having > thrift entities (structs and unions) be immutable by default is advantageous > to both reasoning about code safety and correctness as well as certain > performance optimizations. An example of such a domain/project is found in > Apache Aurora which uses thrift to define its core data model, RPC API and as > its medium for stable storage serialization. Aurora must ensure thrift > structs used as keys in sets and maps are immutable and it must be able to > perform bulk calculations against these objects efficiently. Both > requirements are hindered by the thrift java generated code which is > fundamentally mutable. Today Aurora works around this by generating > immutable wrappers with a custom python script, but ideally it could just say > {{thrift --gen java:immutable}} or {{thrift --gen java_immut}} and opt in to > immutable generated entities. An experiment writing an immutable code > generator using the Facebook [swift|https://github.com/facebook/swift] > library is discussed [here|http://markmail.org/message/a6sdqcelgokw6mwz] and > implemented [here|https://reviews.apache.org/r/42748]. Performance > improvements for the Aurora use cases are documented > [here|https://goo.gl/gR8zgu]. > The concrete proposal is to add a new mode of java codegen (I think a new gen > lang, like {{java_immut}} will probably be needed vs another option to the > existing {{java}} gen lang) that generates immutable thrift entities. This > is harder than it sounds given the goal of leveraging the existing java > support lib as much as possible since the existing supporting thrift java lib > assumes mutable entities at its core with wide ripple. This starts in > [TBase|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TBase.java#L37] > and flows from there up through the [deserialization > stack|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TDeserializer.java#L70-L86]. > Although not insurmountable, some thought will need to be given to a > restructuring of these mutable-assuming APIs into read and write parts that > the existing interfaces can be re-composed on top of to make way for the > immutable option. The alternative is to write a whole new support lib like > {{javame}} has done - not a 1st choice! > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3597) `make check` hangs in go tests
[ https://issues.apache.org/jira/browse/THRIFT-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15131432#comment-15131432 ] John Sirois commented on THRIFT-3597: - https://github.com/apache/thrift/pull/833 > `make check` hangs in go tests > -- > > Key: THRIFT-3597 > URL: https://issues.apache.org/jira/browse/THRIFT-3597 > Project: Thrift > Issue Type: Bug > Components: Go - Library >Reporter: John Sirois > > I noticed this in other work and bisected to THRIFT-3251. Looks like: > {noformat} > git reset --hard f8ca05528e04a24b9f843c82c6600e4de5e42291 > make check > ... > Your bundle is complete! > Use `bundle show [gemname]` to see where a bundled gem is installed. > /home/jsirois/.rvm/gems/ruby-2.1.4@global/bin/bundle exec > /home/jsirois/.rvm/rubies/ruby-2.1.4/bin/ruby -I. test_suite.rb > Loaded suite test_suite > Started > > Finished in 0.001784889 seconds. > -- > 12 tests, 25 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 > notifications > 100% passed > -- > 6723.11 tests/s, 14006.47 assertions/s > make[2]: Leaving directory '/home/jsirois/dev/3rdparty/jsirois-thrift/test/rb' > Making check in go > make[2]: Entering directory > '/home/jsirois/dev/3rdparty/jsirois-thrift/test/go' > Makefile:649: warning: overriding recipe for target 'check' > Makefile:498: warning: ignoring old recipe for target 'check' > mkdir -p src/gen > ../../compiler/cpp/thrift -out src/gen --gen go:thrift_import=thrift > ThriftTest.thrift > [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/go/ThriftTest.thrift:83] > The "byte" type is a compatibility alias for "i8". Use "i8" to emphasize the > signedness of this type. > [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/go/ThriftTest.thrift:44] > No generator named 'noexist' could be found! > [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/go/ThriftTest.thrift:46] > cpp generator does not accept 'noexist' as sub-namespace! > ../../compiler/cpp/thrift -out src/gen --gen go:thrift_import=thrift > ../StressTest.thrift > [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/StressTest.thrift:27] > The "byte" type is a compatibility alias for "i8". Use "i8" to emphasize the > signedness of this type. > [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/StressTest.thrift:31] > Consider using the more efficient "binary" type instead of "list". > [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/StressTest.thrift:31] > Consider using the more efficient "binary" type instead of "list". > ln -nfs ../../../lib/go/thrift src/thrift > GOPATH=`pwd` /usr/bin/go get github.com/golang/mock/gomock > touch gopath > GOPATH=`pwd` /usr/bin/go test -v common/... > === RUN TestAllConnection > SIGQUIT: quit > PC=0x45f9b9 m=0 > goroutine 0 [idle]: > runtime.epollwait(0x4, 0x7fff2e2496b0, 0x0080, 0x0, 0x, > 0x0, 0x0, 0x0, 0x0, 0x0, ...) > /usr/lib/go/src/runtime/sys_linux_amd64.s:420 +0x19 > runtime.netpoll(0xa93501, 0x0) > /usr/lib/go/src/runtime/netpoll_epoll.go:68 +0x94 > runtime.findrunnable(0xc82001c000, 0x0) > /usr/lib/go/src/runtime/proc1.go:1520 +0x598 > runtime.schedule() > /usr/lib/go/src/runtime/proc1.go:1647 +0x267 > runtime.park_m(0xc820001680) > /usr/lib/go/src/runtime/proc1.go:1706 +0x18b > runtime.mcall(0x7fff2e249dc0) > /usr/lib/go/src/runtime/asm_amd64.s:204 +0x5b > goroutine 1 [chan receive]: > testing.RunTests(0x93d3c0, 0xa8a5b0, 0x1, 0x1, 0x1) > /usr/lib/go/src/testing/testing.go:562 +0x8ad > testing.(*M).Run(0xc820051ef8, 0x0) > /usr/lib/go/src/testing/testing.go:494 +0x70 > main.main() > common/_test/_testmain.go:54 +0x116 > goroutine 17 [syscall, locked to thread]: > runtime.goexit() > /usr/lib/go/src/runtime/asm_amd64.s:1721 +0x1 > goroutine 5 [IO wait]: > net.runtime_pollWait(0x7f1b6ce4afa0, 0x72, 0xc820010250) > /usr/lib/go/src/runtime/netpoll.go:157 +0x60 > net.(*pollDesc).Wait(0xc82005c370, 0x72, 0x0, 0x0) > /usr/lib/go/src/net/fd_poll_runtime.go:73 +0x3a > net.(*pollDesc).WaitRead(0xc82005c370, 0x0, 0x0) > /usr/lib/go/src/net/fd_poll_runtime.go:78 +0x36 > net.(*netFD).accept(0xc82005c310, 0x0, 0x7f1b6ce4b060, 0xc82000a9c0) > /usr/lib/go/src/net/fd_unix.go:40
[jira] [Commented] (THRIFT-3597) `make check` hangs in go tests
[ https://issues.apache.org/jira/browse/THRIFT-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15131433#comment-15131433 ] John Sirois commented on THRIFT-3597: - [~dimi...@gmail.com], if you have a chance to review https://github.com/apache/thrift/pull/833 I'd be grateful. > `make check` hangs in go tests > -- > > Key: THRIFT-3597 > URL: https://issues.apache.org/jira/browse/THRIFT-3597 > Project: Thrift > Issue Type: Bug > Components: Go - Library >Reporter: John Sirois > > I noticed this in other work and bisected to THRIFT-3251. Looks like: > {noformat} > git reset --hard f8ca05528e04a24b9f843c82c6600e4de5e42291 > make check > ... > Your bundle is complete! > Use `bundle show [gemname]` to see where a bundled gem is installed. > /home/jsirois/.rvm/gems/ruby-2.1.4@global/bin/bundle exec > /home/jsirois/.rvm/rubies/ruby-2.1.4/bin/ruby -I. test_suite.rb > Loaded suite test_suite > Started > > Finished in 0.001784889 seconds. > -- > 12 tests, 25 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 > notifications > 100% passed > -- > 6723.11 tests/s, 14006.47 assertions/s > make[2]: Leaving directory '/home/jsirois/dev/3rdparty/jsirois-thrift/test/rb' > Making check in go > make[2]: Entering directory > '/home/jsirois/dev/3rdparty/jsirois-thrift/test/go' > Makefile:649: warning: overriding recipe for target 'check' > Makefile:498: warning: ignoring old recipe for target 'check' > mkdir -p src/gen > ../../compiler/cpp/thrift -out src/gen --gen go:thrift_import=thrift > ThriftTest.thrift > [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/go/ThriftTest.thrift:83] > The "byte" type is a compatibility alias for "i8". Use "i8" to emphasize the > signedness of this type. > [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/go/ThriftTest.thrift:44] > No generator named 'noexist' could be found! > [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/go/ThriftTest.thrift:46] > cpp generator does not accept 'noexist' as sub-namespace! > ../../compiler/cpp/thrift -out src/gen --gen go:thrift_import=thrift > ../StressTest.thrift > [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/StressTest.thrift:27] > The "byte" type is a compatibility alias for "i8". Use "i8" to emphasize the > signedness of this type. > [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/StressTest.thrift:31] > Consider using the more efficient "binary" type instead of "list". > [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/StressTest.thrift:31] > Consider using the more efficient "binary" type instead of "list". > ln -nfs ../../../lib/go/thrift src/thrift > GOPATH=`pwd` /usr/bin/go get github.com/golang/mock/gomock > touch gopath > GOPATH=`pwd` /usr/bin/go test -v common/... > === RUN TestAllConnection > SIGQUIT: quit > PC=0x45f9b9 m=0 > goroutine 0 [idle]: > runtime.epollwait(0x4, 0x7fff2e2496b0, 0x0080, 0x0, 0x, > 0x0, 0x0, 0x0, 0x0, 0x0, ...) > /usr/lib/go/src/runtime/sys_linux_amd64.s:420 +0x19 > runtime.netpoll(0xa93501, 0x0) > /usr/lib/go/src/runtime/netpoll_epoll.go:68 +0x94 > runtime.findrunnable(0xc82001c000, 0x0) > /usr/lib/go/src/runtime/proc1.go:1520 +0x598 > runtime.schedule() > /usr/lib/go/src/runtime/proc1.go:1647 +0x267 > runtime.park_m(0xc820001680) > /usr/lib/go/src/runtime/proc1.go:1706 +0x18b > runtime.mcall(0x7fff2e249dc0) > /usr/lib/go/src/runtime/asm_amd64.s:204 +0x5b > goroutine 1 [chan receive]: > testing.RunTests(0x93d3c0, 0xa8a5b0, 0x1, 0x1, 0x1) > /usr/lib/go/src/testing/testing.go:562 +0x8ad > testing.(*M).Run(0xc820051ef8, 0x0) > /usr/lib/go/src/testing/testing.go:494 +0x70 > main.main() > common/_test/_testmain.go:54 +0x116 > goroutine 17 [syscall, locked to thread]: > runtime.goexit() > /usr/lib/go/src/runtime/asm_amd64.s:1721 +0x1 > goroutine 5 [IO wait]: > net.runtime_pollWait(0x7f1b6ce4afa0, 0x72, 0xc820010250) > /usr/lib/go/src/runtime/netpoll.go:157 +0x60 > net.(*pollDesc).Wait(0xc82005c370, 0x72, 0x0, 0x0) > /usr/lib/go/src/net/fd_poll_runtime.go:73 +0x3a > net.(*pollDesc).WaitRead(0xc82005c370, 0x0, 0x0) > /usr/lib/go/src/net/fd_poll_runtime.go:78 +0x36 > net.(*netFD).accept(0xc82005c310, 0x0,
[jira] [Commented] (THRIFT-3597) `make check` hangs in go tests
[ https://issues.apache.org/jira/browse/THRIFT-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15131518#comment-15131518 ] John Sirois commented on THRIFT-3597: - [~nsuke] - I think this is the real fix for your paper-over fix [here|https://github.com/apache/thrift/commit/2eed686406812411199f30e64131548000bbb62a#diff-354f30a63fb0907d4ad57269548329e3R158] Reviews are welcome. > `make check` hangs in go tests > -- > > Key: THRIFT-3597 > URL: https://issues.apache.org/jira/browse/THRIFT-3597 > Project: Thrift > Issue Type: Bug > Components: Go - Library >Reporter: John Sirois > > I noticed this in other work and bisected to THRIFT-3251. Looks like: > {noformat} > git reset --hard f8ca05528e04a24b9f843c82c6600e4de5e42291 > make check > ... > Your bundle is complete! > Use `bundle show [gemname]` to see where a bundled gem is installed. > /home/jsirois/.rvm/gems/ruby-2.1.4@global/bin/bundle exec > /home/jsirois/.rvm/rubies/ruby-2.1.4/bin/ruby -I. test_suite.rb > Loaded suite test_suite > Started > > Finished in 0.001784889 seconds. > -- > 12 tests, 25 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 > notifications > 100% passed > -- > 6723.11 tests/s, 14006.47 assertions/s > make[2]: Leaving directory '/home/jsirois/dev/3rdparty/jsirois-thrift/test/rb' > Making check in go > make[2]: Entering directory > '/home/jsirois/dev/3rdparty/jsirois-thrift/test/go' > Makefile:649: warning: overriding recipe for target 'check' > Makefile:498: warning: ignoring old recipe for target 'check' > mkdir -p src/gen > ../../compiler/cpp/thrift -out src/gen --gen go:thrift_import=thrift > ThriftTest.thrift > [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/go/ThriftTest.thrift:83] > The "byte" type is a compatibility alias for "i8". Use "i8" to emphasize the > signedness of this type. > [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/go/ThriftTest.thrift:44] > No generator named 'noexist' could be found! > [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/go/ThriftTest.thrift:46] > cpp generator does not accept 'noexist' as sub-namespace! > ../../compiler/cpp/thrift -out src/gen --gen go:thrift_import=thrift > ../StressTest.thrift > [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/StressTest.thrift:27] > The "byte" type is a compatibility alias for "i8". Use "i8" to emphasize the > signedness of this type. > [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/StressTest.thrift:31] > Consider using the more efficient "binary" type instead of "list". > [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/StressTest.thrift:31] > Consider using the more efficient "binary" type instead of "list". > ln -nfs ../../../lib/go/thrift src/thrift > GOPATH=`pwd` /usr/bin/go get github.com/golang/mock/gomock > touch gopath > GOPATH=`pwd` /usr/bin/go test -v common/... > === RUN TestAllConnection > SIGQUIT: quit > PC=0x45f9b9 m=0 > goroutine 0 [idle]: > runtime.epollwait(0x4, 0x7fff2e2496b0, 0x0080, 0x0, 0x, > 0x0, 0x0, 0x0, 0x0, 0x0, ...) > /usr/lib/go/src/runtime/sys_linux_amd64.s:420 +0x19 > runtime.netpoll(0xa93501, 0x0) > /usr/lib/go/src/runtime/netpoll_epoll.go:68 +0x94 > runtime.findrunnable(0xc82001c000, 0x0) > /usr/lib/go/src/runtime/proc1.go:1520 +0x598 > runtime.schedule() > /usr/lib/go/src/runtime/proc1.go:1647 +0x267 > runtime.park_m(0xc820001680) > /usr/lib/go/src/runtime/proc1.go:1706 +0x18b > runtime.mcall(0x7fff2e249dc0) > /usr/lib/go/src/runtime/asm_amd64.s:204 +0x5b > goroutine 1 [chan receive]: > testing.RunTests(0x93d3c0, 0xa8a5b0, 0x1, 0x1, 0x1) > /usr/lib/go/src/testing/testing.go:562 +0x8ad > testing.(*M).Run(0xc820051ef8, 0x0) > /usr/lib/go/src/testing/testing.go:494 +0x70 > main.main() > common/_test/_testmain.go:54 +0x116 > goroutine 17 [syscall, locked to thread]: > runtime.goexit() > /usr/lib/go/src/runtime/asm_amd64.s:1721 +0x1 > goroutine 5 [IO wait]: > net.runtime_pollWait(0x7f1b6ce4afa0, 0x72, 0xc820010250) > /usr/lib/go/src/runtime/netpoll.go:157 +0x60 > net.(*pollDesc).Wait(0xc82005c370, 0x72, 0x0, 0x0) > /usr/lib/go/src/net/fd_poll_runtime.go:73 +0x3a > net.(*pollDesc).WaitRead(0xc82005c370, 0x0
[jira] [Comment Edited] (THRIFT-3597) `make check` hangs in go tests
[ https://issues.apache.org/jira/browse/THRIFT-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15131518#comment-15131518 ] John Sirois edited comment on THRIFT-3597 at 2/4/16 1:31 AM: - [~nsuke] - I think this is the real fix for your paper-over fix [here|https://github.com/apache/thrift/commit/2eed686406812411199f30e64131548000bbb62a#diff-354f30a63fb0907d4ad57269548329e3R158] Reviews are appreciated. was (Author: jsirois): [~nsuke] - I think this is the real fix for your paper-over fix [here|https://github.com/apache/thrift/commit/2eed686406812411199f30e64131548000bbb62a#diff-354f30a63fb0907d4ad57269548329e3R158] Reviews are welcome. > `make check` hangs in go tests > -- > > Key: THRIFT-3597 > URL: https://issues.apache.org/jira/browse/THRIFT-3597 > Project: Thrift > Issue Type: Bug > Components: Go - Library >Reporter: John Sirois > > I noticed this in other work and bisected to THRIFT-3251. Looks like: > {noformat} > git reset --hard f8ca05528e04a24b9f843c82c6600e4de5e42291 > make check > ... > Your bundle is complete! > Use `bundle show [gemname]` to see where a bundled gem is installed. > /home/jsirois/.rvm/gems/ruby-2.1.4@global/bin/bundle exec > /home/jsirois/.rvm/rubies/ruby-2.1.4/bin/ruby -I. test_suite.rb > Loaded suite test_suite > Started > > Finished in 0.001784889 seconds. > -- > 12 tests, 25 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 > notifications > 100% passed > -- > 6723.11 tests/s, 14006.47 assertions/s > make[2]: Leaving directory '/home/jsirois/dev/3rdparty/jsirois-thrift/test/rb' > Making check in go > make[2]: Entering directory > '/home/jsirois/dev/3rdparty/jsirois-thrift/test/go' > Makefile:649: warning: overriding recipe for target 'check' > Makefile:498: warning: ignoring old recipe for target 'check' > mkdir -p src/gen > ../../compiler/cpp/thrift -out src/gen --gen go:thrift_import=thrift > ThriftTest.thrift > [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/go/ThriftTest.thrift:83] > The "byte" type is a compatibility alias for "i8". Use "i8" to emphasize the > signedness of this type. > [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/go/ThriftTest.thrift:44] > No generator named 'noexist' could be found! > [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/go/ThriftTest.thrift:46] > cpp generator does not accept 'noexist' as sub-namespace! > ../../compiler/cpp/thrift -out src/gen --gen go:thrift_import=thrift > ../StressTest.thrift > [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/StressTest.thrift:27] > The "byte" type is a compatibility alias for "i8". Use "i8" to emphasize the > signedness of this type. > [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/StressTest.thrift:31] > Consider using the more efficient "binary" type instead of "list". > [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/StressTest.thrift:31] > Consider using the more efficient "binary" type instead of "list". > ln -nfs ../../../lib/go/thrift src/thrift > GOPATH=`pwd` /usr/bin/go get github.com/golang/mock/gomock > touch gopath > GOPATH=`pwd` /usr/bin/go test -v common/... > === RUN TestAllConnection > SIGQUIT: quit > PC=0x45f9b9 m=0 > goroutine 0 [idle]: > runtime.epollwait(0x4, 0x7fff2e2496b0, 0x0080, 0x0, 0x, > 0x0, 0x0, 0x0, 0x0, 0x0, ...) > /usr/lib/go/src/runtime/sys_linux_amd64.s:420 +0x19 > runtime.netpoll(0xa93501, 0x0) > /usr/lib/go/src/runtime/netpoll_epoll.go:68 +0x94 > runtime.findrunnable(0xc82001c000, 0x0) > /usr/lib/go/src/runtime/proc1.go:1520 +0x598 > runtime.schedule() > /usr/lib/go/src/runtime/proc1.go:1647 +0x267 > runtime.park_m(0xc820001680) > /usr/lib/go/src/runtime/proc1.go:1706 +0x18b > runtime.mcall(0x7fff2e249dc0) > /usr/lib/go/src/runtime/asm_amd64.s:204 +0x5b > goroutine 1 [chan receive]: > testing.RunTests(0x93d3c0, 0xa8a5b0, 0x1, 0x1, 0x1) > /usr/lib/go/src/testing/testing.go:562 +0x8ad > testing.(*M).Run(0xc820051ef8, 0x0) > /usr/lib/go/src/testing/testing.go:494 +0x70 > main.main() > common/_test/_testmain.go:54 +0x116 > goroutine 17 [syscall, locked to thread]: > runtime.goexit() > /usr/lib/go/src/runtime/asm_amd64.s:1721 +0
[jira] [Assigned] (THRIFT-3583) Add support for generating immutable java stubs for structs and unions
[ https://issues.apache.org/jira/browse/THRIFT-3583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sirois reassigned THRIFT-3583: --- Assignee: John Sirois > Add support for generating immutable java stubs for structs and unions > -- > > Key: THRIFT-3583 > URL: https://issues.apache.org/jira/browse/THRIFT-3583 > Project: Thrift > Issue Type: New Feature > Components: Java - Compiler, Java - Library >Reporter: John Sirois >Assignee: John Sirois > > In certain domains and depending on consuming project coding style, having > thrift entities (structs and unions) be immutable by default is advantageous > to both reasoning about code safety and correctness as well as certain > performance optimizations. An example of such a domain/project is found in > Apache Aurora which uses thrift to define its core data model, RPC API and as > its medium for stable storage serialization. Aurora must ensure thrift > structs used as keys in sets and maps are immutable and it must be able to > perform bulk calculations against these objects efficiently. Both > requirements are hindered by the thrift java generated code which is > fundamentally mutable. Today Aurora works around this by generating > immutable wrappers with a custom python script, but ideally it could just say > {{thrift --gen java:immutable}} or {{thrift --gen java_immut}} and opt in to > immutable generated entities. An experiment writing an immutable code > generator using the Facebook [swift|https://github.com/facebook/swift] > library is discussed [here|http://markmail.org/message/a6sdqcelgokw6mwz] and > implemented [here|https://reviews.apache.org/r/42748]. Performance > improvements for the Aurora use cases are documented > [here|https://goo.gl/gR8zgu]. > The concrete proposal is to add a new mode of java codegen (I think a new gen > lang, like {{java_immut}} will probably be needed vs another option to the > existing {{java}} gen lang) that generates immutable thrift entities. This > is harder than it sounds given the goal of leveraging the existing java > support lib as much as possible since the existing supporting thrift java lib > assumes mutable entities at its core with wide ripple. This starts in > [TBase|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TBase.java#L37] > and flows from there up through the [deserialization > stack|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TDeserializer.java#L70-L86]. > Although not insurmountable, some thought will need to be given to a > restructuring of these mutable-assuming APIs into read and write parts that > the existing interfaces can be re-composed on top of to make way for the > immutable option. The alternative is to write a whole new support lib like > {{javame}} has done - not a 1st choice! > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3112) [Java] AsyncMethodCallback should be typed in generated AsyncIface
[ https://issues.apache.org/jira/browse/THRIFT-3112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15132962#comment-15132962 ] John Sirois commented on THRIFT-3112: - [~roger.meier] I have begun to dig into the {{make check}} failure here - several causes - but they reveal a good deal of broken-ness in the async client side stack. If you don't object I'd like to assign the issue to myself and carry this home. > [Java] AsyncMethodCallback should be typed in generated AsyncIface > -- > > Key: THRIFT-3112 > URL: https://issues.apache.org/jira/browse/THRIFT-3112 > Project: Thrift > Issue Type: Improvement > Components: Java - Compiler >Affects Versions: 0.9.2 >Reporter: Sergei Egorov >Assignee: Roger Meier > Labels: easyfix > > AsyncMethodCallback is generic, but current Java code generator is not adding > type info to it, and instead of: > {code:java} > public interface AsyncIface { > public void ping(org.apache.thrift.async.AsyncMethodCallback > resultHandler) throws org.apache.thrift.TException; > } > {code} > we have: > {code:java} > public interface AsyncIface { > public void ping(org.apache.thrift.async.AsyncMethodCallback > resultHandler) throws org.apache.thrift.TException; > } > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (THRIFT-2157) generated code would cause ClassCastException
[ https://issues.apache.org/jira/browse/THRIFT-2157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sirois reopened THRIFT-2157: - Assignee: John Sirois (was: Marc Breslow) Re-opening since the ClassCastException is still not solved as described below. I have a patch in-progress to complete thisbut may need to detour to fix THRIFT-3112 1st, which reveals ClassCastExceptions on the success path on client side. > generated code would cause ClassCastException > - > > Key: THRIFT-2157 > URL: https://issues.apache.org/jira/browse/THRIFT-2157 > Project: Thrift > Issue Type: Bug > Components: Java - Compiler >Affects Versions: 0.9.1 >Reporter: Dave Brosius >Assignee: John Sirois >Priority: Trivial > Fix For: 1.0 > > > Looking at the thrift generated code for cassandra, i'm seeing > msg = (org.apache.thrift.TBase)new > org.apache.thrift.TApplicationException(org.apache.thrift.TApplicationException.INTERNAL_ERROR, > e.getMessage()); > as seen here > https://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=blob;f=interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java;h=837acfc0e964249fd62720420e8f1f85d966f1a3;hb=8f202895ab9e17c3d6bd4965924fd5f1ffc27f94#l6095 > i don't see how TApplicationException can be cast to TBase, and so i'd expect > a ClassCastException there. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (THRIFT-2157) generated code would cause ClassCastException
[ https://issues.apache.org/jira/browse/THRIFT-2157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15132966#comment-15132966 ] John Sirois edited comment on THRIFT-2157 at 2/4/16 8:32 PM: - Re-opening since the ClassCastException is still not solved as described above. I have a patch in-progress to complete this but may need to detour to fix THRIFT-3112 1st, which reveals ClassCastExceptions on the success path on client side. was (Author: jsirois): Re-opening since the ClassCastException is still not solved as described below. I have a patch in-progress to complete thisbut may need to detour to fix THRIFT-3112 1st, which reveals ClassCastExceptions on the success path on client side. > generated code would cause ClassCastException > - > > Key: THRIFT-2157 > URL: https://issues.apache.org/jira/browse/THRIFT-2157 > Project: Thrift > Issue Type: Bug > Components: Java - Compiler >Affects Versions: 0.9.1 >Reporter: Dave Brosius >Assignee: John Sirois >Priority: Trivial > Fix For: 1.0 > > > Looking at the thrift generated code for cassandra, i'm seeing > msg = (org.apache.thrift.TBase)new > org.apache.thrift.TApplicationException(org.apache.thrift.TApplicationException.INTERNAL_ERROR, > e.getMessage()); > as seen here > https://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=blob;f=interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java;h=837acfc0e964249fd62720420e8f1f85d966f1a3;hb=8f202895ab9e17c3d6bd4965924fd5f1ffc27f94#l6095 > i don't see how TApplicationException can be cast to TBase, and so i'd expect > a ClassCastException there. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-2157) generated code would cause ClassCastException
[ https://issues.apache.org/jira/browse/THRIFT-2157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15132976#comment-15132976 ] John Sirois commented on THRIFT-2157: - Linking this to THRIFT-3112 which preliminary investigation of reveals similar {{ClassCastException}} issues on the {{onComplete}} path as well - though this time in the java library code. > generated code would cause ClassCastException > - > > Key: THRIFT-2157 > URL: https://issues.apache.org/jira/browse/THRIFT-2157 > Project: Thrift > Issue Type: Bug > Components: Java - Compiler >Affects Versions: 0.9.1 >Reporter: Dave Brosius >Assignee: John Sirois >Priority: Trivial > Fix For: 1.0 > > > Looking at the thrift generated code for cassandra, i'm seeing > msg = (org.apache.thrift.TBase)new > org.apache.thrift.TApplicationException(org.apache.thrift.TApplicationException.INTERNAL_ERROR, > e.getMessage()); > as seen here > https://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=blob;f=interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java;h=837acfc0e964249fd62720420e8f1f85d966f1a3;hb=8f202895ab9e17c3d6bd4965924fd5f1ffc27f94#l6095 > i don't see how TApplicationException can be cast to TBase, and so i'd expect > a ClassCastException there. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3583) Add support for generating immutable java stubs for structs and unions
[ https://issues.apache.org/jira/browse/THRIFT-3583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15132978#comment-15132978 ] John Sirois commented on THRIFT-3583: - Adding another blocker related to THRIFT-2157 and the {{ISerializable}} refactor. This is another class of {{ClassCastException}}s on the {{onComplete}} path (THRIFT-2157 deals with a {{ClassCastException}} on the {{onError}} path). > Add support for generating immutable java stubs for structs and unions > -- > > Key: THRIFT-3583 > URL: https://issues.apache.org/jira/browse/THRIFT-3583 > Project: Thrift > Issue Type: New Feature > Components: Java - Compiler, Java - Library >Reporter: John Sirois >Assignee: John Sirois > > In certain domains and depending on consuming project coding style, having > thrift entities (structs and unions) be immutable by default is advantageous > to both reasoning about code safety and correctness as well as certain > performance optimizations. An example of such a domain/project is found in > Apache Aurora which uses thrift to define its core data model, RPC API and as > its medium for stable storage serialization. Aurora must ensure thrift > structs used as keys in sets and maps are immutable and it must be able to > perform bulk calculations against these objects efficiently. Both > requirements are hindered by the thrift java generated code which is > fundamentally mutable. Today Aurora works around this by generating > immutable wrappers with a custom python script, but ideally it could just say > {{thrift --gen java:immutable}} or {{thrift --gen java_immut}} and opt in to > immutable generated entities. An experiment writing an immutable code > generator using the Facebook [swift|https://github.com/facebook/swift] > library is discussed [here|http://markmail.org/message/a6sdqcelgokw6mwz] and > implemented [here|https://reviews.apache.org/r/42748]. Performance > improvements for the Aurora use cases are documented > [here|https://goo.gl/gR8zgu]. > The concrete proposal is to add a new mode of java codegen (I think a new gen > lang, like {{java_immut}} will probably be needed vs another option to the > existing {{java}} gen lang) that generates immutable thrift entities. This > is harder than it sounds given the goal of leveraging the existing java > support lib as much as possible since the existing supporting thrift java lib > assumes mutable entities at its core with wide ripple. This starts in > [TBase|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TBase.java#L37] > and flows from there up through the [deserialization > stack|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TDeserializer.java#L70-L86]. > Although not insurmountable, some thought will need to be given to a > restructuring of these mutable-assuming APIs into read and write parts that > the existing interfaces can be re-composed on top of to make way for the > immutable option. The alternative is to write a whole new support lib like > {{javame}} has done - not a 1st choice! > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (THRIFT-3583) Add support for generating immutable java stubs for structs and unions
[ https://issues.apache.org/jira/browse/THRIFT-3583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15132978#comment-15132978 ] John Sirois edited comment on THRIFT-3583 at 2/4/16 8:41 PM: - Adding another blocker related to THRIFT-2157 and the {{ISerializable}} refactor. This is another class of {{ClassCastException}} s on the {{onComplete}} path (THRIFT-2157 deals with a {{ClassCastException}} on the {{onError}} path). was (Author: jsirois): Adding another blocker related to THRIFT-2157 and the {{ISerializable}} refactor. This is another class of {{ClassCastException}}s on the {{onComplete}} path (THRIFT-2157 deals with a {{ClassCastException}} on the {{onError}} path). > Add support for generating immutable java stubs for structs and unions > -- > > Key: THRIFT-3583 > URL: https://issues.apache.org/jira/browse/THRIFT-3583 > Project: Thrift > Issue Type: New Feature > Components: Java - Compiler, Java - Library >Reporter: John Sirois >Assignee: John Sirois > > In certain domains and depending on consuming project coding style, having > thrift entities (structs and unions) be immutable by default is advantageous > to both reasoning about code safety and correctness as well as certain > performance optimizations. An example of such a domain/project is found in > Apache Aurora which uses thrift to define its core data model, RPC API and as > its medium for stable storage serialization. Aurora must ensure thrift > structs used as keys in sets and maps are immutable and it must be able to > perform bulk calculations against these objects efficiently. Both > requirements are hindered by the thrift java generated code which is > fundamentally mutable. Today Aurora works around this by generating > immutable wrappers with a custom python script, but ideally it could just say > {{thrift --gen java:immutable}} or {{thrift --gen java_immut}} and opt in to > immutable generated entities. An experiment writing an immutable code > generator using the Facebook [swift|https://github.com/facebook/swift] > library is discussed [here|http://markmail.org/message/a6sdqcelgokw6mwz] and > implemented [here|https://reviews.apache.org/r/42748]. Performance > improvements for the Aurora use cases are documented > [here|https://goo.gl/gR8zgu]. > The concrete proposal is to add a new mode of java codegen (I think a new gen > lang, like {{java_immut}} will probably be needed vs another option to the > existing {{java}} gen lang) that generates immutable thrift entities. This > is harder than it sounds given the goal of leveraging the existing java > support lib as much as possible since the existing supporting thrift java lib > assumes mutable entities at its core with wide ripple. This starts in > [TBase|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TBase.java#L37] > and flows from there up through the [deserialization > stack|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TDeserializer.java#L70-L86]. > Although not insurmountable, some thought will need to be given to a > restructuring of these mutable-assuming APIs into read and write parts that > the existing interfaces can be re-composed on top of to make way for the > immutable option. The alternative is to write a whole new support lib like > {{javame}} has done - not a 1st choice! > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3112) [Java] AsyncMethodCallback should be typed in generated AsyncIface
[ https://issues.apache.org/jira/browse/THRIFT-3112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15134786#comment-15134786 ] John Sirois commented on THRIFT-3112: - I spoke with Roger offline and there are no objections, so assigning this to myself. > [Java] AsyncMethodCallback should be typed in generated AsyncIface > -- > > Key: THRIFT-3112 > URL: https://issues.apache.org/jira/browse/THRIFT-3112 > Project: Thrift > Issue Type: Improvement > Components: Java - Compiler >Affects Versions: 0.9.2 >Reporter: Sergei Egorov >Assignee: Roger Meier > Labels: easyfix > > AsyncMethodCallback is generic, but current Java code generator is not adding > type info to it, and instead of: > {code:java} > public interface AsyncIface { > public void ping(org.apache.thrift.async.AsyncMethodCallback > resultHandler) throws org.apache.thrift.TException; > } > {code} > we have: > {code:java} > public interface AsyncIface { > public void ping(org.apache.thrift.async.AsyncMethodCallback > resultHandler) throws org.apache.thrift.TException; > } > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (THRIFT-3112) [Java] AsyncMethodCallback should be typed in generated AsyncIface
[ https://issues.apache.org/jira/browse/THRIFT-3112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sirois reassigned THRIFT-3112: --- Assignee: John Sirois (was: Roger Meier) > [Java] AsyncMethodCallback should be typed in generated AsyncIface > -- > > Key: THRIFT-3112 > URL: https://issues.apache.org/jira/browse/THRIFT-3112 > Project: Thrift > Issue Type: Improvement > Components: Java - Compiler >Affects Versions: 0.9.2 >Reporter: Sergei Egorov >Assignee: John Sirois > Labels: easyfix > > AsyncMethodCallback is generic, but current Java code generator is not adding > type info to it, and instead of: > {code:java} > public interface AsyncIface { > public void ping(org.apache.thrift.async.AsyncMethodCallback > resultHandler) throws org.apache.thrift.TException; > } > {code} > we have: > {code:java} > public interface AsyncIface { > public void ping(org.apache.thrift.async.AsyncMethodCallback > resultHandler) throws org.apache.thrift.TException; > } > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3112) [Java] AsyncMethodCallback should be typed in generated AsyncIface
[ https://issues.apache.org/jira/browse/THRIFT-3112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137318#comment-15137318 ] John Sirois commented on THRIFT-3112: - The fundamental problem here is that the async client stack is parametrized over its implementation. An example from the tutorial "correctly" parametrized today: {code:java} package tutorial; public class Calculator { public interface AsyncIface extends shared.SharedService .AsyncIface { public void ping(org.apache.thrift.async.AsyncMethodCallback resultHandler) throws org.apache.thrift.TException; public void add(int num1, int num2, org.apache.thrift.async.AsyncMethodCallback resultHandler) throws org.apache.thrift.TException; ... } public static class AsyncClient extends shared.SharedService.AsyncClient implements AsyncIface { public void ping(org.apache.thrift.async.AsyncMethodCallback resultHandler) throws org.apache.thrift.TException { ... } public static class ping_call extends org.apache.thrift.async.TAsyncMethodCall { ... public void getResult() throws org.apache.thrift.TException { ... } } ... } public static class AsyncProcessor extends shared.SharedService.AsyncProcessor { public static class ping extends org.apache.thrift.AsyncProcessFunction { public AsyncMethodCallback getResultHandler(final AsyncFrameBuffer fb, final int seqid) { ... } } public static class add extends org.apache.thrift.AsyncProcessFunction { public AsyncMethodCallback getResultHandler(final AsyncFrameBuffer fb, final int seqid) { ... } } ... } } {code} Notice in the {{AsyncIface}} that: # The client-side AsyncMethodCallback's are not parametrized over the sync interface method return type as you'd expect, they are parametrized by the client implementation of the method call. # The server-side AsyncMethodCallback's are parametrized as you'd expect - over the sync interface method return type. The combination of 1 and 2 is confusing, but worse 1 mixes the implementation - {{AsyncClient}} - into the interface - {{AsyncIface}} and it forces clients to both call {{getResult}} in their {{onComplete}} handler as well as deal with errors. This leaves error handling split between {{onComplete}} and {{onError}}. It seems to me the right long-term answer is to fix the generated client code to match the generated server code and pass the successfull completed result to {{onComplete}} and not the {{TAsyncMethodCall}} implementation. This would be a breaking change though and anyone using AsyncIface today would need to fix there code on upgrade. A [github search|https://github.com/search?utf8=%E2%9C%93&q=new+AsyncMethodCallback+language%3AJava+extension%3Ajava&type=Code&ref=advsearch&l=Java&l=] does reveal folks actually using {{AsyncIface}} today. That said - in the 1st page of results alone there is evidence of folks doing it "wrong" and parametrizing in the natural way. That code must either be unused or else written against an Apache Thrift fork. > [Java] AsyncMethodCallback should be typed in generated AsyncIface > -- > > Key: THRIFT-3112 > URL: https://issues.apache.org/jira/browse/THRIFT-3112 > Project: Thrift > Issue Type: Improvement > Components: Java - Compiler >Affects Versions: 0.9.2 >Reporter: Sergei Egorov >Assignee: John Sirois > Labels: easyfix > > AsyncMethodCallback is generic, but current Java code generator is not adding > type info to it, and instead of: > {code:java} > public interface AsyncIface { > public void ping(org.apache.thrift.async.AsyncMethodCallback > resultHandler) throws org.apache.thrift.TException; > } > {code} > we have: > {code:java} > public interface AsyncIface { > public void ping(org.apache.thrift.async.AsyncMethodCallback > resultHandler) throws org.apache.thrift.TException; > } > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3112) [Java] AsyncMethodCallback should be typed in generated AsyncIface
[ https://issues.apache.org/jira/browse/THRIFT-3112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137540#comment-15137540 ] John Sirois commented on THRIFT-3112: - Proposal thread here: http://markmail.org/message/k77l6helvkh4igrv > [Java] AsyncMethodCallback should be typed in generated AsyncIface > -- > > Key: THRIFT-3112 > URL: https://issues.apache.org/jira/browse/THRIFT-3112 > Project: Thrift > Issue Type: Improvement > Components: Java - Compiler >Affects Versions: 0.9.2 >Reporter: Sergei Egorov >Assignee: John Sirois > Labels: easyfix > > AsyncMethodCallback is generic, but current Java code generator is not adding > type info to it, and instead of: > {code:java} > public interface AsyncIface { > public void ping(org.apache.thrift.async.AsyncMethodCallback > resultHandler) throws org.apache.thrift.TException; > } > {code} > we have: > {code:java} > public interface AsyncIface { > public void ping(org.apache.thrift.async.AsyncMethodCallback > resultHandler) throws org.apache.thrift.TException; > } > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3112) [Java] AsyncMethodCallback should be typed in generated AsyncIface
[ https://issues.apache.org/jira/browse/THRIFT-3112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137591#comment-15137591 ] John Sirois commented on THRIFT-3112: - Straw-man RB that implements the proposal: https://github.com/apache/thrift/pull/840 > [Java] AsyncMethodCallback should be typed in generated AsyncIface > -- > > Key: THRIFT-3112 > URL: https://issues.apache.org/jira/browse/THRIFT-3112 > Project: Thrift > Issue Type: Improvement > Components: Java - Compiler >Affects Versions: 0.9.2 >Reporter: Sergei Egorov >Assignee: John Sirois > Labels: easyfix > > AsyncMethodCallback is generic, but current Java code generator is not adding > type info to it, and instead of: > {code:java} > public interface AsyncIface { > public void ping(org.apache.thrift.async.AsyncMethodCallback > resultHandler) throws org.apache.thrift.TException; > } > {code} > we have: > {code:java} > public interface AsyncIface { > public void ping(org.apache.thrift.async.AsyncMethodCallback > resultHandler) throws org.apache.thrift.TException; > } > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-2157) generated code would cause ClassCastException
[ https://issues.apache.org/jira/browse/THRIFT-2157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137652#comment-15137652 ] John Sirois commented on THRIFT-2157: - Turns out THRIFT-3112 does not reveal similar {{ClassCastException}} issues if you use the generated client code correctly, but the correct usage is odd and inconsistent with the async server code. > generated code would cause ClassCastException > - > > Key: THRIFT-2157 > URL: https://issues.apache.org/jira/browse/THRIFT-2157 > Project: Thrift > Issue Type: Bug > Components: Java - Compiler >Affects Versions: 0.9.1 >Reporter: Dave Brosius >Assignee: John Sirois >Priority: Trivial > Fix For: 1.0 > > > Looking at the thrift generated code for cassandra, i'm seeing > msg = (org.apache.thrift.TBase)new > org.apache.thrift.TApplicationException(org.apache.thrift.TApplicationException.INTERNAL_ERROR, > e.getMessage()); > as seen here > https://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=blob;f=interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java;h=837acfc0e964249fd62720420e8f1f85d966f1a3;hb=8f202895ab9e17c3d6bd4965924fd5f1ffc27f94#l6095 > i don't see how TApplicationException can be cast to TBase, and so i'd expect > a ClassCastException there. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (THRIFT-3112) [Java] AsyncMethodCallback should be typed in generated AsyncIface
[ https://issues.apache.org/jira/browse/THRIFT-3112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sirois updated THRIFT-3112: Labels: (was: easyfix) > [Java] AsyncMethodCallback should be typed in generated AsyncIface > -- > > Key: THRIFT-3112 > URL: https://issues.apache.org/jira/browse/THRIFT-3112 > Project: Thrift > Issue Type: Improvement > Components: Java - Compiler, Java - Library >Affects Versions: 0.9.2 >Reporter: Sergei Egorov >Assignee: John Sirois > > AsyncMethodCallback is generic, but current Java code generator is not adding > type info to it, and instead of: > {code:java} > public interface AsyncIface { > public void ping(org.apache.thrift.async.AsyncMethodCallback > resultHandler) throws org.apache.thrift.TException; > } > {code} > we have: > {code:java} > public interface AsyncIface { > public void ping(org.apache.thrift.async.AsyncMethodCallback > resultHandler) throws org.apache.thrift.TException; > } > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (THRIFT-3112) [Java] AsyncMethodCallback should be typed in generated AsyncIface
[ https://issues.apache.org/jira/browse/THRIFT-3112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sirois updated THRIFT-3112: Component/s: Java - Library > [Java] AsyncMethodCallback should be typed in generated AsyncIface > -- > > Key: THRIFT-3112 > URL: https://issues.apache.org/jira/browse/THRIFT-3112 > Project: Thrift > Issue Type: Improvement > Components: Java - Compiler, Java - Library >Affects Versions: 0.9.2 >Reporter: Sergei Egorov >Assignee: John Sirois > > AsyncMethodCallback is generic, but current Java code generator is not adding > type info to it, and instead of: > {code:java} > public interface AsyncIface { > public void ping(org.apache.thrift.async.AsyncMethodCallback > resultHandler) throws org.apache.thrift.TException; > } > {code} > we have: > {code:java} > public interface AsyncIface { > public void ping(org.apache.thrift.async.AsyncMethodCallback > resultHandler) throws org.apache.thrift.TException; > } > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (THRIFT-2157) generated code would cause ClassCastException
[ https://issues.apache.org/jira/browse/THRIFT-2157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sirois updated THRIFT-2157: Component/s: Java - Library > generated code would cause ClassCastException > - > > Key: THRIFT-2157 > URL: https://issues.apache.org/jira/browse/THRIFT-2157 > Project: Thrift > Issue Type: Bug > Components: Java - Compiler, Java - Library >Affects Versions: 0.9.1 >Reporter: Dave Brosius >Assignee: John Sirois >Priority: Trivial > Fix For: 1.0 > > > Looking at the thrift generated code for cassandra, i'm seeing > msg = (org.apache.thrift.TBase)new > org.apache.thrift.TApplicationException(org.apache.thrift.TApplicationException.INTERNAL_ERROR, > e.getMessage()); > as seen here > https://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=blob;f=interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java;h=837acfc0e964249fd62720420e8f1f85d966f1a3;hb=8f202895ab9e17c3d6bd4965924fd5f1ffc27f94#l6095 > i don't see how TApplicationException can be cast to TBase, and so i'd expect > a ClassCastException there. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3606) TSaslClientTransport props typed too strongly
[ https://issues.apache.org/jira/browse/THRIFT-3606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15140935#comment-15140935 ] John Sirois commented on THRIFT-3606: - [~nattobean], this sounds totally reasonable to me. Are you willing to send up a patch? There are 2 methods described here: https://thrift.apache.org/docs/HowToContribute > TSaslClientTransport props typed too strongly > - > > Key: THRIFT-3606 > URL: https://issues.apache.org/jira/browse/THRIFT-3606 > Project: Thrift > Issue Type: Bug > Components: Java - Library >Affects Versions: 0.9.3 >Reporter: David Schreibman >Priority: Minor > > The constructor in TSaslClientTransport.java with signature: > public TSaslClientTransport( > String mechanism, > String authorizationId, String protocol, > String serverName, > Map props, > CallbackHandler cbh, > TTransport transport) > Is passing along the props Map to Sasl.createSaslClient(). However, > createSaslClient actually accepts props as Map. Having the props > as Map means that we cannot pass legitimate non-string > property values. For example, you cannot pass in the JGSS > javax.security.sasl.policy.credentials property which is an object of type > GSSCredential. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (THRIFT-3608) lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build.
John Sirois created THRIFT-3608: --- Summary: lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build. Key: THRIFT-3608 URL: https://issues.apache.org/jira/browse/THRIFT-3608 Project: Thrift Issue Type: Bug Components: C++ - Library, Test Suite Affects Versions: 0.9.3 Reporter: John Sirois Assignee: John Sirois Priority: Critical The [Thrift-precommit|https://builds.apache.org/job/Thrift-precommit] job now runs in the build/docker/ubuntu container and the [lib/cpp/test/SecurityTest|https://github.com/apache/thrift/blob/master/lib/cpp/test/SecurityTest.cpp] fails most - but not all - of the time. Failures can be seen in the following jobs * https://builds.apache.org/job/Thrift-precommit/60/console * https://builds.apache.org/job/Thrift-precommit/59/console * https://builds.apache.org/job/Thrift-precommit/58/console * https://builds.apache.org/job/Thrift-precommit/57/console They look like so: {noformat} argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest" Running 1 test case... Entering test suite "SecurityTest" Entering test suite "SecurityTest" Entering test case "ssl_security_matrix" TEST: Server = SSLTLS, Client = SSLTLS TEST: Server = SSLTLS, Client = SSLv3 SRV 2b9182317700 Exception: SSL_accept: wrong version number CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure TEST: Server = SSLTLS, Client = TLSv1_0 TEST: Server = SSLTLS, Client = TLSv1_1 TEST: Server = SSLTLS, Client = TLSv1_2 TEST: Server = SSLv3, Client = SSLTLS CLI 2b9181f0a700 Exception: SSL_connect: unsupported protocol SRV 2b9182317700 Exception: SSL_accept: error code: 0 TEST: Server = SSLv3, Client = SSLv3 TEST: Server = SSLv3, Client = TLSv1_0 CLI 2b9181f0a700 Exception: SSL_connect: wrong version number SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure TEST: Server = SSLv3, Client = TLSv1_1 CLI 2b9182317700 Exception: SSL_connect: wrong version number SRV 2b9181f0a700 Exception: SSL_accept: sslv3 alert handshake failure TEST: Server = SSLv3, Client = TLSv1_2 CLI 2b9181f0a700 Exception: SSL_connect: wrong version number SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure TEST: Server = TLSv1_0, Client = SSLTLS TEST: Server = TLSv1_0, Client = SSLv3 SRV 2b9182317700 Exception: SSL_accept: wrong version number CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure TEST: Server = TLSv1_0, Client = TLSv1_0 TEST: Server = TLSv1_0, Client = TLSv1_1 CLI 2b9181f0a700 Exception: SSL_connect: wrong version number SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version TEST: Server = TLSv1_0, Client = TLSv1_2 CLI 2b9182317700 Exception: SSL_connect: wrong version number SRV 2b9181f0a700 Exception: SSL_accept: tlsv1 alert protocol version TEST: Server = TLSv1_1, Client = SSLTLS TEST: Server = TLSv1_1, Client = SSLv3 SRV 2b9181f0a700 Exception: SSL_accept: wrong version number CLI 2b9182317700 Exception: SSL_connect: sslv3 alert handshake failure TEST: Server = TLSv1_1, Client = TLSv1_0 SRV 2b9182317700 Exception: SSL_accept: wrong version number CLI 2b9181f0a700 Exception: SSL_connect: tlsv1 alert protocol version TEST: Server = TLSv1_1, Client = TLSv1_1 TEST: Server = TLSv1_1, Client = TLSv1_2 SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version CLI 2b9181f0a700 Exception: SSL_connect: wrong version number TEST: Server = TLSv1_2, Client = SSLTLS [0;31mFAIL[m: SecurityTest {noformat} {noformat} argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest" Running 1 test case... Entering test suite "SecurityTest" Entering test suite "SecurityTest" Entering test case "ssl_security_matrix" TEST: Server = SSLTLS, Client = SSLTLS TEST: Server = SSLTLS, Client = SSLv3 SRV 2b6bd3245700 Exception: SSL_accept: wrong version number CLI 2b6bd2e38700 Exception: SSL_connect: sslv3 alert handshake failure TEST: Server = SSLTLS, Client = TLSv1_0 TEST: Server = SSLTLS, Client = TLSv1_1 TEST: Server = SSLTLS, Client = TLSv1_2 TEST: Server = SSLv3, Client = SSLTLS CLI 2b6bd2e38700 Exception: SSL_connect: unsupported protocol SRV 2b6bd3245700 Exception: SSL_accept: error code: 0 TEST: Server = SSLv3, Client = SSLv3 TEST: Server = SSLv3, Client = TLSv1_0 CLI 2b6bd2e38700 Exception: SSL_connect: wrong version number SRV 2b6bd3245700 Exception: SSL_accept: sslv3 alert handshake failure TEST: Server = SSLv3, Client = TLSv1_1 CLI 2b6bd3245700 Exception: SSL_connect: wrong version number SRV 2b6bd2e38700 Exception: SSL_accept: sslv3 alert handshake failure TEST: Server = SSLv3, Client = TLSv1_2 SRV 2b6bd3245700 Exception: SSL_accept: sslv3 alert handshake failure CLI 2b6bd2e38700 Exception: SSL_connect: wrong version number TEST: Server = TLSv1_0, Client = SSLTLS TEST: Server = TLSv1_0, Client = SSLv3 SRV 2b6bd3245700 Exception: SSL_accept: wrong version number CLI 2b6bd2e38700 Exception: SSL_connect: ss
[jira] [Commented] (THRIFT-3608) lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build.
[ https://issues.apache.org/jira/browse/THRIFT-3608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15141086#comment-15141086 ] John Sirois commented on THRIFT-3608: - Since there is no assert message printed out in these failures, and the failures happen at ~random points in the test matrix, it seems to me the classic evil of using a fixed port is to blame here. Before understanding the failure cases and/or asynchrony of server socket shutdown SO_LINGER, etc at-play here, I'll try converting over to using an ephemeral port which the test _should_ be doing anyway. > lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build. > - > > Key: THRIFT-3608 > URL: https://issues.apache.org/jira/browse/THRIFT-3608 > Project: Thrift > Issue Type: Bug > Components: C++ - Library, Test Suite >Affects Versions: 0.9.3 >Reporter: John Sirois >Assignee: John Sirois >Priority: Critical > > The [Thrift-precommit|https://builds.apache.org/job/Thrift-precommit] job now > runs in the build/docker/ubuntu container and the > [lib/cpp/test/SecurityTest|https://github.com/apache/thrift/blob/master/lib/cpp/test/SecurityTest.cpp] > fails most - but not all - of the time. > Failures can be seen in the following jobs > * https://builds.apache.org/job/Thrift-precommit/60/console > * https://builds.apache.org/job/Thrift-precommit/59/console > * https://builds.apache.org/job/Thrift-precommit/58/console > * https://builds.apache.org/job/Thrift-precommit/57/console > They look like so: > {noformat} > argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest" > Running 1 test case... > Entering test suite "SecurityTest" > Entering test suite "SecurityTest" > Entering test case "ssl_security_matrix" > TEST: Server = SSLTLS, Client = SSLTLS > TEST: Server = SSLTLS, Client = SSLv3 > SRV 2b9182317700 Exception: SSL_accept: wrong version number > CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = SSLTLS, Client = TLSv1_0 > TEST: Server = SSLTLS, Client = TLSv1_1 > TEST: Server = SSLTLS, Client = TLSv1_2 > TEST: Server = SSLv3, Client = SSLTLS > CLI 2b9181f0a700 Exception: SSL_connect: unsupported protocol > SRV 2b9182317700 Exception: SSL_accept: error code: 0 > TEST: Server = SSLv3, Client = SSLv3 > TEST: Server = SSLv3, Client = TLSv1_0 > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = SSLv3, Client = TLSv1_1 > CLI 2b9182317700 Exception: SSL_connect: wrong version number > SRV 2b9181f0a700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = SSLv3, Client = TLSv1_2 > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = TLSv1_0, Client = SSLTLS > TEST: Server = TLSv1_0, Client = SSLv3 > SRV 2b9182317700 Exception: SSL_accept: wrong version number > CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = TLSv1_0, Client = TLSv1_0 > TEST: Server = TLSv1_0, Client = TLSv1_1 > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version > TEST: Server = TLSv1_0, Client = TLSv1_2 > CLI 2b9182317700 Exception: SSL_connect: wrong version number > SRV 2b9181f0a700 Exception: SSL_accept: tlsv1 alert protocol version > TEST: Server = TLSv1_1, Client = SSLTLS > TEST: Server = TLSv1_1, Client = SSLv3 > SRV 2b9181f0a700 Exception: SSL_accept: wrong version number > CLI 2b9182317700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = TLSv1_1, Client = TLSv1_0 > SRV 2b9182317700 Exception: SSL_accept: wrong version number > CLI 2b9181f0a700 Exception: SSL_connect: tlsv1 alert protocol version > TEST: Server = TLSv1_1, Client = TLSv1_1 > TEST: Server = TLSv1_1, Client = TLSv1_2 > SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > TEST: Server = TLSv1_2, Client = SSLTLS > [0;31mFAIL[m: SecurityTest > {noformat} > {noformat} > argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest" > Running 1 test case... > Entering test suite "SecurityTest" > Entering test suite "SecurityTest" > Entering test case "ssl_security_matrix" > TEST: Server = SSLTLS, Client = SSLTLS > TEST: Server = SSLTLS, Client = SSLv3 > SRV 2b6bd3245700 Exception: SSL_accept: wrong version number > CLI 2b6bd2e38700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = SSLTLS, Client = TLSv1_0 > TEST: Server = SSLTLS, Client = TLSv1_1 > TEST: Server = SSLTLS, Client = TLSv1_2 > TEST: Server = SSLv3, Client = SSLTLS > CLI 2b6bd2e38700 Exception: SSL_connect: unsu
[jira] [Comment Edited] (THRIFT-3608) lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build.
[ https://issues.apache.org/jira/browse/THRIFT-3608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15141086#comment-15141086 ] John Sirois edited comment on THRIFT-3608 at 2/10/16 4:55 PM: -- Since there is no assert message printed out in these failures, and the failures happen at ~random points in the test matrix, it seems to me the classic evil of using a fixed port is to blame here and its leading to port bind failures. Before understanding the failure cases and/or asynchrony of server socket shutdown, SO_LINGER, etc that may be at-play, I'll try converting over to using an ephemeral port which the test _should_ be doing anyway. was (Author: jsirois): Since there is no assert message printed out in these failures, and the failures happen at ~random points in the test matrix, it seems to me the classic evil of using a fixed port is to blame here. Before understanding the failure cases and/or asynchrony of server socket shutdown SO_LINGER, etc at-play here, I'll try converting over to using an ephemeral port which the test _should_ be doing anyway. > lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build. > - > > Key: THRIFT-3608 > URL: https://issues.apache.org/jira/browse/THRIFT-3608 > Project: Thrift > Issue Type: Bug > Components: C++ - Library, Test Suite >Affects Versions: 0.9.3 >Reporter: John Sirois >Assignee: John Sirois >Priority: Critical > > The [Thrift-precommit|https://builds.apache.org/job/Thrift-precommit] job now > runs in the build/docker/ubuntu container and the > [lib/cpp/test/SecurityTest|https://github.com/apache/thrift/blob/master/lib/cpp/test/SecurityTest.cpp] > fails most - but not all - of the time. > Failures can be seen in the following jobs > * https://builds.apache.org/job/Thrift-precommit/60/console > * https://builds.apache.org/job/Thrift-precommit/59/console > * https://builds.apache.org/job/Thrift-precommit/58/console > * https://builds.apache.org/job/Thrift-precommit/57/console > They look like so: > {noformat} > argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest" > Running 1 test case... > Entering test suite "SecurityTest" > Entering test suite "SecurityTest" > Entering test case "ssl_security_matrix" > TEST: Server = SSLTLS, Client = SSLTLS > TEST: Server = SSLTLS, Client = SSLv3 > SRV 2b9182317700 Exception: SSL_accept: wrong version number > CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = SSLTLS, Client = TLSv1_0 > TEST: Server = SSLTLS, Client = TLSv1_1 > TEST: Server = SSLTLS, Client = TLSv1_2 > TEST: Server = SSLv3, Client = SSLTLS > CLI 2b9181f0a700 Exception: SSL_connect: unsupported protocol > SRV 2b9182317700 Exception: SSL_accept: error code: 0 > TEST: Server = SSLv3, Client = SSLv3 > TEST: Server = SSLv3, Client = TLSv1_0 > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = SSLv3, Client = TLSv1_1 > CLI 2b9182317700 Exception: SSL_connect: wrong version number > SRV 2b9181f0a700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = SSLv3, Client = TLSv1_2 > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = TLSv1_0, Client = SSLTLS > TEST: Server = TLSv1_0, Client = SSLv3 > SRV 2b9182317700 Exception: SSL_accept: wrong version number > CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = TLSv1_0, Client = TLSv1_0 > TEST: Server = TLSv1_0, Client = TLSv1_1 > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version > TEST: Server = TLSv1_0, Client = TLSv1_2 > CLI 2b9182317700 Exception: SSL_connect: wrong version number > SRV 2b9181f0a700 Exception: SSL_accept: tlsv1 alert protocol version > TEST: Server = TLSv1_1, Client = SSLTLS > TEST: Server = TLSv1_1, Client = SSLv3 > SRV 2b9181f0a700 Exception: SSL_accept: wrong version number > CLI 2b9182317700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = TLSv1_1, Client = TLSv1_0 > SRV 2b9182317700 Exception: SSL_accept: wrong version number > CLI 2b9181f0a700 Exception: SSL_connect: tlsv1 alert protocol version > TEST: Server = TLSv1_1, Client = TLSv1_1 > TEST: Server = TLSv1_1, Client = TLSv1_2 > SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > TEST: Server = TLSv1_2, Client = SSLTLS > [0;31mFAIL[m: SecurityTest > {noformat} > {noformat} > argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest" > Running 1 test case... > Entering test su
[jira] [Created] (THRIFT-3609) Remove or replace TestPortFixture.h
John Sirois created THRIFT-3609: --- Summary: Remove or replace TestPortFixture.h Key: THRIFT-3609 URL: https://issues.apache.org/jira/browse/THRIFT-3609 Project: Thrift Issue Type: Improvement Components: C++ - Library, Test Suite Reporter: John Sirois Assignee: John Sirois Today 6 cpp lib tests use [TestPortFixture.h|https://github.com/apache/thrift/blob/1f6e380c5d07686e4cd8c2b172300a1ba7fbd8b9/lib/cpp/test/TestPortFixture.h]. THRIFT-3608 corrects one of these cases, but a general cleanup should remove all uses of static ports in the cpp tests to improve robustness and open the path to parallel testing - at least in the cpp libs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3608) lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build.
[ https://issues.apache.org/jira/browse/THRIFT-3608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15141258#comment-15141258 ] John Sirois commented on THRIFT-3608: - https://github.com/apache/thrift/pull/841 > lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build. > - > > Key: THRIFT-3608 > URL: https://issues.apache.org/jira/browse/THRIFT-3608 > Project: Thrift > Issue Type: Bug > Components: C++ - Library, Test Suite >Affects Versions: 0.9.3 >Reporter: John Sirois >Assignee: John Sirois >Priority: Critical > > The [Thrift-precommit|https://builds.apache.org/job/Thrift-precommit] job now > runs in the build/docker/ubuntu container and the > [lib/cpp/test/SecurityTest|https://github.com/apache/thrift/blob/master/lib/cpp/test/SecurityTest.cpp] > fails most - but not all - of the time. > Failures can be seen in the following jobs > * https://builds.apache.org/job/Thrift-precommit/60/console > * https://builds.apache.org/job/Thrift-precommit/59/console > * https://builds.apache.org/job/Thrift-precommit/58/console > * https://builds.apache.org/job/Thrift-precommit/57/console > They look like so: > {noformat} > argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest" > Running 1 test case... > Entering test suite "SecurityTest" > Entering test suite "SecurityTest" > Entering test case "ssl_security_matrix" > TEST: Server = SSLTLS, Client = SSLTLS > TEST: Server = SSLTLS, Client = SSLv3 > SRV 2b9182317700 Exception: SSL_accept: wrong version number > CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = SSLTLS, Client = TLSv1_0 > TEST: Server = SSLTLS, Client = TLSv1_1 > TEST: Server = SSLTLS, Client = TLSv1_2 > TEST: Server = SSLv3, Client = SSLTLS > CLI 2b9181f0a700 Exception: SSL_connect: unsupported protocol > SRV 2b9182317700 Exception: SSL_accept: error code: 0 > TEST: Server = SSLv3, Client = SSLv3 > TEST: Server = SSLv3, Client = TLSv1_0 > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = SSLv3, Client = TLSv1_1 > CLI 2b9182317700 Exception: SSL_connect: wrong version number > SRV 2b9181f0a700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = SSLv3, Client = TLSv1_2 > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = TLSv1_0, Client = SSLTLS > TEST: Server = TLSv1_0, Client = SSLv3 > SRV 2b9182317700 Exception: SSL_accept: wrong version number > CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = TLSv1_0, Client = TLSv1_0 > TEST: Server = TLSv1_0, Client = TLSv1_1 > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version > TEST: Server = TLSv1_0, Client = TLSv1_2 > CLI 2b9182317700 Exception: SSL_connect: wrong version number > SRV 2b9181f0a700 Exception: SSL_accept: tlsv1 alert protocol version > TEST: Server = TLSv1_1, Client = SSLTLS > TEST: Server = TLSv1_1, Client = SSLv3 > SRV 2b9181f0a700 Exception: SSL_accept: wrong version number > CLI 2b9182317700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = TLSv1_1, Client = TLSv1_0 > SRV 2b9182317700 Exception: SSL_accept: wrong version number > CLI 2b9181f0a700 Exception: SSL_connect: tlsv1 alert protocol version > TEST: Server = TLSv1_1, Client = TLSv1_1 > TEST: Server = TLSv1_1, Client = TLSv1_2 > SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > TEST: Server = TLSv1_2, Client = SSLTLS > [0;31mFAIL[m: SecurityTest > {noformat} > {noformat} > argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest" > Running 1 test case... > Entering test suite "SecurityTest" > Entering test suite "SecurityTest" > Entering test case "ssl_security_matrix" > TEST: Server = SSLTLS, Client = SSLTLS > TEST: Server = SSLTLS, Client = SSLv3 > SRV 2b6bd3245700 Exception: SSL_accept: wrong version number > CLI 2b6bd2e38700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = SSLTLS, Client = TLSv1_0 > TEST: Server = SSLTLS, Client = TLSv1_1 > TEST: Server = SSLTLS, Client = TLSv1_2 > TEST: Server = SSLv3, Client = SSLTLS > CLI 2b6bd2e38700 Exception: SSL_connect: unsupported protocol > SRV 2b6bd3245700 Exception: SSL_accept: error code: 0 > TEST: Server = SSLv3, Client = SSLv3 > TEST: Server = SSLv3, Client = TLSv1_0 > CLI 2b6bd2e38700 Exception: SSL_connect: wrong version number > SRV 2b6bd3245700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = SSLv3, Client = TLSv1_1 > CLI 2b6bd3245700 Exception: SSL_con
[jira] [Commented] (THRIFT-3608) lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build.
[ https://issues.apache.org/jira/browse/THRIFT-3608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15143283#comment-15143283 ] John Sirois commented on THRIFT-3608: - Real fix is here: https://github.com/apache/thrift/pull/848 The issue was a bad {{#ifdef linux}} - should be {{#ifdef \_\_linux\_\_}}. Under standards compliance modes, both g++ and clang++ will not define the deprecated {{linux}} symbol. We compile using {{-std=c++11}} and so {{SecutityTest}} would fail to find {{linux}} defined and thus fail to setup ignoring {{SIGPIPE}} which led to flaky test failures. > lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build. > - > > Key: THRIFT-3608 > URL: https://issues.apache.org/jira/browse/THRIFT-3608 > Project: Thrift > Issue Type: Bug > Components: C++ - Library, Test Suite >Affects Versions: 0.9.3 >Reporter: John Sirois >Assignee: John Sirois >Priority: Critical > > The [Thrift-precommit|https://builds.apache.org/job/Thrift-precommit] job now > runs in the build/docker/ubuntu container and the > [lib/cpp/test/SecurityTest|https://github.com/apache/thrift/blob/master/lib/cpp/test/SecurityTest.cpp] > fails most - but not all - of the time. > Failures can be seen in the following jobs > * https://builds.apache.org/job/Thrift-precommit/60/console > * https://builds.apache.org/job/Thrift-precommit/59/console > * https://builds.apache.org/job/Thrift-precommit/58/console > * https://builds.apache.org/job/Thrift-precommit/57/console > They look like so: > {noformat} > argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest" > Running 1 test case... > Entering test suite "SecurityTest" > Entering test suite "SecurityTest" > Entering test case "ssl_security_matrix" > TEST: Server = SSLTLS, Client = SSLTLS > TEST: Server = SSLTLS, Client = SSLv3 > SRV 2b9182317700 Exception: SSL_accept: wrong version number > CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = SSLTLS, Client = TLSv1_0 > TEST: Server = SSLTLS, Client = TLSv1_1 > TEST: Server = SSLTLS, Client = TLSv1_2 > TEST: Server = SSLv3, Client = SSLTLS > CLI 2b9181f0a700 Exception: SSL_connect: unsupported protocol > SRV 2b9182317700 Exception: SSL_accept: error code: 0 > TEST: Server = SSLv3, Client = SSLv3 > TEST: Server = SSLv3, Client = TLSv1_0 > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = SSLv3, Client = TLSv1_1 > CLI 2b9182317700 Exception: SSL_connect: wrong version number > SRV 2b9181f0a700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = SSLv3, Client = TLSv1_2 > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = TLSv1_0, Client = SSLTLS > TEST: Server = TLSv1_0, Client = SSLv3 > SRV 2b9182317700 Exception: SSL_accept: wrong version number > CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = TLSv1_0, Client = TLSv1_0 > TEST: Server = TLSv1_0, Client = TLSv1_1 > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version > TEST: Server = TLSv1_0, Client = TLSv1_2 > CLI 2b9182317700 Exception: SSL_connect: wrong version number > SRV 2b9181f0a700 Exception: SSL_accept: tlsv1 alert protocol version > TEST: Server = TLSv1_1, Client = SSLTLS > TEST: Server = TLSv1_1, Client = SSLv3 > SRV 2b9181f0a700 Exception: SSL_accept: wrong version number > CLI 2b9182317700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = TLSv1_1, Client = TLSv1_0 > SRV 2b9182317700 Exception: SSL_accept: wrong version number > CLI 2b9181f0a700 Exception: SSL_connect: tlsv1 alert protocol version > TEST: Server = TLSv1_1, Client = TLSv1_1 > TEST: Server = TLSv1_1, Client = TLSv1_2 > SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > TEST: Server = TLSv1_2, Client = SSLTLS > [0;31mFAIL[m: SecurityTest > {noformat} > {noformat} > argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest" > Running 1 test case... > Entering test suite "SecurityTest" > Entering test suite "SecurityTest" > Entering test case "ssl_security_matrix" > TEST: Server = SSLTLS, Client = SSLTLS > TEST: Server = SSLTLS, Client = SSLv3 > SRV 2b6bd3245700 Exception: SSL_accept: wrong version number > CLI 2b6bd2e38700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = SSLTLS, Client = TLSv1_0 > TEST: Server = SSLTLS, Client = TLSv1_1 > TEST: Server = SSLTLS, Client = TLSv1_2 > TEST: Server = SSLv3, Client = SSLTLS > CLI 2b6bd2e38700 Exception: SSL_connect
[jira] [Commented] (THRIFT-3608) lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build.
[ https://issues.apache.org/jira/browse/THRIFT-3608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15143290#comment-15143290 ] John Sirois commented on THRIFT-3608: - For posterity - this trick helped seal the diagnosis: {noformat} $ touch foo.cc $ g++ -dM -E foo.cc | grep linux #define __linux 1 #define __linux__ 1 #define __gnu_linux__ 1 #define linux 1 $ g++ -std=c++11 -dM -E foo.cc | grep linux #define __linux 1 #define __linux__ 1 #define __gnu_linux__ 1 $ clang++ -dM -E foo.cc | grep linux #define __gnu_linux__ 1 #define __linux 1 #define __linux__ 1 #define linux 1 $ clang++ -std=c++11 -dM -E foo.cc | grep linux #define __gnu_linux__ 1 #define __linux 1 #define __linux__ 1 {noformat} > lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build. > - > > Key: THRIFT-3608 > URL: https://issues.apache.org/jira/browse/THRIFT-3608 > Project: Thrift > Issue Type: Bug > Components: C++ - Library, Test Suite >Affects Versions: 0.9.3 >Reporter: John Sirois >Assignee: John Sirois >Priority: Critical > > The [Thrift-precommit|https://builds.apache.org/job/Thrift-precommit] job now > runs in the build/docker/ubuntu container and the > [lib/cpp/test/SecurityTest|https://github.com/apache/thrift/blob/master/lib/cpp/test/SecurityTest.cpp] > fails most - but not all - of the time. > Failures can be seen in the following jobs > * https://builds.apache.org/job/Thrift-precommit/60/console > * https://builds.apache.org/job/Thrift-precommit/59/console > * https://builds.apache.org/job/Thrift-precommit/58/console > * https://builds.apache.org/job/Thrift-precommit/57/console > They look like so: > {noformat} > argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest" > Running 1 test case... > Entering test suite "SecurityTest" > Entering test suite "SecurityTest" > Entering test case "ssl_security_matrix" > TEST: Server = SSLTLS, Client = SSLTLS > TEST: Server = SSLTLS, Client = SSLv3 > SRV 2b9182317700 Exception: SSL_accept: wrong version number > CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = SSLTLS, Client = TLSv1_0 > TEST: Server = SSLTLS, Client = TLSv1_1 > TEST: Server = SSLTLS, Client = TLSv1_2 > TEST: Server = SSLv3, Client = SSLTLS > CLI 2b9181f0a700 Exception: SSL_connect: unsupported protocol > SRV 2b9182317700 Exception: SSL_accept: error code: 0 > TEST: Server = SSLv3, Client = SSLv3 > TEST: Server = SSLv3, Client = TLSv1_0 > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = SSLv3, Client = TLSv1_1 > CLI 2b9182317700 Exception: SSL_connect: wrong version number > SRV 2b9181f0a700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = SSLv3, Client = TLSv1_2 > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = TLSv1_0, Client = SSLTLS > TEST: Server = TLSv1_0, Client = SSLv3 > SRV 2b9182317700 Exception: SSL_accept: wrong version number > CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = TLSv1_0, Client = TLSv1_0 > TEST: Server = TLSv1_0, Client = TLSv1_1 > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version > TEST: Server = TLSv1_0, Client = TLSv1_2 > CLI 2b9182317700 Exception: SSL_connect: wrong version number > SRV 2b9181f0a700 Exception: SSL_accept: tlsv1 alert protocol version > TEST: Server = TLSv1_1, Client = SSLTLS > TEST: Server = TLSv1_1, Client = SSLv3 > SRV 2b9181f0a700 Exception: SSL_accept: wrong version number > CLI 2b9182317700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = TLSv1_1, Client = TLSv1_0 > SRV 2b9182317700 Exception: SSL_accept: wrong version number > CLI 2b9181f0a700 Exception: SSL_connect: tlsv1 alert protocol version > TEST: Server = TLSv1_1, Client = TLSv1_1 > TEST: Server = TLSv1_1, Client = TLSv1_2 > SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > TEST: Server = TLSv1_2, Client = SSLTLS > [0;31mFAIL[m: SecurityTest > {noformat} > {noformat} > argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest" > Running 1 test case... > Entering test suite "SecurityTest" > Entering test suite "SecurityTest" > Entering test case "ssl_security_matrix" > TEST: Server = SSLTLS, Client = SSLTLS > TEST: Server = SSLTLS, Client = SSLv3 > SRV 2b6bd3245700 Exception: SSL_accept: wrong version number > CLI 2b6bd2e38700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = SSLTLS, Client = TLSv1_0 > TEST: Server = SSLTLS, Client = TLSv1_1 > TES
[jira] [Issue Comment Deleted] (THRIFT-3608) lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build.
[ https://issues.apache.org/jira/browse/THRIFT-3608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sirois updated THRIFT-3608: Comment: was deleted (was: https://github.com/apache/thrift/pull/841) > lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build. > - > > Key: THRIFT-3608 > URL: https://issues.apache.org/jira/browse/THRIFT-3608 > Project: Thrift > Issue Type: Bug > Components: C++ - Library, Test Suite >Affects Versions: 0.9.3 >Reporter: John Sirois >Assignee: John Sirois >Priority: Critical > > The [Thrift-precommit|https://builds.apache.org/job/Thrift-precommit] job now > runs in the build/docker/ubuntu container and the > [lib/cpp/test/SecurityTest|https://github.com/apache/thrift/blob/master/lib/cpp/test/SecurityTest.cpp] > fails most - but not all - of the time. > Failures can be seen in the following jobs > * https://builds.apache.org/job/Thrift-precommit/60/console > * https://builds.apache.org/job/Thrift-precommit/59/console > * https://builds.apache.org/job/Thrift-precommit/58/console > * https://builds.apache.org/job/Thrift-precommit/57/console > They look like so: > {noformat} > argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest" > Running 1 test case... > Entering test suite "SecurityTest" > Entering test suite "SecurityTest" > Entering test case "ssl_security_matrix" > TEST: Server = SSLTLS, Client = SSLTLS > TEST: Server = SSLTLS, Client = SSLv3 > SRV 2b9182317700 Exception: SSL_accept: wrong version number > CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = SSLTLS, Client = TLSv1_0 > TEST: Server = SSLTLS, Client = TLSv1_1 > TEST: Server = SSLTLS, Client = TLSv1_2 > TEST: Server = SSLv3, Client = SSLTLS > CLI 2b9181f0a700 Exception: SSL_connect: unsupported protocol > SRV 2b9182317700 Exception: SSL_accept: error code: 0 > TEST: Server = SSLv3, Client = SSLv3 > TEST: Server = SSLv3, Client = TLSv1_0 > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = SSLv3, Client = TLSv1_1 > CLI 2b9182317700 Exception: SSL_connect: wrong version number > SRV 2b9181f0a700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = SSLv3, Client = TLSv1_2 > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = TLSv1_0, Client = SSLTLS > TEST: Server = TLSv1_0, Client = SSLv3 > SRV 2b9182317700 Exception: SSL_accept: wrong version number > CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = TLSv1_0, Client = TLSv1_0 > TEST: Server = TLSv1_0, Client = TLSv1_1 > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version > TEST: Server = TLSv1_0, Client = TLSv1_2 > CLI 2b9182317700 Exception: SSL_connect: wrong version number > SRV 2b9181f0a700 Exception: SSL_accept: tlsv1 alert protocol version > TEST: Server = TLSv1_1, Client = SSLTLS > TEST: Server = TLSv1_1, Client = SSLv3 > SRV 2b9181f0a700 Exception: SSL_accept: wrong version number > CLI 2b9182317700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = TLSv1_1, Client = TLSv1_0 > SRV 2b9182317700 Exception: SSL_accept: wrong version number > CLI 2b9181f0a700 Exception: SSL_connect: tlsv1 alert protocol version > TEST: Server = TLSv1_1, Client = TLSv1_1 > TEST: Server = TLSv1_1, Client = TLSv1_2 > SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > TEST: Server = TLSv1_2, Client = SSLTLS > [0;31mFAIL[m: SecurityTest > {noformat} > {noformat} > argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest" > Running 1 test case... > Entering test suite "SecurityTest" > Entering test suite "SecurityTest" > Entering test case "ssl_security_matrix" > TEST: Server = SSLTLS, Client = SSLTLS > TEST: Server = SSLTLS, Client = SSLv3 > SRV 2b6bd3245700 Exception: SSL_accept: wrong version number > CLI 2b6bd2e38700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = SSLTLS, Client = TLSv1_0 > TEST: Server = SSLTLS, Client = TLSv1_1 > TEST: Server = SSLTLS, Client = TLSv1_2 > TEST: Server = SSLv3, Client = SSLTLS > CLI 2b6bd2e38700 Exception: SSL_connect: unsupported protocol > SRV 2b6bd3245700 Exception: SSL_accept: error code: 0 > TEST: Server = SSLv3, Client = SSLv3 > TEST: Server = SSLv3, Client = TLSv1_0 > CLI 2b6bd2e38700 Exception: SSL_connect: wrong version number > SRV 2b6bd3245700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = SSLv3, Client = TLSv1_1 > CLI 2b6bd3245700 Exception: SSL_connect: wrong version numbe
[jira] [Commented] (THRIFT-3608) lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build.
[ https://issues.apache.org/jira/browse/THRIFT-3608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15143369#comment-15143369 ] John Sirois commented on THRIFT-3608: - Thanks [~jensg] - I'll comment over there and close that bug as duplicate to this since I have a solution spinning in jenkins CI now. > lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build. > - > > Key: THRIFT-3608 > URL: https://issues.apache.org/jira/browse/THRIFT-3608 > Project: Thrift > Issue Type: Bug > Components: C++ - Library, Test Suite >Affects Versions: 0.9.3 >Reporter: John Sirois >Assignee: John Sirois >Priority: Critical > > The [Thrift-precommit|https://builds.apache.org/job/Thrift-precommit] job now > runs in the build/docker/ubuntu container and the > [lib/cpp/test/SecurityTest|https://github.com/apache/thrift/blob/master/lib/cpp/test/SecurityTest.cpp] > fails most - but not all - of the time. > Failures can be seen in the following jobs > * https://builds.apache.org/job/Thrift-precommit/60/console > * https://builds.apache.org/job/Thrift-precommit/59/console > * https://builds.apache.org/job/Thrift-precommit/58/console > * https://builds.apache.org/job/Thrift-precommit/57/console > They look like so: > {noformat} > argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest" > Running 1 test case... > Entering test suite "SecurityTest" > Entering test suite "SecurityTest" > Entering test case "ssl_security_matrix" > TEST: Server = SSLTLS, Client = SSLTLS > TEST: Server = SSLTLS, Client = SSLv3 > SRV 2b9182317700 Exception: SSL_accept: wrong version number > CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = SSLTLS, Client = TLSv1_0 > TEST: Server = SSLTLS, Client = TLSv1_1 > TEST: Server = SSLTLS, Client = TLSv1_2 > TEST: Server = SSLv3, Client = SSLTLS > CLI 2b9181f0a700 Exception: SSL_connect: unsupported protocol > SRV 2b9182317700 Exception: SSL_accept: error code: 0 > TEST: Server = SSLv3, Client = SSLv3 > TEST: Server = SSLv3, Client = TLSv1_0 > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = SSLv3, Client = TLSv1_1 > CLI 2b9182317700 Exception: SSL_connect: wrong version number > SRV 2b9181f0a700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = SSLv3, Client = TLSv1_2 > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = TLSv1_0, Client = SSLTLS > TEST: Server = TLSv1_0, Client = SSLv3 > SRV 2b9182317700 Exception: SSL_accept: wrong version number > CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = TLSv1_0, Client = TLSv1_0 > TEST: Server = TLSv1_0, Client = TLSv1_1 > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version > TEST: Server = TLSv1_0, Client = TLSv1_2 > CLI 2b9182317700 Exception: SSL_connect: wrong version number > SRV 2b9181f0a700 Exception: SSL_accept: tlsv1 alert protocol version > TEST: Server = TLSv1_1, Client = SSLTLS > TEST: Server = TLSv1_1, Client = SSLv3 > SRV 2b9181f0a700 Exception: SSL_accept: wrong version number > CLI 2b9182317700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = TLSv1_1, Client = TLSv1_0 > SRV 2b9182317700 Exception: SSL_accept: wrong version number > CLI 2b9181f0a700 Exception: SSL_connect: tlsv1 alert protocol version > TEST: Server = TLSv1_1, Client = TLSv1_1 > TEST: Server = TLSv1_1, Client = TLSv1_2 > SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > TEST: Server = TLSv1_2, Client = SSLTLS > [0;31mFAIL[m: SecurityTest > {noformat} > {noformat} > argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest" > Running 1 test case... > Entering test suite "SecurityTest" > Entering test suite "SecurityTest" > Entering test case "ssl_security_matrix" > TEST: Server = SSLTLS, Client = SSLTLS > TEST: Server = SSLTLS, Client = SSLv3 > SRV 2b6bd3245700 Exception: SSL_accept: wrong version number > CLI 2b6bd2e38700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = SSLTLS, Client = TLSv1_0 > TEST: Server = SSLTLS, Client = TLSv1_1 > TEST: Server = SSLTLS, Client = TLSv1_2 > TEST: Server = SSLv3, Client = SSLTLS > CLI 2b6bd2e38700 Exception: SSL_connect: unsupported protocol > SRV 2b6bd3245700 Exception: SSL_accept: error code: 0 > TEST: Server = SSLv3, Client = SSLv3 > TEST: Server = SSLv3, Client = TLSv1_0 > CLI 2b6bd2e38700 Exception: SSL_connect: wrong version number > SRV 2b6bd3245700 Exception: SSL_accept: sslv3 alert han
[jira] [Commented] (THRIFT-3362) make check fails for C++ at the SecurityTest
[ https://issues.apache.org/jira/browse/THRIFT-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15143375#comment-15143375 ] John Sirois commented on THRIFT-3362: - Noting that I figured this out over in THRIFT-3608. The issue was {{#ifdef linux}} and {{SIGPIPE}} signal handlers not getting installed. I did 1st go down the path of using ephemeral ports, and will address that in THRIFT-3609 since it turned out not to be the issue, but would be good to clean up (globally) anyhow. > make check fails for C++ at the SecurityTest > > > Key: THRIFT-3362 > URL: https://issues.apache.org/jira/browse/THRIFT-3362 > Project: Thrift > Issue Type: Bug > Components: C++ - Library, Test Suite > Environment: openSUSE >Reporter: Jens Geyer >Assignee: James E. King, III > > When I run "make check" inside {{lib/cpp}} I get this error: > {code} > FAIL: SecurityTest > {code} > No more information besides that, and all other tests succeed. When I run > the SecurityTest test alone, it succeeds - it fails only with make check. I > have no idea why, and I can't get the test routine to print more log info to > the console with make check. I already tried adding --log_level > and --report_level to no avail, again this works well when run standalone. > I'm a bit stuck here. Maybe the problem lies in my setup, but I have no idea > what to look for. Any helpful advice would be really great. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (THRIFT-3362) make check fails for C++ at the SecurityTest
[ https://issues.apache.org/jira/browse/THRIFT-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sirois resolved THRIFT-3362. - Resolution: Duplicate Resolving as duplicate of THRIFT-3608 which has a fix spinning through CI here: https://github.com/apache/thrift/pull/848 > make check fails for C++ at the SecurityTest > > > Key: THRIFT-3362 > URL: https://issues.apache.org/jira/browse/THRIFT-3362 > Project: Thrift > Issue Type: Bug > Components: C++ - Library, Test Suite > Environment: openSUSE >Reporter: Jens Geyer >Assignee: James E. King, III > > When I run "make check" inside {{lib/cpp}} I get this error: > {code} > FAIL: SecurityTest > {code} > No more information besides that, and all other tests succeed. When I run > the SecurityTest test alone, it succeeds - it fails only with make check. I > have no idea why, and I can't get the test routine to print more log info to > the console with make check. I already tried adding --log_level > and --report_level to no avail, again this works well when run standalone. > I'm a bit stuck here. Maybe the problem lies in my setup, but I have no idea > what to look for. Any helpful advice would be really great. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3608) lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build.
[ https://issues.apache.org/jira/browse/THRIFT-3608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15143423#comment-15143423 ] John Sirois commented on THRIFT-3608: - Passing now though failing at another ~commmon flaky in python: https://builds.apache.org/job/Thrift-precommit/86/console > lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build. > - > > Key: THRIFT-3608 > URL: https://issues.apache.org/jira/browse/THRIFT-3608 > Project: Thrift > Issue Type: Bug > Components: C++ - Library, Test Suite >Affects Versions: 0.9.3 >Reporter: John Sirois >Assignee: John Sirois >Priority: Critical > > The [Thrift-precommit|https://builds.apache.org/job/Thrift-precommit] job now > runs in the build/docker/ubuntu container and the > [lib/cpp/test/SecurityTest|https://github.com/apache/thrift/blob/master/lib/cpp/test/SecurityTest.cpp] > fails most - but not all - of the time. > Failures can be seen in the following jobs > * https://builds.apache.org/job/Thrift-precommit/60/console > * https://builds.apache.org/job/Thrift-precommit/59/console > * https://builds.apache.org/job/Thrift-precommit/58/console > * https://builds.apache.org/job/Thrift-precommit/57/console > They look like so: > {noformat} > argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest" > Running 1 test case... > Entering test suite "SecurityTest" > Entering test suite "SecurityTest" > Entering test case "ssl_security_matrix" > TEST: Server = SSLTLS, Client = SSLTLS > TEST: Server = SSLTLS, Client = SSLv3 > SRV 2b9182317700 Exception: SSL_accept: wrong version number > CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = SSLTLS, Client = TLSv1_0 > TEST: Server = SSLTLS, Client = TLSv1_1 > TEST: Server = SSLTLS, Client = TLSv1_2 > TEST: Server = SSLv3, Client = SSLTLS > CLI 2b9181f0a700 Exception: SSL_connect: unsupported protocol > SRV 2b9182317700 Exception: SSL_accept: error code: 0 > TEST: Server = SSLv3, Client = SSLv3 > TEST: Server = SSLv3, Client = TLSv1_0 > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = SSLv3, Client = TLSv1_1 > CLI 2b9182317700 Exception: SSL_connect: wrong version number > SRV 2b9181f0a700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = SSLv3, Client = TLSv1_2 > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = TLSv1_0, Client = SSLTLS > TEST: Server = TLSv1_0, Client = SSLv3 > SRV 2b9182317700 Exception: SSL_accept: wrong version number > CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = TLSv1_0, Client = TLSv1_0 > TEST: Server = TLSv1_0, Client = TLSv1_1 > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version > TEST: Server = TLSv1_0, Client = TLSv1_2 > CLI 2b9182317700 Exception: SSL_connect: wrong version number > SRV 2b9181f0a700 Exception: SSL_accept: tlsv1 alert protocol version > TEST: Server = TLSv1_1, Client = SSLTLS > TEST: Server = TLSv1_1, Client = SSLv3 > SRV 2b9181f0a700 Exception: SSL_accept: wrong version number > CLI 2b9182317700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = TLSv1_1, Client = TLSv1_0 > SRV 2b9182317700 Exception: SSL_accept: wrong version number > CLI 2b9181f0a700 Exception: SSL_connect: tlsv1 alert protocol version > TEST: Server = TLSv1_1, Client = TLSv1_1 > TEST: Server = TLSv1_1, Client = TLSv1_2 > SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > TEST: Server = TLSv1_2, Client = SSLTLS > [0;31mFAIL[m: SecurityTest > {noformat} > {noformat} > argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest" > Running 1 test case... > Entering test suite "SecurityTest" > Entering test suite "SecurityTest" > Entering test case "ssl_security_matrix" > TEST: Server = SSLTLS, Client = SSLTLS > TEST: Server = SSLTLS, Client = SSLv3 > SRV 2b6bd3245700 Exception: SSL_accept: wrong version number > CLI 2b6bd2e38700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = SSLTLS, Client = TLSv1_0 > TEST: Server = SSLTLS, Client = TLSv1_1 > TEST: Server = SSLTLS, Client = TLSv1_2 > TEST: Server = SSLv3, Client = SSLTLS > CLI 2b6bd2e38700 Exception: SSL_connect: unsupported protocol > SRV 2b6bd3245700 Exception: SSL_accept: error code: 0 > TEST: Server = SSLv3, Client = SSLv3 > TEST: Server = SSLv3, Client = TLSv1_0 > CLI 2b6bd2e38700 Exception: SSL_connect: wrong version number > SRV 2b6bd3245700 Exception: SSL_accept: sslv3 alert handshake failu
[jira] [Commented] (THRIFT-3608) lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build.
[ https://issues.apache.org/jira/browse/THRIFT-3608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15143424#comment-15143424 ] John Sirois commented on THRIFT-3608: - [~jensg] - If you can confirm this fixes the issue you had as described in THRIFT-3362 I think its ready to merge. > lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build. > - > > Key: THRIFT-3608 > URL: https://issues.apache.org/jira/browse/THRIFT-3608 > Project: Thrift > Issue Type: Bug > Components: C++ - Library, Test Suite >Affects Versions: 0.9.3 >Reporter: John Sirois >Assignee: John Sirois >Priority: Critical > > The [Thrift-precommit|https://builds.apache.org/job/Thrift-precommit] job now > runs in the build/docker/ubuntu container and the > [lib/cpp/test/SecurityTest|https://github.com/apache/thrift/blob/master/lib/cpp/test/SecurityTest.cpp] > fails most - but not all - of the time. > Failures can be seen in the following jobs > * https://builds.apache.org/job/Thrift-precommit/60/console > * https://builds.apache.org/job/Thrift-precommit/59/console > * https://builds.apache.org/job/Thrift-precommit/58/console > * https://builds.apache.org/job/Thrift-precommit/57/console > They look like so: > {noformat} > argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest" > Running 1 test case... > Entering test suite "SecurityTest" > Entering test suite "SecurityTest" > Entering test case "ssl_security_matrix" > TEST: Server = SSLTLS, Client = SSLTLS > TEST: Server = SSLTLS, Client = SSLv3 > SRV 2b9182317700 Exception: SSL_accept: wrong version number > CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = SSLTLS, Client = TLSv1_0 > TEST: Server = SSLTLS, Client = TLSv1_1 > TEST: Server = SSLTLS, Client = TLSv1_2 > TEST: Server = SSLv3, Client = SSLTLS > CLI 2b9181f0a700 Exception: SSL_connect: unsupported protocol > SRV 2b9182317700 Exception: SSL_accept: error code: 0 > TEST: Server = SSLv3, Client = SSLv3 > TEST: Server = SSLv3, Client = TLSv1_0 > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = SSLv3, Client = TLSv1_1 > CLI 2b9182317700 Exception: SSL_connect: wrong version number > SRV 2b9181f0a700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = SSLv3, Client = TLSv1_2 > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = TLSv1_0, Client = SSLTLS > TEST: Server = TLSv1_0, Client = SSLv3 > SRV 2b9182317700 Exception: SSL_accept: wrong version number > CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = TLSv1_0, Client = TLSv1_0 > TEST: Server = TLSv1_0, Client = TLSv1_1 > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version > TEST: Server = TLSv1_0, Client = TLSv1_2 > CLI 2b9182317700 Exception: SSL_connect: wrong version number > SRV 2b9181f0a700 Exception: SSL_accept: tlsv1 alert protocol version > TEST: Server = TLSv1_1, Client = SSLTLS > TEST: Server = TLSv1_1, Client = SSLv3 > SRV 2b9181f0a700 Exception: SSL_accept: wrong version number > CLI 2b9182317700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = TLSv1_1, Client = TLSv1_0 > SRV 2b9182317700 Exception: SSL_accept: wrong version number > CLI 2b9181f0a700 Exception: SSL_connect: tlsv1 alert protocol version > TEST: Server = TLSv1_1, Client = TLSv1_1 > TEST: Server = TLSv1_1, Client = TLSv1_2 > SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > TEST: Server = TLSv1_2, Client = SSLTLS > [0;31mFAIL[m: SecurityTest > {noformat} > {noformat} > argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest" > Running 1 test case... > Entering test suite "SecurityTest" > Entering test suite "SecurityTest" > Entering test case "ssl_security_matrix" > TEST: Server = SSLTLS, Client = SSLTLS > TEST: Server = SSLTLS, Client = SSLv3 > SRV 2b6bd3245700 Exception: SSL_accept: wrong version number > CLI 2b6bd2e38700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = SSLTLS, Client = TLSv1_0 > TEST: Server = SSLTLS, Client = TLSv1_1 > TEST: Server = SSLTLS, Client = TLSv1_2 > TEST: Server = SSLv3, Client = SSLTLS > CLI 2b6bd2e38700 Exception: SSL_connect: unsupported protocol > SRV 2b6bd3245700 Exception: SSL_accept: error code: 0 > TEST: Server = SSLv3, Client = SSLv3 > TEST: Server = SSLv3, Client = TLSv1_0 > CLI 2b6bd2e38700 Exception: SSL_connect: wrong version number > SRV 2b6bd3245700 Exception: SSL_accept: sslv3 alert handshake failure > TE
[jira] [Commented] (THRIFT-3362) make check fails for C++ at the SecurityTest
[ https://issues.apache.org/jira/browse/THRIFT-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15143434#comment-15143434 ] John Sirois commented on THRIFT-3362: - Yeah - realized that late, but I spent 2 days figuring it out so I reserve the right to reverse the flow of time ;) > make check fails for C++ at the SecurityTest > > > Key: THRIFT-3362 > URL: https://issues.apache.org/jira/browse/THRIFT-3362 > Project: Thrift > Issue Type: Bug > Components: C++ - Library, Test Suite > Environment: openSUSE >Reporter: Jens Geyer >Assignee: James E. King, III > > When I run "make check" inside {{lib/cpp}} I get this error: > {code} > FAIL: SecurityTest > {code} > No more information besides that, and all other tests succeed. When I run > the SecurityTest test alone, it succeeds - it fails only with make check. I > have no idea why, and I can't get the test routine to print more log info to > the console with make check. I already tried adding --log_level > and --report_level to no avail, again this works well when run standalone. > I'm a bit stuck here. Maybe the problem lies in my setup, but I have no idea > what to look for. Any helpful advice would be really great. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (THRIFT-3616) lib/py/test/test_sslsocket.py is flaky
John Sirois created THRIFT-3616: --- Summary: lib/py/test/test_sslsocket.py is flaky Key: THRIFT-3616 URL: https://issues.apache.org/jira/browse/THRIFT-3616 Project: Thrift Issue Type: Bug Components: Python - Library, Test Suite Reporter: John Sirois Assignee: John Sirois Priority: Critical As seen here: https://builds.apache.org/job/Thrift-precommit/86/console {noformat} ... /usr/bin/python3 test/thrift_json.py . -- Ran 1 test in 0.011s OK /usr/bin/python3 test/test_sslsocket.py .FFFsys:1: ResourceWarning: unclosed F/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:253: DeprecationWarning: validate is deprecated. Use cert_reqs=ssl.CERT_REQUIRED instead DeprecationWarning) /thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:253: DeprecationWarning: validate is deprecated. Use cert_reqs=ssl.CERT_NONE instead DeprecationWarning) .sys:1: ResourceWarning: unclosed F/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:265: DeprecationWarning: Use cert_reqs instead warnings.warn('Use cert_reqs instead', DeprecationWarning) .sys:1: ResourceWarning: unclosed Exception in thread Thread-11: Traceback (most recent call last): File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner self.run() File "test/test_sslsocket.py", line 60, in run self.client = self._server.accept() File "/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py", line 418, in accept client = self._wrap_socket(plain_client) File "/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py", line 181, in _wrap_socket server_hostname=self._server_hostname) File "/usr/lib/python3.4/ssl.py", line 365, in wrap_socket _context=self) File "/usr/lib/python3.4/ssl.py", line 601, in __init__ self.do_handshake() File "/usr/lib/python3.4/ssl.py", line 828, in do_handshake self._sslobj.do_handshake() ConnectionResetError: [Errno 104] Connection reset by peer PROTOCOL_SSLv2 is not available ..test/test_sslsocket.py:59: ResourceWarning: unclosed self._server.listen() . == FAIL: test_client_cert (__main__.TSSLSocketTest) -- Traceback (most recent call last): File "test/test_sslsocket.py", line 210, in test_client_cert self._assert_connection_success(server, client) File "test/test_sslsocket.py", line 112, in _assert_connection_success self.assertTrue(acc.client is not None) AssertionError: False is not true == FAIL: test_deprecation (__main__.TSSLSocketTest) -- Traceback (most recent call last): File "test/test_sslsocket.py", line 138, in test_deprecation self.assertEqual(len(w), 3) AssertionError: 4 != 3 == FAIL: test_newer_tls (__main__.TSSLSocketTest) -- Traceback (most recent call last): File "test/test_sslsocket.py", line 268, in test_newer_tls self._assert_connection_success(server, client) File "test/test_sslsocket.py", line 112, in _assert_connection_success self.assertTrue(acc.client is not None) AssertionError: False is not true == FAIL: test_server_cert (__main__.TSSLSocketTest) -- Traceback (most recent call last): File "test/test_sslsocket.py", line 172, in test_server_cert self._assert_connection_success(server, client) File "test/test_sslsocket.py", line 112, in _assert_connection_success self.assertTrue(acc.client is not None) AssertionError: False is not true == FAIL: test_set_server_cert (__main__.TSSLSocketTest) -- Traceback (most recent call last): File "test/test_sslsocket.py", line 191, in test_set_server_cert self._assert_connection_success(server, client) File "test/test_sslsocket.py", line 112, in _assert_connection_success self.assertTrue(acc.client is not None) AssertionError: False is not true -- Ran 11 tests in 5.240s FAILED (failures=5) make[3]: *** [py3-test] Error 1 make[3]: Leaving directory `/thrift/lib/py' make[2]: *** [check-am] Error 2 make[2]: Leaving directory `/thrift/lib/py' make[1]: *** [check-recursive] Error 1 make[1]: Leaving directory `/thrift/lib' make: *** [check-recursive] Error 1 Build step 'Execute shell' marked build a
[jira] [Commented] (THRIFT-3608) lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build.
[ https://issues.apache.org/jira/browse/THRIFT-3608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15143459#comment-15143459 ] John Sirois commented on THRIFT-3608: - I filed this for the flaky python issue: THRIFT-3616 > lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build. > - > > Key: THRIFT-3608 > URL: https://issues.apache.org/jira/browse/THRIFT-3608 > Project: Thrift > Issue Type: Bug > Components: C++ - Library, Test Suite >Affects Versions: 0.9.3 >Reporter: John Sirois >Assignee: John Sirois >Priority: Critical > > The [Thrift-precommit|https://builds.apache.org/job/Thrift-precommit] job now > runs in the build/docker/ubuntu container and the > [lib/cpp/test/SecurityTest|https://github.com/apache/thrift/blob/master/lib/cpp/test/SecurityTest.cpp] > fails most - but not all - of the time. > Failures can be seen in the following jobs > * https://builds.apache.org/job/Thrift-precommit/60/console > * https://builds.apache.org/job/Thrift-precommit/59/console > * https://builds.apache.org/job/Thrift-precommit/58/console > * https://builds.apache.org/job/Thrift-precommit/57/console > They look like so: > {noformat} > argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest" > Running 1 test case... > Entering test suite "SecurityTest" > Entering test suite "SecurityTest" > Entering test case "ssl_security_matrix" > TEST: Server = SSLTLS, Client = SSLTLS > TEST: Server = SSLTLS, Client = SSLv3 > SRV 2b9182317700 Exception: SSL_accept: wrong version number > CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = SSLTLS, Client = TLSv1_0 > TEST: Server = SSLTLS, Client = TLSv1_1 > TEST: Server = SSLTLS, Client = TLSv1_2 > TEST: Server = SSLv3, Client = SSLTLS > CLI 2b9181f0a700 Exception: SSL_connect: unsupported protocol > SRV 2b9182317700 Exception: SSL_accept: error code: 0 > TEST: Server = SSLv3, Client = SSLv3 > TEST: Server = SSLv3, Client = TLSv1_0 > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = SSLv3, Client = TLSv1_1 > CLI 2b9182317700 Exception: SSL_connect: wrong version number > SRV 2b9181f0a700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = SSLv3, Client = TLSv1_2 > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = TLSv1_0, Client = SSLTLS > TEST: Server = TLSv1_0, Client = SSLv3 > SRV 2b9182317700 Exception: SSL_accept: wrong version number > CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = TLSv1_0, Client = TLSv1_0 > TEST: Server = TLSv1_0, Client = TLSv1_1 > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version > TEST: Server = TLSv1_0, Client = TLSv1_2 > CLI 2b9182317700 Exception: SSL_connect: wrong version number > SRV 2b9181f0a700 Exception: SSL_accept: tlsv1 alert protocol version > TEST: Server = TLSv1_1, Client = SSLTLS > TEST: Server = TLSv1_1, Client = SSLv3 > SRV 2b9181f0a700 Exception: SSL_accept: wrong version number > CLI 2b9182317700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = TLSv1_1, Client = TLSv1_0 > SRV 2b9182317700 Exception: SSL_accept: wrong version number > CLI 2b9181f0a700 Exception: SSL_connect: tlsv1 alert protocol version > TEST: Server = TLSv1_1, Client = TLSv1_1 > TEST: Server = TLSv1_1, Client = TLSv1_2 > SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > TEST: Server = TLSv1_2, Client = SSLTLS > [0;31mFAIL[m: SecurityTest > {noformat} > {noformat} > argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest" > Running 1 test case... > Entering test suite "SecurityTest" > Entering test suite "SecurityTest" > Entering test case "ssl_security_matrix" > TEST: Server = SSLTLS, Client = SSLTLS > TEST: Server = SSLTLS, Client = SSLv3 > SRV 2b6bd3245700 Exception: SSL_accept: wrong version number > CLI 2b6bd2e38700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = SSLTLS, Client = TLSv1_0 > TEST: Server = SSLTLS, Client = TLSv1_1 > TEST: Server = SSLTLS, Client = TLSv1_2 > TEST: Server = SSLv3, Client = SSLTLS > CLI 2b6bd2e38700 Exception: SSL_connect: unsupported protocol > SRV 2b6bd3245700 Exception: SSL_accept: error code: 0 > TEST: Server = SSLv3, Client = SSLv3 > TEST: Server = SSLv3, Client = TLSv1_0 > CLI 2b6bd2e38700 Exception: SSL_connect: wrong version number > SRV 2b6bd3245700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = SSLv3, Client = TLSv1_1 > CLI 2b6bd3245700 Excepti
[jira] [Commented] (THRIFT-3616) lib/py/test/test_sslsocket.py is flaky
[ https://issues.apache.org/jira/browse/THRIFT-3616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15143462#comment-15143462 ] John Sirois commented on THRIFT-3616: - Seen multiple times over the course of working THRIFT-3608 > lib/py/test/test_sslsocket.py is flaky > -- > > Key: THRIFT-3616 > URL: https://issues.apache.org/jira/browse/THRIFT-3616 > Project: Thrift > Issue Type: Bug > Components: Python - Library, Test Suite >Reporter: John Sirois >Assignee: John Sirois >Priority: Critical > > As seen here: https://builds.apache.org/job/Thrift-precommit/86/console > {noformat} > ... > /usr/bin/python3 test/thrift_json.py > . > -- > Ran 1 test in 0.011s > OK > /usr/bin/python3 test/test_sslsocket.py > .FFFsys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0, > laddr=('::1', 23458, 0, 0), raddr=('::1', 41151, 0, 0)> > F/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:253: > DeprecationWarning: validate is deprecated. Use cert_reqs=ssl.CERT_REQUIRED > instead > DeprecationWarning) > /thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:253: > DeprecationWarning: validate is deprecated. Use cert_reqs=ssl.CERT_NONE > instead > DeprecationWarning) > .sys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0, > laddr=('::1', 23458, 0, 0), raddr=('::1', 41155, 0, 0)> > F/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:265: > DeprecationWarning: Use cert_reqs instead > warnings.warn('Use cert_reqs instead', DeprecationWarning) > .sys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0, > laddr=('::1', 23458, 0, 0), raddr=('::1', 41157, 0, 0)> > Exception in thread Thread-11: > Traceback (most recent call last): > File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner > self.run() > File "test/test_sslsocket.py", line 60, in run > self.client = self._server.accept() > File "/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py", line 418, > in accept > client = self._wrap_socket(plain_client) > File "/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py", line 181, > in _wrap_socket > server_hostname=self._server_hostname) > File "/usr/lib/python3.4/ssl.py", line 365, in wrap_socket > _context=self) > File "/usr/lib/python3.4/ssl.py", line 601, in __init__ > self.do_handshake() > File "/usr/lib/python3.4/ssl.py", line 828, in do_handshake > self._sslobj.do_handshake() > ConnectionResetError: [Errno 104] Connection reset by peer > PROTOCOL_SSLv2 is not available > ..test/test_sslsocket.py:59: ResourceWarning: unclosed family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0> > self._server.listen() > . > == > FAIL: test_client_cert (__main__.TSSLSocketTest) > -- > Traceback (most recent call last): > File "test/test_sslsocket.py", line 210, in test_client_cert > self._assert_connection_success(server, client) > File "test/test_sslsocket.py", line 112, in _assert_connection_success > self.assertTrue(acc.client is not None) > AssertionError: False is not true > == > FAIL: test_deprecation (__main__.TSSLSocketTest) > -- > Traceback (most recent call last): > File "test/test_sslsocket.py", line 138, in test_deprecation > self.assertEqual(len(w), 3) > AssertionError: 4 != 3 > == > FAIL: test_newer_tls (__main__.TSSLSocketTest) > -- > Traceback (most recent call last): > File "test/test_sslsocket.py", line 268, in test_newer_tls > self._assert_connection_success(server, client) > File "test/test_sslsocket.py", line 112, in _assert_connection_success > self.assertTrue(acc.client is not None) > AssertionError: False is not true > == > FAIL: test_server_cert (__main__.TSSLSocketTest) > -- > Traceback (most recent call last): > File "test/test_sslsocket.py", line 172, in test_server_cert > self._assert_connection_success(server, client) > File "test/test_sslsocket.py", line 112, in _assert_connection_success > self.assertTrue(acc.client is not None) > AssertionError: False is not true > ==
[jira] [Commented] (THRIFT-3616) lib/py/test/test_sslsocket.py is flaky
[ https://issues.apache.org/jira/browse/THRIFT-3616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15143466#comment-15143466 ] John Sirois commented on THRIFT-3616: - /cc [~nsuke] - I'm going to work this presently since I'm on a quest to get a green jenkins Thrift-precommit job, but I may turn it over to you since you're expert in this slice of the codebase if I have trouble making progress today. > lib/py/test/test_sslsocket.py is flaky > -- > > Key: THRIFT-3616 > URL: https://issues.apache.org/jira/browse/THRIFT-3616 > Project: Thrift > Issue Type: Bug > Components: Python - Library, Test Suite >Reporter: John Sirois >Assignee: John Sirois >Priority: Critical > > As seen here: https://builds.apache.org/job/Thrift-precommit/86/console > {noformat} > ... > /usr/bin/python3 test/thrift_json.py > . > -- > Ran 1 test in 0.011s > OK > /usr/bin/python3 test/test_sslsocket.py > .FFFsys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0, > laddr=('::1', 23458, 0, 0), raddr=('::1', 41151, 0, 0)> > F/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:253: > DeprecationWarning: validate is deprecated. Use cert_reqs=ssl.CERT_REQUIRED > instead > DeprecationWarning) > /thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:253: > DeprecationWarning: validate is deprecated. Use cert_reqs=ssl.CERT_NONE > instead > DeprecationWarning) > .sys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0, > laddr=('::1', 23458, 0, 0), raddr=('::1', 41155, 0, 0)> > F/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:265: > DeprecationWarning: Use cert_reqs instead > warnings.warn('Use cert_reqs instead', DeprecationWarning) > .sys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0, > laddr=('::1', 23458, 0, 0), raddr=('::1', 41157, 0, 0)> > Exception in thread Thread-11: > Traceback (most recent call last): > File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner > self.run() > File "test/test_sslsocket.py", line 60, in run > self.client = self._server.accept() > File "/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py", line 418, > in accept > client = self._wrap_socket(plain_client) > File "/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py", line 181, > in _wrap_socket > server_hostname=self._server_hostname) > File "/usr/lib/python3.4/ssl.py", line 365, in wrap_socket > _context=self) > File "/usr/lib/python3.4/ssl.py", line 601, in __init__ > self.do_handshake() > File "/usr/lib/python3.4/ssl.py", line 828, in do_handshake > self._sslobj.do_handshake() > ConnectionResetError: [Errno 104] Connection reset by peer > PROTOCOL_SSLv2 is not available > ..test/test_sslsocket.py:59: ResourceWarning: unclosed family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0> > self._server.listen() > . > == > FAIL: test_client_cert (__main__.TSSLSocketTest) > -- > Traceback (most recent call last): > File "test/test_sslsocket.py", line 210, in test_client_cert > self._assert_connection_success(server, client) > File "test/test_sslsocket.py", line 112, in _assert_connection_success > self.assertTrue(acc.client is not None) > AssertionError: False is not true > == > FAIL: test_deprecation (__main__.TSSLSocketTest) > -- > Traceback (most recent call last): > File "test/test_sslsocket.py", line 138, in test_deprecation > self.assertEqual(len(w), 3) > AssertionError: 4 != 3 > == > FAIL: test_newer_tls (__main__.TSSLSocketTest) > -- > Traceback (most recent call last): > File "test/test_sslsocket.py", line 268, in test_newer_tls > self._assert_connection_success(server, client) > File "test/test_sslsocket.py", line 112, in _assert_connection_success > self.assertTrue(acc.client is not None) > AssertionError: False is not true > == > FAIL: test_server_cert (__main__.TSSLSocketTest) > -- > Traceback (most recent call last): > File "test/test_sslsocket.py", line 172, in test_server_cert > self._assert_connection_success(server, client) > File "test/test_ssl
[jira] [Updated] (THRIFT-3609) Remove or replace TestPortFixture.h
[ https://issues.apache.org/jira/browse/THRIFT-3609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sirois updated THRIFT-3609: Description: Today 6 cpp lib tests use [TestPortFixture.h|https://github.com/apache/thrift/blob/1f6e380c5d07686e4cd8c2b172300a1ba7fbd8b9/lib/cpp/test/TestPortFixture.h]. THRIFT-3608 experimented with correcting one of these cases in {{lib/cpp/test/SecurityTest.cpp}}, but a general cleanup should remove all uses of static ports in the cpp tests to improve robustness and open the path to parallel testing - at least in the cpp libs. (was: Today 6 cpp lib tests use [TestPortFixture.h|https://github.com/apache/thrift/blob/1f6e380c5d07686e4cd8c2b172300a1ba7fbd8b9/lib/cpp/test/TestPortFixture.h]. THRIFT-3608 corrects one of these cases, but a general cleanup should remove all uses of static ports in the cpp tests to improve robustness and open the path to parallel testing - at least in the cpp libs.) > Remove or replace TestPortFixture.h > --- > > Key: THRIFT-3609 > URL: https://issues.apache.org/jira/browse/THRIFT-3609 > Project: Thrift > Issue Type: Improvement > Components: C++ - Library, Test Suite >Reporter: John Sirois >Assignee: John Sirois > > Today 6 cpp lib tests use > [TestPortFixture.h|https://github.com/apache/thrift/blob/1f6e380c5d07686e4cd8c2b172300a1ba7fbd8b9/lib/cpp/test/TestPortFixture.h]. > THRIFT-3608 experimented with correcting one of these cases in > {{lib/cpp/test/SecurityTest.cpp}}, but a general cleanup should remove all > uses of static ports in the cpp tests to improve robustness and open the path > to parallel testing - at least in the cpp libs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3616) lib/py/test/test_sslsocket.py is flaky
[ https://issues.apache.org/jira/browse/THRIFT-3616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15143542#comment-15143542 ] John Sirois commented on THRIFT-3616: - OK - I have a delay-free way to fix this that also speeds up the test, just need to polish and I'll have a PR shortly. > lib/py/test/test_sslsocket.py is flaky > -- > > Key: THRIFT-3616 > URL: https://issues.apache.org/jira/browse/THRIFT-3616 > Project: Thrift > Issue Type: Bug > Components: Python - Library, Test Suite >Reporter: John Sirois >Assignee: John Sirois >Priority: Critical > > As seen here: https://builds.apache.org/job/Thrift-precommit/86/console > {noformat} > ... > /usr/bin/python3 test/thrift_json.py > . > -- > Ran 1 test in 0.011s > OK > /usr/bin/python3 test/test_sslsocket.py > .FFFsys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0, > laddr=('::1', 23458, 0, 0), raddr=('::1', 41151, 0, 0)> > F/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:253: > DeprecationWarning: validate is deprecated. Use cert_reqs=ssl.CERT_REQUIRED > instead > DeprecationWarning) > /thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:253: > DeprecationWarning: validate is deprecated. Use cert_reqs=ssl.CERT_NONE > instead > DeprecationWarning) > .sys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0, > laddr=('::1', 23458, 0, 0), raddr=('::1', 41155, 0, 0)> > F/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:265: > DeprecationWarning: Use cert_reqs instead > warnings.warn('Use cert_reqs instead', DeprecationWarning) > .sys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0, > laddr=('::1', 23458, 0, 0), raddr=('::1', 41157, 0, 0)> > Exception in thread Thread-11: > Traceback (most recent call last): > File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner > self.run() > File "test/test_sslsocket.py", line 60, in run > self.client = self._server.accept() > File "/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py", line 418, > in accept > client = self._wrap_socket(plain_client) > File "/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py", line 181, > in _wrap_socket > server_hostname=self._server_hostname) > File "/usr/lib/python3.4/ssl.py", line 365, in wrap_socket > _context=self) > File "/usr/lib/python3.4/ssl.py", line 601, in __init__ > self.do_handshake() > File "/usr/lib/python3.4/ssl.py", line 828, in do_handshake > self._sslobj.do_handshake() > ConnectionResetError: [Errno 104] Connection reset by peer > PROTOCOL_SSLv2 is not available > ..test/test_sslsocket.py:59: ResourceWarning: unclosed family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0> > self._server.listen() > . > == > FAIL: test_client_cert (__main__.TSSLSocketTest) > -- > Traceback (most recent call last): > File "test/test_sslsocket.py", line 210, in test_client_cert > self._assert_connection_success(server, client) > File "test/test_sslsocket.py", line 112, in _assert_connection_success > self.assertTrue(acc.client is not None) > AssertionError: False is not true > == > FAIL: test_deprecation (__main__.TSSLSocketTest) > -- > Traceback (most recent call last): > File "test/test_sslsocket.py", line 138, in test_deprecation > self.assertEqual(len(w), 3) > AssertionError: 4 != 3 > == > FAIL: test_newer_tls (__main__.TSSLSocketTest) > -- > Traceback (most recent call last): > File "test/test_sslsocket.py", line 268, in test_newer_tls > self._assert_connection_success(server, client) > File "test/test_sslsocket.py", line 112, in _assert_connection_success > self.assertTrue(acc.client is not None) > AssertionError: False is not true > == > FAIL: test_server_cert (__main__.TSSLSocketTest) > -- > Traceback (most recent call last): > File "test/test_sslsocket.py", line 172, in test_server_cert > self._assert_connection_success(server, client) > File "test/test_sslsocket.py", line 112, in _assert_connection_success > self.assertTrue(acc.client is not None) > AssertionError: F
[jira] [Comment Edited] (THRIFT-3616) lib/py/test/test_sslsocket.py is flaky
[ https://issues.apache.org/jira/browse/THRIFT-3616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15143542#comment-15143542 ] John Sirois edited comment on THRIFT-3616 at 2/11/16 9:34 PM: -- OK - I have a delay-free way to fix this that also speeds up the test, just need to polish and I'll have a PR shortly. Before: Ran 11 tests in 8.213s After: Ran 11 tests in 0.284s was (Author: jsirois): OK - I have a delay-free way to fix this that also speeds up the test, just need to polish and I'll have a PR shortly. > lib/py/test/test_sslsocket.py is flaky > -- > > Key: THRIFT-3616 > URL: https://issues.apache.org/jira/browse/THRIFT-3616 > Project: Thrift > Issue Type: Bug > Components: Python - Library, Test Suite >Reporter: John Sirois >Assignee: John Sirois >Priority: Critical > > As seen here: https://builds.apache.org/job/Thrift-precommit/86/console > {noformat} > ... > /usr/bin/python3 test/thrift_json.py > . > -- > Ran 1 test in 0.011s > OK > /usr/bin/python3 test/test_sslsocket.py > .FFFsys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0, > laddr=('::1', 23458, 0, 0), raddr=('::1', 41151, 0, 0)> > F/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:253: > DeprecationWarning: validate is deprecated. Use cert_reqs=ssl.CERT_REQUIRED > instead > DeprecationWarning) > /thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:253: > DeprecationWarning: validate is deprecated. Use cert_reqs=ssl.CERT_NONE > instead > DeprecationWarning) > .sys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0, > laddr=('::1', 23458, 0, 0), raddr=('::1', 41155, 0, 0)> > F/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:265: > DeprecationWarning: Use cert_reqs instead > warnings.warn('Use cert_reqs instead', DeprecationWarning) > .sys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0, > laddr=('::1', 23458, 0, 0), raddr=('::1', 41157, 0, 0)> > Exception in thread Thread-11: > Traceback (most recent call last): > File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner > self.run() > File "test/test_sslsocket.py", line 60, in run > self.client = self._server.accept() > File "/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py", line 418, > in accept > client = self._wrap_socket(plain_client) > File "/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py", line 181, > in _wrap_socket > server_hostname=self._server_hostname) > File "/usr/lib/python3.4/ssl.py", line 365, in wrap_socket > _context=self) > File "/usr/lib/python3.4/ssl.py", line 601, in __init__ > self.do_handshake() > File "/usr/lib/python3.4/ssl.py", line 828, in do_handshake > self._sslobj.do_handshake() > ConnectionResetError: [Errno 104] Connection reset by peer > PROTOCOL_SSLv2 is not available > ..test/test_sslsocket.py:59: ResourceWarning: unclosed family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0> > self._server.listen() > . > == > FAIL: test_client_cert (__main__.TSSLSocketTest) > -- > Traceback (most recent call last): > File "test/test_sslsocket.py", line 210, in test_client_cert > self._assert_connection_success(server, client) > File "test/test_sslsocket.py", line 112, in _assert_connection_success > self.assertTrue(acc.client is not None) > AssertionError: False is not true > == > FAIL: test_deprecation (__main__.TSSLSocketTest) > -- > Traceback (most recent call last): > File "test/test_sslsocket.py", line 138, in test_deprecation > self.assertEqual(len(w), 3) > AssertionError: 4 != 3 > == > FAIL: test_newer_tls (__main__.TSSLSocketTest) > -- > Traceback (most recent call last): > File "test/test_sslsocket.py", line 268, in test_newer_tls > self._assert_connection_success(server, client) > File "test/test_sslsocket.py", line 112, in _assert_connection_success > self.assertTrue(acc.client is not None) > AssertionError: False is not true > == > FAIL: test_server_cert (__main__.TSSLSocketTest) > -- > Traceback (most recent call last): >
[jira] [Commented] (THRIFT-3616) lib/py/test/test_sslsocket.py is flaky
[ https://issues.apache.org/jira/browse/THRIFT-3616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15144683#comment-15144683 ] John Sirois commented on THRIFT-3616: - https://github.com/apache/thrift/pull/850 > lib/py/test/test_sslsocket.py is flaky > -- > > Key: THRIFT-3616 > URL: https://issues.apache.org/jira/browse/THRIFT-3616 > Project: Thrift > Issue Type: Bug > Components: Python - Library, Test Suite >Reporter: John Sirois >Assignee: John Sirois >Priority: Critical > > As seen here: https://builds.apache.org/job/Thrift-precommit/86/console > {noformat} > ... > /usr/bin/python3 test/thrift_json.py > . > -- > Ran 1 test in 0.011s > OK > /usr/bin/python3 test/test_sslsocket.py > .FFFsys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0, > laddr=('::1', 23458, 0, 0), raddr=('::1', 41151, 0, 0)> > F/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:253: > DeprecationWarning: validate is deprecated. Use cert_reqs=ssl.CERT_REQUIRED > instead > DeprecationWarning) > /thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:253: > DeprecationWarning: validate is deprecated. Use cert_reqs=ssl.CERT_NONE > instead > DeprecationWarning) > .sys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0, > laddr=('::1', 23458, 0, 0), raddr=('::1', 41155, 0, 0)> > F/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:265: > DeprecationWarning: Use cert_reqs instead > warnings.warn('Use cert_reqs instead', DeprecationWarning) > .sys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0, > laddr=('::1', 23458, 0, 0), raddr=('::1', 41157, 0, 0)> > Exception in thread Thread-11: > Traceback (most recent call last): > File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner > self.run() > File "test/test_sslsocket.py", line 60, in run > self.client = self._server.accept() > File "/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py", line 418, > in accept > client = self._wrap_socket(plain_client) > File "/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py", line 181, > in _wrap_socket > server_hostname=self._server_hostname) > File "/usr/lib/python3.4/ssl.py", line 365, in wrap_socket > _context=self) > File "/usr/lib/python3.4/ssl.py", line 601, in __init__ > self.do_handshake() > File "/usr/lib/python3.4/ssl.py", line 828, in do_handshake > self._sslobj.do_handshake() > ConnectionResetError: [Errno 104] Connection reset by peer > PROTOCOL_SSLv2 is not available > ..test/test_sslsocket.py:59: ResourceWarning: unclosed family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0> > self._server.listen() > . > == > FAIL: test_client_cert (__main__.TSSLSocketTest) > -- > Traceback (most recent call last): > File "test/test_sslsocket.py", line 210, in test_client_cert > self._assert_connection_success(server, client) > File "test/test_sslsocket.py", line 112, in _assert_connection_success > self.assertTrue(acc.client is not None) > AssertionError: False is not true > == > FAIL: test_deprecation (__main__.TSSLSocketTest) > -- > Traceback (most recent call last): > File "test/test_sslsocket.py", line 138, in test_deprecation > self.assertEqual(len(w), 3) > AssertionError: 4 != 3 > == > FAIL: test_newer_tls (__main__.TSSLSocketTest) > -- > Traceback (most recent call last): > File "test/test_sslsocket.py", line 268, in test_newer_tls > self._assert_connection_success(server, client) > File "test/test_sslsocket.py", line 112, in _assert_connection_success > self.assertTrue(acc.client is not None) > AssertionError: False is not true > == > FAIL: test_server_cert (__main__.TSSLSocketTest) > -- > Traceback (most recent call last): > File "test/test_sslsocket.py", line 172, in test_server_cert > self._assert_connection_success(server, client) > File "test/test_sslsocket.py", line 112, in _assert_connection_success > self.assertTrue(acc.client is not None) > AssertionError: False is not true > ===
[jira] [Commented] (THRIFT-3608) lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build.
[ https://issues.apache.org/jira/browse/THRIFT-3608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15144733#comment-15144733 ] John Sirois commented on THRIFT-3608: - I'm not sure I follow the don't wait part. I do need to wait on some committer to review and merge. If you can recommend an appropriate one I'd be grateful. > lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build. > - > > Key: THRIFT-3608 > URL: https://issues.apache.org/jira/browse/THRIFT-3608 > Project: Thrift > Issue Type: Bug > Components: C++ - Library, Test Suite >Affects Versions: 0.9.3 >Reporter: John Sirois >Assignee: John Sirois >Priority: Critical > > The [Thrift-precommit|https://builds.apache.org/job/Thrift-precommit] job now > runs in the build/docker/ubuntu container and the > [lib/cpp/test/SecurityTest|https://github.com/apache/thrift/blob/master/lib/cpp/test/SecurityTest.cpp] > fails most - but not all - of the time. > Failures can be seen in the following jobs > * https://builds.apache.org/job/Thrift-precommit/60/console > * https://builds.apache.org/job/Thrift-precommit/59/console > * https://builds.apache.org/job/Thrift-precommit/58/console > * https://builds.apache.org/job/Thrift-precommit/57/console > They look like so: > {noformat} > argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest" > Running 1 test case... > Entering test suite "SecurityTest" > Entering test suite "SecurityTest" > Entering test case "ssl_security_matrix" > TEST: Server = SSLTLS, Client = SSLTLS > TEST: Server = SSLTLS, Client = SSLv3 > SRV 2b9182317700 Exception: SSL_accept: wrong version number > CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = SSLTLS, Client = TLSv1_0 > TEST: Server = SSLTLS, Client = TLSv1_1 > TEST: Server = SSLTLS, Client = TLSv1_2 > TEST: Server = SSLv3, Client = SSLTLS > CLI 2b9181f0a700 Exception: SSL_connect: unsupported protocol > SRV 2b9182317700 Exception: SSL_accept: error code: 0 > TEST: Server = SSLv3, Client = SSLv3 > TEST: Server = SSLv3, Client = TLSv1_0 > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = SSLv3, Client = TLSv1_1 > CLI 2b9182317700 Exception: SSL_connect: wrong version number > SRV 2b9181f0a700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = SSLv3, Client = TLSv1_2 > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = TLSv1_0, Client = SSLTLS > TEST: Server = TLSv1_0, Client = SSLv3 > SRV 2b9182317700 Exception: SSL_accept: wrong version number > CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = TLSv1_0, Client = TLSv1_0 > TEST: Server = TLSv1_0, Client = TLSv1_1 > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version > TEST: Server = TLSv1_0, Client = TLSv1_2 > CLI 2b9182317700 Exception: SSL_connect: wrong version number > SRV 2b9181f0a700 Exception: SSL_accept: tlsv1 alert protocol version > TEST: Server = TLSv1_1, Client = SSLTLS > TEST: Server = TLSv1_1, Client = SSLv3 > SRV 2b9181f0a700 Exception: SSL_accept: wrong version number > CLI 2b9182317700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = TLSv1_1, Client = TLSv1_0 > SRV 2b9182317700 Exception: SSL_accept: wrong version number > CLI 2b9181f0a700 Exception: SSL_connect: tlsv1 alert protocol version > TEST: Server = TLSv1_1, Client = TLSv1_1 > TEST: Server = TLSv1_1, Client = TLSv1_2 > SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > TEST: Server = TLSv1_2, Client = SSLTLS > [0;31mFAIL[m: SecurityTest > {noformat} > {noformat} > argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest" > Running 1 test case... > Entering test suite "SecurityTest" > Entering test suite "SecurityTest" > Entering test case "ssl_security_matrix" > TEST: Server = SSLTLS, Client = SSLTLS > TEST: Server = SSLTLS, Client = SSLv3 > SRV 2b6bd3245700 Exception: SSL_accept: wrong version number > CLI 2b6bd2e38700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = SSLTLS, Client = TLSv1_0 > TEST: Server = SSLTLS, Client = TLSv1_1 > TEST: Server = SSLTLS, Client = TLSv1_2 > TEST: Server = SSLv3, Client = SSLTLS > CLI 2b6bd2e38700 Exception: SSL_connect: unsupported protocol > SRV 2b6bd3245700 Exception: SSL_accept: error code: 0 > TEST: Server = SSLv3, Client = SSLv3 > TEST: Server = SSLv3, Client = TLSv1_0 > CLI 2b6bd2e38700 Exception: SSL_connect: wrong version number > SRV 2b6bd3245700 Exception: SS
[jira] [Commented] (THRIFT-3616) lib/py/test/test_sslsocket.py is flaky
[ https://issues.apache.org/jira/browse/THRIFT-3616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15144928#comment-15144928 ] John Sirois commented on THRIFT-3616: - Ah yes, I misread 8 and 9 here: https://thrift.apache.org/docs/HowToContribute#contributing-via-github-pull-requests Will do going forward. > lib/py/test/test_sslsocket.py is flaky > -- > > Key: THRIFT-3616 > URL: https://issues.apache.org/jira/browse/THRIFT-3616 > Project: Thrift > Issue Type: Bug > Components: Python - Library, Test Suite >Reporter: John Sirois >Assignee: John Sirois >Priority: Critical > Fix For: 0.9.4 > > > As seen here: https://builds.apache.org/job/Thrift-precommit/86/console > {noformat} > ... > /usr/bin/python3 test/thrift_json.py > . > -- > Ran 1 test in 0.011s > OK > /usr/bin/python3 test/test_sslsocket.py > .FFFsys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0, > laddr=('::1', 23458, 0, 0), raddr=('::1', 41151, 0, 0)> > F/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:253: > DeprecationWarning: validate is deprecated. Use cert_reqs=ssl.CERT_REQUIRED > instead > DeprecationWarning) > /thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:253: > DeprecationWarning: validate is deprecated. Use cert_reqs=ssl.CERT_NONE > instead > DeprecationWarning) > .sys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0, > laddr=('::1', 23458, 0, 0), raddr=('::1', 41155, 0, 0)> > F/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:265: > DeprecationWarning: Use cert_reqs instead > warnings.warn('Use cert_reqs instead', DeprecationWarning) > .sys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0, > laddr=('::1', 23458, 0, 0), raddr=('::1', 41157, 0, 0)> > Exception in thread Thread-11: > Traceback (most recent call last): > File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner > self.run() > File "test/test_sslsocket.py", line 60, in run > self.client = self._server.accept() > File "/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py", line 418, > in accept > client = self._wrap_socket(plain_client) > File "/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py", line 181, > in _wrap_socket > server_hostname=self._server_hostname) > File "/usr/lib/python3.4/ssl.py", line 365, in wrap_socket > _context=self) > File "/usr/lib/python3.4/ssl.py", line 601, in __init__ > self.do_handshake() > File "/usr/lib/python3.4/ssl.py", line 828, in do_handshake > self._sslobj.do_handshake() > ConnectionResetError: [Errno 104] Connection reset by peer > PROTOCOL_SSLv2 is not available > ..test/test_sslsocket.py:59: ResourceWarning: unclosed family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0> > self._server.listen() > . > == > FAIL: test_client_cert (__main__.TSSLSocketTest) > -- > Traceback (most recent call last): > File "test/test_sslsocket.py", line 210, in test_client_cert > self._assert_connection_success(server, client) > File "test/test_sslsocket.py", line 112, in _assert_connection_success > self.assertTrue(acc.client is not None) > AssertionError: False is not true > == > FAIL: test_deprecation (__main__.TSSLSocketTest) > -- > Traceback (most recent call last): > File "test/test_sslsocket.py", line 138, in test_deprecation > self.assertEqual(len(w), 3) > AssertionError: 4 != 3 > == > FAIL: test_newer_tls (__main__.TSSLSocketTest) > -- > Traceback (most recent call last): > File "test/test_sslsocket.py", line 268, in test_newer_tls > self._assert_connection_success(server, client) > File "test/test_sslsocket.py", line 112, in _assert_connection_success > self.assertTrue(acc.client is not None) > AssertionError: False is not true > == > FAIL: test_server_cert (__main__.TSSLSocketTest) > -- > Traceback (most recent call last): > File "test/test_sslsocket.py", line 172, in test_server_cert > self._assert_connection_success(server, client) > File "test/test_sslsocket.py", line 112, in _assert_connection_success > self.as
[jira] [Commented] (THRIFT-3616) lib/py/test/test_sslsocket.py is flaky
[ https://issues.apache.org/jira/browse/THRIFT-3616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15144950#comment-15144950 ] John Sirois commented on THRIFT-3616: - Thanks [~nsuke]! > lib/py/test/test_sslsocket.py is flaky > -- > > Key: THRIFT-3616 > URL: https://issues.apache.org/jira/browse/THRIFT-3616 > Project: Thrift > Issue Type: Bug > Components: Python - Library, Test Suite >Reporter: John Sirois >Assignee: John Sirois >Priority: Critical > Fix For: 0.9.4 > > > As seen here: https://builds.apache.org/job/Thrift-precommit/86/console > {noformat} > ... > /usr/bin/python3 test/thrift_json.py > . > -- > Ran 1 test in 0.011s > OK > /usr/bin/python3 test/test_sslsocket.py > .FFFsys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0, > laddr=('::1', 23458, 0, 0), raddr=('::1', 41151, 0, 0)> > F/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:253: > DeprecationWarning: validate is deprecated. Use cert_reqs=ssl.CERT_REQUIRED > instead > DeprecationWarning) > /thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:253: > DeprecationWarning: validate is deprecated. Use cert_reqs=ssl.CERT_NONE > instead > DeprecationWarning) > .sys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0, > laddr=('::1', 23458, 0, 0), raddr=('::1', 41155, 0, 0)> > F/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:265: > DeprecationWarning: Use cert_reqs instead > warnings.warn('Use cert_reqs instead', DeprecationWarning) > .sys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0, > laddr=('::1', 23458, 0, 0), raddr=('::1', 41157, 0, 0)> > Exception in thread Thread-11: > Traceback (most recent call last): > File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner > self.run() > File "test/test_sslsocket.py", line 60, in run > self.client = self._server.accept() > File "/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py", line 418, > in accept > client = self._wrap_socket(plain_client) > File "/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py", line 181, > in _wrap_socket > server_hostname=self._server_hostname) > File "/usr/lib/python3.4/ssl.py", line 365, in wrap_socket > _context=self) > File "/usr/lib/python3.4/ssl.py", line 601, in __init__ > self.do_handshake() > File "/usr/lib/python3.4/ssl.py", line 828, in do_handshake > self._sslobj.do_handshake() > ConnectionResetError: [Errno 104] Connection reset by peer > PROTOCOL_SSLv2 is not available > ..test/test_sslsocket.py:59: ResourceWarning: unclosed family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0> > self._server.listen() > . > == > FAIL: test_client_cert (__main__.TSSLSocketTest) > -- > Traceback (most recent call last): > File "test/test_sslsocket.py", line 210, in test_client_cert > self._assert_connection_success(server, client) > File "test/test_sslsocket.py", line 112, in _assert_connection_success > self.assertTrue(acc.client is not None) > AssertionError: False is not true > == > FAIL: test_deprecation (__main__.TSSLSocketTest) > -- > Traceback (most recent call last): > File "test/test_sslsocket.py", line 138, in test_deprecation > self.assertEqual(len(w), 3) > AssertionError: 4 != 3 > == > FAIL: test_newer_tls (__main__.TSSLSocketTest) > -- > Traceback (most recent call last): > File "test/test_sslsocket.py", line 268, in test_newer_tls > self._assert_connection_success(server, client) > File "test/test_sslsocket.py", line 112, in _assert_connection_success > self.assertTrue(acc.client is not None) > AssertionError: False is not true > == > FAIL: test_server_cert (__main__.TSSLSocketTest) > -- > Traceback (most recent call last): > File "test/test_sslsocket.py", line 172, in test_server_cert > self._assert_connection_success(server, client) > File "test/test_sslsocket.py", line 112, in _assert_connection_success > self.assertTrue(acc.client is not None) > AssertionError: False is not true > =
[jira] [Commented] (THRIFT-3608) lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build.
[ https://issues.apache.org/jira/browse/THRIFT-3608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15144951#comment-15144951 ] John Sirois commented on THRIFT-3608: - Thank you! Thrift-precommit should hopefully be going green now for github PRs. > lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build. > - > > Key: THRIFT-3608 > URL: https://issues.apache.org/jira/browse/THRIFT-3608 > Project: Thrift > Issue Type: Bug > Components: C++ - Library, Test Suite >Affects Versions: 0.9.3 >Reporter: John Sirois >Assignee: John Sirois >Priority: Critical > Fix For: 0.9.4 > > > The [Thrift-precommit|https://builds.apache.org/job/Thrift-precommit] job now > runs in the build/docker/ubuntu container and the > [lib/cpp/test/SecurityTest|https://github.com/apache/thrift/blob/master/lib/cpp/test/SecurityTest.cpp] > fails most - but not all - of the time. > Failures can be seen in the following jobs > * https://builds.apache.org/job/Thrift-precommit/60/console > * https://builds.apache.org/job/Thrift-precommit/59/console > * https://builds.apache.org/job/Thrift-precommit/58/console > * https://builds.apache.org/job/Thrift-precommit/57/console > They look like so: > {noformat} > argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest" > Running 1 test case... > Entering test suite "SecurityTest" > Entering test suite "SecurityTest" > Entering test case "ssl_security_matrix" > TEST: Server = SSLTLS, Client = SSLTLS > TEST: Server = SSLTLS, Client = SSLv3 > SRV 2b9182317700 Exception: SSL_accept: wrong version number > CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = SSLTLS, Client = TLSv1_0 > TEST: Server = SSLTLS, Client = TLSv1_1 > TEST: Server = SSLTLS, Client = TLSv1_2 > TEST: Server = SSLv3, Client = SSLTLS > CLI 2b9181f0a700 Exception: SSL_connect: unsupported protocol > SRV 2b9182317700 Exception: SSL_accept: error code: 0 > TEST: Server = SSLv3, Client = SSLv3 > TEST: Server = SSLv3, Client = TLSv1_0 > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = SSLv3, Client = TLSv1_1 > CLI 2b9182317700 Exception: SSL_connect: wrong version number > SRV 2b9181f0a700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = SSLv3, Client = TLSv1_2 > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: Server = TLSv1_0, Client = SSLTLS > TEST: Server = TLSv1_0, Client = SSLv3 > SRV 2b9182317700 Exception: SSL_accept: wrong version number > CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = TLSv1_0, Client = TLSv1_0 > TEST: Server = TLSv1_0, Client = TLSv1_1 > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version > TEST: Server = TLSv1_0, Client = TLSv1_2 > CLI 2b9182317700 Exception: SSL_connect: wrong version number > SRV 2b9181f0a700 Exception: SSL_accept: tlsv1 alert protocol version > TEST: Server = TLSv1_1, Client = SSLTLS > TEST: Server = TLSv1_1, Client = SSLv3 > SRV 2b9181f0a700 Exception: SSL_accept: wrong version number > CLI 2b9182317700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = TLSv1_1, Client = TLSv1_0 > SRV 2b9182317700 Exception: SSL_accept: wrong version number > CLI 2b9181f0a700 Exception: SSL_connect: tlsv1 alert protocol version > TEST: Server = TLSv1_1, Client = TLSv1_1 > TEST: Server = TLSv1_1, Client = TLSv1_2 > SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version > CLI 2b9181f0a700 Exception: SSL_connect: wrong version number > TEST: Server = TLSv1_2, Client = SSLTLS > [0;31mFAIL[m: SecurityTest > {noformat} > {noformat} > argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest" > Running 1 test case... > Entering test suite "SecurityTest" > Entering test suite "SecurityTest" > Entering test case "ssl_security_matrix" > TEST: Server = SSLTLS, Client = SSLTLS > TEST: Server = SSLTLS, Client = SSLv3 > SRV 2b6bd3245700 Exception: SSL_accept: wrong version number > CLI 2b6bd2e38700 Exception: SSL_connect: sslv3 alert handshake failure > TEST: Server = SSLTLS, Client = TLSv1_0 > TEST: Server = SSLTLS, Client = TLSv1_1 > TEST: Server = SSLTLS, Client = TLSv1_2 > TEST: Server = SSLv3, Client = SSLTLS > CLI 2b6bd2e38700 Exception: SSL_connect: unsupported protocol > SRV 2b6bd3245700 Exception: SSL_accept: error code: 0 > TEST: Server = SSLv3, Client = SSLv3 > TEST: Server = SSLv3, Client = TLSv1_0 > CLI 2b6bd2e38700 Exception: SSL_connect: wrong version number > SRV 2b6bd3245700 Exception: SSL_accept: sslv3 alert handshake failure > TEST: S
[jira] [Commented] (THRIFT-3112) [Java] AsyncMethodCallback should be typed in generated AsyncIface
[ https://issues.apache.org/jira/browse/THRIFT-3112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145025#comment-15145025 ] John Sirois commented on THRIFT-3112: - The PROPOSAL thread has had 1 positive response from [~nsuke] and no other responses. The PR @ https://github.com/apache/thrift/pull/840 is now green after re-basing against master to pick up fixes for THRIFT-3608 and THRIFT-3616 and ready for review and merging. > [Java] AsyncMethodCallback should be typed in generated AsyncIface > -- > > Key: THRIFT-3112 > URL: https://issues.apache.org/jira/browse/THRIFT-3112 > Project: Thrift > Issue Type: Improvement > Components: Java - Compiler, Java - Library >Affects Versions: 0.9.2 >Reporter: Sergei Egorov >Assignee: John Sirois > > AsyncMethodCallback is generic, but current Java code generator is not adding > type info to it, and instead of: > {code:java} > public interface AsyncIface { > public void ping(org.apache.thrift.async.AsyncMethodCallback > resultHandler) throws org.apache.thrift.TException; > } > {code} > we have: > {code:java} > public interface AsyncIface { > public void ping(org.apache.thrift.async.AsyncMethodCallback > resultHandler) throws org.apache.thrift.TException; > } > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (THRIFT-3620) Cleanup and consolidate Thrift* jenkins jobs.
John Sirois created THRIFT-3620: --- Summary: Cleanup and consolidate Thrift* jenkins jobs. Key: THRIFT-3620 URL: https://issues.apache.org/jira/browse/THRIFT-3620 Project: Thrift Issue Type: Task Components: Test Suite Reporter: John Sirois I spent some time resurrecting the [Thrift-precommit|https://builds.apache.org/job/Thrift-precommit/] job which vets pull-requests to use the pool of docker slaves using a dockerized CI with guidance from [~jfarrell]. The remaining crop of Thrift* jobs should be culled and converted as appropriate. Currently we have the jobs [here|https://builds.apache.org/view/S-Z/view/Thrift/]: {noformat} $ curl --netrc -sS https://builds.apache.org/view/S-Z/view/Thrift/config.xml | xmllint -xpath "//jobNames" - | grep "" | cut -d'>' -f2 | cut -d'<' -f1 Thrift Thrift-all Thrift-Compiler-Linux64 Thrift-Compiler-Windows Thrift-env Thrift-env-Windows Thrift-Test Thrift-Windows {noformat} Of those, [Thrift-env|https://builds.apache.org/job/Thrift-env], [Thrift-env-Windows|https://builds.apache.org/job/Thrift-env-Windows] and [Thrift-Windows|https://builds.apache.org/job/Thrift-Windows] are all disabled and [Thrift|https://builds.apache.org/job/Thrift] and [Thrift-Compiler-Linux64|https://builds.apache.org/job/Thrift-Compiler-Linux64] seem to be unused based on last run dates. This leaves the following active jobs fwict: * [|https://builds.apache.org/job/Thrift-all] * [|https://builds.apache.org/job/Thrift-Compiler-Windows] * [|https://builds.apache.org/job/Thrift-precommit] Of these {{Thrift-all}} is broken in a known-way, it really needs to be converted to use a Dockerized build like {{Thrift-precommit}} to be insulated from the vagaries of the underlying worker machines. I propose deltion of all disabled (3) and inactive (2) jobs listed above and conversion of {{Thrift-all}} to a Dockerized CI job to guard master with. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3620) Cleanup and consolidate Thrift* jenkins jobs.
[ https://issues.apache.org/jira/browse/THRIFT-3620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145116#comment-15145116 ] John Sirois commented on THRIFT-3620: - [~jfarrell], [~jensg], [~roger.meier] and [~nsuke] - I could use your institutional knowledge and feedback here. I'd love to kill the cruft and convert the useful but broken, but I need feedback to do this right. If you can comment on my proposal and set me straight on what I've got wrong, I'd be much appreciative. > Cleanup and consolidate Thrift* jenkins jobs. > - > > Key: THRIFT-3620 > URL: https://issues.apache.org/jira/browse/THRIFT-3620 > Project: Thrift > Issue Type: Task > Components: Test Suite >Reporter: John Sirois > > I spent some time resurrecting the > [Thrift-precommit|https://builds.apache.org/job/Thrift-precommit/] job which > vets pull-requests to use the pool of docker slaves using a dockerized CI > with guidance from [~jfarrell]. The remaining crop of Thrift* jobs should be > culled and converted as appropriate. > Currently we have the jobs > [here|https://builds.apache.org/view/S-Z/view/Thrift/]: > {noformat} > $ curl --netrc -sS https://builds.apache.org/view/S-Z/view/Thrift/config.xml > | xmllint -xpath "//jobNames" - | grep "" | cut -d'>' -f2 | cut -d'<' > -f1 > Thrift > Thrift-all > Thrift-Compiler-Linux64 > Thrift-Compiler-Windows > Thrift-env > Thrift-env-Windows > Thrift-Test > Thrift-Windows > {noformat} > Of those, [Thrift-env|https://builds.apache.org/job/Thrift-env], > [Thrift-env-Windows|https://builds.apache.org/job/Thrift-env-Windows] and > [Thrift-Windows|https://builds.apache.org/job/Thrift-Windows] are all > disabled and [Thrift|https://builds.apache.org/job/Thrift] and > [Thrift-Compiler-Linux64|https://builds.apache.org/job/Thrift-Compiler-Linux64] > seem to be unused based on last run dates. This leaves the following active > jobs fwict: > * [|https://builds.apache.org/job/Thrift-all] > * [|https://builds.apache.org/job/Thrift-Compiler-Windows] > * [|https://builds.apache.org/job/Thrift-precommit] > Of these {{Thrift-all}} is broken in a known-way, it really needs to be > converted to use a Dockerized build like {{Thrift-precommit}} to be insulated > from the vagaries of the underlying worker machines. > I propose deltion of all disabled (3) and inactive (2) jobs listed above and > conversion of {{Thrift-all}} to a Dockerized CI job to guard master with. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (THRIFT-3620) Cleanup and consolidate Thrift* jenkins jobs.
[ https://issues.apache.org/jira/browse/THRIFT-3620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sirois reassigned THRIFT-3620: --- Assignee: John Sirois > Cleanup and consolidate Thrift* jenkins jobs. > - > > Key: THRIFT-3620 > URL: https://issues.apache.org/jira/browse/THRIFT-3620 > Project: Thrift > Issue Type: Task > Components: Test Suite >Reporter: John Sirois >Assignee: John Sirois > > I spent some time resurrecting the > [Thrift-precommit|https://builds.apache.org/job/Thrift-precommit/] job which > vets pull-requests to use the pool of docker slaves using a dockerized CI > with guidance from [~jfarrell]. The remaining crop of Thrift* jobs should be > culled and converted as appropriate. > Currently we have the jobs > [here|https://builds.apache.org/view/S-Z/view/Thrift/]: > {noformat} > $ curl --netrc -sS https://builds.apache.org/view/S-Z/view/Thrift/config.xml > | xmllint -xpath "//jobNames" - | grep "" | cut -d'>' -f2 | cut -d'<' > -f1 > Thrift > Thrift-all > Thrift-Compiler-Linux64 > Thrift-Compiler-Windows > Thrift-env > Thrift-env-Windows > Thrift-Test > Thrift-Windows > {noformat} > Of those, [Thrift-env|https://builds.apache.org/job/Thrift-env], > [Thrift-env-Windows|https://builds.apache.org/job/Thrift-env-Windows] and > [Thrift-Windows|https://builds.apache.org/job/Thrift-Windows] are all > disabled and [Thrift|https://builds.apache.org/job/Thrift] and > [Thrift-Compiler-Linux64|https://builds.apache.org/job/Thrift-Compiler-Linux64] > seem to be unused based on last run dates. This leaves the following active > jobs fwict: > * [|https://builds.apache.org/job/Thrift-all] > * [|https://builds.apache.org/job/Thrift-Compiler-Windows] > * [|https://builds.apache.org/job/Thrift-precommit] > Of these {{Thrift-all}} is broken in a known-way, it really needs to be > converted to use a Dockerized build like {{Thrift-precommit}} to be insulated > from the vagaries of the underlying worker machines. > I propose deltion of all disabled (3) and inactive (2) jobs listed above and > conversion of {{Thrift-all}} to a Dockerized CI job to guard master with. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3620) Cleanup and consolidate Thrift* jenkins jobs.
[ https://issues.apache.org/jira/browse/THRIFT-3620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145263#comment-15145263 ] John Sirois commented on THRIFT-3620: - Ok - it would be nice to have a single job and I had not realized the windows job actually ran on ubuntu and just used mingw. I'll proceed on this in the background to things like THRIFT-3112, THRIFT-2157, THRIFT-3583 which are main main focus now that Thrift-precommit is green and stable. > Cleanup and consolidate Thrift* jenkins jobs. > - > > Key: THRIFT-3620 > URL: https://issues.apache.org/jira/browse/THRIFT-3620 > Project: Thrift > Issue Type: Task > Components: Test Suite >Reporter: John Sirois >Assignee: John Sirois > > I spent some time resurrecting the > [Thrift-precommit|https://builds.apache.org/job/Thrift-precommit/] job which > vets pull-requests to use the pool of docker slaves using a dockerized CI > with guidance from [~jfarrell]. The remaining crop of Thrift* jobs should be > culled and converted as appropriate. > Currently we have the jobs > [here|https://builds.apache.org/view/S-Z/view/Thrift/]: > {noformat} > $ curl --netrc -sS https://builds.apache.org/view/S-Z/view/Thrift/config.xml > | xmllint -xpath "//jobNames" - | grep "" | cut -d'>' -f2 | cut -d'<' > -f1 > Thrift > Thrift-all > Thrift-Compiler-Linux64 > Thrift-Compiler-Windows > Thrift-env > Thrift-env-Windows > Thrift-Test > Thrift-Windows > {noformat} > Of those, [Thrift-env|https://builds.apache.org/job/Thrift-env], > [Thrift-env-Windows|https://builds.apache.org/job/Thrift-env-Windows] and > [Thrift-Windows|https://builds.apache.org/job/Thrift-Windows] are all > disabled and [Thrift|https://builds.apache.org/job/Thrift] and > [Thrift-Compiler-Linux64|https://builds.apache.org/job/Thrift-Compiler-Linux64] > seem to be unused based on last run dates. This leaves the following active > jobs fwict: > * [|https://builds.apache.org/job/Thrift-all] > * [|https://builds.apache.org/job/Thrift-Compiler-Windows] > * [|https://builds.apache.org/job/Thrift-precommit] > Of these {{Thrift-all}} is broken in a known-way, it really needs to be > converted to use a Dockerized build like {{Thrift-precommit}} to be insulated > from the vagaries of the underlying worker machines. > I propose deltion of all disabled (3) and inactive (2) jobs listed above and > conversion of {{Thrift-all}} to a Dockerized CI job to guard master with. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (THRIFT-3621) Fix lib/cpp/test/SecurityTest.cpp to use ephemeral ports
John Sirois created THRIFT-3621: --- Summary: Fix lib/cpp/test/SecurityTest.cpp to use ephemeral ports Key: THRIFT-3621 URL: https://issues.apache.org/jira/browse/THRIFT-3621 Project: Thrift Issue Type: Sub-task Components: C++ - Library, Test Suite Reporter: John Sirois Assignee: John Sirois As part of work on THRIFT-3609, eliminate use of TestPortFixture.h from SecurityTest, -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3621) Fix lib/cpp/test/SecurityTest.cpp to use ephemeral ports
[ https://issues.apache.org/jira/browse/THRIFT-3621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145529#comment-15145529 ] John Sirois commented on THRIFT-3621: - https://github.com/apache/thrift/pull/853 > Fix lib/cpp/test/SecurityTest.cpp to use ephemeral ports > > > Key: THRIFT-3621 > URL: https://issues.apache.org/jira/browse/THRIFT-3621 > Project: Thrift > Issue Type: Sub-task > Components: C++ - Library, Test Suite >Reporter: John Sirois >Assignee: John Sirois > > As part of work on THRIFT-3609, eliminate use of TestPortFixture.h from > SecurityTest, -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (THRIFT-3622) Fix deprecated uses of std::auto_ptr
John Sirois created THRIFT-3622: --- Summary: Fix deprecated uses of std::auto_ptr Key: THRIFT-3622 URL: https://issues.apache.org/jira/browse/THRIFT-3622 Project: Thrift Issue Type: Task Components: C++ - Library Reporter: John Sirois We have some in lib/cpp, like so: {noformat} JSONProtoTest.cpp:35:13: warning: ‘template class std::auto_ptr’ is deprecated [-Wdeprecated-declarations] static std::auto_ptr ooe; ^ In file included from /usr/include/c++/5.3.0/bits/locale_conv.h:41:0, from /usr/include/c++/5.3.0/locale:43, from /usr/include/c++/5.3.0/iomanip:43, from JSONProtoTest.cpp:22: {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (THRIFT-3623) Fix Fix cpp/lib/test/TSSLSocketInterruptTest.cpp to use ephemeral ports
John Sirois created THRIFT-3623: --- Summary: Fix Fix cpp/lib/test/TSSLSocketInterruptTest.cpp to use ephemeral ports Key: THRIFT-3623 URL: https://issues.apache.org/jira/browse/THRIFT-3623 Project: Thrift Issue Type: Sub-task Components: C++ - Library, Test Suite Reporter: John Sirois Assignee: John Sirois As part of work on THRIFT-3609, eliminate use of TestPortFixture.h from TSSLSocketInterruptTest.cpp. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3623) Fix Fix cpp/lib/test/TSSLSocketInterruptTest.cpp to use ephemeral ports
[ https://issues.apache.org/jira/browse/THRIFT-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145634#comment-15145634 ] John Sirois commented on THRIFT-3623: - https://github.com/apache/thrift/pull/854 > Fix Fix cpp/lib/test/TSSLSocketInterruptTest.cpp to use ephemeral ports > --- > > Key: THRIFT-3623 > URL: https://issues.apache.org/jira/browse/THRIFT-3623 > Project: Thrift > Issue Type: Sub-task > Components: C++ - Library, Test Suite >Reporter: John Sirois >Assignee: John Sirois > > As part of work on THRIFT-3609, eliminate use of TestPortFixture.h from > TSSLSocketInterruptTest.cpp. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (THRIFT-3624) Fix lib/cpp/test/TServerSocketTest.cpp to use ephemeral ports
John Sirois created THRIFT-3624: --- Summary: Fix lib/cpp/test/TServerSocketTest.cpp to use ephemeral ports Key: THRIFT-3624 URL: https://issues.apache.org/jira/browse/THRIFT-3624 Project: Thrift Issue Type: Sub-task Components: C++ - Library, Test Suite Reporter: John Sirois Assignee: John Sirois As part of work on THRIFT-3609, eliminate use of TestPortFixture.h from TServerSocketTest. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (THRIFT-3621) Fix lib/cpp/test/SecurityTest.cpp to use ephemeral ports
[ https://issues.apache.org/jira/browse/THRIFT-3621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sirois updated THRIFT-3621: Description: As part of work on THRIFT-3609, eliminate use of TestPortFixture.h from SecurityTest. (was: As part of work on THRIFT-3609, eliminate use of TestPortFixture.h from SecurityTest,) > Fix lib/cpp/test/SecurityTest.cpp to use ephemeral ports > > > Key: THRIFT-3621 > URL: https://issues.apache.org/jira/browse/THRIFT-3621 > Project: Thrift > Issue Type: Sub-task > Components: C++ - Library, Test Suite >Reporter: John Sirois >Assignee: John Sirois > > As part of work on THRIFT-3609, eliminate use of TestPortFixture.h from > SecurityTest. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3624) Fix lib/cpp/test/TServerSocketTest.cpp to use ephemeral ports
[ https://issues.apache.org/jira/browse/THRIFT-3624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145645#comment-15145645 ] John Sirois commented on THRIFT-3624: - https://github.com/apache/thrift/pull/855 > Fix lib/cpp/test/TServerSocketTest.cpp to use ephemeral ports > - > > Key: THRIFT-3624 > URL: https://issues.apache.org/jira/browse/THRIFT-3624 > Project: Thrift > Issue Type: Sub-task > Components: C++ - Library, Test Suite >Reporter: John Sirois >Assignee: John Sirois > > As part of work on THRIFT-3609, eliminate use of TestPortFixture.h from > TServerSocketTest. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (THRIFT-3625) Kill unused #include "TestPortFixture.h" in lib/cpp/test/TServerTransportTest.cpp.
John Sirois created THRIFT-3625: --- Summary: Kill unused #include "TestPortFixture.h" in lib/cpp/test/TServerTransportTest.cpp. Key: THRIFT-3625 URL: https://issues.apache.org/jira/browse/THRIFT-3625 Project: Thrift Issue Type: Sub-task Components: C++ - Library, Test Suite Reporter: John Sirois Assignee: John Sirois As part of work on THRIFT-3609, eliminate use of TestPortFixture.h from TServerTransportTest. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3625) Kill unused #include "TestPortFixture.h" in lib/cpp/test/TServerTransportTest.cpp.
[ https://issues.apache.org/jira/browse/THRIFT-3625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145667#comment-15145667 ] John Sirois commented on THRIFT-3625: - https://github.com/apache/thrift/pull/856 > Kill unused #include "TestPortFixture.h" in > lib/cpp/test/TServerTransportTest.cpp. > -- > > Key: THRIFT-3625 > URL: https://issues.apache.org/jira/browse/THRIFT-3625 > Project: Thrift > Issue Type: Sub-task > Components: C++ - Library, Test Suite >Reporter: John Sirois >Assignee: John Sirois > > As part of work on THRIFT-3609, eliminate use of TestPortFixture.h from > TServerTransportTest. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (THRIFT-3626) Fix lib/cpp/test/TSocketInterruptTest.cpp to use ephemeral ports.
John Sirois created THRIFT-3626: --- Summary: Fix lib/cpp/test/TSocketInterruptTest.cpp to use ephemeral ports. Key: THRIFT-3626 URL: https://issues.apache.org/jira/browse/THRIFT-3626 Project: Thrift Issue Type: Sub-task Components: C++ - Library, Test Suite Reporter: John Sirois Assignee: John Sirois As part of work on THRIFT-3609, eliminate use of TestPortFixture.h from TSocketInterruptTest.cpp. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3626) Fix lib/cpp/test/TSocketInterruptTest.cpp to use ephemeral ports.
[ https://issues.apache.org/jira/browse/THRIFT-3626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145781#comment-15145781 ] John Sirois commented on THRIFT-3626: - https://github.com/apache/thrift/pull/857 > Fix lib/cpp/test/TSocketInterruptTest.cpp to use ephemeral ports. > - > > Key: THRIFT-3626 > URL: https://issues.apache.org/jira/browse/THRIFT-3626 > Project: Thrift > Issue Type: Sub-task > Components: C++ - Library, Test Suite >Reporter: John Sirois >Assignee: John Sirois > > As part of work on THRIFT-3609, eliminate use of TestPortFixture.h from > TSocketInterruptTest.cpp. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3620) Cleanup and consolidate Thrift* jenkins jobs.
[ https://issues.apache.org/jira/browse/THRIFT-3620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145795#comment-15145795 ] John Sirois commented on THRIFT-3620: - I'm agree 100% on the ability for any contributor to enhance CI: {noformat} $ curl --netrc -sS https://builds.apache.org/job/Thrift-precommit/config.xml | xmllint -xpath //command - docker build -t thrift $WORKSPACE/build/docker/ubuntu cat << DOCKERFILE | docker build -t jenkins-thrift - # A thrift image that adds jenkins job specific tools FROM thrift RUN apt-get update && apt-get install -y \ cppcheck \ sloccount DOCKERFILE # TODO(John Sirois): Check the ci script into the repo and have that be the docker entrypoint. docker run --rm -v $WORKSPACE:/thrift -w /thrift -t jenkins-thrift /bin/bash -exc ' sh bootstrap.sh sloccount --duplicates --wide --details lib tutorial test > sloccount.sc cppcheck --error-exitcode=0 --force --xml lib/cpp/src/ lib/c_glib/src/ tutorial/cpp/ test/cpp/ 2> cppcheck-result.xml '# TODO(John Sirois): Check the ci script into the repo and have that be the docker entrypoint. # NB: Hacks used: # 1.) --privileged is a hammer being used to get around issues like intermittent Haxe compile errors of the form: # ... # -I/usr/lib/haxe/lib/hxcpp/3,2,205/include -x c++ -frtti -Wno-invalid-offsetof ./src/thrift/test/TestMapMap_args.cpp -o/thrift/test/haxe/bin/obj/linux64-debug/9931a53a_TestMapMap_args.o # Error: /usr/lib/gcc/x86_64-linux-gnu/4.8/cc1plus: error while loading shared libraries: libcloog-isl.so.4: failed to map segment from shared object: Permission denied # Error: Build failed docker run --privileged --rm -v $WORKSPACE:/thrift -w /thrift -t jenkins-thrift /bin/bash -exc ' sh bootstrap.sh ./configure make check dist ant -f lib/java/build.xml javadoc '# TODO(John Sirois): Check the ci script into the repo and have that be the docker entrypoint. # This is disabled due to a large number of failures in the matrix presently (2/9/2016). # TODO(John Sirois): Fix or remove. #docker run --rm -v $WORKSPACE:/thrift -w /thrift -t jenkins-thrift /bin/bash -exc ' #GOPATH=/thrift/test/go go install bin/... #./test/test.py | tee test.log #' {noformat} > Cleanup and consolidate Thrift* jenkins jobs. > - > > Key: THRIFT-3620 > URL: https://issues.apache.org/jira/browse/THRIFT-3620 > Project: Thrift > Issue Type: Task > Components: Test Suite >Reporter: John Sirois >Assignee: John Sirois > > I spent some time resurrecting the > [Thrift-precommit|https://builds.apache.org/job/Thrift-precommit/] job which > vets pull-requests to use the pool of docker slaves using a dockerized CI > with guidance from [~jfarrell]. The remaining crop of Thrift* jobs should be > culled and converted as appropriate. > Currently we have the jobs > [here|https://builds.apache.org/view/S-Z/view/Thrift/]: > {noformat} > $ curl --netrc -sS https://builds.apache.org/view/S-Z/view/Thrift/config.xml > | xmllint -xpath "//jobNames" - | grep "" | cut -d'>' -f2 | cut -d'<' > -f1 > Thrift > Thrift-all > Thrift-Compiler-Linux64 > Thrift-Compiler-Windows > Thrift-env > Thrift-env-Windows > Thrift-Test > Thrift-Windows > {noformat} > Of those, [Thrift-env|https://builds.apache.org/job/Thrift-env], > [Thrift-env-Windows|https://builds.apache.org/job/Thrift-env-Windows] and > [Thrift-Windows|https://builds.apache.org/job/Thrift-Windows] are all > disabled and [Thrift|https://builds.apache.org/job/Thrift] and > [Thrift-Compiler-Linux64|https://builds.apache.org/job/Thrift-Compiler-Linux64] > seem to be unused based on last run dates. This leaves the following active > jobs fwict: > * [|https://builds.apache.org/job/Thrift-all] > * [|https://builds.apache.org/job/Thrift-Compiler-Windows] > * [|https://builds.apache.org/job/Thrift-precommit] > Of these {{Thrift-all}} is broken in a known-way, it really needs to be > converted to use a Dockerized build like {{Thrift-precommit}} to be insulated > from the vagaries of the underlying worker machines. > I propose deltion of all disabled (3) and inactive (2) jobs listed above and > conversion of {{Thrift-all}} to a Dockerized CI job to guard master with. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (THRIFT-3620) Cleanup and consolidate Thrift* jenkins jobs.
[ https://issues.apache.org/jira/browse/THRIFT-3620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145795#comment-15145795 ] John Sirois edited comment on THRIFT-3620 at 2/13/16 4:41 AM: -- I agree 100% on the ability for any contributor to enhance CI: {noformat} $ curl --netrc -sS https://builds.apache.org/job/Thrift-precommit/config.xml | xmllint -xpath //command - docker build -t thrift $WORKSPACE/build/docker/ubuntu cat << DOCKERFILE | docker build -t jenkins-thrift - # A thrift image that adds jenkins job specific tools FROM thrift RUN apt-get update && apt-get install -y \ cppcheck \ sloccount DOCKERFILE # TODO(John Sirois): Check the ci script into the repo and have that be the docker entrypoint. docker run --rm -v $WORKSPACE:/thrift -w /thrift -t jenkins-thrift /bin/bash -exc ' sh bootstrap.sh sloccount --duplicates --wide --details lib tutorial test > sloccount.sc cppcheck --error-exitcode=0 --force --xml lib/cpp/src/ lib/c_glib/src/ tutorial/cpp/ test/cpp/ 2> cppcheck-result.xml '# TODO(John Sirois): Check the ci script into the repo and have that be the docker entrypoint. # NB: Hacks used: # 1.) --privileged is a hammer being used to get around issues like intermittent Haxe compile errors of the form: # ... # -I/usr/lib/haxe/lib/hxcpp/3,2,205/include -x c++ -frtti -Wno-invalid-offsetof ./src/thrift/test/TestMapMap_args.cpp -o/thrift/test/haxe/bin/obj/linux64-debug/9931a53a_TestMapMap_args.o # Error: /usr/lib/gcc/x86_64-linux-gnu/4.8/cc1plus: error while loading shared libraries: libcloog-isl.so.4: failed to map segment from shared object: Permission denied # Error: Build failed docker run --privileged --rm -v $WORKSPACE:/thrift -w /thrift -t jenkins-thrift /bin/bash -exc ' sh bootstrap.sh ./configure make check dist ant -f lib/java/build.xml javadoc '# TODO(John Sirois): Check the ci script into the repo and have that be the docker entrypoint. # This is disabled due to a large number of failures in the matrix presently (2/9/2016). # TODO(John Sirois): Fix or remove. #docker run --rm -v $WORKSPACE:/thrift -w /thrift -t jenkins-thrift /bin/bash -exc ' #GOPATH=/thrift/test/go go install bin/... #./test/test.py | tee test.log #' {noformat} was (Author: jsirois): I'm agree 100% on the ability for any contributor to enhance CI: {noformat} $ curl --netrc -sS https://builds.apache.org/job/Thrift-precommit/config.xml | xmllint -xpath //command - docker build -t thrift $WORKSPACE/build/docker/ubuntu cat << DOCKERFILE | docker build -t jenkins-thrift - # A thrift image that adds jenkins job specific tools FROM thrift RUN apt-get update && apt-get install -y \ cppcheck \ sloccount DOCKERFILE # TODO(John Sirois): Check the ci script into the repo and have that be the docker entrypoint. docker run --rm -v $WORKSPACE:/thrift -w /thrift -t jenkins-thrift /bin/bash -exc ' sh bootstrap.sh sloccount --duplicates --wide --details lib tutorial test > sloccount.sc cppcheck --error-exitcode=0 --force --xml lib/cpp/src/ lib/c_glib/src/ tutorial/cpp/ test/cpp/ 2> cppcheck-result.xml '# TODO(John Sirois): Check the ci script into the repo and have that be the docker entrypoint. # NB: Hacks used: # 1.) --privileged is a hammer being used to get around issues like intermittent Haxe compile errors of the form: # ... # -I/usr/lib/haxe/lib/hxcpp/3,2,205/include -x c++ -frtti -Wno-invalid-offsetof ./src/thrift/test/TestMapMap_args.cpp -o/thrift/test/haxe/bin/obj/linux64-debug/9931a53a_TestMapMap_args.o # Error: /usr/lib/gcc/x86_64-linux-gnu/4.8/cc1plus: error while loading shared libraries: libcloog-isl.so.4: failed to map segment from shared object: Permission denied # Error: Build failed docker run --privileged --rm -v $WORKSPACE:/thrift -w /thrift -t jenkins-thrift /bin/bash -exc ' sh bootstrap.sh ./configure make check dist ant -f lib/java/build.xml javadoc '# TODO(John Sirois): Check the ci script into the repo and have that be the docker entrypoint. # This is disabled due to a large number of failures in the matrix presently (2/9/2016). # TODO(John Sirois): Fix or remove. #docker run --rm -v $WORKSPACE:/thrift -w /thrift -t jenkins-thrift /bin/bash -exc ' #GOPATH=/thrift/test/go go install bin/... #./test/test.py | tee test.log #' {noformat} > Cleanup and consolidate Thrift* jenkins jobs. > - > > Key: THRIFT-3620 > URL: https://issues.apache.org/jira/browse/THRIFT-3620 > Project: Thrift > Issue Type: Task > Components: Test Suite >Reporter: John Sirois >Assignee: John Sirois > > I spent some time resurrecting the > [Thrift-precommit|https://builds.apache.org/job/Thrift-precommit/] job which > vets pull-requests to use the pool of docker slaves using a dockerized CI > with guidance from [~jfarrel
[jira] [Commented] (THRIFT-3623) Fix Fix cpp/lib/test/TSSLSocketInterruptTest.cpp to use ephemeral ports
[ https://issues.apache.org/jira/browse/THRIFT-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145811#comment-15145811 ] John Sirois commented on THRIFT-3623: - Build went green @ https://builds.apache.org/job/Thrift-precommit/105/ I had the title issue ID messed up. > Fix Fix cpp/lib/test/TSSLSocketInterruptTest.cpp to use ephemeral ports > --- > > Key: THRIFT-3623 > URL: https://issues.apache.org/jira/browse/THRIFT-3623 > Project: Thrift > Issue Type: Sub-task > Components: C++ - Library, Test Suite >Reporter: John Sirois >Assignee: John Sirois > > As part of work on THRIFT-3609, eliminate use of TestPortFixture.h from > TSSLSocketInterruptTest.cpp. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3626) Fix lib/cpp/test/TSocketInterruptTest.cpp to use ephemeral ports.
[ https://issues.apache.org/jira/browse/THRIFT-3626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145825#comment-15145825 ] John Sirois commented on THRIFT-3626: - Now green @ https://builds.apache.org/job/Thrift-precommit/113/ > Fix lib/cpp/test/TSocketInterruptTest.cpp to use ephemeral ports. > - > > Key: THRIFT-3626 > URL: https://issues.apache.org/jira/browse/THRIFT-3626 > Project: Thrift > Issue Type: Sub-task > Components: C++ - Library, Test Suite >Reporter: John Sirois >Assignee: John Sirois > > As part of work on THRIFT-3609, eliminate use of TestPortFixture.h from > TSocketInterruptTest.cpp. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3625) Kill unused #include "TestPortFixture.h" in lib/cpp/test/TServerTransportTest.cpp.
[ https://issues.apache.org/jira/browse/THRIFT-3625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145828#comment-15145828 ] John Sirois commented on THRIFT-3625: - Now green @ https://builds.apache.org/job/Thrift-precommit/114/ > Kill unused #include "TestPortFixture.h" in > lib/cpp/test/TServerTransportTest.cpp. > -- > > Key: THRIFT-3625 > URL: https://issues.apache.org/jira/browse/THRIFT-3625 > Project: Thrift > Issue Type: Sub-task > Components: C++ - Library, Test Suite >Reporter: John Sirois >Assignee: John Sirois > > As part of work on THRIFT-3609, eliminate use of TestPortFixture.h from > TServerTransportTest. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (THRIFT-3628) Fix lib/cpp/test/TServerIntegrationTest.cpp to use ephemeral ports
John Sirois created THRIFT-3628: --- Summary: Fix lib/cpp/test/TServerIntegrationTest.cpp to use ephemeral ports Key: THRIFT-3628 URL: https://issues.apache.org/jira/browse/THRIFT-3628 Project: Thrift Issue Type: Sub-task Components: C++ - Library, Test Suite Reporter: John Sirois Assignee: John Sirois As part of work on THRIFT-3609, eliminate use of TestPortFixture.h from TServerIntegrationTest.cpp. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3623) Fix Fix cpp/lib/test/TSSLSocketInterruptTest.cpp to use ephemeral ports
[ https://issues.apache.org/jira/browse/THRIFT-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15146172#comment-15146172 ] John Sirois commented on THRIFT-3623: - CI went green here: https://builds.apache.org/job/Thrift-precommit/125/ > Fix Fix cpp/lib/test/TSSLSocketInterruptTest.cpp to use ephemeral ports > --- > > Key: THRIFT-3623 > URL: https://issues.apache.org/jira/browse/THRIFT-3623 > Project: Thrift > Issue Type: Sub-task > Components: C++ - Library, Test Suite >Reporter: John Sirois >Assignee: John Sirois > > As part of work on THRIFT-3609, eliminate use of TestPortFixture.h from > TSSLSocketInterruptTest.cpp. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (THRIFT-3622) Fix deprecated uses of std::auto_ptr
[ https://issues.apache.org/jira/browse/THRIFT-3622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sirois reassigned THRIFT-3622: --- Assignee: John Sirois > Fix deprecated uses of std::auto_ptr > > > Key: THRIFT-3622 > URL: https://issues.apache.org/jira/browse/THRIFT-3622 > Project: Thrift > Issue Type: Task > Components: C++ - Library >Reporter: John Sirois >Assignee: John Sirois > > We have some in lib/cpp, like so: > {noformat} > JSONProtoTest.cpp:35:13: warning: ‘template class std::auto_ptr’ is > deprecated [-Wdeprecated-declarations] > static std::auto_ptr ooe; > ^ > In file included from /usr/include/c++/5.3.0/bits/locale_conv.h:41:0, > from /usr/include/c++/5.3.0/locale:43, > from /usr/include/c++/5.3.0/iomanip:43, > from JSONProtoTest.cpp:22: > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3622) Fix deprecated uses of std::auto_ptr
[ https://issues.apache.org/jira/browse/THRIFT-3622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15146218#comment-15146218 ] John Sirois commented on THRIFT-3622: - CI went green here: https://builds.apache.org/job/Thrift-precommit/129/ > Fix deprecated uses of std::auto_ptr > > > Key: THRIFT-3622 > URL: https://issues.apache.org/jira/browse/THRIFT-3622 > Project: Thrift > Issue Type: Task > Components: C++ - Library >Reporter: John Sirois >Assignee: John Sirois > > We have some in lib/cpp, like so: > {noformat} > JSONProtoTest.cpp:35:13: warning: ‘template class std::auto_ptr’ is > deprecated [-Wdeprecated-declarations] > static std::auto_ptr ooe; > ^ > In file included from /usr/include/c++/5.3.0/bits/locale_conv.h:41:0, > from /usr/include/c++/5.3.0/locale:43, > from /usr/include/c++/5.3.0/iomanip:43, > from JSONProtoTest.cpp:22: > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3609) Remove or replace TestPortFixture.h
[ https://issues.apache.org/jira/browse/THRIFT-3609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15146588#comment-15146588 ] John Sirois commented on THRIFT-3609: - Removal: https://github.com/apache/thrift/pull/869 > Remove or replace TestPortFixture.h > --- > > Key: THRIFT-3609 > URL: https://issues.apache.org/jira/browse/THRIFT-3609 > Project: Thrift > Issue Type: Improvement > Components: C++ - Library, Test Suite >Reporter: John Sirois >Assignee: John Sirois > > Today 6 cpp lib tests use > [TestPortFixture.h|https://github.com/apache/thrift/blob/1f6e380c5d07686e4cd8c2b172300a1ba7fbd8b9/lib/cpp/test/TestPortFixture.h]. > THRIFT-3608 experimented with correcting one of these cases in > {{lib/cpp/test/SecurityTest.cpp}}, but a general cleanup should remove all > uses of static ports in the cpp tests to improve robustness and open the path > to parallel testing - at least in the cpp libs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)