[jira] [Created] (THRIFT-3752) nil collections are serialized as empty collections
John Sirois created THRIFT-3752:
---
Summary: nil collections are serialized as empty collections
Key: THRIFT-3752
URL: https://issues.apache.org/jira/browse/THRIFT-3752
Project: Thrift
Issue Type: Bug
Components: Go - Compiler
Reporter: John Sirois
See discussion here: https://reviews.apache.org/r/45193/
This is likely related to THRIFT-3700.
In short, for this struct:
{noformat}
struct TaskQuery {
4: optional set taskIds
}
{noformat}
The following go struct is generated:
{noformat}
type TaskQuery struct {
TaskIds map[string]bool `thrift:"taskIds,4" json:"taskIds"`
}
{noformat}
This is all well and good, since {{TaskQuery{}.TaskIds == nil}}; ie the
{{TaskIds}} collection field is - by default - unset.
The problem is in the serialization for the field, which wipes out the unset
({{nil}}) vs empty collection distinction at the wire level:
{noformat}
func (p *TaskQuery) writeField4(oprot thrift.TProtocol) (err error) {
if err := oprot.WriteFieldBegin("taskIds", thrift.SET, 4); err != nil {
return thrift.PrependError(fmt.Sprintf("%T write field begin
error 4:taskIds: ", p), err)
}
if err := oprot.WriteSetBegin(thrift.STRING, len(p.TaskIds)); err !=
nil {
return thrift.PrependError("error writing set begin: ", err)
}
for v, _ := range p.TaskIds {
if err := oprot.WriteString(string(v)); err != nil {
return thrift.PrependError(fmt.Sprintf("%T. (0) field
write error: ", p), err)
}
}
if err := oprot.WriteSetEnd(); err != nil {
return thrift.PrependError("error writing set end: ", err)
}
if err := oprot.WriteFieldEnd(); err != nil {
return thrift.PrependError(fmt.Sprintf("%T write field end
error 4:taskIds: ", p), err)
}
return err
}
{noformat}
So on the receiving end of the wire, a {{nil}} collection is turned into an
empty collection and so unset-ness cannot be distinguished from set but empty.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (THRIFT-3752) nil collections are serialized as empty collections
[
https://issues.apache.org/jira/browse/THRIFT-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Sirois reassigned THRIFT-3752:
---
Assignee: John Sirois
> nil collections are serialized as empty collections
> ---
>
> Key: THRIFT-3752
> URL: https://issues.apache.org/jira/browse/THRIFT-3752
> Project: Thrift
> Issue Type: Bug
> Components: Go - Compiler
>Reporter: John Sirois
>Assignee: John Sirois
>
> See discussion here: https://reviews.apache.org/r/45193/
> This is likely related to THRIFT-3700.
> In short, for this struct:
> {noformat}
> struct TaskQuery {
> 4: optional set taskIds
> }
> {noformat}
> The following go struct is generated:
> {noformat}
> type TaskQuery struct {
> TaskIds map[string]bool `thrift:"taskIds,4" json:"taskIds"`
> }
> {noformat}
> This is all well and good, since {{TaskQuery{}.TaskIds == nil}}; ie the
> {{TaskIds}} collection field is - by default - unset.
> The problem is in the serialization for the field, which wipes out the unset
> ({{nil}}) vs empty collection distinction at the wire level:
> {noformat}
> func (p *TaskQuery) writeField4(oprot thrift.TProtocol) (err error) {
> if err := oprot.WriteFieldBegin("taskIds", thrift.SET, 4); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T write field begin
> error 4:taskIds: ", p), err)
> }
> if err := oprot.WriteSetBegin(thrift.STRING, len(p.TaskIds)); err !=
> nil {
> return thrift.PrependError("error writing set begin: ", err)
> }
> for v, _ := range p.TaskIds {
> if err := oprot.WriteString(string(v)); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T. (0) field
> write error: ", p), err)
> }
> }
> if err := oprot.WriteSetEnd(); err != nil {
> return thrift.PrependError("error writing set end: ", err)
> }
> if err := oprot.WriteFieldEnd(); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T write field end
> error 4:taskIds: ", p), err)
> }
> return err
> }
> {noformat}
> So on the receiving end of the wire, a {{nil}} collection is turned into an
> empty collection and so unset-ness cannot be distinguished from set but empty.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (THRIFT-3700) Go Map has wrong default value when optional
[
https://issues.apache.org/jira/browse/THRIFT-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208543#comment-15208543
]
John Sirois commented on THRIFT-3700:
-
[~connorgorman] - I filed THRIFT-3752 today which is very similar - down to the
underlying thrift-using project in question! My bug may be dup to yours, but
the specific problem I found was on the serialization side of things. If you
have tie to read that bug report and comment on the mergeability of these two
issues, I'd be grateful. I'm working on a fix for THRIFT-3752 currently and
wouldn't mind knocking off a fix for this issue too.
> Go Map has wrong default value when optional
>
>
> Key: THRIFT-3700
> URL: https://issues.apache.org/jira/browse/THRIFT-3700
> Project: Thrift
> Issue Type: Bug
> Components: Go - Compiler
>Affects Versions: 0.9.3
>Reporter: Connor Gorman
>
> Template:
> {code}
> struct TaskQuery {
> 4: optional set taskIds
> }
> {code}
> renders in Golang as :
> {code}
> TaskIds map[string]bool `thrift:"taskIds,4" json:"taskIds,omitempty"`
> {code}
> In Golang, the default value for a map is simply an empty map and not nil,
> but the optional check
> {code}
> func (p *TaskQuery) IsSetTaskIds() bool {
> return p.TaskIds != nil
> }
> {code}
> checks against nil instead if an empty map
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (THRIFT-3752) nil collections are serialized as empty collections
[
https://issues.apache.org/jira/browse/THRIFT-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208762#comment-15208762
]
John Sirois commented on THRIFT-3752:
-
Hrm, so this may not be a thrift compiler bug per-se. Marking
{{TaskQuery.taskIds}} as optional yields:
{noformat}
type TaskQuery struct {
TaskIds map[string]bool `thrift:"taskIds,4"
json:"taskIds,omitempty"`
}
func (p *TaskQuery) IsSetTaskIds() bool {
return p.TaskIds != nil
}
func (p *TaskQuery) writeField4(oprot thrift.TProtocol) (err error) {
if p.IsSetTaskIds() {
if err := oprot.WriteFieldBegin("taskIds", thrift.SET, 4); err != nil {
return thrift.PrependError(fmt.Sprintf("%T write field begin error
4:taskIds: ", p), err)
}
if err := oprot.WriteSetBegin(thrift.STRING, len(p.TaskIds)); err !=
nil {
return thrift.PrependError("error writing set begin: ", err)
}
for v, _ := range p.TaskIds {
if err := oprot.WriteString(string(v)); err != nil {
return thrift.PrependError(fmt.Sprintf("%T. (0) field write
error: ", p), err)
}
}
if err := oprot.WriteSetEnd(); err != nil {
return thrift.PrependError("error writing set end: ", err)
}
if err := oprot.WriteFieldEnd(); err != nil {
return thrift.PrependError(fmt.Sprintf("%T write field end error
4:taskIds: ", p), err)
}
}
return err
}
{noformat}
The {{IsSetTaskIds}} method is new and its use to skip serialization for an
unset {{taskIds}} field is what we want.
In other words, the thirft compiler for Go has a different notion of unset
requiredness than the java compiler (The TaskQuery receiver in this case is the
Aurora scheduler - written in java). The perils of thrift's global lack of
specificity as to how to handle optional vs required vs is well
documented and the real issue here :/.
[~jfarrell] Has there been any discussion of fixing this, declaring bankruptcy,
or picking some other path to deal with incompatible behavior across the
various language backends wrt requiredness?
> nil collections are serialized as empty collections
> ---
>
> Key: THRIFT-3752
> URL: https://issues.apache.org/jira/browse/THRIFT-3752
> Project: Thrift
> Issue Type: Bug
> Components: Go - Compiler
>Reporter: John Sirois
>Assignee: John Sirois
>
> See discussion here: https://reviews.apache.org/r/45193/
> This is likely related to THRIFT-3700.
> In short, for this struct:
> {noformat}
> struct TaskQuery {
> 4: optional set taskIds
> }
> {noformat}
> The following go struct is generated:
> {noformat}
> type TaskQuery struct {
> TaskIds map[string]bool `thrift:"taskIds,4" json:"taskIds"`
> }
> {noformat}
> This is all well and good, since {{TaskQuery{}.TaskIds == nil}}; ie the
> {{TaskIds}} collection field is - by default - unset.
> The problem is in the serialization for the field, which wipes out the unset
> ({{nil}}) vs empty collection distinction at the wire level:
> {noformat}
> func (p *TaskQuery) writeField4(oprot thrift.TProtocol) (err error) {
> if err := oprot.WriteFieldBegin("taskIds", thrift.SET, 4); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T write field begin
> error 4:taskIds: ", p), err)
> }
> if err := oprot.WriteSetBegin(thrift.STRING, len(p.TaskIds)); err !=
> nil {
> return thrift.PrependError("error writing set begin: ", err)
> }
> for v, _ := range p.TaskIds {
> if err := oprot.WriteString(string(v)); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T. (0) field
> write error: ", p), err)
> }
> }
> if err := oprot.WriteSetEnd(); err != nil {
> return thrift.PrependError("error writing set end: ", err)
> }
> if err := oprot.WriteFieldEnd(); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T write field end
> error 4:taskIds: ", p), err)
> }
> return err
> }
> {noformat}
> So on the receiving end of the wire, a {{nil}} collection is turned into an
> empty collection and so unset-ness cannot be distinguished from set but empty.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (THRIFT-3752) nil collections are serialized as empty collections
[
https://issues.apache.org/jira/browse/THRIFT-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208762#comment-15208762
]
John Sirois edited comment on THRIFT-3752 at 3/23/16 4:55 PM:
--
Hrm, so this may not be a thrift compiler bug per-se. Marking
{{TaskQuery.taskIds}} as optional yields:
{noformat}
type TaskQuery struct {
TaskIds map[string]bool `thrift:"taskIds,4"
json:"taskIds,omitempty"`
}
func (p *TaskQuery) IsSetTaskIds() bool {
return p.TaskIds != nil
}
func (p *TaskQuery) writeField4(oprot thrift.TProtocol) (err error) {
if p.IsSetTaskIds() {
if err := oprot.WriteFieldBegin("taskIds", thrift.SET, 4); err != nil {
return thrift.PrependError(fmt.Sprintf("%T write field begin error
4:taskIds: ", p), err)
}
if err := oprot.WriteSetBegin(thrift.STRING, len(p.TaskIds)); err !=
nil {
return thrift.PrependError("error writing set begin: ", err)
}
for v, _ := range p.TaskIds {
if err := oprot.WriteString(string(v)); err != nil {
return thrift.PrependError(fmt.Sprintf("%T. (0) field write
error: ", p), err)
}
}
if err := oprot.WriteSetEnd(); err != nil {
return thrift.PrependError("error writing set end: ", err)
}
if err := oprot.WriteFieldEnd(); err != nil {
return thrift.PrependError(fmt.Sprintf("%T write field end error
4:taskIds: ", p), err)
}
}
return err
}
{noformat}
The {{IsSetTaskIds}} method is new and its used to skip serialization for an
unset {{taskIds}} field is what we want.
In other words, the thirft compiler for Go has a different notion of unset
requiredness than the java compiler (The TaskQuery receiver in this case is the
Aurora scheduler - written in java). The perils of thrift's global lack of
specificity as to how to handle optional vs required vs is well
documented and the real issue here :/.
[~jfarrell] Has there been any discussion of fixing this, declaring bankruptcy,
or picking some other path to deal with incompatible behavior across the
various language backends wrt requiredness?
was (Author: jsirois):
Hrm, so this may not be a thrift compiler bug per-se. Marking
{{TaskQuery.taskIds}} as optional yields:
{noformat}
type TaskQuery struct {
TaskIds map[string]bool `thrift:"taskIds,4"
json:"taskIds,omitempty"`
}
func (p *TaskQuery) IsSetTaskIds() bool {
return p.TaskIds != nil
}
func (p *TaskQuery) writeField4(oprot thrift.TProtocol) (err error) {
if p.IsSetTaskIds() {
if err := oprot.WriteFieldBegin("taskIds", thrift.SET, 4); err != nil {
return thrift.PrependError(fmt.Sprintf("%T write field begin error
4:taskIds: ", p), err)
}
if err := oprot.WriteSetBegin(thrift.STRING, len(p.TaskIds)); err !=
nil {
return thrift.PrependError("error writing set begin: ", err)
}
for v, _ := range p.TaskIds {
if err := oprot.WriteString(string(v)); err != nil {
return thrift.PrependError(fmt.Sprintf("%T. (0) field write
error: ", p), err)
}
}
if err := oprot.WriteSetEnd(); err != nil {
return thrift.PrependError("error writing set end: ", err)
}
if err := oprot.WriteFieldEnd(); err != nil {
return thrift.PrependError(fmt.Sprintf("%T write field end error
4:taskIds: ", p), err)
}
}
return err
}
{noformat}
The {{IsSetTaskIds}} method is new and its use to skip serialization for an
unset {{taskIds}} field is what we want.
In other words, the thirft compiler for Go has a different notion of unset
requiredness than the java compiler (The TaskQuery receiver in this case is the
Aurora scheduler - written in java). The perils of thrift's global lack of
specificity as to how to handle optional vs required vs is well
documented and the real issue here :/.
[~jfarrell] Has there been any discussion of fixing this, declaring bankruptcy,
or picking some other path to deal with incompatible behavior across the
various language backends wrt requiredness?
> nil collections are serialized as empty collections
> ---
>
> Key: THRIFT-3752
> URL: https://issues.apache.org/jira/browse/THRIFT-3752
> Project: Thrift
> Issue Type: Bug
> Components: Go - Compiler
>Reporter: John Sirois
>Assignee: John Sirois
>
> See discussion here: https://reviews.apache.org/r/45193/
> This is likely related to THRIFT-3700.
> In short, for this struct:
> {noformat}
> struct TaskQuery {
> 4: optional set taskIds
> }
> {noformat}
> The following go struct is generated:
> {noformat}
> type TaskQuery struct {
> TaskIds map[string]bool `thrift:"taskI
[jira] [Commented] (THRIFT-3752) nil collections are serialized as empty collections
[
https://issues.apache.org/jira/browse/THRIFT-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15209376#comment-15209376
]
John Sirois commented on THRIFT-3752:
-
OK - great. Thanks [~jensg]. After hitting this incompat in Go clients of the
java Aurora scheduler, my 1st searches after not finding thrift docs [1][2]
landed me here which made me suspect the non-comformance of thrift compiler
backend verticals was more widespread. I'll pretend its limited to only the Go
backend until I have evidence otherwise!
[1] https://twitter.github.io/scrooge/Semantics.html
[2] http://lionet.livejournal.com/66899.html
> nil collections are serialized as empty collections
> ---
>
> Key: THRIFT-3752
> URL: https://issues.apache.org/jira/browse/THRIFT-3752
> Project: Thrift
> Issue Type: Bug
> Components: Go - Compiler
>Reporter: John Sirois
>Assignee: John Sirois
>
> See discussion here: https://reviews.apache.org/r/45193/
> This is likely related to THRIFT-3700.
> In short, for this struct:
> {noformat}
> struct TaskQuery {
> 4: optional set taskIds
> }
> {noformat}
> The following go struct is generated:
> {noformat}
> type TaskQuery struct {
> TaskIds map[string]bool `thrift:"taskIds,4" json:"taskIds"`
> }
> {noformat}
> This is all well and good, since {{TaskQuery{}.TaskIds == nil}}; ie the
> {{TaskIds}} collection field is - by default - unset.
> The problem is in the serialization for the field, which wipes out the unset
> ({{nil}}) vs empty collection distinction at the wire level:
> {noformat}
> func (p *TaskQuery) writeField4(oprot thrift.TProtocol) (err error) {
> if err := oprot.WriteFieldBegin("taskIds", thrift.SET, 4); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T write field begin
> error 4:taskIds: ", p), err)
> }
> if err := oprot.WriteSetBegin(thrift.STRING, len(p.TaskIds)); err !=
> nil {
> return thrift.PrependError("error writing set begin: ", err)
> }
> for v, _ := range p.TaskIds {
> if err := oprot.WriteString(string(v)); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T. (0) field
> write error: ", p), err)
> }
> }
> if err := oprot.WriteSetEnd(); err != nil {
> return thrift.PrependError("error writing set end: ", err)
> }
> if err := oprot.WriteFieldEnd(); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T write field end
> error 4:taskIds: ", p), err)
> }
> return err
> }
> {noformat}
> So on the receiving end of the wire, a {{nil}} collection is turned into an
> empty collection and so unset-ness cannot be distinguished from set but empty.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (THRIFT-3752) nil collections are serialized as empty collections
[
https://issues.apache.org/jira/browse/THRIFT-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15213646#comment-15213646
]
John Sirois commented on THRIFT-3752:
-
Noting the not-quite-yet-minted docs are here:
https://github.com/Jens-G/thrift/blob/THRIFT-3756/doc/specs/idl.md
I'll proceed with a fix that conforms to those docs.
> nil collections are serialized as empty collections
> ---
>
> Key: THRIFT-3752
> URL: https://issues.apache.org/jira/browse/THRIFT-3752
> Project: Thrift
> Issue Type: Bug
> Components: Go - Compiler
>Reporter: John Sirois
>Assignee: John Sirois
>
> See discussion here: https://reviews.apache.org/r/45193/
> This is likely related to THRIFT-3700.
> In short, for this struct:
> {noformat}
> struct TaskQuery {
> 4: optional set taskIds
> }
> {noformat}
> The following go struct is generated:
> {noformat}
> type TaskQuery struct {
> TaskIds map[string]bool `thrift:"taskIds,4" json:"taskIds"`
> }
> {noformat}
> This is all well and good, since {{TaskQuery{}.TaskIds == nil}}; ie the
> {{TaskIds}} collection field is - by default - unset.
> The problem is in the serialization for the field, which wipes out the unset
> ({{nil}}) vs empty collection distinction at the wire level:
> {noformat}
> func (p *TaskQuery) writeField4(oprot thrift.TProtocol) (err error) {
> if err := oprot.WriteFieldBegin("taskIds", thrift.SET, 4); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T write field begin
> error 4:taskIds: ", p), err)
> }
> if err := oprot.WriteSetBegin(thrift.STRING, len(p.TaskIds)); err !=
> nil {
> return thrift.PrependError("error writing set begin: ", err)
> }
> for v, _ := range p.TaskIds {
> if err := oprot.WriteString(string(v)); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T. (0) field
> write error: ", p), err)
> }
> }
> if err := oprot.WriteSetEnd(); err != nil {
> return thrift.PrependError("error writing set end: ", err)
> }
> if err := oprot.WriteFieldEnd(); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T write field end
> error 4:taskIds: ", p), err)
> }
> return err
> }
> {noformat}
> So on the receiving end of the wire, a {{nil}} collection is turned into an
> empty collection and so unset-ness cannot be distinguished from set but empty.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (THRIFT-3752) nil collections are serialized as empty collections
[
https://issues.apache.org/jira/browse/THRIFT-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15213669#comment-15213669
]
John Sirois commented on THRIFT-3752:
-
Actually [~jensg] - I can't make sense of the spec you added for this case. If
I look at the java generated serialization code for a no-requiredness set
I find:
{noformat}
if (struct.instanceIds != null) {
oprot.writeFieldBegin(INSTANCE_IDS_FIELD_DESC);
{
oprot.writeSetBegin(new
org.apache.thrift.protocol.TSet(org.apache.thrift.protocol.TType.I32,
struct.instanceIds.size()));
for (int _iter143 : struct.instanceIds)
{
oprot.writeI32(_iter143);
}
oprot.writeSetEnd();
}
oprot.writeFieldEnd();
}
{noformat}
Your spec in THRIFT-3756 says "Write: Like required, the fields are always
written.". In the java case though, the field is only written when non-null -
which is a good thing, since it allows the read-side of the wire to correctly
interpret this field as unset.
> nil collections are serialized as empty collections
> ---
>
> Key: THRIFT-3752
> URL: https://issues.apache.org/jira/browse/THRIFT-3752
> Project: Thrift
> Issue Type: Bug
> Components: Go - Compiler
>Reporter: John Sirois
>Assignee: John Sirois
>
> See discussion here: https://reviews.apache.org/r/45193/
> This is likely related to THRIFT-3700.
> In short, for this struct:
> {noformat}
> struct TaskQuery {
> 4: optional set taskIds
> }
> {noformat}
> The following go struct is generated:
> {noformat}
> type TaskQuery struct {
> TaskIds map[string]bool `thrift:"taskIds,4" json:"taskIds"`
> }
> {noformat}
> This is all well and good, since {{TaskQuery{}.TaskIds == nil}}; ie the
> {{TaskIds}} collection field is - by default - unset.
> The problem is in the serialization for the field, which wipes out the unset
> ({{nil}}) vs empty collection distinction at the wire level:
> {noformat}
> func (p *TaskQuery) writeField4(oprot thrift.TProtocol) (err error) {
> if err := oprot.WriteFieldBegin("taskIds", thrift.SET, 4); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T write field begin
> error 4:taskIds: ", p), err)
> }
> if err := oprot.WriteSetBegin(thrift.STRING, len(p.TaskIds)); err !=
> nil {
> return thrift.PrependError("error writing set begin: ", err)
> }
> for v, _ := range p.TaskIds {
> if err := oprot.WriteString(string(v)); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T. (0) field
> write error: ", p), err)
> }
> }
> if err := oprot.WriteSetEnd(); err != nil {
> return thrift.PrependError("error writing set end: ", err)
> }
> if err := oprot.WriteFieldEnd(); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T write field end
> error 4:taskIds: ", p), err)
> }
> return err
> }
> {noformat}
> So on the receiving end of the wire, a {{nil}} collection is turned into an
> empty collection and so unset-ness cannot be distinguished from set but empty.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (THRIFT-4029) Accelerated protocols do not build from thrift-py 0.10.0 on PyPI
[
https://issues.apache.org/jira/browse/THRIFT-4029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15822950#comment-15822950
]
John Sirois commented on THRIFT-4029:
-
This really is the behavior you want - mutable artifacts cause no end of
debugging grief. It's really a good prompt to release 0.10.1 - difficulty or
ease of this aside. Having one good and 1 bad version of the same artifact
seems like it's also asking for debugging trouble and confusion for users.
> Accelerated protocols do not build from thrift-py 0.10.0 on PyPI
>
>
> Key: THRIFT-4029
> URL: https://issues.apache.org/jira/browse/THRIFT-4029
> Project: Thrift
> Issue Type: Bug
> Components: Python - Library
>Affects Versions: 0.10.0
>Reporter: Chandler May
>Assignee: Jake Farrell
>
> The thrift 0.10.0 distribution on PyPI does not include extension headers and
> a C++ (template) file, preventing the accelerated protocols from being built.
> {{pip install}} reports a brief error:
> {code}
> Running thrift-0.10.0/setup.py -q bdist_egg --dist-dir
> /tmp/easy_install-bVU8VN/thrift-0.10.0/egg-dist-tmp-ghJzGL
> cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for
> C/ObjC but not for C++
> src/ext/module.cpp:21:19: fatal error: types.h: No such file or directory
> #include "types.h"
>^
> compilation terminated.
> ()
>
> An error occurred while trying to compile with the C extension enabled
> Attempting to build without the extension now
>
> ()
> {code}
> The list of files that is not included in the distribution is as follows. In
> addition to the headers and source file mentioned above, there's a Windows
> compatibility header missing and a couple of test files missing. It looks
> like there was a file extension filter (accidentally) applied to the C++
> files at least:
> {code}
> src/ext/binary.h
> src/ext/compact.h
> src/ext/endian.h
> src/ext/protocol.h
> src/ext/protocol.tcc
> src/ext/types.h
> compat/win32/stdint.h
> test/_import_local_thrift.py
> test/thrift_json.py
> {code}
> [~jfarrell] were you who made the release? Is it possible to hotfix? My
> team has been waiting on the release of the accelerated compact protocol for
> a while, this discovery is saddening for us.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (THRIFT-3752) nil collections are serialized as empty collections
[
https://issues.apache.org/jira/browse/THRIFT-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15877173#comment-15877173
]
John Sirois commented on THRIFT-3752:
-
[~jking], I don't think so. The compiler still does the {{IsSetXXX,
WriteFieldBegin}} sequence, and the problem noted in this issue is this encodes
a different notion of "unset requiredness" than the java compiler. The issue
was specifically cross-lang incompatibility. So unless the java compiler has
been changed to have the same notion of "unset requiredness" in the intervening
time (aka if the cross-tests are enabled, complete and including all
languages), this is probably still an issue.
> nil collections are serialized as empty collections
> ---
>
> Key: THRIFT-3752
> URL: https://issues.apache.org/jira/browse/THRIFT-3752
> Project: Thrift
> Issue Type: Bug
> Components: Go - Compiler
>Reporter: John Sirois
>Assignee: John Sirois
>
> See discussion here: https://reviews.apache.org/r/45193/
> This is likely related to THRIFT-3700.
> In short, for this struct:
> {noformat}
> struct TaskQuery {
> 4: optional set taskIds
> }
> {noformat}
> The following go struct is generated:
> {noformat}
> type TaskQuery struct {
> TaskIds map[string]bool `thrift:"taskIds,4" json:"taskIds"`
> }
> {noformat}
> This is all well and good, since {{TaskQuery{}.TaskIds == nil}}; ie the
> {{TaskIds}} collection field is - by default - unset.
> The problem is in the serialization for the field, which wipes out the unset
> ({{nil}}) vs empty collection distinction at the wire level:
> {noformat}
> func (p *TaskQuery) writeField4(oprot thrift.TProtocol) (err error) {
> if err := oprot.WriteFieldBegin("taskIds", thrift.SET, 4); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T write field begin
> error 4:taskIds: ", p), err)
> }
> if err := oprot.WriteSetBegin(thrift.STRING, len(p.TaskIds)); err !=
> nil {
> return thrift.PrependError("error writing set begin: ", err)
> }
> for v, _ := range p.TaskIds {
> if err := oprot.WriteString(string(v)); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T. (0) field
> write error: ", p), err)
> }
> }
> if err := oprot.WriteSetEnd(); err != nil {
> return thrift.PrependError("error writing set end: ", err)
> }
> if err := oprot.WriteFieldEnd(); err != nil {
> return thrift.PrependError(fmt.Sprintf("%T write field end
> error 4:taskIds: ", p), err)
> }
> return err
> }
> {noformat}
> So on the receiving end of the wire, a {{nil}} collection is turned into an
> empty collection and so unset-ness cannot be distinguished from set but empty.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (THRIFT-2621) Add basic default hash code method to python
[ https://issues.apache.org/jira/browse/THRIFT-2621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15005212#comment-15005212 ] John Sirois commented on THRIFT-2621: - Maps and lists (and sets) are common struct and service method parameters and it's not uncommon for these container types to contain structs. When this contained structs themselves have maps, lit's or sets, thinges go boom. Is there any way to work around blowing up when reading these types off the wire? > Add basic default hash code method to python > > > Key: THRIFT-2621 > URL: https://issues.apache.org/jira/browse/THRIFT-2621 > Project: Thrift > Issue Type: Improvement > Components: Python - Compiler >Reporter: Jens Geyer >Assignee: Kishor Patil > Fix For: 0.9.2 > > Attachments: > THRIFT-2621_Add_basic_default_hash_code_method_to_python.patch > > > GitHub user kishorvpatil opened a pull request: > https://github.com/apache/thrift/pull/156 > Add basic default hash code method to python, > It doesn't work for maps/lists. This based off Nathan Matz's patch for > storm branch 6d4b008512cd9fcd5a6bb598d09f35eaf2dc7412 > > > You can merge this pull request into a Git repository by running: > $ git pull https://github.com/kishorvpatil/thrift thrift_patch > Alternatively you can review and apply these changes as the patch at: > https://github.com/apache/thrift/pull/156.patch > To close this pull request, make a commit to your master/trunk branch > with (at least) the following in the commit message: > This closes #156 > > > commit f22d578579ab6a78fd07a99a59c2d4af2bb8d3f5 > Author: Kishor Patil > Date: 2014-07-10T20:44:41Z > Adding default hash function to python > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3388) hash doesn't work on set/list
[
https://issues.apache.org/jira/browse/THRIFT-3388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15005591#comment-15005591
]
John Sirois commented on THRIFT-3388:
-
[~jensg] One appropriate fix seems like a revert. THRIFT-2621 ~lifted a patch
striaght from
[here|https://github.com/nathanmarz/thrift/commit/6d4b008512cd9fcd5a6bb598d09f35eaf2dc7412]
that was appropriate in its original context but it does not seem appropriate
to ship with the official Apache thrift. It breaks very common use cases in
thrift, namely using map, set, list. It would help to understand
[~kishorvpatil]'s use case for THRIFT-2621, since a revert would break
presumably break his work. My guess is wanting to avoid writing custom
wrappers or keys for thrift structs for use in unit test comparisons and as
members of application-level dicts and sets.
Increasing the fix complexity, but being sensitive to [~kishorvpatil]'s use
case would mean adding deep hash/eq as an option, there is precedent for this
in {{java:hashcode}}.
Interested in your opinions in particular since I'm not familiar with the
thrift community or its protocols on things like this.
> hash doesn't work on set/list
> -
>
> Key: THRIFT-3388
> URL: https://issues.apache.org/jira/browse/THRIFT-3388
> Project: Thrift
> Issue Type: Bug
> Components: Python - Compiler
>Affects Versions: 0.9.3
> Environment: ubuntu 14.04 python 2.6.9
>Reporter: Michi Mutsuzaki
>
> THRIFT-2621 added hash function to thrift generated classes, but certain
> types like set or list cannot be hashed. One solution is to convert them to
> immutable types (fronzenset, tuple) before calling hash() on them.
> {noformat}
> $ thrift -version
> Thrift version 0.9.3
> $ cat test.thrift
> struct Test {
> 1: required set test
> }
> $ thrift --gen py -out . test.thrift
> $ cat test.py
> from test.ttypes import Test
> from thrift.protocol import TBinaryProtocol
> from thrift.transport import TTransport
> # serialize
> t = Test(test=set(["a"]))
> tout = TTransport.TMemoryBuffer()
> pout = TBinaryProtocol.TBinaryProtocol(tout)
> t.write(pout)
> # deserialize
> tin = TTransport.TMemoryBuffer(tout.getvalue())
> pin = TBinaryProtocol.TBinaryProtocol(tin)
> t2 = Test()
> t2.read(pin)
> # put the deserialized object to a set
> a = set([t2])
> $ python test.py
> Traceback (most recent call last):
> File "test.py", line 18, in
> a = set([t2])
> File "/tmp/test/test/ttypes.py", line 81, in __hash__
> value = (value * 31) ^ hash(self.test)
> TypeError: unhashable type: 'set'
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (THRIFT-3388) hash doesn't work on set/list
[
https://issues.apache.org/jira/browse/THRIFT-3388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15005609#comment-15005609
]
John Sirois commented on THRIFT-3388:
-
My thought is that THRIFT-2621 makes thrift python un-useable under 0.9.2+, so
I'd classify it not as "does not solve the whole issue", but instead as it
breaks thrift python for almost all real world cases. As such I favor a revert
under the premise that ~no-one can actually be using thrift python support
under 0.9.2+ because of this break in the deserialization pipeline inside
thrift itself. Although confident of this, the patch was accepted and so I was
hoping to get [~kishorvpatil]'s thoughts before posting a revert patch as a
proposal.
> hash doesn't work on set/list
> -
>
> Key: THRIFT-3388
> URL: https://issues.apache.org/jira/browse/THRIFT-3388
> Project: Thrift
> Issue Type: Bug
> Components: Python - Compiler
>Affects Versions: 0.9.3
> Environment: ubuntu 14.04 python 2.6.9
>Reporter: Michi Mutsuzaki
>
> THRIFT-2621 added hash function to thrift generated classes, but certain
> types like set or list cannot be hashed. One solution is to convert them to
> immutable types (fronzenset, tuple) before calling hash() on them.
> {noformat}
> $ thrift -version
> Thrift version 0.9.3
> $ cat test.thrift
> struct Test {
> 1: required set test
> }
> $ thrift --gen py -out . test.thrift
> $ cat test.py
> from test.ttypes import Test
> from thrift.protocol import TBinaryProtocol
> from thrift.transport import TTransport
> # serialize
> t = Test(test=set(["a"]))
> tout = TTransport.TMemoryBuffer()
> pout = TBinaryProtocol.TBinaryProtocol(tout)
> t.write(pout)
> # deserialize
> tin = TTransport.TMemoryBuffer(tout.getvalue())
> pin = TBinaryProtocol.TBinaryProtocol(tin)
> t2 = Test()
> t2.read(pin)
> # put the deserialized object to a set
> a = set([t2])
> $ python test.py
> Traceback (most recent call last):
> File "test.py", line 18, in
> a = set([t2])
> File "/tmp/test/test/ttypes.py", line 81, in __hash__
> value = (value * 31) ^ hash(self.test)
> TypeError: unhashable type: 'set'
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (THRIFT-3388) hash doesn't work on set/list
[
https://issues.apache.org/jira/browse/THRIFT-3388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15009596#comment-15009596
]
John Sirois commented on THRIFT-3388:
-
Thank you [~jensg]!
Does 1.0 have a vague timeline for release or totally unknown?
> hash doesn't work on set/list
> -
>
> Key: THRIFT-3388
> URL: https://issues.apache.org/jira/browse/THRIFT-3388
> Project: Thrift
> Issue Type: Bug
> Components: Python - Compiler
>Affects Versions: 0.9.3
> Environment: ubuntu 14.04 python 2.6.9
>Reporter: Michi Mutsuzaki
>Assignee: Jens Geyer
> Fix For: 1.0
>
> Attachments: THRIFT-3388-hash-doesn-t-work-on-set-list.patch
>
>
> THRIFT-2621 added hash function to thrift generated classes, but certain
> types like set or list cannot be hashed. One solution is to convert them to
> immutable types (fronzenset, tuple) before calling hash() on them.
> {noformat}
> $ thrift -version
> Thrift version 0.9.3
> $ cat test.thrift
> struct Test {
> 1: required set test
> }
> $ thrift --gen py -out . test.thrift
> $ cat test.py
> from test.ttypes import Test
> from thrift.protocol import TBinaryProtocol
> from thrift.transport import TTransport
> # serialize
> t = Test(test=set(["a"]))
> tout = TTransport.TMemoryBuffer()
> pout = TBinaryProtocol.TBinaryProtocol(tout)
> t.write(pout)
> # deserialize
> tin = TTransport.TMemoryBuffer(tout.getvalue())
> pin = TBinaryProtocol.TBinaryProtocol(tin)
> t2 = Test()
> t2.read(pin)
> # put the deserialized object to a set
> a = set([t2])
> $ python test.py
> Traceback (most recent call last):
> File "test.py", line 18, in
> a = set([t2])
> File "/tmp/test/test/ttypes.py", line 81, in __hash__
> value = (value * 31) ^ hash(self.test)
> TypeError: unhashable type: 'set'
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (THRIFT-3583) Add support for generating immutable java stubs for structs and unions
John Sirois created THRIFT-3583:
---
Summary: Add support for generating immutable java stubs for
structs and unions
Key: THRIFT-3583
URL: https://issues.apache.org/jira/browse/THRIFT-3583
Project: Thrift
Issue Type: New Feature
Components: Java - Compiler, Java - Library
Reporter: John Sirois
In certain domains and depending on consuming project coding style, having
thrift entities (structs and unions) be immutable by default is advantageous to
both reasoning about code safety and correctness as well as certain performance
optimizations. An example of such a domain/project is found in Apache Aurora
which uses thrift to define its core data model, RPC API and as its medium for
stable storage serialization. Aurora must ensure thrift structs used as keys
in sets and maps are immutable and it must be able to perform bulk calculations
against these objects efficiently. Both requirements are hindered by the
thrift java generated code which is fundamentally mutable. Today Aurora works
around this by generating immutable wrappers with a custom python script, but
ideally it could just say {{thrift --gen java:immutable}} or {{thrift --gen
java_immut}} and opt in to immutable generated entities. An experiment writing
an immutable code generator using the Facebook swift library is discussed
[here|http://markmail.org/message/a6sdqcelgokw6mwz] and implemented
[here|https://reviews.apache.org/r/42748]. Performance improvements for the
Aurora use cases are documented [here|https://goo.gl/gR8zgu].
The concrete proposal is to add a new mode of java codegen (I think a new gen
lang, like {{java_immut}} will probably be needed vs another option to the
existing {{java}} gen lang) that generates immutable thrift entities. This is
harder than it sounds given the goal of leveraging the existing java support
lib as much as possible since the existing supporting thrift java lib assumes
mutable entities at its core with wide ripple. This starts in
[TBase|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TBase.java#L37]
and flows from there up through the [deserialization
stack|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TDeserializer.java#L70-L86].
Although not insurmountable, some thought will need to be given to a
restructuring of these mutable-assuming APIs into read and write parts that the
existing interfaces can be re-composed on top of to make way for the immutable
option. The alternative is to write a whole new support lib like {{javame}}
has done - not a 1st choice!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (THRIFT-3583) Add support for generating immutable java stubs for structs and unions
[
https://issues.apache.org/jira/browse/THRIFT-3583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Sirois updated THRIFT-3583:
Description:
In certain domains and depending on consuming project coding style, having
thrift entities (structs and unions) be immutable by default is advantageous to
both reasoning about code safety and correctness as well as certain performance
optimizations. An example of such a domain/project is found in Apache Aurora
which uses thrift to define its core data model, RPC API and as its medium for
stable storage serialization. Aurora must ensure thrift structs used as keys
in sets and maps are immutable and it must be able to perform bulk calculations
against these objects efficiently. Both requirements are hindered by the
thrift java generated code which is fundamentally mutable. Today Aurora works
around this by generating immutable wrappers with a custom python script, but
ideally it could just say {{thrift --gen java:immutable}} or {{thrift --gen
java_immut}} and opt in to immutable generated entities. An experiment writing
an immutable code generator using the Facebook
[swift|https://github.com/facebook/swift] library is discussed
[here|http://markmail.org/message/a6sdqcelgokw6mwz] and implemented
[here|https://reviews.apache.org/r/42748]. Performance improvements for the
Aurora use cases are documented [here|https://goo.gl/gR8zgu].
The concrete proposal is to add a new mode of java codegen (I think a new gen
lang, like {{java_immut}} will probably be needed vs another option to the
existing {{java}} gen lang) that generates immutable thrift entities. This is
harder than it sounds given the goal of leveraging the existing java support
lib as much as possible since the existing supporting thrift java lib assumes
mutable entities at its core with wide ripple. This starts in
[TBase|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TBase.java#L37]
and flows from there up through the [deserialization
stack|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TDeserializer.java#L70-L86].
Although not insurmountable, some thought will need to be given to a
restructuring of these mutable-assuming APIs into read and write parts that the
existing interfaces can be re-composed on top of to make way for the immutable
option. The alternative is to write a whole new support lib like {{javame}}
has done - not a 1st choice!
was:
In certain domains and depending on consuming project coding style, having
thrift entities (structs and unions) be immutable by default is advantageous to
both reasoning about code safety and correctness as well as certain performance
optimizations. An example of such a domain/project is found in Apache Aurora
which uses thrift to define its core data model, RPC API and as its medium for
stable storage serialization. Aurora must ensure thrift structs used as keys
in sets and maps are immutable and it must be able to perform bulk calculations
against these objects efficiently. Both requirements are hindered by the
thrift java generated code which is fundamentally mutable. Today Aurora works
around this by generating immutable wrappers with a custom python script, but
ideally it could just say {{thrift --gen java:immutable}} or {{thrift --gen
java_immut}} and opt in to immutable generated entities. An experiment writing
an immutable code generator using the Facebook swift library is discussed
[here|http://markmail.org/message/a6sdqcelgokw6mwz] and implemented
[here|https://reviews.apache.org/r/42748]. Performance improvements for the
Aurora use cases are documented [here|https://goo.gl/gR8zgu].
The concrete proposal is to add a new mode of java codegen (I think a new gen
lang, like {{java_immut}} will probably be needed vs another option to the
existing {{java}} gen lang) that generates immutable thrift entities. This is
harder than it sounds given the goal of leveraging the existing java support
lib as much as possible since the existing supporting thrift java lib assumes
mutable entities at its core with wide ripple. This starts in
[TBase|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TBase.java#L37]
and flows from there up through the [deserialization
stack|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TDeserializer.java#L70-L86].
Although not insurmountable, some thought will need to be given to a
restructuring of these mutable-assuming APIs into read and write parts that the
existing interfaces can be re-composed on top of to make way for the immutable
option. The alternative is to write a whole new support lib like {{javame}}
has done - not a 1st choice!
> Add support for generating immutable java stubs for structs and unions
> --
[jira] [Commented] (THRIFT-3583) Add support for generating immutable java stubs for structs and unions
[
https://issues.apache.org/jira/browse/THRIFT-3583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15121734#comment-15121734
]
John Sirois commented on THRIFT-3583:
-
Noting THRIFT-576 which looks to attack immutability by having 2-state objects
with a mutable mode, and a frozen mode triggered with a {{freeze()}} method
fwict.
> Add support for generating immutable java stubs for structs and unions
> --
>
> Key: THRIFT-3583
> URL: https://issues.apache.org/jira/browse/THRIFT-3583
> Project: Thrift
> Issue Type: New Feature
> Components: Java - Compiler, Java - Library
>Reporter: John Sirois
>
> In certain domains and depending on consuming project coding style, having
> thrift entities (structs and unions) be immutable by default is advantageous
> to both reasoning about code safety and correctness as well as certain
> performance optimizations. An example of such a domain/project is found in
> Apache Aurora which uses thrift to define its core data model, RPC API and as
> its medium for stable storage serialization. Aurora must ensure thrift
> structs used as keys in sets and maps are immutable and it must be able to
> perform bulk calculations against these objects efficiently. Both
> requirements are hindered by the thrift java generated code which is
> fundamentally mutable. Today Aurora works around this by generating
> immutable wrappers with a custom python script, but ideally it could just say
> {{thrift --gen java:immutable}} or {{thrift --gen java_immut}} and opt in to
> immutable generated entities. An experiment writing an immutable code
> generator using the Facebook [swift|https://github.com/facebook/swift]
> library is discussed [here|http://markmail.org/message/a6sdqcelgokw6mwz] and
> implemented [here|https://reviews.apache.org/r/42748]. Performance
> improvements for the Aurora use cases are documented
> [here|https://goo.gl/gR8zgu].
> The concrete proposal is to add a new mode of java codegen (I think a new gen
> lang, like {{java_immut}} will probably be needed vs another option to the
> existing {{java}} gen lang) that generates immutable thrift entities. This
> is harder than it sounds given the goal of leveraging the existing java
> support lib as much as possible since the existing supporting thrift java lib
> assumes mutable entities at its core with wide ripple. This starts in
> [TBase|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TBase.java#L37]
> and flows from there up through the [deserialization
> stack|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TDeserializer.java#L70-L86].
> Although not insurmountable, some thought will need to be given to a
> restructuring of these mutable-assuming APIs into read and write parts that
> the existing interfaces can be re-composed on top of to make way for the
> immutable option. The alternative is to write a whole new support lib like
> {{javame}} has done - not a 1st choice!
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (THRIFT-3583) Add support for generating immutable java stubs for structs and unions
[
https://issues.apache.org/jira/browse/THRIFT-3583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15121757#comment-15121757
]
John Sirois commented on THRIFT-3583:
-
Another class of complications comes from collections. Ideally we could adopt
the unofficial offcial java std lib here and use guava
{{Immutable\{List,Set,Map\}}}; however, that leads to all sorts of jar hell
problems for consumers. If we implement our own immutable collections, they'll
both perform worse ~by definition - its a specialized lib - and present an
impedance mismatch to the guava "stdlib" users out there.
In a similar vein, is @Nullable/Optional treatment. Ideally we'd support
modern java coding style here and have nullable fields be represented with
Optional (or at least offfer that as a codegen option), but that requires
raising the floor for compatibility to java 1.8 for the immutable java codegen
option.
> Add support for generating immutable java stubs for structs and unions
> --
>
> Key: THRIFT-3583
> URL: https://issues.apache.org/jira/browse/THRIFT-3583
> Project: Thrift
> Issue Type: New Feature
> Components: Java - Compiler, Java - Library
>Reporter: John Sirois
>
> In certain domains and depending on consuming project coding style, having
> thrift entities (structs and unions) be immutable by default is advantageous
> to both reasoning about code safety and correctness as well as certain
> performance optimizations. An example of such a domain/project is found in
> Apache Aurora which uses thrift to define its core data model, RPC API and as
> its medium for stable storage serialization. Aurora must ensure thrift
> structs used as keys in sets and maps are immutable and it must be able to
> perform bulk calculations against these objects efficiently. Both
> requirements are hindered by the thrift java generated code which is
> fundamentally mutable. Today Aurora works around this by generating
> immutable wrappers with a custom python script, but ideally it could just say
> {{thrift --gen java:immutable}} or {{thrift --gen java_immut}} and opt in to
> immutable generated entities. An experiment writing an immutable code
> generator using the Facebook [swift|https://github.com/facebook/swift]
> library is discussed [here|http://markmail.org/message/a6sdqcelgokw6mwz] and
> implemented [here|https://reviews.apache.org/r/42748]. Performance
> improvements for the Aurora use cases are documented
> [here|https://goo.gl/gR8zgu].
> The concrete proposal is to add a new mode of java codegen (I think a new gen
> lang, like {{java_immut}} will probably be needed vs another option to the
> existing {{java}} gen lang) that generates immutable thrift entities. This
> is harder than it sounds given the goal of leveraging the existing java
> support lib as much as possible since the existing supporting thrift java lib
> assumes mutable entities at its core with wide ripple. This starts in
> [TBase|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TBase.java#L37]
> and flows from there up through the [deserialization
> stack|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TDeserializer.java#L70-L86].
> Although not insurmountable, some thought will need to be given to a
> restructuring of these mutable-assuming APIs into read and write parts that
> the existing interfaces can be re-composed on top of to make way for the
> immutable option. The alternative is to write a whole new support lib like
> {{javame}} has done - not a 1st choice!
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (THRIFT-3583) Add support for generating immutable java stubs for structs and unions
[
https://issues.apache.org/jira/browse/THRIFT-3583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15121911#comment-15121911
]
John Sirois commented on THRIFT-3583:
-
Started a discussion here to vet the idea and the path to take:
http://markmail.org/message/mlxyyauyvlvuxjsf
> Add support for generating immutable java stubs for structs and unions
> --
>
> Key: THRIFT-3583
> URL: https://issues.apache.org/jira/browse/THRIFT-3583
> Project: Thrift
> Issue Type: New Feature
> Components: Java - Compiler, Java - Library
>Reporter: John Sirois
>
> In certain domains and depending on consuming project coding style, having
> thrift entities (structs and unions) be immutable by default is advantageous
> to both reasoning about code safety and correctness as well as certain
> performance optimizations. An example of such a domain/project is found in
> Apache Aurora which uses thrift to define its core data model, RPC API and as
> its medium for stable storage serialization. Aurora must ensure thrift
> structs used as keys in sets and maps are immutable and it must be able to
> perform bulk calculations against these objects efficiently. Both
> requirements are hindered by the thrift java generated code which is
> fundamentally mutable. Today Aurora works around this by generating
> immutable wrappers with a custom python script, but ideally it could just say
> {{thrift --gen java:immutable}} or {{thrift --gen java_immut}} and opt in to
> immutable generated entities. An experiment writing an immutable code
> generator using the Facebook [swift|https://github.com/facebook/swift]
> library is discussed [here|http://markmail.org/message/a6sdqcelgokw6mwz] and
> implemented [here|https://reviews.apache.org/r/42748]. Performance
> improvements for the Aurora use cases are documented
> [here|https://goo.gl/gR8zgu].
> The concrete proposal is to add a new mode of java codegen (I think a new gen
> lang, like {{java_immut}} will probably be needed vs another option to the
> existing {{java}} gen lang) that generates immutable thrift entities. This
> is harder than it sounds given the goal of leveraging the existing java
> support lib as much as possible since the existing supporting thrift java lib
> assumes mutable entities at its core with wide ripple. This starts in
> [TBase|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TBase.java#L37]
> and flows from there up through the [deserialization
> stack|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TDeserializer.java#L70-L86].
> Although not insurmountable, some thought will need to be given to a
> restructuring of these mutable-assuming APIs into read and write parts that
> the existing interfaces can be re-composed on top of to make way for the
> immutable option. The alternative is to write a whole new support lib like
> {{javame}} has done - not a 1st choice!
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (THRIFT-2157) generated code would cause ClassCastException
[
https://issues.apache.org/jira/browse/THRIFT-2157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15131098#comment-15131098
]
John Sirois commented on THRIFT-2157:
-
[~jensg] or [~mbreslow] Is there more coming in this fix? As things stand the
ClassCastException will still be thrown fwict so the issue should be re-opened.
My reasoning:
>From a fresh master:
{noformat}
$ git log -1
commit d094e79de7e0bd61320f006c83c0de669363bce8
Author: Nobuaki Sukegawa
Date: Mon Feb 1 21:47:49 2016 +0900
THRIFT-3592 Add basic test client
This closes #830
{noformat}
The cast to TBase is still present in the compiler generated code; yet
TApplicationException is only a TSerializable now after this fix, not a TBase:
{noformat}
$ git grep -A1 "(org.apache.thrift.TBase)"
compiler/cpp/src/generate/t_java_generator.cc
compiler/cpp/src/generate/t_java_generator.cc:indent(f_service_) << "msg =
(org.apache.thrift.TBase)new "
compiler/cpp/src/generate/t_java_generator.cc-
"org.apache.thrift.TApplicationException(org.apache.thrift."
$ git grep "class TApplicationException" -- lib/java/
lib/java/src/org/apache/thrift/TApplicationException.java:public class
TApplicationException extends TException implements TSerializable {
$ git grep "class TException" -- lib/java/
lib/java/src/org/apache/thrift/TException.java:public class TException extends
Exception {
$ git grep "interface TSerializable" -- lib/java/
lib/java/src/org/apache/thrift/TSerializable.java:public interface
TSerializable {
{noformat}
So [~mbreslow]'s change sets the stage for the full fix but does not complete
it fwict. To complete this fix a change to
{{compiler/cpp/src/generate/t_java_generator.cc}} code generation for
{{AsyncProcessFunction.getResultHandler}}'s {{onError}} implementation is still
needed as well as a fix of the signature of
{{AsyncProcessFunction.sendResponse}}.
A minimal finishing of the fix:
{noformat}
$ git diff --full-index
diff --git a/compiler/cpp/src/generate/t_java_generator.cc
b/compiler/cpp/src/generate/t_java_generator.cc
index
7c610fb8f025e8609d01cf8bc55f498d20076848..d93fd8a28a5d9eaac46215cc51a69f5a5ec17cd2
100644
--- a/compiler/cpp/src/generate/t_java_generator.cc
+++ b/compiler/cpp/src/generate/t_java_generator.cc
@@ -3301,7 +3301,7 @@ void
t_java_generator::generate_process_async_function(t_service* tservice, t_fu
if (!tfunction->is_oneway()) {
indent(f_service_) << "byte msgType =
org.apache.thrift.protocol.TMessageType.REPLY;" << endl;
-indent(f_service_) << "org.apache.thrift.TBase msg;" << endl;
+indent(f_service_) << "org.apache.thrift.TSerializable msg;" << endl;
indent(f_service_) << resultname << " result = new " << resultname <<
"();" << endl;
t_struct* xs = tfunction->get_xceptions();
@@ -3327,7 +3327,7 @@ void
t_java_generator::generate_process_async_function(t_service* tservice, t_fu
indent(f_service_) << "{" << endl;
indent_up();
indent(f_service_) << "msgType =
org.apache.thrift.protocol.TMessageType.EXCEPTION;" << endl;
-indent(f_service_) << "msg = (org.apache.thrift.TBase)new "
+indent(f_service_) << "msg = new "
"org.apache.thrift.TApplicationException(org.apache.thrift."
"TApplicationException.INTERNAL_ERROR,
e.getMessage());" << endl;
indent_down();
diff --git a/lib/java/src/org/apache/thrift/AsyncProcessFunction.java
b/lib/java/src/org/apache/thrift/AsyncProcessFunction.java
index
799e02d58ace7a7fc29e70168f4d785a0b61599f..8537e32518844b682a555e0dcbfd417086c31597
100644
--- a/lib/java/src/org/apache/thrift/AsyncProcessFunction.java
+++ b/lib/java/src/org/apache/thrift/AsyncProcessFunction.java
@@ -43,7 +43,7 @@ public abstract class AsyncProcessFunction {
return methodName;
}
-public void sendResponse(final AbstractNonblockingServer.AsyncFrameBuffer
fb, final TBase result, final byte type, final int seqid) throws TException {
+public void sendResponse(final AbstractNonblockingServer.AsyncFrameBuffer
fb, final TSerializable result, final byte type, final int seqid) throws
TException {
TProtocol oprot = fb.getOutputProtocol();
oprot.writeMessageBegin(new TMessage(getMethodName(), type, seqid));
{noformat}
That passes {{make check}} (well, go tests hang, but that also occurs on clean
master :/, see [\[1\]|#1] below), but that said, it seems to me a unit test
should be required to close out this issue as truly resolved.
I'm happy to work up this missing patch with a unit test if everyone concurs
the fix is incomplete.
{anchor:1} \[1\] master go hang
{noformat}
make check
...
/home/jsirois/.rvm/gems/ruby-2.1.4@global/bin/bundle exec
/home/jsirois/.rvm/rubies/ruby-2.1.4/bin/ruby -I. test_suite.rb
Loaded suite test_suite
Started
Finished in 0.0015732 seconds.
--
[jira] [Commented] (THRIFT-3583) Add support for generating immutable java stubs for structs and unions
[
https://issues.apache.org/jira/browse/THRIFT-3583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15131140#comment-15131140
]
John Sirois commented on THRIFT-3583:
-
Ran into this 1/2-way refactoring of TBase when rebasing my branch :/ :
THRIFT-2157
I'll get that finished so my refactor can properly take it into account.
> Add support for generating immutable java stubs for structs and unions
> --
>
> Key: THRIFT-3583
> URL: https://issues.apache.org/jira/browse/THRIFT-3583
> Project: Thrift
> Issue Type: New Feature
> Components: Java - Compiler, Java - Library
>Reporter: John Sirois
>
> In certain domains and depending on consuming project coding style, having
> thrift entities (structs and unions) be immutable by default is advantageous
> to both reasoning about code safety and correctness as well as certain
> performance optimizations. An example of such a domain/project is found in
> Apache Aurora which uses thrift to define its core data model, RPC API and as
> its medium for stable storage serialization. Aurora must ensure thrift
> structs used as keys in sets and maps are immutable and it must be able to
> perform bulk calculations against these objects efficiently. Both
> requirements are hindered by the thrift java generated code which is
> fundamentally mutable. Today Aurora works around this by generating
> immutable wrappers with a custom python script, but ideally it could just say
> {{thrift --gen java:immutable}} or {{thrift --gen java_immut}} and opt in to
> immutable generated entities. An experiment writing an immutable code
> generator using the Facebook [swift|https://github.com/facebook/swift]
> library is discussed [here|http://markmail.org/message/a6sdqcelgokw6mwz] and
> implemented [here|https://reviews.apache.org/r/42748]. Performance
> improvements for the Aurora use cases are documented
> [here|https://goo.gl/gR8zgu].
> The concrete proposal is to add a new mode of java codegen (I think a new gen
> lang, like {{java_immut}} will probably be needed vs another option to the
> existing {{java}} gen lang) that generates immutable thrift entities. This
> is harder than it sounds given the goal of leveraging the existing java
> support lib as much as possible since the existing supporting thrift java lib
> assumes mutable entities at its core with wide ripple. This starts in
> [TBase|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TBase.java#L37]
> and flows from there up through the [deserialization
> stack|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TDeserializer.java#L70-L86].
> Although not insurmountable, some thought will need to be given to a
> restructuring of these mutable-assuming APIs into read and write parts that
> the existing interfaces can be re-composed on top of to make way for the
> immutable option. The alternative is to write a whole new support lib like
> {{javame}} has done - not a 1st choice!
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (THRIFT-3583) Add support for generating immutable java stubs for structs and unions
[
https://issues.apache.org/jira/browse/THRIFT-3583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15131143#comment-15131143
]
John Sirois commented on THRIFT-3583:
-
Adding THRIFT-2157 as a blocker even though its not really one, just treating
that way to clear a clean path for this work.
> Add support for generating immutable java stubs for structs and unions
> --
>
> Key: THRIFT-3583
> URL: https://issues.apache.org/jira/browse/THRIFT-3583
> Project: Thrift
> Issue Type: New Feature
> Components: Java - Compiler, Java - Library
>Reporter: John Sirois
>
> In certain domains and depending on consuming project coding style, having
> thrift entities (structs and unions) be immutable by default is advantageous
> to both reasoning about code safety and correctness as well as certain
> performance optimizations. An example of such a domain/project is found in
> Apache Aurora which uses thrift to define its core data model, RPC API and as
> its medium for stable storage serialization. Aurora must ensure thrift
> structs used as keys in sets and maps are immutable and it must be able to
> perform bulk calculations against these objects efficiently. Both
> requirements are hindered by the thrift java generated code which is
> fundamentally mutable. Today Aurora works around this by generating
> immutable wrappers with a custom python script, but ideally it could just say
> {{thrift --gen java:immutable}} or {{thrift --gen java_immut}} and opt in to
> immutable generated entities. An experiment writing an immutable code
> generator using the Facebook [swift|https://github.com/facebook/swift]
> library is discussed [here|http://markmail.org/message/a6sdqcelgokw6mwz] and
> implemented [here|https://reviews.apache.org/r/42748]. Performance
> improvements for the Aurora use cases are documented
> [here|https://goo.gl/gR8zgu].
> The concrete proposal is to add a new mode of java codegen (I think a new gen
> lang, like {{java_immut}} will probably be needed vs another option to the
> existing {{java}} gen lang) that generates immutable thrift entities. This
> is harder than it sounds given the goal of leveraging the existing java
> support lib as much as possible since the existing supporting thrift java lib
> assumes mutable entities at its core with wide ripple. This starts in
> [TBase|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TBase.java#L37]
> and flows from there up through the [deserialization
> stack|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TDeserializer.java#L70-L86].
> Although not insurmountable, some thought will need to be given to a
> restructuring of these mutable-assuming APIs into read and write parts that
> the existing interfaces can be re-composed on top of to make way for the
> immutable option. The alternative is to write a whole new support lib like
> {{javame}} has done - not a 1st choice!
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (THRIFT-3597) `make check` hangs in go tests
John Sirois created THRIFT-3597:
---
Summary: `make check` hangs in go tests
Key: THRIFT-3597
URL: https://issues.apache.org/jira/browse/THRIFT-3597
Project: Thrift
Issue Type: Bug
Components: Go - Library
Reporter: John Sirois
I noticed this in other work and bisected to THRIFT-3251. Looks like:
{noformat}
git reset --hard f8ca05528e04a24b9f843c82c6600e4de5e42291
make check
...
Your bundle is complete!
Use `bundle show [gemname]` to see where a bundled gem is installed.
/home/jsirois/.rvm/gems/ruby-2.1.4@global/bin/bundle exec
/home/jsirois/.rvm/rubies/ruby-2.1.4/bin/ruby -I. test_suite.rb
Loaded suite test_suite
Started
Finished in 0.001784889 seconds.
--
12 tests, 25 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0
notifications
100% passed
--
6723.11 tests/s, 14006.47 assertions/s
make[2]: Leaving directory '/home/jsirois/dev/3rdparty/jsirois-thrift/test/rb'
Making check in go
make[2]: Entering directory '/home/jsirois/dev/3rdparty/jsirois-thrift/test/go'
Makefile:649: warning: overriding recipe for target 'check'
Makefile:498: warning: ignoring old recipe for target 'check'
mkdir -p src/gen
../../compiler/cpp/thrift -out src/gen --gen go:thrift_import=thrift
ThriftTest.thrift
[WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/go/ThriftTest.thrift:83]
The "byte" type is a compatibility alias for "i8". Use "i8" to emphasize the
signedness of this type.
[WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/go/ThriftTest.thrift:44]
No generator named 'noexist' could be found!
[WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/go/ThriftTest.thrift:46]
cpp generator does not accept 'noexist' as sub-namespace!
../../compiler/cpp/thrift -out src/gen --gen go:thrift_import=thrift
../StressTest.thrift
[WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/StressTest.thrift:27]
The "byte" type is a compatibility alias for "i8". Use "i8" to emphasize the
signedness of this type.
[WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/StressTest.thrift:31]
Consider using the more efficient "binary" type instead of "list".
[WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/StressTest.thrift:31]
Consider using the more efficient "binary" type instead of "list".
ln -nfs ../../../lib/go/thrift src/thrift
GOPATH=`pwd` /usr/bin/go get github.com/golang/mock/gomock
touch gopath
GOPATH=`pwd` /usr/bin/go test -v common/...
=== RUN TestAllConnection
SIGQUIT: quit
PC=0x45f9b9 m=0
goroutine 0 [idle]:
runtime.epollwait(0x4, 0x7fff2e2496b0, 0x0080, 0x0, 0x,
0x0, 0x0, 0x0, 0x0, 0x0, ...)
/usr/lib/go/src/runtime/sys_linux_amd64.s:420 +0x19
runtime.netpoll(0xa93501, 0x0)
/usr/lib/go/src/runtime/netpoll_epoll.go:68 +0x94
runtime.findrunnable(0xc82001c000, 0x0)
/usr/lib/go/src/runtime/proc1.go:1520 +0x598
runtime.schedule()
/usr/lib/go/src/runtime/proc1.go:1647 +0x267
runtime.park_m(0xc820001680)
/usr/lib/go/src/runtime/proc1.go:1706 +0x18b
runtime.mcall(0x7fff2e249dc0)
/usr/lib/go/src/runtime/asm_amd64.s:204 +0x5b
goroutine 1 [chan receive]:
testing.RunTests(0x93d3c0, 0xa8a5b0, 0x1, 0x1, 0x1)
/usr/lib/go/src/testing/testing.go:562 +0x8ad
testing.(*M).Run(0xc820051ef8, 0x0)
/usr/lib/go/src/testing/testing.go:494 +0x70
main.main()
common/_test/_testmain.go:54 +0x116
goroutine 17 [syscall, locked to thread]:
runtime.goexit()
/usr/lib/go/src/runtime/asm_amd64.s:1721 +0x1
goroutine 5 [IO wait]:
net.runtime_pollWait(0x7f1b6ce4afa0, 0x72, 0xc820010250)
/usr/lib/go/src/runtime/netpoll.go:157 +0x60
net.(*pollDesc).Wait(0xc82005c370, 0x72, 0x0, 0x0)
/usr/lib/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc82005c370, 0x0, 0x0)
/usr/lib/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).accept(0xc82005c310, 0x0, 0x7f1b6ce4b060, 0xc82000a9c0)
/usr/lib/go/src/net/fd_unix.go:408 +0x27c
net.(*TCPListener).AcceptTCP(0xc820030048, 0x, 0x0, 0x0)
/usr/lib/go/src/net/tcpsock_posix.go:254 +0x4d
net.(*TCPListener).Accept(0xc820030048, 0x0, 0x0, 0x0, 0x0)
/usr/lib/go/src/net/tcpsock_posix.go:264 +0x3d
thrift.(*TServerSocket).Accept(0xc820014640, 0x0, 0x0, 0x0, 0x0)
/home/jsirois/dev/3rdparty/jsirois-thrift/test/go/src/thrift/se
[jira] [Commented] (THRIFT-3597) `make check` hangs in go tests
[
https://issues.apache.org/jira/browse/THRIFT-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15131412#comment-15131412
]
John Sirois commented on THRIFT-3597:
-
I can't find how to self-assign but I have a patch and will post shortly.
> `make check` hangs in go tests
> --
>
> Key: THRIFT-3597
> URL: https://issues.apache.org/jira/browse/THRIFT-3597
> Project: Thrift
> Issue Type: Bug
> Components: Go - Library
>Reporter: John Sirois
>
> I noticed this in other work and bisected to THRIFT-3251. Looks like:
> {noformat}
> git reset --hard f8ca05528e04a24b9f843c82c6600e4de5e42291
> make check
> ...
> Your bundle is complete!
> Use `bundle show [gemname]` to see where a bundled gem is installed.
> /home/jsirois/.rvm/gems/ruby-2.1.4@global/bin/bundle exec
> /home/jsirois/.rvm/rubies/ruby-2.1.4/bin/ruby -I. test_suite.rb
> Loaded suite test_suite
> Started
>
> Finished in 0.001784889 seconds.
> --
> 12 tests, 25 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0
> notifications
> 100% passed
> --
> 6723.11 tests/s, 14006.47 assertions/s
> make[2]: Leaving directory '/home/jsirois/dev/3rdparty/jsirois-thrift/test/rb'
> Making check in go
> make[2]: Entering directory
> '/home/jsirois/dev/3rdparty/jsirois-thrift/test/go'
> Makefile:649: warning: overriding recipe for target 'check'
> Makefile:498: warning: ignoring old recipe for target 'check'
> mkdir -p src/gen
> ../../compiler/cpp/thrift -out src/gen --gen go:thrift_import=thrift
> ThriftTest.thrift
> [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/go/ThriftTest.thrift:83]
> The "byte" type is a compatibility alias for "i8". Use "i8" to emphasize the
> signedness of this type.
> [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/go/ThriftTest.thrift:44]
> No generator named 'noexist' could be found!
> [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/go/ThriftTest.thrift:46]
> cpp generator does not accept 'noexist' as sub-namespace!
> ../../compiler/cpp/thrift -out src/gen --gen go:thrift_import=thrift
> ../StressTest.thrift
> [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/StressTest.thrift:27]
> The "byte" type is a compatibility alias for "i8". Use "i8" to emphasize the
> signedness of this type.
> [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/StressTest.thrift:31]
> Consider using the more efficient "binary" type instead of "list".
> [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/StressTest.thrift:31]
> Consider using the more efficient "binary" type instead of "list".
> ln -nfs ../../../lib/go/thrift src/thrift
> GOPATH=`pwd` /usr/bin/go get github.com/golang/mock/gomock
> touch gopath
> GOPATH=`pwd` /usr/bin/go test -v common/...
> === RUN TestAllConnection
> SIGQUIT: quit
> PC=0x45f9b9 m=0
> goroutine 0 [idle]:
> runtime.epollwait(0x4, 0x7fff2e2496b0, 0x0080, 0x0, 0x,
> 0x0, 0x0, 0x0, 0x0, 0x0, ...)
> /usr/lib/go/src/runtime/sys_linux_amd64.s:420 +0x19
> runtime.netpoll(0xa93501, 0x0)
> /usr/lib/go/src/runtime/netpoll_epoll.go:68 +0x94
> runtime.findrunnable(0xc82001c000, 0x0)
> /usr/lib/go/src/runtime/proc1.go:1520 +0x598
> runtime.schedule()
> /usr/lib/go/src/runtime/proc1.go:1647 +0x267
> runtime.park_m(0xc820001680)
> /usr/lib/go/src/runtime/proc1.go:1706 +0x18b
> runtime.mcall(0x7fff2e249dc0)
> /usr/lib/go/src/runtime/asm_amd64.s:204 +0x5b
> goroutine 1 [chan receive]:
> testing.RunTests(0x93d3c0, 0xa8a5b0, 0x1, 0x1, 0x1)
> /usr/lib/go/src/testing/testing.go:562 +0x8ad
> testing.(*M).Run(0xc820051ef8, 0x0)
> /usr/lib/go/src/testing/testing.go:494 +0x70
> main.main()
> common/_test/_testmain.go:54 +0x116
> goroutine 17 [syscall, locked to thread]:
> runtime.goexit()
> /usr/lib/go/src/runtime/asm_amd64.s:1721 +0x1
> goroutine 5 [IO wait]:
> net.runtime_pollWait(0x7f1b6ce4afa0, 0x72, 0xc820010250)
> /usr/lib/go/src/runtime/netpoll.go:157 +0x60
> net.(*pollDesc).Wait(0xc82005c370, 0x72, 0x0, 0x0)
> /usr/lib/go/src/net/fd_poll_runtime.go:73 +0x3a
> net.(*pollDesc).WaitRead(0xc82005c370, 0x0, 0x0)
> /usr/lib/go/src/net/fd_poll_runtime.go:78 +0x36
> net.(*netFD).accept(0xc82005c310, 0x0, 0x7f1b6ce4b060, 0xc82000a9c0)
> /
[jira] [Commented] (THRIFT-3583) Add support for generating immutable java stubs for structs and unions
[
https://issues.apache.org/jira/browse/THRIFT-3583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15131416#comment-15131416
]
John Sirois commented on THRIFT-3583:
-
THRIFT-3597 doesn't really block, but hit this one too and will also fix.
> Add support for generating immutable java stubs for structs and unions
> --
>
> Key: THRIFT-3583
> URL: https://issues.apache.org/jira/browse/THRIFT-3583
> Project: Thrift
> Issue Type: New Feature
> Components: Java - Compiler, Java - Library
>Reporter: John Sirois
>
> In certain domains and depending on consuming project coding style, having
> thrift entities (structs and unions) be immutable by default is advantageous
> to both reasoning about code safety and correctness as well as certain
> performance optimizations. An example of such a domain/project is found in
> Apache Aurora which uses thrift to define its core data model, RPC API and as
> its medium for stable storage serialization. Aurora must ensure thrift
> structs used as keys in sets and maps are immutable and it must be able to
> perform bulk calculations against these objects efficiently. Both
> requirements are hindered by the thrift java generated code which is
> fundamentally mutable. Today Aurora works around this by generating
> immutable wrappers with a custom python script, but ideally it could just say
> {{thrift --gen java:immutable}} or {{thrift --gen java_immut}} and opt in to
> immutable generated entities. An experiment writing an immutable code
> generator using the Facebook [swift|https://github.com/facebook/swift]
> library is discussed [here|http://markmail.org/message/a6sdqcelgokw6mwz] and
> implemented [here|https://reviews.apache.org/r/42748]. Performance
> improvements for the Aurora use cases are documented
> [here|https://goo.gl/gR8zgu].
> The concrete proposal is to add a new mode of java codegen (I think a new gen
> lang, like {{java_immut}} will probably be needed vs another option to the
> existing {{java}} gen lang) that generates immutable thrift entities. This
> is harder than it sounds given the goal of leveraging the existing java
> support lib as much as possible since the existing supporting thrift java lib
> assumes mutable entities at its core with wide ripple. This starts in
> [TBase|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TBase.java#L37]
> and flows from there up through the [deserialization
> stack|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TDeserializer.java#L70-L86].
> Although not insurmountable, some thought will need to be given to a
> restructuring of these mutable-assuming APIs into read and write parts that
> the existing interfaces can be re-composed on top of to make way for the
> immutable option. The alternative is to write a whole new support lib like
> {{javame}} has done - not a 1st choice!
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (THRIFT-3583) Add support for generating immutable java stubs for structs and unions
[
https://issues.apache.org/jira/browse/THRIFT-3583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15131417#comment-15131417
]
John Sirois commented on THRIFT-3583:
-
[~jfarrell] any hints on how I self-assign this issue? Does that take magic
committer powers or just better eyes?
> Add support for generating immutable java stubs for structs and unions
> --
>
> Key: THRIFT-3583
> URL: https://issues.apache.org/jira/browse/THRIFT-3583
> Project: Thrift
> Issue Type: New Feature
> Components: Java - Compiler, Java - Library
>Reporter: John Sirois
>
> In certain domains and depending on consuming project coding style, having
> thrift entities (structs and unions) be immutable by default is advantageous
> to both reasoning about code safety and correctness as well as certain
> performance optimizations. An example of such a domain/project is found in
> Apache Aurora which uses thrift to define its core data model, RPC API and as
> its medium for stable storage serialization. Aurora must ensure thrift
> structs used as keys in sets and maps are immutable and it must be able to
> perform bulk calculations against these objects efficiently. Both
> requirements are hindered by the thrift java generated code which is
> fundamentally mutable. Today Aurora works around this by generating
> immutable wrappers with a custom python script, but ideally it could just say
> {{thrift --gen java:immutable}} or {{thrift --gen java_immut}} and opt in to
> immutable generated entities. An experiment writing an immutable code
> generator using the Facebook [swift|https://github.com/facebook/swift]
> library is discussed [here|http://markmail.org/message/a6sdqcelgokw6mwz] and
> implemented [here|https://reviews.apache.org/r/42748]. Performance
> improvements for the Aurora use cases are documented
> [here|https://goo.gl/gR8zgu].
> The concrete proposal is to add a new mode of java codegen (I think a new gen
> lang, like {{java_immut}} will probably be needed vs another option to the
> existing {{java}} gen lang) that generates immutable thrift entities. This
> is harder than it sounds given the goal of leveraging the existing java
> support lib as much as possible since the existing supporting thrift java lib
> assumes mutable entities at its core with wide ripple. This starts in
> [TBase|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TBase.java#L37]
> and flows from there up through the [deserialization
> stack|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TDeserializer.java#L70-L86].
> Although not insurmountable, some thought will need to be given to a
> restructuring of these mutable-assuming APIs into read and write parts that
> the existing interfaces can be re-composed on top of to make way for the
> immutable option. The alternative is to write a whole new support lib like
> {{javame}} has done - not a 1st choice!
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (THRIFT-3597) `make check` hangs in go tests
[
https://issues.apache.org/jira/browse/THRIFT-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15131432#comment-15131432
]
John Sirois commented on THRIFT-3597:
-
https://github.com/apache/thrift/pull/833
> `make check` hangs in go tests
> --
>
> Key: THRIFT-3597
> URL: https://issues.apache.org/jira/browse/THRIFT-3597
> Project: Thrift
> Issue Type: Bug
> Components: Go - Library
>Reporter: John Sirois
>
> I noticed this in other work and bisected to THRIFT-3251. Looks like:
> {noformat}
> git reset --hard f8ca05528e04a24b9f843c82c6600e4de5e42291
> make check
> ...
> Your bundle is complete!
> Use `bundle show [gemname]` to see where a bundled gem is installed.
> /home/jsirois/.rvm/gems/ruby-2.1.4@global/bin/bundle exec
> /home/jsirois/.rvm/rubies/ruby-2.1.4/bin/ruby -I. test_suite.rb
> Loaded suite test_suite
> Started
>
> Finished in 0.001784889 seconds.
> --
> 12 tests, 25 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0
> notifications
> 100% passed
> --
> 6723.11 tests/s, 14006.47 assertions/s
> make[2]: Leaving directory '/home/jsirois/dev/3rdparty/jsirois-thrift/test/rb'
> Making check in go
> make[2]: Entering directory
> '/home/jsirois/dev/3rdparty/jsirois-thrift/test/go'
> Makefile:649: warning: overriding recipe for target 'check'
> Makefile:498: warning: ignoring old recipe for target 'check'
> mkdir -p src/gen
> ../../compiler/cpp/thrift -out src/gen --gen go:thrift_import=thrift
> ThriftTest.thrift
> [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/go/ThriftTest.thrift:83]
> The "byte" type is a compatibility alias for "i8". Use "i8" to emphasize the
> signedness of this type.
> [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/go/ThriftTest.thrift:44]
> No generator named 'noexist' could be found!
> [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/go/ThriftTest.thrift:46]
> cpp generator does not accept 'noexist' as sub-namespace!
> ../../compiler/cpp/thrift -out src/gen --gen go:thrift_import=thrift
> ../StressTest.thrift
> [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/StressTest.thrift:27]
> The "byte" type is a compatibility alias for "i8". Use "i8" to emphasize the
> signedness of this type.
> [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/StressTest.thrift:31]
> Consider using the more efficient "binary" type instead of "list".
> [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/StressTest.thrift:31]
> Consider using the more efficient "binary" type instead of "list".
> ln -nfs ../../../lib/go/thrift src/thrift
> GOPATH=`pwd` /usr/bin/go get github.com/golang/mock/gomock
> touch gopath
> GOPATH=`pwd` /usr/bin/go test -v common/...
> === RUN TestAllConnection
> SIGQUIT: quit
> PC=0x45f9b9 m=0
> goroutine 0 [idle]:
> runtime.epollwait(0x4, 0x7fff2e2496b0, 0x0080, 0x0, 0x,
> 0x0, 0x0, 0x0, 0x0, 0x0, ...)
> /usr/lib/go/src/runtime/sys_linux_amd64.s:420 +0x19
> runtime.netpoll(0xa93501, 0x0)
> /usr/lib/go/src/runtime/netpoll_epoll.go:68 +0x94
> runtime.findrunnable(0xc82001c000, 0x0)
> /usr/lib/go/src/runtime/proc1.go:1520 +0x598
> runtime.schedule()
> /usr/lib/go/src/runtime/proc1.go:1647 +0x267
> runtime.park_m(0xc820001680)
> /usr/lib/go/src/runtime/proc1.go:1706 +0x18b
> runtime.mcall(0x7fff2e249dc0)
> /usr/lib/go/src/runtime/asm_amd64.s:204 +0x5b
> goroutine 1 [chan receive]:
> testing.RunTests(0x93d3c0, 0xa8a5b0, 0x1, 0x1, 0x1)
> /usr/lib/go/src/testing/testing.go:562 +0x8ad
> testing.(*M).Run(0xc820051ef8, 0x0)
> /usr/lib/go/src/testing/testing.go:494 +0x70
> main.main()
> common/_test/_testmain.go:54 +0x116
> goroutine 17 [syscall, locked to thread]:
> runtime.goexit()
> /usr/lib/go/src/runtime/asm_amd64.s:1721 +0x1
> goroutine 5 [IO wait]:
> net.runtime_pollWait(0x7f1b6ce4afa0, 0x72, 0xc820010250)
> /usr/lib/go/src/runtime/netpoll.go:157 +0x60
> net.(*pollDesc).Wait(0xc82005c370, 0x72, 0x0, 0x0)
> /usr/lib/go/src/net/fd_poll_runtime.go:73 +0x3a
> net.(*pollDesc).WaitRead(0xc82005c370, 0x0, 0x0)
> /usr/lib/go/src/net/fd_poll_runtime.go:78 +0x36
> net.(*netFD).accept(0xc82005c310, 0x0, 0x7f1b6ce4b060, 0xc82000a9c0)
> /usr/lib/go/src/net/fd_unix.go:40
[jira] [Commented] (THRIFT-3597) `make check` hangs in go tests
[
https://issues.apache.org/jira/browse/THRIFT-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15131433#comment-15131433
]
John Sirois commented on THRIFT-3597:
-
[~dimi...@gmail.com], if you have a chance to review
https://github.com/apache/thrift/pull/833 I'd be grateful.
> `make check` hangs in go tests
> --
>
> Key: THRIFT-3597
> URL: https://issues.apache.org/jira/browse/THRIFT-3597
> Project: Thrift
> Issue Type: Bug
> Components: Go - Library
>Reporter: John Sirois
>
> I noticed this in other work and bisected to THRIFT-3251. Looks like:
> {noformat}
> git reset --hard f8ca05528e04a24b9f843c82c6600e4de5e42291
> make check
> ...
> Your bundle is complete!
> Use `bundle show [gemname]` to see where a bundled gem is installed.
> /home/jsirois/.rvm/gems/ruby-2.1.4@global/bin/bundle exec
> /home/jsirois/.rvm/rubies/ruby-2.1.4/bin/ruby -I. test_suite.rb
> Loaded suite test_suite
> Started
>
> Finished in 0.001784889 seconds.
> --
> 12 tests, 25 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0
> notifications
> 100% passed
> --
> 6723.11 tests/s, 14006.47 assertions/s
> make[2]: Leaving directory '/home/jsirois/dev/3rdparty/jsirois-thrift/test/rb'
> Making check in go
> make[2]: Entering directory
> '/home/jsirois/dev/3rdparty/jsirois-thrift/test/go'
> Makefile:649: warning: overriding recipe for target 'check'
> Makefile:498: warning: ignoring old recipe for target 'check'
> mkdir -p src/gen
> ../../compiler/cpp/thrift -out src/gen --gen go:thrift_import=thrift
> ThriftTest.thrift
> [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/go/ThriftTest.thrift:83]
> The "byte" type is a compatibility alias for "i8". Use "i8" to emphasize the
> signedness of this type.
> [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/go/ThriftTest.thrift:44]
> No generator named 'noexist' could be found!
> [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/go/ThriftTest.thrift:46]
> cpp generator does not accept 'noexist' as sub-namespace!
> ../../compiler/cpp/thrift -out src/gen --gen go:thrift_import=thrift
> ../StressTest.thrift
> [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/StressTest.thrift:27]
> The "byte" type is a compatibility alias for "i8". Use "i8" to emphasize the
> signedness of this type.
> [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/StressTest.thrift:31]
> Consider using the more efficient "binary" type instead of "list".
> [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/StressTest.thrift:31]
> Consider using the more efficient "binary" type instead of "list".
> ln -nfs ../../../lib/go/thrift src/thrift
> GOPATH=`pwd` /usr/bin/go get github.com/golang/mock/gomock
> touch gopath
> GOPATH=`pwd` /usr/bin/go test -v common/...
> === RUN TestAllConnection
> SIGQUIT: quit
> PC=0x45f9b9 m=0
> goroutine 0 [idle]:
> runtime.epollwait(0x4, 0x7fff2e2496b0, 0x0080, 0x0, 0x,
> 0x0, 0x0, 0x0, 0x0, 0x0, ...)
> /usr/lib/go/src/runtime/sys_linux_amd64.s:420 +0x19
> runtime.netpoll(0xa93501, 0x0)
> /usr/lib/go/src/runtime/netpoll_epoll.go:68 +0x94
> runtime.findrunnable(0xc82001c000, 0x0)
> /usr/lib/go/src/runtime/proc1.go:1520 +0x598
> runtime.schedule()
> /usr/lib/go/src/runtime/proc1.go:1647 +0x267
> runtime.park_m(0xc820001680)
> /usr/lib/go/src/runtime/proc1.go:1706 +0x18b
> runtime.mcall(0x7fff2e249dc0)
> /usr/lib/go/src/runtime/asm_amd64.s:204 +0x5b
> goroutine 1 [chan receive]:
> testing.RunTests(0x93d3c0, 0xa8a5b0, 0x1, 0x1, 0x1)
> /usr/lib/go/src/testing/testing.go:562 +0x8ad
> testing.(*M).Run(0xc820051ef8, 0x0)
> /usr/lib/go/src/testing/testing.go:494 +0x70
> main.main()
> common/_test/_testmain.go:54 +0x116
> goroutine 17 [syscall, locked to thread]:
> runtime.goexit()
> /usr/lib/go/src/runtime/asm_amd64.s:1721 +0x1
> goroutine 5 [IO wait]:
> net.runtime_pollWait(0x7f1b6ce4afa0, 0x72, 0xc820010250)
> /usr/lib/go/src/runtime/netpoll.go:157 +0x60
> net.(*pollDesc).Wait(0xc82005c370, 0x72, 0x0, 0x0)
> /usr/lib/go/src/net/fd_poll_runtime.go:73 +0x3a
> net.(*pollDesc).WaitRead(0xc82005c370, 0x0, 0x0)
> /usr/lib/go/src/net/fd_poll_runtime.go:78 +0x36
> net.(*netFD).accept(0xc82005c310, 0x0,
[jira] [Commented] (THRIFT-3597) `make check` hangs in go tests
[
https://issues.apache.org/jira/browse/THRIFT-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15131518#comment-15131518
]
John Sirois commented on THRIFT-3597:
-
[~nsuke] - I think this is the real fix for your paper-over fix
[here|https://github.com/apache/thrift/commit/2eed686406812411199f30e64131548000bbb62a#diff-354f30a63fb0907d4ad57269548329e3R158]
Reviews are welcome.
> `make check` hangs in go tests
> --
>
> Key: THRIFT-3597
> URL: https://issues.apache.org/jira/browse/THRIFT-3597
> Project: Thrift
> Issue Type: Bug
> Components: Go - Library
>Reporter: John Sirois
>
> I noticed this in other work and bisected to THRIFT-3251. Looks like:
> {noformat}
> git reset --hard f8ca05528e04a24b9f843c82c6600e4de5e42291
> make check
> ...
> Your bundle is complete!
> Use `bundle show [gemname]` to see where a bundled gem is installed.
> /home/jsirois/.rvm/gems/ruby-2.1.4@global/bin/bundle exec
> /home/jsirois/.rvm/rubies/ruby-2.1.4/bin/ruby -I. test_suite.rb
> Loaded suite test_suite
> Started
>
> Finished in 0.001784889 seconds.
> --
> 12 tests, 25 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0
> notifications
> 100% passed
> --
> 6723.11 tests/s, 14006.47 assertions/s
> make[2]: Leaving directory '/home/jsirois/dev/3rdparty/jsirois-thrift/test/rb'
> Making check in go
> make[2]: Entering directory
> '/home/jsirois/dev/3rdparty/jsirois-thrift/test/go'
> Makefile:649: warning: overriding recipe for target 'check'
> Makefile:498: warning: ignoring old recipe for target 'check'
> mkdir -p src/gen
> ../../compiler/cpp/thrift -out src/gen --gen go:thrift_import=thrift
> ThriftTest.thrift
> [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/go/ThriftTest.thrift:83]
> The "byte" type is a compatibility alias for "i8". Use "i8" to emphasize the
> signedness of this type.
> [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/go/ThriftTest.thrift:44]
> No generator named 'noexist' could be found!
> [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/go/ThriftTest.thrift:46]
> cpp generator does not accept 'noexist' as sub-namespace!
> ../../compiler/cpp/thrift -out src/gen --gen go:thrift_import=thrift
> ../StressTest.thrift
> [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/StressTest.thrift:27]
> The "byte" type is a compatibility alias for "i8". Use "i8" to emphasize the
> signedness of this type.
> [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/StressTest.thrift:31]
> Consider using the more efficient "binary" type instead of "list".
> [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/StressTest.thrift:31]
> Consider using the more efficient "binary" type instead of "list".
> ln -nfs ../../../lib/go/thrift src/thrift
> GOPATH=`pwd` /usr/bin/go get github.com/golang/mock/gomock
> touch gopath
> GOPATH=`pwd` /usr/bin/go test -v common/...
> === RUN TestAllConnection
> SIGQUIT: quit
> PC=0x45f9b9 m=0
> goroutine 0 [idle]:
> runtime.epollwait(0x4, 0x7fff2e2496b0, 0x0080, 0x0, 0x,
> 0x0, 0x0, 0x0, 0x0, 0x0, ...)
> /usr/lib/go/src/runtime/sys_linux_amd64.s:420 +0x19
> runtime.netpoll(0xa93501, 0x0)
> /usr/lib/go/src/runtime/netpoll_epoll.go:68 +0x94
> runtime.findrunnable(0xc82001c000, 0x0)
> /usr/lib/go/src/runtime/proc1.go:1520 +0x598
> runtime.schedule()
> /usr/lib/go/src/runtime/proc1.go:1647 +0x267
> runtime.park_m(0xc820001680)
> /usr/lib/go/src/runtime/proc1.go:1706 +0x18b
> runtime.mcall(0x7fff2e249dc0)
> /usr/lib/go/src/runtime/asm_amd64.s:204 +0x5b
> goroutine 1 [chan receive]:
> testing.RunTests(0x93d3c0, 0xa8a5b0, 0x1, 0x1, 0x1)
> /usr/lib/go/src/testing/testing.go:562 +0x8ad
> testing.(*M).Run(0xc820051ef8, 0x0)
> /usr/lib/go/src/testing/testing.go:494 +0x70
> main.main()
> common/_test/_testmain.go:54 +0x116
> goroutine 17 [syscall, locked to thread]:
> runtime.goexit()
> /usr/lib/go/src/runtime/asm_amd64.s:1721 +0x1
> goroutine 5 [IO wait]:
> net.runtime_pollWait(0x7f1b6ce4afa0, 0x72, 0xc820010250)
> /usr/lib/go/src/runtime/netpoll.go:157 +0x60
> net.(*pollDesc).Wait(0xc82005c370, 0x72, 0x0, 0x0)
> /usr/lib/go/src/net/fd_poll_runtime.go:73 +0x3a
> net.(*pollDesc).WaitRead(0xc82005c370, 0x0
[jira] [Comment Edited] (THRIFT-3597) `make check` hangs in go tests
[
https://issues.apache.org/jira/browse/THRIFT-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15131518#comment-15131518
]
John Sirois edited comment on THRIFT-3597 at 2/4/16 1:31 AM:
-
[~nsuke] - I think this is the real fix for your paper-over fix
[here|https://github.com/apache/thrift/commit/2eed686406812411199f30e64131548000bbb62a#diff-354f30a63fb0907d4ad57269548329e3R158]
Reviews are appreciated.
was (Author: jsirois):
[~nsuke] - I think this is the real fix for your paper-over fix
[here|https://github.com/apache/thrift/commit/2eed686406812411199f30e64131548000bbb62a#diff-354f30a63fb0907d4ad57269548329e3R158]
Reviews are welcome.
> `make check` hangs in go tests
> --
>
> Key: THRIFT-3597
> URL: https://issues.apache.org/jira/browse/THRIFT-3597
> Project: Thrift
> Issue Type: Bug
> Components: Go - Library
>Reporter: John Sirois
>
> I noticed this in other work and bisected to THRIFT-3251. Looks like:
> {noformat}
> git reset --hard f8ca05528e04a24b9f843c82c6600e4de5e42291
> make check
> ...
> Your bundle is complete!
> Use `bundle show [gemname]` to see where a bundled gem is installed.
> /home/jsirois/.rvm/gems/ruby-2.1.4@global/bin/bundle exec
> /home/jsirois/.rvm/rubies/ruby-2.1.4/bin/ruby -I. test_suite.rb
> Loaded suite test_suite
> Started
>
> Finished in 0.001784889 seconds.
> --
> 12 tests, 25 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0
> notifications
> 100% passed
> --
> 6723.11 tests/s, 14006.47 assertions/s
> make[2]: Leaving directory '/home/jsirois/dev/3rdparty/jsirois-thrift/test/rb'
> Making check in go
> make[2]: Entering directory
> '/home/jsirois/dev/3rdparty/jsirois-thrift/test/go'
> Makefile:649: warning: overriding recipe for target 'check'
> Makefile:498: warning: ignoring old recipe for target 'check'
> mkdir -p src/gen
> ../../compiler/cpp/thrift -out src/gen --gen go:thrift_import=thrift
> ThriftTest.thrift
> [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/go/ThriftTest.thrift:83]
> The "byte" type is a compatibility alias for "i8". Use "i8" to emphasize the
> signedness of this type.
> [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/go/ThriftTest.thrift:44]
> No generator named 'noexist' could be found!
> [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/go/ThriftTest.thrift:46]
> cpp generator does not accept 'noexist' as sub-namespace!
> ../../compiler/cpp/thrift -out src/gen --gen go:thrift_import=thrift
> ../StressTest.thrift
> [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/StressTest.thrift:27]
> The "byte" type is a compatibility alias for "i8". Use "i8" to emphasize the
> signedness of this type.
> [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/StressTest.thrift:31]
> Consider using the more efficient "binary" type instead of "list".
> [WARNING:/home/jsirois/dev/3rdparty/jsirois-thrift/test/StressTest.thrift:31]
> Consider using the more efficient "binary" type instead of "list".
> ln -nfs ../../../lib/go/thrift src/thrift
> GOPATH=`pwd` /usr/bin/go get github.com/golang/mock/gomock
> touch gopath
> GOPATH=`pwd` /usr/bin/go test -v common/...
> === RUN TestAllConnection
> SIGQUIT: quit
> PC=0x45f9b9 m=0
> goroutine 0 [idle]:
> runtime.epollwait(0x4, 0x7fff2e2496b0, 0x0080, 0x0, 0x,
> 0x0, 0x0, 0x0, 0x0, 0x0, ...)
> /usr/lib/go/src/runtime/sys_linux_amd64.s:420 +0x19
> runtime.netpoll(0xa93501, 0x0)
> /usr/lib/go/src/runtime/netpoll_epoll.go:68 +0x94
> runtime.findrunnable(0xc82001c000, 0x0)
> /usr/lib/go/src/runtime/proc1.go:1520 +0x598
> runtime.schedule()
> /usr/lib/go/src/runtime/proc1.go:1647 +0x267
> runtime.park_m(0xc820001680)
> /usr/lib/go/src/runtime/proc1.go:1706 +0x18b
> runtime.mcall(0x7fff2e249dc0)
> /usr/lib/go/src/runtime/asm_amd64.s:204 +0x5b
> goroutine 1 [chan receive]:
> testing.RunTests(0x93d3c0, 0xa8a5b0, 0x1, 0x1, 0x1)
> /usr/lib/go/src/testing/testing.go:562 +0x8ad
> testing.(*M).Run(0xc820051ef8, 0x0)
> /usr/lib/go/src/testing/testing.go:494 +0x70
> main.main()
> common/_test/_testmain.go:54 +0x116
> goroutine 17 [syscall, locked to thread]:
> runtime.goexit()
> /usr/lib/go/src/runtime/asm_amd64.s:1721 +0
[jira] [Assigned] (THRIFT-3583) Add support for generating immutable java stubs for structs and unions
[
https://issues.apache.org/jira/browse/THRIFT-3583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Sirois reassigned THRIFT-3583:
---
Assignee: John Sirois
> Add support for generating immutable java stubs for structs and unions
> --
>
> Key: THRIFT-3583
> URL: https://issues.apache.org/jira/browse/THRIFT-3583
> Project: Thrift
> Issue Type: New Feature
> Components: Java - Compiler, Java - Library
>Reporter: John Sirois
>Assignee: John Sirois
>
> In certain domains and depending on consuming project coding style, having
> thrift entities (structs and unions) be immutable by default is advantageous
> to both reasoning about code safety and correctness as well as certain
> performance optimizations. An example of such a domain/project is found in
> Apache Aurora which uses thrift to define its core data model, RPC API and as
> its medium for stable storage serialization. Aurora must ensure thrift
> structs used as keys in sets and maps are immutable and it must be able to
> perform bulk calculations against these objects efficiently. Both
> requirements are hindered by the thrift java generated code which is
> fundamentally mutable. Today Aurora works around this by generating
> immutable wrappers with a custom python script, but ideally it could just say
> {{thrift --gen java:immutable}} or {{thrift --gen java_immut}} and opt in to
> immutable generated entities. An experiment writing an immutable code
> generator using the Facebook [swift|https://github.com/facebook/swift]
> library is discussed [here|http://markmail.org/message/a6sdqcelgokw6mwz] and
> implemented [here|https://reviews.apache.org/r/42748]. Performance
> improvements for the Aurora use cases are documented
> [here|https://goo.gl/gR8zgu].
> The concrete proposal is to add a new mode of java codegen (I think a new gen
> lang, like {{java_immut}} will probably be needed vs another option to the
> existing {{java}} gen lang) that generates immutable thrift entities. This
> is harder than it sounds given the goal of leveraging the existing java
> support lib as much as possible since the existing supporting thrift java lib
> assumes mutable entities at its core with wide ripple. This starts in
> [TBase|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TBase.java#L37]
> and flows from there up through the [deserialization
> stack|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TDeserializer.java#L70-L86].
> Although not insurmountable, some thought will need to be given to a
> restructuring of these mutable-assuming APIs into read and write parts that
> the existing interfaces can be re-composed on top of to make way for the
> immutable option. The alternative is to write a whole new support lib like
> {{javame}} has done - not a 1st choice!
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (THRIFT-3112) [Java] AsyncMethodCallback should be typed in generated AsyncIface
[
https://issues.apache.org/jira/browse/THRIFT-3112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15132962#comment-15132962
]
John Sirois commented on THRIFT-3112:
-
[~roger.meier] I have begun to dig into the {{make check}} failure here -
several causes - but they reveal a good deal of broken-ness in the async client
side stack. If you don't object I'd like to assign the issue to myself and
carry this home.
> [Java] AsyncMethodCallback should be typed in generated AsyncIface
> --
>
> Key: THRIFT-3112
> URL: https://issues.apache.org/jira/browse/THRIFT-3112
> Project: Thrift
> Issue Type: Improvement
> Components: Java - Compiler
>Affects Versions: 0.9.2
>Reporter: Sergei Egorov
>Assignee: Roger Meier
> Labels: easyfix
>
> AsyncMethodCallback is generic, but current Java code generator is not adding
> type info to it, and instead of:
> {code:java}
> public interface AsyncIface {
> public void ping(org.apache.thrift.async.AsyncMethodCallback
> resultHandler) throws org.apache.thrift.TException;
> }
> {code}
> we have:
> {code:java}
> public interface AsyncIface {
> public void ping(org.apache.thrift.async.AsyncMethodCallback
> resultHandler) throws org.apache.thrift.TException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Reopened] (THRIFT-2157) generated code would cause ClassCastException
[ https://issues.apache.org/jira/browse/THRIFT-2157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sirois reopened THRIFT-2157: - Assignee: John Sirois (was: Marc Breslow) Re-opening since the ClassCastException is still not solved as described below. I have a patch in-progress to complete thisbut may need to detour to fix THRIFT-3112 1st, which reveals ClassCastExceptions on the success path on client side. > generated code would cause ClassCastException > - > > Key: THRIFT-2157 > URL: https://issues.apache.org/jira/browse/THRIFT-2157 > Project: Thrift > Issue Type: Bug > Components: Java - Compiler >Affects Versions: 0.9.1 >Reporter: Dave Brosius >Assignee: John Sirois >Priority: Trivial > Fix For: 1.0 > > > Looking at the thrift generated code for cassandra, i'm seeing > msg = (org.apache.thrift.TBase)new > org.apache.thrift.TApplicationException(org.apache.thrift.TApplicationException.INTERNAL_ERROR, > e.getMessage()); > as seen here > https://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=blob;f=interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java;h=837acfc0e964249fd62720420e8f1f85d966f1a3;hb=8f202895ab9e17c3d6bd4965924fd5f1ffc27f94#l6095 > i don't see how TApplicationException can be cast to TBase, and so i'd expect > a ClassCastException there. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (THRIFT-2157) generated code would cause ClassCastException
[ https://issues.apache.org/jira/browse/THRIFT-2157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15132966#comment-15132966 ] John Sirois edited comment on THRIFT-2157 at 2/4/16 8:32 PM: - Re-opening since the ClassCastException is still not solved as described above. I have a patch in-progress to complete this but may need to detour to fix THRIFT-3112 1st, which reveals ClassCastExceptions on the success path on client side. was (Author: jsirois): Re-opening since the ClassCastException is still not solved as described below. I have a patch in-progress to complete thisbut may need to detour to fix THRIFT-3112 1st, which reveals ClassCastExceptions on the success path on client side. > generated code would cause ClassCastException > - > > Key: THRIFT-2157 > URL: https://issues.apache.org/jira/browse/THRIFT-2157 > Project: Thrift > Issue Type: Bug > Components: Java - Compiler >Affects Versions: 0.9.1 >Reporter: Dave Brosius >Assignee: John Sirois >Priority: Trivial > Fix For: 1.0 > > > Looking at the thrift generated code for cassandra, i'm seeing > msg = (org.apache.thrift.TBase)new > org.apache.thrift.TApplicationException(org.apache.thrift.TApplicationException.INTERNAL_ERROR, > e.getMessage()); > as seen here > https://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=blob;f=interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java;h=837acfc0e964249fd62720420e8f1f85d966f1a3;hb=8f202895ab9e17c3d6bd4965924fd5f1ffc27f94#l6095 > i don't see how TApplicationException can be cast to TBase, and so i'd expect > a ClassCastException there. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-2157) generated code would cause ClassCastException
[
https://issues.apache.org/jira/browse/THRIFT-2157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15132976#comment-15132976
]
John Sirois commented on THRIFT-2157:
-
Linking this to THRIFT-3112 which preliminary investigation of reveals similar
{{ClassCastException}} issues on the {{onComplete}} path as well - though this
time in the java library code.
> generated code would cause ClassCastException
> -
>
> Key: THRIFT-2157
> URL: https://issues.apache.org/jira/browse/THRIFT-2157
> Project: Thrift
> Issue Type: Bug
> Components: Java - Compiler
>Affects Versions: 0.9.1
>Reporter: Dave Brosius
>Assignee: John Sirois
>Priority: Trivial
> Fix For: 1.0
>
>
> Looking at the thrift generated code for cassandra, i'm seeing
> msg = (org.apache.thrift.TBase)new
> org.apache.thrift.TApplicationException(org.apache.thrift.TApplicationException.INTERNAL_ERROR,
> e.getMessage());
> as seen here
> https://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=blob;f=interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java;h=837acfc0e964249fd62720420e8f1f85d966f1a3;hb=8f202895ab9e17c3d6bd4965924fd5f1ffc27f94#l6095
> i don't see how TApplicationException can be cast to TBase, and so i'd expect
> a ClassCastException there.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (THRIFT-3583) Add support for generating immutable java stubs for structs and unions
[
https://issues.apache.org/jira/browse/THRIFT-3583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15132978#comment-15132978
]
John Sirois commented on THRIFT-3583:
-
Adding another blocker related to THRIFT-2157 and the {{ISerializable}}
refactor. This is another class of {{ClassCastException}}s on the
{{onComplete}} path (THRIFT-2157 deals with a {{ClassCastException}} on the
{{onError}} path).
> Add support for generating immutable java stubs for structs and unions
> --
>
> Key: THRIFT-3583
> URL: https://issues.apache.org/jira/browse/THRIFT-3583
> Project: Thrift
> Issue Type: New Feature
> Components: Java - Compiler, Java - Library
>Reporter: John Sirois
>Assignee: John Sirois
>
> In certain domains and depending on consuming project coding style, having
> thrift entities (structs and unions) be immutable by default is advantageous
> to both reasoning about code safety and correctness as well as certain
> performance optimizations. An example of such a domain/project is found in
> Apache Aurora which uses thrift to define its core data model, RPC API and as
> its medium for stable storage serialization. Aurora must ensure thrift
> structs used as keys in sets and maps are immutable and it must be able to
> perform bulk calculations against these objects efficiently. Both
> requirements are hindered by the thrift java generated code which is
> fundamentally mutable. Today Aurora works around this by generating
> immutable wrappers with a custom python script, but ideally it could just say
> {{thrift --gen java:immutable}} or {{thrift --gen java_immut}} and opt in to
> immutable generated entities. An experiment writing an immutable code
> generator using the Facebook [swift|https://github.com/facebook/swift]
> library is discussed [here|http://markmail.org/message/a6sdqcelgokw6mwz] and
> implemented [here|https://reviews.apache.org/r/42748]. Performance
> improvements for the Aurora use cases are documented
> [here|https://goo.gl/gR8zgu].
> The concrete proposal is to add a new mode of java codegen (I think a new gen
> lang, like {{java_immut}} will probably be needed vs another option to the
> existing {{java}} gen lang) that generates immutable thrift entities. This
> is harder than it sounds given the goal of leveraging the existing java
> support lib as much as possible since the existing supporting thrift java lib
> assumes mutable entities at its core with wide ripple. This starts in
> [TBase|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TBase.java#L37]
> and flows from there up through the [deserialization
> stack|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TDeserializer.java#L70-L86].
> Although not insurmountable, some thought will need to be given to a
> restructuring of these mutable-assuming APIs into read and write parts that
> the existing interfaces can be re-composed on top of to make way for the
> immutable option. The alternative is to write a whole new support lib like
> {{javame}} has done - not a 1st choice!
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (THRIFT-3583) Add support for generating immutable java stubs for structs and unions
[
https://issues.apache.org/jira/browse/THRIFT-3583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15132978#comment-15132978
]
John Sirois edited comment on THRIFT-3583 at 2/4/16 8:41 PM:
-
Adding another blocker related to THRIFT-2157 and the {{ISerializable}}
refactor. This is another class of {{ClassCastException}} s on the
{{onComplete}} path (THRIFT-2157 deals with a {{ClassCastException}} on the
{{onError}} path).
was (Author: jsirois):
Adding another blocker related to THRIFT-2157 and the {{ISerializable}}
refactor. This is another class of {{ClassCastException}}s on the
{{onComplete}} path (THRIFT-2157 deals with a {{ClassCastException}} on the
{{onError}} path).
> Add support for generating immutable java stubs for structs and unions
> --
>
> Key: THRIFT-3583
> URL: https://issues.apache.org/jira/browse/THRIFT-3583
> Project: Thrift
> Issue Type: New Feature
> Components: Java - Compiler, Java - Library
>Reporter: John Sirois
>Assignee: John Sirois
>
> In certain domains and depending on consuming project coding style, having
> thrift entities (structs and unions) be immutable by default is advantageous
> to both reasoning about code safety and correctness as well as certain
> performance optimizations. An example of such a domain/project is found in
> Apache Aurora which uses thrift to define its core data model, RPC API and as
> its medium for stable storage serialization. Aurora must ensure thrift
> structs used as keys in sets and maps are immutable and it must be able to
> perform bulk calculations against these objects efficiently. Both
> requirements are hindered by the thrift java generated code which is
> fundamentally mutable. Today Aurora works around this by generating
> immutable wrappers with a custom python script, but ideally it could just say
> {{thrift --gen java:immutable}} or {{thrift --gen java_immut}} and opt in to
> immutable generated entities. An experiment writing an immutable code
> generator using the Facebook [swift|https://github.com/facebook/swift]
> library is discussed [here|http://markmail.org/message/a6sdqcelgokw6mwz] and
> implemented [here|https://reviews.apache.org/r/42748]. Performance
> improvements for the Aurora use cases are documented
> [here|https://goo.gl/gR8zgu].
> The concrete proposal is to add a new mode of java codegen (I think a new gen
> lang, like {{java_immut}} will probably be needed vs another option to the
> existing {{java}} gen lang) that generates immutable thrift entities. This
> is harder than it sounds given the goal of leveraging the existing java
> support lib as much as possible since the existing supporting thrift java lib
> assumes mutable entities at its core with wide ripple. This starts in
> [TBase|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TBase.java#L37]
> and flows from there up through the [deserialization
> stack|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/TDeserializer.java#L70-L86].
> Although not insurmountable, some thought will need to be given to a
> restructuring of these mutable-assuming APIs into read and write parts that
> the existing interfaces can be re-composed on top of to make way for the
> immutable option. The alternative is to write a whole new support lib like
> {{javame}} has done - not a 1st choice!
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (THRIFT-3112) [Java] AsyncMethodCallback should be typed in generated AsyncIface
[
https://issues.apache.org/jira/browse/THRIFT-3112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15134786#comment-15134786
]
John Sirois commented on THRIFT-3112:
-
I spoke with Roger offline and there are no objections, so assigning this to
myself.
> [Java] AsyncMethodCallback should be typed in generated AsyncIface
> --
>
> Key: THRIFT-3112
> URL: https://issues.apache.org/jira/browse/THRIFT-3112
> Project: Thrift
> Issue Type: Improvement
> Components: Java - Compiler
>Affects Versions: 0.9.2
>Reporter: Sergei Egorov
>Assignee: Roger Meier
> Labels: easyfix
>
> AsyncMethodCallback is generic, but current Java code generator is not adding
> type info to it, and instead of:
> {code:java}
> public interface AsyncIface {
> public void ping(org.apache.thrift.async.AsyncMethodCallback
> resultHandler) throws org.apache.thrift.TException;
> }
> {code}
> we have:
> {code:java}
> public interface AsyncIface {
> public void ping(org.apache.thrift.async.AsyncMethodCallback
> resultHandler) throws org.apache.thrift.TException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (THRIFT-3112) [Java] AsyncMethodCallback should be typed in generated AsyncIface
[
https://issues.apache.org/jira/browse/THRIFT-3112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Sirois reassigned THRIFT-3112:
---
Assignee: John Sirois (was: Roger Meier)
> [Java] AsyncMethodCallback should be typed in generated AsyncIface
> --
>
> Key: THRIFT-3112
> URL: https://issues.apache.org/jira/browse/THRIFT-3112
> Project: Thrift
> Issue Type: Improvement
> Components: Java - Compiler
>Affects Versions: 0.9.2
>Reporter: Sergei Egorov
>Assignee: John Sirois
> Labels: easyfix
>
> AsyncMethodCallback is generic, but current Java code generator is not adding
> type info to it, and instead of:
> {code:java}
> public interface AsyncIface {
> public void ping(org.apache.thrift.async.AsyncMethodCallback
> resultHandler) throws org.apache.thrift.TException;
> }
> {code}
> we have:
> {code:java}
> public interface AsyncIface {
> public void ping(org.apache.thrift.async.AsyncMethodCallback
> resultHandler) throws org.apache.thrift.TException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (THRIFT-3112) [Java] AsyncMethodCallback should be typed in generated AsyncIface
[
https://issues.apache.org/jira/browse/THRIFT-3112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137318#comment-15137318
]
John Sirois commented on THRIFT-3112:
-
The fundamental problem here is that the async client stack is parametrized
over its implementation. An example from the tutorial "correctly" parametrized
today:
{code:java}
package tutorial;
public class Calculator {
public interface AsyncIface extends shared.SharedService .AsyncIface {
public void
ping(org.apache.thrift.async.AsyncMethodCallback
resultHandler) throws org.apache.thrift.TException;
public void add(int num1, int num2,
org.apache.thrift.async.AsyncMethodCallback
resultHandler) throws org.apache.thrift.TException;
...
}
public static class AsyncClient extends shared.SharedService.AsyncClient
implements AsyncIface {
public void ping(org.apache.thrift.async.AsyncMethodCallback
resultHandler) throws org.apache.thrift.TException {
...
}
public static class ping_call extends
org.apache.thrift.async.TAsyncMethodCall {
...
public void getResult() throws org.apache.thrift.TException {
...
}
}
...
}
public static class AsyncProcessor extends
shared.SharedService.AsyncProcessor {
public static class ping extends
org.apache.thrift.AsyncProcessFunction {
public AsyncMethodCallback getResultHandler(final AsyncFrameBuffer
fb, final int seqid) {
...
}
}
public static class add extends
org.apache.thrift.AsyncProcessFunction {
public AsyncMethodCallback getResultHandler(final
AsyncFrameBuffer fb, final int seqid) {
...
}
}
...
}
}
{code}
Notice in the {{AsyncIface}} that:
# The client-side AsyncMethodCallback's are not parametrized over the sync
interface method return type as you'd expect, they are parametrized by the
client implementation of the method call.
# The server-side AsyncMethodCallback's are parametrized as you'd expect - over
the sync interface method return type.
The combination of 1 and 2 is confusing, but worse 1 mixes the implementation -
{{AsyncClient}} - into the interface - {{AsyncIface}} and it forces clients to
both call {{getResult}} in their {{onComplete}} handler as well as deal with
errors. This leaves error handling split between {{onComplete}} and
{{onError}}.
It seems to me the right long-term answer is to fix the generated client code
to match the generated server code and pass the successfull completed result to
{{onComplete}} and not the {{TAsyncMethodCall}} implementation. This would be
a breaking change though and anyone using AsyncIface today would need to fix
there code on upgrade. A [github
search|https://github.com/search?utf8=%E2%9C%93&q=new+AsyncMethodCallback+language%3AJava+extension%3Ajava&type=Code&ref=advsearch&l=Java&l=]
does reveal folks actually using {{AsyncIface}} today. That said - in the 1st
page of results alone there is evidence of folks doing it "wrong" and
parametrizing in the natural way. That code must either be unused or else
written against an Apache Thrift fork.
> [Java] AsyncMethodCallback should be typed in generated AsyncIface
> --
>
> Key: THRIFT-3112
> URL: https://issues.apache.org/jira/browse/THRIFT-3112
> Project: Thrift
> Issue Type: Improvement
> Components: Java - Compiler
>Affects Versions: 0.9.2
>Reporter: Sergei Egorov
>Assignee: John Sirois
> Labels: easyfix
>
> AsyncMethodCallback is generic, but current Java code generator is not adding
> type info to it, and instead of:
> {code:java}
> public interface AsyncIface {
> public void ping(org.apache.thrift.async.AsyncMethodCallback
> resultHandler) throws org.apache.thrift.TException;
> }
> {code}
> we have:
> {code:java}
> public interface AsyncIface {
> public void ping(org.apache.thrift.async.AsyncMethodCallback
> resultHandler) throws org.apache.thrift.TException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (THRIFT-3112) [Java] AsyncMethodCallback should be typed in generated AsyncIface
[
https://issues.apache.org/jira/browse/THRIFT-3112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137540#comment-15137540
]
John Sirois commented on THRIFT-3112:
-
Proposal thread here: http://markmail.org/message/k77l6helvkh4igrv
> [Java] AsyncMethodCallback should be typed in generated AsyncIface
> --
>
> Key: THRIFT-3112
> URL: https://issues.apache.org/jira/browse/THRIFT-3112
> Project: Thrift
> Issue Type: Improvement
> Components: Java - Compiler
>Affects Versions: 0.9.2
>Reporter: Sergei Egorov
>Assignee: John Sirois
> Labels: easyfix
>
> AsyncMethodCallback is generic, but current Java code generator is not adding
> type info to it, and instead of:
> {code:java}
> public interface AsyncIface {
> public void ping(org.apache.thrift.async.AsyncMethodCallback
> resultHandler) throws org.apache.thrift.TException;
> }
> {code}
> we have:
> {code:java}
> public interface AsyncIface {
> public void ping(org.apache.thrift.async.AsyncMethodCallback
> resultHandler) throws org.apache.thrift.TException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (THRIFT-3112) [Java] AsyncMethodCallback should be typed in generated AsyncIface
[
https://issues.apache.org/jira/browse/THRIFT-3112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137591#comment-15137591
]
John Sirois commented on THRIFT-3112:
-
Straw-man RB that implements the proposal:
https://github.com/apache/thrift/pull/840
> [Java] AsyncMethodCallback should be typed in generated AsyncIface
> --
>
> Key: THRIFT-3112
> URL: https://issues.apache.org/jira/browse/THRIFT-3112
> Project: Thrift
> Issue Type: Improvement
> Components: Java - Compiler
>Affects Versions: 0.9.2
>Reporter: Sergei Egorov
>Assignee: John Sirois
> Labels: easyfix
>
> AsyncMethodCallback is generic, but current Java code generator is not adding
> type info to it, and instead of:
> {code:java}
> public interface AsyncIface {
> public void ping(org.apache.thrift.async.AsyncMethodCallback
> resultHandler) throws org.apache.thrift.TException;
> }
> {code}
> we have:
> {code:java}
> public interface AsyncIface {
> public void ping(org.apache.thrift.async.AsyncMethodCallback
> resultHandler) throws org.apache.thrift.TException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (THRIFT-2157) generated code would cause ClassCastException
[
https://issues.apache.org/jira/browse/THRIFT-2157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137652#comment-15137652
]
John Sirois commented on THRIFT-2157:
-
Turns out THRIFT-3112 does not reveal similar {{ClassCastException}} issues if
you use the generated client code correctly, but the correct usage is odd and
inconsistent with the async server code.
> generated code would cause ClassCastException
> -
>
> Key: THRIFT-2157
> URL: https://issues.apache.org/jira/browse/THRIFT-2157
> Project: Thrift
> Issue Type: Bug
> Components: Java - Compiler
>Affects Versions: 0.9.1
>Reporter: Dave Brosius
>Assignee: John Sirois
>Priority: Trivial
> Fix For: 1.0
>
>
> Looking at the thrift generated code for cassandra, i'm seeing
> msg = (org.apache.thrift.TBase)new
> org.apache.thrift.TApplicationException(org.apache.thrift.TApplicationException.INTERNAL_ERROR,
> e.getMessage());
> as seen here
> https://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=blob;f=interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java;h=837acfc0e964249fd62720420e8f1f85d966f1a3;hb=8f202895ab9e17c3d6bd4965924fd5f1ffc27f94#l6095
> i don't see how TApplicationException can be cast to TBase, and so i'd expect
> a ClassCastException there.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (THRIFT-3112) [Java] AsyncMethodCallback should be typed in generated AsyncIface
[
https://issues.apache.org/jira/browse/THRIFT-3112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Sirois updated THRIFT-3112:
Labels: (was: easyfix)
> [Java] AsyncMethodCallback should be typed in generated AsyncIface
> --
>
> Key: THRIFT-3112
> URL: https://issues.apache.org/jira/browse/THRIFT-3112
> Project: Thrift
> Issue Type: Improvement
> Components: Java - Compiler, Java - Library
>Affects Versions: 0.9.2
>Reporter: Sergei Egorov
>Assignee: John Sirois
>
> AsyncMethodCallback is generic, but current Java code generator is not adding
> type info to it, and instead of:
> {code:java}
> public interface AsyncIface {
> public void ping(org.apache.thrift.async.AsyncMethodCallback
> resultHandler) throws org.apache.thrift.TException;
> }
> {code}
> we have:
> {code:java}
> public interface AsyncIface {
> public void ping(org.apache.thrift.async.AsyncMethodCallback
> resultHandler) throws org.apache.thrift.TException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (THRIFT-3112) [Java] AsyncMethodCallback should be typed in generated AsyncIface
[
https://issues.apache.org/jira/browse/THRIFT-3112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Sirois updated THRIFT-3112:
Component/s: Java - Library
> [Java] AsyncMethodCallback should be typed in generated AsyncIface
> --
>
> Key: THRIFT-3112
> URL: https://issues.apache.org/jira/browse/THRIFT-3112
> Project: Thrift
> Issue Type: Improvement
> Components: Java - Compiler, Java - Library
>Affects Versions: 0.9.2
>Reporter: Sergei Egorov
>Assignee: John Sirois
>
> AsyncMethodCallback is generic, but current Java code generator is not adding
> type info to it, and instead of:
> {code:java}
> public interface AsyncIface {
> public void ping(org.apache.thrift.async.AsyncMethodCallback
> resultHandler) throws org.apache.thrift.TException;
> }
> {code}
> we have:
> {code:java}
> public interface AsyncIface {
> public void ping(org.apache.thrift.async.AsyncMethodCallback
> resultHandler) throws org.apache.thrift.TException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (THRIFT-2157) generated code would cause ClassCastException
[ https://issues.apache.org/jira/browse/THRIFT-2157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sirois updated THRIFT-2157: Component/s: Java - Library > generated code would cause ClassCastException > - > > Key: THRIFT-2157 > URL: https://issues.apache.org/jira/browse/THRIFT-2157 > Project: Thrift > Issue Type: Bug > Components: Java - Compiler, Java - Library >Affects Versions: 0.9.1 >Reporter: Dave Brosius >Assignee: John Sirois >Priority: Trivial > Fix For: 1.0 > > > Looking at the thrift generated code for cassandra, i'm seeing > msg = (org.apache.thrift.TBase)new > org.apache.thrift.TApplicationException(org.apache.thrift.TApplicationException.INTERNAL_ERROR, > e.getMessage()); > as seen here > https://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=blob;f=interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java;h=837acfc0e964249fd62720420e8f1f85d966f1a3;hb=8f202895ab9e17c3d6bd4965924fd5f1ffc27f94#l6095 > i don't see how TApplicationException can be cast to TBase, and so i'd expect > a ClassCastException there. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3606) TSaslClientTransport props typed too strongly
[ https://issues.apache.org/jira/browse/THRIFT-3606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15140935#comment-15140935 ] John Sirois commented on THRIFT-3606: - [~nattobean], this sounds totally reasonable to me. Are you willing to send up a patch? There are 2 methods described here: https://thrift.apache.org/docs/HowToContribute > TSaslClientTransport props typed too strongly > - > > Key: THRIFT-3606 > URL: https://issues.apache.org/jira/browse/THRIFT-3606 > Project: Thrift > Issue Type: Bug > Components: Java - Library >Affects Versions: 0.9.3 >Reporter: David Schreibman >Priority: Minor > > The constructor in TSaslClientTransport.java with signature: > public TSaslClientTransport( > String mechanism, > String authorizationId, String protocol, > String serverName, > Map props, > CallbackHandler cbh, > TTransport transport) > Is passing along the props Map to Sasl.createSaslClient(). However, > createSaslClient actually accepts props as Map. Having the props > as Map means that we cannot pass legitimate non-string > property values. For example, you cannot pass in the JGSS > javax.security.sasl.policy.credentials property which is an object of type > GSSCredential. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (THRIFT-3608) lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build.
John Sirois created THRIFT-3608:
---
Summary: lib/cpp/test/SecurityTest is flaky in jenkins
Thrift-precommit build.
Key: THRIFT-3608
URL: https://issues.apache.org/jira/browse/THRIFT-3608
Project: Thrift
Issue Type: Bug
Components: C++ - Library, Test Suite
Affects Versions: 0.9.3
Reporter: John Sirois
Assignee: John Sirois
Priority: Critical
The [Thrift-precommit|https://builds.apache.org/job/Thrift-precommit] job now
runs in the build/docker/ubuntu container and the
[lib/cpp/test/SecurityTest|https://github.com/apache/thrift/blob/master/lib/cpp/test/SecurityTest.cpp]
fails most - but not all - of the time.
Failures can be seen in the following jobs
* https://builds.apache.org/job/Thrift-precommit/60/console
* https://builds.apache.org/job/Thrift-precommit/59/console
* https://builds.apache.org/job/Thrift-precommit/58/console
* https://builds.apache.org/job/Thrift-precommit/57/console
They look like so:
{noformat}
argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest"
Running 1 test case...
Entering test suite "SecurityTest"
Entering test suite "SecurityTest"
Entering test case "ssl_security_matrix"
TEST: Server = SSLTLS, Client = SSLTLS
TEST: Server = SSLTLS, Client = SSLv3
SRV 2b9182317700 Exception: SSL_accept: wrong version number
CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure
TEST: Server = SSLTLS, Client = TLSv1_0
TEST: Server = SSLTLS, Client = TLSv1_1
TEST: Server = SSLTLS, Client = TLSv1_2
TEST: Server = SSLv3, Client = SSLTLS
CLI 2b9181f0a700 Exception: SSL_connect: unsupported protocol
SRV 2b9182317700 Exception: SSL_accept: error code: 0
TEST: Server = SSLv3, Client = SSLv3
TEST: Server = SSLv3, Client = TLSv1_0
CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure
TEST: Server = SSLv3, Client = TLSv1_1
CLI 2b9182317700 Exception: SSL_connect: wrong version number
SRV 2b9181f0a700 Exception: SSL_accept: sslv3 alert handshake failure
TEST: Server = SSLv3, Client = TLSv1_2
CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure
TEST: Server = TLSv1_0, Client = SSLTLS
TEST: Server = TLSv1_0, Client = SSLv3
SRV 2b9182317700 Exception: SSL_accept: wrong version number
CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure
TEST: Server = TLSv1_0, Client = TLSv1_0
TEST: Server = TLSv1_0, Client = TLSv1_1
CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version
TEST: Server = TLSv1_0, Client = TLSv1_2
CLI 2b9182317700 Exception: SSL_connect: wrong version number
SRV 2b9181f0a700 Exception: SSL_accept: tlsv1 alert protocol version
TEST: Server = TLSv1_1, Client = SSLTLS
TEST: Server = TLSv1_1, Client = SSLv3
SRV 2b9181f0a700 Exception: SSL_accept: wrong version number
CLI 2b9182317700 Exception: SSL_connect: sslv3 alert handshake failure
TEST: Server = TLSv1_1, Client = TLSv1_0
SRV 2b9182317700 Exception: SSL_accept: wrong version number
CLI 2b9181f0a700 Exception: SSL_connect: tlsv1 alert protocol version
TEST: Server = TLSv1_1, Client = TLSv1_1
TEST: Server = TLSv1_1, Client = TLSv1_2
SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version
CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
TEST: Server = TLSv1_2, Client = SSLTLS
[0;31mFAIL[m: SecurityTest
{noformat}
{noformat}
argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest"
Running 1 test case...
Entering test suite "SecurityTest"
Entering test suite "SecurityTest"
Entering test case "ssl_security_matrix"
TEST: Server = SSLTLS, Client = SSLTLS
TEST: Server = SSLTLS, Client = SSLv3
SRV 2b6bd3245700 Exception: SSL_accept: wrong version number
CLI 2b6bd2e38700 Exception: SSL_connect: sslv3 alert handshake failure
TEST: Server = SSLTLS, Client = TLSv1_0
TEST: Server = SSLTLS, Client = TLSv1_1
TEST: Server = SSLTLS, Client = TLSv1_2
TEST: Server = SSLv3, Client = SSLTLS
CLI 2b6bd2e38700 Exception: SSL_connect: unsupported protocol
SRV 2b6bd3245700 Exception: SSL_accept: error code: 0
TEST: Server = SSLv3, Client = SSLv3
TEST: Server = SSLv3, Client = TLSv1_0
CLI 2b6bd2e38700 Exception: SSL_connect: wrong version number
SRV 2b6bd3245700 Exception: SSL_accept: sslv3 alert handshake failure
TEST: Server = SSLv3, Client = TLSv1_1
CLI 2b6bd3245700 Exception: SSL_connect: wrong version number
SRV 2b6bd2e38700 Exception: SSL_accept: sslv3 alert handshake failure
TEST: Server = SSLv3, Client = TLSv1_2
SRV 2b6bd3245700 Exception: SSL_accept: sslv3 alert handshake failure
CLI 2b6bd2e38700 Exception: SSL_connect: wrong version number
TEST: Server = TLSv1_0, Client = SSLTLS
TEST: Server = TLSv1_0, Client = SSLv3
SRV 2b6bd3245700 Exception: SSL_accept: wrong version number
CLI 2b6bd2e38700 Exception: SSL_connect: ss
[jira] [Commented] (THRIFT-3608) lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build.
[
https://issues.apache.org/jira/browse/THRIFT-3608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15141086#comment-15141086
]
John Sirois commented on THRIFT-3608:
-
Since there is no assert message printed out in these failures, and the
failures happen at ~random points in the test matrix, it seems to me the
classic evil of using a fixed port is to blame here. Before understanding the
failure cases and/or asynchrony of server socket shutdown SO_LINGER, etc
at-play here, I'll try converting over to using an ephemeral port which the
test _should_ be doing anyway.
> lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build.
> -
>
> Key: THRIFT-3608
> URL: https://issues.apache.org/jira/browse/THRIFT-3608
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library, Test Suite
>Affects Versions: 0.9.3
>Reporter: John Sirois
>Assignee: John Sirois
>Priority: Critical
>
> The [Thrift-precommit|https://builds.apache.org/job/Thrift-precommit] job now
> runs in the build/docker/ubuntu container and the
> [lib/cpp/test/SecurityTest|https://github.com/apache/thrift/blob/master/lib/cpp/test/SecurityTest.cpp]
> fails most - but not all - of the time.
> Failures can be seen in the following jobs
> * https://builds.apache.org/job/Thrift-precommit/60/console
> * https://builds.apache.org/job/Thrift-precommit/59/console
> * https://builds.apache.org/job/Thrift-precommit/58/console
> * https://builds.apache.org/job/Thrift-precommit/57/console
> They look like so:
> {noformat}
> argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest"
> Running 1 test case...
> Entering test suite "SecurityTest"
> Entering test suite "SecurityTest"
> Entering test case "ssl_security_matrix"
> TEST: Server = SSLTLS, Client = SSLTLS
> TEST: Server = SSLTLS, Client = SSLv3
> SRV 2b9182317700 Exception: SSL_accept: wrong version number
> CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = SSLTLS, Client = TLSv1_0
> TEST: Server = SSLTLS, Client = TLSv1_1
> TEST: Server = SSLTLS, Client = TLSv1_2
> TEST: Server = SSLv3, Client = SSLTLS
> CLI 2b9181f0a700 Exception: SSL_connect: unsupported protocol
> SRV 2b9182317700 Exception: SSL_accept: error code: 0
> TEST: Server = SSLv3, Client = SSLv3
> TEST: Server = SSLv3, Client = TLSv1_0
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = SSLv3, Client = TLSv1_1
> CLI 2b9182317700 Exception: SSL_connect: wrong version number
> SRV 2b9181f0a700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = SSLv3, Client = TLSv1_2
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = TLSv1_0, Client = SSLTLS
> TEST: Server = TLSv1_0, Client = SSLv3
> SRV 2b9182317700 Exception: SSL_accept: wrong version number
> CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = TLSv1_0, Client = TLSv1_0
> TEST: Server = TLSv1_0, Client = TLSv1_1
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version
> TEST: Server = TLSv1_0, Client = TLSv1_2
> CLI 2b9182317700 Exception: SSL_connect: wrong version number
> SRV 2b9181f0a700 Exception: SSL_accept: tlsv1 alert protocol version
> TEST: Server = TLSv1_1, Client = SSLTLS
> TEST: Server = TLSv1_1, Client = SSLv3
> SRV 2b9181f0a700 Exception: SSL_accept: wrong version number
> CLI 2b9182317700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = TLSv1_1, Client = TLSv1_0
> SRV 2b9182317700 Exception: SSL_accept: wrong version number
> CLI 2b9181f0a700 Exception: SSL_connect: tlsv1 alert protocol version
> TEST: Server = TLSv1_1, Client = TLSv1_1
> TEST: Server = TLSv1_1, Client = TLSv1_2
> SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> TEST: Server = TLSv1_2, Client = SSLTLS
> [0;31mFAIL[m: SecurityTest
> {noformat}
> {noformat}
> argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest"
> Running 1 test case...
> Entering test suite "SecurityTest"
> Entering test suite "SecurityTest"
> Entering test case "ssl_security_matrix"
> TEST: Server = SSLTLS, Client = SSLTLS
> TEST: Server = SSLTLS, Client = SSLv3
> SRV 2b6bd3245700 Exception: SSL_accept: wrong version number
> CLI 2b6bd2e38700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = SSLTLS, Client = TLSv1_0
> TEST: Server = SSLTLS, Client = TLSv1_1
> TEST: Server = SSLTLS, Client = TLSv1_2
> TEST: Server = SSLv3, Client = SSLTLS
> CLI 2b6bd2e38700 Exception: SSL_connect: unsu
[jira] [Comment Edited] (THRIFT-3608) lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build.
[
https://issues.apache.org/jira/browse/THRIFT-3608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15141086#comment-15141086
]
John Sirois edited comment on THRIFT-3608 at 2/10/16 4:55 PM:
--
Since there is no assert message printed out in these failures, and the
failures happen at ~random points in the test matrix, it seems to me the
classic evil of using a fixed port is to blame here and its leading to port
bind failures. Before understanding the failure cases and/or asynchrony of
server socket shutdown, SO_LINGER, etc that may be at-play, I'll try converting
over to using an ephemeral port which the test _should_ be doing anyway.
was (Author: jsirois):
Since there is no assert message printed out in these failures, and the
failures happen at ~random points in the test matrix, it seems to me the
classic evil of using a fixed port is to blame here. Before understanding the
failure cases and/or asynchrony of server socket shutdown SO_LINGER, etc
at-play here, I'll try converting over to using an ephemeral port which the
test _should_ be doing anyway.
> lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build.
> -
>
> Key: THRIFT-3608
> URL: https://issues.apache.org/jira/browse/THRIFT-3608
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library, Test Suite
>Affects Versions: 0.9.3
>Reporter: John Sirois
>Assignee: John Sirois
>Priority: Critical
>
> The [Thrift-precommit|https://builds.apache.org/job/Thrift-precommit] job now
> runs in the build/docker/ubuntu container and the
> [lib/cpp/test/SecurityTest|https://github.com/apache/thrift/blob/master/lib/cpp/test/SecurityTest.cpp]
> fails most - but not all - of the time.
> Failures can be seen in the following jobs
> * https://builds.apache.org/job/Thrift-precommit/60/console
> * https://builds.apache.org/job/Thrift-precommit/59/console
> * https://builds.apache.org/job/Thrift-precommit/58/console
> * https://builds.apache.org/job/Thrift-precommit/57/console
> They look like so:
> {noformat}
> argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest"
> Running 1 test case...
> Entering test suite "SecurityTest"
> Entering test suite "SecurityTest"
> Entering test case "ssl_security_matrix"
> TEST: Server = SSLTLS, Client = SSLTLS
> TEST: Server = SSLTLS, Client = SSLv3
> SRV 2b9182317700 Exception: SSL_accept: wrong version number
> CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = SSLTLS, Client = TLSv1_0
> TEST: Server = SSLTLS, Client = TLSv1_1
> TEST: Server = SSLTLS, Client = TLSv1_2
> TEST: Server = SSLv3, Client = SSLTLS
> CLI 2b9181f0a700 Exception: SSL_connect: unsupported protocol
> SRV 2b9182317700 Exception: SSL_accept: error code: 0
> TEST: Server = SSLv3, Client = SSLv3
> TEST: Server = SSLv3, Client = TLSv1_0
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = SSLv3, Client = TLSv1_1
> CLI 2b9182317700 Exception: SSL_connect: wrong version number
> SRV 2b9181f0a700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = SSLv3, Client = TLSv1_2
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = TLSv1_0, Client = SSLTLS
> TEST: Server = TLSv1_0, Client = SSLv3
> SRV 2b9182317700 Exception: SSL_accept: wrong version number
> CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = TLSv1_0, Client = TLSv1_0
> TEST: Server = TLSv1_0, Client = TLSv1_1
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version
> TEST: Server = TLSv1_0, Client = TLSv1_2
> CLI 2b9182317700 Exception: SSL_connect: wrong version number
> SRV 2b9181f0a700 Exception: SSL_accept: tlsv1 alert protocol version
> TEST: Server = TLSv1_1, Client = SSLTLS
> TEST: Server = TLSv1_1, Client = SSLv3
> SRV 2b9181f0a700 Exception: SSL_accept: wrong version number
> CLI 2b9182317700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = TLSv1_1, Client = TLSv1_0
> SRV 2b9182317700 Exception: SSL_accept: wrong version number
> CLI 2b9181f0a700 Exception: SSL_connect: tlsv1 alert protocol version
> TEST: Server = TLSv1_1, Client = TLSv1_1
> TEST: Server = TLSv1_1, Client = TLSv1_2
> SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> TEST: Server = TLSv1_2, Client = SSLTLS
> [0;31mFAIL[m: SecurityTest
> {noformat}
> {noformat}
> argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest"
> Running 1 test case...
> Entering test su
[jira] [Created] (THRIFT-3609) Remove or replace TestPortFixture.h
John Sirois created THRIFT-3609: --- Summary: Remove or replace TestPortFixture.h Key: THRIFT-3609 URL: https://issues.apache.org/jira/browse/THRIFT-3609 Project: Thrift Issue Type: Improvement Components: C++ - Library, Test Suite Reporter: John Sirois Assignee: John Sirois Today 6 cpp lib tests use [TestPortFixture.h|https://github.com/apache/thrift/blob/1f6e380c5d07686e4cd8c2b172300a1ba7fbd8b9/lib/cpp/test/TestPortFixture.h]. THRIFT-3608 corrects one of these cases, but a general cleanup should remove all uses of static ports in the cpp tests to improve robustness and open the path to parallel testing - at least in the cpp libs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3608) lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build.
[
https://issues.apache.org/jira/browse/THRIFT-3608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15141258#comment-15141258
]
John Sirois commented on THRIFT-3608:
-
https://github.com/apache/thrift/pull/841
> lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build.
> -
>
> Key: THRIFT-3608
> URL: https://issues.apache.org/jira/browse/THRIFT-3608
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library, Test Suite
>Affects Versions: 0.9.3
>Reporter: John Sirois
>Assignee: John Sirois
>Priority: Critical
>
> The [Thrift-precommit|https://builds.apache.org/job/Thrift-precommit] job now
> runs in the build/docker/ubuntu container and the
> [lib/cpp/test/SecurityTest|https://github.com/apache/thrift/blob/master/lib/cpp/test/SecurityTest.cpp]
> fails most - but not all - of the time.
> Failures can be seen in the following jobs
> * https://builds.apache.org/job/Thrift-precommit/60/console
> * https://builds.apache.org/job/Thrift-precommit/59/console
> * https://builds.apache.org/job/Thrift-precommit/58/console
> * https://builds.apache.org/job/Thrift-precommit/57/console
> They look like so:
> {noformat}
> argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest"
> Running 1 test case...
> Entering test suite "SecurityTest"
> Entering test suite "SecurityTest"
> Entering test case "ssl_security_matrix"
> TEST: Server = SSLTLS, Client = SSLTLS
> TEST: Server = SSLTLS, Client = SSLv3
> SRV 2b9182317700 Exception: SSL_accept: wrong version number
> CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = SSLTLS, Client = TLSv1_0
> TEST: Server = SSLTLS, Client = TLSv1_1
> TEST: Server = SSLTLS, Client = TLSv1_2
> TEST: Server = SSLv3, Client = SSLTLS
> CLI 2b9181f0a700 Exception: SSL_connect: unsupported protocol
> SRV 2b9182317700 Exception: SSL_accept: error code: 0
> TEST: Server = SSLv3, Client = SSLv3
> TEST: Server = SSLv3, Client = TLSv1_0
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = SSLv3, Client = TLSv1_1
> CLI 2b9182317700 Exception: SSL_connect: wrong version number
> SRV 2b9181f0a700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = SSLv3, Client = TLSv1_2
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = TLSv1_0, Client = SSLTLS
> TEST: Server = TLSv1_0, Client = SSLv3
> SRV 2b9182317700 Exception: SSL_accept: wrong version number
> CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = TLSv1_0, Client = TLSv1_0
> TEST: Server = TLSv1_0, Client = TLSv1_1
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version
> TEST: Server = TLSv1_0, Client = TLSv1_2
> CLI 2b9182317700 Exception: SSL_connect: wrong version number
> SRV 2b9181f0a700 Exception: SSL_accept: tlsv1 alert protocol version
> TEST: Server = TLSv1_1, Client = SSLTLS
> TEST: Server = TLSv1_1, Client = SSLv3
> SRV 2b9181f0a700 Exception: SSL_accept: wrong version number
> CLI 2b9182317700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = TLSv1_1, Client = TLSv1_0
> SRV 2b9182317700 Exception: SSL_accept: wrong version number
> CLI 2b9181f0a700 Exception: SSL_connect: tlsv1 alert protocol version
> TEST: Server = TLSv1_1, Client = TLSv1_1
> TEST: Server = TLSv1_1, Client = TLSv1_2
> SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> TEST: Server = TLSv1_2, Client = SSLTLS
> [0;31mFAIL[m: SecurityTest
> {noformat}
> {noformat}
> argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest"
> Running 1 test case...
> Entering test suite "SecurityTest"
> Entering test suite "SecurityTest"
> Entering test case "ssl_security_matrix"
> TEST: Server = SSLTLS, Client = SSLTLS
> TEST: Server = SSLTLS, Client = SSLv3
> SRV 2b6bd3245700 Exception: SSL_accept: wrong version number
> CLI 2b6bd2e38700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = SSLTLS, Client = TLSv1_0
> TEST: Server = SSLTLS, Client = TLSv1_1
> TEST: Server = SSLTLS, Client = TLSv1_2
> TEST: Server = SSLv3, Client = SSLTLS
> CLI 2b6bd2e38700 Exception: SSL_connect: unsupported protocol
> SRV 2b6bd3245700 Exception: SSL_accept: error code: 0
> TEST: Server = SSLv3, Client = SSLv3
> TEST: Server = SSLv3, Client = TLSv1_0
> CLI 2b6bd2e38700 Exception: SSL_connect: wrong version number
> SRV 2b6bd3245700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = SSLv3, Client = TLSv1_1
> CLI 2b6bd3245700 Exception: SSL_con
[jira] [Commented] (THRIFT-3608) lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build.
[
https://issues.apache.org/jira/browse/THRIFT-3608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15143283#comment-15143283
]
John Sirois commented on THRIFT-3608:
-
Real fix is here: https://github.com/apache/thrift/pull/848
The issue was a bad {{#ifdef linux}} - should be {{#ifdef \_\_linux\_\_}}.
Under standards compliance modes, both g++ and clang++ will not define the
deprecated {{linux}} symbol. We compile using {{-std=c++11}} and so
{{SecutityTest}} would fail to find {{linux}} defined and thus fail to setup
ignoring {{SIGPIPE}} which led to flaky test failures.
> lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build.
> -
>
> Key: THRIFT-3608
> URL: https://issues.apache.org/jira/browse/THRIFT-3608
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library, Test Suite
>Affects Versions: 0.9.3
>Reporter: John Sirois
>Assignee: John Sirois
>Priority: Critical
>
> The [Thrift-precommit|https://builds.apache.org/job/Thrift-precommit] job now
> runs in the build/docker/ubuntu container and the
> [lib/cpp/test/SecurityTest|https://github.com/apache/thrift/blob/master/lib/cpp/test/SecurityTest.cpp]
> fails most - but not all - of the time.
> Failures can be seen in the following jobs
> * https://builds.apache.org/job/Thrift-precommit/60/console
> * https://builds.apache.org/job/Thrift-precommit/59/console
> * https://builds.apache.org/job/Thrift-precommit/58/console
> * https://builds.apache.org/job/Thrift-precommit/57/console
> They look like so:
> {noformat}
> argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest"
> Running 1 test case...
> Entering test suite "SecurityTest"
> Entering test suite "SecurityTest"
> Entering test case "ssl_security_matrix"
> TEST: Server = SSLTLS, Client = SSLTLS
> TEST: Server = SSLTLS, Client = SSLv3
> SRV 2b9182317700 Exception: SSL_accept: wrong version number
> CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = SSLTLS, Client = TLSv1_0
> TEST: Server = SSLTLS, Client = TLSv1_1
> TEST: Server = SSLTLS, Client = TLSv1_2
> TEST: Server = SSLv3, Client = SSLTLS
> CLI 2b9181f0a700 Exception: SSL_connect: unsupported protocol
> SRV 2b9182317700 Exception: SSL_accept: error code: 0
> TEST: Server = SSLv3, Client = SSLv3
> TEST: Server = SSLv3, Client = TLSv1_0
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = SSLv3, Client = TLSv1_1
> CLI 2b9182317700 Exception: SSL_connect: wrong version number
> SRV 2b9181f0a700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = SSLv3, Client = TLSv1_2
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = TLSv1_0, Client = SSLTLS
> TEST: Server = TLSv1_0, Client = SSLv3
> SRV 2b9182317700 Exception: SSL_accept: wrong version number
> CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = TLSv1_0, Client = TLSv1_0
> TEST: Server = TLSv1_0, Client = TLSv1_1
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version
> TEST: Server = TLSv1_0, Client = TLSv1_2
> CLI 2b9182317700 Exception: SSL_connect: wrong version number
> SRV 2b9181f0a700 Exception: SSL_accept: tlsv1 alert protocol version
> TEST: Server = TLSv1_1, Client = SSLTLS
> TEST: Server = TLSv1_1, Client = SSLv3
> SRV 2b9181f0a700 Exception: SSL_accept: wrong version number
> CLI 2b9182317700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = TLSv1_1, Client = TLSv1_0
> SRV 2b9182317700 Exception: SSL_accept: wrong version number
> CLI 2b9181f0a700 Exception: SSL_connect: tlsv1 alert protocol version
> TEST: Server = TLSv1_1, Client = TLSv1_1
> TEST: Server = TLSv1_1, Client = TLSv1_2
> SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> TEST: Server = TLSv1_2, Client = SSLTLS
> [0;31mFAIL[m: SecurityTest
> {noformat}
> {noformat}
> argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest"
> Running 1 test case...
> Entering test suite "SecurityTest"
> Entering test suite "SecurityTest"
> Entering test case "ssl_security_matrix"
> TEST: Server = SSLTLS, Client = SSLTLS
> TEST: Server = SSLTLS, Client = SSLv3
> SRV 2b6bd3245700 Exception: SSL_accept: wrong version number
> CLI 2b6bd2e38700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = SSLTLS, Client = TLSv1_0
> TEST: Server = SSLTLS, Client = TLSv1_1
> TEST: Server = SSLTLS, Client = TLSv1_2
> TEST: Server = SSLv3, Client = SSLTLS
> CLI 2b6bd2e38700 Exception: SSL_connect
[jira] [Commented] (THRIFT-3608) lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build.
[
https://issues.apache.org/jira/browse/THRIFT-3608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15143290#comment-15143290
]
John Sirois commented on THRIFT-3608:
-
For posterity - this trick helped seal the diagnosis:
{noformat}
$ touch foo.cc
$ g++ -dM -E foo.cc | grep linux
#define __linux 1
#define __linux__ 1
#define __gnu_linux__ 1
#define linux 1
$ g++ -std=c++11 -dM -E foo.cc | grep linux
#define __linux 1
#define __linux__ 1
#define __gnu_linux__ 1
$ clang++ -dM -E foo.cc | grep linux
#define __gnu_linux__ 1
#define __linux 1
#define __linux__ 1
#define linux 1
$ clang++ -std=c++11 -dM -E foo.cc | grep linux
#define __gnu_linux__ 1
#define __linux 1
#define __linux__ 1
{noformat}
> lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build.
> -
>
> Key: THRIFT-3608
> URL: https://issues.apache.org/jira/browse/THRIFT-3608
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library, Test Suite
>Affects Versions: 0.9.3
>Reporter: John Sirois
>Assignee: John Sirois
>Priority: Critical
>
> The [Thrift-precommit|https://builds.apache.org/job/Thrift-precommit] job now
> runs in the build/docker/ubuntu container and the
> [lib/cpp/test/SecurityTest|https://github.com/apache/thrift/blob/master/lib/cpp/test/SecurityTest.cpp]
> fails most - but not all - of the time.
> Failures can be seen in the following jobs
> * https://builds.apache.org/job/Thrift-precommit/60/console
> * https://builds.apache.org/job/Thrift-precommit/59/console
> * https://builds.apache.org/job/Thrift-precommit/58/console
> * https://builds.apache.org/job/Thrift-precommit/57/console
> They look like so:
> {noformat}
> argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest"
> Running 1 test case...
> Entering test suite "SecurityTest"
> Entering test suite "SecurityTest"
> Entering test case "ssl_security_matrix"
> TEST: Server = SSLTLS, Client = SSLTLS
> TEST: Server = SSLTLS, Client = SSLv3
> SRV 2b9182317700 Exception: SSL_accept: wrong version number
> CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = SSLTLS, Client = TLSv1_0
> TEST: Server = SSLTLS, Client = TLSv1_1
> TEST: Server = SSLTLS, Client = TLSv1_2
> TEST: Server = SSLv3, Client = SSLTLS
> CLI 2b9181f0a700 Exception: SSL_connect: unsupported protocol
> SRV 2b9182317700 Exception: SSL_accept: error code: 0
> TEST: Server = SSLv3, Client = SSLv3
> TEST: Server = SSLv3, Client = TLSv1_0
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = SSLv3, Client = TLSv1_1
> CLI 2b9182317700 Exception: SSL_connect: wrong version number
> SRV 2b9181f0a700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = SSLv3, Client = TLSv1_2
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = TLSv1_0, Client = SSLTLS
> TEST: Server = TLSv1_0, Client = SSLv3
> SRV 2b9182317700 Exception: SSL_accept: wrong version number
> CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = TLSv1_0, Client = TLSv1_0
> TEST: Server = TLSv1_0, Client = TLSv1_1
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version
> TEST: Server = TLSv1_0, Client = TLSv1_2
> CLI 2b9182317700 Exception: SSL_connect: wrong version number
> SRV 2b9181f0a700 Exception: SSL_accept: tlsv1 alert protocol version
> TEST: Server = TLSv1_1, Client = SSLTLS
> TEST: Server = TLSv1_1, Client = SSLv3
> SRV 2b9181f0a700 Exception: SSL_accept: wrong version number
> CLI 2b9182317700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = TLSv1_1, Client = TLSv1_0
> SRV 2b9182317700 Exception: SSL_accept: wrong version number
> CLI 2b9181f0a700 Exception: SSL_connect: tlsv1 alert protocol version
> TEST: Server = TLSv1_1, Client = TLSv1_1
> TEST: Server = TLSv1_1, Client = TLSv1_2
> SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> TEST: Server = TLSv1_2, Client = SSLTLS
> [0;31mFAIL[m: SecurityTest
> {noformat}
> {noformat}
> argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest"
> Running 1 test case...
> Entering test suite "SecurityTest"
> Entering test suite "SecurityTest"
> Entering test case "ssl_security_matrix"
> TEST: Server = SSLTLS, Client = SSLTLS
> TEST: Server = SSLTLS, Client = SSLv3
> SRV 2b6bd3245700 Exception: SSL_accept: wrong version number
> CLI 2b6bd2e38700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = SSLTLS, Client = TLSv1_0
> TEST: Server = SSLTLS, Client = TLSv1_1
> TES
[jira] [Issue Comment Deleted] (THRIFT-3608) lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build.
[
https://issues.apache.org/jira/browse/THRIFT-3608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Sirois updated THRIFT-3608:
Comment: was deleted
(was: https://github.com/apache/thrift/pull/841)
> lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build.
> -
>
> Key: THRIFT-3608
> URL: https://issues.apache.org/jira/browse/THRIFT-3608
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library, Test Suite
>Affects Versions: 0.9.3
>Reporter: John Sirois
>Assignee: John Sirois
>Priority: Critical
>
> The [Thrift-precommit|https://builds.apache.org/job/Thrift-precommit] job now
> runs in the build/docker/ubuntu container and the
> [lib/cpp/test/SecurityTest|https://github.com/apache/thrift/blob/master/lib/cpp/test/SecurityTest.cpp]
> fails most - but not all - of the time.
> Failures can be seen in the following jobs
> * https://builds.apache.org/job/Thrift-precommit/60/console
> * https://builds.apache.org/job/Thrift-precommit/59/console
> * https://builds.apache.org/job/Thrift-precommit/58/console
> * https://builds.apache.org/job/Thrift-precommit/57/console
> They look like so:
> {noformat}
> argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest"
> Running 1 test case...
> Entering test suite "SecurityTest"
> Entering test suite "SecurityTest"
> Entering test case "ssl_security_matrix"
> TEST: Server = SSLTLS, Client = SSLTLS
> TEST: Server = SSLTLS, Client = SSLv3
> SRV 2b9182317700 Exception: SSL_accept: wrong version number
> CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = SSLTLS, Client = TLSv1_0
> TEST: Server = SSLTLS, Client = TLSv1_1
> TEST: Server = SSLTLS, Client = TLSv1_2
> TEST: Server = SSLv3, Client = SSLTLS
> CLI 2b9181f0a700 Exception: SSL_connect: unsupported protocol
> SRV 2b9182317700 Exception: SSL_accept: error code: 0
> TEST: Server = SSLv3, Client = SSLv3
> TEST: Server = SSLv3, Client = TLSv1_0
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = SSLv3, Client = TLSv1_1
> CLI 2b9182317700 Exception: SSL_connect: wrong version number
> SRV 2b9181f0a700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = SSLv3, Client = TLSv1_2
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = TLSv1_0, Client = SSLTLS
> TEST: Server = TLSv1_0, Client = SSLv3
> SRV 2b9182317700 Exception: SSL_accept: wrong version number
> CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = TLSv1_0, Client = TLSv1_0
> TEST: Server = TLSv1_0, Client = TLSv1_1
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version
> TEST: Server = TLSv1_0, Client = TLSv1_2
> CLI 2b9182317700 Exception: SSL_connect: wrong version number
> SRV 2b9181f0a700 Exception: SSL_accept: tlsv1 alert protocol version
> TEST: Server = TLSv1_1, Client = SSLTLS
> TEST: Server = TLSv1_1, Client = SSLv3
> SRV 2b9181f0a700 Exception: SSL_accept: wrong version number
> CLI 2b9182317700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = TLSv1_1, Client = TLSv1_0
> SRV 2b9182317700 Exception: SSL_accept: wrong version number
> CLI 2b9181f0a700 Exception: SSL_connect: tlsv1 alert protocol version
> TEST: Server = TLSv1_1, Client = TLSv1_1
> TEST: Server = TLSv1_1, Client = TLSv1_2
> SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> TEST: Server = TLSv1_2, Client = SSLTLS
> [0;31mFAIL[m: SecurityTest
> {noformat}
> {noformat}
> argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest"
> Running 1 test case...
> Entering test suite "SecurityTest"
> Entering test suite "SecurityTest"
> Entering test case "ssl_security_matrix"
> TEST: Server = SSLTLS, Client = SSLTLS
> TEST: Server = SSLTLS, Client = SSLv3
> SRV 2b6bd3245700 Exception: SSL_accept: wrong version number
> CLI 2b6bd2e38700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = SSLTLS, Client = TLSv1_0
> TEST: Server = SSLTLS, Client = TLSv1_1
> TEST: Server = SSLTLS, Client = TLSv1_2
> TEST: Server = SSLv3, Client = SSLTLS
> CLI 2b6bd2e38700 Exception: SSL_connect: unsupported protocol
> SRV 2b6bd3245700 Exception: SSL_accept: error code: 0
> TEST: Server = SSLv3, Client = SSLv3
> TEST: Server = SSLv3, Client = TLSv1_0
> CLI 2b6bd2e38700 Exception: SSL_connect: wrong version number
> SRV 2b6bd3245700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = SSLv3, Client = TLSv1_1
> CLI 2b6bd3245700 Exception: SSL_connect: wrong version numbe
[jira] [Commented] (THRIFT-3608) lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build.
[
https://issues.apache.org/jira/browse/THRIFT-3608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15143369#comment-15143369
]
John Sirois commented on THRIFT-3608:
-
Thanks [~jensg] - I'll comment over there and close that bug as duplicate to
this since I have a solution spinning in jenkins CI now.
> lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build.
> -
>
> Key: THRIFT-3608
> URL: https://issues.apache.org/jira/browse/THRIFT-3608
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library, Test Suite
>Affects Versions: 0.9.3
>Reporter: John Sirois
>Assignee: John Sirois
>Priority: Critical
>
> The [Thrift-precommit|https://builds.apache.org/job/Thrift-precommit] job now
> runs in the build/docker/ubuntu container and the
> [lib/cpp/test/SecurityTest|https://github.com/apache/thrift/blob/master/lib/cpp/test/SecurityTest.cpp]
> fails most - but not all - of the time.
> Failures can be seen in the following jobs
> * https://builds.apache.org/job/Thrift-precommit/60/console
> * https://builds.apache.org/job/Thrift-precommit/59/console
> * https://builds.apache.org/job/Thrift-precommit/58/console
> * https://builds.apache.org/job/Thrift-precommit/57/console
> They look like so:
> {noformat}
> argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest"
> Running 1 test case...
> Entering test suite "SecurityTest"
> Entering test suite "SecurityTest"
> Entering test case "ssl_security_matrix"
> TEST: Server = SSLTLS, Client = SSLTLS
> TEST: Server = SSLTLS, Client = SSLv3
> SRV 2b9182317700 Exception: SSL_accept: wrong version number
> CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = SSLTLS, Client = TLSv1_0
> TEST: Server = SSLTLS, Client = TLSv1_1
> TEST: Server = SSLTLS, Client = TLSv1_2
> TEST: Server = SSLv3, Client = SSLTLS
> CLI 2b9181f0a700 Exception: SSL_connect: unsupported protocol
> SRV 2b9182317700 Exception: SSL_accept: error code: 0
> TEST: Server = SSLv3, Client = SSLv3
> TEST: Server = SSLv3, Client = TLSv1_0
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = SSLv3, Client = TLSv1_1
> CLI 2b9182317700 Exception: SSL_connect: wrong version number
> SRV 2b9181f0a700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = SSLv3, Client = TLSv1_2
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = TLSv1_0, Client = SSLTLS
> TEST: Server = TLSv1_0, Client = SSLv3
> SRV 2b9182317700 Exception: SSL_accept: wrong version number
> CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = TLSv1_0, Client = TLSv1_0
> TEST: Server = TLSv1_0, Client = TLSv1_1
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version
> TEST: Server = TLSv1_0, Client = TLSv1_2
> CLI 2b9182317700 Exception: SSL_connect: wrong version number
> SRV 2b9181f0a700 Exception: SSL_accept: tlsv1 alert protocol version
> TEST: Server = TLSv1_1, Client = SSLTLS
> TEST: Server = TLSv1_1, Client = SSLv3
> SRV 2b9181f0a700 Exception: SSL_accept: wrong version number
> CLI 2b9182317700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = TLSv1_1, Client = TLSv1_0
> SRV 2b9182317700 Exception: SSL_accept: wrong version number
> CLI 2b9181f0a700 Exception: SSL_connect: tlsv1 alert protocol version
> TEST: Server = TLSv1_1, Client = TLSv1_1
> TEST: Server = TLSv1_1, Client = TLSv1_2
> SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> TEST: Server = TLSv1_2, Client = SSLTLS
> [0;31mFAIL[m: SecurityTest
> {noformat}
> {noformat}
> argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest"
> Running 1 test case...
> Entering test suite "SecurityTest"
> Entering test suite "SecurityTest"
> Entering test case "ssl_security_matrix"
> TEST: Server = SSLTLS, Client = SSLTLS
> TEST: Server = SSLTLS, Client = SSLv3
> SRV 2b6bd3245700 Exception: SSL_accept: wrong version number
> CLI 2b6bd2e38700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = SSLTLS, Client = TLSv1_0
> TEST: Server = SSLTLS, Client = TLSv1_1
> TEST: Server = SSLTLS, Client = TLSv1_2
> TEST: Server = SSLv3, Client = SSLTLS
> CLI 2b6bd2e38700 Exception: SSL_connect: unsupported protocol
> SRV 2b6bd3245700 Exception: SSL_accept: error code: 0
> TEST: Server = SSLv3, Client = SSLv3
> TEST: Server = SSLv3, Client = TLSv1_0
> CLI 2b6bd2e38700 Exception: SSL_connect: wrong version number
> SRV 2b6bd3245700 Exception: SSL_accept: sslv3 alert han
[jira] [Commented] (THRIFT-3362) make check fails for C++ at the SecurityTest
[
https://issues.apache.org/jira/browse/THRIFT-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15143375#comment-15143375
]
John Sirois commented on THRIFT-3362:
-
Noting that I figured this out over in THRIFT-3608. The issue was {{#ifdef
linux}} and {{SIGPIPE}} signal handlers not getting installed.
I did 1st go down the path of using ephemeral ports, and will address that in
THRIFT-3609 since it turned out not to be the issue, but would be good to clean
up (globally) anyhow.
> make check fails for C++ at the SecurityTest
>
>
> Key: THRIFT-3362
> URL: https://issues.apache.org/jira/browse/THRIFT-3362
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library, Test Suite
> Environment: openSUSE
>Reporter: Jens Geyer
>Assignee: James E. King, III
>
> When I run "make check" inside {{lib/cpp}} I get this error:
> {code}
> FAIL: SecurityTest
> {code}
> No more information besides that, and all other tests succeed. When I run
> the SecurityTest test alone, it succeeds - it fails only with make check. I
> have no idea why, and I can't get the test routine to print more log info to
> the console with make check. I already tried adding --log_level
> and --report_level to no avail, again this works well when run standalone.
> I'm a bit stuck here. Maybe the problem lies in my setup, but I have no idea
> what to look for. Any helpful advice would be really great.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (THRIFT-3362) make check fails for C++ at the SecurityTest
[
https://issues.apache.org/jira/browse/THRIFT-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Sirois resolved THRIFT-3362.
-
Resolution: Duplicate
Resolving as duplicate of THRIFT-3608 which has a fix spinning through CI here:
https://github.com/apache/thrift/pull/848
> make check fails for C++ at the SecurityTest
>
>
> Key: THRIFT-3362
> URL: https://issues.apache.org/jira/browse/THRIFT-3362
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library, Test Suite
> Environment: openSUSE
>Reporter: Jens Geyer
>Assignee: James E. King, III
>
> When I run "make check" inside {{lib/cpp}} I get this error:
> {code}
> FAIL: SecurityTest
> {code}
> No more information besides that, and all other tests succeed. When I run
> the SecurityTest test alone, it succeeds - it fails only with make check. I
> have no idea why, and I can't get the test routine to print more log info to
> the console with make check. I already tried adding --log_level
> and --report_level to no avail, again this works well when run standalone.
> I'm a bit stuck here. Maybe the problem lies in my setup, but I have no idea
> what to look for. Any helpful advice would be really great.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (THRIFT-3608) lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build.
[
https://issues.apache.org/jira/browse/THRIFT-3608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15143423#comment-15143423
]
John Sirois commented on THRIFT-3608:
-
Passing now though failing at another ~commmon flaky in python:
https://builds.apache.org/job/Thrift-precommit/86/console
> lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build.
> -
>
> Key: THRIFT-3608
> URL: https://issues.apache.org/jira/browse/THRIFT-3608
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library, Test Suite
>Affects Versions: 0.9.3
>Reporter: John Sirois
>Assignee: John Sirois
>Priority: Critical
>
> The [Thrift-precommit|https://builds.apache.org/job/Thrift-precommit] job now
> runs in the build/docker/ubuntu container and the
> [lib/cpp/test/SecurityTest|https://github.com/apache/thrift/blob/master/lib/cpp/test/SecurityTest.cpp]
> fails most - but not all - of the time.
> Failures can be seen in the following jobs
> * https://builds.apache.org/job/Thrift-precommit/60/console
> * https://builds.apache.org/job/Thrift-precommit/59/console
> * https://builds.apache.org/job/Thrift-precommit/58/console
> * https://builds.apache.org/job/Thrift-precommit/57/console
> They look like so:
> {noformat}
> argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest"
> Running 1 test case...
> Entering test suite "SecurityTest"
> Entering test suite "SecurityTest"
> Entering test case "ssl_security_matrix"
> TEST: Server = SSLTLS, Client = SSLTLS
> TEST: Server = SSLTLS, Client = SSLv3
> SRV 2b9182317700 Exception: SSL_accept: wrong version number
> CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = SSLTLS, Client = TLSv1_0
> TEST: Server = SSLTLS, Client = TLSv1_1
> TEST: Server = SSLTLS, Client = TLSv1_2
> TEST: Server = SSLv3, Client = SSLTLS
> CLI 2b9181f0a700 Exception: SSL_connect: unsupported protocol
> SRV 2b9182317700 Exception: SSL_accept: error code: 0
> TEST: Server = SSLv3, Client = SSLv3
> TEST: Server = SSLv3, Client = TLSv1_0
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = SSLv3, Client = TLSv1_1
> CLI 2b9182317700 Exception: SSL_connect: wrong version number
> SRV 2b9181f0a700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = SSLv3, Client = TLSv1_2
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = TLSv1_0, Client = SSLTLS
> TEST: Server = TLSv1_0, Client = SSLv3
> SRV 2b9182317700 Exception: SSL_accept: wrong version number
> CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = TLSv1_0, Client = TLSv1_0
> TEST: Server = TLSv1_0, Client = TLSv1_1
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version
> TEST: Server = TLSv1_0, Client = TLSv1_2
> CLI 2b9182317700 Exception: SSL_connect: wrong version number
> SRV 2b9181f0a700 Exception: SSL_accept: tlsv1 alert protocol version
> TEST: Server = TLSv1_1, Client = SSLTLS
> TEST: Server = TLSv1_1, Client = SSLv3
> SRV 2b9181f0a700 Exception: SSL_accept: wrong version number
> CLI 2b9182317700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = TLSv1_1, Client = TLSv1_0
> SRV 2b9182317700 Exception: SSL_accept: wrong version number
> CLI 2b9181f0a700 Exception: SSL_connect: tlsv1 alert protocol version
> TEST: Server = TLSv1_1, Client = TLSv1_1
> TEST: Server = TLSv1_1, Client = TLSv1_2
> SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> TEST: Server = TLSv1_2, Client = SSLTLS
> [0;31mFAIL[m: SecurityTest
> {noformat}
> {noformat}
> argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest"
> Running 1 test case...
> Entering test suite "SecurityTest"
> Entering test suite "SecurityTest"
> Entering test case "ssl_security_matrix"
> TEST: Server = SSLTLS, Client = SSLTLS
> TEST: Server = SSLTLS, Client = SSLv3
> SRV 2b6bd3245700 Exception: SSL_accept: wrong version number
> CLI 2b6bd2e38700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = SSLTLS, Client = TLSv1_0
> TEST: Server = SSLTLS, Client = TLSv1_1
> TEST: Server = SSLTLS, Client = TLSv1_2
> TEST: Server = SSLv3, Client = SSLTLS
> CLI 2b6bd2e38700 Exception: SSL_connect: unsupported protocol
> SRV 2b6bd3245700 Exception: SSL_accept: error code: 0
> TEST: Server = SSLv3, Client = SSLv3
> TEST: Server = SSLv3, Client = TLSv1_0
> CLI 2b6bd2e38700 Exception: SSL_connect: wrong version number
> SRV 2b6bd3245700 Exception: SSL_accept: sslv3 alert handshake failu
[jira] [Commented] (THRIFT-3608) lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build.
[
https://issues.apache.org/jira/browse/THRIFT-3608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15143424#comment-15143424
]
John Sirois commented on THRIFT-3608:
-
[~jensg] - If you can confirm this fixes the issue you had as described in
THRIFT-3362 I think its ready to merge.
> lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build.
> -
>
> Key: THRIFT-3608
> URL: https://issues.apache.org/jira/browse/THRIFT-3608
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library, Test Suite
>Affects Versions: 0.9.3
>Reporter: John Sirois
>Assignee: John Sirois
>Priority: Critical
>
> The [Thrift-precommit|https://builds.apache.org/job/Thrift-precommit] job now
> runs in the build/docker/ubuntu container and the
> [lib/cpp/test/SecurityTest|https://github.com/apache/thrift/blob/master/lib/cpp/test/SecurityTest.cpp]
> fails most - but not all - of the time.
> Failures can be seen in the following jobs
> * https://builds.apache.org/job/Thrift-precommit/60/console
> * https://builds.apache.org/job/Thrift-precommit/59/console
> * https://builds.apache.org/job/Thrift-precommit/58/console
> * https://builds.apache.org/job/Thrift-precommit/57/console
> They look like so:
> {noformat}
> argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest"
> Running 1 test case...
> Entering test suite "SecurityTest"
> Entering test suite "SecurityTest"
> Entering test case "ssl_security_matrix"
> TEST: Server = SSLTLS, Client = SSLTLS
> TEST: Server = SSLTLS, Client = SSLv3
> SRV 2b9182317700 Exception: SSL_accept: wrong version number
> CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = SSLTLS, Client = TLSv1_0
> TEST: Server = SSLTLS, Client = TLSv1_1
> TEST: Server = SSLTLS, Client = TLSv1_2
> TEST: Server = SSLv3, Client = SSLTLS
> CLI 2b9181f0a700 Exception: SSL_connect: unsupported protocol
> SRV 2b9182317700 Exception: SSL_accept: error code: 0
> TEST: Server = SSLv3, Client = SSLv3
> TEST: Server = SSLv3, Client = TLSv1_0
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = SSLv3, Client = TLSv1_1
> CLI 2b9182317700 Exception: SSL_connect: wrong version number
> SRV 2b9181f0a700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = SSLv3, Client = TLSv1_2
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = TLSv1_0, Client = SSLTLS
> TEST: Server = TLSv1_0, Client = SSLv3
> SRV 2b9182317700 Exception: SSL_accept: wrong version number
> CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = TLSv1_0, Client = TLSv1_0
> TEST: Server = TLSv1_0, Client = TLSv1_1
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version
> TEST: Server = TLSv1_0, Client = TLSv1_2
> CLI 2b9182317700 Exception: SSL_connect: wrong version number
> SRV 2b9181f0a700 Exception: SSL_accept: tlsv1 alert protocol version
> TEST: Server = TLSv1_1, Client = SSLTLS
> TEST: Server = TLSv1_1, Client = SSLv3
> SRV 2b9181f0a700 Exception: SSL_accept: wrong version number
> CLI 2b9182317700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = TLSv1_1, Client = TLSv1_0
> SRV 2b9182317700 Exception: SSL_accept: wrong version number
> CLI 2b9181f0a700 Exception: SSL_connect: tlsv1 alert protocol version
> TEST: Server = TLSv1_1, Client = TLSv1_1
> TEST: Server = TLSv1_1, Client = TLSv1_2
> SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> TEST: Server = TLSv1_2, Client = SSLTLS
> [0;31mFAIL[m: SecurityTest
> {noformat}
> {noformat}
> argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest"
> Running 1 test case...
> Entering test suite "SecurityTest"
> Entering test suite "SecurityTest"
> Entering test case "ssl_security_matrix"
> TEST: Server = SSLTLS, Client = SSLTLS
> TEST: Server = SSLTLS, Client = SSLv3
> SRV 2b6bd3245700 Exception: SSL_accept: wrong version number
> CLI 2b6bd2e38700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = SSLTLS, Client = TLSv1_0
> TEST: Server = SSLTLS, Client = TLSv1_1
> TEST: Server = SSLTLS, Client = TLSv1_2
> TEST: Server = SSLv3, Client = SSLTLS
> CLI 2b6bd2e38700 Exception: SSL_connect: unsupported protocol
> SRV 2b6bd3245700 Exception: SSL_accept: error code: 0
> TEST: Server = SSLv3, Client = SSLv3
> TEST: Server = SSLv3, Client = TLSv1_0
> CLI 2b6bd2e38700 Exception: SSL_connect: wrong version number
> SRV 2b6bd3245700 Exception: SSL_accept: sslv3 alert handshake failure
> TE
[jira] [Commented] (THRIFT-3362) make check fails for C++ at the SecurityTest
[
https://issues.apache.org/jira/browse/THRIFT-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15143434#comment-15143434
]
John Sirois commented on THRIFT-3362:
-
Yeah - realized that late, but I spent 2 days figuring it out so I reserve the
right to reverse the flow of time ;)
> make check fails for C++ at the SecurityTest
>
>
> Key: THRIFT-3362
> URL: https://issues.apache.org/jira/browse/THRIFT-3362
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library, Test Suite
> Environment: openSUSE
>Reporter: Jens Geyer
>Assignee: James E. King, III
>
> When I run "make check" inside {{lib/cpp}} I get this error:
> {code}
> FAIL: SecurityTest
> {code}
> No more information besides that, and all other tests succeed. When I run
> the SecurityTest test alone, it succeeds - it fails only with make check. I
> have no idea why, and I can't get the test routine to print more log info to
> the console with make check. I already tried adding --log_level
> and --report_level to no avail, again this works well when run standalone.
> I'm a bit stuck here. Maybe the problem lies in my setup, but I have no idea
> what to look for. Any helpful advice would be really great.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (THRIFT-3616) lib/py/test/test_sslsocket.py is flaky
John Sirois created THRIFT-3616:
---
Summary: lib/py/test/test_sslsocket.py is flaky
Key: THRIFT-3616
URL: https://issues.apache.org/jira/browse/THRIFT-3616
Project: Thrift
Issue Type: Bug
Components: Python - Library, Test Suite
Reporter: John Sirois
Assignee: John Sirois
Priority: Critical
As seen here: https://builds.apache.org/job/Thrift-precommit/86/console
{noformat}
...
/usr/bin/python3 test/thrift_json.py
.
--
Ran 1 test in 0.011s
OK
/usr/bin/python3 test/test_sslsocket.py
.FFFsys:1: ResourceWarning: unclosed
F/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:253:
DeprecationWarning: validate is deprecated. Use cert_reqs=ssl.CERT_REQUIRED
instead
DeprecationWarning)
/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:253:
DeprecationWarning: validate is deprecated. Use cert_reqs=ssl.CERT_NONE instead
DeprecationWarning)
.sys:1: ResourceWarning: unclosed
F/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:265:
DeprecationWarning: Use cert_reqs instead
warnings.warn('Use cert_reqs instead', DeprecationWarning)
.sys:1: ResourceWarning: unclosed
Exception in thread Thread-11:
Traceback (most recent call last):
File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner
self.run()
File "test/test_sslsocket.py", line 60, in run
self.client = self._server.accept()
File "/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py", line 418, in
accept
client = self._wrap_socket(plain_client)
File "/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py", line 181, in
_wrap_socket
server_hostname=self._server_hostname)
File "/usr/lib/python3.4/ssl.py", line 365, in wrap_socket
_context=self)
File "/usr/lib/python3.4/ssl.py", line 601, in __init__
self.do_handshake()
File "/usr/lib/python3.4/ssl.py", line 828, in do_handshake
self._sslobj.do_handshake()
ConnectionResetError: [Errno 104] Connection reset by peer
PROTOCOL_SSLv2 is not available
..test/test_sslsocket.py:59: ResourceWarning: unclosed
self._server.listen()
.
==
FAIL: test_client_cert (__main__.TSSLSocketTest)
--
Traceback (most recent call last):
File "test/test_sslsocket.py", line 210, in test_client_cert
self._assert_connection_success(server, client)
File "test/test_sslsocket.py", line 112, in _assert_connection_success
self.assertTrue(acc.client is not None)
AssertionError: False is not true
==
FAIL: test_deprecation (__main__.TSSLSocketTest)
--
Traceback (most recent call last):
File "test/test_sslsocket.py", line 138, in test_deprecation
self.assertEqual(len(w), 3)
AssertionError: 4 != 3
==
FAIL: test_newer_tls (__main__.TSSLSocketTest)
--
Traceback (most recent call last):
File "test/test_sslsocket.py", line 268, in test_newer_tls
self._assert_connection_success(server, client)
File "test/test_sslsocket.py", line 112, in _assert_connection_success
self.assertTrue(acc.client is not None)
AssertionError: False is not true
==
FAIL: test_server_cert (__main__.TSSLSocketTest)
--
Traceback (most recent call last):
File "test/test_sslsocket.py", line 172, in test_server_cert
self._assert_connection_success(server, client)
File "test/test_sslsocket.py", line 112, in _assert_connection_success
self.assertTrue(acc.client is not None)
AssertionError: False is not true
==
FAIL: test_set_server_cert (__main__.TSSLSocketTest)
--
Traceback (most recent call last):
File "test/test_sslsocket.py", line 191, in test_set_server_cert
self._assert_connection_success(server, client)
File "test/test_sslsocket.py", line 112, in _assert_connection_success
self.assertTrue(acc.client is not None)
AssertionError: False is not true
--
Ran 11 tests in 5.240s
FAILED (failures=5)
make[3]: *** [py3-test] Error 1
make[3]: Leaving directory `/thrift/lib/py'
make[2]: *** [check-am] Error 2
make[2]: Leaving directory `/thrift/lib/py'
make[1]: *** [check-recursive] Error 1
make[1]: Leaving directory `/thrift/lib'
make: *** [check-recursive] Error 1
Build step 'Execute shell' marked build a
[jira] [Commented] (THRIFT-3608) lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build.
[
https://issues.apache.org/jira/browse/THRIFT-3608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15143459#comment-15143459
]
John Sirois commented on THRIFT-3608:
-
I filed this for the flaky python issue: THRIFT-3616
> lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build.
> -
>
> Key: THRIFT-3608
> URL: https://issues.apache.org/jira/browse/THRIFT-3608
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library, Test Suite
>Affects Versions: 0.9.3
>Reporter: John Sirois
>Assignee: John Sirois
>Priority: Critical
>
> The [Thrift-precommit|https://builds.apache.org/job/Thrift-precommit] job now
> runs in the build/docker/ubuntu container and the
> [lib/cpp/test/SecurityTest|https://github.com/apache/thrift/blob/master/lib/cpp/test/SecurityTest.cpp]
> fails most - but not all - of the time.
> Failures can be seen in the following jobs
> * https://builds.apache.org/job/Thrift-precommit/60/console
> * https://builds.apache.org/job/Thrift-precommit/59/console
> * https://builds.apache.org/job/Thrift-precommit/58/console
> * https://builds.apache.org/job/Thrift-precommit/57/console
> They look like so:
> {noformat}
> argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest"
> Running 1 test case...
> Entering test suite "SecurityTest"
> Entering test suite "SecurityTest"
> Entering test case "ssl_security_matrix"
> TEST: Server = SSLTLS, Client = SSLTLS
> TEST: Server = SSLTLS, Client = SSLv3
> SRV 2b9182317700 Exception: SSL_accept: wrong version number
> CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = SSLTLS, Client = TLSv1_0
> TEST: Server = SSLTLS, Client = TLSv1_1
> TEST: Server = SSLTLS, Client = TLSv1_2
> TEST: Server = SSLv3, Client = SSLTLS
> CLI 2b9181f0a700 Exception: SSL_connect: unsupported protocol
> SRV 2b9182317700 Exception: SSL_accept: error code: 0
> TEST: Server = SSLv3, Client = SSLv3
> TEST: Server = SSLv3, Client = TLSv1_0
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = SSLv3, Client = TLSv1_1
> CLI 2b9182317700 Exception: SSL_connect: wrong version number
> SRV 2b9181f0a700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = SSLv3, Client = TLSv1_2
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = TLSv1_0, Client = SSLTLS
> TEST: Server = TLSv1_0, Client = SSLv3
> SRV 2b9182317700 Exception: SSL_accept: wrong version number
> CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = TLSv1_0, Client = TLSv1_0
> TEST: Server = TLSv1_0, Client = TLSv1_1
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version
> TEST: Server = TLSv1_0, Client = TLSv1_2
> CLI 2b9182317700 Exception: SSL_connect: wrong version number
> SRV 2b9181f0a700 Exception: SSL_accept: tlsv1 alert protocol version
> TEST: Server = TLSv1_1, Client = SSLTLS
> TEST: Server = TLSv1_1, Client = SSLv3
> SRV 2b9181f0a700 Exception: SSL_accept: wrong version number
> CLI 2b9182317700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = TLSv1_1, Client = TLSv1_0
> SRV 2b9182317700 Exception: SSL_accept: wrong version number
> CLI 2b9181f0a700 Exception: SSL_connect: tlsv1 alert protocol version
> TEST: Server = TLSv1_1, Client = TLSv1_1
> TEST: Server = TLSv1_1, Client = TLSv1_2
> SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> TEST: Server = TLSv1_2, Client = SSLTLS
> [0;31mFAIL[m: SecurityTest
> {noformat}
> {noformat}
> argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest"
> Running 1 test case...
> Entering test suite "SecurityTest"
> Entering test suite "SecurityTest"
> Entering test case "ssl_security_matrix"
> TEST: Server = SSLTLS, Client = SSLTLS
> TEST: Server = SSLTLS, Client = SSLv3
> SRV 2b6bd3245700 Exception: SSL_accept: wrong version number
> CLI 2b6bd2e38700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = SSLTLS, Client = TLSv1_0
> TEST: Server = SSLTLS, Client = TLSv1_1
> TEST: Server = SSLTLS, Client = TLSv1_2
> TEST: Server = SSLv3, Client = SSLTLS
> CLI 2b6bd2e38700 Exception: SSL_connect: unsupported protocol
> SRV 2b6bd3245700 Exception: SSL_accept: error code: 0
> TEST: Server = SSLv3, Client = SSLv3
> TEST: Server = SSLv3, Client = TLSv1_0
> CLI 2b6bd2e38700 Exception: SSL_connect: wrong version number
> SRV 2b6bd3245700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = SSLv3, Client = TLSv1_1
> CLI 2b6bd3245700 Excepti
[jira] [Commented] (THRIFT-3616) lib/py/test/test_sslsocket.py is flaky
[
https://issues.apache.org/jira/browse/THRIFT-3616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15143462#comment-15143462
]
John Sirois commented on THRIFT-3616:
-
Seen multiple times over the course of working THRIFT-3608
> lib/py/test/test_sslsocket.py is flaky
> --
>
> Key: THRIFT-3616
> URL: https://issues.apache.org/jira/browse/THRIFT-3616
> Project: Thrift
> Issue Type: Bug
> Components: Python - Library, Test Suite
>Reporter: John Sirois
>Assignee: John Sirois
>Priority: Critical
>
> As seen here: https://builds.apache.org/job/Thrift-precommit/86/console
> {noformat}
> ...
> /usr/bin/python3 test/thrift_json.py
> .
> --
> Ran 1 test in 0.011s
> OK
> /usr/bin/python3 test/test_sslsocket.py
> .FFFsys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0,
> laddr=('::1', 23458, 0, 0), raddr=('::1', 41151, 0, 0)>
> F/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:253:
> DeprecationWarning: validate is deprecated. Use cert_reqs=ssl.CERT_REQUIRED
> instead
> DeprecationWarning)
> /thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:253:
> DeprecationWarning: validate is deprecated. Use cert_reqs=ssl.CERT_NONE
> instead
> DeprecationWarning)
> .sys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0,
> laddr=('::1', 23458, 0, 0), raddr=('::1', 41155, 0, 0)>
> F/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:265:
> DeprecationWarning: Use cert_reqs instead
> warnings.warn('Use cert_reqs instead', DeprecationWarning)
> .sys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0,
> laddr=('::1', 23458, 0, 0), raddr=('::1', 41157, 0, 0)>
> Exception in thread Thread-11:
> Traceback (most recent call last):
> File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner
> self.run()
> File "test/test_sslsocket.py", line 60, in run
> self.client = self._server.accept()
> File "/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py", line 418,
> in accept
> client = self._wrap_socket(plain_client)
> File "/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py", line 181,
> in _wrap_socket
> server_hostname=self._server_hostname)
> File "/usr/lib/python3.4/ssl.py", line 365, in wrap_socket
> _context=self)
> File "/usr/lib/python3.4/ssl.py", line 601, in __init__
> self.do_handshake()
> File "/usr/lib/python3.4/ssl.py", line 828, in do_handshake
> self._sslobj.do_handshake()
> ConnectionResetError: [Errno 104] Connection reset by peer
> PROTOCOL_SSLv2 is not available
> ..test/test_sslsocket.py:59: ResourceWarning: unclosed family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0>
> self._server.listen()
> .
> ==
> FAIL: test_client_cert (__main__.TSSLSocketTest)
> --
> Traceback (most recent call last):
> File "test/test_sslsocket.py", line 210, in test_client_cert
> self._assert_connection_success(server, client)
> File "test/test_sslsocket.py", line 112, in _assert_connection_success
> self.assertTrue(acc.client is not None)
> AssertionError: False is not true
> ==
> FAIL: test_deprecation (__main__.TSSLSocketTest)
> --
> Traceback (most recent call last):
> File "test/test_sslsocket.py", line 138, in test_deprecation
> self.assertEqual(len(w), 3)
> AssertionError: 4 != 3
> ==
> FAIL: test_newer_tls (__main__.TSSLSocketTest)
> --
> Traceback (most recent call last):
> File "test/test_sslsocket.py", line 268, in test_newer_tls
> self._assert_connection_success(server, client)
> File "test/test_sslsocket.py", line 112, in _assert_connection_success
> self.assertTrue(acc.client is not None)
> AssertionError: False is not true
> ==
> FAIL: test_server_cert (__main__.TSSLSocketTest)
> --
> Traceback (most recent call last):
> File "test/test_sslsocket.py", line 172, in test_server_cert
> self._assert_connection_success(server, client)
> File "test/test_sslsocket.py", line 112, in _assert_connection_success
> self.assertTrue(acc.client is not None)
> AssertionError: False is not true
> ==
[jira] [Commented] (THRIFT-3616) lib/py/test/test_sslsocket.py is flaky
[
https://issues.apache.org/jira/browse/THRIFT-3616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15143466#comment-15143466
]
John Sirois commented on THRIFT-3616:
-
/cc [~nsuke] - I'm going to work this presently since I'm on a quest to get a
green jenkins Thrift-precommit job, but I may turn it over to you since you're
expert in this slice of the codebase if I have trouble making progress today.
> lib/py/test/test_sslsocket.py is flaky
> --
>
> Key: THRIFT-3616
> URL: https://issues.apache.org/jira/browse/THRIFT-3616
> Project: Thrift
> Issue Type: Bug
> Components: Python - Library, Test Suite
>Reporter: John Sirois
>Assignee: John Sirois
>Priority: Critical
>
> As seen here: https://builds.apache.org/job/Thrift-precommit/86/console
> {noformat}
> ...
> /usr/bin/python3 test/thrift_json.py
> .
> --
> Ran 1 test in 0.011s
> OK
> /usr/bin/python3 test/test_sslsocket.py
> .FFFsys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0,
> laddr=('::1', 23458, 0, 0), raddr=('::1', 41151, 0, 0)>
> F/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:253:
> DeprecationWarning: validate is deprecated. Use cert_reqs=ssl.CERT_REQUIRED
> instead
> DeprecationWarning)
> /thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:253:
> DeprecationWarning: validate is deprecated. Use cert_reqs=ssl.CERT_NONE
> instead
> DeprecationWarning)
> .sys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0,
> laddr=('::1', 23458, 0, 0), raddr=('::1', 41155, 0, 0)>
> F/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:265:
> DeprecationWarning: Use cert_reqs instead
> warnings.warn('Use cert_reqs instead', DeprecationWarning)
> .sys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0,
> laddr=('::1', 23458, 0, 0), raddr=('::1', 41157, 0, 0)>
> Exception in thread Thread-11:
> Traceback (most recent call last):
> File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner
> self.run()
> File "test/test_sslsocket.py", line 60, in run
> self.client = self._server.accept()
> File "/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py", line 418,
> in accept
> client = self._wrap_socket(plain_client)
> File "/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py", line 181,
> in _wrap_socket
> server_hostname=self._server_hostname)
> File "/usr/lib/python3.4/ssl.py", line 365, in wrap_socket
> _context=self)
> File "/usr/lib/python3.4/ssl.py", line 601, in __init__
> self.do_handshake()
> File "/usr/lib/python3.4/ssl.py", line 828, in do_handshake
> self._sslobj.do_handshake()
> ConnectionResetError: [Errno 104] Connection reset by peer
> PROTOCOL_SSLv2 is not available
> ..test/test_sslsocket.py:59: ResourceWarning: unclosed family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0>
> self._server.listen()
> .
> ==
> FAIL: test_client_cert (__main__.TSSLSocketTest)
> --
> Traceback (most recent call last):
> File "test/test_sslsocket.py", line 210, in test_client_cert
> self._assert_connection_success(server, client)
> File "test/test_sslsocket.py", line 112, in _assert_connection_success
> self.assertTrue(acc.client is not None)
> AssertionError: False is not true
> ==
> FAIL: test_deprecation (__main__.TSSLSocketTest)
> --
> Traceback (most recent call last):
> File "test/test_sslsocket.py", line 138, in test_deprecation
> self.assertEqual(len(w), 3)
> AssertionError: 4 != 3
> ==
> FAIL: test_newer_tls (__main__.TSSLSocketTest)
> --
> Traceback (most recent call last):
> File "test/test_sslsocket.py", line 268, in test_newer_tls
> self._assert_connection_success(server, client)
> File "test/test_sslsocket.py", line 112, in _assert_connection_success
> self.assertTrue(acc.client is not None)
> AssertionError: False is not true
> ==
> FAIL: test_server_cert (__main__.TSSLSocketTest)
> --
> Traceback (most recent call last):
> File "test/test_sslsocket.py", line 172, in test_server_cert
> self._assert_connection_success(server, client)
> File "test/test_ssl
[jira] [Updated] (THRIFT-3609) Remove or replace TestPortFixture.h
[
https://issues.apache.org/jira/browse/THRIFT-3609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Sirois updated THRIFT-3609:
Description: Today 6 cpp lib tests use
[TestPortFixture.h|https://github.com/apache/thrift/blob/1f6e380c5d07686e4cd8c2b172300a1ba7fbd8b9/lib/cpp/test/TestPortFixture.h].
THRIFT-3608 experimented with correcting one of these cases in
{{lib/cpp/test/SecurityTest.cpp}}, but a general cleanup should remove all uses
of static ports in the cpp tests to improve robustness and open the path to
parallel testing - at least in the cpp libs. (was: Today 6 cpp lib tests use
[TestPortFixture.h|https://github.com/apache/thrift/blob/1f6e380c5d07686e4cd8c2b172300a1ba7fbd8b9/lib/cpp/test/TestPortFixture.h].
THRIFT-3608 corrects one of these cases, but a general cleanup should remove
all uses of static ports in the cpp tests to improve robustness and open the
path to parallel testing - at least in the cpp libs.)
> Remove or replace TestPortFixture.h
> ---
>
> Key: THRIFT-3609
> URL: https://issues.apache.org/jira/browse/THRIFT-3609
> Project: Thrift
> Issue Type: Improvement
> Components: C++ - Library, Test Suite
>Reporter: John Sirois
>Assignee: John Sirois
>
> Today 6 cpp lib tests use
> [TestPortFixture.h|https://github.com/apache/thrift/blob/1f6e380c5d07686e4cd8c2b172300a1ba7fbd8b9/lib/cpp/test/TestPortFixture.h].
> THRIFT-3608 experimented with correcting one of these cases in
> {{lib/cpp/test/SecurityTest.cpp}}, but a general cleanup should remove all
> uses of static ports in the cpp tests to improve robustness and open the path
> to parallel testing - at least in the cpp libs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (THRIFT-3616) lib/py/test/test_sslsocket.py is flaky
[
https://issues.apache.org/jira/browse/THRIFT-3616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15143542#comment-15143542
]
John Sirois commented on THRIFT-3616:
-
OK - I have a delay-free way to fix this that also speeds up the test, just
need to polish and I'll have a PR shortly.
> lib/py/test/test_sslsocket.py is flaky
> --
>
> Key: THRIFT-3616
> URL: https://issues.apache.org/jira/browse/THRIFT-3616
> Project: Thrift
> Issue Type: Bug
> Components: Python - Library, Test Suite
>Reporter: John Sirois
>Assignee: John Sirois
>Priority: Critical
>
> As seen here: https://builds.apache.org/job/Thrift-precommit/86/console
> {noformat}
> ...
> /usr/bin/python3 test/thrift_json.py
> .
> --
> Ran 1 test in 0.011s
> OK
> /usr/bin/python3 test/test_sslsocket.py
> .FFFsys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0,
> laddr=('::1', 23458, 0, 0), raddr=('::1', 41151, 0, 0)>
> F/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:253:
> DeprecationWarning: validate is deprecated. Use cert_reqs=ssl.CERT_REQUIRED
> instead
> DeprecationWarning)
> /thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:253:
> DeprecationWarning: validate is deprecated. Use cert_reqs=ssl.CERT_NONE
> instead
> DeprecationWarning)
> .sys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0,
> laddr=('::1', 23458, 0, 0), raddr=('::1', 41155, 0, 0)>
> F/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:265:
> DeprecationWarning: Use cert_reqs instead
> warnings.warn('Use cert_reqs instead', DeprecationWarning)
> .sys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0,
> laddr=('::1', 23458, 0, 0), raddr=('::1', 41157, 0, 0)>
> Exception in thread Thread-11:
> Traceback (most recent call last):
> File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner
> self.run()
> File "test/test_sslsocket.py", line 60, in run
> self.client = self._server.accept()
> File "/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py", line 418,
> in accept
> client = self._wrap_socket(plain_client)
> File "/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py", line 181,
> in _wrap_socket
> server_hostname=self._server_hostname)
> File "/usr/lib/python3.4/ssl.py", line 365, in wrap_socket
> _context=self)
> File "/usr/lib/python3.4/ssl.py", line 601, in __init__
> self.do_handshake()
> File "/usr/lib/python3.4/ssl.py", line 828, in do_handshake
> self._sslobj.do_handshake()
> ConnectionResetError: [Errno 104] Connection reset by peer
> PROTOCOL_SSLv2 is not available
> ..test/test_sslsocket.py:59: ResourceWarning: unclosed family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0>
> self._server.listen()
> .
> ==
> FAIL: test_client_cert (__main__.TSSLSocketTest)
> --
> Traceback (most recent call last):
> File "test/test_sslsocket.py", line 210, in test_client_cert
> self._assert_connection_success(server, client)
> File "test/test_sslsocket.py", line 112, in _assert_connection_success
> self.assertTrue(acc.client is not None)
> AssertionError: False is not true
> ==
> FAIL: test_deprecation (__main__.TSSLSocketTest)
> --
> Traceback (most recent call last):
> File "test/test_sslsocket.py", line 138, in test_deprecation
> self.assertEqual(len(w), 3)
> AssertionError: 4 != 3
> ==
> FAIL: test_newer_tls (__main__.TSSLSocketTest)
> --
> Traceback (most recent call last):
> File "test/test_sslsocket.py", line 268, in test_newer_tls
> self._assert_connection_success(server, client)
> File "test/test_sslsocket.py", line 112, in _assert_connection_success
> self.assertTrue(acc.client is not None)
> AssertionError: False is not true
> ==
> FAIL: test_server_cert (__main__.TSSLSocketTest)
> --
> Traceback (most recent call last):
> File "test/test_sslsocket.py", line 172, in test_server_cert
> self._assert_connection_success(server, client)
> File "test/test_sslsocket.py", line 112, in _assert_connection_success
> self.assertTrue(acc.client is not None)
> AssertionError: F
[jira] [Comment Edited] (THRIFT-3616) lib/py/test/test_sslsocket.py is flaky
[
https://issues.apache.org/jira/browse/THRIFT-3616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15143542#comment-15143542
]
John Sirois edited comment on THRIFT-3616 at 2/11/16 9:34 PM:
--
OK - I have a delay-free way to fix this that also speeds up the test, just
need to polish and I'll have a PR shortly.
Before: Ran 11 tests in 8.213s
After: Ran 11 tests in 0.284s
was (Author: jsirois):
OK - I have a delay-free way to fix this that also speeds up the test, just
need to polish and I'll have a PR shortly.
> lib/py/test/test_sslsocket.py is flaky
> --
>
> Key: THRIFT-3616
> URL: https://issues.apache.org/jira/browse/THRIFT-3616
> Project: Thrift
> Issue Type: Bug
> Components: Python - Library, Test Suite
>Reporter: John Sirois
>Assignee: John Sirois
>Priority: Critical
>
> As seen here: https://builds.apache.org/job/Thrift-precommit/86/console
> {noformat}
> ...
> /usr/bin/python3 test/thrift_json.py
> .
> --
> Ran 1 test in 0.011s
> OK
> /usr/bin/python3 test/test_sslsocket.py
> .FFFsys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0,
> laddr=('::1', 23458, 0, 0), raddr=('::1', 41151, 0, 0)>
> F/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:253:
> DeprecationWarning: validate is deprecated. Use cert_reqs=ssl.CERT_REQUIRED
> instead
> DeprecationWarning)
> /thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:253:
> DeprecationWarning: validate is deprecated. Use cert_reqs=ssl.CERT_NONE
> instead
> DeprecationWarning)
> .sys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0,
> laddr=('::1', 23458, 0, 0), raddr=('::1', 41155, 0, 0)>
> F/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:265:
> DeprecationWarning: Use cert_reqs instead
> warnings.warn('Use cert_reqs instead', DeprecationWarning)
> .sys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0,
> laddr=('::1', 23458, 0, 0), raddr=('::1', 41157, 0, 0)>
> Exception in thread Thread-11:
> Traceback (most recent call last):
> File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner
> self.run()
> File "test/test_sslsocket.py", line 60, in run
> self.client = self._server.accept()
> File "/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py", line 418,
> in accept
> client = self._wrap_socket(plain_client)
> File "/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py", line 181,
> in _wrap_socket
> server_hostname=self._server_hostname)
> File "/usr/lib/python3.4/ssl.py", line 365, in wrap_socket
> _context=self)
> File "/usr/lib/python3.4/ssl.py", line 601, in __init__
> self.do_handshake()
> File "/usr/lib/python3.4/ssl.py", line 828, in do_handshake
> self._sslobj.do_handshake()
> ConnectionResetError: [Errno 104] Connection reset by peer
> PROTOCOL_SSLv2 is not available
> ..test/test_sslsocket.py:59: ResourceWarning: unclosed family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0>
> self._server.listen()
> .
> ==
> FAIL: test_client_cert (__main__.TSSLSocketTest)
> --
> Traceback (most recent call last):
> File "test/test_sslsocket.py", line 210, in test_client_cert
> self._assert_connection_success(server, client)
> File "test/test_sslsocket.py", line 112, in _assert_connection_success
> self.assertTrue(acc.client is not None)
> AssertionError: False is not true
> ==
> FAIL: test_deprecation (__main__.TSSLSocketTest)
> --
> Traceback (most recent call last):
> File "test/test_sslsocket.py", line 138, in test_deprecation
> self.assertEqual(len(w), 3)
> AssertionError: 4 != 3
> ==
> FAIL: test_newer_tls (__main__.TSSLSocketTest)
> --
> Traceback (most recent call last):
> File "test/test_sslsocket.py", line 268, in test_newer_tls
> self._assert_connection_success(server, client)
> File "test/test_sslsocket.py", line 112, in _assert_connection_success
> self.assertTrue(acc.client is not None)
> AssertionError: False is not true
> ==
> FAIL: test_server_cert (__main__.TSSLSocketTest)
> --
> Traceback (most recent call last):
>
[jira] [Commented] (THRIFT-3616) lib/py/test/test_sslsocket.py is flaky
[
https://issues.apache.org/jira/browse/THRIFT-3616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15144683#comment-15144683
]
John Sirois commented on THRIFT-3616:
-
https://github.com/apache/thrift/pull/850
> lib/py/test/test_sslsocket.py is flaky
> --
>
> Key: THRIFT-3616
> URL: https://issues.apache.org/jira/browse/THRIFT-3616
> Project: Thrift
> Issue Type: Bug
> Components: Python - Library, Test Suite
>Reporter: John Sirois
>Assignee: John Sirois
>Priority: Critical
>
> As seen here: https://builds.apache.org/job/Thrift-precommit/86/console
> {noformat}
> ...
> /usr/bin/python3 test/thrift_json.py
> .
> --
> Ran 1 test in 0.011s
> OK
> /usr/bin/python3 test/test_sslsocket.py
> .FFFsys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0,
> laddr=('::1', 23458, 0, 0), raddr=('::1', 41151, 0, 0)>
> F/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:253:
> DeprecationWarning: validate is deprecated. Use cert_reqs=ssl.CERT_REQUIRED
> instead
> DeprecationWarning)
> /thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:253:
> DeprecationWarning: validate is deprecated. Use cert_reqs=ssl.CERT_NONE
> instead
> DeprecationWarning)
> .sys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0,
> laddr=('::1', 23458, 0, 0), raddr=('::1', 41155, 0, 0)>
> F/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:265:
> DeprecationWarning: Use cert_reqs instead
> warnings.warn('Use cert_reqs instead', DeprecationWarning)
> .sys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0,
> laddr=('::1', 23458, 0, 0), raddr=('::1', 41157, 0, 0)>
> Exception in thread Thread-11:
> Traceback (most recent call last):
> File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner
> self.run()
> File "test/test_sslsocket.py", line 60, in run
> self.client = self._server.accept()
> File "/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py", line 418,
> in accept
> client = self._wrap_socket(plain_client)
> File "/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py", line 181,
> in _wrap_socket
> server_hostname=self._server_hostname)
> File "/usr/lib/python3.4/ssl.py", line 365, in wrap_socket
> _context=self)
> File "/usr/lib/python3.4/ssl.py", line 601, in __init__
> self.do_handshake()
> File "/usr/lib/python3.4/ssl.py", line 828, in do_handshake
> self._sslobj.do_handshake()
> ConnectionResetError: [Errno 104] Connection reset by peer
> PROTOCOL_SSLv2 is not available
> ..test/test_sslsocket.py:59: ResourceWarning: unclosed family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0>
> self._server.listen()
> .
> ==
> FAIL: test_client_cert (__main__.TSSLSocketTest)
> --
> Traceback (most recent call last):
> File "test/test_sslsocket.py", line 210, in test_client_cert
> self._assert_connection_success(server, client)
> File "test/test_sslsocket.py", line 112, in _assert_connection_success
> self.assertTrue(acc.client is not None)
> AssertionError: False is not true
> ==
> FAIL: test_deprecation (__main__.TSSLSocketTest)
> --
> Traceback (most recent call last):
> File "test/test_sslsocket.py", line 138, in test_deprecation
> self.assertEqual(len(w), 3)
> AssertionError: 4 != 3
> ==
> FAIL: test_newer_tls (__main__.TSSLSocketTest)
> --
> Traceback (most recent call last):
> File "test/test_sslsocket.py", line 268, in test_newer_tls
> self._assert_connection_success(server, client)
> File "test/test_sslsocket.py", line 112, in _assert_connection_success
> self.assertTrue(acc.client is not None)
> AssertionError: False is not true
> ==
> FAIL: test_server_cert (__main__.TSSLSocketTest)
> --
> Traceback (most recent call last):
> File "test/test_sslsocket.py", line 172, in test_server_cert
> self._assert_connection_success(server, client)
> File "test/test_sslsocket.py", line 112, in _assert_connection_success
> self.assertTrue(acc.client is not None)
> AssertionError: False is not true
> ===
[jira] [Commented] (THRIFT-3608) lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build.
[
https://issues.apache.org/jira/browse/THRIFT-3608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15144733#comment-15144733
]
John Sirois commented on THRIFT-3608:
-
I'm not sure I follow the don't wait part. I do need to wait on some committer
to review and merge. If you can recommend an appropriate one I'd be grateful.
> lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build.
> -
>
> Key: THRIFT-3608
> URL: https://issues.apache.org/jira/browse/THRIFT-3608
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library, Test Suite
>Affects Versions: 0.9.3
>Reporter: John Sirois
>Assignee: John Sirois
>Priority: Critical
>
> The [Thrift-precommit|https://builds.apache.org/job/Thrift-precommit] job now
> runs in the build/docker/ubuntu container and the
> [lib/cpp/test/SecurityTest|https://github.com/apache/thrift/blob/master/lib/cpp/test/SecurityTest.cpp]
> fails most - but not all - of the time.
> Failures can be seen in the following jobs
> * https://builds.apache.org/job/Thrift-precommit/60/console
> * https://builds.apache.org/job/Thrift-precommit/59/console
> * https://builds.apache.org/job/Thrift-precommit/58/console
> * https://builds.apache.org/job/Thrift-precommit/57/console
> They look like so:
> {noformat}
> argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest"
> Running 1 test case...
> Entering test suite "SecurityTest"
> Entering test suite "SecurityTest"
> Entering test case "ssl_security_matrix"
> TEST: Server = SSLTLS, Client = SSLTLS
> TEST: Server = SSLTLS, Client = SSLv3
> SRV 2b9182317700 Exception: SSL_accept: wrong version number
> CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = SSLTLS, Client = TLSv1_0
> TEST: Server = SSLTLS, Client = TLSv1_1
> TEST: Server = SSLTLS, Client = TLSv1_2
> TEST: Server = SSLv3, Client = SSLTLS
> CLI 2b9181f0a700 Exception: SSL_connect: unsupported protocol
> SRV 2b9182317700 Exception: SSL_accept: error code: 0
> TEST: Server = SSLv3, Client = SSLv3
> TEST: Server = SSLv3, Client = TLSv1_0
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = SSLv3, Client = TLSv1_1
> CLI 2b9182317700 Exception: SSL_connect: wrong version number
> SRV 2b9181f0a700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = SSLv3, Client = TLSv1_2
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = TLSv1_0, Client = SSLTLS
> TEST: Server = TLSv1_0, Client = SSLv3
> SRV 2b9182317700 Exception: SSL_accept: wrong version number
> CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = TLSv1_0, Client = TLSv1_0
> TEST: Server = TLSv1_0, Client = TLSv1_1
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version
> TEST: Server = TLSv1_0, Client = TLSv1_2
> CLI 2b9182317700 Exception: SSL_connect: wrong version number
> SRV 2b9181f0a700 Exception: SSL_accept: tlsv1 alert protocol version
> TEST: Server = TLSv1_1, Client = SSLTLS
> TEST: Server = TLSv1_1, Client = SSLv3
> SRV 2b9181f0a700 Exception: SSL_accept: wrong version number
> CLI 2b9182317700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = TLSv1_1, Client = TLSv1_0
> SRV 2b9182317700 Exception: SSL_accept: wrong version number
> CLI 2b9181f0a700 Exception: SSL_connect: tlsv1 alert protocol version
> TEST: Server = TLSv1_1, Client = TLSv1_1
> TEST: Server = TLSv1_1, Client = TLSv1_2
> SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> TEST: Server = TLSv1_2, Client = SSLTLS
> [0;31mFAIL[m: SecurityTest
> {noformat}
> {noformat}
> argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest"
> Running 1 test case...
> Entering test suite "SecurityTest"
> Entering test suite "SecurityTest"
> Entering test case "ssl_security_matrix"
> TEST: Server = SSLTLS, Client = SSLTLS
> TEST: Server = SSLTLS, Client = SSLv3
> SRV 2b6bd3245700 Exception: SSL_accept: wrong version number
> CLI 2b6bd2e38700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = SSLTLS, Client = TLSv1_0
> TEST: Server = SSLTLS, Client = TLSv1_1
> TEST: Server = SSLTLS, Client = TLSv1_2
> TEST: Server = SSLv3, Client = SSLTLS
> CLI 2b6bd2e38700 Exception: SSL_connect: unsupported protocol
> SRV 2b6bd3245700 Exception: SSL_accept: error code: 0
> TEST: Server = SSLv3, Client = SSLv3
> TEST: Server = SSLv3, Client = TLSv1_0
> CLI 2b6bd2e38700 Exception: SSL_connect: wrong version number
> SRV 2b6bd3245700 Exception: SS
[jira] [Commented] (THRIFT-3616) lib/py/test/test_sslsocket.py is flaky
[
https://issues.apache.org/jira/browse/THRIFT-3616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15144928#comment-15144928
]
John Sirois commented on THRIFT-3616:
-
Ah yes, I misread 8 and 9 here:
https://thrift.apache.org/docs/HowToContribute#contributing-via-github-pull-requests
Will do going forward.
> lib/py/test/test_sslsocket.py is flaky
> --
>
> Key: THRIFT-3616
> URL: https://issues.apache.org/jira/browse/THRIFT-3616
> Project: Thrift
> Issue Type: Bug
> Components: Python - Library, Test Suite
>Reporter: John Sirois
>Assignee: John Sirois
>Priority: Critical
> Fix For: 0.9.4
>
>
> As seen here: https://builds.apache.org/job/Thrift-precommit/86/console
> {noformat}
> ...
> /usr/bin/python3 test/thrift_json.py
> .
> --
> Ran 1 test in 0.011s
> OK
> /usr/bin/python3 test/test_sslsocket.py
> .FFFsys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0,
> laddr=('::1', 23458, 0, 0), raddr=('::1', 41151, 0, 0)>
> F/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:253:
> DeprecationWarning: validate is deprecated. Use cert_reqs=ssl.CERT_REQUIRED
> instead
> DeprecationWarning)
> /thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:253:
> DeprecationWarning: validate is deprecated. Use cert_reqs=ssl.CERT_NONE
> instead
> DeprecationWarning)
> .sys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0,
> laddr=('::1', 23458, 0, 0), raddr=('::1', 41155, 0, 0)>
> F/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:265:
> DeprecationWarning: Use cert_reqs instead
> warnings.warn('Use cert_reqs instead', DeprecationWarning)
> .sys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0,
> laddr=('::1', 23458, 0, 0), raddr=('::1', 41157, 0, 0)>
> Exception in thread Thread-11:
> Traceback (most recent call last):
> File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner
> self.run()
> File "test/test_sslsocket.py", line 60, in run
> self.client = self._server.accept()
> File "/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py", line 418,
> in accept
> client = self._wrap_socket(plain_client)
> File "/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py", line 181,
> in _wrap_socket
> server_hostname=self._server_hostname)
> File "/usr/lib/python3.4/ssl.py", line 365, in wrap_socket
> _context=self)
> File "/usr/lib/python3.4/ssl.py", line 601, in __init__
> self.do_handshake()
> File "/usr/lib/python3.4/ssl.py", line 828, in do_handshake
> self._sslobj.do_handshake()
> ConnectionResetError: [Errno 104] Connection reset by peer
> PROTOCOL_SSLv2 is not available
> ..test/test_sslsocket.py:59: ResourceWarning: unclosed family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0>
> self._server.listen()
> .
> ==
> FAIL: test_client_cert (__main__.TSSLSocketTest)
> --
> Traceback (most recent call last):
> File "test/test_sslsocket.py", line 210, in test_client_cert
> self._assert_connection_success(server, client)
> File "test/test_sslsocket.py", line 112, in _assert_connection_success
> self.assertTrue(acc.client is not None)
> AssertionError: False is not true
> ==
> FAIL: test_deprecation (__main__.TSSLSocketTest)
> --
> Traceback (most recent call last):
> File "test/test_sslsocket.py", line 138, in test_deprecation
> self.assertEqual(len(w), 3)
> AssertionError: 4 != 3
> ==
> FAIL: test_newer_tls (__main__.TSSLSocketTest)
> --
> Traceback (most recent call last):
> File "test/test_sslsocket.py", line 268, in test_newer_tls
> self._assert_connection_success(server, client)
> File "test/test_sslsocket.py", line 112, in _assert_connection_success
> self.assertTrue(acc.client is not None)
> AssertionError: False is not true
> ==
> FAIL: test_server_cert (__main__.TSSLSocketTest)
> --
> Traceback (most recent call last):
> File "test/test_sslsocket.py", line 172, in test_server_cert
> self._assert_connection_success(server, client)
> File "test/test_sslsocket.py", line 112, in _assert_connection_success
> self.as
[jira] [Commented] (THRIFT-3616) lib/py/test/test_sslsocket.py is flaky
[
https://issues.apache.org/jira/browse/THRIFT-3616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15144950#comment-15144950
]
John Sirois commented on THRIFT-3616:
-
Thanks [~nsuke]!
> lib/py/test/test_sslsocket.py is flaky
> --
>
> Key: THRIFT-3616
> URL: https://issues.apache.org/jira/browse/THRIFT-3616
> Project: Thrift
> Issue Type: Bug
> Components: Python - Library, Test Suite
>Reporter: John Sirois
>Assignee: John Sirois
>Priority: Critical
> Fix For: 0.9.4
>
>
> As seen here: https://builds.apache.org/job/Thrift-precommit/86/console
> {noformat}
> ...
> /usr/bin/python3 test/thrift_json.py
> .
> --
> Ran 1 test in 0.011s
> OK
> /usr/bin/python3 test/test_sslsocket.py
> .FFFsys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0,
> laddr=('::1', 23458, 0, 0), raddr=('::1', 41151, 0, 0)>
> F/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:253:
> DeprecationWarning: validate is deprecated. Use cert_reqs=ssl.CERT_REQUIRED
> instead
> DeprecationWarning)
> /thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:253:
> DeprecationWarning: validate is deprecated. Use cert_reqs=ssl.CERT_NONE
> instead
> DeprecationWarning)
> .sys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0,
> laddr=('::1', 23458, 0, 0), raddr=('::1', 41155, 0, 0)>
> F/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py:265:
> DeprecationWarning: Use cert_reqs instead
> warnings.warn('Use cert_reqs instead', DeprecationWarning)
> .sys:1: ResourceWarning: unclosed family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0,
> laddr=('::1', 23458, 0, 0), raddr=('::1', 41157, 0, 0)>
> Exception in thread Thread-11:
> Traceback (most recent call last):
> File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner
> self.run()
> File "test/test_sslsocket.py", line 60, in run
> self.client = self._server.accept()
> File "/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py", line 418,
> in accept
> client = self._wrap_socket(plain_client)
> File "/thrift/lib/py/build/lib/thrift/transport/TSSLSocket.py", line 181,
> in _wrap_socket
> server_hostname=self._server_hostname)
> File "/usr/lib/python3.4/ssl.py", line 365, in wrap_socket
> _context=self)
> File "/usr/lib/python3.4/ssl.py", line 601, in __init__
> self.do_handshake()
> File "/usr/lib/python3.4/ssl.py", line 828, in do_handshake
> self._sslobj.do_handshake()
> ConnectionResetError: [Errno 104] Connection reset by peer
> PROTOCOL_SSLv2 is not available
> ..test/test_sslsocket.py:59: ResourceWarning: unclosed family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0>
> self._server.listen()
> .
> ==
> FAIL: test_client_cert (__main__.TSSLSocketTest)
> --
> Traceback (most recent call last):
> File "test/test_sslsocket.py", line 210, in test_client_cert
> self._assert_connection_success(server, client)
> File "test/test_sslsocket.py", line 112, in _assert_connection_success
> self.assertTrue(acc.client is not None)
> AssertionError: False is not true
> ==
> FAIL: test_deprecation (__main__.TSSLSocketTest)
> --
> Traceback (most recent call last):
> File "test/test_sslsocket.py", line 138, in test_deprecation
> self.assertEqual(len(w), 3)
> AssertionError: 4 != 3
> ==
> FAIL: test_newer_tls (__main__.TSSLSocketTest)
> --
> Traceback (most recent call last):
> File "test/test_sslsocket.py", line 268, in test_newer_tls
> self._assert_connection_success(server, client)
> File "test/test_sslsocket.py", line 112, in _assert_connection_success
> self.assertTrue(acc.client is not None)
> AssertionError: False is not true
> ==
> FAIL: test_server_cert (__main__.TSSLSocketTest)
> --
> Traceback (most recent call last):
> File "test/test_sslsocket.py", line 172, in test_server_cert
> self._assert_connection_success(server, client)
> File "test/test_sslsocket.py", line 112, in _assert_connection_success
> self.assertTrue(acc.client is not None)
> AssertionError: False is not true
> =
[jira] [Commented] (THRIFT-3608) lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build.
[
https://issues.apache.org/jira/browse/THRIFT-3608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15144951#comment-15144951
]
John Sirois commented on THRIFT-3608:
-
Thank you! Thrift-precommit should hopefully be going green now for github PRs.
> lib/cpp/test/SecurityTest is flaky in jenkins Thrift-precommit build.
> -
>
> Key: THRIFT-3608
> URL: https://issues.apache.org/jira/browse/THRIFT-3608
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library, Test Suite
>Affects Versions: 0.9.3
>Reporter: John Sirois
>Assignee: John Sirois
>Priority: Critical
> Fix For: 0.9.4
>
>
> The [Thrift-precommit|https://builds.apache.org/job/Thrift-precommit] job now
> runs in the build/docker/ubuntu container and the
> [lib/cpp/test/SecurityTest|https://github.com/apache/thrift/blob/master/lib/cpp/test/SecurityTest.cpp]
> fails most - but not all - of the time.
> Failures can be seen in the following jobs
> * https://builds.apache.org/job/Thrift-precommit/60/console
> * https://builds.apache.org/job/Thrift-precommit/59/console
> * https://builds.apache.org/job/Thrift-precommit/58/console
> * https://builds.apache.org/job/Thrift-precommit/57/console
> They look like so:
> {noformat}
> argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest"
> Running 1 test case...
> Entering test suite "SecurityTest"
> Entering test suite "SecurityTest"
> Entering test case "ssl_security_matrix"
> TEST: Server = SSLTLS, Client = SSLTLS
> TEST: Server = SSLTLS, Client = SSLv3
> SRV 2b9182317700 Exception: SSL_accept: wrong version number
> CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = SSLTLS, Client = TLSv1_0
> TEST: Server = SSLTLS, Client = TLSv1_1
> TEST: Server = SSLTLS, Client = TLSv1_2
> TEST: Server = SSLv3, Client = SSLTLS
> CLI 2b9181f0a700 Exception: SSL_connect: unsupported protocol
> SRV 2b9182317700 Exception: SSL_accept: error code: 0
> TEST: Server = SSLv3, Client = SSLv3
> TEST: Server = SSLv3, Client = TLSv1_0
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = SSLv3, Client = TLSv1_1
> CLI 2b9182317700 Exception: SSL_connect: wrong version number
> SRV 2b9181f0a700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = SSLv3, Client = TLSv1_2
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> SRV 2b9182317700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: Server = TLSv1_0, Client = SSLTLS
> TEST: Server = TLSv1_0, Client = SSLv3
> SRV 2b9182317700 Exception: SSL_accept: wrong version number
> CLI 2b9181f0a700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = TLSv1_0, Client = TLSv1_0
> TEST: Server = TLSv1_0, Client = TLSv1_1
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version
> TEST: Server = TLSv1_0, Client = TLSv1_2
> CLI 2b9182317700 Exception: SSL_connect: wrong version number
> SRV 2b9181f0a700 Exception: SSL_accept: tlsv1 alert protocol version
> TEST: Server = TLSv1_1, Client = SSLTLS
> TEST: Server = TLSv1_1, Client = SSLv3
> SRV 2b9181f0a700 Exception: SSL_accept: wrong version number
> CLI 2b9182317700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = TLSv1_1, Client = TLSv1_0
> SRV 2b9182317700 Exception: SSL_accept: wrong version number
> CLI 2b9181f0a700 Exception: SSL_connect: tlsv1 alert protocol version
> TEST: Server = TLSv1_1, Client = TLSv1_1
> TEST: Server = TLSv1_1, Client = TLSv1_2
> SRV 2b9182317700 Exception: SSL_accept: tlsv1 alert protocol version
> CLI 2b9181f0a700 Exception: SSL_connect: wrong version number
> TEST: Server = TLSv1_2, Client = SSLTLS
> [0;31mFAIL[m: SecurityTest
> {noformat}
> {noformat}
> argv[0] = "/thrift/lib/cpp/test/.libs/lt-SecurityTest"
> Running 1 test case...
> Entering test suite "SecurityTest"
> Entering test suite "SecurityTest"
> Entering test case "ssl_security_matrix"
> TEST: Server = SSLTLS, Client = SSLTLS
> TEST: Server = SSLTLS, Client = SSLv3
> SRV 2b6bd3245700 Exception: SSL_accept: wrong version number
> CLI 2b6bd2e38700 Exception: SSL_connect: sslv3 alert handshake failure
> TEST: Server = SSLTLS, Client = TLSv1_0
> TEST: Server = SSLTLS, Client = TLSv1_1
> TEST: Server = SSLTLS, Client = TLSv1_2
> TEST: Server = SSLv3, Client = SSLTLS
> CLI 2b6bd2e38700 Exception: SSL_connect: unsupported protocol
> SRV 2b6bd3245700 Exception: SSL_accept: error code: 0
> TEST: Server = SSLv3, Client = SSLv3
> TEST: Server = SSLv3, Client = TLSv1_0
> CLI 2b6bd2e38700 Exception: SSL_connect: wrong version number
> SRV 2b6bd3245700 Exception: SSL_accept: sslv3 alert handshake failure
> TEST: S
[jira] [Commented] (THRIFT-3112) [Java] AsyncMethodCallback should be typed in generated AsyncIface
[
https://issues.apache.org/jira/browse/THRIFT-3112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145025#comment-15145025
]
John Sirois commented on THRIFT-3112:
-
The PROPOSAL thread has had 1 positive response from [~nsuke] and no other
responses. The PR @ https://github.com/apache/thrift/pull/840 is now green
after re-basing against master to pick up fixes for THRIFT-3608 and THRIFT-3616
and ready for review and merging.
> [Java] AsyncMethodCallback should be typed in generated AsyncIface
> --
>
> Key: THRIFT-3112
> URL: https://issues.apache.org/jira/browse/THRIFT-3112
> Project: Thrift
> Issue Type: Improvement
> Components: Java - Compiler, Java - Library
>Affects Versions: 0.9.2
>Reporter: Sergei Egorov
>Assignee: John Sirois
>
> AsyncMethodCallback is generic, but current Java code generator is not adding
> type info to it, and instead of:
> {code:java}
> public interface AsyncIface {
> public void ping(org.apache.thrift.async.AsyncMethodCallback
> resultHandler) throws org.apache.thrift.TException;
> }
> {code}
> we have:
> {code:java}
> public interface AsyncIface {
> public void ping(org.apache.thrift.async.AsyncMethodCallback
> resultHandler) throws org.apache.thrift.TException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (THRIFT-3620) Cleanup and consolidate Thrift* jenkins jobs.
John Sirois created THRIFT-3620:
---
Summary: Cleanup and consolidate Thrift* jenkins jobs.
Key: THRIFT-3620
URL: https://issues.apache.org/jira/browse/THRIFT-3620
Project: Thrift
Issue Type: Task
Components: Test Suite
Reporter: John Sirois
I spent some time resurrecting the
[Thrift-precommit|https://builds.apache.org/job/Thrift-precommit/] job which
vets pull-requests to use the pool of docker slaves using a dockerized CI with
guidance from [~jfarrell]. The remaining crop of Thrift* jobs should be culled
and converted as appropriate.
Currently we have the jobs
[here|https://builds.apache.org/view/S-Z/view/Thrift/]:
{noformat}
$ curl --netrc -sS https://builds.apache.org/view/S-Z/view/Thrift/config.xml |
xmllint -xpath "//jobNames" - | grep "" | cut -d'>' -f2 | cut -d'<' -f1
Thrift
Thrift-all
Thrift-Compiler-Linux64
Thrift-Compiler-Windows
Thrift-env
Thrift-env-Windows
Thrift-Test
Thrift-Windows
{noformat}
Of those, [Thrift-env|https://builds.apache.org/job/Thrift-env],
[Thrift-env-Windows|https://builds.apache.org/job/Thrift-env-Windows] and
[Thrift-Windows|https://builds.apache.org/job/Thrift-Windows] are all disabled
and [Thrift|https://builds.apache.org/job/Thrift] and
[Thrift-Compiler-Linux64|https://builds.apache.org/job/Thrift-Compiler-Linux64]
seem to be unused based on last run dates. This leaves the following active
jobs fwict:
* [|https://builds.apache.org/job/Thrift-all]
* [|https://builds.apache.org/job/Thrift-Compiler-Windows]
* [|https://builds.apache.org/job/Thrift-precommit]
Of these {{Thrift-all}} is broken in a known-way, it really needs to be
converted to use a Dockerized build like {{Thrift-precommit}} to be insulated
from the vagaries of the underlying worker machines.
I propose deltion of all disabled (3) and inactive (2) jobs listed above and
conversion of {{Thrift-all}} to a Dockerized CI job to guard master with.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (THRIFT-3620) Cleanup and consolidate Thrift* jenkins jobs.
[
https://issues.apache.org/jira/browse/THRIFT-3620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145116#comment-15145116
]
John Sirois commented on THRIFT-3620:
-
[~jfarrell], [~jensg], [~roger.meier] and [~nsuke] - I could use your
institutional knowledge and feedback here. I'd love to kill the cruft and
convert the useful but broken, but I need feedback to do this right. If you
can comment on my proposal and set me straight on what I've got wrong, I'd be
much appreciative.
> Cleanup and consolidate Thrift* jenkins jobs.
> -
>
> Key: THRIFT-3620
> URL: https://issues.apache.org/jira/browse/THRIFT-3620
> Project: Thrift
> Issue Type: Task
> Components: Test Suite
>Reporter: John Sirois
>
> I spent some time resurrecting the
> [Thrift-precommit|https://builds.apache.org/job/Thrift-precommit/] job which
> vets pull-requests to use the pool of docker slaves using a dockerized CI
> with guidance from [~jfarrell]. The remaining crop of Thrift* jobs should be
> culled and converted as appropriate.
> Currently we have the jobs
> [here|https://builds.apache.org/view/S-Z/view/Thrift/]:
> {noformat}
> $ curl --netrc -sS https://builds.apache.org/view/S-Z/view/Thrift/config.xml
> | xmllint -xpath "//jobNames" - | grep "" | cut -d'>' -f2 | cut -d'<'
> -f1
> Thrift
> Thrift-all
> Thrift-Compiler-Linux64
> Thrift-Compiler-Windows
> Thrift-env
> Thrift-env-Windows
> Thrift-Test
> Thrift-Windows
> {noformat}
> Of those, [Thrift-env|https://builds.apache.org/job/Thrift-env],
> [Thrift-env-Windows|https://builds.apache.org/job/Thrift-env-Windows] and
> [Thrift-Windows|https://builds.apache.org/job/Thrift-Windows] are all
> disabled and [Thrift|https://builds.apache.org/job/Thrift] and
> [Thrift-Compiler-Linux64|https://builds.apache.org/job/Thrift-Compiler-Linux64]
> seem to be unused based on last run dates. This leaves the following active
> jobs fwict:
> * [|https://builds.apache.org/job/Thrift-all]
> * [|https://builds.apache.org/job/Thrift-Compiler-Windows]
> * [|https://builds.apache.org/job/Thrift-precommit]
> Of these {{Thrift-all}} is broken in a known-way, it really needs to be
> converted to use a Dockerized build like {{Thrift-precommit}} to be insulated
> from the vagaries of the underlying worker machines.
> I propose deltion of all disabled (3) and inactive (2) jobs listed above and
> conversion of {{Thrift-all}} to a Dockerized CI job to guard master with.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (THRIFT-3620) Cleanup and consolidate Thrift* jenkins jobs.
[
https://issues.apache.org/jira/browse/THRIFT-3620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Sirois reassigned THRIFT-3620:
---
Assignee: John Sirois
> Cleanup and consolidate Thrift* jenkins jobs.
> -
>
> Key: THRIFT-3620
> URL: https://issues.apache.org/jira/browse/THRIFT-3620
> Project: Thrift
> Issue Type: Task
> Components: Test Suite
>Reporter: John Sirois
>Assignee: John Sirois
>
> I spent some time resurrecting the
> [Thrift-precommit|https://builds.apache.org/job/Thrift-precommit/] job which
> vets pull-requests to use the pool of docker slaves using a dockerized CI
> with guidance from [~jfarrell]. The remaining crop of Thrift* jobs should be
> culled and converted as appropriate.
> Currently we have the jobs
> [here|https://builds.apache.org/view/S-Z/view/Thrift/]:
> {noformat}
> $ curl --netrc -sS https://builds.apache.org/view/S-Z/view/Thrift/config.xml
> | xmllint -xpath "//jobNames" - | grep "" | cut -d'>' -f2 | cut -d'<'
> -f1
> Thrift
> Thrift-all
> Thrift-Compiler-Linux64
> Thrift-Compiler-Windows
> Thrift-env
> Thrift-env-Windows
> Thrift-Test
> Thrift-Windows
> {noformat}
> Of those, [Thrift-env|https://builds.apache.org/job/Thrift-env],
> [Thrift-env-Windows|https://builds.apache.org/job/Thrift-env-Windows] and
> [Thrift-Windows|https://builds.apache.org/job/Thrift-Windows] are all
> disabled and [Thrift|https://builds.apache.org/job/Thrift] and
> [Thrift-Compiler-Linux64|https://builds.apache.org/job/Thrift-Compiler-Linux64]
> seem to be unused based on last run dates. This leaves the following active
> jobs fwict:
> * [|https://builds.apache.org/job/Thrift-all]
> * [|https://builds.apache.org/job/Thrift-Compiler-Windows]
> * [|https://builds.apache.org/job/Thrift-precommit]
> Of these {{Thrift-all}} is broken in a known-way, it really needs to be
> converted to use a Dockerized build like {{Thrift-precommit}} to be insulated
> from the vagaries of the underlying worker machines.
> I propose deltion of all disabled (3) and inactive (2) jobs listed above and
> conversion of {{Thrift-all}} to a Dockerized CI job to guard master with.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (THRIFT-3620) Cleanup and consolidate Thrift* jenkins jobs.
[
https://issues.apache.org/jira/browse/THRIFT-3620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145263#comment-15145263
]
John Sirois commented on THRIFT-3620:
-
Ok - it would be nice to have a single job and I had not realized the windows
job actually ran on ubuntu and just used mingw. I'll proceed on this in the
background to things like THRIFT-3112, THRIFT-2157, THRIFT-3583 which are main
main focus now that Thrift-precommit is green and stable.
> Cleanup and consolidate Thrift* jenkins jobs.
> -
>
> Key: THRIFT-3620
> URL: https://issues.apache.org/jira/browse/THRIFT-3620
> Project: Thrift
> Issue Type: Task
> Components: Test Suite
>Reporter: John Sirois
>Assignee: John Sirois
>
> I spent some time resurrecting the
> [Thrift-precommit|https://builds.apache.org/job/Thrift-precommit/] job which
> vets pull-requests to use the pool of docker slaves using a dockerized CI
> with guidance from [~jfarrell]. The remaining crop of Thrift* jobs should be
> culled and converted as appropriate.
> Currently we have the jobs
> [here|https://builds.apache.org/view/S-Z/view/Thrift/]:
> {noformat}
> $ curl --netrc -sS https://builds.apache.org/view/S-Z/view/Thrift/config.xml
> | xmllint -xpath "//jobNames" - | grep "" | cut -d'>' -f2 | cut -d'<'
> -f1
> Thrift
> Thrift-all
> Thrift-Compiler-Linux64
> Thrift-Compiler-Windows
> Thrift-env
> Thrift-env-Windows
> Thrift-Test
> Thrift-Windows
> {noformat}
> Of those, [Thrift-env|https://builds.apache.org/job/Thrift-env],
> [Thrift-env-Windows|https://builds.apache.org/job/Thrift-env-Windows] and
> [Thrift-Windows|https://builds.apache.org/job/Thrift-Windows] are all
> disabled and [Thrift|https://builds.apache.org/job/Thrift] and
> [Thrift-Compiler-Linux64|https://builds.apache.org/job/Thrift-Compiler-Linux64]
> seem to be unused based on last run dates. This leaves the following active
> jobs fwict:
> * [|https://builds.apache.org/job/Thrift-all]
> * [|https://builds.apache.org/job/Thrift-Compiler-Windows]
> * [|https://builds.apache.org/job/Thrift-precommit]
> Of these {{Thrift-all}} is broken in a known-way, it really needs to be
> converted to use a Dockerized build like {{Thrift-precommit}} to be insulated
> from the vagaries of the underlying worker machines.
> I propose deltion of all disabled (3) and inactive (2) jobs listed above and
> conversion of {{Thrift-all}} to a Dockerized CI job to guard master with.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (THRIFT-3621) Fix lib/cpp/test/SecurityTest.cpp to use ephemeral ports
John Sirois created THRIFT-3621: --- Summary: Fix lib/cpp/test/SecurityTest.cpp to use ephemeral ports Key: THRIFT-3621 URL: https://issues.apache.org/jira/browse/THRIFT-3621 Project: Thrift Issue Type: Sub-task Components: C++ - Library, Test Suite Reporter: John Sirois Assignee: John Sirois As part of work on THRIFT-3609, eliminate use of TestPortFixture.h from SecurityTest, -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3621) Fix lib/cpp/test/SecurityTest.cpp to use ephemeral ports
[ https://issues.apache.org/jira/browse/THRIFT-3621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145529#comment-15145529 ] John Sirois commented on THRIFT-3621: - https://github.com/apache/thrift/pull/853 > Fix lib/cpp/test/SecurityTest.cpp to use ephemeral ports > > > Key: THRIFT-3621 > URL: https://issues.apache.org/jira/browse/THRIFT-3621 > Project: Thrift > Issue Type: Sub-task > Components: C++ - Library, Test Suite >Reporter: John Sirois >Assignee: John Sirois > > As part of work on THRIFT-3609, eliminate use of TestPortFixture.h from > SecurityTest, -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (THRIFT-3622) Fix deprecated uses of std::auto_ptr
John Sirois created THRIFT-3622:
---
Summary: Fix deprecated uses of std::auto_ptr
Key: THRIFT-3622
URL: https://issues.apache.org/jira/browse/THRIFT-3622
Project: Thrift
Issue Type: Task
Components: C++ - Library
Reporter: John Sirois
We have some in lib/cpp, like so:
{noformat}
JSONProtoTest.cpp:35:13: warning: ‘template class std::auto_ptr’ is
deprecated [-Wdeprecated-declarations]
static std::auto_ptr ooe;
^
In file included from /usr/include/c++/5.3.0/bits/locale_conv.h:41:0,
from /usr/include/c++/5.3.0/locale:43,
from /usr/include/c++/5.3.0/iomanip:43,
from JSONProtoTest.cpp:22:
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (THRIFT-3623) Fix Fix cpp/lib/test/TSSLSocketInterruptTest.cpp to use ephemeral ports
John Sirois created THRIFT-3623: --- Summary: Fix Fix cpp/lib/test/TSSLSocketInterruptTest.cpp to use ephemeral ports Key: THRIFT-3623 URL: https://issues.apache.org/jira/browse/THRIFT-3623 Project: Thrift Issue Type: Sub-task Components: C++ - Library, Test Suite Reporter: John Sirois Assignee: John Sirois As part of work on THRIFT-3609, eliminate use of TestPortFixture.h from TSSLSocketInterruptTest.cpp. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3623) Fix Fix cpp/lib/test/TSSLSocketInterruptTest.cpp to use ephemeral ports
[ https://issues.apache.org/jira/browse/THRIFT-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145634#comment-15145634 ] John Sirois commented on THRIFT-3623: - https://github.com/apache/thrift/pull/854 > Fix Fix cpp/lib/test/TSSLSocketInterruptTest.cpp to use ephemeral ports > --- > > Key: THRIFT-3623 > URL: https://issues.apache.org/jira/browse/THRIFT-3623 > Project: Thrift > Issue Type: Sub-task > Components: C++ - Library, Test Suite >Reporter: John Sirois >Assignee: John Sirois > > As part of work on THRIFT-3609, eliminate use of TestPortFixture.h from > TSSLSocketInterruptTest.cpp. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (THRIFT-3624) Fix lib/cpp/test/TServerSocketTest.cpp to use ephemeral ports
John Sirois created THRIFT-3624: --- Summary: Fix lib/cpp/test/TServerSocketTest.cpp to use ephemeral ports Key: THRIFT-3624 URL: https://issues.apache.org/jira/browse/THRIFT-3624 Project: Thrift Issue Type: Sub-task Components: C++ - Library, Test Suite Reporter: John Sirois Assignee: John Sirois As part of work on THRIFT-3609, eliminate use of TestPortFixture.h from TServerSocketTest. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (THRIFT-3621) Fix lib/cpp/test/SecurityTest.cpp to use ephemeral ports
[ https://issues.apache.org/jira/browse/THRIFT-3621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sirois updated THRIFT-3621: Description: As part of work on THRIFT-3609, eliminate use of TestPortFixture.h from SecurityTest. (was: As part of work on THRIFT-3609, eliminate use of TestPortFixture.h from SecurityTest,) > Fix lib/cpp/test/SecurityTest.cpp to use ephemeral ports > > > Key: THRIFT-3621 > URL: https://issues.apache.org/jira/browse/THRIFT-3621 > Project: Thrift > Issue Type: Sub-task > Components: C++ - Library, Test Suite >Reporter: John Sirois >Assignee: John Sirois > > As part of work on THRIFT-3609, eliminate use of TestPortFixture.h from > SecurityTest. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3624) Fix lib/cpp/test/TServerSocketTest.cpp to use ephemeral ports
[ https://issues.apache.org/jira/browse/THRIFT-3624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145645#comment-15145645 ] John Sirois commented on THRIFT-3624: - https://github.com/apache/thrift/pull/855 > Fix lib/cpp/test/TServerSocketTest.cpp to use ephemeral ports > - > > Key: THRIFT-3624 > URL: https://issues.apache.org/jira/browse/THRIFT-3624 > Project: Thrift > Issue Type: Sub-task > Components: C++ - Library, Test Suite >Reporter: John Sirois >Assignee: John Sirois > > As part of work on THRIFT-3609, eliminate use of TestPortFixture.h from > TServerSocketTest. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (THRIFT-3625) Kill unused #include "TestPortFixture.h" in lib/cpp/test/TServerTransportTest.cpp.
John Sirois created THRIFT-3625: --- Summary: Kill unused #include "TestPortFixture.h" in lib/cpp/test/TServerTransportTest.cpp. Key: THRIFT-3625 URL: https://issues.apache.org/jira/browse/THRIFT-3625 Project: Thrift Issue Type: Sub-task Components: C++ - Library, Test Suite Reporter: John Sirois Assignee: John Sirois As part of work on THRIFT-3609, eliminate use of TestPortFixture.h from TServerTransportTest. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3625) Kill unused #include "TestPortFixture.h" in lib/cpp/test/TServerTransportTest.cpp.
[ https://issues.apache.org/jira/browse/THRIFT-3625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145667#comment-15145667 ] John Sirois commented on THRIFT-3625: - https://github.com/apache/thrift/pull/856 > Kill unused #include "TestPortFixture.h" in > lib/cpp/test/TServerTransportTest.cpp. > -- > > Key: THRIFT-3625 > URL: https://issues.apache.org/jira/browse/THRIFT-3625 > Project: Thrift > Issue Type: Sub-task > Components: C++ - Library, Test Suite >Reporter: John Sirois >Assignee: John Sirois > > As part of work on THRIFT-3609, eliminate use of TestPortFixture.h from > TServerTransportTest. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (THRIFT-3626) Fix lib/cpp/test/TSocketInterruptTest.cpp to use ephemeral ports.
John Sirois created THRIFT-3626: --- Summary: Fix lib/cpp/test/TSocketInterruptTest.cpp to use ephemeral ports. Key: THRIFT-3626 URL: https://issues.apache.org/jira/browse/THRIFT-3626 Project: Thrift Issue Type: Sub-task Components: C++ - Library, Test Suite Reporter: John Sirois Assignee: John Sirois As part of work on THRIFT-3609, eliminate use of TestPortFixture.h from TSocketInterruptTest.cpp. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3626) Fix lib/cpp/test/TSocketInterruptTest.cpp to use ephemeral ports.
[ https://issues.apache.org/jira/browse/THRIFT-3626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145781#comment-15145781 ] John Sirois commented on THRIFT-3626: - https://github.com/apache/thrift/pull/857 > Fix lib/cpp/test/TSocketInterruptTest.cpp to use ephemeral ports. > - > > Key: THRIFT-3626 > URL: https://issues.apache.org/jira/browse/THRIFT-3626 > Project: Thrift > Issue Type: Sub-task > Components: C++ - Library, Test Suite >Reporter: John Sirois >Assignee: John Sirois > > As part of work on THRIFT-3609, eliminate use of TestPortFixture.h from > TSocketInterruptTest.cpp. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3620) Cleanup and consolidate Thrift* jenkins jobs.
[
https://issues.apache.org/jira/browse/THRIFT-3620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145795#comment-15145795
]
John Sirois commented on THRIFT-3620:
-
I'm agree 100% on the ability for any contributor to enhance CI:
{noformat}
$ curl --netrc -sS https://builds.apache.org/job/Thrift-precommit/config.xml |
xmllint -xpath //command -
docker build -t thrift $WORKSPACE/build/docker/ubuntu
cat << DOCKERFILE | docker build -t jenkins-thrift -
# A thrift image that adds jenkins job specific tools
FROM thrift
RUN apt-get update && apt-get install -y \
cppcheck \
sloccount
DOCKERFILE
# TODO(John Sirois): Check the ci script into the repo and
have that be the docker entrypoint.
docker run --rm -v $WORKSPACE:/thrift -w /thrift -t jenkins-thrift /bin/bash
-exc '
sh bootstrap.sh
sloccount --duplicates --wide --details lib tutorial test > sloccount.sc
cppcheck --error-exitcode=0 --force --xml lib/cpp/src/ lib/c_glib/src/
tutorial/cpp/ test/cpp/ 2> cppcheck-result.xml
'# TODO(John Sirois): Check the ci script into the repo and
have that be the docker entrypoint.
# NB: Hacks used:
# 1.) --privileged is a hammer being used to get around issues like
intermittent Haxe compile errors of the form:
# ...
# -I/usr/lib/haxe/lib/hxcpp/3,2,205/include -x c++ -frtti
-Wno-invalid-offsetof ./src/thrift/test/TestMapMap_args.cpp
-o/thrift/test/haxe/bin/obj/linux64-debug/9931a53a_TestMapMap_args.o
# Error: /usr/lib/gcc/x86_64-linux-gnu/4.8/cc1plus: error while loading
shared libraries: libcloog-isl.so.4: failed to map segment from shared object:
Permission denied
# Error: Build failed
docker run --privileged --rm -v $WORKSPACE:/thrift -w /thrift -t jenkins-thrift
/bin/bash -exc '
sh bootstrap.sh
./configure
make check dist
ant -f lib/java/build.xml javadoc
'# TODO(John Sirois): Check the ci script into the repo and
have that be the docker entrypoint.
# This is disabled due to a large number of failures in the matrix presently
(2/9/2016).
# TODO(John Sirois): Fix or remove.
#docker run --rm -v $WORKSPACE:/thrift -w /thrift -t jenkins-thrift /bin/bash
-exc '
#GOPATH=/thrift/test/go go install bin/...
#./test/test.py | tee test.log
#'
{noformat}
> Cleanup and consolidate Thrift* jenkins jobs.
> -
>
> Key: THRIFT-3620
> URL: https://issues.apache.org/jira/browse/THRIFT-3620
> Project: Thrift
> Issue Type: Task
> Components: Test Suite
>Reporter: John Sirois
>Assignee: John Sirois
>
> I spent some time resurrecting the
> [Thrift-precommit|https://builds.apache.org/job/Thrift-precommit/] job which
> vets pull-requests to use the pool of docker slaves using a dockerized CI
> with guidance from [~jfarrell]. The remaining crop of Thrift* jobs should be
> culled and converted as appropriate.
> Currently we have the jobs
> [here|https://builds.apache.org/view/S-Z/view/Thrift/]:
> {noformat}
> $ curl --netrc -sS https://builds.apache.org/view/S-Z/view/Thrift/config.xml
> | xmllint -xpath "//jobNames" - | grep "" | cut -d'>' -f2 | cut -d'<'
> -f1
> Thrift
> Thrift-all
> Thrift-Compiler-Linux64
> Thrift-Compiler-Windows
> Thrift-env
> Thrift-env-Windows
> Thrift-Test
> Thrift-Windows
> {noformat}
> Of those, [Thrift-env|https://builds.apache.org/job/Thrift-env],
> [Thrift-env-Windows|https://builds.apache.org/job/Thrift-env-Windows] and
> [Thrift-Windows|https://builds.apache.org/job/Thrift-Windows] are all
> disabled and [Thrift|https://builds.apache.org/job/Thrift] and
> [Thrift-Compiler-Linux64|https://builds.apache.org/job/Thrift-Compiler-Linux64]
> seem to be unused based on last run dates. This leaves the following active
> jobs fwict:
> * [|https://builds.apache.org/job/Thrift-all]
> * [|https://builds.apache.org/job/Thrift-Compiler-Windows]
> * [|https://builds.apache.org/job/Thrift-precommit]
> Of these {{Thrift-all}} is broken in a known-way, it really needs to be
> converted to use a Dockerized build like {{Thrift-precommit}} to be insulated
> from the vagaries of the underlying worker machines.
> I propose deltion of all disabled (3) and inactive (2) jobs listed above and
> conversion of {{Thrift-all}} to a Dockerized CI job to guard master with.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (THRIFT-3620) Cleanup and consolidate Thrift* jenkins jobs.
[
https://issues.apache.org/jira/browse/THRIFT-3620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145795#comment-15145795
]
John Sirois edited comment on THRIFT-3620 at 2/13/16 4:41 AM:
--
I agree 100% on the ability for any contributor to enhance CI:
{noformat}
$ curl --netrc -sS https://builds.apache.org/job/Thrift-precommit/config.xml |
xmllint -xpath //command -
docker build -t thrift $WORKSPACE/build/docker/ubuntu
cat << DOCKERFILE | docker build -t jenkins-thrift -
# A thrift image that adds jenkins job specific tools
FROM thrift
RUN apt-get update && apt-get install -y \
cppcheck \
sloccount
DOCKERFILE
# TODO(John Sirois): Check the ci script into the repo and
have that be the docker entrypoint.
docker run --rm -v $WORKSPACE:/thrift -w /thrift -t jenkins-thrift /bin/bash
-exc '
sh bootstrap.sh
sloccount --duplicates --wide --details lib tutorial test > sloccount.sc
cppcheck --error-exitcode=0 --force --xml lib/cpp/src/ lib/c_glib/src/
tutorial/cpp/ test/cpp/ 2> cppcheck-result.xml
'# TODO(John Sirois): Check the ci script into the repo and
have that be the docker entrypoint.
# NB: Hacks used:
# 1.) --privileged is a hammer being used to get around issues like
intermittent Haxe compile errors of the form:
# ...
# -I/usr/lib/haxe/lib/hxcpp/3,2,205/include -x c++ -frtti
-Wno-invalid-offsetof ./src/thrift/test/TestMapMap_args.cpp
-o/thrift/test/haxe/bin/obj/linux64-debug/9931a53a_TestMapMap_args.o
# Error: /usr/lib/gcc/x86_64-linux-gnu/4.8/cc1plus: error while loading
shared libraries: libcloog-isl.so.4: failed to map segment from shared object:
Permission denied
# Error: Build failed
docker run --privileged --rm -v $WORKSPACE:/thrift -w /thrift -t jenkins-thrift
/bin/bash -exc '
sh bootstrap.sh
./configure
make check dist
ant -f lib/java/build.xml javadoc
'# TODO(John Sirois): Check the ci script into the repo and
have that be the docker entrypoint.
# This is disabled due to a large number of failures in the matrix presently
(2/9/2016).
# TODO(John Sirois): Fix or remove.
#docker run --rm -v $WORKSPACE:/thrift -w /thrift -t jenkins-thrift /bin/bash
-exc '
#GOPATH=/thrift/test/go go install bin/...
#./test/test.py | tee test.log
#'
{noformat}
was (Author: jsirois):
I'm agree 100% on the ability for any contributor to enhance CI:
{noformat}
$ curl --netrc -sS https://builds.apache.org/job/Thrift-precommit/config.xml |
xmllint -xpath //command -
docker build -t thrift $WORKSPACE/build/docker/ubuntu
cat << DOCKERFILE | docker build -t jenkins-thrift -
# A thrift image that adds jenkins job specific tools
FROM thrift
RUN apt-get update && apt-get install -y \
cppcheck \
sloccount
DOCKERFILE
# TODO(John Sirois): Check the ci script into the repo and
have that be the docker entrypoint.
docker run --rm -v $WORKSPACE:/thrift -w /thrift -t jenkins-thrift /bin/bash
-exc '
sh bootstrap.sh
sloccount --duplicates --wide --details lib tutorial test > sloccount.sc
cppcheck --error-exitcode=0 --force --xml lib/cpp/src/ lib/c_glib/src/
tutorial/cpp/ test/cpp/ 2> cppcheck-result.xml
'# TODO(John Sirois): Check the ci script into the repo and
have that be the docker entrypoint.
# NB: Hacks used:
# 1.) --privileged is a hammer being used to get around issues like
intermittent Haxe compile errors of the form:
# ...
# -I/usr/lib/haxe/lib/hxcpp/3,2,205/include -x c++ -frtti
-Wno-invalid-offsetof ./src/thrift/test/TestMapMap_args.cpp
-o/thrift/test/haxe/bin/obj/linux64-debug/9931a53a_TestMapMap_args.o
# Error: /usr/lib/gcc/x86_64-linux-gnu/4.8/cc1plus: error while loading
shared libraries: libcloog-isl.so.4: failed to map segment from shared object:
Permission denied
# Error: Build failed
docker run --privileged --rm -v $WORKSPACE:/thrift -w /thrift -t jenkins-thrift
/bin/bash -exc '
sh bootstrap.sh
./configure
make check dist
ant -f lib/java/build.xml javadoc
'# TODO(John Sirois): Check the ci script into the repo and
have that be the docker entrypoint.
# This is disabled due to a large number of failures in the matrix presently
(2/9/2016).
# TODO(John Sirois): Fix or remove.
#docker run --rm -v $WORKSPACE:/thrift -w /thrift -t jenkins-thrift /bin/bash
-exc '
#GOPATH=/thrift/test/go go install bin/...
#./test/test.py | tee test.log
#'
{noformat}
> Cleanup and consolidate Thrift* jenkins jobs.
> -
>
> Key: THRIFT-3620
> URL: https://issues.apache.org/jira/browse/THRIFT-3620
> Project: Thrift
> Issue Type: Task
> Components: Test Suite
>Reporter: John Sirois
>Assignee: John Sirois
>
> I spent some time resurrecting the
> [Thrift-precommit|https://builds.apache.org/job/Thrift-precommit/] job which
> vets pull-requests to use the pool of docker slaves using a dockerized CI
> with guidance from [~jfarrel
[jira] [Commented] (THRIFT-3623) Fix Fix cpp/lib/test/TSSLSocketInterruptTest.cpp to use ephemeral ports
[ https://issues.apache.org/jira/browse/THRIFT-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145811#comment-15145811 ] John Sirois commented on THRIFT-3623: - Build went green @ https://builds.apache.org/job/Thrift-precommit/105/ I had the title issue ID messed up. > Fix Fix cpp/lib/test/TSSLSocketInterruptTest.cpp to use ephemeral ports > --- > > Key: THRIFT-3623 > URL: https://issues.apache.org/jira/browse/THRIFT-3623 > Project: Thrift > Issue Type: Sub-task > Components: C++ - Library, Test Suite >Reporter: John Sirois >Assignee: John Sirois > > As part of work on THRIFT-3609, eliminate use of TestPortFixture.h from > TSSLSocketInterruptTest.cpp. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3626) Fix lib/cpp/test/TSocketInterruptTest.cpp to use ephemeral ports.
[ https://issues.apache.org/jira/browse/THRIFT-3626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145825#comment-15145825 ] John Sirois commented on THRIFT-3626: - Now green @ https://builds.apache.org/job/Thrift-precommit/113/ > Fix lib/cpp/test/TSocketInterruptTest.cpp to use ephemeral ports. > - > > Key: THRIFT-3626 > URL: https://issues.apache.org/jira/browse/THRIFT-3626 > Project: Thrift > Issue Type: Sub-task > Components: C++ - Library, Test Suite >Reporter: John Sirois >Assignee: John Sirois > > As part of work on THRIFT-3609, eliminate use of TestPortFixture.h from > TSocketInterruptTest.cpp. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3625) Kill unused #include "TestPortFixture.h" in lib/cpp/test/TServerTransportTest.cpp.
[ https://issues.apache.org/jira/browse/THRIFT-3625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145828#comment-15145828 ] John Sirois commented on THRIFT-3625: - Now green @ https://builds.apache.org/job/Thrift-precommit/114/ > Kill unused #include "TestPortFixture.h" in > lib/cpp/test/TServerTransportTest.cpp. > -- > > Key: THRIFT-3625 > URL: https://issues.apache.org/jira/browse/THRIFT-3625 > Project: Thrift > Issue Type: Sub-task > Components: C++ - Library, Test Suite >Reporter: John Sirois >Assignee: John Sirois > > As part of work on THRIFT-3609, eliminate use of TestPortFixture.h from > TServerTransportTest. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (THRIFT-3628) Fix lib/cpp/test/TServerIntegrationTest.cpp to use ephemeral ports
John Sirois created THRIFT-3628: --- Summary: Fix lib/cpp/test/TServerIntegrationTest.cpp to use ephemeral ports Key: THRIFT-3628 URL: https://issues.apache.org/jira/browse/THRIFT-3628 Project: Thrift Issue Type: Sub-task Components: C++ - Library, Test Suite Reporter: John Sirois Assignee: John Sirois As part of work on THRIFT-3609, eliminate use of TestPortFixture.h from TServerIntegrationTest.cpp. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (THRIFT-3623) Fix Fix cpp/lib/test/TSSLSocketInterruptTest.cpp to use ephemeral ports
[ https://issues.apache.org/jira/browse/THRIFT-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15146172#comment-15146172 ] John Sirois commented on THRIFT-3623: - CI went green here: https://builds.apache.org/job/Thrift-precommit/125/ > Fix Fix cpp/lib/test/TSSLSocketInterruptTest.cpp to use ephemeral ports > --- > > Key: THRIFT-3623 > URL: https://issues.apache.org/jira/browse/THRIFT-3623 > Project: Thrift > Issue Type: Sub-task > Components: C++ - Library, Test Suite >Reporter: John Sirois >Assignee: John Sirois > > As part of work on THRIFT-3609, eliminate use of TestPortFixture.h from > TSSLSocketInterruptTest.cpp. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (THRIFT-3622) Fix deprecated uses of std::auto_ptr
[
https://issues.apache.org/jira/browse/THRIFT-3622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Sirois reassigned THRIFT-3622:
---
Assignee: John Sirois
> Fix deprecated uses of std::auto_ptr
>
>
> Key: THRIFT-3622
> URL: https://issues.apache.org/jira/browse/THRIFT-3622
> Project: Thrift
> Issue Type: Task
> Components: C++ - Library
>Reporter: John Sirois
>Assignee: John Sirois
>
> We have some in lib/cpp, like so:
> {noformat}
> JSONProtoTest.cpp:35:13: warning: ‘template class std::auto_ptr’ is
> deprecated [-Wdeprecated-declarations]
> static std::auto_ptr ooe;
> ^
> In file included from /usr/include/c++/5.3.0/bits/locale_conv.h:41:0,
> from /usr/include/c++/5.3.0/locale:43,
> from /usr/include/c++/5.3.0/iomanip:43,
> from JSONProtoTest.cpp:22:
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (THRIFT-3622) Fix deprecated uses of std::auto_ptr
[
https://issues.apache.org/jira/browse/THRIFT-3622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15146218#comment-15146218
]
John Sirois commented on THRIFT-3622:
-
CI went green here: https://builds.apache.org/job/Thrift-precommit/129/
> Fix deprecated uses of std::auto_ptr
>
>
> Key: THRIFT-3622
> URL: https://issues.apache.org/jira/browse/THRIFT-3622
> Project: Thrift
> Issue Type: Task
> Components: C++ - Library
>Reporter: John Sirois
>Assignee: John Sirois
>
> We have some in lib/cpp, like so:
> {noformat}
> JSONProtoTest.cpp:35:13: warning: ‘template class std::auto_ptr’ is
> deprecated [-Wdeprecated-declarations]
> static std::auto_ptr ooe;
> ^
> In file included from /usr/include/c++/5.3.0/bits/locale_conv.h:41:0,
> from /usr/include/c++/5.3.0/locale:43,
> from /usr/include/c++/5.3.0/iomanip:43,
> from JSONProtoTest.cpp:22:
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (THRIFT-3609) Remove or replace TestPortFixture.h
[
https://issues.apache.org/jira/browse/THRIFT-3609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15146588#comment-15146588
]
John Sirois commented on THRIFT-3609:
-
Removal: https://github.com/apache/thrift/pull/869
> Remove or replace TestPortFixture.h
> ---
>
> Key: THRIFT-3609
> URL: https://issues.apache.org/jira/browse/THRIFT-3609
> Project: Thrift
> Issue Type: Improvement
> Components: C++ - Library, Test Suite
>Reporter: John Sirois
>Assignee: John Sirois
>
> Today 6 cpp lib tests use
> [TestPortFixture.h|https://github.com/apache/thrift/blob/1f6e380c5d07686e4cd8c2b172300a1ba7fbd8b9/lib/cpp/test/TestPortFixture.h].
> THRIFT-3608 experimented with correcting one of these cases in
> {{lib/cpp/test/SecurityTest.cpp}}, but a general cleanup should remove all
> uses of static ports in the cpp tests to improve robustness and open the path
> to parallel testing - at least in the cpp libs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
