[jira] [Closed] (THRIFT-4197) Implement transparent gzip compression for HTTP transport
[ https://issues.apache.org/jira/browse/THRIFT-4197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Can Celasun closed THRIFT-4197. --- > Implement transparent gzip compression for HTTP transport > - > > Key: THRIFT-4197 > URL: https://issues.apache.org/jira/browse/THRIFT-4197 > Project: Thrift > Issue Type: Improvement > Components: Go - Library >Affects Versions: 0.10.0 >Reporter: Can Celasun >Assignee: Can Celasun >Priority: Minor > Fix For: 0.11.0 > > > HTTP transport should check if the client supports gzip compressed responses > and do transparent compression if it does. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (THRIFT-4197) Implement transparent gzip compression for HTTP transport
[ https://issues.apache.org/jira/browse/THRIFT-4197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16009347#comment-16009347 ] ASF GitHub Bot commented on THRIFT-4197: Github user dcelasun commented on the issue: https://github.com/apache/thrift/pull/1266 Yes I did, works just fine. > Implement transparent gzip compression for HTTP transport > - > > Key: THRIFT-4197 > URL: https://issues.apache.org/jira/browse/THRIFT-4197 > Project: Thrift > Issue Type: Improvement > Components: Go - Library >Affects Versions: 0.10.0 >Reporter: Can Celasun >Assignee: Can Celasun >Priority: Minor > Fix For: 0.11.0 > > > HTTP transport should check if the client supports gzip compressed responses > and do transparent compression if it does. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] thrift issue #1266: THRIFT-4197 [Go] Transparent gzip support for HTTP trans...
Github user dcelasun commented on the issue: https://github.com/apache/thrift/pull/1266 Yes I did, works just fine. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (THRIFT-4197) Implement transparent gzip compression for HTTP transport
[ https://issues.apache.org/jira/browse/THRIFT-4197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16009345#comment-16009345 ] ASF GitHub Bot commented on THRIFT-4197: Github user mrwiggles commented on the issue: https://github.com/apache/thrift/pull/1266 Seems very straightforward. Were you able to test it out? > Implement transparent gzip compression for HTTP transport > - > > Key: THRIFT-4197 > URL: https://issues.apache.org/jira/browse/THRIFT-4197 > Project: Thrift > Issue Type: Improvement > Components: Go - Library >Affects Versions: 0.10.0 >Reporter: Can Celasun >Assignee: Can Celasun >Priority: Minor > Fix For: 0.11.0 > > > HTTP transport should check if the client supports gzip compressed responses > and do transparent compression if it does. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (THRIFT-4195) Compilation to GO produces broken code
[ https://issues.apache.org/jira/browse/THRIFT-4195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16009343#comment-16009343 ] Allen George commented on THRIFT-4195: -- [~jensg] [~jking3] So, not sure what the correct approach is here. Tested this out locally, and it seems like the code in master does the appropriate thing. > Compilation to GO produces broken code > -- > > Key: THRIFT-4195 > URL: https://issues.apache.org/jira/browse/THRIFT-4195 > Project: Thrift > Issue Type: Bug > Components: Go - Compiler >Affects Versions: 0.10.0 > Environment: MacOS 10.12.4 (16E195) > go version go1.8.1 darwin/amd64 >Reporter: Stanislav Baranov >Priority: Critical > Attachments: GoUnusedProtection__.go, sample-consts.go, sample.go, > sample.thrift > > > Sample source file (see attached sample.thrift): > {code} > namespace go reports > typedef string ReportState > const ReportState ReportStateQueued = "QUEUED"; > const ReportState ReportStateCalculating = "CALCULATING"; > const ReportState ReportStateImporting = "IMPORTING"; > const ReportState ReportStateFinished = "FINISHED"; > const ReportState ReportStateFailed = "FAILED"; > struct ReportFilter { > 1: i64 quantity, > 2: i64 dataVersion, > 3: string subscriptionID, > 4: optional string principalID, > 5: optional string createdOn, > 6: optional list states, > } > {code} > after compilation as: > {{thrift -r -v -o ./ -out ./go --gen go:package_prefix="git.4tree.de/" > ./sample.thrift}} > Produces code (see attached sample.go) which contains: > {code} > func (p *ReportFilter) ReadField6(iprot thrift.TProtocol) error { > _, size, err := iprot.ReadListBegin() > if err != nil { > return thrift.PrependError("error reading list begin: ", err) > } > tSlice := make([]ReportState, 0, size) > p.States = tSlice > for i := 0; i < size; i ++ { > var _elem0 string > if v, err := iprot.ReadString(); err != nil { > return thrift.PrependError("error reading field 0: ", err) > } else { > _elem0 = v > } > p.States = append(p.States, _elem0) > } > if err := iprot.ReadListEnd(); err != nil { > return thrift.PrependError("error reading list end: ", err) > } > return nil > } > {code} > When trying to compile this code, GO triggers an error: > {code} > cannot use _elem0 (type string) as type ReportState in append > {code} > at line: > {code} > p.States = append(p.States, _elem0) > {code} > This code compiles normal using thrift 0.9.3 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (THRIFT-4188) Error while flushing write buffer of size 70 to transport, only wrote 0 bytes: write tcp 127.0.0.1:5000->127.0.0.1:58174: write: broken pipe
[ https://issues.apache.org/jira/browse/THRIFT-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16009339#comment-16009339 ] Jens Geyer edited comment on THRIFT-4188 at 5/13/17 1:13 PM: - I just tried to reproduce and I could. Although it's really weird: One run produces about 1400 errors (5000 calls), the next run none. The third run then 1100, and so on. Putting a print command before {{client.Ping()}} brings it down to about 300 errors, probably due to the reduced load on the connection caused by the additional print command. That makes me think that there might be some limited resource involved and some timeout at some place that maybe is a bit too agressive. Thoughts? was (Author: jensg): I just tried to reproduce and I could. Although it's really weird: One run produces about 1400 errors (5000 calls), the next run none. The third run then 1100, and so on. Putting a print command before {{client.Ping()}} brings it down to about 300 errors. Taht makes me guess that there is some limited resource involved and some timeout at some place that maybe is a bit too agressive. Thoughts? > Error while flushing write buffer of size 70 to transport, only wrote 0 > bytes: write tcp 127.0.0.1:5000->127.0.0.1:58174: write: broken pipe > > > Key: THRIFT-4188 > URL: https://issues.apache.org/jira/browse/THRIFT-4188 > Project: Thrift > Issue Type: Bug > Components: Go - Library >Affects Versions: 0.10.0 >Reporter: brian.kang > Attachments: client.go, handler.go, main.go, main.go, server.go, > shared.thrift, tutorial.thrift > > > {code} > if *server { > if err := runServer(transportFactory, protocolFactory, *addr, > *secure); err != nil { > fmt.Println("error running server:", err) > } > } else { > done := make(chan int32) > for i := 0; i < 5000; i++ { > go func(i int) { > err, s := runClient(transportFactory, > protocolFactory, *addr, *secure, int32(i)) > if err != nil { > fmt.Println("error running client:", > err) > } > done <- s > }(i) > } > for i := 0; i < 5000; i++ { > fmt.Println(<-done) > } > } > {code} > too many errors,eg: > {code} > Error while flushing write buffer of size 17 to transport, only wrote > 02017/05/04 13:29:00 error processing request: write tcp > 127.0.0.1:9090->127.0.0.1:58932: write: broken pipe > bytes: write tcp 127.0.0.1:9090->127.0.0.1:58915: write: broken pipe > 2017/05/04 13:29:00 error processing request: write tcp > 127.0.0.1:9090->127.0.0.1:58922: write: broken pipe > Error while flushing write buffer of size 2017/05/04 13:29:00 error > processing request: write tcp 127.0.0.1:9090->127.0.0.1:58928: write: broken > pipe > 17 to transport, only wrote 0 bytes: write tcp > 127.0.0.1:9090->127.0.0.1:58923: write: broken pipe2017/05/04 13:29:00 error > processing request: write tcp 127.0.0.1:9090->127.0.0.1:58915: write: broken > pipe > Error while flushing write buffer of size 17 to transport, only wrote 0 > bytes: write tcp 127.0.0.1:9090->127.0.0.1:58930: write: broken pipe > 2017/05/04 13:29:00 error processing request: write tcp > 127.0.0.1:9090->127.0.0.1:58923: write: broken pipe > Error while flushing write buffer of size 17 to transport, only wrote 0 > bytes: write tcp 127.0.0.1:9090->127.0.0.1:58926: write: broken pipe > 2017/05/04 13:29:00 error processing request: write tcp > 127.0.0.1:9090->127.0.0.1:58930: write: broken pipe > Error while flushing write buffer of size 2017/05/04 13:29:00 error > processing request: write tcp 127.0.0.1:9090->127.0.0.1:58923: write: broken > pipe > 17 to transport, only wrote 0 bytes: write tcp > 127.0.0.1:9090->127.0.0.1:58920: write: broken pipe > 2017/05/04 13:29:00 error processing request: write tcp > 127.0.0.1:9090->127.0.0.1:58932: write: broken pipe > 2017/05/04 13:29:00 error processing request: write tcp > 127.0.0.1:9090->127.0.0.1:58930: write: broken pipe > Error while flushing write buffer of size 17 to transport, only wrote > 2017/05/04 13:29:00 error processing request: write tcp > 127.0.0.1:9090->127.0.0.1:58926: write: broken pipe > 0 bytes: write tcp 127.0.0.1:9090->127.0.0.1:58912: write: broken pipe > Error while flushing write buffer of size 172017/05/04 13:29:00 error > processing request: write tcp 127.0.0.1:9090->127.0.0.1:58912: write: broken > pipe > to transport, only wrote 0 bytes: write tcp
[jira] [Commented] (THRIFT-4188) Error while flushing write buffer of size 70 to transport, only wrote 0 bytes: write tcp 127.0.0.1:5000->127.0.0.1:58174: write: broken pipe
[ https://issues.apache.org/jira/browse/THRIFT-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16009339#comment-16009339 ] Jens Geyer commented on THRIFT-4188: I just tried to reproduce and I could. Although it's really weird: One run produces about 1400 errors (5000 calls), the next run none. The third run then 1100, and so on. Putting a print command before {{client.Ping()}} brings it down to about 300 errors. Taht makes me guess that there is some limited resource involved and some timeout at some place that maybe is a bit too agressive. Thoughts? > Error while flushing write buffer of size 70 to transport, only wrote 0 > bytes: write tcp 127.0.0.1:5000->127.0.0.1:58174: write: broken pipe > > > Key: THRIFT-4188 > URL: https://issues.apache.org/jira/browse/THRIFT-4188 > Project: Thrift > Issue Type: Bug > Components: Go - Library >Affects Versions: 0.10.0 >Reporter: brian.kang > Attachments: client.go, handler.go, main.go, main.go, server.go, > shared.thrift, tutorial.thrift > > > {code} > if *server { > if err := runServer(transportFactory, protocolFactory, *addr, > *secure); err != nil { > fmt.Println("error running server:", err) > } > } else { > done := make(chan int32) > for i := 0; i < 5000; i++ { > go func(i int) { > err, s := runClient(transportFactory, > protocolFactory, *addr, *secure, int32(i)) > if err != nil { > fmt.Println("error running client:", > err) > } > done <- s > }(i) > } > for i := 0; i < 5000; i++ { > fmt.Println(<-done) > } > } > {code} > too many errors,eg: > {code} > Error while flushing write buffer of size 17 to transport, only wrote > 02017/05/04 13:29:00 error processing request: write tcp > 127.0.0.1:9090->127.0.0.1:58932: write: broken pipe > bytes: write tcp 127.0.0.1:9090->127.0.0.1:58915: write: broken pipe > 2017/05/04 13:29:00 error processing request: write tcp > 127.0.0.1:9090->127.0.0.1:58922: write: broken pipe > Error while flushing write buffer of size 2017/05/04 13:29:00 error > processing request: write tcp 127.0.0.1:9090->127.0.0.1:58928: write: broken > pipe > 17 to transport, only wrote 0 bytes: write tcp > 127.0.0.1:9090->127.0.0.1:58923: write: broken pipe2017/05/04 13:29:00 error > processing request: write tcp 127.0.0.1:9090->127.0.0.1:58915: write: broken > pipe > Error while flushing write buffer of size 17 to transport, only wrote 0 > bytes: write tcp 127.0.0.1:9090->127.0.0.1:58930: write: broken pipe > 2017/05/04 13:29:00 error processing request: write tcp > 127.0.0.1:9090->127.0.0.1:58923: write: broken pipe > Error while flushing write buffer of size 17 to transport, only wrote 0 > bytes: write tcp 127.0.0.1:9090->127.0.0.1:58926: write: broken pipe > 2017/05/04 13:29:00 error processing request: write tcp > 127.0.0.1:9090->127.0.0.1:58930: write: broken pipe > Error while flushing write buffer of size 2017/05/04 13:29:00 error > processing request: write tcp 127.0.0.1:9090->127.0.0.1:58923: write: broken > pipe > 17 to transport, only wrote 0 bytes: write tcp > 127.0.0.1:9090->127.0.0.1:58920: write: broken pipe > 2017/05/04 13:29:00 error processing request: write tcp > 127.0.0.1:9090->127.0.0.1:58932: write: broken pipe > 2017/05/04 13:29:00 error processing request: write tcp > 127.0.0.1:9090->127.0.0.1:58930: write: broken pipe > Error while flushing write buffer of size 17 to transport, only wrote > 2017/05/04 13:29:00 error processing request: write tcp > 127.0.0.1:9090->127.0.0.1:58926: write: broken pipe > 0 bytes: write tcp 127.0.0.1:9090->127.0.0.1:58912: write: broken pipe > Error while flushing write buffer of size 172017/05/04 13:29:00 error > processing request: write tcp 127.0.0.1:9090->127.0.0.1:58912: write: broken > pipe > to transport, only wrote 0 bytes: write tcp 127.0.0.1:9090->127.0.0.1:58918: > write: broken pipe > 2017/05/04 13:29:00 error processing request: write tcp > 127.0.0.1:9090->127.0.0.1:58918: write: broken pipe > 2017/05/04 13:29:00 error processing request: write tcp > 127.0.0.1:9090->127.0.0.1:58920: write: broken pipe > 2017/05/04 13:29:00 error processing request: write tcp > 127.0.0.1:9090->127.0.0.1:58920: write: broken pipe > 2017/05/04 13:29:00 error processing request: write tcp > 127.0.0.1:9090->127.0.0.1:58915: write: broken pipe > 2017/05/04 13:29:00 error processing request: write tcp >
[jira] [Updated] (THRIFT-4188) Error while flushing write buffer of size 70 to transport, only wrote 0 bytes: write tcp 127.0.0.1:5000->127.0.0.1:58174: write: broken pipe
[ https://issues.apache.org/jira/browse/THRIFT-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jens Geyer updated THRIFT-4188: --- Description: {code} if *server { if err := runServer(transportFactory, protocolFactory, *addr, *secure); err != nil { fmt.Println("error running server:", err) } } else { done := make(chan int32) for i := 0; i < 5000; i++ { go func(i int) { err, s := runClient(transportFactory, protocolFactory, *addr, *secure, int32(i)) if err != nil { fmt.Println("error running client:", err) } done <- s }(i) } for i := 0; i < 5000; i++ { fmt.Println(<-done) } } {code} too many errors,eg: {code} Error while flushing write buffer of size 17 to transport, only wrote 02017/05/04 13:29:00 error processing request: write tcp 127.0.0.1:9090->127.0.0.1:58932: write: broken pipe bytes: write tcp 127.0.0.1:9090->127.0.0.1:58915: write: broken pipe 2017/05/04 13:29:00 error processing request: write tcp 127.0.0.1:9090->127.0.0.1:58922: write: broken pipe Error while flushing write buffer of size 2017/05/04 13:29:00 error processing request: write tcp 127.0.0.1:9090->127.0.0.1:58928: write: broken pipe 17 to transport, only wrote 0 bytes: write tcp 127.0.0.1:9090->127.0.0.1:58923: write: broken pipe2017/05/04 13:29:00 error processing request: write tcp 127.0.0.1:9090->127.0.0.1:58915: write: broken pipe Error while flushing write buffer of size 17 to transport, only wrote 0 bytes: write tcp 127.0.0.1:9090->127.0.0.1:58930: write: broken pipe 2017/05/04 13:29:00 error processing request: write tcp 127.0.0.1:9090->127.0.0.1:58923: write: broken pipe Error while flushing write buffer of size 17 to transport, only wrote 0 bytes: write tcp 127.0.0.1:9090->127.0.0.1:58926: write: broken pipe 2017/05/04 13:29:00 error processing request: write tcp 127.0.0.1:9090->127.0.0.1:58930: write: broken pipe Error while flushing write buffer of size 2017/05/04 13:29:00 error processing request: write tcp 127.0.0.1:9090->127.0.0.1:58923: write: broken pipe 17 to transport, only wrote 0 bytes: write tcp 127.0.0.1:9090->127.0.0.1:58920: write: broken pipe 2017/05/04 13:29:00 error processing request: write tcp 127.0.0.1:9090->127.0.0.1:58932: write: broken pipe 2017/05/04 13:29:00 error processing request: write tcp 127.0.0.1:9090->127.0.0.1:58930: write: broken pipe Error while flushing write buffer of size 17 to transport, only wrote 2017/05/04 13:29:00 error processing request: write tcp 127.0.0.1:9090->127.0.0.1:58926: write: broken pipe 0 bytes: write tcp 127.0.0.1:9090->127.0.0.1:58912: write: broken pipe Error while flushing write buffer of size 172017/05/04 13:29:00 error processing request: write tcp 127.0.0.1:9090->127.0.0.1:58912: write: broken pipe to transport, only wrote 0 bytes: write tcp 127.0.0.1:9090->127.0.0.1:58918: write: broken pipe 2017/05/04 13:29:00 error processing request: write tcp 127.0.0.1:9090->127.0.0.1:58918: write: broken pipe 2017/05/04 13:29:00 error processing request: write tcp 127.0.0.1:9090->127.0.0.1:58920: write: broken pipe 2017/05/04 13:29:00 error processing request: write tcp 127.0.0.1:9090->127.0.0.1:58920: write: broken pipe 2017/05/04 13:29:00 error processing request: write tcp 127.0.0.1:9090->127.0.0.1:58915: write: broken pipe 2017/05/04 13:29:00 error processing request: write tcp 127.0.0.1:9090->127.0.0.1:58912: write: broken pipe 2017/05/04 13:29:00 error processing request: write tcp 127.0.0.1:9090->127.0.0.1:58918: write: broken pipe 2017/05/04 13:29:00 error processing request: write tcp 127.0.0.1:9090->127.0.0.1:58926: write: broken pipe {code} was: if *server { if err := runServer(transportFactory, protocolFactory, *addr, *secure); err != nil { fmt.Println("error running server:", err) } } else { done := make(chan int32) for i := 0; i < 5000; i++ { go func(i int) { err, s := runClient(transportFactory, protocolFactory, *addr, *secure, int32(i)) if err != nil { fmt.Println("error running client:", err) } done <- s }(i) } for i := 0; i < 5000; i++ { fmt.Println(<-done) } } too many errors,eg: Error while flushing write buffer of size 17 to transport, only wrote 02017/05/04 13:29:00
[jira] [Resolved] (THRIFT-4197) Implement transparent gzip compression for HTTP transport
[ https://issues.apache.org/jira/browse/THRIFT-4197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jens Geyer resolved THRIFT-4197. Resolution: Fixed Fix Version/s: 0.11.0 Nice addition, committed. > Implement transparent gzip compression for HTTP transport > - > > Key: THRIFT-4197 > URL: https://issues.apache.org/jira/browse/THRIFT-4197 > Project: Thrift > Issue Type: Improvement > Components: Go - Library >Affects Versions: 0.10.0 >Reporter: Can Celasun >Assignee: Can Celasun >Priority: Minor > Fix For: 0.11.0 > > > HTTP transport should check if the client supports gzip compressed responses > and do transparent compression if it does. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (THRIFT-4190) improve C# TThreadPoolServer defaults
[ https://issues.apache.org/jira/browse/THRIFT-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16009212#comment-16009212 ] ASF GitHub Bot commented on THRIFT-4190: Github user Jens-G commented on the issue: https://github.com/apache/thrift/pull/1268 The NPM timed out, no other errors. If there are no objections against it, I'll commit that shortly. So please speak now (or be quiet forever, as they say) ... :-) > improve C# TThreadPoolServer defaults > - > > Key: THRIFT-4190 > URL: https://issues.apache.org/jira/browse/THRIFT-4190 > Project: Thrift > Issue Type: Improvement > Components: C# - Library >Reporter: Jens Geyer >Assignee: Jens Geyer > > The TThreadPoolServer uses hardcoded values to specify min/max number of > threads, if the caller does not specify them. This is rather unexpected in my > opinion, since the global C# ThreadPool (which is used internally) comes with > its own defaults for all 4 values - yes, 4, not 2: there are different > settings for the number of threads on one hand and the number of asyn IO > completion ports on the other, and they are not necessary identical numbers. > For example, on my machine I get these numbers by default: > - min 4 threads and 4 I/O completion ports > - max 37267 threads and 1000 I/O completion ports > There are several *problems* with this approach: > # There is really no way to bypass the defaults of min 10/10 and max 100/100 > that are hard-coded into TThreadPoolServer and use the defaults provided by > the NET framework instead, since we can only pass number which is then used > for threads AND io ports. In my example, no matter what value I pass, 37267 > or 1000, it will be something other than the defaults. > # It is rather unexpected to have Thrift override the default settings of the > global thread pool object if I don't even provide values by calling one of > the simpler TThreadPoolServer CTORs. > # I'm not sure where the defaults are come from. Both numbers look like wild > guesswork to me. The defaults provided by the runtime make much more sense, > as they automatically adapt to the machine's capabilities. > My *proposal* to solve it comes in two parts: > # Change the CTOR in a way that interprets 0 or negative values as intention > to stick with the NET default settings. I think that is the best way to > handle it, as the current implementation would just throw in a very defined > way, so we don't get any compatibility conflicts here that pass undetectedly. > # Additionally make the default values {{DEFAULT_MAX_THREADS}} and > {{DEFAULT_MIN_THREADS}} both 0 (or negative) to enforce the system's > defaults. Since this will be a breaking change, as it changes the current > default behaviour, I'd like to know the opinions of the community before I > commit that part of the changes. > Further reference > - [SetMinThreads > method|https://msdn.microsoft.com/en-us/library/system.threading.threadpool.setminthreads(v=vs.110).aspx] > - [SetMaxThreads > method|https://msdn.microsoft.com/en-us/library/system.threading.threadpool.setmaxthreads(v=vs.110).aspx] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] thrift issue #1268: THRIFT-4190 Improve C# TThreadPoolServer defaults (part ...
Github user Jens-G commented on the issue: https://github.com/apache/thrift/pull/1268 The NPM timed out, no other errors. If there are no objections against it, I'll commit that shortly. So please speak now (or be quiet forever, as they say) ... :-) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---