[ https://issues.apache.org/jira/browse/THRIFT-2619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Frank Schroeder updated THRIFT-2619: ------------------------------------ Attachment: THRIFT-2619-handle-EOF-correctly-1.patch > Go lib http transport does not handle EOF correctly > --------------------------------------------------- > > Key: THRIFT-2619 > URL: https://issues.apache.org/jira/browse/THRIFT-2619 > Project: Thrift > Issue Type: Bug > Components: Go - Library > Affects Versions: 0.9.2 > Environment: osx > Reporter: Frank Schroeder > Attachments: THRIFT-2619-handle-EOF-correctly-1.patch > > > Executing the tests with Go 1.3 fails with the following errors: > {code} > frschroeder@Frank:~/git/apache/thrift/lib/go/thrift (master)$ go test > --- FAIL: TestReadWriteBinaryProtocol (0.00 seconds) > protocol_test.go:220: ReadWriteBool: *thrift.TBinaryProtocol > *thrift.THttpClient "EOF" Error reading bool at index 4: %!q(bool=true) > protocol_test.go:223: ReadWriteBool: index 4 > &{%!q(*thrift.THttpClient=&{0xc208060510 0xc20802a4d0 0xc20802a540 > map[Content-Type:[application/x-thrift]] 0 0}) > %!q(*thrift.THttpClient=&{0xc208060510 0xc20802a4d0 0xc20802a540 > map[Content-Type:[application/x-thrift]] 0 0}) > %!q(*thrift.THttpClient=&{0xc208060510 0xc20802a4d0 0xc20802a540 > map[Content-Type:[application/x-thrift]] 0 0}) > %!q(*thrift.THttpClient=&{0xc208060510 0xc20802a4d0 0xc20802a540 > map[Content-Type:[application/x-thrift]] 0 0}) %!q(bool=false) %!q(bool=true) > "\x00\x00\x00\x05\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"} > &{%!q(*http.Response=&{200 OK 200 HTTP/1.1 1 1 map[Date:[Thu, 10 Jul 2014 > 09:03:19 GMT] Content-Length:[10] Content-Type:[application/octet-stream]] > 0xc20803c8c0 10 [] false map[] 0xc208028d00 <nil>}) %!q(*url.URL=&{http > <nil> 127.0.0.1:40000 }) %!q(*bytes.Buffer=&{[] 0 [0 0 0 0] [0 0 0 0 0 0 0 > 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 > 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] 0}) > map["Content-Type":["application/x-thrift"]] '\x00' '\x00'} %!q(bool=true) != > %!q(bool=false) > protocol_test.go:269: ReadWriteByte: *thrift.TBinaryProtocol > *thrift.THttpClient "EOF" Error reading byte at index 6: 'ÿ' > protocol_test.go:272: ReadWriteByte: *thrift.TBinaryProtocol > *thrift.THttpClient 255 != 0 > protocol_test.go:269: ReadWriteByte: *thrift.TBinaryProtocol > *thrift.THttpClient "EOF" Error reading byte at index 6: 'ÿ' > protocol_test.go:272: ReadWriteByte: *thrift.TBinaryProtocol > *thrift.THttpClient 255 != 0 > --- FAIL: TestReadWriteCompactProtocol (0.00 seconds) > protocol_test.go:220: ReadWriteBool: *thrift.TCompactProtocol > *thrift.THttpClient "EOF" Error reading bool at index 4: %!q(bool=true) > protocol_test.go:223: ReadWriteBool: index 4 > &{%!q(*thrift.THttpClient=&{0xc2080d0000 0xc208092150 0xc2080921c0 > map[Content-Type:[application/x-thrift]] 0 0}) > %!q(*thrift.THttpClient=&{0xc2080d0000 0xc208092150 0xc2080921c0 > map[Content-Type:[application/x-thrift]] 0 0}) [] '\x00' "" '\x00' > %!q(bool=false) %!q(bool=false) %!q(bool=false) > "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"} > &{%!q(*http.Response=&{200 OK 200 HTTP/1.1 1 1 map[Date:[Thu, 10 Jul 2014 > 09:03:19 GMT] Content-Length:[6] Content-Type:[application/octet-stream]] > 0xc20803d480 6 [] false map[] 0xc2080ce000 <nil>}) %!q(*url.URL=&{http <nil> > 127.0.0.1:40000 }) %!q(*bytes.Buffer=&{[] 0 [0 0 0 0] [0 0 0 0 0 0 0 0 0 0 > 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 > 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] 0}) > map["Content-Type":["application/x-thrift"]] '\x00' '\x00'} %!q(bool=true) != > %!q(bool=false) > protocol_test.go:269: ReadWriteByte: *thrift.TCompactProtocol > *thrift.THttpClient "EOF" Error reading byte at index 6: 'ÿ' > protocol_test.go:272: ReadWriteByte: *thrift.TCompactProtocol > *thrift.THttpClient 255 != 0 > protocol_test.go:306: ReadWriteI16: *thrift.TCompactProtocol > *thrift.THttpClient "EOF" Error reading int16 at index 7: %!q(int16=-32768) > protocol_test.go:309: ReadWriteI16: *thrift.TCompactProtocol > *thrift.THttpClient -32768 != 0 > protocol_test.go:343: ReadWriteI32: *thrift.TCompactProtocol > *thrift.THttpClient "EOF" Error reading int32 at index 8: > %!q(int32=-2147483535) > protocol_test.go:346: ReadWriteI32: *thrift.TCompactProtocol > *thrift.THttpClient -2147483535 != 0 > protocol_test.go:379: ReadWriteI64: *thrift.TCompactProtocol > *thrift.THttpClient "EOF" Error reading int64 at index 12: > %!q(int64=9223372036854775807) > protocol_test.go:382: ReadWriteI64: *thrift.TCompactProtocol > *thrift.THttpClient %!q(int64=9223372036854775807) != '\x00' > protocol_test.go:379: ReadWriteI64: *thrift.TCompactProtocol > *thrift.THttpClient "EOF" Error reading int64 at index 12: > %!q(int64=9223372036854775807) > protocol_test.go:382: ReadWriteI64: *thrift.TCompactProtocol > *thrift.THttpClient %!q(int64=9223372036854775807) != '\x00' > protocol_test.go:269: ReadWriteByte: *thrift.TCompactProtocol > *thrift.THttpClient "EOF" Error reading byte at index 6: 'ÿ' > protocol_test.go:272: ReadWriteByte: *thrift.TCompactProtocol > *thrift.THttpClient 255 != 0 > --- FAIL: TestHttpClient (0.00 seconds) > transport_test.go:86: Transport *thrift.THttpClient cannot read binary > data 2 of total length 4096 from offset 0: EOF > FAIL > exit status 1 > FAIL _/Users/frschroeder/git/apache/thrift/lib/go/thrift 0.029s > frschroeder@Frank:~/git/apache/thrift/lib/go/thrift (master)$ git log | head > commit b0350dbc40d3bc442f02bbd5980e2c2b5d83194d > Author: jfarrell <jfarr...@apache.org> > Date: Wed Jul 9 23:39:34 2014 -0400 > Thrift-2601:Fix vagrant to work again for builds again > Client: build process > Patch: jfarrell > Updates debian packaging to work with ubuntu 14.04 deps > {code} > The reason for this is that the http.bodyEOFSignal transport changed its > behavior when reaching EOF. With go 1.2 it sent a nil error on the last byte > it read and with go 1.3 it sends the data and EOF in the error. > According to the documented behavior of an io.Reader both is correct and > callers must handle both conditions. > (http://golang.org/pkg/io/#Reader) > {code} > When Read encounters an error or end-of-file condition after successfully > reading n > 0 bytes, it returns the number of bytes read. It may return the > (non-nil) error from the same call or return the error (and n == 0) from a > subsequent call. An instance of this general case is that a Reader returning > a non-zero number of bytes at the end of the input stream may return either > err == EOF or err == nil. The next Read should return 0, EOF regardless. > Callers should always process the n > 0 bytes returned before considering the > error err. Doing so correctly handles I/O errors that happen after reading > some bytes and also both of the allowed EOF behaviors. > {code} > The code in the Go thrift lib does not check for err == io.EOF and assumes > that err != nil is an error. Therefore, the last chunk of data is not read. > I've created a patch and some tests but I'd appreciate if someone could > double check that what I'm doing there makes sense since we're not using the > HTTP transport. Ran the tests with both Go 1.2 and 1.3. -- This message was sent by Atlassian JIRA (v6.2#6252)