[
https://issues.apache.org/jira/browse/THRIFT-847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christian Lavoie updated THRIFT-847:
------------------------------------
Attachment: v1-WORK_IN_PROGRESS-unified_tests.tar.gz
So, here's a very rough draft of unfinished cross language tests. To run,
{{cd}} into a working thrift tree, extract the tar, it'll create a
{{crosstest/}} subdir with a few shell scripts:
{noformat}
$ ls
SimpleTest.cpp SimpleTest.thrift java-build.sh
SimpleTest.hs cpp-build.sh java-run.sh
SimpleTest.java hs-build.sh
{noformat}
To build the tests:
{noformat}
$ ./java-build.sh && ./cpp-build.sh && ./hs-build.sh
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
[ 1 of 14] Compiling Thrift.Transport ( ../lib/hs/src/Thrift/Transport.hs,
build-hs/Thrift/Transport.o )
[ 2 of 14] Compiling Thrift.Transport.Handle (
../lib/hs/src/Thrift/Transport/Handle.hs, build-hs/Thrift/Transport/Handle.o )
[ 3 of 14] Compiling Thrift.Protocol ( ../lib/hs/src/Thrift/Protocol.hs,
build-hs/Thrift/Protocol.o )
[ 4 of 14] Compiling Thrift.Protocol.Binary (
../lib/hs/src/Thrift/Protocol/Binary.hs, build-hs/Thrift/Protocol/Binary.o )
[ 5 of 14] Compiling Thrift ( ../lib/hs/src/Thrift.hs,
build-hs/Thrift.o )
[ 6 of 14] Compiling Thrift.Server ( ../lib/hs/src/Thrift/Server.hs,
build-hs/Thrift/Server.o )
[ 7 of 14] Compiling SimpleTest_Types ( gen-hs/SimpleTest_Types.hs,
build-hs/SimpleTest_Types.o )
[ 8 of 14] Compiling Base_Iface ( gen-hs/Base_Iface.hs,
build-hs/Base_Iface.o )
[ 9 of 14] Compiling Simple_Iface ( gen-hs/Simple_Iface.hs,
build-hs/Simple_Iface.o )
[10 of 14] Compiling Base ( gen-hs/Base.hs, build-hs/Base.o )
[11 of 14] Compiling Simple ( gen-hs/Simple.hs, build-hs/Simple.o )
[12 of 14] Compiling Base_Client ( gen-hs/Base_Client.hs,
build-hs/Base_Client.o )
[13 of 14] Compiling Simple_Client ( gen-hs/Simple_Client.hs,
build-hs/Simple_Client.o )
[14 of 14] Compiling Main ( SimpleTest.hs, build-hs/Main.o )
Linking hs-test ...
{noformat}
And to run the tests in {{both}} mode (each language is run against itself, but
through a TCP socket):
{noformat}
$ ./cpp-simple-test both && ./java-run.sh both && ./hs-test both
Running in both client-and-server mode
Running byte tests
arg: both
Starting server on port 9090 ...
Running byte numeric tests
Running i16 numeric tests
Running i32 numeric tests
Running i64 numeric tests
Running double numeric tests
SUCCESS
Runing in both client and server mode
Testing Simple.addBytes...
{noformat}
You can also run each pair of executables with one in {{server}} and one in
{{client}} mode:
{noformat}
$ ./java-run.sh server & sleep 5 && ./cpp-simple-test client && ./hs-test
client && sleep 1 && kill $(jobs -p)
[1] 12018
arg: server
Starting server on port 9090 ...
Running in client-only mode
Running byte tests
Runing in client-only mode
Testing Simple.addBytes...
{noformat}
On my Mac machine, I can get all pairs (and all three tests in {{both}} modes)
to run with 0.5.0, but NOT with HEAD; the cpp client seems to be broken since
dreiss' facebook patches; just guessing here, but the error message is telling:
{noformat}
$ ./hs-test server & sleep 3 && ./cpp-simple-test client
[1] 22775
Runing in server-only mode
Running in client-only mode
Running byte tests
terminate called after throwing an instance of
'apache::thrift::transport::TTransportException'
what(): Base TTransport cannot write.
Abort trap
{noformat}
So, here's my plan, and my RFC:
# Finish the simplest possible tests, and implement a handful of clients (I
suspect I'll get perl, haskell, c++ and java done) and a handful of servers
(haskell, c++ and java; most likely)
# Get this hooked to the autotools build system and {{make check}}
# Define a set of {{base}} tests: you aren't a proper thrift bindings unless
you pass those
# Define different set of {{advanced}} tests: you match specific features
exactly as the reference languages do (== java, cpp, I think). In particular, I
introduced a bug where Haskell consider {{byte}} to be unsigned, and java/cpp
don't. Under this definition, Haskell would fail the advanced tests that check
for numerical type implementation details
# Create some sort of score card for each language -- to see how closely they
match the reference implementation behaviour
Please comment on the approach, I'd like some feedback before I spend too much
time implementing tests nobody will like.
> Test Framework harmonization across all languages
> -------------------------------------------------
>
> Key: THRIFT-847
> URL: https://issues.apache.org/jira/browse/THRIFT-847
> Project: Thrift
> Issue Type: Improvement
> Components: C# - Library, C++ - Library, Cocoa - Library, Erlang -
> Library, Haskell - Library, Java - Library, JavaScript - Library, OCaml -
> Library, Perl - Library, PHP - Library, Python - Library, Ruby - Library,
> Test Suite
> Affects Versions: 0.1, 0.2, 0.3
> Reporter: Roger Meier
> Fix For: 0.6
>
> Attachments: v1-WORK_IN_PROGRESS-unified_tests.tar.gz
>
>
> Today each Language supported by Thrift, have its own unit test, all are
> using the same Thrift IDL's located at the test directory. But the behavior
> of these tests seems to be different from language to language... this makes
> it difficult to do tests and bug fixing across different Languages. e.g.
> * C++ Test and JavaScript Test Server written in Java have different
> responses for the same services
> * C# and Java Test Server have different responses for testException as C++
> I propose the following steps:
> * identify the language with the reference implementation (well defined
> return values for all test cases)
> * update the ThriftTest.thrift with details about the required return values
> that have to be implemented
> * update test implementations and move language tests into their appropriate
> library directory (THRIFT-35)
> * a public test server that supports multiple protocols and transports could
> be another enhancement for testing purposes
> I'm ready to help preparing patches, just tell me what you need!
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
