Re: local test server (Was: Re: [jira] Commented: (HARMONY-71) java.net.URLConnection.setUseCaches throws unspecified IllegalAccessError)
Yes, Jetty is a good option :-) Richard Liang China Software Development Lab, IBM Geir Magnusson Jr wrote: Stepan Mishura wrote: Tim, George, I'd like to resume this topic Thanks for posting to the dev list :) There is no way to force a server to send you a chunked response using regular HTTP headers, so in this case the server and client have an understanding that when the client asks for a particular resource the server will send it back in chunks. I've created a small local server that just demonstrates common approach only. The idea is the next: a simple local server provides a way for a unit test to configure a server's response (also it may be improved to save a request message to be check by a unit test): ---invoke------request--- A unit teststested API a local server instance | ---result------response--- ^ | | ---configure server I believe this approach may be applied for most of cases and allow us to eliminate dependency on an external server. And give us tight control to be able to mimic misbehavior on the server side to be sure that the library behaves correctly. I've created JIRA (see Harmony-164) and attached a file with local server and 2 JUnit test cases (one of them sends a chunked response, please note there is no server's code specific for a selected test). Could you please review the code and let me know what you think? I took a quick look, and while I really am convinced that our own framework is necessary for all sorts of reasons I and others have listed already, I do wonder if we could simply do better via using jetty or tomcat in some way? geir
Re: local test server (Was: Re: [jira] Commented: (HARMONY-71) java.net.URLConnection.setUseCaches throws unspecified IllegalAccessError)
On 3/3/06, Geir Magnusson Jr [EMAIL PROTECTED] wrote: ...snip... I believe this approach may be applied for most of cases and allow us to eliminate dependency on an external server. And give us tight control to be able to mimic misbehavior on the server side to be sure that the library behaves correctly. Sure, it is quite important to verify that the implementation functions properly in case of incorrect server's respond. And this can not be tested with a 'real' server because as was noted previously there is no way to force a server to send you a chunked response using regular HTTP headers. I can add that there is no way to force a server to send you a corrupted respond. ...snip... I took a quick look, and while I really am convinced that our own framework is necessary for all sorts of reasons I and others have listed already, I do wonder if we could simply do better via using jetty or tomcat in some way? Sorry, I didn't catch your suggestion with using jetty or tomcat. I don't understand the next: why we should introduce external dependency when there is simple solution? When I see that a complicated way is invented instead of using simple and obvious (from my point of view) solution I always assume two options here: - I missed something important and someone should point me out where I'm wrong. - A person tends to create sophisticated solutions due to some reasons, for example, she/he worked with some http server for years and now believes that it should be used everywhere. Returning back to our case, I do believe that there must be strong arguments for using external server for unit testing. Arguments like a server is free and easy configurable don't work against 40 lines of code and absence of special configuration. Thanks, Stepan On 3/3/06, Geir Magnusson Jr [EMAIL PROTECTED] wrote: Stepan Mishura wrote: Tim, George, I'd like to resume this topic Thanks for posting to the dev list :) There is no way to force a server to send you a chunked response using regular HTTP headers, so in this case the server and client have an understanding that when the client asks for a particular resource the server will send it back in chunks. I've created a small local server that just demonstrates common approach only. The idea is the next: a simple local server provides a way for a unit test to configure a server's response (also it may be improved to save a request message to be check by a unit test): ---invoke------request--- A unit teststested API a local server instance | ---result------response--- ^ | | ---configure server I believe this approach may be applied for most of cases and allow us to eliminate dependency on an external server. And give us tight control to be able to mimic misbehavior on the server side to be sure that the library behaves correctly. I've created JIRA (see Harmony-164) and attached a file with local server and 2 JUnit test cases (one of them sends a chunked response, please note there is no server's code specific for a selected test). Could you please review the code and let me know what you think? I took a quick look, and while I really am convinced that our own framework is necessary for all sorts of reasons I and others have listed already, I do wonder if we could simply do better via using jetty or tomcat in some way? geir -- Thanks, Stepan Mishura Intel Middleware Products Division
Re: local test server (Was: Re: [jira] Commented: (HARMONY-71) java.net.URLConnection.setUseCaches throws unspecified IllegalAccessError)
Tim, George, I'd like to resume this topic There is no way to force a server to send you a chunked response using regular HTTP headers, so in this case the server and client have an understanding that when the client asks for a particular resource the server will send it back in chunks. I've created a small local server that just demonstrates common approach only. The idea is the next: a simple local server provides a way for a unit test to configure a server's response (also it may be improved to save a request message to be check by a unit test): ---invoke------request--- A unit teststested API a local server instance | ---result------response--- ^ | | ---configure server I believe this approach may be applied for most of cases and allow us to eliminate dependency on an external server. I've created JIRA (see Harmony-164) and attached a file with local server and 2 JUnit test cases (one of them sends a chunked response, please note there is no server's code specific for a selected test). Could you please review the code and let me know what you think? Thanks, Stepan On 2/22/06, Tim Ellison [EMAIL PROTECTED] wrote: Stepan Mishura wrote: On 2/21/06, Tim Ellison [EMAIL PROTECTED] wrote: Stepan Mishura wrote: I glanced through a local server code from HARMONY-57 contribution without looking into implementation details and ... this made me think. I catch hold of the following: 1) The server contains some testing code that looks not good for me Care to explain? Support_HttpServer.java has the following code: class ServerThread implements Runnable { public void run() { // call the test function based on the requested resource if (resourceName.equals(CHUNKEDTEST)) chunkedTest(); ... lot of 'else if' ... ... } private void chunkedTest() { ... construct server's respond and send it ... } I'd prefer to avoid this and to have a server implementation that provides a way for unit test to check received request and to configure a server's respond. and I thought you wanted it simple ;-) There is no way to force a server to send you a chunked response using regular HTTP headers, so in this case the server and client have an understanding that when the client asks for a particular resource the server will send it back in chunks. If that understanding was encoded differently, say in the search field of the URL rather than the path would that make you happier? do you have some other idea? snip Does it interfere with what you want to do? Sure, we have javax.security.auth.kerberos.KerberosTicket class that currently contains a stub. I was thinking about implementing the stub and how to test it. So I played a little bit with developing Kerberos stub server. When I saw that tests for URLConnection class failed because they need local stub server then I got interested how to solve this similar issue. Not clear how that interferes. You should feel free to reuse the server code and extend it as you need to do a faux Kerberos server. Regards, Tim -- Tim Ellison ([EMAIL PROTECTED]) IBM Java technology centre, UK. -- Thanks, Stepan Mishura Intel Middleware Products Division
Re: local test server (Was: Re: [jira] Commented: (HARMONY-71) java.net.URLConnection.setUseCaches throws unspecified IllegalAccessError)
Stepan Mishura wrote: Tim, George, I'd like to resume this topic Thanks for posting to the dev list :) There is no way to force a server to send you a chunked response using regular HTTP headers, so in this case the server and client have an understanding that when the client asks for a particular resource the server will send it back in chunks. I've created a small local server that just demonstrates common approach only. The idea is the next: a simple local server provides a way for a unit test to configure a server's response (also it may be improved to save a request message to be check by a unit test): ---invoke------request--- A unit teststested API a local server instance | ---result------response--- ^ | | ---configure server I believe this approach may be applied for most of cases and allow us to eliminate dependency on an external server. And give us tight control to be able to mimic misbehavior on the server side to be sure that the library behaves correctly. I've created JIRA (see Harmony-164) and attached a file with local server and 2 JUnit test cases (one of them sends a chunked response, please note there is no server's code specific for a selected test). Could you please review the code and let me know what you think? I took a quick look, and while I really am convinced that our own framework is necessary for all sorts of reasons I and others have listed already, I do wonder if we could simply do better via using jetty or tomcat in some way? geir
Re: local test server (Was: Re: [jira] Commented: (HARMONY-71) java.net.URLConnection.setUseCaches throws unspecified IllegalAccessError)
Geir Magnusson Jr wrote: George Harley wrote: Geir Magnusson Jr wrote: George Harley wrote: What is the problem with using httpd to test Harmony ? It's a rather large external dependency. Reminds me of the time we got the TCK for JavaMail (a story in itself) and the only practical way to use it was to get Sun's IMAP server which only ran on Solaris. So what I was getting at is can we use something smaller/lighter that we can include in the project for this? What is it required to do? The tests require an HTTP server, an FTP server and a SOCKS server to be listening on certain (configurable) ports. All of which are available in this toaster language that we like to write in... The respective servers need to have certain resources - text files, jars etc - deployed on them as a number of the java.net.* tests try and access them from the configured locations. The tests check that the expected content is available, that it is the expected size, contains the expected data etc etc. Sure - of course. There are a number of mature, stable, freely available HTTP/FTP/SOCKS servers out there that will do just great for satisfying the demands of these tests so I don't propose to recommend anything in particular (although I have had no problems running Apache HTTPD on my local Windows machine). I don't see any justification for someone to spend time developing a new server to run these tests. But, hey, it's not my time that will be wasted... The goal should always be : $ svn co $URL $ ant $ ant test and all should work. Now, I know that is just an ideal - there may be things one may have to do. But running a target test server is fairly strightforward, I would think. There's an FTP server somewhere in incubator, tomcat clearly has HTTP support and if you don't want that, use jetty - it's embeddable. Have a canned webapp that you deploy to deliver resources in a neat bundle... Right - and as the tests do not mandate any particular HTTP/FTP/SOCKS server then integrating executable versions of the freely available servers you describe into the test suite sounds like it should not be much of a problem. Best regards, George IBM UK What I'm trying to have happen is it be easy for any new user or potential contributor to get things up and running. (I know that when I'm looking around at new things, and I have to do something external to the project or modify my environmnet, I mentally throw a ManWhatAPainInTheBacksideWhyAreTheyWastingMyTimeException and move on...) Also, I'd like to avoid becoming a httpd help desk for our users/contributors/etc geir
Re: local test server (Was: Re: [jira] Commented: (HARMONY-71) java.net.URLConnection.setUseCaches throws unspecified IllegalAccessError)
I agree -- thanks for the patch Stepan. Regards, Tim Stepan Mishura wrote: Tim, ..snip.. I agree that the tests in HARMONY-71 only need something to accept their connection, but I was simply pointing out that there will be a local server we can use if that contribution is accepted into the project, so probably not worth writing another. I've developed a prototype of local server to be run with net unit tests, started to play with it and ... realized that the current regression tests for URLConnection class don't need any server at all :-) I've filled JIRA bug attached a patch to fix tests. Please review it. (see HARMONY-138) Also I'd like to make a note on testing URLConnection class: I found that creating an object of this class to be tested the following way in not correct: URLConnection u = new URL(http://www.apache.org;).openConnection(); openConnection() returns a default implementation of HttpURLConnection class, for example com.ibm.oti.net.www.protocol.http.HttpURLConnection, that may override a tested method. IMHO, the correct approach is to subclass URLConnection directly. Thanks, Stepan I'd like to start with the first issue (BTW, is it ok to discuss details of contribution implementation if it is not accepted yet?). Sure -- I would expect people to be evaluating it thoroughly so that they can vote the code in/out. The PPMC will shout early if the paperwork implies a contribution is unacceptable. Can we avoid adding a code that is specific for some test case to a local server? Does it interfere with what you want to do? In some cases, the test case needs to know what is on the server a priori, so that it can test the client behavior correctly. Regards, Tim On 2/16/06, Tim Ellison wrote: I would rather that you just opened a new JIRA issue for that as it doesn't relate to the original problem. FYI there is a local server incoming in the HARMONY-57 contribution, so don't spend any time on it, see: HARMONY-57\Harmony\Harmony_Tests\src\test\java\tests\support\Support_HttpServer.java Regards, Tim Stepan Mishura (JIRA) wrote: [ http://issues.apache.org/jira/browse/HARMONY-71?page=comments#action_12366637 ] Stepan Mishura commented on HARMONY-71: --- Tim, Added test case (test_setUseCachesZ) depends on network and it fails if it can not create connection with apache.org. The same for HARMONY-72. I'm going to attach a patch file with a local server in it. Could you reopen this JIRA? Thanks, Stepan. java.net.URLConnection.setUseCaches throws unspecified IllegalAccessError -- Key: HARMONY-71 URL: http://issues.apache.org/jira/browse/HARMONY-71 Project: Harmony Type: Bug Components: Classlib Reporter: Svetlana Samoilenko Assignee: Tim Ellison According to j2se 1.4.2 specification method java.net.URLConnection.setUseCaches(boolean) throws IllegalStateException, if already connected. Harmony throws java.lang.IllegalAccessErrorinstead, that contradicts the specification. Code to reproduce: import java.io.IOException; import java.net.*; public class test2 { public static void main(String[] args) { HttpURLConnection u=null; try { u=(HttpURLConnection)(new URL(http://intel.com ).openConnection()); u.connect(); } catch (MalformedURLException e) { System.out.println(unexpected MalformedURLException+e); } catch (IOException f) { System.out.println(unexpected IOException+f); } try { u.setUseCaches(true); } catch (IllegalStateException e) { System.out.println(OK. Expected IllegalStateException); e.printStackTrace(); }; } } Steps to Reproduce: 1. Build Harmony (check-out on 2006-01-30) j2se subset as described in README.txt. 2. Compile test2.java using BEA 1.4 javac javac -d . test2.java 3. Run java using compatible VM (J9) java -showversion test2 Output: C:\tmpC:\jrockit-j2sdk1.4.2_04\bin\java.exe -showversion test2 java version 1.4.2_04 Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_04-b05) BEA WebLogic JRockit(TM) 1.4.2_04 JVM (build ari-31788-20040616-1132-win-ia32, Native Threads, GC strategy: parallel) OK. Expected IllegalStateException java.lang.IllegalStateException: Already connected at java.net.URLConnection.setUseCaches(Z)V(URLConnection.java :828) at test2.main([Ljava.lang.String;)V(test2.java:17) C:\tmpC:\harmony\trunk\deploy\jre\bin\java -showversion test2 (c) Copyright 1991, 2005 The Apache Software Foundation or its licensors, as applicable. java.lang.IllegalAccessError: Connection already established at java.net.URLConnection.setUseCaches(URLConnection.java :923) at test2.main(test2.java:17)
Re: local test server (Was: Re: [jira] Commented: (HARMONY-71) java.net.URLConnection.setUseCaches throws unspecified IllegalAccessError)
George Harley wrote: Geir Magnusson Jr wrote: George Harley wrote: What is the problem with using httpd to test Harmony ? It's a rather large external dependency. Reminds me of the time we got the TCK for JavaMail (a story in itself) and the only practical way to use it was to get Sun's IMAP server which only ran on Solaris. So what I was getting at is can we use something smaller/lighter that we can include in the project for this? What is it required to do? The tests require an HTTP server, an FTP server and a SOCKS server to be listening on certain (configurable) ports. All of which are available in this toaster language that we like to write in... The respective servers need to have certain resources - text files, jars etc - deployed on them as a number of the java.net.* tests try and access them from the configured locations. The tests check that the expected content is available, that it is the expected size, contains the expected data etc etc. Sure - of course. There are a number of mature, stable, freely available HTTP/FTP/SOCKS servers out there that will do just great for satisfying the demands of these tests so I don't propose to recommend anything in particular (although I have had no problems running Apache HTTPD on my local Windows machine). I don't see any justification for someone to spend time developing a new server to run these tests. But, hey, it's not my time that will be wasted... The goal should always be : $ svn co $URL $ ant $ ant test and all should work. Now, I know that is just an ideal - there may be things one may have to do. But running a target test server is fairly strightforward, I would think. There's an FTP server somewhere in incubator, tomcat clearly has HTTP support and if you don't want that, use jetty - it's embeddable. Have a canned webapp that you deploy to deliver resources in a neat bundle... What I'm trying to have happen is it be easy for any new user or potential contributor to get things up and running. (I know that when I'm looking around at new things, and I have to do something external to the project or modify my environmnet, I mentally throw a ManWhatAPainInTheBacksideWhyAreTheyWastingMyTimeException and move on...) Also, I'd like to avoid becoming a httpd help desk for our users/contributors/etc geir
Re: local test server (Was: Re: [jira] Commented: (HARMONY-71) java.net.URLConnection.setUseCaches throws unspecified IllegalAccessError)
Mark Hindess wrote: But this doesn't mean George is wrong! Because *if* there was a publically accessible Internet server that already had Apache httpd, twoftpd (my favourite ftp server this week), Dante socks, etc, then the scenario I like to optimise becomes possible. And doesn't work offline. The effort of setting up one hosting server is definitely cheaper than the effort of implementing stubs - I know because I set up the server George tests against and it didn't take much time at all. Right - but how about just having it as part of the setup for test, to run it locally, and then teardown at the end of testing? Having one server means we wont get 1000's of users asking Apache httpd, twoftpd, Dante socks configuration questions on our mailing lists. It also means we have a way to see the other half of the results - that is, the server logs. (Stubs should make this easier and this cost should be considered too but if we ran the server we could make this easier.) It also means it isn't easy for anyone to tweak it. [snip] I think it basically comes down to whether or not we can provide a central server. I'm not sure it's a simple question but for me this question is key to this issue. Or perhaps I'm optimising for the wrong scenario? I think so. I think we do want a simple, easy local server so people that want to add new tests can (and test them before giving to us), etc... I guess the big question is - how hard is it to run locally? Is there a list or such that defines all the things needed for http/ftp/socks? geir
Re: local test server (Was: Re: [jira] Commented: (HARMONY-71) java.net.URLConnection.setUseCaches throws unspecified IllegalAccessError)
Tim Ellison wrote: I'd echo those sentiments. Geir: how close are you to putting HARMONY-57 to the vote? I think I have everything, but fear the JIRA! Fear the JIRA! :) Regards, Tim George Harley wrote: Just want to emphasise something that has possibly got lost in this thread. To date, all of the discussion on this topic seems to have been based on eye balling of the code in issue HARMONY-57. It would almost certainly help move things forward if people were to start actually running the tests (particularly those pesky java.net.* ones). Once people start running these tests we can discuss experiences of set-up etc and work to *evolve* them into something more suitable to the needs of the Harmony development community. It's only natural that improvements in test coverage, test quality and test complexity/simplicity are going to occur. The simpler it is to set up and run the tests, the better for all of us. My own comments on this thread have been mainly focussed about what is actually there *today* in HARMONY-57 ; it would be great to move up to discussions on the running tests and then for us all to participate in their evolution. My 2 Euro Cents. George IBM UK George Harley wrote: Mark Hindess wrote: I think it might help this disagreement if we step back and decide what scenarios for running tests we are trying to optimise. Disagreement ? What ? On this mailing list ? :-) Personally, whenever I write tests I'm doing it to optimise the scenario where a new users comes to the project and does: 1$ svn co ... classlib # or wget/tar if you prefer 2$ cd classlib/make 3$ ant test I do this because I want the tests to be easy to run by new users particularly on new platforms. I hope making it this easy means we get more people running tests and that we get a broader set of results than we could acheive ourselves. This scenario becomes slightly less pleasant if, between steps 2 and 3, [*] you have to find your nearest Linux machine and install Apache httpd, your-favourite-ftp-server, Dante socks server, etc. This typically means that I'd be inclined to write stub servers to test against. But this doesn't mean George is wrong! Because *if* there was a publically accessible Internet server that already had Apache httpd, twoftpd (my favourite ftp server this week), Dante socks, etc, then the scenario I like to optimise becomes possible. The effort of setting up one hosting server is definitely cheaper than the effort of implementing stubs - I know because I set up the server George tests against and it didn't take much time at all. Agreed : a central, publicly available test server that could be used for network-related testing would IMHO be advantageous to us all. I wonder if such a server can be supplied through the auspices of the ASF ? Incidentally, it took me only a small amount of time to set up my laptop for running the tests on locally. Having one server means we wont get 1000's of users asking Apache httpd, twoftpd, Dante socks configuration questions on our mailing lists. It also means we have a way to see the other half of the results - that is, the server logs. (Stubs should make this easier and this cost should be considered too but if we ran the server we could make this easier.) George's approach continues to be cheaper even if a few groups have to set up there own servers - though I think the set of users who don't have Internet access but do have a mechanism for getting up to date Harmony code should be small and getting smaller. Of course, it becomes much more expensive if everyone has to do it. Agreed. I've never tried using George's approach on past projects because I've either also been writing server code, that can/should be used by tests, or because the server was massive and I didn't have the experience or resources to run a real server. Because of this, initially, I was inclined to support the implemention of stubs, but now I'm not so certain. I think it basically comes down to whether or not we can provide a central server. I'm not sure it's a simple question but for me this question is key to this issue. Or perhaps I'm optimising for the wrong scenario? Regards, Mark. [*] More likely people would run step 3 find it goes horribly wrong then read the README they ignored initially and discover why. ;-)
Re: local test server (Was: Re: [jira] Commented: (HARMONY-71) java.net.URLConnection.setUseCaches throws unspecified IllegalAccessError)
Stepan Mishura wrote: I feel that I need to study more details about net testing. Also to made discussion more concrete and constructive I'll to try to provide 'a proof of concept'. I believe that this won't be a wasted effort and it won't take too much time. I'm going to select a number of 'common testing scenarios', for example, I will try to force a server to send a chunked response without creating a special resource to be requested. And demonstrate that it is possible to run them without external server or sophisticated internal server. Anyway I was going to continue to work with Kerberos server stub (because I'm not delighted with configuring external Kerberos server in order to run unit tests) and this is a good change for me to review this issue from another angle. Thanks, Stepan Mishura Intel Middleware Products Division Hi Stepan, Sounds good to me. Best regards, George IBM UK On 2/23/06, *George Harley* george.c.harley at googlemail.com http://googlemail.com wrote: Tim Ellison wrote: George Harley wrote: Geir Magnusson Jr wrote: George Harley wrote: Geir Magnusson Jr wrote: So, does this summarize to : 1) We need a new server or we should enahnce the current one? 2) We need instructions on how to do this. ? (I hope we don't need httpd to test Harmony...) Could you remind me what the current one is ? The one in the pending JIRA There is no server in the pending JIRA. (psst) take a look at HARMONY-57\Harmony\Harmony_Tests\src\test\java\tests\support\Support_HttpServer.java Regards, Tim Hi Tim, Funny. That's a test support class that is used to mock up an HTTP server for one, admittedly large, test case method (test_getInputStream in tests.api.java.net.URLConnectionTest). It is not the server. There are a significantly larger number of test case methods that rely on using the configured external HTTP/FTP/SOCKS servers to test out their functionality. Best regards, George IBM UK As briefly mentioned somewhere else in this thread, the submitted zip for that JIRA contains configuration code and properties files that permits the tester to use any server in any location (including localhost) and test resources for deploying on those servers. What is the problem with using httpd to test Harmony ? It's a rather large external dependency. Reminds me of the time we got the TCK for JavaMail (a story in itself) and the only practical way to use it was to get Sun's IMAP server which only ran on Solaris. So what I was getting at is can we use something smaller/lighter that we can include in the project for this? What is it required to do? The tests require an HTTP server, an FTP server and a SOCKS server to be listening on certain (configurable) ports. The respective servers need to have certain resources - text files, jars etc - deployed on them as a number of the java.net.* tests try and access them from the configured locations. The tests check that the expected content is available, that it is the expected size, contains the expected data etc etc. There are a number of mature, stable, freely available HTTP/FTP/SOCKS servers out there that will do just great for satisfying the demands of these tests so I don't propose to recommend anything in particular (although I have had no problems running Apache HTTPD on my local Windows machine). I don't see any justification for someone to spend time developing a new server to run these tests. But, hey, it's not my time that will be wasted... geir Stepan Mishura wrote: Hi Richard, Yes, I'm always for simplifying testing routine. And creating a detail guide for configuring some testing server is a possible way to do it. However I'd prefer to try to develop a simple local test server that is easy to configure. I believe that it will cover most of cases. And I assume that there may be a number of sophisticated test cases that requires a special server set up. But I'd place them separately this was discussed in '[testing] code for exotic configurations' topic. Otherwise there is a chance to get the following detailed instructions: 1) Download http server from http://www.apache.org ... 50) Do this and that ... 100) Now you can run net tests Thanks, Stepan On 2/22/06, Richard Liang [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: Dears, Agree that server (maybe not local) is required for some
Re: local test server (Was: Re: [jira] Commented: (HARMONY-71) java.net.URLConnection.setUseCaches throws unspecified IllegalAccessError)
This sounds like the discussion we had a while ago along the lines of 'what is a reasonable environment we should expect developers/testers to set-up locally, and what should be stubbed in the test suite itself.' i.e. which of the following are reasonable items to put in the readme? go and set-up a http server, ftp server, socks server, LDAP server, kerberos server, smtp server, ... Regards, Tim Stepan Mishura wrote: I feel that I need to study more details about net testing. Also to made discussion more concrete and constructive I'll to try to provide 'a proof of concept'. I believe that this won't be a wasted effort and it won't take too much time. I'm going to select a number of 'common testing scenarios', for example, I will try to force a server to send a chunked response without creating a special resource to be requested. And demonstrate that it is possible to run them without external server or sophisticated internal server. Anyway I was going to continue to work with Kerberos server stub (because I'm not delighted with configuring external Kerberos server in order to run unit tests) and this is a good change for me to review this issue from another angle. Thanks, Stepan Mishura Intel Middleware Products Division On 2/23/06, George Harley george.c.harley at googlemail.com wrote: Tim Ellison wrote: George Harley wrote: Geir Magnusson Jr wrote: George Harley wrote: Geir Magnusson Jr wrote: So, does this summarize to : 1) We need a new server or we should enahnce the current one? 2) We need instructions on how to do this. ? (I hope we don't need httpd to test Harmony...) Could you remind me what the current one is ? The one in the pending JIRA There is no server in the pending JIRA. (psst) take a look at HARMONY-57\Harmony\Harmony_Tests\src\test\java\tests\support\Support_HttpServer.java Regards, Tim Hi Tim, Funny. That's a test support class that is used to mock up an HTTP server for one, admittedly large, test case method (test_getInputStream in tests.api.java.net.URLConnectionTest). It is not the server. There are a significantly larger number of test case methods that rely on using the configured external HTTP/FTP/SOCKS servers to test out their functionality. Best regards, George IBM UK As briefly mentioned somewhere else in this thread, the submitted zip for that JIRA contains configuration code and properties files that permits the tester to use any server in any location (including localhost) and test resources for deploying on those servers. What is the problem with using httpd to test Harmony ? It's a rather large external dependency. Reminds me of the time we got the TCK for JavaMail (a story in itself) and the only practical way to use it was to get Sun's IMAP server which only ran on Solaris. So what I was getting at is can we use something smaller/lighter that we can include in the project for this? What is it required to do? The tests require an HTTP server, an FTP server and a SOCKS server to be listening on certain (configurable) ports. The respective servers need to have certain resources - text files, jars etc - deployed on them as a number of the java.net.* tests try and access them from the configured locations. The tests check that the expected content is available, that it is the expected size, contains the expected data etc etc. There are a number of mature, stable, freely available HTTP/FTP/SOCKS servers out there that will do just great for satisfying the demands of these tests so I don't propose to recommend anything in particular (although I have had no problems running Apache HTTPD on my local Windows machine). I don't see any justification for someone to spend time developing a new server to run these tests. But, hey, it's not my time that will be wasted... geir Stepan Mishura wrote: Hi Richard, Yes, I'm always for simplifying testing routine. And creating a detail guide for configuring some testing server is a possible way to do it. However I'd prefer to try to develop a simple local test server that is easy to configure. I believe that it will cover most of cases. And I assume that there may be a number of sophisticated test cases that requires a special server set up. But I'd place them separately this was discussed in '[testing] code for exotic configurations' topic. Otherwise there is a chance to get the following detailed instructions: 1) Download http server from http://www.apache.org ... 50) Do this and that ... 100) Now you can run net tests Thanks, Stepan On 2/22/06, Richard Liang [EMAIL PROTECTED] wrote: Dears, Agree that server (maybe not local) is required for some sophisticated test cases. To make thing simply, we may provide more detail guide (step-by-step) on how to setup/configure a typical testing server. Or shall we setup a test server and allow user to share our testing server? Richard Liang
Re: local test server (Was: Re: [jira] Commented: (HARMONY-71) java.net.URLConnection.setUseCaches throws unspecified IllegalAccessError)
Mark Hindess wrote: I think it might help this disagreement if we step back and decide what scenarios for running tests we are trying to optimise. Personally, whenever I write tests I'm doing it to optimise the scenario where a new users comes to the project and does: 1$ svn co ... classlib # or wget/tar if you prefer 2$ cd classlib/make 3$ ant test I do this because I want the tests to be easy to run by new users particularly on new platforms. I hope making it this easy means we get more people running tests and that we get a broader set of results than we could acheive ourselves. This scenario becomes slightly less pleasant if, between steps 2 and 3, [*] you have to find your nearest Linux machine and install Apache httpd, your-favourite-ftp-server, Dante socks server, etc. This typically means that I'd be inclined to write stub servers to test against. But this doesn't mean George is wrong! Because *if* there was a publically accessible Internet server that already had Apache httpd, twoftpd (my favourite ftp server this week), Dante socks, etc, then the scenario I like to optimise becomes possible. The effort of setting up one hosting server is definitely cheaper than the effort of implementing stubs - I know because I set up the server George tests against and it didn't take much time at all. While I don't object to having such publically accessible services (though I would understand if ASF did), that cannot be the required set-up for the tests. We need to maintain the ability to test locally (can you imagine Geir's Skyfone charges otherwise ;-) ) Having one server means we wont get 1000's of users asking Apache httpd, twoftpd, Dante socks configuration questions on our mailing lists. It also means we have a way to see the other half of the results - that is, the server logs. (Stubs should make this easier and this cost should be considered too but if we ran the server we could make this easier.) George's approach continues to be cheaper even if a few groups have to set up there own servers - though I think the set of users who don't have Internet access but do have a mechanism for getting up to date Harmony code should be small and getting smaller. Of course, it becomes much more expensive if everyone has to do it. The size of development 'group' sharing the test server resource can be 1 upwards, so while sharing makes sense it should not be required. Regards, Tim I've never tried using George's approach on past projects because I've either also been writing server code, that can/should be used by tests, or because the server was massive and I didn't have the experience or resources to run a real server. Because of this, initially, I was inclined to support the implemention of stubs, but now I'm not so certain. I think it basically comes down to whether or not we can provide a central server. I'm not sure it's a simple question but for me this question is key to this issue. Or perhaps I'm optimising for the wrong scenario? Regards, Mark. [*] More likely people would run step 3 find it goes horribly wrong then read the README they ignored initially and discover why. ;-) -- Tim Ellison ([EMAIL PROTECTED]) IBM Java technology centre, UK.
Re: local test server (Was: Re: [jira] Commented: (HARMONY-71) java.net.URLConnection.setUseCaches throws unspecified IllegalAccessError)
Hi Richard, Yes, I'm always for simplifying testing routine. And creating a detail guide for configuring some testing server is a possible way to do it. However I'd prefer to try to develop a simple local test server that is easy to configure. I believe that it will cover most of cases. And I assume that there may be a number of sophisticated test cases that requires a special server set up. But I'd place them separately this was discussed in '[testing] code for exotic configurations' topic. Otherwise there is a chance to get the following detailed instructions: 1) Download http server from http://www.apache.org ... 50) Do this and that ... 100) Now you can run net tests Thanks, Stepan On 2/22/06, Richard Liang [EMAIL PROTECTED] wrote: Dears, Agree that server (maybe not local) is required for some sophisticated test cases. To make thing simply, we may provide more detail guide (step-by-step) on how to setup/configure a typical testing server. Or shall we setup a test server and allow user to share our testing server? Richard Liang China Software Development Lab, IBM Tim Ellison wrote: Stepan Mishura wrote: I glanced through a local server code from HARMONY-57 contribution without looking into implementation details and ... this made me think. I catch hold of the following: 1) The server contains some testing code that looks not good for me Care to explain? 2) It is not as simple as I expected - my impression was that a local server implementation should be very simple because we are not going to test whether it corresponds to http spec. or not. IMHO, it should only provide a test case with: - a port number to open communication - access to a server's input/output streams to verify a request to a server or to form a server's response. Naive? Well, I'm aware that I'm not so experienced in java.netdevelopment and testing and may be I'm wrong. The local server I referenced can be used for a number of the java.net tests, some of which do require more sophisticated behavior, like basic authentication, eTags, etc. I agree that the tests in HARMONY-71 only need something to accept their connection, but I was simply pointing out that there will be a local server we can use if that contribution is accepted into the project, so probably not worth writing another. I'd like to start with the first issue (BTW, is it ok to discuss details of contribution implementation if it is not accepted yet?). Sure -- I would expect people to be evaluating it thoroughly so that they can vote the code in/out. The PPMC will shout early if the paperwork implies a contribution is unacceptable. Can we avoid adding a code that is specific for some test case to a local server? Does it interfere with what you want to do? In some cases, the test case needs to know what is on the server a priori, so that it can test the client behavior correctly. Regards, Tim On 2/16/06, Tim Ellison wrote: I would rather that you just opened a new JIRA issue for that as it doesn't relate to the original problem. FYI there is a local server incoming in the HARMONY-57 contribution, so don't spend any time on it, see: HARMONY-57\Harmony\Harmony_Tests\src\test\java\tests\support\Support_HttpServer.java Regards, Tim Stepan Mishura (JIRA) wrote: [ http://issues.apache.org/jira/browse/HARMONY-71?page=comments#action_12366637 ] Stepan Mishura commented on HARMONY-71: --- Tim, Added test case (test_setUseCachesZ) depends on network and it fails if it can not create connection with apache.org. The same for HARMONY-72. I'm going to attach a patch file with a local server in it. Could you reopen this JIRA? Thanks, Stepan. java.net.URLConnection.setUseCaches throws unspecified IllegalAccessError -- Key: HARMONY-71 URL: http://issues.apache.org/jira/browse/HARMONY-71 Project: Harmony Type: Bug Components: Classlib Reporter: Svetlana Samoilenko Assignee: Tim Ellison According to j2se 1.4.2 specification method java.net.URLConnection.setUseCaches(boolean) throws IllegalStateException, if already connected. Harmony throws java.lang.IllegalAccessErrorinstead, that contradicts the specification. Code to reproduce: import java.io.IOException; import java.net.*; public class test2 { public static void main(String[] args) { HttpURLConnection u=null; try { u=(HttpURLConnection)(new URL(http://intel.com ).openConnection()); u.connect(); } catch (MalformedURLException e) { System.out.println(unexpected MalformedURLException+e); } catch (IOException f) {
Re: local test server (Was: Re: [jira] Commented: (HARMONY-71) java.net.URLConnection.setUseCaches throws unspecified IllegalAccessError)
Stepan Mishura wrote: On 2/21/06, Tim Ellison [EMAIL PROTECTED] wrote: Stepan Mishura wrote: I glanced through a local server code from HARMONY-57 contribution without looking into implementation details and ... this made me think. I catch hold of the following: 1) The server contains some testing code that looks not good for me Care to explain? Support_HttpServer.java has the following code: class ServerThread implements Runnable { public void run() { // call the test function based on the requested resource if (resourceName.equals(CHUNKEDTEST)) chunkedTest(); ... lot of 'else if' ... ... } private void chunkedTest() { ... construct server's respond and send it ... } I'd prefer to avoid this and to have a server implementation that provides a way for unit test to check received request and to configure a server's respond. and I thought you wanted it simple ;-) There is no way to force a server to send you a chunked response using regular HTTP headers, so in this case the server and client have an understanding that when the client asks for a particular resource the server will send it back in chunks. If that understanding was encoded differently, say in the search field of the URL rather than the path would that make you happier? do you have some other idea? snip Does it interfere with what you want to do? Sure, we have javax.security.auth.kerberos.KerberosTicket class that currently contains a stub. I was thinking about implementing the stub and how to test it. So I played a little bit with developing Kerberos stub server. When I saw that tests for URLConnection class failed because they need local stub server then I got interested how to solve this similar issue. Not clear how that interferes. You should feel free to reuse the server code and extend it as you need to do a faux Kerberos server. Regards, Tim -- Tim Ellison ([EMAIL PROTECTED]) IBM Java technology centre, UK.
Re: local test server (Was: Re: [jira] Commented: (HARMONY-71) java.net.URLConnection.setUseCaches throws unspecified IllegalAccessError)
So, does this summarize to : 1) We need a new server or we should enahnce the current one? 2) We need instructions on how to do this. ? (I hope we don't need httpd to test Harmony...) Stepan Mishura wrote: Hi Richard, Yes, I'm always for simplifying testing routine. And creating a detail guide for configuring some testing server is a possible way to do it. However I'd prefer to try to develop a simple local test server that is easy to configure. I believe that it will cover most of cases. And I assume that there may be a number of sophisticated test cases that requires a special server set up. But I'd place them separately this was discussed in '[testing] code for exotic configurations' topic. Otherwise there is a chance to get the following detailed instructions: 1) Download http server from http://www.apache.org ... 50) Do this and that ... 100) Now you can run net tests Thanks, Stepan On 2/22/06, Richard Liang [EMAIL PROTECTED] wrote: Dears, Agree that server (maybe not local) is required for some sophisticated test cases. To make thing simply, we may provide more detail guide (step-by-step) on how to setup/configure a typical testing server. Or shall we setup a test server and allow user to share our testing server? Richard Liang China Software Development Lab, IBM Tim Ellison wrote: Stepan Mishura wrote: I glanced through a local server code from HARMONY-57 contribution without looking into implementation details and ... this made me think. I catch hold of the following: 1) The server contains some testing code that looks not good for me Care to explain? 2) It is not as simple as I expected - my impression was that a local server implementation should be very simple because we are not going to test whether it corresponds to http spec. or not. IMHO, it should only provide a test case with: - a port number to open communication - access to a server's input/output streams to verify a request to a server or to form a server's response. Naive? Well, I'm aware that I'm not so experienced in java.netdevelopment and testing and may be I'm wrong. The local server I referenced can be used for a number of the java.net tests, some of which do require more sophisticated behavior, like basic authentication, eTags, etc. I agree that the tests in HARMONY-71 only need something to accept their connection, but I was simply pointing out that there will be a local server we can use if that contribution is accepted into the project, so probably not worth writing another. I'd like to start with the first issue (BTW, is it ok to discuss details of contribution implementation if it is not accepted yet?). Sure -- I would expect people to be evaluating it thoroughly so that they can vote the code in/out. The PPMC will shout early if the paperwork implies a contribution is unacceptable. Can we avoid adding a code that is specific for some test case to a local server? Does it interfere with what you want to do? In some cases, the test case needs to know what is on the server a priori, so that it can test the client behavior correctly. Regards, Tim On 2/16/06, Tim Ellison wrote: I would rather that you just opened a new JIRA issue for that as it doesn't relate to the original problem. FYI there is a local server incoming in the HARMONY-57 contribution, so don't spend any time on it, see: HARMONY-57\Harmony\Harmony_Tests\src\test\java\tests\support\Support_HttpServer.java Regards, Tim Stepan Mishura (JIRA) wrote: [ http://issues.apache.org/jira/browse/HARMONY-71?page=comments#action_12366637 ] Stepan Mishura commented on HARMONY-71: --- Tim, Added test case (test_setUseCachesZ) depends on network and it fails if it can not create connection with apache.org. The same for HARMONY-72. I'm going to attach a patch file with a local server in it. Could you reopen this JIRA? Thanks, Stepan. java.net.URLConnection.setUseCaches throws unspecified IllegalAccessError -- Key: HARMONY-71 URL: http://issues.apache.org/jira/browse/HARMONY-71 Project: Harmony Type: Bug Components: Classlib Reporter: Svetlana Samoilenko Assignee: Tim Ellison According to j2se 1.4.2 specification method java.net.URLConnection.setUseCaches(boolean) throws IllegalStateException, if already connected. Harmony throws java.lang.IllegalAccessErrorinstead, that contradicts the specification. Code to reproduce: import java.io.IOException; import java.net.*; public class test2 { public static void main(String[] args) { HttpURLConnection u=null; try { u=(HttpURLConnection)(new URL(http://intel.com ).openConnection()); u.connect(); } catch (MalformedURLException e) { System.out.println(unexpected MalformedURLException+e); } catch (IOException f) {
Re: local test server (Was: Re: [jira] Commented: (HARMONY-71) java.net.URLConnection.setUseCaches throws unspecified IllegalAccessError)
Geir Magnusson Jr wrote: So, does this summarize to : 1) We need a new server or we should enahnce the current one? 2) We need instructions on how to do this. ? (I hope we don't need httpd to test Harmony...) Could you remind me what the current one is ? What is the problem with using httpd to test Harmony ? Stepan Mishura wrote: Hi Richard, Yes, I'm always for simplifying testing routine. And creating a detail guide for configuring some testing server is a possible way to do it. However I'd prefer to try to develop a simple local test server that is easy to configure. I believe that it will cover most of cases. And I assume that there may be a number of sophisticated test cases that requires a special server set up. But I'd place them separately this was discussed in '[testing] code for exotic configurations' topic. Otherwise there is a chance to get the following detailed instructions: 1) Download http server from http://www.apache.org ... 50) Do this and that ... 100) Now you can run net tests Thanks, Stepan On 2/22/06, Richard Liang [EMAIL PROTECTED] wrote: Dears, Agree that server (maybe not local) is required for some sophisticated test cases. To make thing simply, we may provide more detail guide (step-by-step) on how to setup/configure a typical testing server. Or shall we setup a test server and allow user to share our testing server? Richard Liang China Software Development Lab, IBM Tim Ellison wrote: Stepan Mishura wrote: I glanced through a local server code from HARMONY-57 contribution without looking into implementation details and ... this made me think. I catch hold of the following: 1) The server contains some testing code that looks not good for me Care to explain? 2) It is not as simple as I expected - my impression was that a local server implementation should be very simple because we are not going to test whether it corresponds to http spec. or not. IMHO, it should only provide a test case with: - a port number to open communication - access to a server's input/output streams to verify a request to a server or to form a server's response. Naive? Well, I'm aware that I'm not so experienced in java.netdevelopment and testing and may be I'm wrong. The local server I referenced can be used for a number of the java.net tests, some of which do require more sophisticated behavior, like basic authentication, eTags, etc. I agree that the tests in HARMONY-71 only need something to accept their connection, but I was simply pointing out that there will be a local server we can use if that contribution is accepted into the project, so probably not worth writing another. I'd like to start with the first issue (BTW, is it ok to discuss details of contribution implementation if it is not accepted yet?). Sure -- I would expect people to be evaluating it thoroughly so that they can vote the code in/out. The PPMC will shout early if the paperwork implies a contribution is unacceptable. Can we avoid adding a code that is specific for some test case to a local server? Does it interfere with what you want to do? In some cases, the test case needs to know what is on the server a priori, so that it can test the client behavior correctly. Regards, Tim On 2/16/06, Tim Ellison wrote: I would rather that you just opened a new JIRA issue for that as it doesn't relate to the original problem. FYI there is a local server incoming in the HARMONY-57 contribution, so don't spend any time on it, see: HARMONY-57\Harmony\Harmony_Tests\src\test\java\tests\support\Support_HttpServer.java Regards, Tim Stepan Mishura (JIRA) wrote: [ http://issues.apache.org/jira/browse/HARMONY-71?page=comments#action_12366637 ] Stepan Mishura commented on HARMONY-71: --- Tim, Added test case (test_setUseCachesZ) depends on network and it fails if it can not create connection with apache.org. The same for HARMONY-72. I'm going to attach a patch file with a local server in it. Could you reopen this JIRA? Thanks, Stepan. java.net.URLConnection.setUseCaches throws unspecified IllegalAccessError -- Key: HARMONY-71 URL: http://issues.apache.org/jira/browse/HARMONY-71 Project: Harmony Type: Bug Components: Classlib Reporter: Svetlana Samoilenko Assignee: Tim Ellison According to j2se 1.4.2 specification method java.net.URLConnection.setUseCaches(boolean) throws IllegalStateException, if already connected. Harmony throws java.lang.IllegalAccessErrorinstead, that contradicts the specification. Code to reproduce: import java.io.IOException; import java.net.*; public class test2 { public static void main(String[] args) { HttpURLConnection u=null; try { u=(HttpURLConnection)(new URL(http://intel.com ).openConnection()); u.connect();
Re: local test server (Was: Re: [jira] Commented: (HARMONY-71) java.net.URLConnection.setUseCaches throws unspecified IllegalAccessError)
George Harley wrote: Geir Magnusson Jr wrote: George Harley wrote: Geir Magnusson Jr wrote: So, does this summarize to : 1) We need a new server or we should enahnce the current one? 2) We need instructions on how to do this. ? (I hope we don't need httpd to test Harmony...) Could you remind me what the current one is ? The one in the pending JIRA There is no server in the pending JIRA. (psst) take a look at HARMONY-57\Harmony\Harmony_Tests\src\test\java\tests\support\Support_HttpServer.java Regards, Tim As briefly mentioned somewhere else in this thread, the submitted zip for that JIRA contains configuration code and properties files that permits the tester to use any server in any location (including localhost) and test resources for deploying on those servers. What is the problem with using httpd to test Harmony ? It's a rather large external dependency. Reminds me of the time we got the TCK for JavaMail (a story in itself) and the only practical way to use it was to get Sun's IMAP server which only ran on Solaris. So what I was getting at is can we use something smaller/lighter that we can include in the project for this? What is it required to do? The tests require an HTTP server, an FTP server and a SOCKS server to be listening on certain (configurable) ports. The respective servers need to have certain resources - text files, jars etc - deployed on them as a number of the java.net.* tests try and access them from the configured locations. The tests check that the expected content is available, that it is the expected size, contains the expected data etc etc. There are a number of mature, stable, freely available HTTP/FTP/SOCKS servers out there that will do just great for satisfying the demands of these tests so I don't propose to recommend anything in particular (although I have had no problems running Apache HTTPD on my local Windows machine). I don't see any justification for someone to spend time developing a new server to run these tests. But, hey, it's not my time that will be wasted... geir Stepan Mishura wrote: Hi Richard, Yes, I'm always for simplifying testing routine. And creating a detail guide for configuring some testing server is a possible way to do it. However I'd prefer to try to develop a simple local test server that is easy to configure. I believe that it will cover most of cases. And I assume that there may be a number of sophisticated test cases that requires a special server set up. But I'd place them separately this was discussed in '[testing] code for exotic configurations' topic. Otherwise there is a chance to get the following detailed instructions: 1) Download http server from http://www.apache.org ... 50) Do this and that ... 100) Now you can run net tests Thanks, Stepan On 2/22/06, Richard Liang [EMAIL PROTECTED] wrote: Dears, Agree that server (maybe not local) is required for some sophisticated test cases. To make thing simply, we may provide more detail guide (step-by-step) on how to setup/configure a typical testing server. Or shall we setup a test server and allow user to share our testing server? Richard Liang China Software Development Lab, IBM Tim Ellison wrote: Stepan Mishura wrote: I glanced through a local server code from HARMONY-57 contribution without looking into implementation details and ... this made me think. I catch hold of the following: 1) The server contains some testing code that looks not good for me Care to explain? 2) It is not as simple as I expected - my impression was that a local server implementation should be very simple because we are not going to test whether it corresponds to http spec. or not. IMHO, it should only provide a test case with: - a port number to open communication - access to a server's input/output streams to verify a request to a server or to form a server's response. Naive? Well, I'm aware that I'm not so experienced in java.netdevelopment and testing and may be I'm wrong. The local server I referenced can be used for a number of the java.net tests, some of which do require more sophisticated behavior, like basic authentication, eTags, etc. I agree that the tests in HARMONY-71 only need something to accept their connection, but I was simply pointing out that there will be a local server we can use if that contribution is accepted into the project, so probably not worth writing another. I'd like to start with the first issue (BTW, is it ok to discuss details of contribution implementation if it is not accepted yet?). Sure -- I would expect people to be evaluating it thoroughly so that they can vote the code in/out. The PPMC will shout early if the paperwork implies a contribution is unacceptable. Can we avoid adding a code that is specific for some test case to a local server? Does it interfere with what you want to do? In some cases, the test
Re: local test server (Was: Re: [jira] Commented: (HARMONY-71) java.net.URLConnection.setUseCaches throws unspecified IllegalAccessError)
Tim Ellison wrote: George Harley wrote: Geir Magnusson Jr wrote: George Harley wrote: Geir Magnusson Jr wrote: So, does this summarize to : 1) We need a new server or we should enahnce the current one? 2) We need instructions on how to do this. ? (I hope we don't need httpd to test Harmony...) Could you remind me what the current one is ? The one in the pending JIRA There is no server in the pending JIRA. (psst) take a look at HARMONY-57\Harmony\Harmony_Tests\src\test\java\tests\support\Support_HttpServer.java Regards, Tim Hi Tim, Funny. That's a test support class that is used to mock up an HTTP server for one, admittedly large, test case method (test_getInputStream in tests.api.java.net.URLConnectionTest). It is not the server. There are a significantly larger number of test case methods that rely on using the configured external HTTP/FTP/SOCKS servers to test out their functionality. Best regards, George IBM UK As briefly mentioned somewhere else in this thread, the submitted zip for that JIRA contains configuration code and properties files that permits the tester to use any server in any location (including localhost) and test resources for deploying on those servers. What is the problem with using httpd to test Harmony ? It's a rather large external dependency. Reminds me of the time we got the TCK for JavaMail (a story in itself) and the only practical way to use it was to get Sun's IMAP server which only ran on Solaris. So what I was getting at is can we use something smaller/lighter that we can include in the project for this? What is it required to do? The tests require an HTTP server, an FTP server and a SOCKS server to be listening on certain (configurable) ports. The respective servers need to have certain resources - text files, jars etc - deployed on them as a number of the java.net.* tests try and access them from the configured locations. The tests check that the expected content is available, that it is the expected size, contains the expected data etc etc. There are a number of mature, stable, freely available HTTP/FTP/SOCKS servers out there that will do just great for satisfying the demands of these tests so I don't propose to recommend anything in particular (although I have had no problems running Apache HTTPD on my local Windows machine). I don't see any justification for someone to spend time developing a new server to run these tests. But, hey, it's not my time that will be wasted... geir Stepan Mishura wrote: Hi Richard, Yes, I'm always for simplifying testing routine. And creating a detail guide for configuring some testing server is a possible way to do it. However I'd prefer to try to develop a simple local test server that is easy to configure. I believe that it will cover most of cases. And I assume that there may be a number of sophisticated test cases that requires a special server set up. But I'd place them separately this was discussed in '[testing] code for exotic configurations' topic. Otherwise there is a chance to get the following detailed instructions: 1) Download http server from http://www.apache.org ... 50) Do this and that ... 100) Now you can run net tests Thanks, Stepan On 2/22/06, Richard Liang [EMAIL PROTECTED] wrote: Dears, Agree that server (maybe not local) is required for some sophisticated test cases. To make thing simply, we may provide more detail guide (step-by-step) on how to setup/configure a typical testing server. Or shall we setup a test server and allow user to share our testing server? Richard Liang China Software Development Lab, IBM Tim Ellison wrote: Stepan Mishura wrote: I glanced through a local server code from HARMONY-57 contribution without looking into implementation details and ... this made me think. I catch hold of the following: 1) The server contains some testing code that looks not good for me Care to explain? 2) It is not as simple as I expected - my impression was that a local server implementation should be very simple because we are not going to test whether it corresponds to http spec. or not. IMHO, it should only provide a test case with: - a port number to open communication - access to a server's input/output streams to verify a request to a server or to form a server's response. Naive? Well, I'm aware that I'm not so experienced in java.netdevelopment and testing and may be I'm wrong. The local server I referenced can be used for a number of the java.net tests, some of which do require more sophisticated behavior, like basic authentication, eTags, etc. I agree
Re: local test server (Was: Re: [jira] Commented: (HARMONY-71) java.net.URLConnection.setUseCaches throws unspecified IllegalAccessError)
I feel that I need to study more details about net testing. Also to made discussion more concrete and constructive I'll to try to provide 'a proof of concept'. I believe that this won't be a wasted effort and it won't take too much time. I'm going to select a number of 'common testing scenarios', for example, I will try to force a server to send a chunked response without creating a special resource to be requested. And demonstrate that it is possible to run them without external server or sophisticated internal server. Anyway I was going to continue to work with Kerberos server stub (because I'm not delighted with configuring external Kerberos server in order to run unit tests) and this is a good change for me to review this issue from another angle. Thanks, Stepan Mishura Intel Middleware Products Division On 2/23/06, George Harley george.c.harley at googlemail.com wrote: Tim Ellison wrote: George Harley wrote: Geir Magnusson Jr wrote: George Harley wrote: Geir Magnusson Jr wrote: So, does this summarize to : 1) We need a new server or we should enahnce the current one? 2) We need instructions on how to do this. ? (I hope we don't need httpd to test Harmony...) Could you remind me what the current one is ? The one in the pending JIRA There is no server in the pending JIRA. (psst) take a look at HARMONY-57\Harmony\Harmony_Tests\src\test\java\tests\support\Support_HttpServer.java Regards, Tim Hi Tim, Funny. That's a test support class that is used to mock up an HTTP server for one, admittedly large, test case method (test_getInputStream in tests.api.java.net.URLConnectionTest). It is not the server. There are a significantly larger number of test case methods that rely on using the configured external HTTP/FTP/SOCKS servers to test out their functionality. Best regards, George IBM UK As briefly mentioned somewhere else in this thread, the submitted zip for that JIRA contains configuration code and properties files that permits the tester to use any server in any location (including localhost) and test resources for deploying on those servers. What is the problem with using httpd to test Harmony ? It's a rather large external dependency. Reminds me of the time we got the TCK for JavaMail (a story in itself) and the only practical way to use it was to get Sun's IMAP server which only ran on Solaris. So what I was getting at is can we use something smaller/lighter that we can include in the project for this? What is it required to do? The tests require an HTTP server, an FTP server and a SOCKS server to be listening on certain (configurable) ports. The respective servers need to have certain resources - text files, jars etc - deployed on them as a number of the java.net.* tests try and access them from the configured locations. The tests check that the expected content is available, that it is the expected size, contains the expected data etc etc. There are a number of mature, stable, freely available HTTP/FTP/SOCKS servers out there that will do just great for satisfying the demands of these tests so I don't propose to recommend anything in particular (although I have had no problems running Apache HTTPD on my local Windows machine). I don't see any justification for someone to spend time developing a new server to run these tests. But, hey, it's not my time that will be wasted... geir Stepan Mishura wrote: Hi Richard, Yes, I'm always for simplifying testing routine. And creating a detail guide for configuring some testing server is a possible way to do it. However I'd prefer to try to develop a simple local test server that is easy to configure. I believe that it will cover most of cases. And I assume that there may be a number of sophisticated test cases that requires a special server set up. But I'd place them separately this was discussed in '[testing] code for exotic configurations' topic. Otherwise there is a chance to get the following detailed instructions: 1) Download http server from http://www.apache.org ... 50) Do this and that ... 100) Now you can run net tests Thanks, Stepan On 2/22/06, Richard Liang [EMAIL PROTECTED] wrote: Dears, Agree that server (maybe not local) is required for some sophisticated test cases. To make thing simply, we may provide more detail guide (step-by-step) on how to setup/configure a typical testing server. Or shall we setup a test server and allow user to share our testing server? Richard Liang China Software Development Lab, IBM Tim Ellison wrote: Stepan Mishura wrote: I glanced through a local server code from HARMONY-57 contribution without looking into implementation details and ... this made me think. I catch hold of the following: 1) The server contains some testing code that
Re: local test server (Was: Re: [jira] Commented: (HARMONY-71) java.net.URLConnection.setUseCaches throws unspecified IllegalAccessError)
I think it might help this disagreement if we step back and decide what scenarios for running tests we are trying to optimise. Personally, whenever I write tests I'm doing it to optimise the scenario where a new users comes to the project and does: 1$ svn co ... classlib # or wget/tar if you prefer 2$ cd classlib/make 3$ ant test I do this because I want the tests to be easy to run by new users particularly on new platforms. I hope making it this easy means we get more people running tests and that we get a broader set of results than we could acheive ourselves. This scenario becomes slightly less pleasant if, between steps 2 and 3, [*] you have to find your nearest Linux machine and install Apache httpd, your-favourite-ftp-server, Dante socks server, etc. This typically means that I'd be inclined to write stub servers to test against. But this doesn't mean George is wrong! Because *if* there was a publically accessible Internet server that already had Apache httpd, twoftpd (my favourite ftp server this week), Dante socks, etc, then the scenario I like to optimise becomes possible. The effort of setting up one hosting server is definitely cheaper than the effort of implementing stubs - I know because I set up the server George tests against and it didn't take much time at all. Having one server means we wont get 1000's of users asking Apache httpd, twoftpd, Dante socks configuration questions on our mailing lists. It also means we have a way to see the other half of the results - that is, the server logs. (Stubs should make this easier and this cost should be considered too but if we ran the server we could make this easier.) George's approach continues to be cheaper even if a few groups have to set up there own servers - though I think the set of users who don't have Internet access but do have a mechanism for getting up to date Harmony code should be small and getting smaller. Of course, it becomes much more expensive if everyone has to do it. I've never tried using George's approach on past projects because I've either also been writing server code, that can/should be used by tests, or because the server was massive and I didn't have the experience or resources to run a real server. Because of this, initially, I was inclined to support the implemention of stubs, but now I'm not so certain. I think it basically comes down to whether or not we can provide a central server. I'm not sure it's a simple question but for me this question is key to this issue. Or perhaps I'm optimising for the wrong scenario? Regards, Mark. [*] More likely people would run step 3 find it goes horribly wrong then read the README they ignored initially and discover why. ;-)
Re: local test server (Was: Re: [jira] Commented: (HARMONY-71) java.net.URLConnection.setUseCaches throws unspecified IllegalAccessError)
Stepan Mishura wrote: I glanced through a local server code from HARMONY-57 contribution without looking into implementation details and ... this made me think. I catch hold of the following: 1) The server contains some testing code that looks not good for me Care to explain? 2) It is not as simple as I expected - my impression was that a local server implementation should be very simple because we are not going to test whether it corresponds to http spec. or not. IMHO, it should only provide a test case with: - a port number to open communication - access to a server's input/output streams to verify a request to a server or to form a server's response. Naive? Well, I'm aware that I'm not so experienced in java.net development and testing and may be I'm wrong. The local server I referenced can be used for a number of the java.net tests, some of which do require more sophisticated behavior, like basic authentication, eTags, etc. I agree that the tests in HARMONY-71 only need something to accept their connection, but I was simply pointing out that there will be a local server we can use if that contribution is accepted into the project, so probably not worth writing another. I'd like to start with the first issue (BTW, is it ok to discuss details of contribution implementation if it is not accepted yet?). Sure -- I would expect people to be evaluating it thoroughly so that they can vote the code in/out. The PPMC will shout early if the paperwork implies a contribution is unacceptable. Can we avoid adding a code that is specific for some test case to a local server? Does it interfere with what you want to do? In some cases, the test case needs to know what is on the server a priori, so that it can test the client behavior correctly. Regards, Tim On 2/16/06, Tim Ellison wrote: I would rather that you just opened a new JIRA issue for that as it doesn't relate to the original problem. FYI there is a local server incoming in the HARMONY-57 contribution, so don't spend any time on it, see: HARMONY-57\Harmony\Harmony_Tests\src\test\java\tests\support\Support_HttpServer.java Regards, Tim Stepan Mishura (JIRA) wrote: [ http://issues.apache.org/jira/browse/HARMONY-71?page=comments#action_12366637] Stepan Mishura commented on HARMONY-71: --- Tim, Added test case (test_setUseCachesZ) depends on network and it fails if it can not create connection with apache.org. The same for HARMONY-72. I'm going to attach a patch file with a local server in it. Could you reopen this JIRA? Thanks, Stepan. java.net.URLConnection.setUseCaches throws unspecified IllegalAccessError -- Key: HARMONY-71 URL: http://issues.apache.org/jira/browse/HARMONY-71 Project: Harmony Type: Bug Components: Classlib Reporter: Svetlana Samoilenko Assignee: Tim Ellison According to j2se 1.4.2 specification method java.net.URLConnection.setUseCaches(boolean) throws IllegalStateException, if already connected. Harmony throws java.lang.IllegalAccessError instead, that contradicts the specification. Code to reproduce: import java.io.IOException; import java.net.*; public class test2 { public static void main(String[] args) { HttpURLConnection u=null; try { u=(HttpURLConnection)(new URL(http://intel.com ).openConnection()); u.connect(); } catch (MalformedURLException e) { System.out.println(unexpected MalformedURLException+e); } catch (IOException f) { System.out.println(unexpected IOException+f); } try { u.setUseCaches(true); } catch (IllegalStateException e) { System.out.println(OK. Expected IllegalStateException); e.printStackTrace(); }; } } Steps to Reproduce: 1. Build Harmony (check-out on 2006-01-30) j2se subset as described in README.txt. 2. Compile test2.java using BEA 1.4 javac javac -d . test2.java 3. Run java using compatible VM (J9) java -showversion test2 Output: C:\tmpC:\jrockit-j2sdk1.4.2_04\bin\java.exe -showversion test2 java version 1.4.2_04 Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_04-b05) BEA WebLogic JRockit(TM) 1.4.2_04 JVM (build ari-31788-20040616-1132-win-ia32, Native Threads, GC strategy: parallel) OK. Expected IllegalStateException java.lang.IllegalStateException: Already connected at java.net.URLConnection.setUseCaches(Z)V(URLConnection.java :828) at test2.main([Ljava.lang.String;)V(test2.java:17) C:\tmpC:\harmony\trunk\deploy\jre\bin\java -showversion test2 (c) Copyright 1991, 2005 The Apache Software Foundation or its licensors, as applicable. java.lang.IllegalAccessError: Connection already established at java.net.URLConnection.setUseCaches(URLConnection.java:923)
Re: local test server (Was: Re: [jira] Commented: (HARMONY-71) java.net.URLConnection.setUseCaches throws unspecified IllegalAccessError)
Dears, Agree that server (maybe not local) is required for some sophisticated test cases. To make thing simply, we may provide more detail guide (step-by-step) on how to setup/configure a typical testing server. Or shall we setup a test server and allow user to share our testing server? Richard Liang China Software Development Lab, IBM Tim Ellison wrote: Stepan Mishura wrote: I glanced through a local server code from HARMONY-57 contribution without looking into implementation details and ... this made me think. I catch hold of the following: 1) The server contains some testing code that looks not good for me Care to explain? 2) It is not as simple as I expected - my impression was that a local server implementation should be very simple because we are not going to test whether it corresponds to http spec. or not. IMHO, it should only provide a test case with: - a port number to open communication - access to a server's input/output streams to verify a request to a server or to form a server's response. Naive? Well, I'm aware that I'm not so experienced in java.net development and testing and may be I'm wrong. The local server I referenced can be used for a number of the java.net tests, some of which do require more sophisticated behavior, like basic authentication, eTags, etc. I agree that the tests in HARMONY-71 only need something to accept their connection, but I was simply pointing out that there will be a local server we can use if that contribution is accepted into the project, so probably not worth writing another. I'd like to start with the first issue (BTW, is it ok to discuss details of contribution implementation if it is not accepted yet?). Sure -- I would expect people to be evaluating it thoroughly so that they can vote the code in/out. The PPMC will shout early if the paperwork implies a contribution is unacceptable. Can we avoid adding a code that is specific for some test case to a local server? Does it interfere with what you want to do? In some cases, the test case needs to know what is on the server a priori, so that it can test the client behavior correctly. Regards, Tim On 2/16/06, Tim Ellison wrote: I would rather that you just opened a new JIRA issue for that as it doesn't relate to the original problem. FYI there is a local server incoming in the HARMONY-57 contribution, so don't spend any time on it, see: HARMONY-57\Harmony\Harmony_Tests\src\test\java\tests\support\Support_HttpServer.java Regards, Tim Stepan Mishura (JIRA) wrote: [ http://issues.apache.org/jira/browse/HARMONY-71?page=comments#action_12366637] Stepan Mishura commented on HARMONY-71: --- Tim, Added test case (test_setUseCachesZ) depends on network and it fails if it can not create connection with apache.org. The same for HARMONY-72. I'm going to attach a patch file with a local server in it. Could you reopen this JIRA? Thanks, Stepan. java.net.URLConnection.setUseCaches throws unspecified IllegalAccessError -- Key: HARMONY-71 URL: http://issues.apache.org/jira/browse/HARMONY-71 Project: Harmony Type: Bug Components: Classlib Reporter: Svetlana Samoilenko Assignee: Tim Ellison According to j2se 1.4.2 specification method java.net.URLConnection.setUseCaches(boolean) throws IllegalStateException, if already connected. Harmony throws java.lang.IllegalAccessError instead, that contradicts the specification. Code to reproduce: import java.io.IOException; import java.net.*; public class test2 { public static void main(String[] args) { HttpURLConnection u=null; try { u=(HttpURLConnection)(new URL(http://intel.com ).openConnection()); u.connect(); } catch (MalformedURLException e) { System.out.println(unexpected MalformedURLException+e); } catch (IOException f) { System.out.println(unexpected IOException+f); } try { u.setUseCaches(true); } catch (IllegalStateException e) { System.out.println(OK. Expected IllegalStateException); e.printStackTrace(); }; } } Steps to Reproduce: 1. Build Harmony (check-out on 2006-01-30) j2se subset as described in README.txt. 2. Compile test2.java using BEA 1.4 javac javac -d . test2.java 3. Run java using compatible VM (J9) java -showversion test2 Output: C:\tmpC:\jrockit-j2sdk1.4.2_04\bin\java.exe -showversion test2 java version 1.4.2_04 Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_04-b05) BEA WebLogic JRockit(TM) 1.4.2_04 JVM (build ari-31788-20040616-1132-win-ia32, Native Threads, GC strategy:
Re: local test server (Was: Re: [jira] Commented: (HARMONY-71) java.net.URLConnection.setUseCaches throws unspecified IllegalAccessError)
Richard Liang wrote: Dears, Agree that server (maybe not local) is required for some sophisticated test cases. To make thing simply, we may provide more detail guide (step-by-step) on how to setup/configure a typical testing server. Or shall we setup a test server and allow user to share our testing server? I think that instructions for local setup is best. Or better, an ant script :) Richard Liang China Software Development Lab, IBM Tim Ellison wrote: Stepan Mishura wrote: I glanced through a local server code from HARMONY-57 contribution without looking into implementation details and ... this made me think. I catch hold of the following: 1) The server contains some testing code that looks not good for me Care to explain? 2) It is not as simple as I expected - my impression was that a local server implementation should be very simple because we are not going to test whether it corresponds to http spec. or not. IMHO, it should only provide a test case with: - a port number to open communication - access to a server's input/output streams to verify a request to a server or to form a server's response. Naive? Well, I'm aware that I'm not so experienced in java.net development and testing and may be I'm wrong. The local server I referenced can be used for a number of the java.net tests, some of which do require more sophisticated behavior, like basic authentication, eTags, etc. I agree that the tests in HARMONY-71 only need something to accept their connection, but I was simply pointing out that there will be a local server we can use if that contribution is accepted into the project, so probably not worth writing another. I'd like to start with the first issue (BTW, is it ok to discuss details of contribution implementation if it is not accepted yet?). Sure -- I would expect people to be evaluating it thoroughly so that they can vote the code in/out. The PPMC will shout early if the paperwork implies a contribution is unacceptable. Can we avoid adding a code that is specific for some test case to a local server? Does it interfere with what you want to do? In some cases, the test case needs to know what is on the server a priori, so that it can test the client behavior correctly. Regards, Tim On 2/16/06, Tim Ellison wrote: I would rather that you just opened a new JIRA issue for that as it doesn't relate to the original problem. FYI there is a local server incoming in the HARMONY-57 contribution, so don't spend any time on it, see: HARMONY-57\Harmony\Harmony_Tests\src\test\java\tests\support\Support_HttpServer.java Regards, Tim Stepan Mishura (JIRA) wrote: [ http://issues.apache.org/jira/browse/HARMONY-71?page=comments#action_12366637] Stepan Mishura commented on HARMONY-71: --- Tim, Added test case (test_setUseCachesZ) depends on network and it fails if it can not create connection with apache.org. The same for HARMONY-72. I'm going to attach a patch file with a local server in it. Could you reopen this JIRA? Thanks, Stepan. java.net.URLConnection.setUseCaches throws unspecified IllegalAccessError -- Key: HARMONY-71 URL: http://issues.apache.org/jira/browse/HARMONY-71 Project: Harmony Type: Bug Components: Classlib Reporter: Svetlana Samoilenko Assignee: Tim Ellison According to j2se 1.4.2 specification method java.net.URLConnection.setUseCaches(boolean) throws IllegalStateException, if already connected. Harmony throws java.lang.IllegalAccessError instead, that contradicts the specification. Code to reproduce: import java.io.IOException; import java.net.*; public class test2 { public static void main(String[] args) { HttpURLConnection u=null; try { u=(HttpURLConnection)(new URL(http://intel.com ).openConnection()); u.connect(); } catch (MalformedURLException e) { System.out.println(unexpected MalformedURLException+e); } catch (IOException f) { System.out.println(unexpected IOException+f); } try { u.setUseCaches(true); } catch (IllegalStateException e) { System.out.println(OK. Expected IllegalStateException); e.printStackTrace(); }; } } Steps to Reproduce: 1. Build Harmony (check-out on 2006-01-30) j2se subset as described in README.txt. 2. Compile test2.java using BEA 1.4 javac javac -d . test2.java 3. Run java using compatible VM (J9) java -showversion test2 Output: C:\tmpC:\jrockit-j2sdk1.4.2_04\bin\java.exe -showversion test2 java version 1.4.2_04 Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_04-b05) BEA
Re: local test server (Was: Re: [jira] Commented: (HARMONY-71) java.net.URLConnection.setUseCaches throws unspecified IllegalAccessError)
On 2/21/06, Tim Ellison [EMAIL PROTECTED] wrote: Stepan Mishura wrote: I glanced through a local server code from HARMONY-57 contribution without looking into implementation details and ... this made me think. I catch hold of the following: 1) The server contains some testing code that looks not good for me Care to explain? Support_HttpServer.java has the following code: class ServerThread implements Runnable { public void run() { // call the test function based on the requested resource if (resourceName.equals(CHUNKEDTEST)) chunkedTest(); ... lot of 'else if' ... ... } private void chunkedTest() { ... construct server's respond and send it ... } I'd prefer to avoid this and to have a server implementation that provides a way for unit test to check received request and to configure a server's respond. 2) It is not as simple as I expected - my impression was that a local server implementation should be very simple because we are not going to test whether it corresponds to http spec. or not. IMHO, it should only provide a test case with: - a port number to open communication - access to a server's input/output streams to verify a request to a server or to form a server's response. Naive? Well, I'm aware that I'm not so experienced in java.netdevelopment and testing and may be I'm wrong. The local server I referenced can be used for a number of the java.net tests, some of which do require more sophisticated behavior, like basic authentication, eTags, etc. I agree that the tests in HARMONY-71 only need something to accept their connection, but I was simply pointing out that there will be a local server we can use if that contribution is accepted into the project, so probably not worth writing another. I'd like to start with the first issue (BTW, is it ok to discuss details of contribution implementation if it is not accepted yet?). Sure -- I would expect people to be evaluating it thoroughly so that they can vote the code in/out. The PPMC will shout early if the paperwork implies a contribution is unacceptable. Can we avoid adding a code that is specific for some test case to a local server? Does it interfere with what you want to do? Sure, we have javax.security.auth.kerberos.KerberosTicket class that currently contains a stub. I was thinking about implementing the stub and how to test it. So I played a little bit with developing Kerberos stub server. When I saw that tests for URLConnection class failed because they need local stub server then I got interested how to solve this similar issue. Thanks, Stepan. In some cases, the test case needs to know what is on the server a priori, so that it can test the client behavior correctly. Regards, Tim On 2/16/06, Tim Ellison wrote: I would rather that you just opened a new JIRA issue for that as it doesn't relate to the original problem. FYI there is a local server incoming in the HARMONY-57 contribution, so don't spend any time on it, see: HARMONY-57\Harmony\Harmony_Tests\src\test\java\tests\support\Support_HttpServer.java Regards, Tim Stepan Mishura (JIRA) wrote: [ http://issues.apache.org/jira/browse/HARMONY-71?page=comments#action_12366637 ] Stepan Mishura commented on HARMONY-71: --- Tim, Added test case (test_setUseCachesZ) depends on network and it fails if it can not create connection with apache.org. The same for HARMONY-72. I'm going to attach a patch file with a local server in it. Could you reopen this JIRA? Thanks, Stepan. java.net.URLConnection.setUseCaches throws unspecified IllegalAccessError -- Key: HARMONY-71 URL: http://issues.apache.org/jira/browse/HARMONY-71 Project: Harmony Type: Bug Components: Classlib Reporter: Svetlana Samoilenko Assignee: Tim Ellison According to j2se 1.4.2 specification method java.net.URLConnection.setUseCaches(boolean) throws IllegalStateException, if already connected. Harmony throws java.lang.IllegalAccessErrorinstead, that contradicts the specification. Code to reproduce: import java.io.IOException; import java.net.*; public class test2 { public static void main(String[] args) { HttpURLConnection u=null; try { u=(HttpURLConnection)(new URL(http://intel.com ).openConnection()); u.connect(); } catch (MalformedURLException e) { System.out.println(unexpected MalformedURLException+e); } catch (IOException f) { System.out.println(unexpected IOException+f); } try { u.setUseCaches(true); } catch (IllegalStateException e) {