On 2 May 2013 17:34, Julien Plu <julien....@redaction-developpez.com> wrote:
>>Use the system properties http.proxyUser and http.proxyPassword.
>>Usually, system properties are problematic, but in this case, they're
>>ok: 1. we use system properties for the proxy settings anyway, 2. we
>>don't have to change the download config, and 3. we can more easily
>>reuse the solution elsewhere.
>
> I think I badly explained myself. You have to give your proxy logs, so you
> have to write them somewhere, and into the code is the worst solution ever,
> and if you pass them through the jvm args like "-Dhttp.proxyUser=xxx
> -Dhttp.proxyPassword=xxx" it doesn't works.
I know that simply setting these system properties doesn't help. You
need the Authenticator.setAuthenticator(...) code that we're
discussing here. The question is: how does that code get the username
and password?
One possibility is what you did - add config parameters, read them,
pass them to the code. That works, but wherever you want to enable
proxy authentication, you have to change the code that handles the
configuration.
Another possibility is: the code reads the system properties
http.proxyUser and http.proxyPassword:
String user = System.getProperty("http.proxyUser", "");
String password = System.getProperty("http.proxyPassword", "");
In Java, you can set and read whichever system property you want.
(Well, it depends on your security settings, but that doesn't matter
in our case.) These two properties are ignored by the Java classes
that handle proxy requests, that's correct. But that doesn't mean we
can't use them for our own purposes.
For more details, please see the code at
http://stackoverflow.com/questions/1626549/authenticated-http-proxy-with-java/16340273#16340273
I think that's the best and safest implementation of setting the
authenticator. There's more to this issue than it seems. If I
understand correctly, the Authenticator is used for *all* HTTP
requests, not just for proxy requests.
> I tested it. So a good solution
> is to write them into a property file.
The problem with this is that we have to change the configuration of
every piece of code in the framework that uses HTTP. When the code
that sets the authenticator reads the system properties, that's not
necessary.
System properties are often not a good choice for configuration,
mostly because they affect the whole system and there is no way for
different modules within one JVM to use different settings. But in our
case, the system is rather small - there are no sub-modules - and it's
very unlikely that we want different proxy settings.
Additionally, the user has to configure the proxy host and port in the
pom.xml anyway. I think it's a lot more intuitive to have
<jvmArg>-Dhttp.proxyHost=proxy.server.com</jvmArg>
<jvmArg>-Dhttp.proxyPort=80</jvmArg>
<jvmArg>-Dhttp.proxyUser=foo</jvmArg>
<jvmArg>-Dhttp.proxyPassword=bar</jvmArg>
in pom.xml than to have
<jvmArg>-Dhttp.proxyHost=proxy.server.com</jvmArg>
<jvmArg>-Dhttp.proxyPort=80</jvmArg>
in pom.xml and
proxyUser=foo
proxyPassword=bar
in some other configuration file.
As for the proxy settings for the extraction: I was wrong, you
probably do need them there too. We don't really make http calls, but
some code deep down in Java's XML handling probably does. It's silly
and we should turn it off, but that requires a bit of coding for which
we didn't have time yet.
http://sourceforge.net/mailarchive/message.php?msg_id=30590876
JC
>
> Next, create a "util" class is, indeed, a good way to reuse it easily.
>
> Best.
>
> Julien.
>
>
> 2013/5/2 Jona Christopher Sahnwaldt <j...@sahnwaldt.de>
>>
>> P.S.: ProxyAuthentication is a good class name as well. :-)
>>
>> On 2 May 2013 17:04, Jona Christopher Sahnwaldt <j...@sahnwaldt.de> wrote:
>> > On 2 May 2013 12:55, Julien Plu <julien....@redaction-developpez.com>
>> > wrote:
>> >> Ok, now that works like a charm. I added just two properties inside
>> >> "download.minimal.properties" file which are : "user" and "password"
>> >> and the
>> >> authentication is made only is these parameters are set. And I created
>> >> a
>> >> class "ProxyAuthentication.scala".
>> >>
>> >
>> > A few recommendations:
>> >
>> > - Use the system properties http.proxyUser and http.proxyPassword.
>> > Usually, system properties are problematic, but in this case, they're
>> > ok: 1. we use system properties for the proxy settings anyway, 2. we
>> > don't have to change the download config, and 3. we can more easily
>> > reuse the solution elsewhere. See below.
>> >
>> > - I just googled a bit and think we should make the implementation a
>> > bit safer:
>> > http://stackoverflow.com/questions/1626549/authenticated-http-proxy-with-java/16340273#16340273
>> >
>> > - Put the code in a new util class ProxyUtils.scala in the core module
>> > in the util package
>> >
>> > https://github.com/dbpedia/extraction-framework/tree/master/core/src/main/java/org/dbpedia/extraction/util
>> > , or you can just use ProxyUtils.java and create a new util package in
>> >
>> > https://github.com/dbpedia/extraction-framework/tree/master/core/src/main/java/org/dbpedia/extraction
>> > , then you won't have to port the code to Scala.
>> >
>> > We make HTTP calls in a few other places, so we will have to call the
>> > method from there as well, so we should have an implementation in the
>> > core module.
>> >
>> >> I have to do the same thing for "Extraction.scala" class or this class
>> >> doesn't need an internet connexion ?
>> >
>> > I'm pretty sure it doesn't. I don't think we make any requests during
>> > extraction.
>> >
>> > But there are other places. You can look in the pom.xml files, they
>> > should contain Scala launchers for all main classes that we have. For
>> > example, all the launchers in core/pom.xml download some stuff over
>> > HTTP, so if you want you can add a call to your new method in each of
>> > their main() methods. When your code uses the system properties, you
>> > just have to add one line: ProxyUtils.configureProxyAuthenticator();
>> >
>> > Cheers,
>> > JC
>> >
>> >
>> >
>> >>
>> >>
>> >> 2013/4/30 Julien Plu <julien....@redaction-developpez.com>
>> >>>
>> >>> That's a really good idea ! I will do it thursday because now I
>> >>> finished
>> >>> my working day and tomorrow is a national day off in France.
>> >>>
>> >>> I have also try to solve some bugs in the extraction with separators.
>> >>>
>> >>> Best.
>> >>>
>> >>> Julien.
>> >>>
>> >>>
>> >>> 2013/4/30 Jona Christopher Sahnwaldt <j...@sahnwaldt.de>
>> >>>>
>> >>>>
>> >>>>
>> >>>> https://github.com/dbpedia/extraction-framework/blob/master/dump/src/main/scala/org/dbpedia/extraction/dump/download/Download.scala
>> >>>>
>> >>>> It would be cool if you send a pull request when you are done! As a
>> >>>> first step, you can simply copy & paste the code from SO and
>> >>>> hard-code
>> >>>> your settings to test if that solves your problem at all, but it
>> >>>> would
>> >>>> be nice if you could make the code a little smarter so others can use
>> >>>> it: call System.getProperty("http.proxyUser") and "...Password" and
>> >>>> only install the authenticator if both are set. Good luck!
>> >>>>
>> >>>> On 30 April 2013 19:47, Julien Plu
>> >>>> <julien....@redaction-developpez.com>
>> >>>> wrote:
>> >>>> > Ok so apparently it's normal that it's doesn't works with command
>> >>>> > line,
>> >>>> > I
>> >>>> > have to modify the main. So which file contains the main function ?
>> >>>> >
>> >>>> > Best.
>> >>>> >
>> >>>> > Julien.
>> >>>> >
>> >>>> >
>> >>>> > 2013/4/30 Jona Christopher Sahnwaldt <j...@sahnwaldt.de>
>> >>>> >>
>> >>>> >> Maybe you have to tweak the code a bit:
>> >>>> >>
>> >>>> >>
>> >>>> >>
>> >>>> >> http://stackoverflow.com/questions/1626549/authenticated-http-proxy-with-java
>> >>>> >>
>> >>>> >> On 30 April 2013 19:28, Julien Plu
>> >>>> >> <julien....@redaction-developpez.com>
>> >>>> >> wrote:
>> >>>> >> > Doesn't works :-(
>> >>>> >> >
>> >>>> >> > Best
>> >>>> >> >
>> >>>> >> > Julien.
>> >>>> >> >
>> >>>> >> >
>> >>>> >> > 2013/4/30 Jona Christopher Sahnwaldt <j...@sahnwaldt.de>
>> >>>> >> >>
>> >>>> >> >> Hi Julien,
>> >>>> >> >>
>> >>>> >> >> try adding the settings for https as well, e.g.
>> >>>> >> >>
>> >>>> >> >> <jvmArg>-Dhttps.proxyHost=myproxyhost</jvmArg>
>> >>>> >> >>
>> >>>> >> >> Might help, I'm not sure.
>> >>>> >> >>
>> >>>> >> >> JC
>> >>>> >> >>
>> >>>> >> >> On 30 April 2013 16:57, Julien Plu
>> >>>> >> >> <julien....@redaction-developpez.com>
>> >>>> >> >> wrote:
>> >>>> >> >> > Hi,
>> >>>> >> >> >
>> >>>> >> >> > I try to use the extraction framework on my Windows machine,
>> >>>> >> >> > but
>> >>>> >> >> > I
>> >>>> >> >> > fall
>> >>>> >> >> > on a
>> >>>> >> >> > 407 http proxy authentication error during the dumps
>> >>>> >> >> > download. I
>> >>>> >> >> > put
>> >>>> >> >> > these
>> >>>> >> >> > lines inside the dump/pom.xml :
>> >>>> >> >> >
>> >>>> >> >> > <jvmArg>-Dhttp.proxyHost=myproxyhost</jvmArg>
>> >>>> >> >> > <jvmArg>-Dhttp.proxyPort=port</jvmArg>
>> >>>> >> >> > <jvmArg>-Dhttp.proxyUser=myuser</jvmArg>
>> >>>> >> >> > <jvmArg>-Dhttp.proxyPassword=mypass</jvmArg>
>> >>>> >> >> >
>> >>>> >> >> > Even this :
>> >>>> >> >> >
>> >>>> >> >> > <jvmArg>-Djava.net.useSystemProxies=true</jvmArg>
>> >>>> >> >> >
>> >>>> >> >> > Doesn't work.
>> >>>> >> >> >
>> >>>> >> >> > There is something more to use ?
>> >>>> >> >> >
>> >>>> >> >> > Best.
>> >>>> >> >> >
>> >>>> >> >> > Julien.
>> >>>> >> >> >
>> >>>> >> >> >
>> >>>> >> >> >
>> >>>> >> >> >
>> >>>> >> >> >
>> >>>> >> >> > ------------------------------------------------------------------------------
>> >>>> >> >> > Introducing AppDynamics Lite, a free troubleshooting tool for
>> >>>> >> >> > Java/.NET
>> >>>> >> >> > Get 100% visibility into your production application - at no
>> >>>> >> >> > cost.
>> >>>> >> >> > Code-level diagnostics for performance bottlenecks with <2%
>> >>>> >> >> > overhead
>> >>>> >> >> > Download for free and get started troubleshooting in minutes.
>> >>>> >> >> > http://p.sf.net/sfu/appdyn_d2d_ap1
>> >>>> >> >> > _______________________________________________
>> >>>> >> >> > Dbpedia-discussion mailing list
>> >>>> >> >> > Dbpedia-discussion@lists.sourceforge.net
>> >>>> >> >> >
>> >>>> >> >> > https://lists.sourceforge.net/lists/listinfo/dbpedia-discussion
>> >>>> >> >> >
>> >>>> >> >
>> >>>> >> >
>> >>>> >
>> >>>> >
>> >>>
>> >>>
>> >>
>
>
------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite
It's a free troubleshooting tool designed for production
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://p.sf.net/sfu/appdyn_d2d_ap2
_______________________________________________
Dbpedia-discussion mailing list
Dbpedia-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dbpedia-discussion
