On 2 May 2013 20:58, Julien Plu <julien....@redaction-developpez.com> wrote:
> Ok, so if I well understood what you mean is writing this inside the
> dump/pom.xml :
>
> <jvmArg>-Dhttp.proxyHost=proxy.server.com</jvmArg>
> <jvmArg>-Dhttp.proxyPort=80</jvmArg>
> <jvmArg>-Dhttp.proxyUser=foo</jvmArg>
> <jvmArg>-Dhttp.proxyPassword=bar</jvmArg>
>
> And retrieve the value of http.proxyUser and http.proxyPassword with this :
>
> String user = System.getProperty("http.proxyUser", "");
> String password = System.getProperty("http.proxyPassword", "");
>
> For using them inside the ProxyAuthenticator class. If it's that so ok I'm
> totally agree with, and I will modify my code for taking this way :-)Exactly. :-) As I said, you should look at the code at http://stackoverflow.com/questions/1626549/authenticated-http-proxy-with-java/16340273#16340273 There are some security issues. Basically, you can just copy that code, or port it to Scala if you want. > > Best. > > Julien. > > > 2013/5/2 Jona Christopher Sahnwaldt <j...@sahnwaldt.de> >> >> On 2 May 2013 17:34, Julien Plu <julien....@redaction-developpez.com> >> wrote: >> >>Use the system properties http.proxyUser and http.proxyPassword. >> >>Usually, system properties are problematic, but in this case, they're >> >>ok: 1. we use system properties for the proxy settings anyway, 2. we >> >>don't have to change the download config, and 3. we can more easily >> >>reuse the solution elsewhere. >> > >> > I think I badly explained myself. You have to give your proxy logs, so >> > you >> > have to write them somewhere, and into the code is the worst solution >> > ever, >> > and if you pass them through the jvm args like "-Dhttp.proxyUser=xxx >> > -Dhttp.proxyPassword=xxx" it doesn't works. >> >> I know that simply setting these system properties doesn't help. You >> need the Authenticator.setAuthenticator(...) code that we're >> discussing here. The question is: how does that code get the username >> and password? >> >> One possibility is what you did - add config parameters, read them, >> pass them to the code. That works, but wherever you want to enable >> proxy authentication, you have to change the code that handles the >> configuration. >> >> Another possibility is: the code reads the system properties >> http.proxyUser and http.proxyPassword: >> >> String user = System.getProperty("http.proxyUser", ""); >> String password = System.getProperty("http.proxyPassword", ""); >> >> In Java, you can set and read whichever system property you want. >> (Well, it depends on your security settings, but that doesn't matter >> in our case.) These two properties are ignored by the Java classes >> that handle proxy requests, that's correct. But that doesn't mean we >> can't use them for our own purposes. >> >> For more details, please see the code at >> >> >> http://stackoverflow.com/questions/1626549/authenticated-http-proxy-with-java/16340273#16340273 >> >> I think that's the best and safest implementation of setting the >> authenticator. There's more to this issue than it seems. If I >> understand correctly, the Authenticator is used for *all* HTTP >> requests, not just for proxy requests. >> >> > I tested it. So a good solution >> > is to write them into a property file. >> >> The problem with this is that we have to change the configuration of >> every piece of code in the framework that uses HTTP. When the code >> that sets the authenticator reads the system properties, that's not >> necessary. >> >> System properties are often not a good choice for configuration, >> mostly because they affect the whole system and there is no way for >> different modules within one JVM to use different settings. But in our >> case, the system is rather small - there are no sub-modules - and it's >> very unlikely that we want different proxy settings. >> >> Additionally, the user has to configure the proxy host and port in the >> pom.xml anyway. I think it's a lot more intuitive to have >> >> <jvmArg>-Dhttp.proxyHost=proxy.server.com</jvmArg> >> <jvmArg>-Dhttp.proxyPort=80</jvmArg> >> <jvmArg>-Dhttp.proxyUser=foo</jvmArg> >> <jvmArg>-Dhttp.proxyPassword=bar</jvmArg> >> >> in pom.xml than to have >> >> <jvmArg>-Dhttp.proxyHost=proxy.server.com</jvmArg> >> <jvmArg>-Dhttp.proxyPort=80</jvmArg> >> >> in pom.xml and >> >> proxyUser=foo >> proxyPassword=bar >> >> in some other configuration file. >> >> As for the proxy settings for the extraction: I was wrong, you >> probably do need them there too. We don't really make http calls, but >> some code deep down in Java's XML handling probably does. It's silly >> and we should turn it off, but that requires a bit of coding for which >> we didn't have time yet. >> >> http://sourceforge.net/mailarchive/message.php?msg_id=30590876 >> >> JC >> >> > >> > Next, create a "util" class is, indeed, a good way to reuse it easily. >> > >> > Best. >> > >> > Julien. >> > >> > >> > 2013/5/2 Jona Christopher Sahnwaldt <j...@sahnwaldt.de> >> >> >> >> P.S.: ProxyAuthentication is a good class name as well. :-) >> >> >> >> On 2 May 2013 17:04, Jona Christopher Sahnwaldt <j...@sahnwaldt.de> >> >> wrote: >> >> > On 2 May 2013 12:55, Julien Plu <julien....@redaction-developpez.com> >> >> > wrote: >> >> >> Ok, now that works like a charm. I added just two properties inside >> >> >> "download.minimal.properties" file which are : "user" and "password" >> >> >> and the >> >> >> authentication is made only is these parameters are set. And I >> >> >> created >> >> >> a >> >> >> class "ProxyAuthentication.scala". >> >> >> >> >> > >> >> > A few recommendations: >> >> > >> >> > - Use the system properties http.proxyUser and http.proxyPassword. >> >> > Usually, system properties are problematic, but in this case, they're >> >> > ok: 1. we use system properties for the proxy settings anyway, 2. we >> >> > don't have to change the download config, and 3. we can more easily >> >> > reuse the solution elsewhere. See below. >> >> > >> >> > - I just googled a bit and think we should make the implementation a >> >> > bit safer: >> >> > >> >> > http://stackoverflow.com/questions/1626549/authenticated-http-proxy-with-java/16340273#16340273 >> >> > >> >> > - Put the code in a new util class ProxyUtils.scala in the core >> >> > module >> >> > in the util package >> >> > >> >> > >> >> > https://github.com/dbpedia/extraction-framework/tree/master/core/src/main/java/org/dbpedia/extraction/util >> >> > , or you can just use ProxyUtils.java and create a new util package >> >> > in >> >> > >> >> > >> >> > https://github.com/dbpedia/extraction-framework/tree/master/core/src/main/java/org/dbpedia/extraction >> >> > , then you won't have to port the code to Scala. >> >> > >> >> > We make HTTP calls in a few other places, so we will have to call the >> >> > method from there as well, so we should have an implementation in the >> >> > core module. >> >> > >> >> >> I have to do the same thing for "Extraction.scala" class or this >> >> >> class >> >> >> doesn't need an internet connexion ? >> >> > >> >> > I'm pretty sure it doesn't. I don't think we make any requests during >> >> > extraction. >> >> > >> >> > But there are other places. You can look in the pom.xml files, they >> >> > should contain Scala launchers for all main classes that we have. For >> >> > example, all the launchers in core/pom.xml download some stuff over >> >> > HTTP, so if you want you can add a call to your new method in each of >> >> > their main() methods. When your code uses the system properties, you >> >> > just have to add one line: ProxyUtils.configureProxyAuthenticator(); >> >> > >> >> > Cheers, >> >> > JC >> >> > >> >> > >> >> > >> >> >> >> >> >> >> >> >> 2013/4/30 Julien Plu <julien....@redaction-developpez.com> >> >> >>> >> >> >>> That's a really good idea ! I will do it thursday because now I >> >> >>> finished >> >> >>> my working day and tomorrow is a national day off in France. >> >> >>> >> >> >>> I have also try to solve some bugs in the extraction with >> >> >>> separators. >> >> >>> >> >> >>> Best. >> >> >>> >> >> >>> Julien. >> >> >>> >> >> >>> >> >> >>> 2013/4/30 Jona Christopher Sahnwaldt <j...@sahnwaldt.de> >> >> >>>> >> >> >>>> >> >> >>>> >> >> >>>> >> >> >>>> https://github.com/dbpedia/extraction-framework/blob/master/dump/src/main/scala/org/dbpedia/extraction/dump/download/Download.scala >> >> >>>> >> >> >>>> It would be cool if you send a pull request when you are done! As >> >> >>>> a >> >> >>>> first step, you can simply copy & paste the code from SO and >> >> >>>> hard-code >> >> >>>> your settings to test if that solves your problem at all, but it >> >> >>>> would >> >> >>>> be nice if you could make the code a little smarter so others can >> >> >>>> use >> >> >>>> it: call System.getProperty("http.proxyUser") and "...Password" >> >> >>>> and >> >> >>>> only install the authenticator if both are set. Good luck! >> >> >>>> >> >> >>>> On 30 April 2013 19:47, Julien Plu >> >> >>>> <julien....@redaction-developpez.com> >> >> >>>> wrote: >> >> >>>> > Ok so apparently it's normal that it's doesn't works with >> >> >>>> > command >> >> >>>> > line, >> >> >>>> > I >> >> >>>> > have to modify the main. So which file contains the main >> >> >>>> > function ? >> >> >>>> > >> >> >>>> > Best. >> >> >>>> > >> >> >>>> > Julien. >> >> >>>> > >> >> >>>> > >> >> >>>> > 2013/4/30 Jona Christopher Sahnwaldt <j...@sahnwaldt.de> >> >> >>>> >> >> >> >>>> >> Maybe you have to tweak the code a bit: >> >> >>>> >> >> >> >>>> >> >> >> >>>> >> >> >> >>>> >> >> >> >>>> >> http://stackoverflow.com/questions/1626549/authenticated-http-proxy-with-java >> >> >>>> >> >> >> >>>> >> On 30 April 2013 19:28, Julien Plu >> >> >>>> >> <julien....@redaction-developpez.com> >> >> >>>> >> wrote: >> >> >>>> >> > Doesn't works :-( >> >> >>>> >> > >> >> >>>> >> > Best >> >> >>>> >> > >> >> >>>> >> > Julien. >> >> >>>> >> > >> >> >>>> >> > >> >> >>>> >> > 2013/4/30 Jona Christopher Sahnwaldt <j...@sahnwaldt.de> >> >> >>>> >> >> >> >> >>>> >> >> Hi Julien, >> >> >>>> >> >> >> >> >>>> >> >> try adding the settings for https as well, e.g. >> >> >>>> >> >> >> >> >>>> >> >> <jvmArg>-Dhttps.proxyHost=myproxyhost</jvmArg> >> >> >>>> >> >> >> >> >>>> >> >> Might help, I'm not sure. >> >> >>>> >> >> >> >> >>>> >> >> JC >> >> >>>> >> >> >> >> >>>> >> >> On 30 April 2013 16:57, Julien Plu >> >> >>>> >> >> <julien....@redaction-developpez.com> >> >> >>>> >> >> wrote: >> >> >>>> >> >> > Hi, >> >> >>>> >> >> > >> >> >>>> >> >> > I try to use the extraction framework on my Windows >> >> >>>> >> >> > machine, >> >> >>>> >> >> > but >> >> >>>> >> >> > I >> >> >>>> >> >> > fall >> >> >>>> >> >> > on a >> >> >>>> >> >> > 407 http proxy authentication error during the dumps >> >> >>>> >> >> > download. I >> >> >>>> >> >> > put >> >> >>>> >> >> > these >> >> >>>> >> >> > lines inside the dump/pom.xml : >> >> >>>> >> >> > >> >> >>>> >> >> > <jvmArg>-Dhttp.proxyHost=myproxyhost</jvmArg> >> >> >>>> >> >> > <jvmArg>-Dhttp.proxyPort=port</jvmArg> >> >> >>>> >> >> > <jvmArg>-Dhttp.proxyUser=myuser</jvmArg> >> >> >>>> >> >> > <jvmArg>-Dhttp.proxyPassword=mypass</jvmArg> >> >> >>>> >> >> > >> >> >>>> >> >> > Even this : >> >> >>>> >> >> > >> >> >>>> >> >> > <jvmArg>-Djava.net.useSystemProxies=true</jvmArg> >> >> >>>> >> >> > >> >> >>>> >> >> > Doesn't work. >> >> >>>> >> >> > >> >> >>>> >> >> > There is something more to use ? >> >> >>>> >> >> > >> >> >>>> >> >> > Best. >> >> >>>> >> >> > >> >> >>>> >> >> > Julien. >> >> >>>> >> >> > >> >> >>>> >> >> > >> >> >>>> >> >> > >> >> >>>> >> >> > >> >> >>>> >> >> > >> >> >>>> >> >> > >> >> >>>> >> >> > ------------------------------------------------------------------------------ >> >> >>>> >> >> > Introducing AppDynamics Lite, a free troubleshooting tool >> >> >>>> >> >> > for >> >> >>>> >> >> > Java/.NET >> >> >>>> >> >> > Get 100% visibility into your production application - at >> >> >>>> >> >> > no >> >> >>>> >> >> > cost. >> >> >>>> >> >> > Code-level diagnostics for performance bottlenecks with >> >> >>>> >> >> > <2% >> >> >>>> >> >> > overhead >> >> >>>> >> >> > Download for free and get started troubleshooting in >> >> >>>> >> >> > minutes. >> >> >>>> >> >> > http://p.sf.net/sfu/appdyn_d2d_ap1 >> >> >>>> >> >> > _______________________________________________ >> >> >>>> >> >> > Dbpedia-discussion mailing list >> >> >>>> >> >> > Dbpedia-discussion@lists.sourceforge.net >> >> >>>> >> >> > >> >> >>>> >> >> > >> >> >>>> >> >> > https://lists.sourceforge.net/lists/listinfo/dbpedia-discussion >> >> >>>> >> >> > >> >> >>>> >> > >> >> >>>> >> > >> >> >>>> > >> >> >>>> > >> >> >>> >> >> >>> >> >> >> >> > >> > > > ------------------------------------------------------------------------------ Get 100% visibility into Java/.NET code with AppDynamics Lite It's a free troubleshooting tool designed for production Get down to code-level detail for bottlenecks, with <2% overhead. Download for free and get started troubleshooting in minutes. http://p.sf.net/sfu/appdyn_d2d_ap2 _______________________________________________ Dbpedia-discussion mailing list Dbpedia-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dbpedia-discussion
