I've found the problem with a sample on top of wagon-http. The response is gzip'ed by the server but the Apache HTTP client or wagon-http does not unzip the stream -> parsing the response leads to no links detected !!!!
Jeff On Thu, Jun 13, 2013 at 2:52 PM, Olivier Lamy <[email protected]> wrote: > not sure you will be able to get that working with current plugins. > But wagon api has a method to get file list from a directory ( > > http://maven.apache.org/wagon/wagon-provider-api/apidocs/org/apache/maven/wagon/Wagon.html#getFileList(java.lang.String) > ) > > So "easy" to traverse a remote site tree and download all files (but > IMHO you will need to write your own plugin or improve one) > > 2013/6/13 Jeff MAURY <[email protected]>: > > Yes, but this should be converted by Maven as it is not possible to > specify > > a File in a POM. > > I'm working with a sample but I think this is related to the returned > HTML > > file format. > > > > Jeff > > > > > > On Thu, Jun 13, 2013 at 12:12 PM, Martin Gainty <[email protected]> > wrote: > > > >> Jeff download toDir is a file not a string > >> http://mojo.codehaus.org/wagon-maven-plugin/download-mojo.html > >> > >> try copy > >> http://mojo.codehaus.org/wagon-maven-plugin/usage.html > >> > >> Martin > >> ______________________________________________ > >> Verzicht und Vertraulichkeitanmerkung/Note de déni et de confidentialité > >> > >> Diese Nachricht ist vertraulich. Sollten Sie nicht der vorgesehene > >> Empfaenger sein, so bitten wir hoeflich um eine Mitteilung. Jede > unbefugte > >> Weiterleitung oder Fertigung einer Kopie ist unzulaessig. Diese > Nachricht > >> dient lediglich dem Austausch von Informationen und entfaltet keine > >> rechtliche Bindungswirkung. Aufgrund der leichten Manipulierbarkeit von > >> E-Mails koennen wir keine Haftung fuer den Inhalt uebernehmen. > >> > >> Ce message est confidentiel et peut être privilégié. Si vous n'êtes pas > le > >> destinataire prévu, nous te demandons avec bonté que pour satisfaire > >> informez l'expéditeur. N'importe quelle diffusion non autorisée ou la > copie > >> de ceci est interdite. Ce message sert à l'information seulement et > n'aura > >> pas n'importe quel effet légalement obligatoire. Étant donné que les > email > >> peuvent facilement être sujets à la manipulation, nous ne pouvons > accepter > >> aucune responsabilité pour le contenu fourni. > >> > >> > >> > From: [email protected] > >> > Date: Thu, 13 Jun 2013 10:55:39 +0200 > >> > Subject: Can't manage to download a site with wagon maven plugin > >> > To: [email protected] > >> > > >> > Hello, > >> > > >> > I want to setup my pom in order to dowload a remote web site through > >> HTTP. > >> > Here is my POM: > >> > > >> > <project xmlns="http://maven.apache.org/POM/4.0.0"; xmlns:xsi=" > >> > http://www.w3.org/2001/XMLSchema-instance"; > >> > xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 > >> > http://maven.apache.org/xsd/maven-4.0.0.xsd";> > >> > <modelVersion>4.0.0</modelVersion> > >> > > >> > <groupId>com.jeffmaury</groupId> > >> > <artifactId>wagon-http-test</artifactId> > >> > <version>0.0.1-SNAPSHOT</version> > >> > <packaging>jar</packaging> > >> > > >> > <name>wagon-http-test</name> > >> > <url>http://maven.apache.org</url> > >> > > >> > <properties> > >> > <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> > >> > </properties> > >> > > >> > <build> > >> > <extensions> > >> > <extension> > >> > <groupId>org.apache.maven.wagon</groupId> > >> > <artifactId>wagon-http</artifactId> > >> > <version>2.4</version> > >> > </extension> > >> > </extensions> > >> > <plugins> > >> > <plugin> > >> > <groupId>org.codehaus.mojo</groupId> > >> > <artifactId>wagon-maven-plugin</artifactId> > >> > <version>1.0-beta-4</version> > >> > <executions> > >> > <execution> > >> > <phase>generate-resources</phase> > >> > <goals> > >> > <goal>download</goal> > >> > </goals> > >> > </execution> > >> > </executions> > >> > <configuration> > >> > <url>http://www.syspertec.com</url> > >> > <fromDir>/</fromDir> > >> > <toDir>${project.outputDirectory}/test</toDir> > >> > <includes>**/*</includes> > >> > </configuration> > >> > </plugin> > >> > </plugins> > >> > </build> > >> > </project> > >> > > >> > Here is the output: > >> > > >> > [INFO] Scanning for projects... > >> > [INFO] > >> > [INFO] > >> > > ------------------------------------------------------------------------ > >> > [INFO] Building wagon-http-test 0.0.1-SNAPSHOT > >> > [INFO] > >> > > ------------------------------------------------------------------------ > >> > [INFO] > >> > *[INFO] --- wagon-maven-plugin:1.0-beta-4:download (default) @ > >> > wagon-http-test ---* > >> > *[INFO] Scanning remote file system: http://www.syspertec.com ...* > >> > *[INFO] Nothing to download.* > >> > [INFO] > >> > [INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ > >> > wagon-http-test --- > >> > [debug] execute contextualize > >> > [INFO] Using 'UTF-8' encoding to copy filtered resources. > >> > [INFO] skip non existing resourceDirectory > >> > d:\workspace\wagon-http-test\src\main\resources > >> > [INFO] > >> > [INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ > >> > wagon-http-test --- > >> > [INFO] Nothing to compile - all classes are up to date > >> > [INFO] > >> > > ------------------------------------------------------------------------ > >> > [INFO] BUILD SUCCESS > >> > [INFO] > >> > > ------------------------------------------------------------------------ > >> > [INFO] Total time: 3.162s > >> > [INFO] Finished at: Thu Jun 13 10:53:50 CEST 2013 > >> > [INFO] Final Memory: 6M/73M > >> > [INFO] > >> > > ------------------------------------------------------------------------ > >> > /d/workspace/wagon-http-test > >> > > >> > Any help ? > >> > -- > >> > Jeff MAURY > >> > PS; I'm posting to Maven user list instead of Mojo because I suspect > this > >> > is related to wagon-http > >> > > >> > > >> > "Legacy code" often differs from its suggested alternative by actually > >> > working and scaling. > >> > - Bjarne Stroustrup > >> > > >> > http://www.jeffmaury.com > >> > http://riadiscuss.jeffmaury.com > >> > http://www.twitter.com/jeffmaury > >> > >> > > > > > > > > -- > > Jeff MAURY > > > > > > "Legacy code" often differs from its suggested alternative by actually > > working and scaling. > > - Bjarne Stroustrup > > > > http://www.jeffmaury.com > > http://riadiscuss.jeffmaury.com > > http://www.twitter.com/jeffmaury > > > > -- > Olivier Lamy > Ecetera: http://ecetera.com.au > http://twitter.com/olamy | http://linkedin.com/in/olamy > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > -- Jeff MAURY "Legacy code" often differs from its suggested alternative by actually working and scaling. - Bjarne Stroustrup http://www.jeffmaury.com http://riadiscuss.jeffmaury.com http://www.twitter.com/jeffmaury
