Wow. Sounds great, Claude! On Mon, Jan 2, 2017 at 2:49 AM, <cbris...@apache.org> wrote:
> Author: cbrisson > Date: Mon Jan 2 10:49:55 2017 > New Revision: 1776916 > > URL: http://svn.apache.org/viewvc?rev=1776916&view=rev > Log: > [tools] ImportTool and XmlTool reenginering, along with the addition of a > JsonTool. > > The XmlTool now uses the standard JRE parser rather than the external jdom > library. Thus, the velocity-tools-xml module has been removed, and XmlTool > has been incorporated to the generic and view modules. > > ImportTool and ImportSupport have been splitted between the generic and > the view packages, as detailed below. The terminology used by the > ImportSupport has been reviewed (use local/remote URL instead of > 'absolute'/'relative' URL). > > The generic.ImportSupport and view.ViewImportSupport classes give a basic > API for: > - reading a local resource FIRST from a File (generic version) or from > the webapp (view version), THEN, if not found from the classpath. > - fetching a local URL (only applicable for the view version), which > allows to mix several J2EE view technologies in the same container. > - fetching a remote URL (which is forbidden in safe mode for the view > version). > > generic.ImportTool and view.ImportTool expose read(String resource) and > fetch(String url) methods, based on the ImportSupport and ViewImportSupport > APIs. > > generic.XmlTool and view.XmlTool expose parse(String xml), read(String > xmlResource) and fetch(String url) methods, based on the ImportSupport and > ViewImportSupport APIs. Both accept the 'resource=<resource_path>' and > 'source=<url>' configuration parameters. > > generic.JsonTool and view.JsonTool expose parse(String json), read(String > jsonResource) and fetch(String url) methods, based on the ImportSupport and > ViewImportSupport APIs. Both accept the 'resource=<resource_path>' and > 'source=<url>' configuration parameters. > > Whenever no specific configuration source has been given, view.XmlTool and > view.JsonTool add the ability to parse the posted request body, if it > corresponds to an xml or json mime type. > > Some remarks about the implementation: > - I choosed to shade the org.json library, to avoid another dependency. > - The XmlUtils class provides a static pool of xml parsers along with > several xml utilities > - feel free to make alternate proposals for the tools API and > configuration ; in particular, for the the "resource=<local resource>" and > "source=<url>" terminology. > > Happy new year, by the way. > > > Added: > velocity/tools/trunk/velocity-tools-examples/velocity-tools- > examples-showcase/src/main/webapp/json.vm > velocity/tools/trunk/velocity-tools-examples/velocity-tools- > examples-showcase/src/main/webapp/post_json.vm > velocity/tools/trunk/velocity-tools-generic/src/main/java/ > org/apache/velocity/tools/XmlUtils.java > velocity/tools/trunk/velocity-tools-generic/src/main/java/ > org/apache/velocity/tools/generic/ImportSupport.java > velocity/tools/trunk/velocity-tools-generic/src/main/java/ > org/apache/velocity/tools/generic/ImportTool.java > - copied, changed from r1770862, velocity/tools/trunk/velocity- > tools-view/src/main/java/org/apache/velocity/tools/view/ImportTool.java > velocity/tools/trunk/velocity-tools-generic/src/main/java/ > org/apache/velocity/tools/generic/JsonTool.java > velocity/tools/trunk/velocity-tools-generic/src/main/java/ > org/apache/velocity/tools/generic/XmlTool.java > velocity/tools/trunk/velocity-tools-generic/src/test/java/ > org/apache/velocity/tools/generic/JsonToolTests.java > - copied, changed from r1770862, velocity/tools/trunk/velocity- > tools-generic/src/test/java/org/apache/velocity/tools/ > generic/ClassToolTests.java > velocity/tools/trunk/velocity-tools-generic/src/test/java/ > org/apache/velocity/tools/generic/XmlToolTests.java > velocity/tools/trunk/velocity-tools-generic/src/test/ > resources/file.xml > velocity/tools/trunk/velocity-tools-generic/src/test/ > resources/foo.json > velocity/tools/trunk/velocity-tools-view/src/main/java/org/ > apache/velocity/tools/view/JsonTool.java > velocity/tools/trunk/velocity-tools-view/src/main/java/org/ > apache/velocity/tools/view/ViewImportSupport.java > - copied, changed from r1776915, velocity/tools/trunk/velocity- > tools-view/src/main/java/org/apache/velocity/tools/view/ImportSupport.java > velocity/tools/trunk/velocity-tools-view/src/main/java/org/ > apache/velocity/tools/view/XmlTool.java > Removed: > velocity/tools/trunk/velocity-tools-view/src/main/java/org/ > apache/velocity/tools/view/ImportSupport.java > velocity/tools/trunk/velocity-tools-xml/ > Modified: > velocity/tools/trunk/pom.xml > velocity/tools/trunk/src/changes/changes.xml > velocity/tools/trunk/velocity-tools-assembly/pom.xml > velocity/tools/trunk/velocity-tools-assembly/src/main/assembly/all.xml > velocity/tools/trunk/velocity-tools-examples/velocity-tools- > examples-showcase/pom.xml > velocity/tools/trunk/velocity-tools-examples/velocity-tools- > examples-showcase/src/main/resources/resources.properties > velocity/tools/trunk/velocity-tools-examples/velocity-tools- > examples-showcase/src/main/webapp/collection.vm > velocity/tools/trunk/velocity-tools-examples/velocity-tools- > examples-showcase/src/main/webapp/toolmenu.vm > velocity/tools/trunk/velocity-tools-examples/velocity-tools- > examples-showcase/src/test/java/org/apache/velocity/ > examples/showcase/ViewToolsIT.java > velocity/tools/trunk/velocity-tools-generic/pom.xml > velocity/tools/trunk/velocity-tools-generic/src/main/java/ > org/apache/velocity/tools/ConversionUtils.java > velocity/tools/trunk/velocity-tools-generic/src/main/ > resources/org/apache/velocity/tools/generic/tools.xml > velocity/tools/trunk/velocity-tools-generic/src/test/java/ > org/apache/velocity/tools/test/whitebox/GenericToolsTests.java > velocity/tools/trunk/velocity-tools-uberjar/pom.xml > velocity/tools/trunk/velocity-tools-view/src/main/java/org/ > apache/velocity/tools/view/BrowserTool.java > velocity/tools/trunk/velocity-tools-view/src/main/java/org/ > apache/velocity/tools/view/ImportTool.java > velocity/tools/trunk/velocity-tools-view/src/main/resources/ > org/apache/velocity/tools/view/tools.xml > > Modified: velocity/tools/trunk/pom.xml > URL: http://svn.apache.org/viewvc/velocity/tools/trunk/pom.xml? > rev=1776916&r1=1776915&r2=1776916&view=diff > ============================================================ > ================== > --- velocity/tools/trunk/pom.xml (original) > +++ velocity/tools/trunk/pom.xml Mon Jan 2 10:49:55 2017 > @@ -128,7 +128,6 @@ > </scm> > <modules> > <module>velocity-tools-generic</module> > - <module>velocity-tools-xml</module> > <module>velocity-tools-view</module> > <module>velocity-tools-view-jsp</module> > <module>velocity-tools-uberjar</module> > > Modified: velocity/tools/trunk/src/changes/changes.xml > URL: http://svn.apache.org/viewvc/velocity/tools/trunk/src/ > changes/changes.xml?rev=1776916&r1=1776915&r2=1776916&view=diff > ============================================================ > ================== > --- velocity/tools/trunk/src/changes/changes.xml (original) > +++ velocity/tools/trunk/src/changes/changes.xml Mon Jan 2 10:49:55 2017 > @@ -26,6 +26,44 @@ > <body> > > <release version="3.0-SNAPSHOT" date="In Subversion"> > + <action type="add" dev="cbrisson"> > + ImportTool reenginering: > + <ul> > + <li>the ImportSupport utility class has been splitted between > o.a.v.generic.ImportSupport and o.a.v.view.ViewImportSupport</li> > + <li>the ImportTool now has a generic version (for remote URLs > import) and a view version (for local URLs import) which cannot use remote > URLs in safe mode</li> > + </ul> > + </action> > + <action type="add" dev="cbrisson"> > + XmlTool now uses the standard JRE XML parser instead of the > org.jdom API ; it is now in two flavors, the generic tools one and the view > tools one. The view tools flavor > + allows the parsing of http query xml post data. > + </action> > + <action type="add" dev="cbrisson"> > + Added a new JsonTool for parsing json. It is in two flavors, the > generic tools one and the view tools one. The view tools flavor > + allows the parsing of http query json post data. > + </action> > + <action type="add" dev="cbrisson"> > + Added an EscapeTool.unurl(String) unescaping method > + </action> > + <action type="add" dev="cbrisson"> > + Deprecated ConversionTool: date/time parsing and formatting > methods belong to DateTool, while number parsing > + and formatting methods belong to NumberTool ; toLocale() method > is now in LocaleConfig > + </action> > + <action type="add" dev="cbrisson"> > + Deprecated SortTool, and added a CollectionTool to gather lists > sorting/splitting methods > + </action> > + <action type="add" dev="cbrisson"> > + Deprecated MathTool number parsing methods, which are redundant > with NumberTool ones > + </action> > + <action type="add" dev="cbrisson"> > + Deprecated AlternateTool > + </action> > + <action type="add" dev="cbrisson"> > + DateTool reenginering: > + <ul> > + <li>added iso and iso_tz date/datetime standard formats</li> > + <li>added intl and intl_tz for human-readable international > format (time zone displayed by id)</li> > + </ul> > + </action> > <action type="fix" dev="cbrisson"> > use static logging in Tools classes > </action> > > Modified: velocity/tools/trunk/velocity-tools-assembly/pom.xml > URL: http://svn.apache.org/viewvc/velocity/tools/trunk/velocity- > tools-assembly/pom.xml?rev=1776916&r1=1776915&r2=1776916&view=diff > ============================================================ > ================== > --- velocity/tools/trunk/velocity-tools-assembly/pom.xml (original) > +++ velocity/tools/trunk/velocity-tools-assembly/pom.xml Mon Jan 2 > 10:49:55 2017 > @@ -73,13 +73,6 @@ > </dependency> > <dependency> > <groupId>org.apache.velocity</groupId> > - <artifactId>velocity-tools-xml</artifactId> > - <version>${project.version}</version> > - <type>jar</type> > - <scope>compile</scope> > - </dependency> > - <dependency> > - <groupId>org.apache.velocity</groupId> > <artifactId>velocity-tools-examples-simple</artifactId> > <version>${project.version}</version> > <type>war</type> > > Modified: velocity/tools/trunk/velocity-tools-assembly/src/main/ > assembly/all.xml > URL: http://svn.apache.org/viewvc/velocity/tools/trunk/velocity- > tools-assembly/src/main/assembly/all.xml?rev=1776916& > r1=1776915&r2=1776916&view=diff > ============================================================ > ================== > --- velocity/tools/trunk/velocity-tools-assembly/src/main/assembly/all.xml > (original) > +++ velocity/tools/trunk/velocity-tools-assembly/src/main/assembly/all.xml > Mon Jan 2 10:49:55 2017 > @@ -83,14 +83,6 @@ > </includes> > </fileSet> > <fileSet> > - <directory>../velocity-tools-xml</directory> > - <outputDirectory>src/velocity-tools-xml</outputDirectory> > - <includes> > - <include>pom.xml</include> > - <include>src/</include> > - </includes> > - </fileSet> > - <fileSet> > <directory>../velocity-tools-view</directory> > <outputDirectory>src/velocity-tools-view</outputDirectory> > <includes> > @@ -155,10 +147,6 @@ > <outputDirectory>docs/velocity-tools-generic</ > outputDirectory> > </fileSet> > <fileSet> > - <directory>../velocity-tools-xml/target/site</directory> > - <outputDirectory>docs/velocity-tools-xml</outputDirectory> > - </fileSet> > - <fileSet> > <directory>../velocity-tools-view/target/site</directory> > <outputDirectory>docs/velocity-tools-view</outputDirectory> > </fileSet> > > Modified: velocity/tools/trunk/velocity-tools-examples/velocity-tools- > examples-showcase/pom.xml > URL: http://svn.apache.org/viewvc/velocity/tools/trunk/velocity- > tools-examples/velocity-tools-examples-showcase/pom.xml?rev= > 1776916&r1=1776915&r2=1776916&view=diff > ============================================================ > ================== > --- > velocity/tools/trunk/velocity-tools-examples/velocity-tools-examples-showcase/pom.xml > (original) > +++ > velocity/tools/trunk/velocity-tools-examples/velocity-tools-examples-showcase/pom.xml > Mon Jan 2 10:49:55 2017 > @@ -63,7 +63,7 @@ > <properties> > <cargo.jvmargs> > -Xdebug > - -Xrunjdwp:transport=dt_socket, > server=y,suspend=n,address=5005 > + -Xrunjdwp:transport=dt_socket, > server=n,suspend=y,address=5005 > -Xnoagent > -Djava.compiler=NONE > </cargo.jvmargs> > @@ -126,12 +126,6 @@ > </dependency> > <dependency> > <groupId>org.apache.velocity</groupId> > - <artifactId>velocity-tools-xml</artifactId> > - <version>${project.version}</version> > - <scope>runtime</scope> > - </dependency> > - <dependency> > - <groupId>org.apache.velocity</groupId> > <artifactId>velocity-tools-view-jsp</artifactId> > <version>${project.version}</version> > <type>jar</type> > > Modified: velocity/tools/trunk/velocity-tools-examples/velocity-tools- > examples-showcase/src/main/resources/resources.properties > URL: http://svn.apache.org/viewvc/velocity/tools/trunk/velocity- > tools-examples/velocity-tools-examples-showcase/src/main/ > resources/resources.properties?rev=1776916&r1=1776915&r2=1776916&view=diff > ============================================================ > ================== > --- velocity/tools/trunk/velocity-tools-examples/velocity-tools- > examples-showcase/src/main/resources/resources.properties (original) > +++ velocity/tools/trunk/velocity-tools-examples/velocity-tools- > examples-showcase/src/main/resources/resources.properties Mon Jan 2 > 10:49:55 2017 > @@ -76,6 +76,7 @@ tools.esc = EscapeTool > tools.field = FieldTool > tools.include = IncludeTool > tools.import = ImportTool > +tools.json = JsonTool > tools.jsp = JSP taglib > tools.link = LinkTool > tools.lists = ListTool > @@ -164,6 +165,7 @@ class.isStrict = Returns <code>true</cod > class.supportsNewInstance = Returns <code>true</code> if a new instance > of the class being inspected can be created via $class.type.newInstance(). > class.toString = Returns the result of $class.type.toString(). > > +collection.intro = Collection tool which provides String splitting > methods and various collections sorting methods. > collection.getStringsDelimiter = Returns the delimiter to be used when > splitting a string. > collection.getStringsTrim = Returns whether or not to trim string > elements found when splitting a string. > collection.sort_Collection = Sort a collection. > @@ -340,6 +342,9 @@ the name parameter is returned. > include.find_StringString.param1 = ''demo.vm'' > include.find_StringString.param2 = ''fr'' > > +# json.vm resources > +json.intro = Json parsing tool > + > # loop.vm resources > loop.intro = This tool is a convenience tool to use with #foreach loops. > It wraps a \ > list to let you prematurely stop iterating, skip iterations, sync > iteration over \ > > Modified: velocity/tools/trunk/velocity-tools-examples/velocity-tools- > examples-showcase/src/main/webapp/collection.vm > URL: http://svn.apache.org/viewvc/velocity/tools/trunk/velocity- > tools-examples/velocity-tools-examples-showcase/src/main/ > webapp/collection.vm?rev=1776916&r1=1776915&r2=1776916&view=diff > ============================================================ > ================== > --- velocity/tools/trunk/velocity-tools-examples/velocity-tools- > examples-showcase/src/main/webapp/collection.vm (original) > +++ velocity/tools/trunk/velocity-tools-examples/velocity-tools- > examples-showcase/src/main/webapp/collection.vm Mon Jan 2 10:49:55 2017 > @@ -14,18 +14,17 @@ > ## KIND, either express or implied. See the License for the > ## specific language governing permissions and limitations > ## under the License. > -#title( 'ConversionTool' ) > +#title( 'CollectionTool' ) > <p> > $text.demo.thisPage.insert("#doclink( 'CollectionTool' true )"). > +</p> > +<p> > $text.collection.intro > </p> > > -## The demo.vm template expects the following values to be set > -#set( $toollink = $doclink ) > -#set( $toolname = 'collection' ) > -#set( $toolclass = $collection.class ) > -#set( $toolDemo = > -"" > -) > +#demoTableStart() > + > +#demo1( 'collection' 'split' 5 'split string' ) > +#demoCustom( "sorter.sort(['b','c','a'])" ) > > -#parse( 'demo.vm' ) > +</table> > > Added: velocity/tools/trunk/velocity-tools-examples/velocity-tools- > examples-showcase/src/main/webapp/json.vm > URL: http://svn.apache.org/viewvc/velocity/tools/trunk/velocity- > tools-examples/velocity-tools-examples-showcase/src/main/ > webapp/json.vm?rev=1776916&view=auto > ============================================================ > ================== > --- velocity/tools/trunk/velocity-tools-examples/velocity-tools- > examples-showcase/src/main/webapp/json.vm (added) > +++ velocity/tools/trunk/velocity-tools-examples/velocity-tools- > examples-showcase/src/main/webapp/json.vm Mon Jan 2 10:49:55 2017 > @@ -0,0 +1,29 @@ > +## Licensed to the Apache Software Foundation (ASF) under one > +## or more contributor license agreements. See the NOTICE file > +## distributed with this work for additional information > +## regarding copyright ownership. The ASF licenses this file > +## to you under the Apache License, Version 2.0 (the > +## "License"); you may not use this file except in compliance > +## with the License. You may obtain a copy of the License at > +## > +## http://www.apache.org/licenses/LICENSE-2.0 > +## > +## Unless required by applicable law or agreed to in writing, > +## software distributed under the License is distributed on an > +## "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY > +## KIND, either express or implied. See the License for the > +## specific language governing permissions and limitations > +## under the License. > +#title( 'JsonTool' ) > +<p> > +$text.demo.thisPage.insert("#doclink( 'JsonTool' true )"). > +</p> > +<p> > +$text.json.intro > +</p> > + > +#demoTableStart() > + > +#demo1( 'json' 'parse' 5 'parse JSON string' ) > + > +</table> > > Added: velocity/tools/trunk/velocity-tools-examples/velocity-tools- > examples-showcase/src/main/webapp/post_json.vm > URL: http://svn.apache.org/viewvc/velocity/tools/trunk/velocity- > tools-examples/velocity-tools-examples-showcase/src/main/ > webapp/post_json.vm?rev=1776916&view=auto > ============================================================ > ================== > --- velocity/tools/trunk/velocity-tools-examples/velocity-tools- > examples-showcase/src/main/webapp/post_json.vm (added) > +++ velocity/tools/trunk/velocity-tools-examples/velocity-tools- > examples-showcase/src/main/webapp/post_json.vm Mon Jan 2 10:49:55 2017 > @@ -0,0 +1,18 @@ > +## Licensed to the Apache Software Foundation (ASF) under one > +## or more contributor license agreements. See the NOTICE file > +## distributed with this work for additional information > +## regarding copyright ownership. The ASF licenses this file > +## to you under the Apache License, Version 2.0 (the > +## "License"); you may not use this file except in compliance > +## with the License. You may obtain a copy of the License at > +## > +## http://www.apache.org/licenses/LICENSE-2.0 > +## > +## Unless required by applicable law or agreed to in writing, > +## software distributed under the License is distributed on an > +## "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY > +## KIND, either express or implied. See the License for the > +## specific language governing permissions and limitations > +## under the License. > +#title( 'JsonTool test template' ) > +<span id="foo">$json.foo</span> > > Modified: velocity/tools/trunk/velocity-tools-examples/velocity-tools- > examples-showcase/src/main/webapp/toolmenu.vm > URL: http://svn.apache.org/viewvc/velocity/tools/trunk/velocity- > tools-examples/velocity-tools-examples-showcase/src/main/ > webapp/toolmenu.vm?rev=1776916&r1=1776915&r2=1776916&view=diff > ============================================================ > ================== > --- velocity/tools/trunk/velocity-tools-examples/velocity-tools- > examples-showcase/src/main/webapp/toolmenu.vm (original) > +++ velocity/tools/trunk/velocity-tools-examples/velocity-tools- > examples-showcase/src/main/webapp/toolmenu.vm Mon Jan 2 10:49:55 2017 > @@ -35,6 +35,7 @@ > #toolMenuItem( $llink 'field' ) > #toolMenuItem( $llink 'import' ) > #toolMenuItem( $llink 'include' ) > +#toolMenuItem( $llink 'json' ) > #toolMenuItem( $llink 'jsp' ) > #toolMenuItem( $llink 'link' ) > #toolMenuItem( $llink 'loop' ) > > Modified: velocity/tools/trunk/velocity-tools-examples/velocity-tools- > examples-showcase/src/test/java/org/apache/velocity/ > examples/showcase/ViewToolsIT.java > URL: http://svn.apache.org/viewvc/velocity/tools/trunk/velocity- > tools-examples/velocity-tools-examples-showcase/src/test/ > java/org/apache/velocity/examples/showcase/ViewToolsIT. > java?rev=1776916&r1=1776915&r2=1776916&view=diff > ============================================================ > ================== > --- velocity/tools/trunk/velocity-tools-examples/velocity-tools- > examples-showcase/src/test/java/org/apache/velocity/ > examples/showcase/ViewToolsIT.java (original) > +++ velocity/tools/trunk/velocity-tools-examples/velocity-tools- > examples-showcase/src/test/java/org/apache/velocity/ > examples/showcase/ViewToolsIT.java Mon Jan 2 10:49:55 2017 > @@ -24,11 +24,16 @@ import static org.junit.Assert.assertNot > import static org.junit.Assert.assertTrue; > import static org.junit.Assert.fail; > > +import java.io.ByteArrayInputStream; > import java.io.IOException; > +import java.io.InputStream; > +import java.io.InputStreamReader; > import java.io.PrintWriter; > +import java.nio.charset.Charset; > import java.util.regex.Matcher; > import java.util.regex.Pattern; > > +import com.meterware.httpunit.PostMethodWebRequest; > import org.junit.BeforeClass; > import org.junit.Test; > > @@ -143,7 +148,7 @@ public class ViewToolsIT { > */ > private WebResponse submitWithParam(WebResponse orig, String > formname, String paramname, String value) throws Exception { > WebForm form = orig.getFormWithName(formname); > - form.setParameter(paramname,value); > + form.setParameter(paramname, value); > return form.submit(); > } > > @@ -187,24 +192,24 @@ public class ViewToolsIT { > WebResponse resp = conv.getResponse(req); > > /* check that getThis() is a ViewToolContext instance */ > - checkTextStart(resp,"getThis()","org.apache.velocity.tools. > view.ViewToolContext"); > + checkTextStart(resp, "getThis()", "org.apache.velocity.tools. > view.ViewToolContext"); > > /* check contains('context') */ > resp = submitWithParam(resp,"contains_Object","contains_ > Object1","'context'"); > - checkText(resp,"contains(java.lang.Object)","true"); > + checkText(resp, "contains(java.lang.Object)", "true"); > > /* check get('context') */ > - resp = submitWithParam(resp,"get_Object","get_Object1","' > context'"); > - checkTextStart(resp,"get(java.lang.Object)","org.apache. > velocity.tools.view.ViewContextTool"); > + resp = submitWithParam(resp, "get_Object", "get_Object1", > "'context'"); > + checkTextStart(resp, "get(java.lang.Object)", > "org.apache.velocity.tools.view.ViewContextTool"); > > /* check keys (the only expected uppercase is in 'velocityCount') > */ > - checkTextRegex(resp,"getKeys()","^\\[[a-z_A-Z]+(?:,\\s*[a-z_ > A-Z]+)*\\]$"); > + checkTextRegex(resp, "getKeys()", "^\\[[a-z_A-Z]+(?:,\\s*[a-z_A- > Z]+)*\\]$"); > > /* check toolbox */ > checkTextRegex(resp,"getToolbox()","^\\{[a-z_A-Z]+= > .*(?:,\\s*[a-z_A-Z]+=.*)*\\}$"); > > /* check values */ > - checkTextStartEnd(resp,"getValues()","[","]"); > + checkTextStartEnd(resp, "getValues()", "[", "]"); > } > > public @Test void testLinkTool() throws Exception { > @@ -316,4 +321,14 @@ public class ViewToolsIT { > /* check all */ > checkTextRegex(resp,"all","^\\{.*\\}$"); > } > + > + public @Test void testJsonTool() throws Exception > + { > + String json = "{\"foo\":\"bar\"}"; > + WebConversation conv = new WebConversation(); > + WebRequest req = new PostMethodWebRequest(ROOT_URL+"post_json.vm", > new ByteArrayInputStream(json.getBytes(Charset.forName("UTF-8"))), > "application/json"); > + WebResponse resp = conv.getResponse(req); > + checkText(resp, "foo", "bar"); > + } > + > } > > Modified: velocity/tools/trunk/velocity-tools-generic/pom.xml > URL: http://svn.apache.org/viewvc/velocity/tools/trunk/velocity- > tools-generic/pom.xml?rev=1776916&r1=1776915&r2=1776916&view=diff > ============================================================ > ================== > --- velocity/tools/trunk/velocity-tools-generic/pom.xml (original) > +++ velocity/tools/trunk/velocity-tools-generic/pom.xml Mon Jan 2 > 10:49:55 2017 > @@ -30,6 +30,38 @@ > <artifactId>velocity-tools-generic</artifactId> > <name>Apache Velocity Tools - Generic tools</name> > <description>Generic tools that can be used in any > context.</description> > + <build> > + <plugins> > + <plugin> > + <groupId>org.apache.maven.plugins</groupId> > + <artifactId>maven-shade-plugin</artifactId> > + <version>2.4.3</version> > + <executions> > + <execution> > + <id>shade</id> > + <phase>package</phase> > + <goals> > + <goal>shade</goal> > + </goals> > + <configuration> > + <artifactSet> > + <includes> > + <include>org.json</include> > + </includes> > + </artifactSet> > + <relocations> > + <relocation> > + <pattern>org.json</pattern> > + <shadedPattern>org.apache. > velocity.tools.shaded.org.json</shadedPattern> > + </relocation> > + </relocations> > + <minimizeJar>true</minimizeJar> > + </configuration> > + </execution> > + </executions> > + </plugin> > + </plugins> > + </build> > <dependencies> > <dependency> > <groupId>org.apache.velocity</groupId> > @@ -63,5 +95,10 @@ > <version>${slf4j.version}</version> > <scope>test</scope> > </dependency> > + <dependency> > + <groupId>org.json</groupId> > + <artifactId>json</artifactId> > + <version>20160810</version> > + </dependency> > </dependencies> > </project> > > Modified: velocity/tools/trunk/velocity-tools-generic/src/main/java/ > org/apache/velocity/tools/ConversionUtils.java > URL: http://svn.apache.org/viewvc/velocity/tools/trunk/velocity- > tools-generic/src/main/java/org/apache/velocity/tools/ > ConversionUtils.java?rev=1776916&r1=1776915&r2=1776916&view=diff > ============================================================ > ================== > --- velocity/tools/trunk/velocity-tools-generic/src/main/java/ > org/apache/velocity/tools/ConversionUtils.java (original) > +++ velocity/tools/trunk/velocity-tools-generic/src/main/java/ > org/apache/velocity/tools/ConversionUtils.java Mon Jan 2 10:49:55 2017 > @@ -813,7 +813,7 @@ public class ConversionUtils > * @see File > * @see ClassUtils#getResource(String,Object) > * @see URL > - */ > + */ > public static URL toURL(String value, Object caller) > { > try > > Added: velocity/tools/trunk/velocity-tools-generic/src/main/java/ > org/apache/velocity/tools/XmlUtils.java > URL: http://svn.apache.org/viewvc/velocity/tools/trunk/velocity- > tools-generic/src/main/java/org/apache/velocity/tools/ > XmlUtils.java?rev=1776916&view=auto > ============================================================ > ================== > --- velocity/tools/trunk/velocity-tools-generic/src/main/java/ > org/apache/velocity/tools/XmlUtils.java (added) > +++ velocity/tools/trunk/velocity-tools-generic/src/main/java/ > org/apache/velocity/tools/XmlUtils.java Mon Jan 2 10:49:55 2017 > @@ -0,0 +1,529 @@ > +/* > + * Licensed to the Apache Software Foundation (ASF) under one > + * or more contributor license agreements. See the NOTICE file > + * distributed with this work for additional information > + * regarding copyright ownership. The ASF licenses this file > + * to you under the Apache License, Version 2.0 (the > + * "License"); you may not use this file except in compliance > + * with the License. You may obtain a copy of the License at > + * > + * http://www.apache.org/licenses/LICENSE-2.0 > + * > + * Unless required by applicable law or agreed to in writing, > + * software distributed under the License is distributed on an > + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY > + * KIND, either express or implied. See the License for the > + * specific language governing permissions and limitations > + * under the License. > + */ > +package org.apache.velocity.tools; > + > +import org.apache.velocity.runtime.RuntimeConstants; > +import org.slf4j.Logger; > +import org.slf4j.LoggerFactory; > +import org.w3c.dom.Attr; > +import org.w3c.dom.Element; > +import org.w3c.dom.NamedNodeMap; > +import org.w3c.dom.Node; > +import org.w3c.dom.NodeList; > +import org.xml.sax.ErrorHandler; > +import org.xml.sax.InputSource; > +import org.xml.sax.SAXException; > +import org.xml.sax.SAXParseException; > + > +import java.io.Reader; > +import java.io.StringReader; > +import java.io.StringWriter; > +import java.lang.ref.SoftReference; > +import java.util.Stack; > +import java.util.concurrent.LinkedBlockingDeque; > + > +import javax.xml.XMLConstants; > +import javax.xml.parsers.DocumentBuilder; > +import javax.xml.parsers.DocumentBuilderFactory; > +import javax.xml.parsers.ParserConfigurationException; > +import javax.xml.transform.OutputKeys; > +import javax.xml.transform.Transformer; > +import javax.xml.transform.TransformerException; > +import javax.xml.transform.TransformerFactory; > +import javax.xml.transform.dom.DOMSource; > +import javax.xml.transform.stream.StreamResult; > +import javax.xml.xpath.XPath; > +import javax.xml.xpath.XPathConstants; > +import javax.xml.xpath.XPathExpression; > +import javax.xml.xpath.XPathExpressionException; > +import javax.xml.xpath.XPathFactory; > + > +/** > + * <p>Utility class for simplifying parsing of xml documents. Documents > are not validated, and > + * loading of external files (xinclude, external entities, DTDs, etc.) > are disabled.</p> > + * > + * @author Claude Brisson > + * @since 3.0 > + * @version $$ > + */ > +public final class XmlUtils > +{ > + /* several pieces of code were borrowed from the Apache Shindig > XmlUtil class.*/ > + > + private static final Logger LOGGER = LoggerFactory.getLogger( > XmlUtils.class); > + > + /** Handles xml errors so that they're not logged to stderr. > + */ > + private static final ErrorHandler errorHandler = new ErrorHandler() > + { > + public void error(SAXParseException exception) throws SAXException > + { > + throw exception; > + } > + public void fatalError(SAXParseException exception) throws > SAXException > + { > + throw exception; > + } > + public void warning(SAXParseException exception) > + { > + LOGGER.info("warning during parsing", exception); > + } > + }; > + > + private static boolean canReuseBuilders = false; > + > + private static final DocumentBuilderFactory builderFactory = > DocumentBuilderFactory.newInstance(); > + > + private static final ThreadLocal<DocumentBuilder> reusableBuilder > + = new ThreadLocal<DocumentBuilder>() { > + @Override > + protected DocumentBuilder initialValue() { > + try > + { > + LOGGER.trace("Created a new document builder"); > + return builderFactory.newDocumentBuilder(); > + } > + catch (ParserConfigurationException e) > + { > + throw new RuntimeException(e); > + } > + } > + }; > + > + static > + { > + // Namespace support is required for <os:> elements > + builderFactory.setNamespaceAware(true); > + > + // Disable various insecure and/or expensive options. > + builderFactory.setValidating(false); > + > + // Can't disable doctypes entirely because they're usually > harmless. External entity > + // resolution, however, is both expensive and insecure. > + try > + { > + builderFactory.setAttribute("http://xml.org/sax/features/ > external-general-entities", false); > + } > + catch (IllegalArgumentException e) > + { > + // Not supported by some very old parsers. > + LOGGER.info("Error parsing external general entities: ", e); > + } > + > + try > + { > + builderFactory.setAttribute("http://xml.org/sax/features/ > external-parameter-entities", false); > + } > + catch (IllegalArgumentException e) > + { > + // Not supported by some very old parsers. > + LOGGER.info("Error parsing external parameter entities: ", e); > + } > + > + try > + { > + builderFactory.setAttribute("http://apache.org/xml/features/ > nonvalidating/load-external-dtd", false); > + } > + catch (IllegalArgumentException e) > + { > + // Only supported by Apache's XML parsers. > + LOGGER.info("Error parsing external DTD: ", e); > + } > + > + try > + { > + > builderFactory.setAttribute(XMLConstants.FEATURE_SECURE_PROCESSING, > true); > + } > + catch (IllegalArgumentException e) > + { > + // Not supported by older parsers. > + LOGGER.info("Error parsing secure XML: ", e); > + } > + > + try > + { > + DocumentBuilder builder = builderFactory. > newDocumentBuilder(); > + builder.reset(); > + canReuseBuilders = true; > + LOGGER.trace("reusing document builders"); > + } > + catch (UnsupportedOperationException e) > + { > + // Only supported by newer parsers (xerces 2.8.x+ for > instance). > + canReuseBuilders = false; > + LOGGER.trace("not reusing document builders"); > + } > + catch (ParserConfigurationException e) > + { > + // Only supported by newer parsers (xerces 2.8.x+ for > instance). > + canReuseBuilders = false; > + LOGGER.trace("not reusing document builders"); > + } > + } > + > + private static LinkedBlockingDeque<SoftReference<DocumentBuilder>> > builderPool = new LinkedBlockingDeque<SoftReference<DocumentBuilder>>(); > // contains only idle builders > + private static int maxBuildersCount = 100; > + private static int currentBuildersCount = 0; > + private static final String BUILDER_MAX_INSTANCES_KEY = > "velocity.tools.xml.documentbuilder.max.instances"; > + > + static > + { > + /* We're in a static code portion, so use a system property so > that the DocumentBuilder pool size > + * remains configurable. */ > + try > + { > + String configuredMax = System.getProperty(BUILDER_ > MAX_INSTANCES_KEY); > + if (configuredMax != null) > + { > + maxBuildersCount = Integer.parseInt(configuredMax); > + } > + } > + catch(Exception e) > + { > + LOGGER.error("could not configure XML document builder max > instances count", e); > + } > + } > + > + private static synchronized DocumentBuilder getDocumentBuilder() > + { > + DocumentBuilder builder = null; > + if (canReuseBuilders && builderPool.size() > 0) > + { > + builder = builderPool.pollFirst().get(); > + } > + if (builder == null) > + { > + if(!canReuseBuilders || currentBuildersCount < > maxBuildersCount) > + { > + try > + { > + builder = builderFactory.newDocumentBuilder(); > + builder.setErrorHandler(errorHandler); > + ++currentBuildersCount; > + } > + catch(Exception e) > + { > + /* this is a fatal error */ > + throw new RuntimeException("could not create a new > XML DocumentBuilder instance", e); > + } > + } > + else > + { > + try > + { > + LOGGER.warn("reached XML DocumentBuilder pool size > limit, current thread needs to wait; you can increase pool size with the {} > system property", BUILDER_MAX_INSTANCES_KEY); > + builder = builderPool.takeFirst().get(); > + } > + catch(InterruptedException ie) > + { > + LOGGER.warn("caught an InterruptedException while > waiting for a DocumentBuilder instance"); > + } > + } > + } > + return builder; > + } > + > + private static synchronized void releaseBuilder(DocumentBuilder > builder) > + { > + builder.reset(); > + builderPool.addLast(new SoftReference<DocumentBuilder>(builder)); > + } > + > + private XmlUtils() {} > + > + /** > + * Extracts an attribute from a node. > + * > + * @param node > + * @param attr > + * @param def > + * @return The value of the attribute, or def > + */ > + public static String getAttribute(Node node, String attr, String def) > + { > + NamedNodeMap attrs = node.getAttributes(); > + Node val = attrs.getNamedItem(attr); > + if (val != null) > + { > + return val.getNodeValue(); > + } > + return def; > + } > + > + /** > + * @param node > + * @param attr > + * @return The value of the given attribute, or null if not present. > + */ > + public static String getAttribute(Node node, String attr) > + { > + return getAttribute(node, attr, null); > + } > + > + /** > + * Retrieves an attribute as a boolean. > + * > + * @param node > + * @param attr > + * @param def > + * @return True if the attribute exists and is not equal to "false" > + * false if equal to "false", and def if not present. > + */ > + public static boolean getBoolAttribute(Node node, String attr, > boolean def) > + { > + String value = getAttribute(node, attr); > + if (value == null) > + { > + return def; > + } > + return Boolean.parseBoolean(value); > + } > + > + /** > + * @param node > + * @param attr > + * @return True if the attribute exists and is not equal to "false" > + * false otherwise. > + */ > + public static boolean getBoolAttribute(Node node, String attr) > + { > + return getBoolAttribute(node, attr, false); > + } > + > + /** > + * @return An attribute coerced to an integer. > + */ > + public static int getIntAttribute(Node node, String attr, int def) > + { > + String value = getAttribute(node, attr); > + if (value == null) > + { > + return def; > + } > + try > + { > + return Integer.parseInt(value); > + } > + catch (NumberFormatException e) > + { > + return def; > + } > + } > + > + /** > + * @return An attribute coerced to an integer. > + */ > + public static int getIntAttribute(Node node, String attr) > + { > + return getIntAttribute(node, attr, 0); > + } > + > + /** > + * Attempts to parse the input xml into a single element. > + * @param xml > + * @return The document object > + */ > + public static Element parse(Reader xml) > + { > + Element ret = null; > + DocumentBuilder builder = getDocumentBuilder(); > + try > + { > + ret = builder.parse(new InputSource(xml)). > getDocumentElement(); > + releaseBuilder(builder); > + return ret; > + } > + catch(Exception e) > + { > + LOGGER.error("could not parse given xml", e); > + } > + finally > + { > + releaseBuilder(builder); > + } > + return ret; > + } > + > + /** > + * Attempts to parse the input xml into a single element. > + * @param xml > + * @return The document object > + */ > + public static Element parse(String xml) > + { > + return parse(new StringReader(xml)); > + } > + > + public static NodeList search(String xpath, Node context) > + { > + NodeList ret = null; > + try > + { > + XPath xp = XPathFactory.newInstance().newXPath(); > + XPathExpression exp = xp.compile(xpath); > + ret = (NodeList)exp.evaluate(context, > XPathConstants.NODESET); > + } > + catch (XPathExpressionException xpe) > + { > + LOGGER.error("could not process xpath expression {}", xpath, > xpe); > + } > + return ret; > + } > + > + /** > + * <p>Builds the xpath expression for a node (tries to use id/name > nodes when possible to get a unique path)</p> > + * > + */ > + // (borrow from http://stackoverflow.com/questions/5046174/get-xpath- > from-the-org-w3c-dom-node ) > + public static String nodePath(Node n) > + { > + // abort early > + if (null == n) > + return null; > + > + // declarations > + Node parent = null; > + Stack<Node> hierarchy = new Stack<Node>(); > + StringBuffer buffer = new StringBuffer('/'); > + > + // push element on stack > + hierarchy.push(n); > + > + switch (n.getNodeType()) { > + case Node.ATTRIBUTE_NODE: > + parent = ((Attr) n).getOwnerElement(); > + break; > + case Node.ELEMENT_NODE: > + parent = n.getParentNode(); > + break; > + case Node.DOCUMENT_NODE: > + parent = n.getParentNode(); > + break; > + default: > + throw new IllegalStateException("Unexpected Node type" + > n.getNodeType()); > + } > + > + while (null != parent && parent.getNodeType() != > Node.DOCUMENT_NODE) { > + // push on stack > + hierarchy.push(parent); > + > + // get parent of parent > + parent = parent.getParentNode(); > + } > + > + // construct xpath > + Object obj = null; > + while (!hierarchy.isEmpty() && null != (obj = hierarchy.pop())) { > + Node node = (Node) obj; > + boolean handled = false; > + > + if (node.getNodeType() == Node.ELEMENT_NODE) > + { > + Element e = (Element) node; > + > + // is this the root element? > + if (buffer.length() == 1) > + { > + // root element - simply append element name > + buffer.append(node.getNodeName()); > + } > + else > + { > + // child element - append slash and element name > + buffer.append("/"); > + buffer.append(node.getNodeName()); > + > + if (node.hasAttributes()) > + { > + // see if the element has a name or id attribute > + if (e.hasAttribute("id")) > + { > + // id attribute found - use that > + buffer.append("[@id='" + e.getAttribute("id") > + "']"); > + handled = true; > + } > + else if (e.hasAttribute("name")) > + { > + // name attribute found - use that > + buffer.append("[@name='" + > e.getAttribute("name") + "']"); > + handled = true; > + } > + } > + > + if (!handled) > + { > + // no known attribute we could use - get sibling > index > + int prev_siblings = 1; > + Node prev_sibling = node.getPreviousSibling(); > + while (null != prev_sibling) > + { > + if (prev_sibling.getNodeType() == > node.getNodeType()) > + { > + if (prev_sibling.getNodeName(). > equalsIgnoreCase( > + node.getNodeName())) > + { > + prev_siblings++; > + } > + } > + prev_sibling = prev_sibling. > getPreviousSibling(); > + } > + buffer.append("[" + prev_siblings + "]"); > + } > + } > + } > + else if (node.getNodeType() == Node.ATTRIBUTE_NODE) > + { > + buffer.append("/@"); > + buffer.append(node.getNodeName()); > + } > + } > + // return buffer > + return buffer.toString(); > + } > + > + public static String nodeToString(Node node) > + > + { > + StringWriter sw = new StringWriter(); > + try > + { > + Transformer t = TransformerFactory. > newInstance().newTransformer(); > + t.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "yes"); > + t.setOutputProperty(OutputKeys.INDENT, "no"); > + /* CB - Since everything is already stored as strings in > memory why shoud an encoding be required here? */ > + t.setOutputProperty(OutputKeys.ENCODING, > RuntimeConstants.ENCODING_DEFAULT); > + t.transform(new DOMSource(node), new StreamResult(sw)); > + } > + catch (TransformerException te) > + { > + LOGGER.error("could not convert XML node to string", te); > + } > + return sw.toString(); > + } > + > + public static boolean isXmlMimeType(String mimeType) > + { > + return mimeType != null && > + ( > + "text/xml".equals(mimeType) || > + "application/xml".equals(mimeType) || > + mimeType.endsWith("+xml") > + ); > + } > +} > > Added: velocity/tools/trunk/velocity-tools-generic/src/main/java/ > org/apache/velocity/tools/generic/ImportSupport.java > URL: http://svn.apache.org/viewvc/velocity/tools/trunk/velocity- > tools-generic/src/main/java/org/apache/velocity/tools/ > generic/ImportSupport.java?rev=1776916&view=auto > ============================================================ > ================== > --- velocity/tools/trunk/velocity-tools-generic/src/main/java/ > org/apache/velocity/tools/generic/ImportSupport.java (added) > +++ velocity/tools/trunk/velocity-tools-generic/src/main/java/ > org/apache/velocity/tools/generic/ImportSupport.java Mon Jan 2 10:49:55 > 2017 > @@ -0,0 +1,588 @@ > +package org.apache.velocity.tools.generic; > + > +/* > + * Licensed to the Apache Software Foundation (ASF) under one > + * or more contributor license agreements. See the NOTICE file > + * distributed with this work for additional information > + * regarding copyright ownership. The ASF licenses this file > + * to you under the Apache License, Version 2.0 (the > + * "License"); you may not use this file except in compliance > + * with the License. You may obtain a copy of the License at > + * > + * http://www.apache.org/licenses/LICENSE-2.0 > + * > + * Unless required by applicable law or agreed to in writing, > + * software distributed under the License is distributed on an > + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY > + * KIND, either express or implied. See the License for the > + * specific language governing permissions and limitations > + * under the License. > + */ > + > +import org.apache.velocity.runtime.RuntimeConstants; > +import org.apache.velocity.tools.ClassUtils; > +import org.apache.velocity.tools.ConversionUtils; > +import org.apache.velocity.tools.Scope; > +import org.apache.velocity.tools.config.InvalidScope; > + > +import java.io.BufferedReader; > +import java.io.File; > +import java.io.IOException; > +import java.io.InputStream; > +import java.io.InputStreamReader; > +import java.io.Reader; > +import java.io.UnsupportedEncodingException; > +import java.net.HttpURLConnection; > +import java.net.URL; > +import java.net.URLConnection; > + > +/** > + * <p>Provides methods to import arbitrary local or remote resources as > strings, generic version.</p> > + * <p>Based on ImportSupport from the JSTL taglib by Shawn Bayern</p> > + * > + * @author <a href="mailto:mari...@centrum.is";>Marino A. Jonsson</a> > + * @author Claude Brisson > + * @since VelocityTools 3.0 > + * @version $$ > + */ > +@InvalidScope({Scope.APPLICATION, Scope.SESSION, Scope.REQUEST}) /* this > tool is not meant to be used directly*/ > +public class ImportSupport extends SafeConfig > +{ > + protected static final String VALID_SCHEME_CHARS = > + "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567 > 89+.-"; > + > + /** Configuration key for XmlTool and JsonTool, used to specify a > local resource > + */ > + public static final String RESOURCE_KEY = "resource"; > + > + /** Configuration key for ImportTool, XmlTool and JsonTool, used to > specify a local or remote source URL > + */ > + public static final String URL_KEY = "url"; > + > + //********************************************************** > *********** > + // URL importation logic > + > + /* > + * Overall strategy: we have two entry points, acquireString() and > + * acquireReader(). The latter passes data through unbuffered if > + * possible (but note that it is not always possible -- specifically > + * for cases where we must use the RequestDispatcher. The remaining > + * methods handle the common.core logic of loading either a URL or a > local > + * resource. > + * > + * We consider the 'natural' form of remote URLs to be Readers and > + * local URLs to be Strings. Thus, to avoid doing extra work, > + * acquireString() and acquireReader() delegate to one another as > + * appropriate. (Perhaps I could have spelled things out more > clearly, > + * but I thought this implementation was instructive, not to mention > + * somewhat cute...) > + * > + * CB: Changes with VelocityTools 3.0 implementation: ImportSupport > is now splitted in two classes, > + * o.a.v.tools.generic.ImportSupport and > o.a.v.tools.view.ViewImportSupport > inheriting the former one. > + * In the generic version, only remote urls are supported, while the > view version will work as aforementioned. > + */ > + > + /** > + * configure import support > + * @param values > + */ > + protected void configure(ValueParser values) > + { > + super.configure(values); > + } > + > + /** > + * > + * @param url the URL resource to return as string > + * @return the URL resource as string > + * @throws IOException > + */ > + public String acquireString(String url) throws IOException > + { > + getLog().debug("acquire URL {}", url); > + if (isRemoteURL(url)) > + { > + return acquireRemoteURLString(url); > + } > + else > + { > + return acquireLocalURLString(url); > + } > + } > + > + /** > + * Aquire the content of a remote URL. > + * @param url remote URL > + * @return the URL resource as string > + * @throws IOException > + */ > + protected String acquireRemoteURLString(String url) throws IOException > + { > + // delegate to our peer > + BufferedReader r = null; > + try > + { > + r = new BufferedReader(acquireRemoteURLReader(url)); > + StringBuilder sb = new StringBuilder(); > + int i; > + // under JIT, testing seems to show this simple loop is as > fast > + // as any of the alternatives > + while ((i = r.read()) != -1) > + { > + sb.append((char)i); > + } > + return sb.toString(); > + } > + finally > + { > + if(r != null) > + { > + try > + { > + r.close(); > + } > + catch (IOException ioe) > + { > + getLog().error("Could not close reader.", ioe); > + } > + } > + } > + } > + > + /** > + * Aquire the content of a local URL. > + * @param url local URL > + * @return the URL resource as string > + * @throws IOException > + */ > + protected String acquireLocalURLString(String url) throws IOException > + { > + throw new IOException("Only remote URLs are supported"); > + } > + > + /** > + * Acquire a reader to an URL > + * @param url the URL to read > + * @return a Reader for the InputStream created from the supplied URL > + * @throws IOException > + * @throws java.lang.Exception > + */ > + public Reader acquireReader(String url) throws IOException > + { > + getLog().debug("acquire URL {}", url); > + if (isRemoteURL(url)) > + { > + return acquireRemoteURLReader(url); > + } > + else > + { > + return acquireLocalURLReader(url); > + } > + } > + > + /** > + * Acquire a reader to a remote URL > + * @param url the URL to read > + * @return a Reader for the InputStream created from the supplied URL > + * @throws IOException > + * @throws java.lang.Exception > + */ > + protected Reader acquireRemoteURLReader(String url) throws > IOException > + { > + // remote URL > + URLConnection uc = null; > + HttpURLConnection huc = null; > + InputStream i = null; > + > + try > + { > + // handle remote URLs ourselves, using java.net.URL > + URL u = ConversionUtils.toURL(url); > + uc = u.openConnection(); > + i = uc.getInputStream(); > + > + // check response code for HTTP URLs, per spec, > + if (uc instanceof HttpURLConnection) > + { > + huc = (HttpURLConnection)uc; > + > + int status = huc.getResponseCode(); > + if (status < 200 || status > 299) > + { > + throw new IOException(status + " " + url); > + } > + } > + > + // okay, we've got a stream; encode it appropriately > + Reader r = null; > + String charSet; > + > + // charSet extracted according to RFC 2045, section 5.1 > + String contentType = uc.getContentType(); > + if (contentType != null) > + { > + charSet = ImportSupport.getContentTypeAttribute(contentType, > "charset"); > + if (charSet == null) > + { > + charSet = RuntimeConstants.ENCODING_DEFAULT; > + } > + } > + else > + { > + charSet = RuntimeConstants.ENCODING_DEFAULT; > + } > + > + try > + { > + r = new InputStreamReader(i, charSet); > + } > + catch (UnsupportedEncodingException ueex) > + { > + r = new InputStreamReader(i, RuntimeConstants.ENCODING_ > DEFAULT); > + } > + > + if (huc == null) > + { > + return r; > + } > + else > + { > + return new SafeClosingHttpURLConnectionReader(r, huc); > + } > + } > + catch (IOException ex) > + { > + if (i != null) > + { > + try > + { > + i.close(); > + } > + catch (IOException ioe) > + { > + getLog().error("Could not close InputStream", ioe); > + } > + } > + > + if (huc != null) > + { > + huc.disconnect(); > + } > + throw new IOException("Problem accessing the remote URL \"" > + + url + "\". " + ex); > + } > + catch (RuntimeException ex) > + { > + if (i != null) > + { > + try > + { > + i.close(); > + } > + catch (IOException ioe) > + { > + getLog().error("Could not close InputStream", ioe); > + } > + } > + > + if (huc != null) > + { > + huc.disconnect(); > + } > + // because the spec makes us > + throw new IOException("Problem accessing the remote URL \"" + > url + "\" :" + ex.getMessage(), ex); > + } > + } > + > + /** > + * Acquire a reader to a local URL - non applicable to the generic > version of ImportSupport > + * @param url the URL to read > + * @return a Reader for the InputStream created from the supplied URL > + * @throws IOException > + * @throws java.lang.Exception > + */ > + protected Reader acquireLocalURLReader(String url) throws IOException > + { > + throw new IOException("Only remote URLs are supported"); > + } > + > + protected static class SafeClosingHttpURLConnectionReader extends > Reader > + { > + private final HttpURLConnection huc; > + private final Reader wrappedReader; > + > + SafeClosingHttpURLConnectionReader(Reader r, HttpURLConnection > huc) > + { > + this.wrappedReader = r; > + this.huc = huc; > + } > + > + public void close() throws IOException > + { > + if(null != huc) > + { > + huc.disconnect(); > + } > + > + wrappedReader.close(); > + } > + > + // Pass-through methods. > + public void mark(int readAheadLimit) throws IOException > + { > + wrappedReader.mark(readAheadLimit); > + } > + > + public boolean markSupported() > + { > + return wrappedReader.markSupported(); > + } > + > + public int read() throws IOException > + { > + return wrappedReader.read(); > + } > + > + public int read(char[] buf) throws IOException > + { > + return wrappedReader.read(buf); > + } > + > + public int read(char[] buf, int off, int len) throws IOException > + { > + return wrappedReader.read(buf, off, len); > + } > + > + public boolean ready() throws IOException > + { > + return wrappedReader.ready(); > + } > + > + public void reset() throws IOException > + { > + wrappedReader.reset(); > + } > + > + public long skip(long n) throws IOException > + { > + return wrappedReader.skip(n); > + } > + } > + > + //********************************************************** > *********** > + // Public utility methods > + > + /** > + * Returns whether an URL is remote or local > + * > + * @param url the url to check out > + * @return wether the URL is remote > + */ > + public static boolean isRemoteURL(String url) > + { > + return getProtocol(url) == null; > + } > + > + /** > + * Returns protocol, or null for a local URL > + * > + * @param url the url to check out > + * @return found protocol or null for a local URL > + */ > + public static String getProtocol(String url) > + { > + // a null URL is not remote, by our definition > + if (url == null) > + { > + return null; > + } > + > + // do a fast, simple check first > + int colonPos; > + if ((colonPos = url.indexOf(':')) == -1) > + { > + return null; > + } > + > + // if we DO have a colon, make sure that every character > + // leading up to it is a valid scheme character > + for (int i = 0; i < colonPos; i++) > + { > + if (VALID_SCHEME_CHARS.indexOf(url.charAt(i)) == -1) > + { > + return null; > + } > + } > + // if so, we've got a remote url > + return url.substring(0, colonPos); > + } > + > + /** > + * Get the value associated with a content-type attribute. > + * Syntax defined in RFC 2045, section 5.1. > + * > + * @param input the string containing the attributes > + * @param name the name of the content-type attribute > + * @return the value associated with a content-type attribute > + */ > + public static String getContentTypeAttribute(String input, String > name) > + { > + int begin; > + int end; > + int index = input.toUpperCase().indexOf(name.toUpperCase()); > + if (index == -1) > + { > + return null; > + } > + index = index + name.length(); // positioned after the attribute > name > + index = input.indexOf('=', index); // positioned at the '=' > + if (index == -1) > + { > + return null; > + } > + index += 1; // positioned after the '=' > + input = input.substring(index).trim(); > + > + if (input.charAt(0) == '"') > + { > + // attribute value is a quoted string > + begin = 1; > + end = input.indexOf('"', begin); > + if (end == -1) > + { > + return null; > + } > + } > + else > + { > + begin = 0; > + end = input.indexOf(';'); > + if (end == -1) > + { > + end = input.indexOf(' '); > + } > + if (end == -1) > + { > + end = input.length(); > + } > + } > + return input.substring(begin, end).trim(); > + } > + > + //********************************************************** > *********** > + // Fetch local resource > + > + /** > + * Fetch a local resource, first trying with a file (or a webapp > resource for the view flavor) > + * then with a classpath entry. > + * @param resource the resource to read > + * @return the content of the resource > + * @throws IOException > + * @throws java.lang.Exception > + */ > + public String getResourceString(String resource) > + { > + String ret = null; > + try > + { > + Reader rawReader = getResourceReader(resource); > + if (rawReader != null) > + { > + BufferedReader reader = new BufferedReader(rawReader); > + StringBuilder sb = new StringBuilder(); > + int i; > + // under JIT, testing seems to show this simple loop is > as fast > + // as any of the alternatives > + while ((i = reader.read()) != -1) > + { > + sb.append((char) i); > + } > + ret = sb.toString(); > + } > + } > + catch (IOException ioe) > + { > + getLog().error("could not load resource {}", resource, ioe); > + } > + return ret; > + } > + > + /** > + * Get a reader of a local resource, first trying with a file (or a > webapp resource for the view flavor) > + * then with a classpath entry. > + * @param resource the resource to read > + * @return a reader of the resource > + * @throws IOException > + * @throws java.lang.Exception > + */ > + public Reader getResourceReader(String resource) > + { > + getLog().debug("get resource {}", resource); > + URL url = null; > + Reader reader = null; > + try > + { > + url = getFileResource(resource); > + if (url == null) > + { > + url = getClasspathResource(resource); > + } > + if (url != null) > + { > + URLConnection uc = url.openConnection(); > + InputStream is = uc.getInputStream(); > + String charSet; > + // charSet extracted according to RFC 2045, section 5.1 > + String contentType = uc.getContentType(); > + if (contentType != null) > + { > + charSet = > ImportSupport.getContentTypeAttribute(contentType, > "charset"); > + if (charSet == null) > + { > + charSet = RuntimeConstants.ENCODING_DEFAULT; > + } > + } > + else > + { > + charSet = RuntimeConstants.ENCODING_DEFAULT; > + } > + reader = new InputStreamReader(is, charSet); > + } > + } > + catch (Exception e) > + { > + getLog().error("could not get resource {}", resource, e); > + } > + return reader; > + } > + > + /** > + * Overridable local file URL builder. > + * @param resource the resource to read > + * @return the content of the resource > + * @throws IOException > + * @throws java.lang.Exception > + */ > + protected URL getFileResource(String resource) throws Exception > + { > + URL url = null; > + File file = new File(resource); > + if (file.exists() && file.isFile() && file.canRead()) > + { > + url = file.toURI().toURL(); > + } > + return url; > + } > + > + /** > + * Classpath entry URL builder > + * @param resource the resource to read > + * @return the content of the resource > + * @throws IOException > + * @throws java.lang.Exception > + */ > + protected URL getClasspathResource(String resource) throws Exception > + { > + return ClassUtils.getResource(resource, this); > + } > +} > > Copied: velocity/tools/trunk/velocity-tools-generic/src/main/java/ > org/apache/velocity/tools/generic/ImportTool.java (from r1770862, > velocity/tools/trunk/velocity-tools-view/src/main/java/org/ > apache/velocity/tools/view/ImportTool.java) > URL: http://svn.apache.org/viewvc/velocity/tools/trunk/velocity- > tools-generic/src/main/java/org/apache/velocity/tools/ > generic/ImportTool.java?p2=velocity/tools/trunk/velocity- > tools-generic/src/main/java/org/apache/velocity/tools/ > generic/ImportTool.java&p1=velocity/tools/trunk/velocity- > tools-view/src/main/java/org/apache/velocity/tools/view/ > ImportTool.java&r1=1770862&r2=1776916&rev=1776916&view=diff > ============================================================ > ================== > --- velocity/tools/trunk/velocity-tools-view/src/main/java/org/ > apache/velocity/tools/view/ImportTool.java (original) > +++ velocity/tools/trunk/velocity-tools-generic/src/main/java/ > org/apache/velocity/tools/generic/ImportTool.java Mon Jan 2 10:49:55 2017 > @@ -1,4 +1,4 @@ > -package org.apache.velocity.tools.view; > +package org.apache.velocity.tools.generic; > > /* > * Licensed to the Apache Software Foundation (ASF) under one > @@ -22,12 +22,10 @@ package org.apache.velocity.tools.view; > import org.apache.velocity.tools.Scope; > import org.apache.velocity.tools.config.DefaultKey; > import org.apache.velocity.tools.config.ValidScope; > -import org.apache.velocity.tools.view.ImportSupport; > > /** > - * General-purpose text-importing view tool for templates. > - * <p>Usage:<br /> > - * Just call $import.read("http://www.foo.com/bleh.jsp?sneh=bar";) to > insert the contents of the named > + * General-purpose text-importing tool for templates. > + * <p>Usage: just call $import.read("http://www.foo.com/bleh.jsp?sneh=bar";) > to insert the contents of the named > * resource into the template. > * </p> > * <p><pre> > @@ -40,39 +38,92 @@ import org.apache.velocity.tools.view.Im > * </pre></p> > * > * @author <a href="mailto:mari...@centrum.is";>Marino A. Jonsson</a> > - * @since VelocityTools 2.0 > - * @version $Revision$ $Date$ > + * @since VelocityTools 3.0 > + * @version $Id$ > */ > > @DefaultKey("import") > @ValidScope(Scope.REQUEST) > -public class ImportTool extends ImportSupport > +public class ImportTool extends SafeConfig > { > /** > + * ImportSupport utility which provides underlying i/o > + */ > + protected ImportSupport importSupport = null; > + > + /** > + * Importsupport initialization > + * @param config > + */ > + protected void initializeImportSupport(ValueParser config) > + { > + importSupport = new ImportSupport(); > + importSupport.configure(config); > + } > + > + /** > + * Configuration > + * @param values > + */ > + protected void configure(ValueParser values) > + { > + initializeImportSupport(values); > + } > + > + /** > + * Returns the supplied resource rendered as a String. > + * > + * @param resource the URL to import > + * @return the URL as a string > + */ > + public String read(String resource) > + { > + if (resource == null) > + { > + getLog().warn("resource is null!"); > + return null; > + } > + if (resource.length() == 0) > + { > + getLog().warn("resource is empty string!"); > + return null; > + } > + try > + { > + return importSupport.getResourceString(resource); > + } > + catch (Exception ex) > + { > + getLog().error("Exception while getting '{}'", resource, ex); > + return null; > + } > + } > + > + /** > * Returns the supplied URL rendered as a String. > * > - * @param obj the URL to import > + * @param url the URL to import > * @return the URL as a string > */ > - public String read(Object obj) { > - if (obj == null) > + public String fetch(String url) > + { > + if (url == null) > { > - getLog().warn("ImportTool.read(): url is null!"); > + getLog().warn("URL is null!"); > return null; > } > - String url = String.valueOf(obj).trim(); > if (url.length() == 0) > { > - getLog().warn("ImportTool.read(): url is empty string!"); > + getLog().warn("URL is empty string!"); > return null; > } > try > { > - return acquireString(url); > + return importSupport.acquireString(url); > } > catch (Exception ex) > { > - getLog().error("ImportTool.read(): Exception while aquiring > '{}'", url, ex); > + getLog().error("Exception while acquiring '{}'", url, ex); > return null; > } > } > > Added: velocity/tools/trunk/velocity-tools-generic/src/main/java/ > org/apache/velocity/tools/generic/JsonTool.java > URL: http://svn.apache.org/viewvc/velocity/tools/trunk/velocity- > tools-generic/src/main/java/org/apache/velocity/tools/ > generic/JsonTool.java?rev=1776916&view=auto > ============================================================ > ================== > --- velocity/tools/trunk/velocity-tools-generic/src/main/java/ > org/apache/velocity/tools/generic/JsonTool.java (added) > +++ velocity/tools/trunk/velocity-tools-generic/src/main/java/ > org/apache/velocity/tools/generic/JsonTool.java Mon Jan 2 10:49:55 2017 > @@ -0,0 +1,372 @@ > +package org.apache.velocity.tools.generic; > + > +/* > + * Licensed to the Apache Software Foundation (ASF) under one > + * or more contributor license agreements. See the NOTICE file > + * distributed with this work for additional information > + * regarding copyright ownership. The ASF licenses this file > + * to you under the Apache License, Version 2.0 (the > + * "License"); you may not use this file except in compliance > + * with the License. You may obtain a copy of the License at > + * > + * http://www.apache.org/licenses/LICENSE-2.0 > + * > + * Unless required by applicable law or agreed to in writing, > + * software distributed under the License is distributed on an > + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY > + * KIND, either express or implied. See the License for the > + * specific language governing permissions and limitations > + * under the License. > + */ > + > +import java.io.BufferedReader; > +import java.io.InputStreamReader; > +import java.io.Reader; > +import java.io.StringReader; > +import java.net.URL; > +import java.util.Iterator; > +import java.util.Set; > + > +import org.apache.velocity.tools.ConversionUtils; > +import org.apache.velocity.tools.XmlUtils; > +import org.json.JSONArray; > +import org.json.JSONObject; > +import org.json.JSONTokener; > + > +import org.apache.velocity.tools.Scope; > +import org.apache.velocity.tools.config.DefaultKey; > +import org.apache.velocity.tools.config.InvalidScope; > +import org.w3c.dom.Document; > +import org.w3c.dom.Node; > + > +/** > + * <p>Tool which can parse a JSON file.</o> > + * <p>Usage:</p> > + * <p> > + * <ul> > + * <li>$json.parse(<i>json string</i>)</li> > + * <li>$json.read(<i>file or classpath resource</i>)</li> > + * <li>$json.fetch(<i>URL</i>)</li> > + * </ul> > + * </p> > + * <p>Configuration parameters:</p> > + * <p> > + * <ul> > + * <li><code>resource</code>=<i>file or classpath > resource</i></li> > + * <li><code>source</code>=<i>URL</i></li> > + * </ul> > + * </p> > + * <p> > + * <pre> > + * Example configuration: > + * <tools> > + * <toolbox scope="request"> > + * <tool class="org.apache.velocity.tools.generic.JsonTool" > + * key="foo" resource="doc.xml"/> > + * </toolbox> > + * </tools> > + * > + * </pre> > + * </p> > + * @author Claude Brisson > + * @since VelocityTools 3.0 > + * @version $Id:$ > + */ > + > +// JSONObject isn't (yet?) Serializable, so session scope is invalid > +@DefaultKey("json") > +@InvalidScope(Scope.SESSION) > +public class JsonTool extends ImportSupport implements Iterable > +{ > + /** > + * ImportSupport utility which provides underlying i/o > + */ > + protected ImportSupport importSupport = null; > + > + /** > + * ImportSupport initialization > + * @param config > + */ > + protected void initializeImportSupport(ValueParser config) > + { > + importSupport = new ImportSupport(); > + importSupport.configure(config); > + } > + > + /** > + * JSONObject content > + */ > + private JSONObject jsonObject = null; > + > + /** > + * JSONArray content > + */ > + private JSONArray jsonArray = null; > + > + /** > + * Looks for the "file" parameter and automatically uses > + * {@link #initJSON(String)} to parse the file (searched in > filesystem current path and classpath) and set the > + * resulting JSON object as the root node for this instance. > + */ > + protected void configure(ValueParser values) > + { > + super.configure(values); > + initializeImportSupport(values); > + String resource = values.getString(ImportSupport.RESOURCE_KEY); > + if (resource != null) > + { > + read(resource); > + } > + else > + { > + String url = values.getString(ImportSupport.URL_KEY); > + if (url != null) > + { > + fetch(url); > + } > + } > + } > + > + /** > + * Initialize JSON content from a string. > + * @param json > + */ > + protected void initJSON(String json) > + { > + if (json != null) > + { > + initJSON(new StringReader(json)); > + } > + } > + > + /** > + * Initialize JSON content from a reader. > + * @param reader > + */ > + protected void initJSON(Reader reader) > + { > + try > + { > + final int lookahead = 100; > + int jsonType = 0; // 1 = object, 2 = array > + if (!reader.markSupported()) > + { > + reader = new BufferedReader(reader); > + } > + reader.mark(lookahead); > + char buffer[] = new char[lookahead]; > + int read = reader.read(buffer); > + reader.reset(); > + for (int i = 0; i < read; ++i) > + { > + switch (buffer[i]) > + { > + case '{': > + jsonType = 1; > + break; > + case '[': > + jsonType = 2; > + break; > + case ' ': > + case '\t': > + case '\r': > + case '\n': > + break; > + default: > + { > + String msg = "could not pase JSON: invalid > character at position " + i + ": '" + buffer[i] + "'"; > + throw new Exception(msg); > + } > + } > + if (jsonType != 0) > + { > + break; > + } > + } > + switch (jsonType) > + { > + case 0: > + { > + String msg = "could not pase JSON: did not find any > '{' or '[' in the first " + lookahead + " characters"; > + throw new Exception(msg); > + } > + case 1: > + jsonArray = null; > + jsonObject = new JSONObject(new JSONTokener(reader)); > + break; > + case 2: > + jsonObject = null; > + jsonArray = new JSONArray(new JSONTokener(reader)); > + } > + } > + catch (Exception e) > + { > + getLog().error("error while setting up JSON source", e); > + } > + } > + > + /** > + * Parses the given JSON string and uses the resulting {@link > Document} > + * as the root {@link Node}. > + */ > + public void parse(String xml) throws Exception > + { > + if (xml != null) > + { > + try > + { > + initJSON(xml); > + } > + catch (Exception e) > + { > + getLog().error("could not parse given JSON string", e); > + } > + } > + } > + > + /** > + * Reads and parses a local JSON resource file > + */ > + public void read(String resource) > + { > + if (resource != null) > + { > + try > + { > + Reader reader = importSupport. > getResourceReader(resource); > + if (reader != null) > + { > + initJSON(reader); > + } > + } > + catch (Exception e) > + { > + getLog().error("could not read JSON resource {}", > resource, e); > + } > + } > + } > + > + /** > + * Reads and parses a remote or local URL > + */ > + public void fetch(String url) > + { > + if (url != null) > + { > + try > + { > + Reader reader = importSupport.acquireReader(url); > + if (reader != null) > + { > + initJSON(reader); > + } > + } > + catch (Exception e) > + { > + getLog().error("could not fetch JSON content from URL > {}", url, e); > + } > + } > + } > + > + /** > + * Get JSON root object. > + * @return root object or array > + */ > + public Object root() > + { > + return jsonObject != null ? jsonObject : jsonArray ; > + } > + > + /** > + * Get nth element from root json array. > + * @param index n > + * @return nth element, or null if root object is null or not an array > + */ > + public Object get(int index) > + { > + return jsonArray == null ? null : jsonArray.get(index); > + } > + > + /** > + * Get a property from root object > + * @param key > + * @return property value, or null > + */ > + public Object get(String key) > + { > + return jsonObject == null ? null : jsonObject.get(key); > + } > + > + /** > + * Iterate keys of root object. > + * @return iterator > + */ > + public Iterator<String> keys() > + { > + return jsonObject == null ? null : jsonObject.keys(); > + } > + > + /** > + * Get set of root object keys. > + * @return > + */ > + public Set<String> keySet() > + { > + return jsonObject == null ? null : jsonObject.keySet(); > + } > + > + /** > + * Get an iterator. For a root object, returns an iterator over key > names. For a root array, returns an iterator > + * over contained objects. > + * @return iterator > + */ > + public Iterator iterator() > + { > + if (jsonObject != null) > + { > + return jsonObject. keys(); > + } > + else if (jsonArray != null) > + { > + return jsonArray.iterator(); > + } > + return null; > + } > + > + /** > + * Get size of root object or array. > + * @return size > + */ > + public int length() > + { > + return jsonObject == null ? jsonArray == null ? null : > jsonArray.length() : jsonObject.length(); > + } > + > + /** > + * Get array of root object keys. > + * @return array of keys > + */ > + public JSONArray names() > + { > + return jsonObject == null ? null : jsonObject.names(); > + } > + > + /** > + * Query root object or array using a JSON pointer > + * @param jsonPointer > + * @return result > + */ > + public Object query(String jsonPointer) > + { > + return jsonObject == null ? jsonArray == null ? null : > jsonArray.query(jsonPointer) : jsonObject.query(jsonPointer); > + } > + > + /** > + * Convert JSON object or array into string > + * @return JSON representation of the root object or array > + */ > + public String toString() > + { > + return jsonObject == null ? jsonArray == null ? "null" : > jsonArray.toString() : jsonObject.toString(); > + } > +} > > >
