Hi DJ,
I did some research with David and figured out that if you use a Reader then
the encoding set in the xml file is ignored. if you use a InputStream then
the encoding is set based on the value in the xml file.
So it is most appropriate to use InputStream for unmarshalling all xml
files. Then we can specify in the file its encoding.
I think your ideas for CastorUtils and the Castor DAO class make a lot of
sense.
Thanks for noticing!
Matt
On Tue, Jun 17, 2008 at 12:04 AM, DJ Gregor <[EMAIL PROTECTED]> wrote:
> David,
>
> I saw your commit today where you changed from using a Reader to an
> InputSource (from an InputStream) to fix bug #2526 (UTF-8 support for
> the model importer). Should we be doing this for some more of our XML
> files or all of them?
>
> Both the CastorUtils class, which has helper static methods for
> marshalling/unmarshalling, and the AbstractCastorConfigDao class
> currently use a Reader. AbstractCastorConfigDao is the base of a
> handful of Castor DAOs, and I would like to eventually see it used for
> all of them. I'm wondering if these should be switched over. It
> probably also makes sense to add a "unmarshal" method to CastorUtils
> that takes a Spring Resource, and that could be used both by
> AbstractCastorConfigDao and the importer's SpecFile class.
>
>
> - deej
>
>
> References:
>
> Bug #2526: http://bugzilla.opennms.org/show_bug.cgi?id=2526
> SpecFile.java changes in revision 9350:
>
> http://opennms.svn.sourceforge.net/viewvc/opennms/opennms/branches/1.6-testing/opennms-import/src/main/java/org/opennms/netmgt/importer/specification/SpecFile.java?r1=9158&r2=9350
> CastorUtils.java:
>
> http://opennms.svn.sourceforge.net/viewvc/opennms/opennms/trunk/opennms-dao/src/main/java/org/opennms/netmgt/dao/castor/CastorUtils.java?view=markup
> AbstractCastorConfigDao.java:
>
> http://opennms.svn.sourceforge.net/viewvc/opennms/opennms/trunk/opennms-dao/src/main/java/org/opennms/netmgt/dao/castor/AbstractCastorConfigDao.java?view=markup
>
>
> On Mon, 16 Jun 2008 13:55:38 -0700, [EMAIL PROTECTED] said:
> > Revision: 9350
> > http://opennms.svn.sourceforge.net/opennms/?rev=9350&view=rev
> > Author: dhustace
> > Date: 2008-06-16 13:55:25 -0700 (Mon, 16 Jun 2008)
> >
> > Log Message:
> > -----------
> > Fix for http://bugzilla.opennms.org/show_bug.cgi?id=2526... converted
> > Marshaller to user Stream vs. Reader. Deprecated the Reader version.
> >
> > Modified Paths:
> > --------------
> >
> opennms/branches/1.6-testing/opennms-import/src/main/java/org/opennms/netmgt/importer/specification/SpecFile.java
> >
> opennms/branches/1.6-testing/opennms-import/src/test/java/org/opennms/netmgt/importer/ImporterServiceTest.java
> >
> opennms/branches/1.6-testing/opennms-import/src/test/java/org/opennms/netmgt/importer/ModelImporterTest.java
> >
> > Added Paths:
> > -----------
> >
> opennms/branches/1.6-testing/opennms-import/src/test/resources/utf-8.xml
> >
> > Modified:
> >
> opennms/branches/1.6-testing/opennms-import/src/main/java/org/opennms/netmgt/importer/specification/SpecFile.java
> > ===================================================================
> > ---
> >
> opennms/branches/1.6-testing/opennms-import/src/main/java/org/opennms/netmgt/importer/specification/SpecFile.java
> > 2008-06-16 20:38:40 UTC (rev 9349)
> > +++
> >
> opennms/branches/1.6-testing/opennms-import/src/main/java/org/opennms/netmgt/importer/specification/SpecFile.java
> > 2008-06-16 20:55:25 UTC (rev 9350)
> > @@ -34,6 +34,7 @@
> > package org.opennms.netmgt.importer.specification;
> >
> > import java.io.IOException;
> > +import java.io.InputStream;
> > import java.io.InputStreamReader;
> > import java.io.Reader;
> >
> > @@ -47,35 +48,56 @@
> > import org.opennms.netmgt.config.modelimport.Node;
> > import org.opennms.netmgt.importer.ModelImportException;
> > import org.springframework.core.io.Resource;
> > +import org.xml.sax.InputSource;
> >
> > public class SpecFile {
> >
> > private ModelImport m_mi;
> >
> > public void loadResource(Resource resource) throws
> > ModelImportException, IOException {
> > - Reader reader = new
> > InputStreamReader(resource.getInputStream());
> > - unmarshall(reader);
> > - closeQuietly(reader);
> > -
> > + InputStream inputStream = null;
> > + try {
> > + inputStream = resource.getInputStream();
> > + unmarshall(inputStream);
> > + } finally {
> > + closeQuietly(inputStream);
> > + }
> > }
> >
> > - private void closeQuietly(Reader reader) {
> > + private void closeQuietly(InputStream stream) {
> > try {
> > - if (reader != null) reader.close();
> > + if (stream != null) stream.close();
> > } catch (IOException e) {
> > // ignore failed close
> > }
> > }
> >
> > - public void unmarshall(Reader rdr) throws ModelImportException {
> > + public void unmarshall(InputStream stream) throws
> ModelImportException
> > {
> > try {
> > - m_mi =
> > (ModelImport)Unmarshaller.unmarshal(ModelImport.class, rdr);
> > + InputSource source = new InputSource(stream);
> > + m_mi =
> > (ModelImport)Unmarshaller.unmarshal(ModelImport.class, source);
> > } catch (MarshalException e) {
> > throw new ModelImportException("Exception while marshalling
> > import: "+e, e);
> > } catch (ValidationException e) {
> > throw new ModelImportException("Exception while validating
> > import "+e);
> > }
> > }
> > +
> > + /**
> > + * @deprecated
> > + * @param rdr
> > + * @throws ModelImportException
> > + */
> > + public void unmarshall(Reader rdr) throws ModelImportException {
> > + try {
> > + InputSource source = new InputSource(rdr);
> > + m_mi =
> > (ModelImport)Unmarshaller.unmarshal(ModelImport.class, source);
> > + } catch (MarshalException e) {
> > + throw new ModelImportException("Exception while marshalling
> > import: "+e, e);
> > + } catch (ValidationException e) {
> > + throw new ModelImportException("Exception while validating
> > import "+e);
> > + }
> > + }
> >
> > public void visitImport(ImportVisitor visitor) {
> > doVisitImport(visitor);
> >
> > Modified:
> >
> opennms/branches/1.6-testing/opennms-import/src/test/java/org/opennms/netmgt/importer/ImporterServiceTest.java
> > ===================================================================
> > ---
> >
> opennms/branches/1.6-testing/opennms-import/src/test/java/org/opennms/netmgt/importer/ImporterServiceTest.java
> > 2008-06-16 20:38:40 UTC (rev 9349)
> > +++
> >
> opennms/branches/1.6-testing/opennms-import/src/test/java/org/opennms/netmgt/importer/ImporterServiceTest.java
> > 2008-06-16 20:55:25 UTC (rev 9350)
> > @@ -40,7 +40,6 @@
> >
> > import org.opennms.netmgt.EventConstants;
> > import
> >
>
> org.opennms.netmgt.dao.db.AbstractTransactionalTemporaryDatabaseSpringContextTests;
> > -import org.opennms.netmgt.importer.ImporterService;
> > import org.opennms.netmgt.mock.MockEventIpcManager;
> > import org.opennms.netmgt.utils.EventBuilder;
> > import org.opennms.netmgt.xml.event.Event;
> >
> > Modified:
> >
> opennms/branches/1.6-testing/opennms-import/src/test/java/org/opennms/netmgt/importer/ModelImporterTest.java
> > ===================================================================
> > ---
> >
> opennms/branches/1.6-testing/opennms-import/src/test/java/org/opennms/netmgt/importer/ModelImporterTest.java
> > 2008-06-16 20:38:40 UTC (rev 9349)
> > +++
> >
> opennms/branches/1.6-testing/opennms-import/src/test/java/org/opennms/netmgt/importer/ModelImporterTest.java
> > 2008-06-16 20:55:25 UTC (rev 9350)
> > @@ -266,6 +266,30 @@
> > *
> > * @throws ModelImportException
> > */
> > + public void testImportUtf8() throws Exception {
> > + createAndFlushServiceTypes();
> > + createAndFlushCategories();
> > +
> > + //Initialize the database
> > + ModelImporter mi = m_importer;
> > + String specFile = "/utf-8.xml";
> > + mi.importModelFromResource(new ClassPathResource(specFile));
> > +
> > + assertEquals(1, mi.getNodeDao().countAll());
> > + assertEquals("\x96ode2", mi.getNodeDao().get(1).getLabel());
> > +
> > +// ImportVisitor visitor = new ImportAccountant();
> > +//
> > mi.loadResource(File.separator+"tec_dump.xml.smalltest.delete");
> > +// mi.visitImport(visitor);
> > +
> > + }
> > +
> > + /**
> > + * This test first bulk imports 10 nodes then runs update with 1
> > node missing
> > + * from the import file.
> > + *
> > + * @throws ModelImportException
> > + */
> > public void testDelete() throws Exception {
> > createAndFlushServiceTypes();
> > createAndFlushCategories();
> >
> > Added:
> > opennms/branches/1.6-testing/opennms-import/src/test/resources/utf-8.xml
> > ===================================================================
> > ---
> > opennms/branches/1.6-testing/opennms-import/src/test/resources/utf-8.xml
> > (rev 0)
> > +++
> > opennms/branches/1.6-testing/opennms-import/src/test/resources/utf-8.xml
> > 2008-06-16 20:55:25 UTC (rev 9350)
> > @@ -0,0 +1,21 @@
> > +<?xml version="1.0" encoding="UTF-8"?>
> > +<model-import xmlns="http://www.example.org/model-import";
> > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
> > xsi:schemalLocation="http://www.example.org/model-import
> > +model-import.xsd" date-stamp="2005-12-27T09:12:54">
> > + <node node-label="ñode2" parent-node-label="ñode2" foreign-id="1">
> > + <category name="AC" />
> > + <category name="BE" />
> > + <category name="high" />
> > + <category name="Park Plaza" />
> > + <interface ip-addr="192.168.2.1" status="1"
> snmp-primary="P"
> > descr="VPN interface">
> > + <monitored-service service-name="ICMP"/>
> > + <monitored-service service-name="SNMP"/>
> > + </interface>
> > + <interface ip-addr="192.168.2.2" status="1"
> snmp-primary="S"
> > descr="Management interface">
> > + <monitored-service service-name="ICMP"/>
> > + <monitored-service service-name="HTTP"/>
> > + </interface>
> > + <interface ip-addr="192.168.2.3" status="1"
> snmp-primary="N"
> > descr="Gateway interface">
> > + <monitored-service service-name="ICMP"/>
> > + </interface>
> > + </node>
> > +</model-import>
> >
> >
> > This was sent by the SourceForge.net collaborative development platform,
> > the world's largest Open Source development site.
> >
> > -------------------------------------------------------------------------
> > Check out the new SourceForge.net Marketplace.
> > It's the best place to buy or sell services for
> > just about anything Open Source.
> > http://sourceforge.net/services/buy/index.php
> > _______________________________________________
> > Please read the OpenNMS Mailing List FAQ:
> > http://www.opennms.org/wiki/index.php?page=MailingListFaq
> > opennms-cvs mailing list
> >
> > To *unsubscribe* or change your subscription options, see the bottom of
> > this page:
> > https://lists.sourceforge.net/lists/listinfo/opennms-cvs
>
> -------------------------------------------------------------------------
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services for
> just about anything Open Source.
> http://sourceforge.net/services/buy/index.php
> _______________________________________________
> Please read the OpenNMS Mailing List FAQ:
> http://www.opennms.org/index.php/Mailing_List_FAQ
>
> opennms-devel mailing list
>
> To *unsubscribe* or change your subscription options, see the bottom of
> this page:
> https://lists.sourceforge.net/lists/listinfo/opennms-devel
>
-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ
opennms-devel mailing list
To *unsubscribe* or change your subscription options, see the bottom of this
page:
https://lists.sourceforge.net/lists/listinfo/opennms-devel
