Yeah, that works fine.
I've created a test that reproduces the problem. I think it can only occur
when an Object has a Map as a field. This is the only way we can get a
period into something Mongo uses as a key (I'm guesstimating here) since
fields names themselves obviously can't use a period.
I can think of a few solutions. Two had problems though. Editing the actual
object passed into the converter via reflection or editing the map produced
by the current converter[1] mean you need to recursively check every map
for another map, probably all the way up the object hierarchy too. It's a
bit messy and in what's likely to be the majority of the use cases a
pointless overhead!
Given this I created a converter that maps the object to a string. This
lets Jackson take care of the details and leaves us to simply edit a string
before creating the DBObject.
String json = MongoDbBasicConverters.objectMapper.writeValueAsString(value);
json = json.replace(".", ",");
answer = (DBObject) JSON.parse(json);
This works in all the tests (including the new period key test) except
those that pass a String array of Json. The object to json string object
mapper ends up creating some kind of weird json of json, I suppose that's
to be expected. If I leave the original converter [1] in but change the
signature to the following:
public static DBObject fromAnyArrayToDBObject(final Object[] value).
all the tests pass (since array objects use the mapper conversion
strategy[1]).
This does leave a use case open where somebody could have an array of maps,
so perhaps some kind of check would be required on the elements...if it's
worth worrying about for now.
I can't figure out how to get the type conversion to trigger on the way
back out of the database though, e.g. in a find operation. Any help would
be appreciated there.
Perhaps offline so as to avoid spamming the user list.
Thanks!
P.S. I guess there could be a property in the endpoint uri to allow users
to configure the period replacement character(s) if they didn't want to use
a comma for some reason.
[1] Line 67 in
https://fisheye6.atlassian.com/browse/camel/trunk/components/camel-mongodb/src/main/java/org/apache/camel/component/mongodb/converters/MongoDbBasicConverters.java?hb=true
On 6 October 2012 02:19, Raul Kripalani <r...@evosent.com> wrote:
> Hi Mark,
>
> I just committed a fix for the tests in r1394865. It turns out that the
> PropertiesComponent was being added too late in the route lifecycle, which
> worked before r1373402, but not after.
>
> You should now be able to run the tests locally simply by starting MongoDB.
> The tests are prepared to connect to the default Mongo port @ localhost. If
> your config is different, just adjust mongodb.test.properties accordingly.
>
> Thanks in advance for your contribution ;)
>
> Regards,
>
> *Raúl Kripalani*
> Apache Camel Committer
> Enterprise Architect, Program Manager, Open Source Integration specialist
> http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
> http://blog.raulkr.net | twitter: @raulvk <http://twitter.com/raulvk>
>
> On Fri, Oct 5, 2012 at 6:08 PM, Raul Kripalani <r...@evosent.com> wrote:
>
> > Apologies for not being able to look at this yesterday. Thanks for all
> the
> > feedback from your side.
> >
> > Unfortunately the unit tests of this component are disabled on our
> Jenkins
> > because MongoDB is not embeddable and our unit tests are really
> > "integration tests" from the purist point of view, because they target a
> > real MongoDB instance. Probably some framework change between 2.10.0 and
> > now have introduced this regression.
> >
> > I'll try and take a look at this during the weekend.
> >
> > Sometime ago I was in talks with a MongoDB cloud-hosting company about a
> > free MongoDB cloud instance for the Camel project just for our automated
> > testing. I have to pick up that conversation again.
> >
> > Regards,
> >
> > *Raúl Kripalani*
> > Apache Camel Committer
> > Enterprise Architect, Program Manager, Open Source Integration specialist
> > http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
> > http://blog.raulkr.net | twitter: @raulvk <http://twitter.com/raulvk>
> >
> > On Fri, Oct 5, 2012 at 5:53 PM, Mark Doyle <markjohndo...@gmail.com
> >wrote:
> >
> >> I have a little more info.
> >>
> >> The properties issue is evident on both the command line (mvn install)
> and
> >> running the junit test directly in elipse.
> >>
> >> I tried the spring 3.1 profile on the command line (mvn clean install -P
> >> spring3.1) and got the same problem.
> >>
> >> What is weird is it suddenly stopped finding the latest
> camel-test-spring
> >> dependency, no version is specified in the POM so unless it's managed
> in a
> >> parent I guess it will go for the latest snapshot. I had to
> >> explicitly specify <version>2.11-SNAPSHOT</version> in the pom to get it
> >> to
> >> compile again. Having said all that, I still get the Properties issue :)
> >>
> >> On 4 October 2012 14:20, Raul Kripalani <r...@evosent.com> wrote:
> >>
> >> > Strange, will check this evening and will send you a direct email with
> >> my
> >> > findings.
> >> > Thanks for your collaboration ;)
> >> >
> >> > Regards,
> >> >
> >> > *Raúl Kripalani*
> >> > Apache Camel Committer
> >> > Enterprise Architect, Program Manager, Open Source Integration
> >> specialist
> >> > http://about.me/raulkripalani |
> >> http://www.linkedin.com/in/raulkripalani
> >> > http://blog.raulkr.net | twitter: @raulvk <http://twitter.com/raulvk>
> >> >
> >> > On Wed, Oct 3, 2012 at 10:23 PM, Mark Doyle <markjohndo...@gmail.com>
> >> > wrote:
> >> >
> >> > > Hi Raul,
> >> > >
> >> > > I'm getting the trace below when I try to run the unit tests. Looks
> >> like
> >> > a
> >> > > problem adding the Properties component (using
> >> mongdb.test.properties) to
> >> > > the camel spring context. Any ideas?
> >> > >
> >> > > java.lang.IllegalArgumentException: Cannot add component as its
> >> already
> >> > > previously added: properties
> >> > > at
> >> > >
> >> > >
> >> >
> >>
> org.apache.camel.impl.DefaultCamelContext.addComponent(DefaultCamelContext.java:291)
> >> > > at
> >> > >
> >> > >
> >> >
> >>
> org.apache.camel.component.mongodb.MongoDbOperationsTest$1.configure(MongoDbOperationsTest.java:227)
> >> > > at
> >> > >
> >> > >
> >> >
> >>
> org.apache.camel.builder.RouteBuilder.checkInitialized(RouteBuilder.java:322)
> >> > > at
> >> > >
> >> > >
> >> >
> >>
> org.apache.camel.builder.RouteBuilder.configureRoutes(RouteBuilder.java:276)
> >> > > at
> >> > >
> >> > >
> >> >
> >>
> org.apache.camel.builder.RouteBuilder.addRoutesToCamelContext(RouteBuilder.java:262)
> >> > > at
> >> > >
> >> > >
> >> >
> >>
> org.apache.camel.impl.DefaultCamelContext.addRoutes(DefaultCamelContext.java:633)
> >> > > at
> >> > >
> >> > >
> >> >
> >>
> org.apache.camel.test.junit4.CamelTestSupport.doSetUp(CamelTestSupport.java:293)
> >> > > at
> >> > >
> >> > >
> >> >
> >>
> org.apache.camel.test.junit4.CamelTestSupport.setUp(CamelTestSupport.java:213)
> >> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >> > > at
> >> > >
> >> > >
> >> >
> >>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> >> > > at
> >> > >
> >> > >
> >> >
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >> > > at java.lang.reflect.Method.invoke(Method.java:597)
> >> > > at
> >> > >
> >> > >
> >> >
> >>
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> >> > > at
> >> > >
> >> > >
> >> >
> >>
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> >> > > at
> >> > >
> >> > >
> >> >
> >>
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> >> > > at
> >> > >
> >> > >
> >> >
> >>
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)
> >> > > at
> >> > >
> >> >
> >>
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
> >> > > at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:47)
> >> > > at org.junit.rules.RunRules.evaluate(RunRules.java:18)
> >> > > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> >> > > at
> >> > >
> >> > >
> >> >
> >>
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> >> > > at
> >> > >
> >> > >
> >> >
> >>
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> >> > > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> >> > > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> >> > > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> >> > > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> >> > > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> >> > > at
> >> > >
> >> > >
> >> >
> >>
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> >> > > at
> >> > >
> >> >
> >>
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
> >> > > at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> >> > > at
> >> > >
> >> > >
> >> >
> >>
> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
> >> > > at
> >> > >
> >> > >
> >> >
> >>
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> >> > > at
> >> > >
> >> > >
> >> >
> >>
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
> >> > > at
> >> > >
> >> > >
> >> >
> >>
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
> >> > > at
> >> > >
> >> > >
> >> >
> >>
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
> >> > > at
> >> > >
> >> > >
> >> >
> >>
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
> >> > >
> >> > > java.lang.NullPointerException
> >> > > at
> >> > >
> >> > >
> >> >
> >>
> org.apache.camel.component.mongodb.AbstractMongoDbTest.cleanup(AbstractMongoDbTest.java:96)
> >> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >> > > at
> >> > >
> >> > >
> >> >
> >>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> >> > > at
> >> > >
> >> > >
> >> >
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >> > > at java.lang.reflect.Method.invoke(Method.java:597)
> >> > > at
> >> > >
> >> > >
> >> >
> >>
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> >> > > at
> >> > >
> >> > >
> >> >
> >>
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> >> > > at
> >> > >
> >> > >
> >> >
> >>
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> >> > > at
> >> > >
> >> >
> >>
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:36)
> >> > > at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:47)
> >> > > at org.junit.rules.RunRules.evaluate(RunRules.java:18)
> >> > > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> >> > > at
> >> > >
> >> > >
> >> >
> >>
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> >> > > at
> >> > >
> >> > >
> >> >
> >>
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> >> > > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> >> > > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> >> > > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> >> > > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> >> > > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> >> > > at
> >> > >
> >> > >
> >> >
> >>
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> >> > > at
> >> > >
> >> >
> >>
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
> >> > > at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> >> > > at
> >> > >
> >> > >
> >> >
> >>
> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
> >> > > at
> >> > >
> >> > >
> >> >
> >>
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> >> > > at
> >> > >
> >> > >
> >> >
> >>
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
> >> > > at
> >> > >
> >> > >
> >> >
> >>
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
> >> > > at
> >> > >
> >> > >
> >> >
> >>
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
> >> > > at
> >> > >
> >> > >
> >> >
> >>
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
> >> > >
> >> > >
> >> > > On 2 October 2012 21:26, Mark Doyle <markjohndo...@gmail.com>
> wrote:
> >> > >
> >> > > > Indeed I would; first job is to figure out how to create a JIRA...
> >> > > >
> >> > > > as for your question, yes, you assume correctly. It's the
> following
> >> > > simple
> >> > > > model object:
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> https://github.com/JohannesKlug/hbird/blob/master/src/core/commons/src/main/java/org/hbird/core/commons/tmtcgroups/HummingbirdParameter.java
> >> > > >
> >> > > > Which I assume triggers the fromAnyObjectToDBObject method in the
> >> > > > MongoDbBasicConverters class? The qualified name field always
> >> contains
> >> > > > periods.
> >> > > >
> >> > > >
> >> > > > On 2 October 2012 20:11, Raul Kripalani <r...@evosent.com> wrote:
> >> > > >
> >> > > >> Hi Mark,
> >> > > >>
> >> > > >> Correct. Mongo doesn't accept dot characters in field naming
> >> because
> >> > it
> >> > > >> serves as a separator for nested fields, so it's reserved.
> >> > > >>
> >> > > >> It seems like a great idea. Please feel free to create a JIRA.
> >> Would
> >> > you
> >> > > >> be
> >> > > >> happy to work on a patch? If you want I can provide you with some
> >> > > guidance
> >> > > >> since I know this component inside out because I created it ;)
> >> > > >>
> >> > > >> There are a number of type converters that MongoDB provides...
> >> What's
> >> > > the
> >> > > >> original payload type before it hits the MongoDB endpoint?
> (Object,
> >> > > >> HashMap, etc.). I assume it's an Object but please confirm.
> >> > > >>
> >> > > >> Thanks,
> >> > > >>
> >> > > >> --
> >> > > >> *Raúl Kripalani*
> >> > > >> Apache Camel Committer
> >> > > >> Enterprise Architect, Program Manager, Open Source Integration
> >> > > specialist
> >> > > >> http://about.me/raulkripalani |
> >> > > http://www.linkedin.com/in/raulkripalani
> >> > > >> http://blog.raulkr.net | twitter: @raulvk <
> >> http://twitter.com/raulvk>
> >> > > >>
> >> > > >>
> >> > > >> On Tue, Oct 2, 2012 at 4:22 PM, Mark Doyle <
> >> markjohndo...@gmail.com>
> >> > > >> wrote:
> >> > > >>
> >> > > >> > I have a problem where a String field contains a period. This
> >> causes
> >> > > the
> >> > > >> > following exception:
> >> > > >> >
> >> > > >> > Caused by: java.lang.IllegalArgumentException: fields stored in
> >> the
> >> > db
> >> > > >> > can't have . in them. (Bad Key: 'Stock6.tm.Azimuth')
> >> > > >> >
> >> > > >> > which I believe stems from restrictions in the depths of the
> >> mongo!
> >> > > >> >
> >> > > >> > The solution seems to be to convert the character to something
> >> else
> >> > > >> (e.g. a
> >> > > >> > comma) before inserting and back again after finding. Does this
> >> seem
> >> > > >> like
> >> > > >> > something the camel-mongo component should offer as an option,
> or
> >> > even
> >> > > >> > silently deal with by default? Could it be a change in the type
> >> > > >> converters?
> >> > > >> > I think that uses Jackson to build the DB object and therefore
> >> could
> >> > > >> filter
> >> > > >> > out and replace periods. I suppose a user could run into
> >> problems if
> >> > > >> they
> >> > > >> > overrode the converters with their own.
> >> > > >> >
> >> > > >> > I don't know how camel works under the covers so this might not
> >> > work,
> >> > > >> > especially if the converters aren't used to create a real
> object
> >> > from
> >> > > >> > the DBObject returned by Mongo.
> >> > > >> >
> >> > > >>
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>
