Interesting idea. I'll look into the JSONLayout as a possible solution.
On Thu, Apr 25, 2013 at 1:13 PM, Ralph Goers <[email protected]>wrote: > I would think that for something like MongoDB you would want to use a > JSONLayout and then just insert that. I can tell you that for Cassandra it > is considerably more complicated. > > Ralph > > > > > On Apr 25, 2013, at 9:25 AM, Nicholas Williams wrote: > > > On Thu, Apr 25, 2013 at 11:17 AM, Nicholas Williams < > [email protected]> wrote: > >> >> On Thu, Apr 25, 2013 at 11:15 AM, Nicholas Williams < >> [email protected]> wrote: >> >>> >>> On Thu, Apr 25, 2013 at 11:06 AM, Gary Gregory >>> <[email protected]>wrote: >>> >>>> I thought it included a vendor neutral NoSQL API in its stack >>>> someplace. >>>> >>> >>> From what I can tell, it includes a vendor-neutral way to connect to its >>> cloud-based application engine. But you must either connect to a hosted >>> cloud application engine, or you must download and install the cloud >>> application engine on your own machine(s) and then connect to it. There >>> doesn't appear to be an actual, extractable database API that can be used >>> to connect directly to generic NoSQL databases. >>> >>> But I could be wrong. I'm not finding a whole lot of useful information >>> on their website. >>> >> >> From their GitHub repository: >> >> "AppScale is a platform that allows users to deploy and host their own >> Google App Engine applications. It executes automatically over Amazon EC2, >> Rackspace, Eucalyptus, CloudStack, OpenStack, as well as Xen, VirtualBox, >> VMWare, and KVM. It has been developed and is maintained by AppScale >> Systems, Inc., in Santa Barbara, CA. >> >> "It supports the Python, Java, and Go Google App Engine platforms." >> >> > Doing some Googling, there doesn't seem to be a vendor-neutral Java API > for NoSQL databases at all--which doesn't surprise me. But if you think > about it, even if there were, we don't need something that large and (for > us) cumbersome. All we need is a straightforward API for inserting simple > POJOs. I think it would be more efficient/easier to simply create that > two-interface API and implement it for each NoSQL database supported. > > >> >> >>> >>> >>>> >>>> Gary >>>> >>>> On Apr 25, 2013, at 11:47, Nicholas Williams < >>>> [email protected]> wrote: >>>> >>>> >>>> On Thu, Apr 25, 2013 at 10:43 AM, Gary Gregory >>>> <[email protected]>wrote: >>>> >>>>> On Thu, Apr 25, 2013 at 11:37 AM, Nicholas Williams < >>>>> [email protected]> wrote: >>>>> >>>>>> >>>>>> On Thu, Apr 25, 2013 at 10:01 AM, Remko Popma <[email protected]>wrote: >>>>>> >>>>>>> I think this is the link Gary is talking about: (from the wiki) >>>>>>> Build a NoSQL Appender, maybe with >>>>>>> AppScale<http://wiki.apache.org/logging/AppScale> >>>>>>> : http://appscale.cs.ucsb.edu/datastores.html Inspiration came from >>>>>>> the log4j1 appender for redis: >>>>>>> https://github.com/pavlobaron/log4j2redis >>>>>>> >>>>>>> >>>>>> Unless I'm misunderstanding something (very possible), AppScale seems >>>>>> like a lot of overhead and overkill. However, it should certainly be >>>>>> possible to create a NoSqlAppender that has a "type" property where you >>>>>> specify the database type (Mongo, Redis, etc.). >>>>>> >>>>>> One thing we need to keep in mind is dependencies: appending to Mongo >>>>>> will require the org.mongodb:mongo-java-driver artifact (and its >>>>>> dependencies), Redis will require the redis.clients:jedis artifact (and >>>>>> its >>>>>> dependencies), etc. So, as far as I see it, we have two options: >>>>>> >>>>>> A) Have one NoSqlAppender that initially supports MongoDB and can >>>>>> easily be made to support other NoSQL databases, put it in the log4j-core >>>>>> artifact, and mark all dependencies as OPTIONAL or PROVIDED (thoughts? >>>>>> opinions?) so as not to cause using projects to download unnecessary >>>>>> dependencies. >>>>>> >>>>>> B) Put NoSQL appenders in separate artifacts so that dependencies >>>>>> don't have to be optional/provided, but as a result each NoSQL will have >>>>>> to >>>>>> have its own appender and each will result in a separate log4j artifact. >>>>>> >>>>>> I REALLY don't like option B, but optional/provided dependencies also >>>>>> require careful, explicit documentation and, my bet, end up with lots of >>>>>> mailing list emails "Why am I getting a NoClassDefError?". >>>>>> >>>>> >>>>> That's not an issue IMO since you only get the error when the NoSQL >>>>> appender class would be loaded. If you use plain log4j, you'd never see >>>>> the >>>>> error. >>>>> >>>>> If we do a NoSQL appender, it seems to me that each will have it's own >>>>> idiosyncrasies and configuration tweak, so a single generic class is not >>>>> going to be pretty so I would do: >>>>> >>>>> - an AppScale appender or equivalent JDBC-like vendor neutral API >>>>> - If need be our own, with a NoSQL abstract appender and vendor >>>>> specific subclasses. >>>>> >>>> >>>> I may not understand what AppScale is exactly. It looks like just a >>>> cloud service. Is that correct? >>>> >>>> >>>>> >>>>> Gary >>>>> >>>>> >>>>>> Agree with Gary on keeping things simple. Also agree every new >>>>>>> feature needs to be in beta for a while to shake out bugs etc. I don't >>>>>>> have >>>>>>> an opinion on whether to include jdbc appenders in the first 2.0 >>>>>>> release or >>>>>>> add them later. >>>>>>> >>>>>>> You know, I was actually thinking to write a tutorial for custom >>>>>>> plugin developers, called "(How to) Write Your Own Darn JdbcAppender!" >>>>>>> :-) >>>>>>> >>>>>>> Sent from my iPhone >>>>>>> >>>>>>> On 2013/04/25, at 23:51, Gary Gregory <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>> On Thu, Apr 25, 2013 at 10:39 AM, Nicholas Williams < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> First, a quick question: do we anticipate the next version to be >>>>>>>> beta6 or rc1? Just curious. >>>>>>>> >>>>>>> >>>>>>> As long as we are adding features, I'd like to keep rolling Betas. >>>>>>> But it would also be OK to release 2.0 and add appenders later. >>>>>>> >>>>>>> I tried porting our app to 2.0 a couple of weeks ago but ran into >>>>>>> lots of issues, so I'll need to take another stab at it in a couple of >>>>>>> weeks again. We rely on a lot of 1.0 guts so I'll have to think about >>>>>>> that >>>>>>> some more... >>>>>>> >>>>>>> >>>>>>>> I'm currently working on cleaning up compiler warnings throughout >>>>>>>> the project and should have that completed soon. >>>>>>>> >>>>>>> >>>>>>> Great! >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> I want to go ahead and get the conversation started about database >>>>>>>> appenders. I'd like to see two appenders: >>>>>>>> >>>>>>>> - A JdbcAppender that is capable of logging to any RDBMS for which >>>>>>>> there is a JDBC driver. >>>>>>>> - A MongoAppender that is capable of logging to a MongoDB database. >>>>>>>> >>>>>>> >>>>>>> We should not need a MongoDB appender if there is a JDBC driver for >>>>>>> it: docs.mongodb.org/ecosystem/drivers/java/ >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> The JdbcAppender and MongoAppender would, as far as I can tell, >>>>>>>> need properties for mapping all of the possible logging event >>>>>>>> properties to >>>>>>>> table columns (or Mongo equivalent). I don't really see any other way >>>>>>>> to >>>>>>>> accomplish that. We could use layout patterns from the PatternLayout to >>>>>>>> achieve this: <column name="columnName" pattern="PatternLayout >>>>>>>> equivalent-pattern" /> >>>>>>>> >>>>>>> >>>>>>> You can look at Log4J 1 for inspiration. Keep it simple for a start. >>>>>>> I think version 1 just let's you specify a SQL INSERT instead of using >>>>>>> some >>>>>>> XML for mapping. >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> I imagine the JdbcAppender having mutually exclusive properties for >>>>>>>> JDBC URL/username/password, DataSource JNDI URL, and >>>>>>>> class.staticFactoryMethod for obtaining a DataSource. >>>>>>>> >>>>>>> >>>>>>> Keep is simple for the first cut ;) >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> The MongoAppender would similarly have mutually exclusive >>>>>>>> properties for connection information and class.statucFactoryMethod for >>>>>>>> obtaining a Mongo instance. >>>>>>>> >>>>>>>> I'd like to take a stab at these after I complete fixing compiler >>>>>>>> warnings, and wanted to start getting feedback/ideas and also see if >>>>>>>> anyone >>>>>>>> has use cases for other NoSQL appenders. >>>>>>>> >>>>>>> >>>>>>> Search the ML for my note on NoSQL, it looks like there is a >>>>>>> JDBC-like API for NoSQL DBs. >>>>>>> >>>>>>> Gary >>>>>>> >>>>>>> -- >>>>>>> E-Mail: [email protected] | [email protected] >>>>>>> Java Persistence with Hibernate, Second >>>>>>> Edition<http://www.manning.com/bauer3/> >>>>>>> JUnit in Action, Second Edition <http://www.manning.com/tahchiev/> >>>>>>> Spring Batch in Action <http://www.manning.com/templier/> >>>>>>> Blog: http://garygregory.wordpress.com >>>>>>> Home: http://garygregory.com/ >>>>>>> Tweet! http://twitter.com/GaryGregory >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> E-Mail: [email protected] | [email protected] >>>>> Java Persistence with Hibernate, Second >>>>> Edition<http://www.manning.com/bauer3/> >>>>> JUnit in Action, Second Edition <http://www.manning.com/tahchiev/> >>>>> Spring Batch in Action <http://www.manning.com/templier/> >>>>> Blog: http://garygregory.wordpress.com >>>>> Home: http://garygregory.com/ >>>>> Tweet! http://twitter.com/GaryGregory >>>>> >>>> >>>> >>> >> > >
