Hi Alberto, We agree completely :-) it's great that you push as much querying down to the backend. I was only suggesting you start with making something that is "slow and stupid, but functional" and then optimize. But I think you're already doing much better :-)
Consider making this as a fork to the MetaModel source. At least, I think we could integrate this quite soon into the main MM source tree. I do think however we would need a contributor license agreement from you to accept it (it's not just a patch, but more substantial). Have you already made an ICLA for ASF? Best regards, Kasper 2014-09-12 22:10 GMT+02:00 Alberto Rodriguez <[email protected]>: > Hi Kasper, > > Ok got it! thank you for explaining to me. > > I will implement the materializeMainSchemaTable but as you said I think is > a good idea to take advantage of the very powerful ElasticSearch API which > has a DSL that support natively queries and filters, have a look at them: > Query > DSL > < > http://www.elasticsearch.org/guide/en/elasticsearch/client/java-api/current/query-dsl-queries.html > > > , Query dsl filters > < > http://www.elasticsearch.org/guide/en/elasticsearch/client/java-api/current/query-dsl-filters.html > >. > By using these APIs we would also get a huge performance improvement > because we were using Apache Lucene under-the-hood. > > I will let you know if there is any further update. Thank you!! > > Kind regards, > > Alberto > > > > > 2014-09-12 14:52 GMT+02:00 Kasper Sørensen <[email protected] > >: > > > Hi Alberto, > > > > Yes you are right that these ones do override executeQuery(Query). In the > > case of JDBC, it's a completely different story so let's leave that out > :-) > > In the case of MongoDB this method is overridden to catch a few optimized > > corner cases, which is fine. But in case those corner cases are not met, > it > > delegates to super.executeQuery(Query), which in turn will delegate to > > materializeMainSchemaTable(...) then! That is why, if you implement > > materializeMainSchemaTable(...) then at least you are 100% functional, > > because MM can then always revert to doing any filtering and aggregation > in > > memory! It may not be performing well at all, but it's a great start to > be > > functional and compatible with the query model. > > > > I would in deed recommend you catch the corner cases of the WHERE items > > that you can support natively. And also to implement > executeCountQuery(...) > > would be good. > > > > Best regards, > > Kasper > > > > 2014-09-12 14:35 GMT+02:00 Alberto Rodriguez <[email protected]>: > > > > > Hi Kasper, > > > > > > thank you for the feedback! > > > > > > I am not sure if I fully understand your piece of advice related to not > > > implement the executeQuery(query) method. I've been having a look at > the > > > different modules (jdbc, mongo...) and they do override this method. > I've > > > just taken a look at the materializeTable method signature and it does > > not > > > include, for instance, the FilterItem list so I wouldn't have acces to > > the > > > filters within this method. > > > > > > I've just commited some changes that allow to add simple where/isEquals > > > clause to the query (although they are inside the executeQuery). > > > > > > I have also just upgraded the dependency to the latest version (thank > > you!) > > > > > > Kind regards, > > > > > > Alberto > > > > > > 2014-09-12 13:58 GMT+02:00 Kasper Sørensen < > > [email protected] > > > >: > > > > > > > Hi Alberto, > > > > > > > > I gave you project a quick look, and it's already quite awesome work! > > > :-) I > > > > like the way you discover metadata ... I didn't know of that > approach, > > > so I > > > > am looking forward to trying it out. > > > > > > > > A quick thing I noticed is that your DataContext overrides the > > > > executeQuery(Query) method. You probably shouldn't do that, since > your > > > > superclass does all the hard work. Simply implement the > > > > materializeTable(...) method and you're much better off. > > > > > > > > If you upgrade your dependency to rely on the latest version of MM > > (4.2) > > > > then you will even have a lot more helper methods to implement in > your > > > > DataContext. Such as executePrimaryKeyLookupQuery and others. Doing > > that > > > > will quickly get your going to a very broad query support and better > > > > performance. > > > > > > > > Best regards, > > > > Kasper > > > > > > > > > > > > 2014-09-12 13:27 GMT+02:00 Hans Drexler < > > [email protected] > > > >: > > > > > > > > > Just a small correction. MetaModel is not my project. I am not > nearly > > > > > smart enough for that. But I'll do my best to give you some > > worthwhile > > > > > feedback. > > > > > > > > > > Cheers, > > > > > > > > > > Hans > > > > > > > > > > -----Original Message----- > > > > > From: Alberto Rodriguez [mailto:[email protected]] > > > > > Sent: Friday, September 12, 2014 1:03 PM > > > > > To: [email protected] > > > > > Subject: Re: ElasticSearch Module > > > > > > > > > > Hi Hans, > > > > > > > > > > no worries!! I think your project is awesome and I would like to > > > > > contribute somehow. > > > > > > > > > > I will keep working on the module. > > > > > > > > > > Regards, > > > > > > > > > > Alberto > > > > > > > > > > 2014-09-12 12:58 GMT+02:00 Hans Drexler < > > > [email protected] > > > > >: > > > > > > > > > > > Hi Alberto, > > > > > > > > > > > > What great news. I will checkout your code and try to give > > comments. > > > > > > Thank you for your contributon. > > > > > > > > > > > > Regards, > > > > > > > > > > > > Hans > > > > > > > > > > > > Op 12 sep. 2014 11:13 schreef Alberto Rodriguez < > [email protected] > > >: > > > > > > Hi there, > > > > > > > > > > > > as per our previous discussion I have been taking a look at the > > > > > > elasticsearch module for metamodel. > > > > > > > > > > > > I have been working on it and I have a very simple version which > is > > > > > > already fetching the metadata from ES. I have also managed to > make > > > > > > simple queries work (queries without filters and where clauses). > > > > > > > > > > > > Could you please have a look at the project on my github account > > and > > > > > > let me know what you think about it?: Metamodel-elasticsearch > > module > > > > > > <https://github.com/ardlema/metamodel-elasticsearch> > > > > > > > > > > > > Thank you in advance. > > > > > > > > > > > > Kind regards, > > > > > > > > > > > > Alberto > > > > > > > > > > > > > > > > > > > > >
