Re: Issues with using Paoding to index Chinese characters
I'm still trying to find out which version it is compatible for, but the document which I've followed is written in 2012. http://java.dzone.com/articles/indexing-chinese-solr Regards, Edwin On 12 June 2015 at 20:15, Upayavira u...@odoko.co.uk wrote: Not knowing anything about paoding, it seems that this library isn't compatible with the current version of Solr/Lucene. Have a look at the version that it was compiled for. Having looked at the date of the latest download (2008) Lucene has changed a LOT since then, so some conversion work will definitely be needed to make it work. Upayavira On Fri, Jun 12, 2015, at 08:28 AM, Zheng Lin Edwin Yeo wrote: I'm trying to use Paoding to index Chinese characters in Solr. I'm using Solr 5.1, have downloaded the dictionary to shard1\dic and shard2\dic, and have configured the following in schema,xml fieldType name=text_chinese class=solr.TextField analyzer class=net.paoding.analysis.analyzer.PaodingAnalyzer/ /fieldType I've also included -DPAODING_DIC_HOME=/dic during my startup of Solr However, when I tried to start Solr, I get the following error: java.lang.VerifyError: class net.paoding.analysis.analyzer.PaodingAnalyzerBean overrides final method tokenStream.(Ljava/lang/String;Ljava/io/Reader;)Lorg/apache/lucene/analysis/TokenStream; at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(Unknown Source) at java.security.SecureClassLoader.defineClass(Unknown Source) at java.net.URLClassLoader.defineClass(Unknown Source) at java.net.URLClassLoader.access$100(Unknown Source) at java.net.URLClassLoader$1.run(Unknown Source) at java.net.URLClassLoader$1.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(Unknown Source) at org.eclipse.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:421) at org.eclipse.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:383) at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(Unknown Source) at java.security.SecureClassLoader.defineClass(Unknown Source) at java.net.URLClassLoader.defineClass(Unknown Source) at java.net.URLClassLoader.access$100(Unknown Source) at java.net.URLClassLoader$1.run(Unknown Source) at java.net.URLClassLoader$1.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(Unknown Source) at org.eclipse.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:421) at java.lang.ClassLoader.loadClass(Unknown Source) at java.net.FactoryURLClassLoader.loadClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) at java.net.FactoryURLClassLoader.loadClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Unknown Source) at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:476) at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:423) at org.apache.solr.schema.FieldTypePluginLoader.readAnalyzer(FieldTypePluginLoader.java:262) at org.apache.solr.schema.FieldTypePluginLoader.create(FieldTypePluginLoader.java:94) at org.apache.solr.schema.FieldTypePluginLoader.create(FieldTypePluginLoader.java:42) at org.apache.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:151) at org.apache.solr.schema.IndexSchema.readSchema(IndexSchema.java:489) at org.apache.solr.schema.IndexSchema.init(IndexSchema.java:175) at org.apache.solr.schema.IndexSchemaFactory.create(IndexSchemaFactory.java:55) at org.apache.solr.schema.IndexSchemaFactory.buildIndexSchema(IndexSchemaFactory.java:69) at org.apache.solr.core.ConfigSetService.createIndexSchema(ConfigSetService.java:102) at org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:74) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:516) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:283) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:277) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Is there anything which I've done wrong or missed out? Regards, Edwin
Re: Division with Stats Component when Grouping in Solr
I was able to get the new version of Solr installed. This query gets me
really close, but it is averaging the rows BEFORE the grouping so it's not
totally accurate. I need it to sum the visits and bounces by keyword and
then perform the division. The avg here probably seems confusing and
pointless, but it wouldn't let me just put the div directly in the facet
without wrapping it with a function.
So instead of summing all the rows into one group and performing the divide,
it is diving each row one by one and then averaging them together which
creates skewed results since one day may have more data than the other.
It seems dividing is possible if only I can tell it to divide the grouped by
keyword result and not the individual rows and having to average them
together, etc.
Here is what I have (granted it's a simplified version for testing)
json.facet={
keywords:{
type:terms,
limit:10,
field:keyword,
facet:{
bounces_sum:sum(bounces),
visits_sum:sum(visits),
bounce_rate:avg(div(sum(bounces),sum(visits)))
}
}
}
What I really want is:
bounce_rate: div(bounces_sum, visits_sum) ... but this doesn't work.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Division-with-Stats-Component-when-Grouping-in-Solr-tp4211402p4211639.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Division with Stats Component when Grouping in Solr
Not sure why but half of my posts are showing up as not accepted by the mailing list. I've made a few replies to others that haven't gone through. I am not sure if it's because I'm replying via email or what the issue is. -- View this message in context: http://lucene.472066.n3.nabble.com/Division-with-Stats-Component-when-Grouping-in-Solr-tp4211402p4211631.html Sent from the Solr - User mailing list archive at Nabble.com.
Parent/Child (Nested Document) Faceting
Hey Folks, I'd love some feedback on the interface for nested document
faceting (or rather switching facet domains to/from parent/child).
See the bottom of this blog:
http://yonik.com/solr-nested-objects/
Issue #1: How to specify that one should change domains before faceting?
I originally started out with a new facet type (like query facet, but
switches domains).
So if you started out querying a child of type book, you would first
do a blockParent facet to map the domain to parents, and then put
the actual facet you wanted as a sub-facet.
q=book_review:xx /* query some child-doc of book */
json.facet=
{ // NOTE: this was my first pass... not the current interface
books : {
type: blockParent,
parentFilter : type:book
facet : {
authors : {
type : terms,
field : author
}
}
}
Although having a separate facet type to map domains is logically very
clean, it does introduce an additional level of indentation which may
not be desired.
So then I thought about including domain switching operations under a
domain directive in the facet itself:
json.facet=
{ // current form a domain switching facet
authors : {
type: terms,
field: author,
domain : {blockParent:type:book}
}
}
I envision some future other options for domain including the
ability to reset the domain with another query (ignoring your parent
domain), or adding additional filters to the domain before faceting,
or normal (non-block) joins.
Issue #2: Naming
I avoided toParent and toChild because people cloud be confused that
it would work on any sort of parent/child relationship (i.e. other
than nested documents).
I used blockParent and blockChildren because I was thinking about
block join.
One alternative that might be better could be nested (i.e. nestedParent).
Pluralization:
I picked the singular for blockParent and plural for blockChildren
since a single block as one parent and multiple children. But you
could think about it in other ways since we're mapping a set of
documents at a time (i.e. both could be pluralized).
Options:
nestedParent, nestedChildren // current option
nestedParents, nestedChildren // both plural
nestedChild, nestedParent// both singular
Feedback appreciated!
-Yonik
Re: Division with Stats Component when Grouping in Solr
kingofhypocrites: Usually that's because your e-mail formats with html or some other non-plain-text format. Try sending them as plain text. On Sat, Jun 13, 2015 at 5:26 PM, kingofhypocrites kingofhypocri...@gmail.com wrote: Not sure why but half of my posts are showing up as not accepted by the mailing list. I've made a few replies to others that haven't gone through. I am not sure if it's because I'm replying via email or what the issue is. -- View this message in context: http://lucene.472066.n3.nabble.com/Division-with-Stats-Component-when-Grouping-in-Solr-tp4211402p4211631.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Division with Stats Component when Grouping in Solr
@Billnbell What did you conclude with the Analytics component? It sounds like you are saying it does the same thing as the stats component but it has several other features that aren't supported by the stats library. I'd love to have a talk with you offline if possible. -- View this message in context: http://lucene.472066.n3.nabble.com/Division-with-Stats-Component-when-Grouping-in-Solr-tp4211402p4211635.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Division with Stats Component when Grouping in Solr
@Yonik, Thanks for this! I was actually just looking at your blog earlier
today and thinking that the json facet feature may be just what I need. I'm
using Solr. 4.3 currently as that is what comes with DataStax, so I'm trying
to create a new build with the latest Solr version so i can test this
feature. For the sort I am assuming this would be sorting on sum(visits) for
the given keyword correct? Also can you confirm if it's possible to do a
division in the facet? Something like facet: { bouncerate: div(sum(bounces)
/ sum(visits)) } Because of the large number of results, I would need to
precalculate this (division operation) if they happen to sort on it. I don't
see anything like this mentioned in the api docs, so maybe it's not
possible.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Division-with-Stats-Component-when-Grouping-in-Solr-tp4211402p4211634.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Division with Stats Component when Grouping in Solr
This looks very promising if only I could get it to work: https://issues.apache.org/jira/browse/SOLR-5302 https://issues.apache.org/jira/secure/attachment/12606793/Search%20Analytics%20Component.pdf Various links it points to are broken now and i can't find anything about it online, but the PDF indicates I can set olap=true to turn it on, although this doesn't seem to do anything. The docs say it supports limiting the results and doing math operations on statistics which is exactly what I need. I'm not clear if I need to install this or if this component is even used anymore. On Fri, Jun 12, 2015 at 12:00 PM Joel Bernstein [via Lucene] ml-node+s472066n4211422...@n3.nabble.com wrote: https://issues.apache.org/jira/browse/SOLR-7560, will almost support this in Solr 5.3. The compound function support won't be there yet though. But it will be there in the near future. Joel Bernstein http://joelsolr.blogspot.com/ On Fri, Jun 12, 2015 at 9:30 AM, kingofhypocrites [hidden email] http:///user/SendEmail.jtp?type=nodenode=4211422i=0 wrote: I am migrating a database from SQL Server to Cassandra. Currently I have a setup as follows: - Log data in Cassandra - Summarize data in Spark and put into Cassandra summary tables - Query data in Solr Everything fits beautifully until I need to do stats on groups. I am hoping to get this to work with Solr so I can stick to one database, but I am not sure it's possible. If I had it in SQL Server, I could do it like so: SELECT site_id, keyword, SUM(visits) as visits, CONVERT(DECIMAL(13, 3), SUM(bounces)) / SUM(visits) as bounce_rate, SUM(pageviews) as pageviews, CONVERT(DECIMAL(13, 3), SUM(pageviews)) / SUM(visits) as avg_pages_per_visit FROM report_all_keywords_daily WHERE site_id = 55 AND date_key = '20150606' AND date_key = '20150608' GROUP BY site_id, keyword ORDER BY visits DESC Now I need to replicate this in Solr. The closest I could get to this is by using the Stats component and then using field collapsing. group=truegroup.field=keywordstats=truestats.field=visitsstats.facet=keyword And here are some results I get back: http://pastebin.com/raw.php?i=Fxhe2RA0 However, I need to do able to divide certain metrics. I tried including functions in the stats.field such as div(sum(bounce_rate), (sum(visits)) but it doesn't recognize the functions. Also it seems to ignoring the paging for the stats results and returns all groups regardless. Ultimately I'd like something like this which is what I would get in SQL: http://lucene.472066.n3.nabble.com/file/n4211402/pic.png Is this possible or do I have to give up on the prospect of using Solr? I have to query this data dynamically so I can't pre-summarize all of it. To clarify I having the following two problems: - Paging is ignored for stats data - I can't figure out how to divide two stats together to get a third stat. Note: In some cases I would need to be able to sort on this combined stat -- View this message in context: http://lucene.472066.n3.nabble.com/Division-with-Stats-Component-when-Grouping-in-Solr-tp4211402.html Sent from the Solr - User mailing list archive at Nabble.com. -- If you reply to this email, your message will be added to the discussion below: http://lucene.472066.n3.nabble.com/Division-with-Stats-Component-when-Grouping-in-Solr-tp4211402p4211422.html To unsubscribe from Division with Stats Component when Grouping in Solr, click here http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_codenode=4211402code=a2luZ29maHlwb2NyaXRlc0BnbWFpbC5jb218NDIxMTQwMnwtNDY4MDgyMzk1 . NAML http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewerid=instant_html%21nabble%3Aemail.namlbase=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespacebreadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml -- View this message in context: http://lucene.472066.n3.nabble.com/Division-with-Stats-Component-when-Grouping-in-Solr-tp4211402p4211525.html Sent from the Solr - User mailing list archive at Nabble.com.
How to use https://issues.apache.org/jira/browse/SOLR-7274
How do you set this up? -- Bill Bell billnb...@gmail.com cell 720-256-8076
Re: What's wrong
What does does your exact query parameter look like? The parentheses in your message make it unclear. You have a comma in your query as if you expect this has some functional purpose. Technically, it should get analyzed away, but why did you include it? Do any queries find that document, or do all other queries find it and only this one fails to find it? Are you sure that you committed the document? Does a query by id find the document? Does your field for details have indexed=TRUE? -- Jack Krupansky On Sat, Jun 13, 2015 at 5:54 AM, Test Test andymish...@yahoo.fr wrote: Hi, I have solr document, composed like this, with 2 fields : id = 1details = London is the capital and most-populous city of United Kingdom. When i request solr with this parameter (details:london, details:city), i don't get the document.The details field is a type text_general fieldType name=text_general class=solr.TextField positionIncrementGap=100analyzer type=index tokenizer class=solr.StandardTokenizerFactory/filter class=solr.StopFilterFactory ignoreCase=true words=stopwords.txt / !-- in this example, we will only use synonyms at query time filter class=solr.SynonymFilterFactory synonyms=index_synonyms.txt ignoreCase=true expand=false/ --filter class=solr.LowerCaseFilterFactory/ /analyzeranalyzer type=querytokenizer class=solr.StandardTokenizerFactory/filter class=solr.StopFilterFactory ignoreCase=true words=stopwords.txt / filter class=solr.SynonymFilterFactory synonyms=synonyms.txt ignoreCase=true expand=true/filter class=solr.LowerCaseFilterFactory//analyzer/fieldType What's wrong?
What's wrong
Hi, I have solr document, composed like this, with 2 fields : id = 1details = London is the capital and most-populous city of United Kingdom. When i request solr with this parameter (details:london, details:city), i don't get the document.The details field is a type text_general fieldType name=text_general class=solr.TextField positionIncrementGap=100 analyzer type=index tokenizer class=solr.StandardTokenizerFactory/ filter class=solr.StopFilterFactory ignoreCase=true words=stopwords.txt / !-- in this example, we will only use synonyms at query time filter class=solr.SynonymFilterFactory synonyms=index_synonyms.txt ignoreCase=true expand=false/ -- filter class=solr.LowerCaseFilterFactory/ /analyzer analyzer type=query tokenizer class=solr.StandardTokenizerFactory/ filter class=solr.StopFilterFactory ignoreCase=true words=stopwords.txt / filter class=solr.SynonymFilterFactory synonyms=synonyms.txt ignoreCase=true expand=true/ filter class=solr.LowerCaseFilterFactory/ /analyzer /fieldType What's wrong?
solr 5.2 and dovecot
Hi list,
I wanted to get SOLR 5.2 with dovecot running. My installation of solr 4.10
with dovecot works so far.
It's a very basic installation with almost no configuration, so I gave 5.2 a
try and just copied the conf directory of 4.10 to the fresh 5.2 core.
Dovecot reports that sent mails are being indexed according to logs:
Jun 13 11:08:42 mailtest dovecot: indexer-worker(user): Debug: http-client:
conn 127.0.0.1:8983 [0]: Got 200 response for request [POST
http://localhost:8983/solr/mail3/update] (took 33 ms + 0 ms in queue)
Jun 13 11:08:42 mailtest dovecot: indexer-worker(user): Debug: http-client:
conn 127.0.0.1:8983 [0]: Response payload stream destroyed (0 ms after initial
response)
Jun 13 11:08:42 mailtest dovecot: indexer-worker(user): Debug: http-client:
request [POST http://localhost:8983/solr/mail3/update]: Finished
However, when trying to search I get an error in dovecot:
Jun 13 11:11:43 mailtest dovecot: imap(user): Debug: http-client: queue
http://localhost:8983: Connection to peer 127.0.0.1:8983 claimed request [GET
http://localhost:8983/solr/mail3/select?fl=uid,scorerows=3sort=uid+ascq=subject:%22blubb%22fq=%2Bbox:b624920e60057c554e4adf211752+%2Buser:%22user%22]
Jun 13 11:11:43 mailtest dovecot: imap(user): Debug: http-client: conn
127.0.0.1:8983 [0]: Claimed request [GET
http://localhost:8983/solr/mail3/select?fl=uid,scorerows=3sort=uid+ascq=subject:%22blubb%22fq=%2Bbox:b624920e60057c554e4adf211752+%2Buser:%22user%22]
Jun 13 11:11:43 mailtest dovecot: imap(user): Debug: http-client: request [GET
http://localhost:8983/solr/mail3/select?fl=uid,scorerows=3sort=uid+ascq=subject:%22blubb%22fq=%2Bbox:b624920e60057c554e4adf211752+%2Buser:%22user%22]:
Sent header
Jun 13 11:11:43 mailtest dovecot: imap(user): Debug: http-client: conn
127.0.0.1:8983 [0]: Got 400 response for request [GET
http://localhost:8983/solr/mail3/select?fl=uid,scorerows=3sort=uid+ascq=subject:%22blubb%22fq=%2Bbox:b624920e60057c554e4adf211752+%2Buser:%22user%22]
(took 4 ms + 1 ms in queue)
Jun 13 11:11:43 mailtest dovecot: imap(user): Error: fts_solr: Lookup failed:
Bad Request
In SOLR webadmin I get the message org.apache.solr.common.SolrException: Bad
contentType for search handler :text/xml
request={sort=uid+ascfl=uid,scoreq=subject:blubbfq=%2Bbox:b624920e60057c554e4adf211752+%2Buser:lukasrows=3}
or to be precise:
org.apache.solr.common.SolrException: Bad contentType for search handler
:text/xml
request={sort=uid+ascfl=uid,scoreq=subject:blubbfq=%2Bbox:b624920e60057c554e4adf211752+%2Buser:lukasrows=3}
at
org.apache.solr.request.json.RequestUtil.processParams(RequestUtil.java:74)
at
org.apache.solr.util.SolrPluginUtils.setDefaults(SolrPluginUtils.java:167)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:140)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2064)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:640)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:436)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:497)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at
org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
Re: AngularJS
Oooh, yes. Thx! Keep them coming! Upayavira On Sat, Jun 13, 2015, at 04:52 AM, William Bell wrote: 1. With the angular index.html, when selecting a CORE, the right side of the screen does not refresh and show info for the core I selected. 2. It looks like it just needs whitespace abbr style=display:inline title=Total Rows Fetched class=ng-bindingFetched/abbr : 11,310 span style=display:inline ng-show=doc.speed class=ng-binding251/s/span On Wed, Jun 10, 2015 at 3:28 AM, Upayavira u...@odoko.co.uk wrote: On Wed, Jun 10, 2015, at 05:52 AM, William Bell wrote: Finding DIH issue with the new AngularJS DIH section, while indexing... 1,22613/s ? Last Update: 22:50:50 *Indexing since 0:1:38.204* Requests: 1, Fetched: 1,22613/s, Skipped: 0, Processed: 1,22613/s Started: 3 minutes ago Ahh, great - real feedback! :-) What does the old UI say at that point? Could you use inspect element in your browser, and paste a few nodes around this for both the old and the new UI? We can, and probably should, do this in a JIRA ticket. You willing to file one? Many thanks! Upayavira -- Bill Bell billnb...@gmail.com cell 720-256-8076
Re: What's wrong
Use the analysis tab of the admin UI to try out your sentence against the text_general analyzer. See how your sentence is analysed at index and query time. Upayavira On Sat, Jun 13, 2015, at 10:54 AM, Test Test wrote: Hi, I have solr document, composed like this, with 2 fields : id = 1details = London is the capital and most-populous city of United Kingdom. When i request solr with this parameter (details:london, details:city), i don't get the document.The details field is a type text_general fieldType name=text_general class=solr.TextField positionIncrementGap=100 analyzer type=index tokenizer class=solr.StandardTokenizerFactory/ filter class=solr.StopFilterFactory ignoreCase=true words=stopwords.txt / !-- in this example, we will only use synonyms at query time filter class=solr.SynonymFilterFactory synonyms=index_synonyms.txt ignoreCase=true expand=false/ -- filter class=solr.LowerCaseFilterFactory/ /analyzer analyzer type=query tokenizer class=solr.StandardTokenizerFactory/ filter class=solr.StopFilterFactory ignoreCase=true words=stopwords.txt / filter class=solr.SynonymFilterFactory synonyms=synonyms.txt ignoreCase=true expand=true/ filter class=solr.LowerCaseFilterFactory/ /analyzer /fieldType What's wrong?
Re: Division with Stats Component when Grouping in Solr
OK more info requestHandler name=standard class=solr.StandardRequestHandler arr name=components strquery/str strfacet/str stranalytics/str strhighlight/str strdebug/str strexpand/str /arr /requestHandler searchComponent name=analytics class=org.apache.solr.handler.component.AnalyticsComponent / I am going to try that after adding it to solrconfig.xml. On Sat, Jun 13, 2015 at 1:11 PM, William Bell billnb...@gmail.com wrote: Same here. What do we need to add to solrconfig.xml to get it to work? 1. SOLR-5302 https://issues.apache.org/jira/browse/SOLR-5302 2. 3. Help/ On Sat, Jun 13, 2015 at 8:34 AM, kingofhypocrites kingofhypocri...@gmail.com wrote: This looks very promising if only I could get it to work: https://issues.apache.org/jira/browse/SOLR-5302 https://issues.apache.org/jira/secure/attachment/12606793/Search%20Analytics%20Component.pdf Various links it points to are broken now and i can't find anything about it online, but the PDF indicates I can set olap=true to turn it on, although this doesn't seem to do anything. The docs say it supports limiting the results and doing math operations on statistics which is exactly what I need. I'm not clear if I need to install this or if this component is even used anymore. On Fri, Jun 12, 2015 at 12:00 PM Joel Bernstein [via Lucene] ml-node+s472066n4211422...@n3.nabble.com wrote: https://issues.apache.org/jira/browse/SOLR-7560, will almost support this in Solr 5.3. The compound function support won't be there yet though. But it will be there in the near future. Joel Bernstein http://joelsolr.blogspot.com/ On Fri, Jun 12, 2015 at 9:30 AM, kingofhypocrites [hidden email] http:///user/SendEmail.jtp?type=nodenode=4211422i=0 wrote: I am migrating a database from SQL Server to Cassandra. Currently I have a setup as follows: - Log data in Cassandra - Summarize data in Spark and put into Cassandra summary tables - Query data in Solr Everything fits beautifully until I need to do stats on groups. I am hoping to get this to work with Solr so I can stick to one database, but I am not sure it's possible. If I had it in SQL Server, I could do it like so: SELECT site_id, keyword, SUM(visits) as visits, CONVERT(DECIMAL(13, 3), SUM(bounces)) / SUM(visits) as bounce_rate, SUM(pageviews) as pageviews, CONVERT(DECIMAL(13, 3), SUM(pageviews)) / SUM(visits) as avg_pages_per_visit FROM report_all_keywords_daily WHERE site_id = 55 AND date_key = '20150606' AND date_key = '20150608' GROUP BY site_id, keyword ORDER BY visits DESC Now I need to replicate this in Solr. The closest I could get to this is by using the Stats component and then using field collapsing. group=truegroup.field=keywordstats=truestats.field=visitsstats.facet=keyword And here are some results I get back: http://pastebin.com/raw.php?i=Fxhe2RA0 However, I need to do able to divide certain metrics. I tried including functions in the stats.field such as div(sum(bounce_rate), (sum(visits)) but it doesn't recognize the functions. Also it seems to ignoring the paging for the stats results and returns all groups regardless. Ultimately I'd like something like this which is what I would get in SQL: http://lucene.472066.n3.nabble.com/file/n4211402/pic.png Is this possible or do I have to give up on the prospect of using Solr? I have to query this data dynamically so I can't pre-summarize all of it. To clarify I having the following two problems: - Paging is ignored for stats data - I can't figure out how to divide two stats together to get a third stat. Note: In some cases I would need to be able to sort on this combined stat -- View this message in context: http://lucene.472066.n3.nabble.com/Division-with-Stats-Component-when-Grouping-in-Solr-tp4211402.html Sent from the Solr - User mailing list archive at Nabble.com. -- If you reply to this email, your message will be added to the discussion below: http://lucene.472066.n3.nabble.com/Division-with-Stats-Component-when-Grouping-in-Solr-tp4211402p4211422.html To unsubscribe from Division with Stats Component when Grouping in Solr, click here http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_codenode=4211402code=a2luZ29maHlwb2NyaXRlc0BnbWFpbC5jb218NDIxMTQwMnwtNDY4MDgyMzk1 . NAML
Re: Division with Stats Component when Grouping in Solr
Not you need to enable docValues to get range stuff to work. docValues=true on the field. On Sat, Jun 13, 2015 at 1:37 PM, William Bell billnb...@gmail.com wrote: OK. That works with one more change. lib dir=../../../dist/ regex=solr-analytics-.*\.jar / lib dir=../../../dist/ regex=solr-analysis-.*\.jar / http://localhost:8983/solr/select?q=*%3A*wt=jsonindent=truestats=trueolap=trueolap.overall_score.statistic.sum=sum(overall_score) On Sat, Jun 13, 2015 at 1:16 PM, William Bell billnb...@gmail.com wrote: OK more info requestHandler name=standard class=solr.StandardRequestHandler arr name=components strquery/str strfacet/str stranalytics/str strhighlight/str strdebug/str strexpand/str /arr /requestHandler searchComponent name=analytics class=org.apache.solr.handler.component.AnalyticsComponent / I am going to try that after adding it to solrconfig.xml. On Sat, Jun 13, 2015 at 1:11 PM, William Bell billnb...@gmail.com wrote: Same here. What do we need to add to solrconfig.xml to get it to work? 1. SOLR-5302 https://issues.apache.org/jira/browse/SOLR-5302 2. 3. Help/ On Sat, Jun 13, 2015 at 8:34 AM, kingofhypocrites kingofhypocri...@gmail.com wrote: This looks very promising if only I could get it to work: https://issues.apache.org/jira/browse/SOLR-5302 https://issues.apache.org/jira/secure/attachment/12606793/Search%20Analytics%20Component.pdf Various links it points to are broken now and i can't find anything about it online, but the PDF indicates I can set olap=true to turn it on, although this doesn't seem to do anything. The docs say it supports limiting the results and doing math operations on statistics which is exactly what I need. I'm not clear if I need to install this or if this component is even used anymore. On Fri, Jun 12, 2015 at 12:00 PM Joel Bernstein [via Lucene] ml-node+s472066n4211422...@n3.nabble.com wrote: https://issues.apache.org/jira/browse/SOLR-7560, will almost support this in Solr 5.3. The compound function support won't be there yet though. But it will be there in the near future. Joel Bernstein http://joelsolr.blogspot.com/ On Fri, Jun 12, 2015 at 9:30 AM, kingofhypocrites [hidden email] http:// /user/SendEmail.jtp?type=nodenode=4211422i=0 wrote: I am migrating a database from SQL Server to Cassandra. Currently I have a setup as follows: - Log data in Cassandra - Summarize data in Spark and put into Cassandra summary tables - Query data in Solr Everything fits beautifully until I need to do stats on groups. I am hoping to get this to work with Solr so I can stick to one database, but I am not sure it's possible. If I had it in SQL Server, I could do it like so: SELECT site_id, keyword, SUM(visits) as visits, CONVERT(DECIMAL(13, 3), SUM(bounces)) / SUM(visits) as bounce_rate, SUM(pageviews) as pageviews, CONVERT(DECIMAL(13, 3), SUM(pageviews)) / SUM(visits) as avg_pages_per_visit FROM report_all_keywords_daily WHERE site_id = 55 AND date_key = '20150606' AND date_key = '20150608' GROUP BY site_id, keyword ORDER BY visits DESC Now I need to replicate this in Solr. The closest I could get to this is by using the Stats component and then using field collapsing. group=truegroup.field=keywordstats=truestats.field=visitsstats.facet=keyword And here are some results I get back: http://pastebin.com/raw.php?i=Fxhe2RA0 However, I need to do able to divide certain metrics. I tried including functions in the stats.field such as div(sum(bounce_rate), (sum(visits)) but it doesn't recognize the functions. Also it seems to ignoring the paging for the stats results and returns all groups regardless. Ultimately I'd like something like this which is what I would get in SQL: http://lucene.472066.n3.nabble.com/file/n4211402/pic.png Is this possible or do I have to give up on the prospect of using Solr? I have to query this data dynamically so I can't pre-summarize all of it. To clarify I having the following two problems: - Paging is ignored for stats data - I can't figure out how to divide two stats together to get a third stat. Note: In some cases I would need to be able to sort on this combined stat -- View this message in context: http://lucene.472066.n3.nabble.com/Division-with-Stats-Component-when-Grouping-in-Solr-tp4211402.html Sent from the Solr - User mailing list archive at Nabble.com. -- If you reply to this email, your message will be added to the discussion below: http://lucene.472066.n3.nabble.com/Division-with-Stats-Component-when-Grouping-in-Solr-tp4211402p4211422.html To unsubscribe from Division with Stats Component when Grouping in
Re: Division with Stats Component when Grouping in Solr
OK. Kinda like pivoting stats...
http://localhost:8983/solr/select?q=*%3A*wt=jsonindent=trueolap=trueolap.req1.fieldfacet=overall_scorefacet=truefacet.field=overall_scoreolap.req1.statistic.count=count(overall_score)
Basically this does the same think in olap and facet.
- response:
{
- numFound: 63061,
- start: 0,
- docs:
[]
},
- facet_counts:
{
- facet_queries: { },
- facet_fields:
{
- overall_score:
[
- 1,
- 40138,
- 5,
- 17487,
- 2,
- 2299,
- 4,
- 1810,
- 3,
- 1314
]
},
- facet_dates: { },
- facet_ranges: { },
- facet_intervals: { },
- facet_heatmaps: { }
},
- stats:
[
- req1,
-
[
- count,
- 63048,
- fieldFacets,
-
[
- overall_score,
-
[
- 1,
-
[
- count,
- 40138
],
- 2,
-
[
- count,
- 2299
],
- 3,
-
[
- count,
- 1314
],
- 4,
-
[
- count,
- 1810
],
- 5,
-
[
- count,
- 17487
]
]
],
- rangeFacets,
- [ ],
- queryFacets,
- [ ]
]
]
}
On Sat, Jun 13, 2015 at 2:06 PM, William Bell billnb...@gmail.com wrote:
Having a hard time getting this to work:
http://localhost:8983/solr/select?q=*%3A*wt=jsonindent=trueolap=trueolap.req1.fieldfacet=overall_score
and even tried... I made sure docValues was set for overall_score too.
http://hgsolr2devmstr:8983/solr/survey/select?q=*%3A*wt=jsonindent=trueolap=trueolap.fieldfacet=overall_score
field name=overall_score type=int indexed=true stored=true
docValues=true /
On Sat, Jun 13, 2015 at 2:02 PM, William Bell billnb...@gmail.com wrote:
Not you need to enable docValues to get range stuff to work.
docValues=true on the field.
On Sat, Jun 13, 2015 at 1:37 PM, William Bell billnb...@gmail.com
wrote:
OK. That works with one more change.
lib dir=../../../dist/ regex=solr-analytics-.*\.jar /
lib dir=../../../dist/ regex=solr-analysis-.*\.jar /
http://localhost:8983/solr/select?q=*%3A*wt=jsonindent=truestats=trueolap=trueolap.overall_score.statistic.sum=sum(overall_score)
On Sat, Jun 13, 2015 at 1:16 PM, William Bell billnb...@gmail.com
wrote:
OK more info
requestHandler name=standard class=solr.StandardRequestHandler
arr name=components
strquery/str
strfacet/str
stranalytics/str
strhighlight/str
strdebug/str
strexpand/str
/arr
/requestHandler
searchComponent name=analytics
class=org.apache.solr.handler.component.AnalyticsComponent /
I am going to try that after adding it to solrconfig.xml.
On Sat, Jun 13, 2015 at 1:11 PM, William Bell billnb...@gmail.com
wrote:
Same here.
What do we need to add to solrconfig.xml to get it to work?
1. SOLR-5302 https://issues.apache.org/jira/browse/SOLR-5302
2.
3. Help/
On Sat, Jun 13, 2015 at 8:34 AM, kingofhypocrites
kingofhypocri...@gmail.com wrote:
This looks very promising if only I could get it to work:
https://issues.apache.org/jira/browse/SOLR-5302
https://issues.apache.org/jira/secure/attachment/12606793/Search%20Analytics%20Component.pdf
Various links it points to are broken now and i can't find anything
about
it online, but the PDF indicates I can set olap=true to turn it on,
although this doesn't seem to do anything. The docs say it supports
limiting the results and doing math operations on statistics which is
exactly what I need. I'm not clear if I need to install this or if
this
component is even used anymore.
On Fri, Jun 12, 2015 at 12:00 PM Joel Bernstein [via Lucene]
ml-node+s472066n4211422...@n3.nabble.com wrote:
https://issues.apache.org/jira/browse/SOLR-7560, will almost
support this
in Solr 5.3. The compound function support won't be there yet
though. But
it will be there in the near future.
Joel Bernstein
http://joelsolr.blogspot.com/
On Fri, Jun 12, 2015 at 9:30 AM, kingofhypocrites
[hidden email] http://
/user/SendEmail.jtp?type=nodenode=4211422i=0
wrote:
I am migrating a database from SQL Server to Cassandra. Currently
I have
a
setup as follows:
- Log data in Cassandra
- Summarize data in Spark and put into Cassandra summary tables
- Query data in Solr
Everything fits beautifully until I need to do stats
heatmaps
How do you use the heatmaps feature to show the PNG or use the int2D to
show the heatmap easily just for testing?
http://localhost:8983/solr/select?q=*%3A*wt=jsonrows=0indent=truefacet=truefacet.heatmap=store_geohashfacet.heatmap.format=png
{
- responseHeader:
{
- status: 0,
- QTime: 163,
- params:
{
- facet.heatmap.format: png,
- q: *:*,
- facet.heatmap: store_geohash,
- indent: true,
- rows: 0,
- wt: json,
- facet: true
}
},
- response:
{
- numFound: 2664396,
- start: 0,
- docs: [ ]
},
- facet_counts:
{
- facet_queries: { },
- facet_fields: { },
- facet_dates: { },
- facet_ranges: { },
- facet_intervals: { },
- facet_heatmaps:
{
- store_geohash:
[
- gridLevel,
- 2,
- columns,
- 32,
- rows,
- 32,
- minX,
- -180,
- maxX,
- 180,
- minY,
- -90,
- maxY,
- 90,
- counts_png,
-
iVBORw0KGgoNSUhEUgAAACAgCAYAAABzenr00UlEQVR42mNgYGD4P8B41AEwBuNAOoD7PwMbP5BmGygHyP9nENf5z8C45D8Dk+ZAp4FNA+wA5pn/GZfz/GeQzv3PkL3hP4Pw+f8MLGuBctl0cgCv1n+Gton/GSUs/jPGL/jP+uvCf467IAdMoFcUyPxnaS78zzCN6T+TKMt/5g+T/jNJraZzOXBc6T/D9bv/mfp//2f8GDqABZFCzX/GQh86OED1338Grm34FVrMoEcITKdDtmsZrYxGHTDqAFIdwDgaBaMOGHXAqANGHTC8HQAAQf8YdbR5KkIASUVORK5CYII=
]
}
}
}
--
Bill Bell
billnb...@gmail.com
cell 720-256-8076
Re: Division with Stats Component when Grouping in Solr
Same here. What do we need to add to solrconfig.xml to get it to work? 1. SOLR-5302 https://issues.apache.org/jira/browse/SOLR-5302 2. 3. Help/ On Sat, Jun 13, 2015 at 8:34 AM, kingofhypocrites kingofhypocri...@gmail.com wrote: This looks very promising if only I could get it to work: https://issues.apache.org/jira/browse/SOLR-5302 https://issues.apache.org/jira/secure/attachment/12606793/Search%20Analytics%20Component.pdf Various links it points to are broken now and i can't find anything about it online, but the PDF indicates I can set olap=true to turn it on, although this doesn't seem to do anything. The docs say it supports limiting the results and doing math operations on statistics which is exactly what I need. I'm not clear if I need to install this or if this component is even used anymore. On Fri, Jun 12, 2015 at 12:00 PM Joel Bernstein [via Lucene] ml-node+s472066n4211422...@n3.nabble.com wrote: https://issues.apache.org/jira/browse/SOLR-7560, will almost support this in Solr 5.3. The compound function support won't be there yet though. But it will be there in the near future. Joel Bernstein http://joelsolr.blogspot.com/ On Fri, Jun 12, 2015 at 9:30 AM, kingofhypocrites [hidden email] http:///user/SendEmail.jtp?type=nodenode=4211422i=0 wrote: I am migrating a database from SQL Server to Cassandra. Currently I have a setup as follows: - Log data in Cassandra - Summarize data in Spark and put into Cassandra summary tables - Query data in Solr Everything fits beautifully until I need to do stats on groups. I am hoping to get this to work with Solr so I can stick to one database, but I am not sure it's possible. If I had it in SQL Server, I could do it like so: SELECT site_id, keyword, SUM(visits) as visits, CONVERT(DECIMAL(13, 3), SUM(bounces)) / SUM(visits) as bounce_rate, SUM(pageviews) as pageviews, CONVERT(DECIMAL(13, 3), SUM(pageviews)) / SUM(visits) as avg_pages_per_visit FROM report_all_keywords_daily WHERE site_id = 55 AND date_key = '20150606' AND date_key = '20150608' GROUP BY site_id, keyword ORDER BY visits DESC Now I need to replicate this in Solr. The closest I could get to this is by using the Stats component and then using field collapsing. group=truegroup.field=keywordstats=truestats.field=visitsstats.facet=keyword And here are some results I get back: http://pastebin.com/raw.php?i=Fxhe2RA0 However, I need to do able to divide certain metrics. I tried including functions in the stats.field such as div(sum(bounce_rate), (sum(visits)) but it doesn't recognize the functions. Also it seems to ignoring the paging for the stats results and returns all groups regardless. Ultimately I'd like something like this which is what I would get in SQL: http://lucene.472066.n3.nabble.com/file/n4211402/pic.png Is this possible or do I have to give up on the prospect of using Solr? I have to query this data dynamically so I can't pre-summarize all of it. To clarify I having the following two problems: - Paging is ignored for stats data - I can't figure out how to divide two stats together to get a third stat. Note: In some cases I would need to be able to sort on this combined stat -- View this message in context: http://lucene.472066.n3.nabble.com/Division-with-Stats-Component-when-Grouping-in-Solr-tp4211402.html Sent from the Solr - User mailing list archive at Nabble.com. -- If you reply to this email, your message will be added to the discussion below: http://lucene.472066.n3.nabble.com/Division-with-Stats-Component-when-Grouping-in-Solr-tp4211402p4211422.html To unsubscribe from Division with Stats Component when Grouping in Solr, click here http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_codenode=4211402code=a2luZ29maHlwb2NyaXRlc0BnbWFpbC5jb218NDIxMTQwMnwtNDY4MDgyMzk1 . NAML http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewerid=instant_html%21nabble%3Aemail.namlbase=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespacebreadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml -- View this message in context: http://lucene.472066.n3.nabble.com/Division-with-Stats-Component-when-Grouping-in-Solr-tp4211402p4211525.html Sent from the Solr - User mailing list archive at Nabble.com. -- Bill Bell billnb...@gmail.com cell 720-256-8076
Re: Division with Stats Component when Grouping in Solr
OK. That works with one more change. lib dir=../../../dist/ regex=solr-analytics-.*\.jar / lib dir=../../../dist/ regex=solr-analysis-.*\.jar / http://localhost:8983/solr/select?q=*%3A*wt=jsonindent=truestats=trueolap=trueolap.overall_score.statistic.sum=sum(overall_score) On Sat, Jun 13, 2015 at 1:16 PM, William Bell billnb...@gmail.com wrote: OK more info requestHandler name=standard class=solr.StandardRequestHandler arr name=components strquery/str strfacet/str stranalytics/str strhighlight/str strdebug/str strexpand/str /arr /requestHandler searchComponent name=analytics class=org.apache.solr.handler.component.AnalyticsComponent / I am going to try that after adding it to solrconfig.xml. On Sat, Jun 13, 2015 at 1:11 PM, William Bell billnb...@gmail.com wrote: Same here. What do we need to add to solrconfig.xml to get it to work? 1. SOLR-5302 https://issues.apache.org/jira/browse/SOLR-5302 2. 3. Help/ On Sat, Jun 13, 2015 at 8:34 AM, kingofhypocrites kingofhypocri...@gmail.com wrote: This looks very promising if only I could get it to work: https://issues.apache.org/jira/browse/SOLR-5302 https://issues.apache.org/jira/secure/attachment/12606793/Search%20Analytics%20Component.pdf Various links it points to are broken now and i can't find anything about it online, but the PDF indicates I can set olap=true to turn it on, although this doesn't seem to do anything. The docs say it supports limiting the results and doing math operations on statistics which is exactly what I need. I'm not clear if I need to install this or if this component is even used anymore. On Fri, Jun 12, 2015 at 12:00 PM Joel Bernstein [via Lucene] ml-node+s472066n4211422...@n3.nabble.com wrote: https://issues.apache.org/jira/browse/SOLR-7560, will almost support this in Solr 5.3. The compound function support won't be there yet though. But it will be there in the near future. Joel Bernstein http://joelsolr.blogspot.com/ On Fri, Jun 12, 2015 at 9:30 AM, kingofhypocrites [hidden email] http:// /user/SendEmail.jtp?type=nodenode=4211422i=0 wrote: I am migrating a database from SQL Server to Cassandra. Currently I have a setup as follows: - Log data in Cassandra - Summarize data in Spark and put into Cassandra summary tables - Query data in Solr Everything fits beautifully until I need to do stats on groups. I am hoping to get this to work with Solr so I can stick to one database, but I am not sure it's possible. If I had it in SQL Server, I could do it like so: SELECT site_id, keyword, SUM(visits) as visits, CONVERT(DECIMAL(13, 3), SUM(bounces)) / SUM(visits) as bounce_rate, SUM(pageviews) as pageviews, CONVERT(DECIMAL(13, 3), SUM(pageviews)) / SUM(visits) as avg_pages_per_visit FROM report_all_keywords_daily WHERE site_id = 55 AND date_key = '20150606' AND date_key = '20150608' GROUP BY site_id, keyword ORDER BY visits DESC Now I need to replicate this in Solr. The closest I could get to this is by using the Stats component and then using field collapsing. group=truegroup.field=keywordstats=truestats.field=visitsstats.facet=keyword And here are some results I get back: http://pastebin.com/raw.php?i=Fxhe2RA0 However, I need to do able to divide certain metrics. I tried including functions in the stats.field such as div(sum(bounce_rate), (sum(visits)) but it doesn't recognize the functions. Also it seems to ignoring the paging for the stats results and returns all groups regardless. Ultimately I'd like something like this which is what I would get in SQL: http://lucene.472066.n3.nabble.com/file/n4211402/pic.png Is this possible or do I have to give up on the prospect of using Solr? I have to query this data dynamically so I can't pre-summarize all of it. To clarify I having the following two problems: - Paging is ignored for stats data - I can't figure out how to divide two stats together to get a third stat. Note: In some cases I would need to be able to sort on this combined stat -- View this message in context: http://lucene.472066.n3.nabble.com/Division-with-Stats-Component-when-Grouping-in-Solr-tp4211402.html Sent from the Solr - User mailing list archive at Nabble.com. -- If you reply to this email, your message will be added to the discussion below: http://lucene.472066.n3.nabble.com/Division-with-Stats-Component-when-Grouping-in-Solr-tp4211402p4211422.html To unsubscribe from Division with Stats Component when Grouping in Solr, click here
Re: Division with Stats Component when Grouping in Solr
Having a hard time getting this to work: http://localhost:8983/solr/select?q=*%3A*wt=jsonindent=trueolap=trueolap.req1.fieldfacet=overall_score and even tried... I made sure docValues was set for overall_score too. http://hgsolr2devmstr:8983/solr/survey/select?q=*%3A*wt=jsonindent=trueolap=trueolap.fieldfacet=overall_score field name=overall_score type=int indexed=true stored=true docValues=true / On Sat, Jun 13, 2015 at 2:02 PM, William Bell billnb...@gmail.com wrote: Not you need to enable docValues to get range stuff to work. docValues=true on the field. On Sat, Jun 13, 2015 at 1:37 PM, William Bell billnb...@gmail.com wrote: OK. That works with one more change. lib dir=../../../dist/ regex=solr-analytics-.*\.jar / lib dir=../../../dist/ regex=solr-analysis-.*\.jar / http://localhost:8983/solr/select?q=*%3A*wt=jsonindent=truestats=trueolap=trueolap.overall_score.statistic.sum=sum(overall_score) On Sat, Jun 13, 2015 at 1:16 PM, William Bell billnb...@gmail.com wrote: OK more info requestHandler name=standard class=solr.StandardRequestHandler arr name=components strquery/str strfacet/str stranalytics/str strhighlight/str strdebug/str strexpand/str /arr /requestHandler searchComponent name=analytics class=org.apache.solr.handler.component.AnalyticsComponent / I am going to try that after adding it to solrconfig.xml. On Sat, Jun 13, 2015 at 1:11 PM, William Bell billnb...@gmail.com wrote: Same here. What do we need to add to solrconfig.xml to get it to work? 1. SOLR-5302 https://issues.apache.org/jira/browse/SOLR-5302 2. 3. Help/ On Sat, Jun 13, 2015 at 8:34 AM, kingofhypocrites kingofhypocri...@gmail.com wrote: This looks very promising if only I could get it to work: https://issues.apache.org/jira/browse/SOLR-5302 https://issues.apache.org/jira/secure/attachment/12606793/Search%20Analytics%20Component.pdf Various links it points to are broken now and i can't find anything about it online, but the PDF indicates I can set olap=true to turn it on, although this doesn't seem to do anything. The docs say it supports limiting the results and doing math operations on statistics which is exactly what I need. I'm not clear if I need to install this or if this component is even used anymore. On Fri, Jun 12, 2015 at 12:00 PM Joel Bernstein [via Lucene] ml-node+s472066n4211422...@n3.nabble.com wrote: https://issues.apache.org/jira/browse/SOLR-7560, will almost support this in Solr 5.3. The compound function support won't be there yet though. But it will be there in the near future. Joel Bernstein http://joelsolr.blogspot.com/ On Fri, Jun 12, 2015 at 9:30 AM, kingofhypocrites [hidden email] http:// /user/SendEmail.jtp?type=nodenode=4211422i=0 wrote: I am migrating a database from SQL Server to Cassandra. Currently I have a setup as follows: - Log data in Cassandra - Summarize data in Spark and put into Cassandra summary tables - Query data in Solr Everything fits beautifully until I need to do stats on groups. I am hoping to get this to work with Solr so I can stick to one database, but I am not sure it's possible. If I had it in SQL Server, I could do it like so: SELECT site_id, keyword, SUM(visits) as visits, CONVERT(DECIMAL(13, 3), SUM(bounces)) / SUM(visits) as bounce_rate, SUM(pageviews) as pageviews, CONVERT(DECIMAL(13, 3), SUM(pageviews)) / SUM(visits) as avg_pages_per_visit FROM report_all_keywords_daily WHERE site_id = 55 AND date_key = '20150606' AND date_key = '20150608' GROUP BY site_id, keyword ORDER BY visits DESC Now I need to replicate this in Solr. The closest I could get to this is by using the Stats component and then using field collapsing. group=truegroup.field=keywordstats=truestats.field=visitsstats.facet=keyword And here are some results I get back: http://pastebin.com/raw.php?i=Fxhe2RA0 However, I need to do able to divide certain metrics. I tried including functions in the stats.field such as div(sum(bounce_rate), (sum(visits)) but it doesn't recognize the functions. Also it seems to ignoring the paging for the stats results and returns all groups regardless. Ultimately I'd like something like this which is what I would get in SQL: http://lucene.472066.n3.nabble.com/file/n4211402/pic.png Is this possible or do I have to give up on the prospect of using Solr? I have to query this data dynamically so I can't pre-summarize all of it. To clarify I having the following two problems: - Paging is ignored for stats data - I can't figure out how to divide two stats together to get a third stat. Note: In some cases I would need to be able to sort on this combined stat -- View this message in
Re: heatmaps
OK. I was able to get a PNG - but it looks tiny and a blob. Not sure how to
use it.
echo iVBORw0KGgoNSUhEUgAAACAgCAYAAABzenr00UlEQVR42mNg
YGD4P8B41AEwBuNAOoD7PwMbP5BmGygHyP9nENf5z8C45D8Dk+ZAp4FNA+
wA5pn/GZfz/GeQzv3PkL3hP4Pw+f8MLGuBctl0cgCv1n+Gton/GSUs/jPGL/jP+
uvCf467IAdMoFcUyPxnaS78zzCN6T+TKMt/5g+T/jNJraZzOXBc6T/D9bv/
mfp//2f8GDqABZFCzX/GQh86OED1338Grm34FVrMoEcITKdDt
msZrYxGHTDqAFIdwDgaBaMOGHXAqANGHTC8HQAAQf8YdbR5KkIASUVORK5CYII= |
base64 --decode test.png
Very strange.
Maybe I need to zoom in?
On Sat, Jun 13, 2015 at 2:18 PM, William Bell billnb...@gmail.com wrote:
How do you use the heatmaps feature to show the PNG or use the int2D to
show the heatmap easily just for testing?
http://localhost:8983/solr/select?q=*%3A*wt=jsonrows=0indent=truefacet=truefacet.heatmap=store_geohashfacet.heatmap.format=png
{
- responseHeader:
{
- status: 0,
- QTime: 163,
- params:
{
- facet.heatmap.format: png,
- q: *:*,
- facet.heatmap: store_geohash,
- indent: true,
- rows: 0,
- wt: json,
- facet: true
}
},
- response:
{
- numFound: 2664396,
- start: 0,
- docs: [ ]
},
- facet_counts:
{
- facet_queries: { },
- facet_fields: { },
- facet_dates: { },
- facet_ranges: { },
- facet_intervals: { },
- facet_heatmaps:
{
- store_geohash:
[
- gridLevel,
- 2,
- columns,
- 32,
- rows,
- 32,
- minX,
- -180,
- maxX,
- 180,
- minY,
- -90,
- maxY,
- 90,
- counts_png,
-
iVBORw0KGgoNSUhEUgAAACAgCAYAAABzenr00UlEQVR42mNgYGD4P8B41AEwBuNAOoD7PwMbP5BmGygHyP9nENf5z8C45D8Dk+ZAp4FNA+wA5pn/GZfz/GeQzv3PkL3hP4Pw+f8MLGuBctl0cgCv1n+Gton/GSUs/jPGL/jP+uvCf467IAdMoFcUyPxnaS78zzCN6T+TKMt/5g+T/jNJraZzOXBc6T/D9bv/mfp//2f8GDqABZFCzX/GQh86OED1338Grm34FVrMoEcITKdDtmsZrYxGHTDqAFIdwDgaBaMOGHXAqANGHTC8HQAAQf8YdbR5KkIASUVORK5CYII=
]
}
}
}
--
Bill Bell
billnb...@gmail.com
cell 720-256-8076
--
Bill Bell
billnb...@gmail.com
cell 720-256-8076
Re: Division with Stats Component when Grouping in Solr
On Fri, Jun 12, 2015 at 10:30 AM, kingofhypocrites
kingofhypocri...@gmail.com wrote:
I am migrating a database from SQL Server to Cassandra. Currently I have a
setup as follows:
- Log data in Cassandra
- Summarize data in Spark and put into Cassandra summary tables
- Query data in Solr
Everything fits beautifully until I need to do stats on groups. I am hoping
to get this to work with Solr so I can stick to one database, but I am not
sure it's possible.
If I had it in SQL Server, I could do it like so:
SELECT
site_id,
keyword,
SUM(visits) as visits,
CONVERT(DECIMAL(13, 3), SUM(bounces)) / SUM(visits) as bounce_rate,
SUM(pageviews) as pageviews,
CONVERT(DECIMAL(13, 3), SUM(pageviews)) / SUM(visits) as
avg_pages_per_visit
FROM
report_all_keywords_daily
WHERE
site_id = 55 AND date_key = '20150606' AND date_key = '20150608'
GROUP BY
site_id, keyword
ORDER BY visits DESC
This is the closest we can get with the JSON Facet API today:
json.facet={
sites: {
type : terms,
field : site_id,
sort : visits desc,
facet : {
visits : sum(visits),
bounces : sum(bounces),
pageviews : sum(pageviews)
}
}
}
That doesn't take into account keyword when sorting the buckets.
You could nest a ketword facet inside a site facet and thus calculate
the stats for the top N keywords per site:
json.facet={
sites: {
type : terms,
field : site_id,
facet : {
keywords: {
type : terms,
field : keyword,
sort : visits desc,
facet : {
visits : sum(visits),
bounces : sum(bounces),
pageviews : sum(pageviews)
}
}
}
}
More info here: http://yonik.com/json-facet-api/
-Yonik
Re: Division with Stats Component when Grouping in Solr
It would be cool to be able to set 2 group by with facets GROUP BY site_id, keyword Bill Bell Sent from mobile On Jun 13, 2015, at 2:28 PM, Yonik Seeley ysee...@gmail.com wrote: GROUP BY site_id, keyword
