[ https://issues.apache.org/jira/browse/SOLR-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13011264#comment-13011264 ]
Yonik Seeley commented on SOLR-1566: ------------------------------------ I've tried to go over my brainstorming notes and pull out some stuff that might make sense (as opposed to the stuff that no one will understand w/o a lot more context, like why approach X or approach Y won't work). * Know when not to load any stored fields at all (fields could all come from pseudofields, including external fields) ** example: fl={!func key=id}id // use the fieldcache to load an id (important option for distributed search) ** example: fl={!func key=id},mul(popularity,editorial_score)&rows=1000000000 // stream the entire thing * General transformer class: something that adds, changes, or removes fields based on an external/database lookup ** may need to work in batches of external ids (a getNextDocument() iterator pattern could allow buffering and then an external request with a bunch of ids) ** probably wants to work in batches of SolrDocument, not docids ** may need to inspect the FieldsSpec to see what fields were requested *** ask: did the request ask for field "x" that I am responsible for? *** ask: did the request *specifically* ask for field "x" * function query that needs to work in-order for good efficiency (query/scorer is one example) ** need to work in potentially big blocks (100 docids at a time), sort the ids then retrieve values in order ** don't want to instantiate all SolrDocuments at once... just keep a float[100] or Object[100] ** if multiple function queries, don't make each sort it's own list of docids, share somehow? *** the sharing mechanism could just be a private implementation to this type of transformer * transformers that need the Index... don't make each one look up the right AtomicReaderContext, share it somehow * external fields should work with highlighting (eventually) ** needs to be possible for highlighting to work on SolrDocuments and only run the transformers once... replacing the original DocList with something else * different DocLists need different transformer lists ** example: multiple DocLists as the result of grouping ** example: allow overriding "fl" per grouping command * need more than one level of pseudo-fields... ** example: user may specify fl=add(a,b)... that's "global" unless overridden but the grouping code may want to add the _group_ pseudo-field when using the flattened/simple format > Allow components to add fields to outgoing documents > ---------------------------------------------------- > > Key: SOLR-1566 > URL: https://issues.apache.org/jira/browse/SOLR-1566 > Project: Solr > Issue Type: New Feature > Components: search > Reporter: Noble Paul > Assignee: Grant Ingersoll > Fix For: Next > > Attachments: SOLR-1566-DocTransformer.patch, > SOLR-1566-DocTransformer.patch, SOLR-1566-DocTransformer.patch, > SOLR-1566-DocTransformer.patch, SOLR-1566-DocTransformer.patch, > SOLR-1566-DocTransformer.patch, SOLR-1566-gsi.patch, SOLR-1566-rm.patch, > SOLR-1566-rm.patch, SOLR-1566-rm.patch, SOLR-1566-rm.patch, > SOLR-1566-rm.patch, SOLR-1566.patch, SOLR-1566.patch, SOLR-1566.patch, > SOLR-1566.patch, SOLR-1566_parsing.patch > > > Currently it is not possible for components to add fields to outgoing > documents which are not in the the stored fields of the document. This makes > it cumbersome to add computed fields/metadata . -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org