[jira] [Updated] (LUCENE-5674) A new token filter: SubSequence
[ https://issues.apache.org/jira/browse/LUCENE-5674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nitzan Shaked updated LUCENE-5674: -- Attachment: 0001-SubSeqFilter.patch > A new token filter: SubSequence > --- > > Key: LUCENE-5674 > URL: https://issues.apache.org/jira/browse/LUCENE-5674 > Project: Lucene - Core > Issue Type: Improvement > Components: core/other >Reporter: Nitzan Shaked >Priority: Minor > Attachments: 0001-SubSeqFilter.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > A new configurable token filter which, given a token breaks it into sub-parts > and outputs consecutive sub-sequences of those sub-parts. > Useful for, for example, using during indexing to generate variations on > domain names, so that "www.google.com" can be found by searching for > "google.com", or "www.google.com". > Parameters: > sepRegexp: A regular expression used split incoming tokens into sub-parts. > glue: A string used to concatenate sub-parts together when creating > sub-sequences. > minLen: Minimum length (in sub-parts) of output sub-sequences > maxLen: Maximum length (in sub-parts) of output sub-sequences (0 for > unlimited; negative numbers for token length in sub-parts minus specified > length) > anchor: Anchor.START to output only prefixes, or Anchor.END to output only > suffixes, or Anchor.NONE to output any sub-sequence > withOriginal: whether to output also the original token > Note: tests will follow, I currently believe BaseTokenStreamTestCase is > broken/mis-designed for such use cases, I will open a separate Jira for that. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5674) A new token filter: SubSequence
[ https://issues.apache.org/jira/browse/LUCENE-5674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nitzan Shaked updated LUCENE-5674: -- Attachment: subseqfilter.patch Updated patch, including tests > A new token filter: SubSequence > --- > > Key: LUCENE-5674 > URL: https://issues.apache.org/jira/browse/LUCENE-5674 > Project: Lucene - Core > Issue Type: Improvement > Components: core/other >Reporter: Nitzan Shaked >Priority: Minor > Attachments: subseqfilter.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > A new configurable token filter which, given a token breaks it into sub-parts > and outputs consecutive sub-sequences of those sub-parts. > Useful for, for example, using during indexing to generate variations on > domain names, so that "www.google.com" can be found by searching for > "google.com", or "www.google.com". > Parameters: > sepRegexp: A regular expression used split incoming tokens into sub-parts. > glue: A string used to concatenate sub-parts together when creating > sub-sequences. > minLen: Minimum length (in sub-parts) of output sub-sequences > maxLen: Maximum length (in sub-parts) of output sub-sequences (0 for > unlimited; negative numbers for token length in sub-parts minus specified > length) > anchor: Anchor.START to output only prefixes, or Anchor.END to output only > suffixes, or Anchor.NONE to output any sub-sequence > withOriginal: whether to output also the original token > Note: tests will follow, I currently believe BaseTokenStreamTestCase is > broken/mis-designed for such use cases, I will open a separate Jira for that. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5674) A new token filter: SubSequence
[ https://issues.apache.org/jira/browse/LUCENE-5674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nitzan Shaked updated LUCENE-5674: -- Description: A new configurable token filter which, given a token breaks it into sub-parts and outputs consecutive sub-sequences of those sub-parts. Useful for, for example, using during indexing to generate variations on domain names, so that "www.google.com" can be found by searching for "google.com", or "www.google.com". Parameters: sepRegexp: A regular expression used split incoming tokens into sub-parts. glue: A string used to concatenate sub-parts together when creating sub-sequences. minLen: Minimum length (in sub-parts) of output sub-sequences maxLen: Maximum length (in sub-parts) of output sub-sequences (0 for unlimited; negative numbers for token length in sub-parts minus specified length) anchor: Anchor.START to output only prefixes, or Anchor.END to output only suffixes, or Anchor.NONE to output any sub-sequence withOriginal: whether to output also the original token EDIT: now includes tests for filter and for factory. was: A new configurable token filter which, given a token breaks it into sub-parts and outputs consecutive sub-sequences of those sub-parts. Useful for, for example, using during indexing to generate variations on domain names, so that "www.google.com" can be found by searching for "google.com", or "www.google.com". Parameters: sepRegexp: A regular expression used split incoming tokens into sub-parts. glue: A string used to concatenate sub-parts together when creating sub-sequences. minLen: Minimum length (in sub-parts) of output sub-sequences maxLen: Maximum length (in sub-parts) of output sub-sequences (0 for unlimited; negative numbers for token length in sub-parts minus specified length) anchor: Anchor.START to output only prefixes, or Anchor.END to output only suffixes, or Anchor.NONE to output any sub-sequence withOriginal: whether to output also the original token Note: tests will follow, I currently believe BaseTokenStreamTestCase is broken/mis-designed for such use cases, I will open a separate Jira for that. > A new token filter: SubSequence > --- > > Key: LUCENE-5674 > URL: https://issues.apache.org/jira/browse/LUCENE-5674 > Project: Lucene - Core > Issue Type: Improvement > Components: core/other >Reporter: Nitzan Shaked >Priority: Minor > Attachments: subseqfilter.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > A new configurable token filter which, given a token breaks it into sub-parts > and outputs consecutive sub-sequences of those sub-parts. > Useful for, for example, using during indexing to generate variations on > domain names, so that "www.google.com" can be found by searching for > "google.com", or "www.google.com". > Parameters: > sepRegexp: A regular expression used split incoming tokens into sub-parts. > glue: A string used to concatenate sub-parts together when creating > sub-sequences. > minLen: Minimum length (in sub-parts) of output sub-sequences > maxLen: Maximum length (in sub-parts) of output sub-sequences (0 for > unlimited; negative numbers for token length in sub-parts minus specified > length) > anchor: Anchor.START to output only prefixes, or Anchor.END to output only > suffixes, or Anchor.NONE to output any sub-sequence > withOriginal: whether to output also the original token > EDIT: now includes tests for filter and for factory. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5674) A new token filter: SubSequence
[ https://issues.apache.org/jira/browse/LUCENE-5674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nitzan Shaked updated LUCENE-5674: -- Attachment: (was: 0001-SubSeqFilter.patch) > A new token filter: SubSequence > --- > > Key: LUCENE-5674 > URL: https://issues.apache.org/jira/browse/LUCENE-5674 > Project: Lucene - Core > Issue Type: Improvement > Components: core/other >Reporter: Nitzan Shaked >Priority: Minor > Original Estimate: 24h > Remaining Estimate: 24h > > A new configurable token filter which, given a token breaks it into sub-parts > and outputs consecutive sub-sequences of those sub-parts. > Useful for, for example, using during indexing to generate variations on > domain names, so that "www.google.com" can be found by searching for > "google.com", or "www.google.com". > Parameters: > sepRegexp: A regular expression used split incoming tokens into sub-parts. > glue: A string used to concatenate sub-parts together when creating > sub-sequences. > minLen: Minimum length (in sub-parts) of output sub-sequences > maxLen: Maximum length (in sub-parts) of output sub-sequences (0 for > unlimited; negative numbers for token length in sub-parts minus specified > length) > anchor: Anchor.START to output only prefixes, or Anchor.END to output only > suffixes, or Anchor.NONE to output any sub-sequence > withOriginal: whether to output also the original token > Note: tests will follow, I currently believe BaseTokenStreamTestCase is > broken/mis-designed for such use cases, I will open a separate Jira for that. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5674) A new token filter: SubSequence
[ https://issues.apache.org/jira/browse/LUCENE-5674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nitzan Shaked updated LUCENE-5674: -- Attachment: (was: subseqfilter.patch) > A new token filter: SubSequence > --- > > Key: LUCENE-5674 > URL: https://issues.apache.org/jira/browse/LUCENE-5674 > Project: Lucene - Core > Issue Type: Improvement > Components: core/other >Reporter: Nitzan Shaked >Priority: Minor > Original Estimate: 24h > Remaining Estimate: 24h > > A new configurable token filter which, given a token breaks it into sub-parts > and outputs consecutive sub-sequences of those sub-parts. > Useful for, for example, using during indexing to generate variations on > domain names, so that "www.google.com" can be found by searching for > "google.com", or "www.google.com". > Parameters: > sepRegexp: A regular expression used split incoming tokens into sub-parts. > glue: A string used to concatenate sub-parts together when creating > sub-sequences. > minLen: Minimum length (in sub-parts) of output sub-sequences > maxLen: Maximum length (in sub-parts) of output sub-sequences (0 for > unlimited; negative numbers for token length in sub-parts minus specified > length) > anchor: Anchor.START to output only prefixes, or Anchor.END to output only > suffixes, or Anchor.NONE to output any sub-sequence > withOriginal: whether to output also the original token > EDIT: now includes tests for filter and for factory. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5674) A new token filter: SubSequence
[
https://issues.apache.org/jira/browse/LUCENE-5674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nitzan Shaked updated LUCENE-5674:
--
Attachment: subseqfilter.patch
Fixes for {{ant precommit}}
> A new token filter: SubSequence
> ---
>
> Key: LUCENE-5674
> URL: https://issues.apache.org/jira/browse/LUCENE-5674
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/other
>Reporter: Nitzan Shaked
>Priority: Minor
> Attachments: subseqfilter.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> A new configurable token filter which, given a token breaks it into sub-parts
> and outputs consecutive sub-sequences of those sub-parts.
> Useful for, for example, using during indexing to generate variations on
> domain names, so that "www.google.com" can be found by searching for
> "google.com", or "www.google.com".
> Parameters:
> sepRegexp: A regular expression used split incoming tokens into sub-parts.
> glue: A string used to concatenate sub-parts together when creating
> sub-sequences.
> minLen: Minimum length (in sub-parts) of output sub-sequences
> maxLen: Maximum length (in sub-parts) of output sub-sequences (0 for
> unlimited; negative numbers for token length in sub-parts minus specified
> length)
> anchor: Anchor.START to output only prefixes, or Anchor.END to output only
> suffixes, or Anchor.NONE to output any sub-sequence
> withOriginal: whether to output also the original token
> EDIT: now includes tests for filter and for factory.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5674) A new token filter: SubSequence
[ https://issues.apache.org/jira/browse/LUCENE-5674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nitzan Shaked updated LUCENE-5674: -- Attachment: (was: subseqfilter.patch) > A new token filter: SubSequence > --- > > Key: LUCENE-5674 > URL: https://issues.apache.org/jira/browse/LUCENE-5674 > Project: Lucene - Core > Issue Type: Improvement > Components: core/other >Reporter: Nitzan Shaked >Priority: Minor > Original Estimate: 24h > Remaining Estimate: 24h > > A new configurable token filter which, given a token breaks it into sub-parts > and outputs consecutive sub-sequences of those sub-parts. > Useful for, for example, using during indexing to generate variations on > domain names, so that "www.google.com" can be found by searching for > "google.com", or "www.google.com". > Parameters: > sepRegexp: A regular expression used split incoming tokens into sub-parts. > glue: A string used to concatenate sub-parts together when creating > sub-sequences. > minLen: Minimum length (in sub-parts) of output sub-sequences > maxLen: Maximum length (in sub-parts) of output sub-sequences (0 for > unlimited; negative numbers for token length in sub-parts minus specified > length) > anchor: Anchor.START to output only prefixes, or Anchor.END to output only > suffixes, or Anchor.NONE to output any sub-sequence > withOriginal: whether to output also the original token > EDIT: now includes tests for filter and for factory. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5674) A new token filter: SubSequence
[ https://issues.apache.org/jira/browse/LUCENE-5674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nitzan Shaked updated LUCENE-5674: -- Attachment: subseqfilter.patch Latest patch, all commits squashed into one > A new token filter: SubSequence > --- > > Key: LUCENE-5674 > URL: https://issues.apache.org/jira/browse/LUCENE-5674 > Project: Lucene - Core > Issue Type: Improvement > Components: core/other >Reporter: Nitzan Shaked >Priority: Minor > Attachments: subseqfilter.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > A new configurable token filter which, given a token breaks it into sub-parts > and outputs consecutive sub-sequences of those sub-parts. > Useful for, for example, using during indexing to generate variations on > domain names, so that "www.google.com" can be found by searching for > "google.com", or "www.google.com". > Parameters: > sepRegexp: A regular expression used split incoming tokens into sub-parts. > glue: A string used to concatenate sub-parts together when creating > sub-sequences. > minLen: Minimum length (in sub-parts) of output sub-sequences > maxLen: Maximum length (in sub-parts) of output sub-sequences (0 for > unlimited; negative numbers for token length in sub-parts minus specified > length) > anchor: Anchor.START to output only prefixes, or Anchor.END to output only > suffixes, or Anchor.NONE to output any sub-sequence > withOriginal: whether to output also the original token > EDIT: now includes tests for filter and for factory. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5674) A new token filter: SubSequence
[ https://issues.apache.org/jira/browse/LUCENE-5674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nitzan Shaked updated LUCENE-5674: -- Attachment: (was: subseqfilter.patch) > A new token filter: SubSequence > --- > > Key: LUCENE-5674 > URL: https://issues.apache.org/jira/browse/LUCENE-5674 > Project: Lucene - Core > Issue Type: Improvement > Components: core/other >Reporter: Nitzan Shaked >Priority: Minor > Original Estimate: 24h > Remaining Estimate: 24h > > A new configurable token filter which, given a token breaks it into sub-parts > and outputs consecutive sub-sequences of those sub-parts. > Useful for, for example, using during indexing to generate variations on > domain names, so that "www.google.com" can be found by searching for > "google.com", or "www.google.com". > Parameters: > sepRegexp: A regular expression used split incoming tokens into sub-parts. > glue: A string used to concatenate sub-parts together when creating > sub-sequences. > minLen: Minimum length (in sub-parts) of output sub-sequences > maxLen: Maximum length (in sub-parts) of output sub-sequences (0 for > unlimited; negative numbers for token length in sub-parts minus specified > length) > anchor: Anchor.START to output only prefixes, or Anchor.END to output only > suffixes, or Anchor.NONE to output any sub-sequence > withOriginal: whether to output also the original token > EDIT: now includes tests for filter and for factory. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5674) A new token filter: SubSequence
[ https://issues.apache.org/jira/browse/LUCENE-5674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nitzan Shaked updated LUCENE-5674: -- Attachment: subseqfilter.patch Updated patch, contains "new format header" for the one place that used the "old format header" > A new token filter: SubSequence > --- > > Key: LUCENE-5674 > URL: https://issues.apache.org/jira/browse/LUCENE-5674 > Project: Lucene - Core > Issue Type: Improvement > Components: core/other >Reporter: Nitzan Shaked >Priority: Minor > Attachments: subseqfilter.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > A new configurable token filter which, given a token breaks it into sub-parts > and outputs consecutive sub-sequences of those sub-parts. > Useful for, for example, using during indexing to generate variations on > domain names, so that "www.google.com" can be found by searching for > "google.com", or "www.google.com". > Parameters: > sepRegexp: A regular expression used split incoming tokens into sub-parts. > glue: A string used to concatenate sub-parts together when creating > sub-sequences. > minLen: Minimum length (in sub-parts) of output sub-sequences > maxLen: Maximum length (in sub-parts) of output sub-sequences (0 for > unlimited; negative numbers for token length in sub-parts minus specified > length) > anchor: Anchor.START to output only prefixes, or Anchor.END to output only > suffixes, or Anchor.NONE to output any sub-sequence > withOriginal: whether to output also the original token > EDIT: now includes tests for filter and for factory. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
