[
https://issues.apache.org/jira/browse/LUCENENET-616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shad Storhaug updated LUCENENET-616:
------------------------------------
Description:
The collection types in {{Lucene.Net.Support}} were originally sourced to
support Lucene.Net itself. While they were made public, they were not
considered to be features that would be used by anyone except for advanced
users.
However, it has become clear by user reports that some parts of Lucene's design
require specialized collections that don't exist in the .NET Framework in order
to properly function (see
[LUCENENET-612|https://issues.apache.org/jira/projects/LUCENENET/issues/LUCENENET-612]
and
[LUCENENET-615|https://issues.apache.org/jira/projects/LUCENENET/issues/LUCENENET-615].
.NET users are generally not familiar with these specialized collection types
and assume that when {{IDictionary<TKey, TValue>}} is required by an API that
using {{Dictionary<TKey, TValue>}} is their best choice. We need to improve
documentation and increase visibility of the specialized collection types in
order to help them along.
Some of the existing collections are composed of other nested collection
objects and should be replaced with a lower-level implementation, if possible.
Since these are breaking API changes, they should be done before the official
release of Lucene.Net 4.8.0.
Additionally, many parts of Lucene.NET expect the collections to:
* Be structurally equal
* Format their contents in the ToString() method
The safest and most thorough way to achieve this is to replace the usage of all
built-in .NET collections with collections that we own that implement this
functionality. .NET provides interfaces to help achieve this:
* {{IStructuralEquatable}}
* {{IFormattable}}
But neither interface is implemented in any of the built-in collections in .NET.
h2. *Requirements for Lucene.NET Collections*
We must have the following collections for internal use within Lucene.NET, and
each should also be made available to end users.
* {{J2N.Collections.Generic.List<T>}}
** Subclass of {{System.Collections.Generic.List<T>}}
** Implements {{IStructuralEquatable}}
** Implements {{IFormattable}}
* {{J2N.Collections.Generic.HashSet<T>}}
** Subclass of {{System.Collections.Generic.List<T>}}
** Implements {{IStructuralEquatable}}
** Implements {{IFormattable}}
* {{J2N.Collections.Generic.SortedSet<T>}}
** Subclass of {{System.Collections.Generic.SortedSet<T>}}
** Implements {{IStructuralEquatable}}
** Implements {{IFormattable}}
* {{J2N.Collections.Generic.Dictionary<TKey, TValue>}}
** Same interface as {{System.Collections.Generic.Dictionary<TKey, TValue>}}
** Supports null keys
** Implements {{IStructuralEquatable}}
** Implements {{IFormattable}}
* {{J2N.Collections.Generic.SortedDictionary<TKey, TValue>}}
** Same interface as {{System.Collections.Generic.SortedDictionary<TKey,
TValue>}}
** Supports null keys
** Implements {{IStructuralEquatable}}
** Implements {{IFormattable}}
* {{J2N.Collections.Generic.LinkedDictionary<TKey, TValue>}}
** Same interface as {{System.Collections.Generic.Dictionary<TKey, TValue>}}
** Preserves insertion order across adds/deletes
** Supports null keys
** Implements {{IStructuralEquatable}}
** Implements {{IFormattable}}
* {{J2N.Collections.Generic.PriorityQueue<T>}}
** Migrate {{Lucene.Net.Support.PriorityQueue<T>}} implementation, but clean
up the API for .NET (AddAll() > AddRange(), etc.)
* {{J2N.Collections.Generic.IdentityDictionary<TKey, TValue>}} (See
{{Lucene.Net.Support.IdentityHashMap<TKey, TValue>}})
** Same interface as {{System.Collections.Generic.Dictionary<TKey, TValue>}}
** Supports null keys
** Implements {{IStructuralEquatable}}
** Implements {{IFormattable}}
* {{J2N.Collections.Generic.IdentityHashSet<T>}} (See
{{Lucene.Net.Support.IdentityHashSet<T>}})
** Subclass of {{System.Collections.Generic.HashSet<T>}}
** Implements {{IStructuralEquatable}}
** Implements {{IFormattable}}
* {{ConcurrentHashSet<T>}} - (See LUCENENET-634)
We should have the following collections for internal use within Lucene.NET.
* {{J2N.Collections.Generic.LinkedHashSet<T>}}
** Same interface as {{System.Collections.Generic.List<T>}}
** Preserves insertion order across adds/deletes
** Implements {{IStructuralEquatable}}
** Implements {{IFormattable}}
* {{J2N.Runtime.CompilerServices.ConditionalWeakTable<TKey, TValue>}} (See
LUCENENET-636)
h3. Special Cases
There are a few uses where the above collections won't suffice because they use
low-level methods that don't exist in .NET collections. However, these are
closed systems and we don't have to worry about supporting end users in these
cases.
* {{Lucene.Net.TestFramework.Analysis.MockCharFilter}} > Use C5's
{{TreeDictionary<TKey, TValue}}
* {{Lucene.Net.Grouping.AbstractGroupFacetCollector}} > Use C5's {{TreeSet<T>}}
* {{Lucene.Net.Highlighter.PostingsHighlight.PostingsHighlighter}} > Use C5's
{{TreeSet<T>}}
Note that C5's {{TreeSet<T>}} does not currently support {{ISet<T>}}, and we
will need to push our implementation of it back to them to support this change.
Structural Equality
J2N has {{J2N.Collections.Generic.ListEqualityComparer<T>}},
{{J2N.Collections.Generic.SetEqualityComparer<T>}},
{{J2N.Collections.Generic.DictionaryEqualityComparer<TKey, TValue>}}, and
{{J2N.Collections.StructuralEqualityComparer<T>}} that implement the structural
equality behavior for each type of collection. These types should be passed in
through each collection's constructor, as each has a {{Default}} and an
{{Aggressive}} implementation.
{{Default}} is more efficient and should be used in all closed scenarios within
Lucene.NET (where the collection is not passed in from outside of the class).
{{Aggressive}} is designed to interoperate with built-in .NET collections.
See [these minimal examples of
implementations|https://github.com/NightOwl888/J2N/tree/a50d41668cf459f3fe827dff5f1746422155c4f6/src/J2N.TestFramework/Util].
Collection Formatting
J2N has a {{StringFormatter}} class that can be used to format collections, as
follows.
{code:c#}
// IFormattable
public string ToString(IFormatProvider provider)
{
return string.Format(provider, "{0}", this);
}
public override string ToString()
{
return ToString(StringFormatter.CurrentCulture);
}
{code}
Nullable Keys
Use the attached {{NullableKey<T>}} and {{NullableKeyDictionary<TKey, TValue>}}
implementations. {{NullableKeyDictionary<TKey, TValue>}} is intended to be a
base class for all nullable key dictionary implementations. The backing
dictionary (the one we are trying to mimic) is then passed in through its
constructor.
was:
The collection types in {{Lucene.Net.Support}} were originally sourced to
support Lucene.Net itself. While they were made public, they were not
considered to be features that would be used by anyone except for advanced
users.
However, it has become clear by user reports that some parts of Lucene's design
require specialized collections that don't exist in the .NET Framework in order
to properly function (see
[LUCENENET-612|https://issues.apache.org/jira/projects/LUCENENET/issues/LUCENENET-612]
and
[LUCENENET-615|https://issues.apache.org/jira/projects/LUCENENET/issues/LUCENENET-615].
.NET users are generally not familiar with these specialized collection types
and assume that when {{IDictionary<TKey, TValue>}} is required by an API that
using {{Dictionary<TKey, TValue>}} is their best choice. We need to improve
documentation and increase visibility of the specialized collection types in
order to help them along.
# The collection types should be moved to a new
{{Lucene.Net.Collections.Specialized}} namespace
# The collection types should stay in the {{Support}} folder, but be moved
into a {{Collections/Specialized}} subfolder. The {{Collections}} class needs
to be renamed to prevent a naming conflict - I suggest following the same
approach that was done in [ICU4N's Support
folder|https://github.com/NightOwl888/ICU4N/tree/master/src/ICU4N/Support], by
making many of them into extension methods.
# The collection types should be named consistently according to .NET
conventions, for example {{HashMap}} should be renamed {{HashDictionary}}
# We should ensure we have a full set of tests for each collection, and that
each collection is fully implemented
# All collections must implement the expected interfaces from
{{System.Collections}}, for example a generic dictionary should implement
{{System.Collections.Generic.IDictionary<TKey, TValue>}}
# The documentation for each collection should not just indicate where it was
grabbed from, but explain fully how it behaves and how its behavior differs
from other collection types
# APIs where it is known that specialized collections were intended to be used
should also be documented to indicate the choices the end user has so they
don't automatically assume they are constrained by the .NET framework's
collection types
# There should be a "namespace document" in {{Lucene.Net.Collections.Generic}}
that lists all of the collection types and a summary of their behavior, so they
can be comparison shopped easily
Since these are breaking API changes, they should be done before the official
release of Lucene.Net 4.8.0.
Generally speaking, collections were fully implemented and tested to prevent
odd bugs from creeping into Lucene.Net but it wouldn't hurt to review to ensure
that is the case.
> Make Collections from Lucene.Net.Support into a 1st Class Feature
> -----------------------------------------------------------------
>
> Key: LUCENENET-616
> URL: https://issues.apache.org/jira/browse/LUCENENET-616
> Project: Lucene.Net
> Issue Type: Improvement
> Components: Lucene.Net Core
> Affects Versions: Lucene.Net 4.8.0
> Reporter: Shad Storhaug
> Assignee: Shad Storhaug
> Priority: Major
>
> The collection types in {{Lucene.Net.Support}} were originally sourced to
> support Lucene.Net itself. While they were made public, they were not
> considered to be features that would be used by anyone except for advanced
> users.
> However, it has become clear by user reports that some parts of Lucene's
> design require specialized collections that don't exist in the .NET Framework
> in order to properly function (see
> [LUCENENET-612|https://issues.apache.org/jira/projects/LUCENENET/issues/LUCENENET-612]
> and
> [LUCENENET-615|https://issues.apache.org/jira/projects/LUCENENET/issues/LUCENENET-615].
> .NET users are generally not familiar with these specialized collection
> types and assume that when {{IDictionary<TKey, TValue>}} is required by an
> API that using {{Dictionary<TKey, TValue>}} is their best choice. We need to
> improve documentation and increase visibility of the specialized collection
> types in order to help them along.
> Some of the existing collections are composed of other nested collection
> objects and should be replaced with a lower-level implementation, if
> possible. Since these are breaking API changes, they should be done before
> the official release of Lucene.Net 4.8.0.
> Additionally, many parts of Lucene.NET expect the collections to:
> * Be structurally equal
> * Format their contents in the ToString() method
> The safest and most thorough way to achieve this is to replace the usage of
> all built-in .NET collections with collections that we own that implement
> this functionality. .NET provides interfaces to help achieve this:
> * {{IStructuralEquatable}}
> * {{IFormattable}}
> But neither interface is implemented in any of the built-in collections in
> .NET.
> h2. *Requirements for Lucene.NET Collections*
> We must have the following collections for internal use within Lucene.NET,
> and each should also be made available to end users.
> * {{J2N.Collections.Generic.List<T>}}
> ** Subclass of {{System.Collections.Generic.List<T>}}
> ** Implements {{IStructuralEquatable}}
> ** Implements {{IFormattable}}
> * {{J2N.Collections.Generic.HashSet<T>}}
> ** Subclass of {{System.Collections.Generic.List<T>}}
> ** Implements {{IStructuralEquatable}}
> ** Implements {{IFormattable}}
> * {{J2N.Collections.Generic.SortedSet<T>}}
> ** Subclass of {{System.Collections.Generic.SortedSet<T>}}
> ** Implements {{IStructuralEquatable}}
> ** Implements {{IFormattable}}
> * {{J2N.Collections.Generic.Dictionary<TKey, TValue>}}
> ** Same interface as {{System.Collections.Generic.Dictionary<TKey, TValue>}}
> ** Supports null keys
> ** Implements {{IStructuralEquatable}}
> ** Implements {{IFormattable}}
> * {{J2N.Collections.Generic.SortedDictionary<TKey, TValue>}}
> ** Same interface as {{System.Collections.Generic.SortedDictionary<TKey,
> TValue>}}
> ** Supports null keys
> ** Implements {{IStructuralEquatable}}
> ** Implements {{IFormattable}}
> * {{J2N.Collections.Generic.LinkedDictionary<TKey, TValue>}}
> ** Same interface as {{System.Collections.Generic.Dictionary<TKey, TValue>}}
> ** Preserves insertion order across adds/deletes
> ** Supports null keys
> ** Implements {{IStructuralEquatable}}
> ** Implements {{IFormattable}}
> * {{J2N.Collections.Generic.PriorityQueue<T>}}
> ** Migrate {{Lucene.Net.Support.PriorityQueue<T>}} implementation, but clean
> up the API for .NET (AddAll() > AddRange(), etc.)
> * {{J2N.Collections.Generic.IdentityDictionary<TKey, TValue>}} (See
> {{Lucene.Net.Support.IdentityHashMap<TKey, TValue>}})
> ** Same interface as {{System.Collections.Generic.Dictionary<TKey, TValue>}}
> ** Supports null keys
> ** Implements {{IStructuralEquatable}}
> ** Implements {{IFormattable}}
> * {{J2N.Collections.Generic.IdentityHashSet<T>}} (See
> {{Lucene.Net.Support.IdentityHashSet<T>}})
> ** Subclass of {{System.Collections.Generic.HashSet<T>}}
> ** Implements {{IStructuralEquatable}}
> ** Implements {{IFormattable}}
> * {{ConcurrentHashSet<T>}} - (See LUCENENET-634)
> We should have the following collections for internal use within Lucene.NET.
> * {{J2N.Collections.Generic.LinkedHashSet<T>}}
> ** Same interface as {{System.Collections.Generic.List<T>}}
> ** Preserves insertion order across adds/deletes
> ** Implements {{IStructuralEquatable}}
> ** Implements {{IFormattable}}
> * {{J2N.Runtime.CompilerServices.ConditionalWeakTable<TKey, TValue>}} (See
> LUCENENET-636)
> h3. Special Cases
> There are a few uses where the above collections won't suffice because they
> use low-level methods that don't exist in .NET collections. However, these
> are closed systems and we don't have to worry about supporting end users in
> these cases.
> * {{Lucene.Net.TestFramework.Analysis.MockCharFilter}} > Use C5's
> {{TreeDictionary<TKey, TValue}}
> * {{Lucene.Net.Grouping.AbstractGroupFacetCollector}} > Use C5's
> {{TreeSet<T>}}
> * {{Lucene.Net.Highlighter.PostingsHighlight.PostingsHighlighter}} > Use
> C5's {{TreeSet<T>}}
> Note that C5's {{TreeSet<T>}} does not currently support {{ISet<T>}}, and we
> will need to push our implementation of it back to them to support this
> change.
> Structural Equality
> J2N has {{J2N.Collections.Generic.ListEqualityComparer<T>}},
> {{J2N.Collections.Generic.SetEqualityComparer<T>}},
> {{J2N.Collections.Generic.DictionaryEqualityComparer<TKey, TValue>}}, and
> {{J2N.Collections.StructuralEqualityComparer<T>}} that implement the
> structural equality behavior for each type of collection. These types should
> be passed in through each collection's constructor, as each has a {{Default}}
> and an {{Aggressive}} implementation.
> {{Default}} is more efficient and should be used in all closed scenarios
> within Lucene.NET (where the collection is not passed in from outside of the
> class). {{Aggressive}} is designed to interoperate with built-in .NET
> collections.
> See [these minimal examples of
> implementations|https://github.com/NightOwl888/J2N/tree/a50d41668cf459f3fe827dff5f1746422155c4f6/src/J2N.TestFramework/Util].
> Collection Formatting
> J2N has a {{StringFormatter}} class that can be used to format collections,
> as follows.
> {code:c#}
> // IFormattable
> public string ToString(IFormatProvider provider)
> {
> return string.Format(provider, "{0}", this);
> }
> public override string ToString()
> {
> return ToString(StringFormatter.CurrentCulture);
> }
> {code}
> Nullable Keys
> Use the attached {{NullableKey<T>}} and {{NullableKeyDictionary<TKey,
> TValue>}} implementations. {{NullableKeyDictionary<TKey, TValue>}} is
> intended to be a base class for all nullable key dictionary implementations.
> The backing dictionary (the one we are trying to mimic) is then passed in
> through its constructor.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
