[
https://issues.apache.org/jira/browse/LUCENENET-616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shad Storhaug updated LUCENENET-616:
------------------------------------
Attachment: NullableKey.cs
NullableKeyDictionary.cs
Fix Version/s: Lucene.Net 4.8.0
Flags: Important
Description:
The collection types in {{Lucene.Net.Support}} were originally sourced to
support Lucene.Net itself. While they were made public, they were not
considered to be features that would be used by anyone except for advanced
users.
However, it has become clear by user reports that some parts of Lucene's design
require specialized collections that don't exist in the .NET Framework in order
to properly function (see
[LUCENENET-612|https://issues.apache.org/jira/projects/LUCENENET/issues/LUCENENET-612]
and
[LUCENENET-615|https://issues.apache.org/jira/projects/LUCENENET/issues/LUCENENET-615].
.NET users are generally not familiar with these specialized collection types
and assume that when {{IDictionary<TKey, TValue>}} is required by an API that
using {{Dictionary<TKey, TValue>}} is their best choice. We need to improve
documentation and increase visibility of the specialized collection types in
order to help them along.
Some of the existing collections are composed of other nested collection
objects and should be replaced with a lower-level implementation, if possible.
Since these are breaking API changes, they should be done before the official
release of Lucene.Net 4.8.0.
Additionally, many parts of Lucene.NET expect the collections to:
* Be structurally equal
* Format their contents in the ToString() method
The safest and most thorough way to achieve this is to replace the usage of all
built-in .NET collections with collections that we own that implement this
functionality. .NET provides interfaces to help achieve this:
* {{IStructuralEquatable}}
* {{IFormattable}}
But neither interface is implemented in any of the built-in collections in .NET.
h2. *Requirements for Lucene.NET Collections*
We must have the following collections for internal use within Lucene.NET, and
each should also be made available to end users.
* {{J2N.Collections.Generic.List<T>}}
** Subclass of {{System.Collections.Generic.List<T>}}
** Implements {{IStructuralEquatable}}
** Implements {{IFormattable}}
* {{J2N.Collections.Generic.HashSet<T>}}
** Subclass of {{System.Collections.Generic.List<T>}}
** Implements {{IStructuralEquatable}}
** Implements {{IFormattable}}
* {{J2N.Collections.Generic.SortedSet<T>}}
** Subclass of {{System.Collections.Generic.SortedSet<T>}}
** Implements {{IStructuralEquatable}}
** Implements {{IFormattable}}
* {{J2N.Collections.Generic.Dictionary<TKey, TValue>}}
** Same interface as {{System.Collections.Generic.Dictionary<TKey, TValue>}}
** Supports null keys
** Implements {{IStructuralEquatable}}
** Implements {{IFormattable}}
* {{J2N.Collections.Generic.SortedDictionary<TKey, TValue>}}
** Same interface as {{System.Collections.Generic.SortedDictionary<TKey,
TValue>}}
** Supports null keys
** Implements {{IStructuralEquatable}}
** Implements {{IFormattable}}
* {{J2N.Collections.Generic.LinkedDictionary<TKey, TValue>}}
** Same interface as {{System.Collections.Generic.Dictionary<TKey, TValue>}}
** Preserves insertion order across adds/deletes
** Supports null keys
** Implements {{IStructuralEquatable}}
** Implements {{IFormattable}}
* {{J2N.Collections.Generic.PriorityQueue<T>}}
** Migrate {{Lucene.Net.Support.PriorityQueue<T>}} implementation, but clean
up the API for .NET (AddAll() > AddRange(), etc.)
* {{J2N.Collections.Generic.IdentityDictionary<TKey, TValue>}} (See
{{Lucene.Net.Support.IdentityHashMap<TKey, TValue>}})
** Same interface as {{System.Collections.Generic.Dictionary<TKey, TValue>}}
** Supports null keys
** Implements {{IStructuralEquatable}}
** Implements {{IFormattable}}
* {{J2N.Collections.Generic.IdentityHashSet<T>}} (See
{{Lucene.Net.Support.IdentityHashSet<T>}})
** Subclass of {{System.Collections.Generic.HashSet<T>}}
** Implements {{IStructuralEquatable}}
** Implements {{IFormattable}}
* {{ConcurrentHashSet<T>}} - (See LUCENENET-634)
We should have the following collections for internal use within Lucene.NET.
* {{J2N.Collections.Generic.LinkedHashSet<T>}}
** Same interface as {{System.Collections.Generic.List<T>}}
** Preserves insertion order across adds/deletes
** Implements {{IStructuralEquatable}}
** Implements {{IFormattable}}
* {{J2N.Runtime.CompilerServices.ConditionalWeakTable<TKey, TValue>}} (See
LUCENENET-636)
h3. *Special Cases*
There are a few uses where the above collections won't suffice because they use
low-level methods that don't exist in .NET collections. However, these are
closed systems and we don't have to worry about supporting end users in these
cases.
* {{Lucene.Net.TestFramework.Analysis.MockCharFilter}} > Use C5's
{{TreeDictionary<TKey, TValue}}
* {{Lucene.Net.Grouping.AbstractGroupFacetCollector}} > Use C5's {{TreeSet<T>}}
* {{Lucene.Net.Highlighter.PostingsHighlight.PostingsHighlighter}} > Use C5's
{{TreeSet<T>}}
Note that C5's {{TreeSet<T>}} does not currently support {{ISet<T>}}, and we
will need to push our implementation of it back to them to support this change.
h3. *Structural Equality*
J2N has {{J2N.Collections.Generic.ListEqualityComparer<T>}},
{{J2N.Collections.Generic.SetEqualityComparer<T>}},
{{J2N.Collections.Generic.DictionaryEqualityComparer<TKey, TValue>}}, and
{{J2N.Collections.StructuralEqualityComparer<T>}} that implement the structural
equality behavior for each type of collection. These types should be passed in
through each collection's constructor, as each has a {{Default}} and an
{{Aggressive}} implementation.
{{Default}} is more efficient and should be used in all closed scenarios within
Lucene.NET (where the collection is not passed in from outside of the class).
{{Aggressive}} is designed to interoperate with built-in .NET collections.
See [these minimal examples of
implementations|https://github.com/NightOwl888/J2N/tree/a50d41668cf459f3fe827dff5f1746422155c4f6/src/J2N.TestFramework/Util].
h3. *Collection Formatting*
J2N has a {{StringFormatter}} class that can be used to format collections, as
follows.
{code:c#}
// IFormattable
public string ToString(IFormatProvider provider)
{
return string.Format(provider, "{0}", this);
}
public override string ToString()
{
return ToString(StringFormatter.CurrentCulture);
}
{code}
h3. *Nullable Keys*
Use the attached {{NullableKey<T>}} and {{NullableKeyDictionary<TKey, TValue>}}
implementations. {{NullableKeyDictionary<TKey, TValue>}} is intended to be a
base class for all nullable key dictionary implementations. The backing
dictionary (the one we are trying to mimic) is then passed in through its
constructor.
h3. *Integration*
After the collections have been implemented, we can eliminate the usage of
{{Lucene.Net.Support.Collections.Equals()}},
{{Lucene.Net.Support.Collections.GetHashCode(), and
{{Lucene.Net.Support.Collections.ToString()}}.
Usage in of {{Lucene.Net.Support.Collections.Equals()}} and
{{Lucene.Net.Support.Collections.GetHashCode() in Lucene.Net can be changed to
the equality comparer matching the collection type.
For string formatting, the usage can simply be changed to
{{collection.ToString()}} in production code, and we can use in
{{collection.ToString(J2N.Text.StringFormatter.InvariantCulture)}} to patch
tests.
For types where collections are passed from the outside that may contain nested
collections, we should either use {{Aggressive}} structural comparison, or
document that if nested collections are needed, the end user should use
collections that implement {{IStructuralEquatable}}, such as those from J2N.
was:
The collection types in {{Lucene.Net.Support}} were originally sourced to
support Lucene.Net itself. While they were made public, they were not
considered to be features that would be used by anyone except for advanced
users.
However, it has become clear by user reports that some parts of Lucene's design
require specialized collections that don't exist in the .NET Framework in order
to properly function (see
[LUCENENET-612|https://issues.apache.org/jira/projects/LUCENENET/issues/LUCENENET-612]
and
[LUCENENET-615|https://issues.apache.org/jira/projects/LUCENENET/issues/LUCENENET-615].
.NET users are generally not familiar with these specialized collection types
and assume that when {{IDictionary<TKey, TValue>}} is required by an API that
using {{Dictionary<TKey, TValue>}} is their best choice. We need to improve
documentation and increase visibility of the specialized collection types in
order to help them along.
Some of the existing collections are composed of other nested collection
objects and should be replaced with a lower-level implementation, if possible.
Since these are breaking API changes, they should be done before the official
release of Lucene.Net 4.8.0.
Additionally, many parts of Lucene.NET expect the collections to:
* Be structurally equal
* Format their contents in the ToString() method
The safest and most thorough way to achieve this is to replace the usage of all
built-in .NET collections with collections that we own that implement this
functionality. .NET provides interfaces to help achieve this:
* {{IStructuralEquatable}}
* {{IFormattable}}
But neither interface is implemented in any of the built-in collections in .NET.
h2. *Requirements for Lucene.NET Collections*
We must have the following collections for internal use within Lucene.NET, and
each should also be made available to end users.
* {{J2N.Collections.Generic.List<T>}}
** Subclass of {{System.Collections.Generic.List<T>}}
** Implements {{IStructuralEquatable}}
** Implements {{IFormattable}}
* {{J2N.Collections.Generic.HashSet<T>}}
** Subclass of {{System.Collections.Generic.List<T>}}
** Implements {{IStructuralEquatable}}
** Implements {{IFormattable}}
* {{J2N.Collections.Generic.SortedSet<T>}}
** Subclass of {{System.Collections.Generic.SortedSet<T>}}
** Implements {{IStructuralEquatable}}
** Implements {{IFormattable}}
* {{J2N.Collections.Generic.Dictionary<TKey, TValue>}}
** Same interface as {{System.Collections.Generic.Dictionary<TKey, TValue>}}
** Supports null keys
** Implements {{IStructuralEquatable}}
** Implements {{IFormattable}}
* {{J2N.Collections.Generic.SortedDictionary<TKey, TValue>}}
** Same interface as {{System.Collections.Generic.SortedDictionary<TKey,
TValue>}}
** Supports null keys
** Implements {{IStructuralEquatable}}
** Implements {{IFormattable}}
* {{J2N.Collections.Generic.LinkedDictionary<TKey, TValue>}}
** Same interface as {{System.Collections.Generic.Dictionary<TKey, TValue>}}
** Preserves insertion order across adds/deletes
** Supports null keys
** Implements {{IStructuralEquatable}}
** Implements {{IFormattable}}
* {{J2N.Collections.Generic.PriorityQueue<T>}}
** Migrate {{Lucene.Net.Support.PriorityQueue<T>}} implementation, but clean
up the API for .NET (AddAll() > AddRange(), etc.)
* {{J2N.Collections.Generic.IdentityDictionary<TKey, TValue>}} (See
{{Lucene.Net.Support.IdentityHashMap<TKey, TValue>}})
** Same interface as {{System.Collections.Generic.Dictionary<TKey, TValue>}}
** Supports null keys
** Implements {{IStructuralEquatable}}
** Implements {{IFormattable}}
* {{J2N.Collections.Generic.IdentityHashSet<T>}} (See
{{Lucene.Net.Support.IdentityHashSet<T>}})
** Subclass of {{System.Collections.Generic.HashSet<T>}}
** Implements {{IStructuralEquatable}}
** Implements {{IFormattable}}
* {{ConcurrentHashSet<T>}} - (See LUCENENET-634)
We should have the following collections for internal use within Lucene.NET.
* {{J2N.Collections.Generic.LinkedHashSet<T>}}
** Same interface as {{System.Collections.Generic.List<T>}}
** Preserves insertion order across adds/deletes
** Implements {{IStructuralEquatable}}
** Implements {{IFormattable}}
* {{J2N.Runtime.CompilerServices.ConditionalWeakTable<TKey, TValue>}} (See
LUCENENET-636)
h3. Special Cases
There are a few uses where the above collections won't suffice because they use
low-level methods that don't exist in .NET collections. However, these are
closed systems and we don't have to worry about supporting end users in these
cases.
* {{Lucene.Net.TestFramework.Analysis.MockCharFilter}} > Use C5's
{{TreeDictionary<TKey, TValue}}
* {{Lucene.Net.Grouping.AbstractGroupFacetCollector}} > Use C5's {{TreeSet<T>}}
* {{Lucene.Net.Highlighter.PostingsHighlight.PostingsHighlighter}} > Use C5's
{{TreeSet<T>}}
Note that C5's {{TreeSet<T>}} does not currently support {{ISet<T>}}, and we
will need to push our implementation of it back to them to support this change.
Structural Equality
J2N has {{J2N.Collections.Generic.ListEqualityComparer<T>}},
{{J2N.Collections.Generic.SetEqualityComparer<T>}},
{{J2N.Collections.Generic.DictionaryEqualityComparer<TKey, TValue>}}, and
{{J2N.Collections.StructuralEqualityComparer<T>}} that implement the structural
equality behavior for each type of collection. These types should be passed in
through each collection's constructor, as each has a {{Default}} and an
{{Aggressive}} implementation.
{{Default}} is more efficient and should be used in all closed scenarios within
Lucene.NET (where the collection is not passed in from outside of the class).
{{Aggressive}} is designed to interoperate with built-in .NET collections.
See [these minimal examples of
implementations|https://github.com/NightOwl888/J2N/tree/a50d41668cf459f3fe827dff5f1746422155c4f6/src/J2N.TestFramework/Util].
Collection Formatting
J2N has a {{StringFormatter}} class that can be used to format collections, as
follows.
{code:c#}
// IFormattable
public string ToString(IFormatProvider provider)
{
return string.Format(provider, "{0}", this);
}
public override string ToString()
{
return ToString(StringFormatter.CurrentCulture);
}
{code}
Nullable Keys
Use the attached {{NullableKey<T>}} and {{NullableKeyDictionary<TKey, TValue>}}
implementations. {{NullableKeyDictionary<TKey, TValue>}} is intended to be a
base class for all nullable key dictionary implementations. The backing
dictionary (the one we are trying to mimic) is then passed in through its
constructor.
Remaining Estimate: 60h
Original Estimate: 60h
> Make Collections from Lucene.Net.Support into a 1st Class Feature
> -----------------------------------------------------------------
>
> Key: LUCENENET-616
> URL: https://issues.apache.org/jira/browse/LUCENENET-616
> Project: Lucene.Net
> Issue Type: Improvement
> Components: Lucene.Net Core
> Affects Versions: Lucene.Net 4.8.0
> Reporter: Shad Storhaug
> Assignee: Shad Storhaug
> Priority: Major
> Fix For: Lucene.Net 4.8.0
>
> Attachments: NullableKey.cs, NullableKeyDictionary.cs
>
> Original Estimate: 60h
> Remaining Estimate: 60h
>
> The collection types in {{Lucene.Net.Support}} were originally sourced to
> support Lucene.Net itself. While they were made public, they were not
> considered to be features that would be used by anyone except for advanced
> users.
> However, it has become clear by user reports that some parts of Lucene's
> design require specialized collections that don't exist in the .NET Framework
> in order to properly function (see
> [LUCENENET-612|https://issues.apache.org/jira/projects/LUCENENET/issues/LUCENENET-612]
> and
> [LUCENENET-615|https://issues.apache.org/jira/projects/LUCENENET/issues/LUCENENET-615].
> .NET users are generally not familiar with these specialized collection
> types and assume that when {{IDictionary<TKey, TValue>}} is required by an
> API that using {{Dictionary<TKey, TValue>}} is their best choice. We need to
> improve documentation and increase visibility of the specialized collection
> types in order to help them along.
> Some of the existing collections are composed of other nested collection
> objects and should be replaced with a lower-level implementation, if
> possible. Since these are breaking API changes, they should be done before
> the official release of Lucene.Net 4.8.0.
> Additionally, many parts of Lucene.NET expect the collections to:
> * Be structurally equal
> * Format their contents in the ToString() method
> The safest and most thorough way to achieve this is to replace the usage of
> all built-in .NET collections with collections that we own that implement
> this functionality. .NET provides interfaces to help achieve this:
> * {{IStructuralEquatable}}
> * {{IFormattable}}
> But neither interface is implemented in any of the built-in collections in
> .NET.
> h2. *Requirements for Lucene.NET Collections*
> We must have the following collections for internal use within Lucene.NET,
> and each should also be made available to end users.
> * {{J2N.Collections.Generic.List<T>}}
> ** Subclass of {{System.Collections.Generic.List<T>}}
> ** Implements {{IStructuralEquatable}}
> ** Implements {{IFormattable}}
> * {{J2N.Collections.Generic.HashSet<T>}}
> ** Subclass of {{System.Collections.Generic.List<T>}}
> ** Implements {{IStructuralEquatable}}
> ** Implements {{IFormattable}}
> * {{J2N.Collections.Generic.SortedSet<T>}}
> ** Subclass of {{System.Collections.Generic.SortedSet<T>}}
> ** Implements {{IStructuralEquatable}}
> ** Implements {{IFormattable}}
> * {{J2N.Collections.Generic.Dictionary<TKey, TValue>}}
> ** Same interface as {{System.Collections.Generic.Dictionary<TKey, TValue>}}
> ** Supports null keys
> ** Implements {{IStructuralEquatable}}
> ** Implements {{IFormattable}}
> * {{J2N.Collections.Generic.SortedDictionary<TKey, TValue>}}
> ** Same interface as {{System.Collections.Generic.SortedDictionary<TKey,
> TValue>}}
> ** Supports null keys
> ** Implements {{IStructuralEquatable}}
> ** Implements {{IFormattable}}
> * {{J2N.Collections.Generic.LinkedDictionary<TKey, TValue>}}
> ** Same interface as {{System.Collections.Generic.Dictionary<TKey, TValue>}}
> ** Preserves insertion order across adds/deletes
> ** Supports null keys
> ** Implements {{IStructuralEquatable}}
> ** Implements {{IFormattable}}
> * {{J2N.Collections.Generic.PriorityQueue<T>}}
> ** Migrate {{Lucene.Net.Support.PriorityQueue<T>}} implementation, but clean
> up the API for .NET (AddAll() > AddRange(), etc.)
> * {{J2N.Collections.Generic.IdentityDictionary<TKey, TValue>}} (See
> {{Lucene.Net.Support.IdentityHashMap<TKey, TValue>}})
> ** Same interface as {{System.Collections.Generic.Dictionary<TKey, TValue>}}
> ** Supports null keys
> ** Implements {{IStructuralEquatable}}
> ** Implements {{IFormattable}}
> * {{J2N.Collections.Generic.IdentityHashSet<T>}} (See
> {{Lucene.Net.Support.IdentityHashSet<T>}})
> ** Subclass of {{System.Collections.Generic.HashSet<T>}}
> ** Implements {{IStructuralEquatable}}
> ** Implements {{IFormattable}}
> * {{ConcurrentHashSet<T>}} - (See LUCENENET-634)
> We should have the following collections for internal use within Lucene.NET.
> * {{J2N.Collections.Generic.LinkedHashSet<T>}}
> ** Same interface as {{System.Collections.Generic.List<T>}}
> ** Preserves insertion order across adds/deletes
> ** Implements {{IStructuralEquatable}}
> ** Implements {{IFormattable}}
> * {{J2N.Runtime.CompilerServices.ConditionalWeakTable<TKey, TValue>}} (See
> LUCENENET-636)
> h3. *Special Cases*
> There are a few uses where the above collections won't suffice because they
> use low-level methods that don't exist in .NET collections. However, these
> are closed systems and we don't have to worry about supporting end users in
> these cases.
> * {{Lucene.Net.TestFramework.Analysis.MockCharFilter}} > Use C5's
> {{TreeDictionary<TKey, TValue}}
> * {{Lucene.Net.Grouping.AbstractGroupFacetCollector}} > Use C5's
> {{TreeSet<T>}}
> * {{Lucene.Net.Highlighter.PostingsHighlight.PostingsHighlighter}} > Use
> C5's {{TreeSet<T>}}
> Note that C5's {{TreeSet<T>}} does not currently support {{ISet<T>}}, and we
> will need to push our implementation of it back to them to support this
> change.
> h3. *Structural Equality*
> J2N has {{J2N.Collections.Generic.ListEqualityComparer<T>}},
> {{J2N.Collections.Generic.SetEqualityComparer<T>}},
> {{J2N.Collections.Generic.DictionaryEqualityComparer<TKey, TValue>}}, and
> {{J2N.Collections.StructuralEqualityComparer<T>}} that implement the
> structural equality behavior for each type of collection. These types should
> be passed in through each collection's constructor, as each has a {{Default}}
> and an {{Aggressive}} implementation.
> {{Default}} is more efficient and should be used in all closed scenarios
> within Lucene.NET (where the collection is not passed in from outside of the
> class). {{Aggressive}} is designed to interoperate with built-in .NET
> collections.
> See [these minimal examples of
> implementations|https://github.com/NightOwl888/J2N/tree/a50d41668cf459f3fe827dff5f1746422155c4f6/src/J2N.TestFramework/Util].
> h3. *Collection Formatting*
> J2N has a {{StringFormatter}} class that can be used to format collections,
> as follows.
> {code:c#}
> // IFormattable
> public string ToString(IFormatProvider provider)
> {
> return string.Format(provider, "{0}", this);
> }
> public override string ToString()
> {
> return ToString(StringFormatter.CurrentCulture);
> }
> {code}
> h3. *Nullable Keys*
> Use the attached {{NullableKey<T>}} and {{NullableKeyDictionary<TKey,
> TValue>}} implementations. {{NullableKeyDictionary<TKey, TValue>}} is
> intended to be a base class for all nullable key dictionary implementations.
> The backing dictionary (the one we are trying to mimic) is then passed in
> through its constructor.
> h3. *Integration*
> After the collections have been implemented, we can eliminate the usage of
> {{Lucene.Net.Support.Collections.Equals()}},
> {{Lucene.Net.Support.Collections.GetHashCode(), and
> {{Lucene.Net.Support.Collections.ToString()}}.
> Usage in of {{Lucene.Net.Support.Collections.Equals()}} and
> {{Lucene.Net.Support.Collections.GetHashCode() in Lucene.Net can be changed
> to the equality comparer matching the collection type.
> For string formatting, the usage can simply be changed to
> {{collection.ToString()}} in production code, and we can use in
> {{collection.ToString(J2N.Text.StringFormatter.InvariantCulture)}} to patch
> tests.
> For types where collections are passed from the outside that may contain
> nested collections, we should either use {{Aggressive}} structural
> comparison, or document that if nested collections are needed, the end user
> should use collections that implement {{IStructuralEquatable}}, such as those
> from J2N.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
