[jira] [Commented] (SPARK-2354) BitSet Range Expanded when creating new one
[
https://issues.apache.org/jira/browse/SPARK-2354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14061920#comment-14061920
]
Sean Owen commented on SPARK-2354:
--
I think it should be closed if there is no existing bug or code change.
> BitSet Range Expanded when creating new one
> ---
>
> Key: SPARK-2354
> URL: https://issues.apache.org/jira/browse/SPARK-2354
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
>Affects Versions: 1.0.0, 1.1.0
>Reporter: Yijie Shen
>Priority: Minor
>
> BitSet has a constructor parameter named "numBits: Int" and indicate the bit
> num inside.
> And also, there is a function called "capacity" which represents the long
> words number to hold the bits.
> When creating new BitSet,for example in '|', I thought the new created one
> shouldn't be the size of longer words' length, instead, it should be the
> longer set's num of bit
> {code}def |(other: BitSet): BitSet = {
> val newBS = new BitSet(math.max(numBits, other.numBits))
> // I know by now the numBits isn't a field
> {code}
> Does it have any other reason to expand the BitSet range I don't know?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (SPARK-2354) BitSet Range Expanded when creating new one
[
https://issues.apache.org/jira/browse/SPARK-2354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14061787#comment-14061787
]
Yijie Shen commented on SPARK-2354:
---
No issue yet, I just think it's a minor bug, as the Priority tag suggested.
> BitSet Range Expanded when creating new one
> ---
>
> Key: SPARK-2354
> URL: https://issues.apache.org/jira/browse/SPARK-2354
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
>Affects Versions: 1.0.0, 1.1.0
>Reporter: Yijie Shen
>Priority: Minor
>
> BitSet has a constructor parameter named "numBits: Int" and indicate the bit
> num inside.
> And also, there is a function called "capacity" which represents the long
> words number to hold the bits.
> When creating new BitSet,for example in '|', I thought the new created one
> shouldn't be the size of longer words' length, instead, it should be the
> longer set's num of bit
> {code}def |(other: BitSet): BitSet = {
> val newBS = new BitSet(math.max(numBits, other.numBits))
> // I know by now the numBits isn't a field
> {code}
> Does it have any other reason to expand the BitSet range I don't know?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (SPARK-2354) BitSet Range Expanded when creating new one
[
https://issues.apache.org/jira/browse/SPARK-2354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060521#comment-14060521
]
Sean Owen commented on SPARK-2354:
--
Yes, you could not just naively flip all the bits to make a new complement
BitSet. These methods don't exist, right? And the resulting size of the new
BitSet would still be the same. If you're just saying these methods in the
future would need to be implemented correctly, that's true. Is there an issue
to report on current code?
> BitSet Range Expanded when creating new one
> ---
>
> Key: SPARK-2354
> URL: https://issues.apache.org/jira/browse/SPARK-2354
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
>Affects Versions: 1.0.0, 1.1.0
>Reporter: Yijie Shen
>Priority: Minor
>
> BitSet has a constructor parameter named "numBits: Int" and indicate the bit
> num inside.
> And also, there is a function called "capacity" which represents the long
> words number to hold the bits.
> When creating new BitSet,for example in '|', I thought the new created one
> shouldn't be the size of longer words' length, instead, it should be the
> longer set's num of bit
> {code}def |(other: BitSet): BitSet = {
> val newBS = new BitSet(math.max(numBits, other.numBits))
> // I know by now the numBits isn't a field
> {code}
> Does it have any other reason to expand the BitSet range I don't know?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (SPARK-2354) BitSet Range Expanded when creating new one
[
https://issues.apache.org/jira/browse/SPARK-2354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060279#comment-14060279
]
Yijie Shen commented on SPARK-2354:
---
For the methods available in BitSet, they have the same effect.
But what if I want to implement a `complement` or `xnor` method? Since the
Iterator's hasNext method only check for nextSetBit's index is >= 0, when
iterating the complement bitset, indexes out of range will return, between the
numBits and capacity.
> BitSet Range Expanded when creating new one
> ---
>
> Key: SPARK-2354
> URL: https://issues.apache.org/jira/browse/SPARK-2354
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
>Affects Versions: 1.0.0, 1.1.0
>Reporter: Yijie Shen
>Priority: Minor
>
> BitSet has a constructor parameter named "numBits: Int" and indicate the bit
> num inside.
> And also, there is a function called "capacity" which represents the long
> words number to hold the bits.
> When creating new BitSet,for example in '|', I thought the new created one
> shouldn't be the size of longer words' length, instead, it should be the
> longer set's num of bit
> {code}def |(other: BitSet): BitSet = {
> val newBS = new BitSet(math.max(numBits, other.numBits))
> // I know by now the numBits isn't a field
> {code}
> Does it have any other reason to expand the BitSet range I don't know?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (SPARK-2354) BitSet Range Expanded when creating new one
[
https://issues.apache.org/jira/browse/SPARK-2354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060156#comment-14060156
]
Sean Owen commented on SPARK-2354:
--
These end up with the same effect.
Let's say A is created with numBits=50 and B is created with numBits=70. A will
have a capacity of 64 and B will have a capacity of 128, since they internally
allocate 1 and 2 longs of storage, respectively.
A|B needs to accommodate at least 70 bits, yes. Whether it is created with
numBits=70 (your suggestion) or numBits=128 (the current code), you end up with
a capacity of 128.
Nothing is being expanded needlessly; the result is the same.
I think the current code is fine.
> BitSet Range Expanded when creating new one
> ---
>
> Key: SPARK-2354
> URL: https://issues.apache.org/jira/browse/SPARK-2354
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
>Affects Versions: 1.0.0, 1.1.0
>Reporter: Yijie Shen
>Priority: Minor
>
> BitSet has a constructor parameter named "numBits: Int" and indicate the bit
> num inside.
> And also, there is a function called "capacity" which represents the long
> words number to hold the bits.
> When creating new BitSet,for example in '|', I thought the new created one
> shouldn't be the size of longer words' length, instead, it should be the
> longer set's num of bit
> {code}def |(other: BitSet): BitSet = {
> val newBS = new BitSet(math.max(numBits, other.numBits))
> // I know by now the numBits isn't a field
> {code}
> Does it have any other reason to expand the BitSet range I don't know?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
