[jira] [Commented] (ARROW-3613) [Go] Resize does not correctly update the length
[
https://issues.apache.org/jira/browse/ARROW-3613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16702469#comment-16702469
]
Francois Saint-Jacques commented on ARROW-3613:
---
[~alexandreyc] see https://github.com/apache/arrow/pull/3048
> [Go] Resize does not correctly update the length
>
>
> Key: ARROW-3613
> URL: https://issues.apache.org/jira/browse/ARROW-3613
> Project: Apache Arrow
> Issue Type: Bug
> Components: Go
>Reporter: Jonathan A Sternberg
>Assignee: Francois Saint-Jacques
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.12.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> If you have the following code:
> {code:java}
> package main
> import (
> "fmt"
> "github.com/apache/arrow/go/arrow/array"
> "github.com/apache/arrow/go/arrow/memory"
> )
> func main() {
> builder := array.NewFloat64Builder(memory.DefaultAllocator)
> fmt.Println(builder.Len(), builder.Cap())
> builder.Reserve(44)
> fmt.Println(builder.Len(), builder.Cap())
> builder.Resize(5)
> fmt.Println(builder.Len(), builder.Cap())
> builder.Reserve(44)
> for i := 0; i < 44; i++ {
> builder.Append(0)
> }
> fmt.Println(builder.Len(), builder.Cap())
> builder.Resize(5)
> fmt.Println(builder.Len(), builder.Cap())
> }
> {code}
> It gives the following output:
> {code:java}
> 0 0
> 0 64
> 0 32
> 44 64
> 44 32
> {code}
> For whatever reason, the length is not recorded as 5. I understand why the
> capacity might not be 5, but it does seem like the length should be set to 5
> if the array is resized to a length smaller than its current capacity.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-3613) [Go] Resize does not correctly update the length
[
https://issues.apache.org/jira/browse/ARROW-3613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16702418#comment-16702418
]
Alexandre Crayssac commented on ARROW-3613:
---
Good catch!
I wiil look at it but currently I don't have a "simple" solution.
> [Go] Resize does not correctly update the length
>
>
> Key: ARROW-3613
> URL: https://issues.apache.org/jira/browse/ARROW-3613
> Project: Apache Arrow
> Issue Type: Bug
> Components: Go
>Reporter: Jonathan A Sternberg
>Assignee: Francois Saint-Jacques
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.12.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> If you have the following code:
> {code:java}
> package main
> import (
> "fmt"
> "github.com/apache/arrow/go/arrow/array"
> "github.com/apache/arrow/go/arrow/memory"
> )
> func main() {
> builder := array.NewFloat64Builder(memory.DefaultAllocator)
> fmt.Println(builder.Len(), builder.Cap())
> builder.Reserve(44)
> fmt.Println(builder.Len(), builder.Cap())
> builder.Resize(5)
> fmt.Println(builder.Len(), builder.Cap())
> builder.Reserve(44)
> for i := 0; i < 44; i++ {
> builder.Append(0)
> }
> fmt.Println(builder.Len(), builder.Cap())
> builder.Resize(5)
> fmt.Println(builder.Len(), builder.Cap())
> }
> {code}
> It gives the following output:
> {code:java}
> 0 0
> 0 64
> 0 32
> 44 64
> 44 32
> {code}
> For whatever reason, the length is not recorded as 5. I understand why the
> capacity might not be 5, but it does seem like the length should be set to 5
> if the array is resized to a length smaller than its current capacity.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-3613) [Go] Resize does not correctly update the length
[
https://issues.apache.org/jira/browse/ARROW-3613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16702391#comment-16702391
]
Wes McKinney commented on ARROW-3613:
-
Reopen?
> [Go] Resize does not correctly update the length
>
>
> Key: ARROW-3613
> URL: https://issues.apache.org/jira/browse/ARROW-3613
> Project: Apache Arrow
> Issue Type: Bug
> Components: Go
>Reporter: Jonathan A Sternberg
>Assignee: Jonathan A Sternberg
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.12.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> If you have the following code:
> {code:java}
> package main
> import (
> "fmt"
> "github.com/apache/arrow/go/arrow/array"
> "github.com/apache/arrow/go/arrow/memory"
> )
> func main() {
> builder := array.NewFloat64Builder(memory.DefaultAllocator)
> fmt.Println(builder.Len(), builder.Cap())
> builder.Reserve(44)
> fmt.Println(builder.Len(), builder.Cap())
> builder.Resize(5)
> fmt.Println(builder.Len(), builder.Cap())
> builder.Reserve(44)
> for i := 0; i < 44; i++ {
> builder.Append(0)
> }
> fmt.Println(builder.Len(), builder.Cap())
> builder.Resize(5)
> fmt.Println(builder.Len(), builder.Cap())
> }
> {code}
> It gives the following output:
> {code:java}
> 0 0
> 0 64
> 0 32
> 44 64
> 44 32
> {code}
> For whatever reason, the length is not recorded as 5. I understand why the
> capacity might not be 5, but it does seem like the length should be set to 5
> if the array is resized to a length smaller than its current capacity.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-3613) [Go] Resize does not correctly update the length
[
https://issues.apache.org/jira/browse/ARROW-3613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16702356#comment-16702356
]
Francois Saint-Jacques commented on ARROW-3613:
---
Seems like this issue is fully solved as null counts is not properly updated.
> [Go] Resize does not correctly update the length
>
>
> Key: ARROW-3613
> URL: https://issues.apache.org/jira/browse/ARROW-3613
> Project: Apache Arrow
> Issue Type: Bug
> Components: Go
>Reporter: Jonathan A Sternberg
>Assignee: Jonathan A Sternberg
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.12.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> If you have the following code:
> {code:java}
> package main
> import (
> "fmt"
> "github.com/apache/arrow/go/arrow/array"
> "github.com/apache/arrow/go/arrow/memory"
> )
> func main() {
> builder := array.NewFloat64Builder(memory.DefaultAllocator)
> fmt.Println(builder.Len(), builder.Cap())
> builder.Reserve(44)
> fmt.Println(builder.Len(), builder.Cap())
> builder.Resize(5)
> fmt.Println(builder.Len(), builder.Cap())
> builder.Reserve(44)
> for i := 0; i < 44; i++ {
> builder.Append(0)
> }
> fmt.Println(builder.Len(), builder.Cap())
> builder.Resize(5)
> fmt.Println(builder.Len(), builder.Cap())
> }
> {code}
> It gives the following output:
> {code:java}
> 0 0
> 0 64
> 0 32
> 44 64
> 44 32
> {code}
> For whatever reason, the length is not recorded as 5. I understand why the
> capacity might not be 5, but it does seem like the length should be set to 5
> if the array is resized to a length smaller than its current capacity.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-3613) [Go] Resize does not correctly update the length
[
https://issues.apache.org/jira/browse/ARROW-3613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681405#comment-16681405
]
Alexandre Crayssac commented on ARROW-3613:
---
Just submitted a PR : [https://github.com/apache/arrow/pull/2927]
Need review though.
> [Go] Resize does not correctly update the length
>
>
> Key: ARROW-3613
> URL: https://issues.apache.org/jira/browse/ARROW-3613
> Project: Apache Arrow
> Issue Type: Bug
> Components: Go
>Reporter: Jonathan A Sternberg
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> If you have the following code:
> {code:java}
> package main
> import (
> "fmt"
> "github.com/apache/arrow/go/arrow/array"
> "github.com/apache/arrow/go/arrow/memory"
> )
> func main() {
> builder := array.NewFloat64Builder(memory.DefaultAllocator)
> fmt.Println(builder.Len(), builder.Cap())
> builder.Reserve(44)
> fmt.Println(builder.Len(), builder.Cap())
> builder.Resize(5)
> fmt.Println(builder.Len(), builder.Cap())
> builder.Reserve(44)
> for i := 0; i < 44; i++ {
> builder.Append(0)
> }
> fmt.Println(builder.Len(), builder.Cap())
> builder.Resize(5)
> fmt.Println(builder.Len(), builder.Cap())
> }
> {code}
> It gives the following output:
> {code:java}
> 0 0
> 0 64
> 0 32
> 44 64
> 44 32
> {code}
> For whatever reason, the length is not recorded as 5. I understand why the
> capacity might not be 5, but it does seem like the length should be set to 5
> if the array is resized to a length smaller than its current capacity.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-3613) [Go] Resize does not correctly update the length
[
https://issues.apache.org/jira/browse/ARROW-3613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681230#comment-16681230
]
Alexandre Crayssac commented on ARROW-3613:
---
Ok, still investigating on it since it looks like the bug has others
ramifications.
> [Go] Resize does not correctly update the length
>
>
> Key: ARROW-3613
> URL: https://issues.apache.org/jira/browse/ARROW-3613
> Project: Apache Arrow
> Issue Type: Bug
> Components: Go
>Reporter: Jonathan A Sternberg
>Priority: Major
>
> If you have the following code:
> {code:java}
> package main
> import (
> "fmt"
> "github.com/apache/arrow/go/arrow/array"
> "github.com/apache/arrow/go/arrow/memory"
> )
> func main() {
> builder := array.NewFloat64Builder(memory.DefaultAllocator)
> fmt.Println(builder.Len(), builder.Cap())
> builder.Reserve(44)
> fmt.Println(builder.Len(), builder.Cap())
> builder.Resize(5)
> fmt.Println(builder.Len(), builder.Cap())
> builder.Reserve(44)
> for i := 0; i < 44; i++ {
> builder.Append(0)
> }
> fmt.Println(builder.Len(), builder.Cap())
> builder.Resize(5)
> fmt.Println(builder.Len(), builder.Cap())
> }
> {code}
> It gives the following output:
> {code:java}
> 0 0
> 0 64
> 0 32
> 44 64
> 44 32
> {code}
> For whatever reason, the length is not recorded as 5. I understand why the
> capacity might not be 5, but it does seem like the length should be set to 5
> if the array is resized to a length smaller than its current capacity.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-3613) [Go] Resize does not correctly update the length
[
https://issues.apache.org/jira/browse/ARROW-3613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680398#comment-16680398
]
Wes McKinney commented on ARROW-3613:
-
Looks like a bug to me
> [Go] Resize does not correctly update the length
>
>
> Key: ARROW-3613
> URL: https://issues.apache.org/jira/browse/ARROW-3613
> Project: Apache Arrow
> Issue Type: Bug
> Components: Go
>Reporter: Jonathan A Sternberg
>Priority: Major
>
> If you have the following code:
> {code:java}
> package main
> import (
> "fmt"
> "github.com/apache/arrow/go/arrow/array"
> "github.com/apache/arrow/go/arrow/memory"
> )
> func main() {
> builder := array.NewFloat64Builder(memory.DefaultAllocator)
> fmt.Println(builder.Len(), builder.Cap())
> builder.Reserve(44)
> fmt.Println(builder.Len(), builder.Cap())
> builder.Resize(5)
> fmt.Println(builder.Len(), builder.Cap())
> builder.Reserve(44)
> for i := 0; i < 44; i++ {
> builder.Append(0)
> }
> fmt.Println(builder.Len(), builder.Cap())
> builder.Resize(5)
> fmt.Println(builder.Len(), builder.Cap())
> }
> {code}
> It gives the following output:
> {code:java}
> 0 0
> 0 64
> 0 32
> 44 64
> 44 32
> {code}
> For whatever reason, the length is not recorded as 5. I understand why the
> capacity might not be 5, but it does seem like the length should be set to 5
> if the array is resized to a length smaller than its current capacity.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-3613) [Go] Resize does not correctly update the length
[
https://issues.apache.org/jira/browse/ARROW-3613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680379#comment-16680379
]
Alexandre Crayssac commented on ARROW-3613:
---
Hello,
After digging into the code I confirm that the field `length` of
`Float64Builder.builder` is not updated when calls to `Float64Builder.Resize`
are made.
Note that it is also the case with all the numeric types since they are all
generated by a common template and they all use the `builder` field to retrieve
the length. By the way, should not we use the field `data` to retrieve the
lenght?
It looks like the problem is rooted in the function `resize` of the `builder`
field which does not update its `length` field. Note that the `capacity` field
is updated so I think we might end up with `length > capacity`. (Note that the
length of `data` is correctly updated.)
Can someone confirm that's a bug?
Disclaimer: I don't have tons of experience with Go and Arrow.
Alexandre
> [Go] Resize does not correctly update the length
>
>
> Key: ARROW-3613
> URL: https://issues.apache.org/jira/browse/ARROW-3613
> Project: Apache Arrow
> Issue Type: Bug
> Components: Go
>Reporter: Jonathan A Sternberg
>Priority: Major
>
> If you have the following code:
> {code:java}
> package main
> import (
> "fmt"
> "github.com/apache/arrow/go/arrow/array"
> "github.com/apache/arrow/go/arrow/memory"
> )
> func main() {
> builder := array.NewFloat64Builder(memory.DefaultAllocator)
> fmt.Println(builder.Len(), builder.Cap())
> builder.Reserve(44)
> fmt.Println(builder.Len(), builder.Cap())
> builder.Resize(5)
> fmt.Println(builder.Len(), builder.Cap())
> builder.Reserve(44)
> for i := 0; i < 44; i++ {
> builder.Append(0)
> }
> fmt.Println(builder.Len(), builder.Cap())
> builder.Resize(5)
> fmt.Println(builder.Len(), builder.Cap())
> }
> {code}
> It gives the following output:
> {code:java}
> 0 0
> 0 64
> 0 32
> 44 64
> 44 32
> {code}
> For whatever reason, the length is not recorded as 5. I understand why the
> capacity might not be 5, but it does seem like the length should be set to 5
> if the array is resized to a length smaller than its current capacity.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
