[jira] [Commented] (ARROW-16543) [JS] Timestamp types are all the same
[
https://issues.apache.org/jira/browse/ARROW-16543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552616#comment-17552616
]
Teodor Kostov commented on ARROW-16543:
---
Here is a full test:
{code:javascript}
const dataType = new arrow.Struct([
new arrow.Field('time', new arrow.TimestampSecond()),
new arrow.Field('value', new arrow.Float64()),
])
const builder = arrow.makeBuilder({ type: dataType, nullValues: [null,
undefined] })
const date = new Date()
const timestampSeconds = Math.floor(date.getTime() / 1000)
const timestamp = timestampSeconds * 1000
builder.append({ time: date, value: 1.2 })
builder.append({ time: date, value: 3.3 })
builder.finish()
const vector = builder.toVector()
const schema = new arrow.Schema(dataType.children)
const recordBatch = new arrow.RecordBatch(schema, vector.data[0])
const table = new arrow.Table(recordBatch)
console.log(timestamp)
console.log(timestampSeconds)
console.log(table.get(0).time)
console.log(table.get(0).time === timestamp) // should be false
console.log(table.get(0).time === timestampSeconds) // should be true
{code}
> [JS] Timestamp types are all the same
> -
>
> Key: ARROW-16543
> URL: https://issues.apache.org/jira/browse/ARROW-16543
> Project: Apache Arrow
> Issue Type: Bug
> Components: JavaScript
>Reporter: Teodor Kostov
>Priority: Major
>
> Current timestamp types are all the same. They have the same representation.
> And also the same precision.
> For example, {{TimestampSecond}} and {{TimestampMillisecond}} return the
> values as {{165211818}}. Instead, I would expect the {{TimestampSecond}}
> to drop the 3 zeros when returning a value, e.g. {{1652118180}}. Also, the
> representation underneath is still an {{int32}} array. Even though for
> {{TimestampSecond}} every second value is {{0}}, the array still has double
> the amount of integers.
> I also got an error when trying to read a {{Date}} as {{TimestampNanosecond}}
> - {{TypeError: can't convert 165211818 to BigInt}}.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (ARROW-16543) [JS] Timestamp types are all the same
[
https://issues.apache.org/jira/browse/ARROW-16543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17548109#comment-17548109
]
Teodor Kostov commented on ARROW-16543:
---
Hello [~paul.e.taylor]. I used a {{Struct}} type and a builder.
{code:javascript}
const myType = new arrow.Struct([
new arrow.Field('time', new arrow.TimestampSecond()),
new arrow.Field('value', new arrow.Float64()),
])
const myBuilder = arrow.makeBuilder({ type: myType, nullValues: [null,
undefined] })
...
myBuilder.append(...)
...
myBuilder.finish()
const vector = myBuilder.toVector()
{code}
I am using another library that is sensitive to the timestamp format, whether
in milliseconds or seconds. And for my application, I do not need the
millisecond accuracy. At the same time, I would expect the data not to need any
transformation when I get it from the table. I would also expect the table to
use the most efficient format to store my data.
> [JS] Timestamp types are all the same
> -
>
> Key: ARROW-16543
> URL: https://issues.apache.org/jira/browse/ARROW-16543
> Project: Apache Arrow
> Issue Type: Bug
> Components: JavaScript
>Reporter: Teodor Kostov
>Priority: Major
>
> Current timestamp types are all the same. They have the same representation.
> And also the same precision.
> For example, {{TimestampSecond}} and {{TimestampMillisecond}} return the
> values as {{165211818}}. Instead, I would expect the {{TimestampSecond}}
> to drop the 3 zeros when returning a value, e.g. {{1652118180}}. Also, the
> representation underneath is still an {{int32}} array. Even though for
> {{TimestampSecond}} every second value is {{0}}, the array still has double
> the amount of integers.
> I also got an error when trying to read a {{Date}} as {{TimestampNanosecond}}
> - {{TypeError: can't convert 165211818 to BigInt}}.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (ARROW-16543) [JS] Timestamp types are all the same
[
https://issues.apache.org/jira/browse/ARROW-16543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17545241#comment-17545241
]
Paul Taylor commented on ARROW-16543:
-
[~terusus] how did you construct the Timestamp Vectors? The semantic meaning of
the type is " since the epoch," so it's valid for the different
duration dtypes to have the same underlying representation.
> [JS] Timestamp types are all the same
> -
>
> Key: ARROW-16543
> URL: https://issues.apache.org/jira/browse/ARROW-16543
> Project: Apache Arrow
> Issue Type: Bug
> Components: JavaScript
>Reporter: Teodor Kostov
>Priority: Major
>
> Current timestamp types are all the same. They have the same representation.
> And also the same precision.
> For example, {{TimestampSecond}} and {{TimestampMillisecond}} return the
> values as {{165211818}}. Instead, I would expect the {{TimestampSecond}}
> to drop the 3 zeros when returning a value, e.g. {{1652118180}}. Also, the
> representation underneath is still an {{int32}} array. Even though for
> {{TimestampSecond}} every second value is {{0}}, the array still has double
> the amount of integers.
> I also got an error when trying to read a {{Date}} as {{TimestampNanosecond}}
> - {{TypeError: can't convert 165211818 to BigInt}}.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
