Hi,
Thanks for looking into this! Let me try to help by clarifying some of
the questions that you've raised:
- I think it's fine to break pyfuse3 backwards compatibility, as long
as we are explicit about it and increment the major version
number. That is what it's there for.
- I would rather not break backwards compatibility in S3QL, since this
means that we have to write upgrade code and everyone has to upgrade
their filesystem.
- I do not think S3QL needs to support nanosecond resolution (the
VFS does not expect this from filesystems in general
either). However, we should do better than seconds.
- Currently, S3QL stores data as an int64 of nanoseconds (in both
Python and SQLite). I think this should be fine - in 64 bits we can
store 9223372036854775808 ns which is more than 292 years (starting
at 1970).
- pyfuse3 is not intended to use floating point values at all (see
e.g. http://www.rath.org/pyfuse3-docs/data.html#pyfuse3.EntryAttributes). My
best guess is that code that uses "1e9" should read "1 000 000 000"
and that I (or whoever wrote the code) forgot that 1e9 gives a float
rather than an integer.
All the best,
-Nikolaus
On Nov 23 2020, Grunthos wrote:
> I've added a bug to pyfuse3; the ability for me to consider a patch depends
> in very large part on what level of API compatibility you want to break!
>
> Since 64 bits is insufficient to represent a nano-second timestamp,
> something has to become incompatible. I think the choice is down to:
>
>- add numpy (or similar) to the pyfuse dependencies
>- remove the *_ns properties (or at least make them return a
>struct/object instead)
>
> On Monday, November 23, 2020 at 10:42:47 PM UTC+11 Grunthos wrote:
>
>> Further, if my analysis is correct, then it suggests anything that uses
>> the pyfuse composite (double/float) field may have breakage.
>>
>> On Monday, November 23, 2020 at 10:30:52 PM UTC+11 Grunthos wrote:
>>
>>> *definitely* looks like a rounding problem with using a double-precision
>>> value to represent seconds + ns since 1970. For a standard double, only 6
>>> decimal digits work...for ns, you need 9 places.
>>>
>>> Try this in python:
>>>
>>> >>> 157788.999
>>>
>>> <- this is the source of the problem.
>>>
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "s3ql" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to s3ql+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/s3ql/0a4a5cb4-bf43-4f72-bacd-69636d26d37en%40googlegroups.com.
--
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
»Time flies like an arrow, fruit flies like a Banana.«
--
You received this message because you are subscribed to the Google Groups
"s3ql" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to s3ql+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/s3ql/87h7pfds31.fsf%40vostro.rath.org.
