Re: ANN: jedi-time 0.1.4

[dimitris]

Hi Sean,

I'm back home and trying to understand/internalize this...Unfortunately, 
this kind of (flat & arg-less) navigation is not going to be very useful 
for the majority of java.time (datafied) objects. That is for two 
reasons... First of all the datafied maps I'm returning are nested. This 
means that for example to get to the `YearMonth` object, you would need 
to navigate to the [:year :month] path, and in the absence of `nav-in` 
this is somewhat awkward. Secondly, most of the interesting/useful 
conversions (in the context of date-times), almost always requires some 
sort of argument (e.g. `Instant` to `LocalDateTime`), and so if the last 
arg to `nav` has to be either nil (for missing keys), or match the 
actual value in the map, then there is no room left for arguments.


It is true that I'm probably trying to do too much with `nav`, but now 
that I'm understanding its purpose better, I get the feeling that it's 
not going to be as useful as I originally thought (in the context of 
this lib). Yes, I can pull all the clever stuff into distinct functions, 
but ultimately for `nav` to be useful I would have to either:


1. Change the datafied representation to something flat, OR

2. accept that navigating to pure data (via `get-in`) will be done with 
real paths (e.g. `[:year :month]`), whereas navigating to objects (via 
`nav`) will be done with bogus keys (e.g. `:month-of-year`).


As things stand (with my current nested representation), only LocalDate, 
LocalDateTime, OffsetDateTime & ZonedDateTime can have useful navigations:


- LocalDate => :week-day , :year-month

- LocalDateTime => :local-date, :local-time

- OffsetDateTime => :local-datetime, :instant

- ZonedDateTime => :offset-datetime, :local-datetime, :instant

That is pretty much it in terms of `nav`...

Does that make (more) sense?


Many thanks in advance...

Dimitris

ps:  Sean I can be on slack but with my work email


On 04/02/2020 05:18, Sean Corfield wrote:

You're misunderstanding me. I'll try again.

I'm not saying you can't navigate to keys that don't exist in the data 
-- but since there would be no corresponding value, the nav call would 
be (nav coll k nil) essentially.


If (get coll k) produces some value v, then (nav coll k v) will take 
you from the right side (pure data) to the left side (objects) to the 
object that "corresponds" to the equivalent navigation on the right 
(i.e., within the data).


object -> datafy -> pure data
pure data -> get etc -> new pure data
pure data -> nav -> new object "corresponding" to new pure data

On Mon, Feb 3, 2020 at 3:38 AM Dimitrios Jim Piliouras 
mailto:jimpil1...@gmail.com>> wrote:


This is what I've done but it contradicts what we said earlier...

If I navigate to some existing key and it gives me back a Java
object, then it means that the datafied representation had a key
pointing to non data!

I have read your blog post multiple times ;), but I think the
situation you're describing  with the foreign keys is rather unique...

The datafied datetime cannot possibly include all its possible
formats, nor all the possible alternatives - that would be
extremely wasteful and meaningless the way I see it.


Let's take an Instant as an example...it datafies to map of two
keys (:epoch, :second). Does it make sense to add a :format-iso
key in there pointing to a String? Is there any point navigating
to that key? Is there any point navigating to :epoch or :second?
The answer is no, right? Is there a point in navigating to
:zoned-datetime given a zone id? I would think yes...

On Mon, 3 Feb 2020, 04:47 Sean Corfield, mailto:s...@corfield.org>> wrote:

Think of it as a square:

You start with an object of some sort (left side) -> datafy ->
turns it into pure Clojure data (including metadata). (right side)

Given pure Clojure data, you can navigate through it with get
etc and you stay in the right side (pure data).

Given that pure Clojure data, you can navigate back to the
left hand wide with nav, mimicking how get etc work.

So datafy is L -> R, get is R -> R, nav is R -> L on a
"diagonal" that takes you back to the object world on the
left, corresponding to the place on the right that you'd get
to via get etc.

See if this blog post helps
https://corfield.org/blog/2018/12/03/datafy-nav/

On Sun, Feb 2, 2020 at 1:22 AM Dimitrios Jim Piliouras
mailto:jimpil1...@gmail.com>> wrote:

Hi Sean,

Admittedly, I’ve never used REBL, and I did struggle with
the shape and name of the `nav` arguments...

In particular I’m struggling to understand why would
anyone use `nav` to navigate to a key that already exists
in the map...Can’t we just use `get` or `get-in`?

You used the :format as an example, which works with nil,
:i

Re: ANN: jedi-time 0.1.4

[Sean Corfield]

Yes, I agree with all of that I think.

For nested navigation, consider that (get-in data [:year :month) is
equivalent to (get data :month (get data :year)) so you could nav one step
at a time. Calling nav (& then datafy) on the intermediate steps would just
bring you back to the data world at the same point as the inner (top-level)
get in that case.

nav-in would be a strange operation since it would need to call datafy
after each step to get the arguments needed for the next nav call. REBL
provides nav-> which does this behind the scenes while it is threading data
through the pipeline of nav operations (so there is a precedent).

Even with an equivalent to nav-in (or nav->) I think that using datafy/nav
on Java Time objects may be an incomplete mapping -- and probably somewhat
hard to work with. When you first posted, I was more focused on the
confusion using non-core datafy/nav would be and interop with REBL -- I
didn't look too deep into the _actual_ navigation you were proposing,
sorry.

On Sun, Feb 9, 2020 at 1:19 AM dimitris  wrote:

> Hi Sean,
>
> I'm back home and trying to understand/internalize this...Unfortunately,
> this kind of (flat & arg-less) navigation is not going to be very useful
> for the majority of java.time (datafied) objects. That is for two
> reasons... First of all the datafied maps I'm returning are nested. This
> means that for example to get to the `YearMonth` object, you would need to
> navigate to the [:year :month] path, and in the absence of `nav-in` this is
> somewhat awkward. Secondly, most of the interesting/useful conversions (in
> the context of date-times), almost always requires some sort of argument
> (e.g. `Instant` to `LocalDateTime`), and so if the last arg to `nav` has to
> be either nil (for missing keys), or match the actual value in the map,
> then there is no room left for arguments.
>
> It is true that I'm probably trying to do too much with `nav`, but now
> that I'm understanding its purpose better, I get the feeling that it's not
> going to be as useful as I originally thought (in the context of this lib).
> Yes, I can pull all the clever stuff into distinct functions, but
> ultimately for `nav` to be useful I would have to either:
>
> 1. Change the datafied representation to something flat, OR
>
> 2. accept that navigating to pure data (via `get-in`) will be done with
> real paths (e.g. `[:year :month]`), whereas navigating to objects (via
> `nav`) will be done with bogus keys (e.g. `:month-of-year`).
>
> As things stand (with my current nested representation), only LocalDate,
> LocalDateTime, OffsetDateTime & ZonedDateTime can have useful navigations:
>
> - LocalDate => :week-day , :year-month
>
> - LocalDateTime => :local-date, :local-time
>
> - OffsetDateTime => :local-datetime, :instant
>
> - ZonedDateTime => :offset-datetime, :local-datetime, :instant
>
> That is pretty much it in terms of `nav`...
>
> Does that make (more) sense?
>
>
> Many thanks in advance...
>
> Dimitris
>
> ps:  Sean I can be on slack but with my work email
>
>
> On 04/02/2020 05:18, Sean Corfield wrote:
>
> You're misunderstanding me. I'll try again.
>
> I'm not saying you can't navigate to keys that don't exist in the data --
> but since there would be no corresponding value, the nav call would be (nav
> coll k nil) essentially.
>
> If (get coll k) produces some value v, then (nav coll k v) will take you
> from the right side (pure data) to the left side (objects) to the object
> that "corresponds" to the equivalent navigation on the right (i.e., within
> the data).
>
> object -> datafy -> pure data
> pure data -> get etc -> new pure data
> pure data -> nav -> new object "corresponding" to new pure data
>
> On Mon, Feb 3, 2020 at 3:38 AM Dimitrios Jim Piliouras <
> jimpil1...@gmail.com> wrote:
>
>> This is what I've done but it contradicts what we said earlier...
>>
>> If I navigate to some existing key and it gives me back a Java object,
>> then it means that the datafied representation had a key pointing to non
>> data!
>>
>> I have read your blog post multiple times ;), but I think the situation
>> you're describing  with the foreign keys is rather unique...
>>
>> The datafied datetime cannot possibly include all its possible formats,
>> nor all the possible alternatives - that would be extremely wasteful and
>> meaningless the way I see it.
>>
>>
>> Let's take an Instant as an example...it datafies to map of two keys
>> (:epoch, :second). Does it make sense to add a :format-iso key in there
>> pointing to a String? Is there any point navigating to that key? Is there
>> any point navigating to :epoch or :second? The answer is no, right? Is
>> there a point in navigating to :zoned-datetime given a zone id? I would
>> think yes...
>>
>> On Mon, 3 Feb 2020, 04:47 Sean Corfield,  wrote:
>>
>>> Think of it as a square:
>>>
>>> You start with an object of some sort (left side) -> datafy -> turns it
>>> into pure Clojure data (including metadata). (right side)
>>>
>>> Given pur

Re: ANN: jedi-time 0.1.4

[dimitris]

Hi again Sean and folks,

I've had another stub at this, mostly by flattening the model but also 
by separating navigation from query/comparing/formatting etc. Most 
datafied representations are now navigable on their keys (returning 
either a Java object or some base value), and some on certain extra keys.


I believe this version will play much nicer with REBL, and all the 
wrapper style fns now live in `jedi-time.datafied` namespace and 
backed-by `jedi-time.protocols`. In other words, navigation is now 
purely for traversing the graph - everything else has its own protocol.


Unfortunately, now Instant can't really navigate to anything interesting 
(it falls back to data navigation via `get`).


I would be very interested in feedback on the new approach (git-sha: 
8e756ecb71bbfa0b081e00d71a21c47037f1eae4). If anything, it separates 
navigation from the other capabilities, and makes sure that there is 
always a navigation path to either upgrade (by making assumptions), or 
downgrade (by losing information) your datafied representation .


As always, thanks in advance...

Dimitris


On 09/02/2020 19:35, Sean Corfield wrote:

Yes, I agree with all of that I think.

For nested navigation, consider that (get-in data [:year :month) is 
equivalent to (get data :month (get data :year)) so you could nav one 
step at a time. Calling nav (& then datafy) on the intermediate steps 
would just bring you back to the data world at the same point as the 
inner (top-level) get in that case.


nav-in would be a strange operation since it would need to call datafy 
after each step to get the arguments needed for the next nav call. 
REBL provides nav-> which does this behind the scenes while it is 
threading data through the pipeline of nav operations (so there is a 
precedent).


Even with an equivalent to nav-in (or nav->) I think that using 
datafy/nav on Java Time objects may be an incomplete mapping -- and 
probably somewhat hard to work with. When you first posted, I was more 
focused on the confusion using non-core datafy/nav would be and 
interop with REBL -- I didn't look too deep into the _actual_ 
navigation you were proposing, sorry.


On Sun, Feb 9, 2020 at 1:19 AM dimitris > wrote:


Hi Sean,

I'm back home and trying to understand/internalize
this...Unfortunately, this kind of (flat & arg-less) navigation is
not going to be very useful for the majority of java.time
(datafied) objects. That is for two reasons... First of all the
datafied maps I'm returning are nested. This means that for
example to get to the `YearMonth` object, you would need to
navigate to the [:year :month] path, and in the absence of
`nav-in` this is somewhat awkward. Secondly, most of the
interesting/useful conversions (in the context of date-times),
almost always requires some sort of argument (e.g. `Instant` to
`LocalDateTime`), and so if the last arg to `nav` has to be either
nil (for missing keys), or match the actual value in the map, then
there is no room left for arguments.

It is true that I'm probably trying to do too much with `nav`, but
now that I'm understanding its purpose better, I get the feeling
that it's not going to be as useful as I originally thought (in
the context of this lib). Yes, I can pull all the clever stuff
into distinct functions, but ultimately for `nav` to be useful I
would have to either:

1. Change the datafied representation to something flat, OR

2. accept that navigating to pure data (via `get-in`) will be done
with real paths (e.g. `[:year :month]`), whereas navigating to
objects (via `nav`) will be done with bogus keys (e.g.
`:month-of-year`).

As things stand (with my current nested representation), only
LocalDate, LocalDateTime, OffsetDateTime & ZonedDateTime can have
useful navigations:

- LocalDate => :week-day , :year-month

- LocalDateTime => :local-date, :local-time

- OffsetDateTime => :local-datetime, :instant

- ZonedDateTime => :offset-datetime, :local-datetime, :instant

That is pretty much it in terms of `nav`...

Does that make (more) sense?


Many thanks in advance...

Dimitris

ps:  Sean I can be on slack but with my work email


On 04/02/2020 05:18, Sean Corfield wrote:

You're misunderstanding me. I'll try again.

I'm not saying you can't navigate to keys that don't exist in the
data -- but since there would be no corresponding value, the nav
call would be (nav coll k nil) essentially.

If (get coll k) produces some value v, then (nav coll k v) will
take you from the right side (pure data) to the left side
(objects) to the object that "corresponds" to the equivalent
navigation on the right (i.e., within the data).

object -> datafy -> pure data
pure data -> get etc -> new pure data
pure data -> nav -> new object "corresponding" to new pure data

On Mon, Fe

Re: ANN: jedi-time 0.1.4

[Sean Corfield]

That is starting to work nicely with REBL for the basic datafy/nav
functionality. Thank you!

Once I'd verified that, I tried this: (assoc (d/datafy
(java.time.LocalDateTime/now)) :format :iso) hoping that if I nav'd to the
new :format key, it would "navigate" to a formatted version but it didn't.
Then I realized you hadn't added support for that in your nav
implementation (you're assuming folks explicitly use your protocol-based
functions on the original objects, I think?). That would be the next
logical step: being able to augment a datafied date/time with additional
key/value pairs that would be recognized by the implementation of nav. The
datafication wouldn't need to add these keys (although, if you wanted an
obvious "default" behavior, you could add some) but the navigation would
need to recognize them and do the appropriate calculation/conversion. Does
that make sense?

On Sun, Feb 9, 2020 at 1:46 PM dimitris  wrote:

> Hi again Sean and folks,
>
> I've had another stub at this, mostly by flattening the model but also by
> separating navigation from query/comparing/formatting etc. Most datafied
> representations are now navigable on their keys (returning either a Java
> object or some base value), and some on certain extra keys.
>
> I believe this version will play much nicer with REBL, and all the wrapper
> style fns now live in `jedi-time.datafied` namespace and backed-by
> `jedi-time.protocols`. In other words, navigation is now purely for
> traversing the graph - everything else has its own protocol.
>
> Unfortunately, now Instant can't really navigate to anything interesting
> (it falls back to data navigation via `get`).
>
> I would be very interested in feedback on the new approach (git-sha:
> 8e756ecb71bbfa0b081e00d71a21c47037f1eae4). If anything, it separates
> navigation from the other capabilities, and makes sure that there is always
> a navigation path to either upgrade (by making assumptions), or downgrade
> (by losing information) your datafied representation .
>
> As always, thanks in advance...
>
> Dimitris
>
>
> On 09/02/2020 19:35, Sean Corfield wrote:
>
> Yes, I agree with all of that I think.
>
> For nested navigation, consider that (get-in data [:year :month) is
> equivalent to (get data :month (get data :year)) so you could nav one step
> at a time. Calling nav (& then datafy) on the intermediate steps would just
> bring you back to the data world at the same point as the inner (top-level)
> get in that case.
>
> nav-in would be a strange operation since it would need to call datafy
> after each step to get the arguments needed for the next nav call. REBL
> provides nav-> which does this behind the scenes while it is threading data
> through the pipeline of nav operations (so there is a precedent).
>
> Even with an equivalent to nav-in (or nav->) I think that using datafy/nav
> on Java Time objects may be an incomplete mapping -- and probably somewhat
> hard to work with. When you first posted, I was more focused on the
> confusion using non-core datafy/nav would be and interop with REBL -- I
> didn't look too deep into the _actual_ navigation you were proposing,
> sorry.
>
> On Sun, Feb 9, 2020 at 1:19 AM dimitris  wrote:
>
>> Hi Sean,
>>
>> I'm back home and trying to understand/internalize this...Unfortunately,
>> this kind of (flat & arg-less) navigation is not going to be very useful
>> for the majority of java.time (datafied) objects. That is for two
>> reasons... First of all the datafied maps I'm returning are nested. This
>> means that for example to get to the `YearMonth` object, you would need to
>> navigate to the [:year :month] path, and in the absence of `nav-in` this is
>> somewhat awkward. Secondly, most of the interesting/useful conversions (in
>> the context of date-times), almost always requires some sort of argument
>> (e.g. `Instant` to `LocalDateTime`), and so if the last arg to `nav` has to
>> be either nil (for missing keys), or match the actual value in the map,
>> then there is no room left for arguments.
>>
>> It is true that I'm probably trying to do too much with `nav`, but now
>> that I'm understanding its purpose better, I get the feeling that it's not
>> going to be as useful as I originally thought (in the context of this lib).
>> Yes, I can pull all the clever stuff into distinct functions, but
>> ultimately for `nav` to be useful I would have to either:
>>
>> 1. Change the datafied representation to something flat, OR
>>
>> 2. accept that navigating to pure data (via `get-in`) will be done with
>> real paths (e.g. `[:year :month]`), whereas navigating to objects (via
>> `nav`) will be done with bogus keys (e.g. `:month-of-year`).
>>
>> As things stand (with my current nested representation), only LocalDate,
>> LocalDateTime, OffsetDateTime & ZonedDateTime can have useful navigations:
>>
>> - LocalDate => :week-day , :year-month
>>
>> - LocalDateTime => :local-date, :local-time
>>
>> - OffsetDateTime => :local-datetime, :instant
>>
>> - ZonedDa