Re: Tags and object IDs

[William Reade]

On Mon, Jan 25, 2016 at 12:07 PM, Nate Finch 
wrote:

> I was really trying not to give too much information about this exact
> case, so we could avoid talking about a specific implementation, and focus
> on the more general question of how we identify objects.  Yes, we get the
> bytes using an HTTP request, but that is irrelevant to my question :)
>

I thought I did answer the question:

But whenever we do record the unit-X-uses-resource-Y info I assume we'll
>>> have much the same stuff available in the apiserver, in which case I think
>>> you just want to pass the *Unit back into state; without it, you just need
>>> to read the doc from the DB all over again to make appropriate
>>> liveness/existence checks [0], and why bother unless you've already hit an
>>> assertion failure in your first txn attempt?
>>>
>>
...but perhaps I misunderstood what you were looking for?

Cheers
William
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Tags and object IDs

[Nate Finch]

I was really trying not to give too much information about this exact case,
so we could avoid talking about a specific implementation, and focus on the
more general question of how we identify objects.  Yes, we get the bytes
using an HTTP request, but that is irrelevant to my question :)

On Mon, Jan 25, 2016 at 2:00 AM John Meinel  wrote:

> On Sat, Jan 23, 2016 at 1:28 AM, William Reade <
> william.re...@canonical.com> wrote:
>
>> On Fri, Jan 22, 2016 at 9:53 PM, Nate Finch 
>> wrote:
>>
>>> Working in the model layer on the server between the API and the DB.
>>> Specifically in my instance, an API call comes in from a unit, requesting
>>> the bytes for a resource.  We want to record that this unit is now using
>>> the bytes from that specific revision of the resource.  I have a pointer to
>>> a state.Unit, and a function that takes a Resource metadata object and some
>>> reference to the unit, and does the actual transaction to the DB to store
>>> the unit's ID and the resource information.
>>>
>>
>> I'm a bit surprised that we'd be transferring those bytes over an API
>> call in the first place (is json-over-websocket really a great way to send
>> potential gigabytes? shouldn't we be getting URL+SHA256 from the apiserver
>> as we do for charms, and downloading separately? and do we really want to
>> enforce charmstore == apiserver?); and I'd point out that merely having
>> agreed to deliver some bytes to a client is no indication that the client
>> will actually be using those bytes for anything; but we should probably
>> chat about those elsewhere, I'm evidently missing some context.
>>
>
> So I would have expected that we'd rather use a similar raw
> HTTP-to-get-content instead of a JSON request (given the intent of
> resources is that they may be GB in size), but regardless it is the intent
> that you download the bytes from the charm rather from the store directly.
> Similar to how we currently fetch the charm archive content itself.
> As for "will you be using it", the specific request from the charm is when
> it calls "resource-get" which is very specifically the time when the charm
> wants to go do something with those bytes.
>
> John
> =:->
>
>
>> But whenever we do record the unit-X-uses-resource-Y info I assume we'll
>> have much the same stuff available in the apiserver, in which case I think
>> you just want to pass the *Unit back into state; without it, you just need
>> to read the doc from the DB all over again to make appropriate
>> liveness/existence checks [0], and why bother unless you've already hit an
>> assertion failure in your first txn attempt?
>>
>> Cheers
>> William
>>
>> [0] I imagine you're not just dumping (unit, resource) pairs into the DB
>> without checking that they're sane? that's really not safe
>>
>>
>>> On Fri, Jan 22, 2016 at 3:34 PM William Reade <
>>> william.re...@canonical.com> wrote:
>>>
 Need a bit more context here. What layer are you working in?

 In general terms, entity references in the API *must* use tags; entity
 references that leak out to users *must not* use tags; otherwise it's a
 matter of judgment and convenience. In state code, it's annoying to use
 tags because we've already got the globalKey convention; in worker code
 it's often justifiable if not exactly awesome. See
 https://github.com/juju/juju/wiki/Managing-complexity#workers

 Cheers
 William

 On Fri, Jan 22, 2016 at 6:02 PM, Nate Finch 
 wrote:

> I have a function that is recording which unit is using a specific
> resource.  I wrote the function to take a UnitTag, because that's the
> closest thing we have to an ID type. However, I and others seem to 
> remember
> hearing that Tags are really only supposed to be used for the API. That
> leaves me with a problem - what can I pass to this function to indicate
> which unit I'm talking about?  I'd be fine passing a pointer to the unit
> object itself, but we're trying to avoid direct dependencies on state.
> People have suggested just passing a string (presumably
> unit.Tag().String()), but then my API is too lenient - it appears to say
> "give me any string you want for an id", but what it really means is "give
> me a serialized UnitTag".
>
> I think most places in the code just use a string for an ID, but this
> opens up the code to abuses and developer errors.
>
> Can someone explain why tags should only be used in the API? It seems
> like the perfect type to pass around to indicate the ID of a specific
> object.
>
> -Nate
>
> --
> Juju-dev mailing list
> Juju-dev@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>
>
>>
>> --
>> Juju-dev mailing list
>> Juju-dev@lists.ubuntu.com
>> Modify settings or unsubscribe at:
>> 

Re: Tags and object IDs

[John Meinel]

I think to William's point, we should have already authenticated the unit
as part of the API request, thus we should have a Unit object hanging
around somewhere close to where that request is being made, and can just
pass it into state.

John
=:->

On Mon, Jan 25, 2016 at 3:07 PM, Nate Finch 
wrote:

> I was really trying not to give too much information about this exact
> case, so we could avoid talking about a specific implementation, and focus
> on the more general question of how we identify objects.  Yes, we get the
> bytes using an HTTP request, but that is irrelevant to my question :)
>
> On Mon, Jan 25, 2016 at 2:00 AM John Meinel 
> wrote:
>
>> On Sat, Jan 23, 2016 at 1:28 AM, William Reade <
>> william.re...@canonical.com> wrote:
>>
>>> On Fri, Jan 22, 2016 at 9:53 PM, Nate Finch 
>>> wrote:
>>>
 Working in the model layer on the server between the API and the DB.
 Specifically in my instance, an API call comes in from a unit, requesting
 the bytes for a resource.  We want to record that this unit is now using
 the bytes from that specific revision of the resource.  I have a pointer to
 a state.Unit, and a function that takes a Resource metadata object and some
 reference to the unit, and does the actual transaction to the DB to store
 the unit's ID and the resource information.

>>>
>>> I'm a bit surprised that we'd be transferring those bytes over an API
>>> call in the first place (is json-over-websocket really a great way to send
>>> potential gigabytes? shouldn't we be getting URL+SHA256 from the apiserver
>>> as we do for charms, and downloading separately? and do we really want to
>>> enforce charmstore == apiserver?); and I'd point out that merely having
>>> agreed to deliver some bytes to a client is no indication that the client
>>> will actually be using those bytes for anything; but we should probably
>>> chat about those elsewhere, I'm evidently missing some context.
>>>
>>
>> So I would have expected that we'd rather use a similar raw
>> HTTP-to-get-content instead of a JSON request (given the intent of
>> resources is that they may be GB in size), but regardless it is the intent
>> that you download the bytes from the charm rather from the store directly.
>> Similar to how we currently fetch the charm archive content itself.
>> As for "will you be using it", the specific request from the charm is
>> when it calls "resource-get" which is very specifically the time when the
>> charm wants to go do something with those bytes.
>>
>> John
>> =:->
>>
>>
>>> But whenever we do record the unit-X-uses-resource-Y info I assume we'll
>>> have much the same stuff available in the apiserver, in which case I think
>>> you just want to pass the *Unit back into state; without it, you just need
>>> to read the doc from the DB all over again to make appropriate
>>> liveness/existence checks [0], and why bother unless you've already hit an
>>> assertion failure in your first txn attempt?
>>>
>>> Cheers
>>> William
>>>
>>> [0] I imagine you're not just dumping (unit, resource) pairs into the DB
>>> without checking that they're sane? that's really not safe
>>>
>>>
 On Fri, Jan 22, 2016 at 3:34 PM William Reade <
 william.re...@canonical.com> wrote:

> Need a bit more context here. What layer are you working in?
>
> In general terms, entity references in the API *must* use tags; entity
> references that leak out to users *must not* use tags; otherwise it's a
> matter of judgment and convenience. In state code, it's annoying to use
> tags because we've already got the globalKey convention; in worker code
> it's often justifiable if not exactly awesome. See
> https://github.com/juju/juju/wiki/Managing-complexity#workers
>
> Cheers
> William
>
> On Fri, Jan 22, 2016 at 6:02 PM, Nate Finch 
> wrote:
>
>> I have a function that is recording which unit is using a specific
>> resource.  I wrote the function to take a UnitTag, because that's the
>> closest thing we have to an ID type. However, I and others seem to 
>> remember
>> hearing that Tags are really only supposed to be used for the API. That
>> leaves me with a problem - what can I pass to this function to indicate
>> which unit I'm talking about?  I'd be fine passing a pointer to the unit
>> object itself, but we're trying to avoid direct dependencies on state.
>> People have suggested just passing a string (presumably
>> unit.Tag().String()), but then my API is too lenient - it appears to say
>> "give me any string you want for an id", but what it really means is 
>> "give
>> me a serialized UnitTag".
>>
>> I think most places in the code just use a string for an ID, but this
>> opens up the code to abuses and developer errors.
>>
>> Can someone explain why tags should only 

Re: Tags and object IDs

[John Meinel]

On Sat, Jan 23, 2016 at 1:28 AM, William Reade 
wrote:

> On Fri, Jan 22, 2016 at 9:53 PM, Nate Finch 
> wrote:
>
>> Working in the model layer on the server between the API and the DB.
>> Specifically in my instance, an API call comes in from a unit, requesting
>> the bytes for a resource.  We want to record that this unit is now using
>> the bytes from that specific revision of the resource.  I have a pointer to
>> a state.Unit, and a function that takes a Resource metadata object and some
>> reference to the unit, and does the actual transaction to the DB to store
>> the unit's ID and the resource information.
>>
>
> I'm a bit surprised that we'd be transferring those bytes over an API call
> in the first place (is json-over-websocket really a great way to send
> potential gigabytes? shouldn't we be getting URL+SHA256 from the apiserver
> as we do for charms, and downloading separately? and do we really want to
> enforce charmstore == apiserver?); and I'd point out that merely having
> agreed to deliver some bytes to a client is no indication that the client
> will actually be using those bytes for anything; but we should probably
> chat about those elsewhere, I'm evidently missing some context.
>

So I would have expected that we'd rather use a similar raw
HTTP-to-get-content instead of a JSON request (given the intent of
resources is that they may be GB in size), but regardless it is the intent
that you download the bytes from the charm rather from the store directly.
Similar to how we currently fetch the charm archive content itself.
As for "will you be using it", the specific request from the charm is when
it calls "resource-get" which is very specifically the time when the charm
wants to go do something with those bytes.

John
=:->


> But whenever we do record the unit-X-uses-resource-Y info I assume we'll
> have much the same stuff available in the apiserver, in which case I think
> you just want to pass the *Unit back into state; without it, you just need
> to read the doc from the DB all over again to make appropriate
> liveness/existence checks [0], and why bother unless you've already hit an
> assertion failure in your first txn attempt?
>
> Cheers
> William
>
> [0] I imagine you're not just dumping (unit, resource) pairs into the DB
> without checking that they're sane? that's really not safe
>
>
>> On Fri, Jan 22, 2016 at 3:34 PM William Reade <
>> william.re...@canonical.com> wrote:
>>
>>> Need a bit more context here. What layer are you working in?
>>>
>>> In general terms, entity references in the API *must* use tags; entity
>>> references that leak out to users *must not* use tags; otherwise it's a
>>> matter of judgment and convenience. In state code, it's annoying to use
>>> tags because we've already got the globalKey convention; in worker code
>>> it's often justifiable if not exactly awesome. See
>>> https://github.com/juju/juju/wiki/Managing-complexity#workers
>>>
>>> Cheers
>>> William
>>>
>>> On Fri, Jan 22, 2016 at 6:02 PM, Nate Finch 
>>> wrote:
>>>
 I have a function that is recording which unit is using a specific
 resource.  I wrote the function to take a UnitTag, because that's the
 closest thing we have to an ID type. However, I and others seem to remember
 hearing that Tags are really only supposed to be used for the API. That
 leaves me with a problem - what can I pass to this function to indicate
 which unit I'm talking about?  I'd be fine passing a pointer to the unit
 object itself, but we're trying to avoid direct dependencies on state.
 People have suggested just passing a string (presumably
 unit.Tag().String()), but then my API is too lenient - it appears to say
 "give me any string you want for an id", but what it really means is "give
 me a serialized UnitTag".

 I think most places in the code just use a string for an ID, but this
 opens up the code to abuses and developer errors.

 Can someone explain why tags should only be used in the API? It seems
 like the perfect type to pass around to indicate the ID of a specific
 object.

 -Nate

 --
 Juju-dev mailing list
 Juju-dev@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju-dev


>
> --
> Juju-dev mailing list
> Juju-dev@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>
>
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Tags and object IDs

[Nate Finch]

Working in the model layer on the server between the API and the DB.
Specifically in my instance, an API call comes in from a unit, requesting
the bytes for a resource.  We want to record that this unit is now using
the bytes from that specific revision of the resource.  I have a pointer to
a state.Unit, and a function that takes a Resource metadata object and some
reference to the unit, and does the actual transaction to the DB to store
the unit's ID and the resource information.

On Fri, Jan 22, 2016 at 3:34 PM William Reade 
wrote:

> Need a bit more context here. What layer are you working in?
>
> In general terms, entity references in the API *must* use tags; entity
> references that leak out to users *must not* use tags; otherwise it's a
> matter of judgment and convenience. In state code, it's annoying to use
> tags because we've already got the globalKey convention; in worker code
> it's often justifiable if not exactly awesome. See
> https://github.com/juju/juju/wiki/Managing-complexity#workers
>
> Cheers
> William
>
> On Fri, Jan 22, 2016 at 6:02 PM, Nate Finch 
> wrote:
>
>> I have a function that is recording which unit is using a specific
>> resource.  I wrote the function to take a UnitTag, because that's the
>> closest thing we have to an ID type. However, I and others seem to remember
>> hearing that Tags are really only supposed to be used for the API. That
>> leaves me with a problem - what can I pass to this function to indicate
>> which unit I'm talking about?  I'd be fine passing a pointer to the unit
>> object itself, but we're trying to avoid direct dependencies on state.
>> People have suggested just passing a string (presumably
>> unit.Tag().String()), but then my API is too lenient - it appears to say
>> "give me any string you want for an id", but what it really means is "give
>> me a serialized UnitTag".
>>
>> I think most places in the code just use a string for an ID, but this
>> opens up the code to abuses and developer errors.
>>
>> Can someone explain why tags should only be used in the API? It seems
>> like the perfect type to pass around to indicate the ID of a specific
>> object.
>>
>> -Nate
>>
>> --
>> Juju-dev mailing list
>> Juju-dev@lists.ubuntu.com
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>>
>>
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Tags and object IDs

[William Reade]

On Fri, Jan 22, 2016 at 10:28 PM, William Reade  wrote:
>
> [0] I imagine you're not just dumping (unit, resource) pairs into the DB
> without checking that they're sane? that's really not safe
>

(Specifically, you need to check that they're *still* sane whenever that
txn happens to apply. Just because they were sane at the time of the api
call is no indication of anything.)
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Tags and object IDs

[William Reade]

Need a bit more context here. What layer are you working in?

In general terms, entity references in the API *must* use tags; entity
references that leak out to users *must not* use tags; otherwise it's a
matter of judgment and convenience. In state code, it's annoying to use
tags because we've already got the globalKey convention; in worker code
it's often justifiable if not exactly awesome. See
https://github.com/juju/juju/wiki/Managing-complexity#workers

Cheers
William

On Fri, Jan 22, 2016 at 6:02 PM, Nate Finch 
wrote:

> I have a function that is recording which unit is using a specific
> resource.  I wrote the function to take a UnitTag, because that's the
> closest thing we have to an ID type. However, I and others seem to remember
> hearing that Tags are really only supposed to be used for the API. That
> leaves me with a problem - what can I pass to this function to indicate
> which unit I'm talking about?  I'd be fine passing a pointer to the unit
> object itself, but we're trying to avoid direct dependencies on state.
> People have suggested just passing a string (presumably
> unit.Tag().String()), but then my API is too lenient - it appears to say
> "give me any string you want for an id", but what it really means is "give
> me a serialized UnitTag".
>
> I think most places in the code just use a string for an ID, but this
> opens up the code to abuses and developer errors.
>
> Can someone explain why tags should only be used in the API? It seems like
> the perfect type to pass around to indicate the ID of a specific object.
>
> -Nate
>
> --
> Juju-dev mailing list
> Juju-dev@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>
>
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Tags and object IDs

[Nate Finch]

I have a function that is recording which unit is using a specific
resource.  I wrote the function to take a UnitTag, because that's the
closest thing we have to an ID type. However, I and others seem to remember
hearing that Tags are really only supposed to be used for the API. That
leaves me with a problem - what can I pass to this function to indicate
which unit I'm talking about?  I'd be fine passing a pointer to the unit
object itself, but we're trying to avoid direct dependencies on state.
People have suggested just passing a string (presumably
unit.Tag().String()), but then my API is too lenient - it appears to say
"give me any string you want for an id", but what it really means is "give
me a serialized UnitTag".

I think most places in the code just use a string for an ID, but this opens
up the code to abuses and developer errors.

Can someone explain why tags should only be used in the API? It seems like
the perfect type to pass around to indicate the ID of a specific object.

-Nate
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev