...you'll only see the bar.jpg job... no record of the job is left...
Well there's also the changelog. I think right now I'm only adding a
message that says (this job was changed), but that could easily include
_what_ the change was. Same for deletes.
So, you basically only have 60 seconds max to edit/delete a job
That's only for it to have _no_ impact. I checked in a production
database and found that the most common runtime for a job is 178 hours,
and the vast majority are at least 48. You effectively have the entire
runtime of a job to "fix" it if need be, but earlier is better. In the
current system, for 178 hours your DS is just hosed, no do-overs and no
emergency stop.
On 7/31/19 9:54 AM, Jeremy Mitchell wrote:
Yeah PUT seems like it could cause problems. Think of this scenario:
1. Bob creates an invalidation request like this:
https://my.origin.com/foo.jpg
10 minutes pass and foo.jpg has been completely "revalidated" across the
entire cdn (every mid has gone back to the origin to get a new copy of
foo.jpg)
2. on minute 15, bob realizes that he got it wrong and edits that job to
https://my.origin.com/bar.jpg
bob really created 2 jobs because both foo.jpg and bar.jpg were
"revalidated" but if you look in https://tp.domain.com/#!/jobs
<https://tp.staging.cdnlab.comcast.net/#!/jobs>, you'll only see the
bar.jpg job and people will scratch their heads "hmm, why was foo.jpg
refetched from the origin? weird. don't see any revalidation job for
it..."
and actually, delete seems like it could cause problems too. you create a
job, it completes, then you delete it. no record of the job is left.
sadly, there is currently no concept of when a job is "done". but
really to
me, once that revalidation job is pushed to even 1 cache, it is done and
after that the job should be immutable. the damage is already done. no do
overs.
another thing to think about: currently caches check the reval_pending
flag
every 60 seconds. So, you basically only have 60 seconds max to
edit/delete
a job. maybe we should rethink that 60 seconds to give the user more time
to rethink jobs?
jeremy
On Tue, Jul 30, 2019 at 4:49 PM Rawlin Peters <[email protected]>
wrote:
In my opinion, introducing PATCH methods seems like unnecessary
complexity. We don't really have a good way in TO-Go to support
partial object updates in a holistic manner today, and there are some
difficulties around determining which fields were actually sent by a
client with a null value (e.g. `"foo": null`) vs fields that were
entirely omitted by the client. It would also add to the burden of
testing and maintenance (when a simple PUT implementation would
suffice), and I don't think there's a great way for the TO Go client
to marshal a lib/go-tc struct into a PATCH request that only contains
the fields you'd like to update (sometimes with null/empty values).
As for PUT, I think we could get by with a POST and a DELETE without a
PUT for this particular endpoint, but I'm not sure I really feel
strongly about that. Providing the ability to PUT kind of encourages
the idea that you don't really have to get your invalidations right
the first time, or that you can just update an existing invalidation
job to change the regex instead of creating a new invalidation with a
different regex (when really they should be created as separate jobs).
If you have a bad revalidation deployed, your first priority should
probably be to get rid of it as quickly as possible (via DELETE)
instead of trying to replace it with a different regex (via PUT). In
that case, I'd think it would be advantageous to only provide the
DELETE option instead of both DELETE and PUT. First delete the bad
invalidation ASAP, then work on a better regex.
- Rawlin
On Tue, Jul 30, 2019 at 10:31 AM ocket8888 <[email protected]> wrote:
I have had this PR open for a while:
https://github.com/apache/trafficcontrol/pull/3744
I meant to bring this to the mailing list earlier, but I forgot :P
The reason this merits discussion is that the PR adds several method
handlers to the /jobs endpoint that didn't exist in Perl:
- POST
lets users create new jobs directly at this endpoint. My hope is
that the /user/current/jobs endpoint will fall into disuse, and we can
consolidate some functionality in one place. Obviously, this
necessitates a CDN-wide queue of reval updates.
- PUT
allows jobs to be replaced. This queues reval updates CDN-wide.
- PATCH
allows jobs to be edited. This also queues reval updates CDN-wide
- DELETE
deletes jobs. This, too, queues reval updates CDN-wide
Which I think is a good idea. Without any way to mutate created jobs, a
typo can have dire consequences that can't be taken back. But since
POST->DELETE->POST is really just editing with more steps, a PUT/PATCH
seemed to make sense.
thoughts?
