Sorry, point of clarification - after the proposed change, will the
Import/Export buttons on the Traffic Ops UI still continue to work (but using a
different format)? Or will the Import/Export functionality be gone from the
Traffic Ops UI?
Thanks,
Mike Sandman
On 9/18/18, 9:33 AM, "Dave Neuman" <neu...@apache.org> wrote:
I am +1 on removing the old Export/Import in favor of using the API. I
don't think we need to add a new endpoint to support importing the old
version of the export just because someone somewhere might have an old
profile export that is not in Traffic Ops anymore and they need to import
it. I don't think we should put development effort into supporting
something that we don't actually know needs to be supported and just
because we can do something doesn't mean that we should.
On Mon, Sep 17, 2018 at 4:29 PM Robert Butts <r...@apache.org> wrote:
> >So actually we don't really need to add a new API endpoint at all
>
> Sure. I don't object to renaming, if people want to. But the existing one
> does work, yes.
>
> >Those endpoints should be sufficient enough to support the legacy format
> import/export
>
> There's no reason to support legacy export, just import. I'm 100% +1 on
> deprecating and removing the old export.
>
> >Going forward the import/export in TP should be done with the
> /api/*/profiles format (assuming we add parameters support to that API),
> with a separate button for "legacy" import which uses the deprecated
> /api/*/profiles/import format
>
> +1
>
> >But IMO there's not really a good reason to export profiles and save them
> outside of TO for long periods of time
>
> I personally don't have a use case either; but I'm sure someone does, or
> will.
>
>
> On Mon, Sep 17, 2018 at 4:11 PM Rawlin Peters <rawlin.pet...@gmail.com>
> wrote:
>
> > So actually we don't really need to add a new API endpoint at all if
> > my understanding is correct. We currently have
> > /api/*/profiles/:id/export and /api/*/profiles/import. Those endpoints
> > should be sufficient enough to support the legacy format import/export
> > without the need for new endpoints, right? The real issue is exposing
> > them via TP. Going forward the import/export in TP should be done with
> > the /api/*/profiles format (assuming we add parameters support to that
> > API), with a separate button for "legacy" import which uses the
> > deprecated /api/*/profiles/import format. But IMO there's not really a
> > good reason to export profiles and save them outside of TO for long
> > periods of time, so I think the whole import/export thing is really
> > meant for default profiles or for quickly exporting profiles from one
> > TO instance and importing into another. As long as the default
> > profiles are updated to the new format, I think that would solve most
> > of the problem for our users.
> >
> > - Rawlin
> >
> > On Mon, Sep 17, 2018 at 3:05 PM Robert Butts <r...@apache.org> wrote:
> > >
> > > >It just seems wrong to me to add a new API endpoint to support a
> > > deprecated format.
> > >
> > > So people who've exported their data are just out of luck? We just
> "don't
> > > support" migrating your data forward? It seems egregiously wrong to me
> > > _not_ to support importing old data, for any app or service, ever. If
> we
> > > gave people a way to create exported data, it's our job to continue to
> > > provide them a way to import it. Without them having to pay for
> > development
> > > time for an engineer to do a data conversion.
> > >
> > > >IMO if we do add a new endpoint just to support the deprecated legacy
> > > format, then it should also be marked "deprecated" for removal in the
> > next
> > > release.
> > >
> > > The entire point of supporting importing legacy data is _not_ to
> > deprecate
> > > it. This isn't about API versioning, and deprecating old undesirable
> > > protocols. That's absolutely the right way to progress the API itself.
> > This
> > > is about providing a way to move data forward. Users have exported
> files,
> > > from an export we gave them. It's the job of any well-behaved product
> to
> > > provide a mechanism to import old formats, when the format changes.
> > >
> > > How would you feel if you were using a product, say Google Docs, or
> > > Microsoft Excel, and they suddenly said, "Sorry, your old documents
> don't
> > > work in the latest version." "Oh, but we provide a command-line script
> to
> > > migrate them, you just have to open MS-DOS (or maybe Powershell?), and
> > run
> > > this script, with these arguments. On all your documents." Sure, as an
> > > engineer you could do that, however inconvenient; how would you feel
if
> > you
> > > read that and were an average user, who'd never seen a command line
> > before,
> > > and only ever used the GUI?
> > >
> > >
> > > On Mon, Sep 17, 2018 at 2:44 PM Rawlin Peters <rawlin.pet...@gmail.com
> >
> > > wrote:
> > >
> > > > It just seems wrong to me to add a new API endpoint to support a
> > > > deprecated format. There's much more involved in maintaining an API
> > > > endpoint as opposed to a one-off script to convert formats. IMO if
we
> > > > do add a new endpoint just to support the deprecated legacy format,
> > > > then it should also be marked "deprecated" for removal in the next
> > > > release.
> > > >
> > > > - Rawlin
> > > >
> > > > On Mon, Sep 17, 2018 at 1:51 PM Robert Butts <r...@apache.org>
wrote:
> > > > >
> > > > > >Do we really want to add a new API endpoint just to support the
> > legacy
> > > > > format
> > > > >
> > > > > An endpoint and UI page are easier to use for Self-Service CDN
> > consumers.
> > > > > Some CDN tenants might not know how to run a script or use a
> command
> > > > line.
> > > > > I'm not sure I understand the objection. Why is it such a big deal
> > to add
> > > > > an endpoint? And/or to support importing old formats? IMO
importing
> > old
> > > > > data formats should be a first-class citizen, in any API or UI.
> Data
> > > > jails
> > > > > are bad.
> > > > >
> > > > > >traffic_ops/app/bin/config2json
> > > > >
> > > > > The difference is that the TO config is used by system
> > administrators. As
> > > > > Self-Service moves forward, profiles and other API data will be
> used
> > by
> > > > CDN
> > > > > tenants, who may have very little technical knowledge. We need to
> > make
> > > > sure
> > > > > anything a Self-Service tenant may need to do has a simple
> graphical
> > way
> > > > to
> > > > > do it, and also an API so people deploying TC can write their own
> > UIs if
> > > > > necessary.
> > > > >
> > > > >
> > > > > On Mon, Sep 17, 2018 at 1:23 PM Rawlin Peters <
> > rawlin.pet...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Do we really want to add a new API endpoint just to support the
> > legacy
> > > > > > format? Couldn't we just provide a script that converts between
> the
> > > > > > two formats, similar to what we did for cdn.conf with
> > > > > > traffic_ops/app/bin/config2json?
> > > > > >
> > > > > > - Rawlin
> > > > > > On Mon, Sep 17, 2018 at 11:04 AM Robert Butts <r...@apache.org>
> > wrote:
> > > > > > >
> > > > > > > Ah, excellent. It wasn't clear from the docs that `POST
> > > > > > /api/1.2/profiles`
> > > > > > > and `GET /api/1.2/profiles/:id` accept parameters. We should
> fix
> > > > that.
> > > > > > >
> > > > > > > That said, I would be in favor of adding a new endpoint, to
> > import
> > > > in the
> > > > > > > old format, `/api/profile/import-legacy` or something. Just so
> > people
> > > > > > with
> > > > > > > old exports can import them back in. Data jails are bad (I
know
> > it's
> > > > > > > human-readable and not that difficult to convert, but it'd
> still
> > be
> > > > > > > frustrating).
> > > > > > >
> > > > > > > I made Github issues for both:
> > > > > > > https://github.com/apache/trafficcontrol/issues/2827
> > > > > > > https://github.com/apache/trafficcontrol/issues/2826
> > > > > > >
> > > > > > > On Mon, Sep 17, 2018 at 10:08 AM Dan Kirkwood <
> dang...@gmail.com
> > >
> > > > wrote:
> > > > > > >
> > > > > > > > yes -- that's really what I meant. Import/Export should
not
> > be a
> > > > > > separate
> > > > > > > > operation as documented here:
> > > > > > > >
> > > > > > > >
> > > > > >
> > > >
> >
>
https://traffic-control-cdn.readthedocs.io/en/latest/admin/traffic_ops/default_profiles.html?highlight=import
> > > > > > > > ,
> > > > > > > > but should be part of the "normal" API. The default profile
> > files
> > > > we
> > > > > > keep
> > > > > > > > in http://trafficcontrol.apache.org/downloads/profiles/ are
> > in the
> > > > > > format
> > > > > > > > I'm describing. They should be in the more standard form
> and
> > > > should
> > > > > > > > associate all parameters in the file with the profile.
> > > > > > > >
> > > > > > > > The Go `POST api/1.3/profiles` endpoint already loads the
> > > > parameters
> > > > > > into
> > > > > > > > memory, but ignores the parameters list.
> > > > > > > >
> > > > > > > > -dan
> > > > > > > >
> > > > > > > > On Mon, Sep 17, 2018 at 9:58 AM Jeremy Mitchell <
> > > > mitchell...@gmail.com
> > > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Maybe this is what Dan said but imo there should be no
> > > > import/export
> > > > > > > > > endpoints but these should do the job:
> > > > > > > > >
> > > > > > > > > GET /api/*/profiles/:id <-- this is essentially an export.
> it
> > > > gives
> > > > > > you
> > > > > > > > the
> > > > > > > > > json of a profile (along with all the parameters). save
the
> > json
> > > > to a
> > > > > > > > file
> > > > > > > > > and you've essentially exported the profile.
> > > > > > > > > POST /api/*/profiles <-- this is how you create profiles.
> > if you
> > > > > > supply
> > > > > > > > > parameters as well you've essentially done an import.
> > > > > > > > >
> > > > > > > > > jeremy
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Mon, Sep 17, 2018 at 9:16 AM Robert Butts <
> r...@apache.org
> > >
> > > > wrote:
> > > > > > > > >
> > > > > > > > > > Can you clarify, are you referring to `/profile/import`
> and
> > > > > > > > > > `/profile/:id/export`? Or
> > `/api/$version/profiles/:id/export`
> > > > and
> > > > > > > > > > `/api/$version/profiles/import`?
> > > > > > > > > >
> > > > > > > > > > We should definitely deprecate and remove the non-API
> > > > endpoints.
> > > > > > But
> > > > > > > > IMO
> > > > > > > > > we
> > > > > > > > > > need to keep a way to create and get an entire profile,
> > with
> > > > > > > > parameters,
> > > > > > > > > in
> > > > > > > > > > a single request. Users shouldn't have to make a dozen
> > > > requests to
> > > > > > > > import
> > > > > > > > > > and export a profile.
> > > > > > > > > >
> > > > > > > > > > Also +1 on changing them to real booleans (not sure if
> the
> > api
> > > > > > > > > > export/import is, they appear to be undocumented).
Though
> > we
> > > > can't
> > > > > > > > break
> > > > > > > > > > the current API; we could make the POST accept either,
> but
> > the
> > > > GET
> > > > > > > > would
> > > > > > > > > > have to be a new endpoint I think.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Sat, Sep 15, 2018 at 3:15 PM Dan Kirkwood <
> > > > dang...@apache.org>
> > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > I’d like to propose deprecating the import/export
> format
> > of
> > > > > > profiles.
> > > > > > > > > The
> > > > > > > > > > > current format (Perl-based) is inconsistent with the
> > standard
> > > > > > POST
> > > > > > > > > > > api/x.x/profiles format. Import/Export can/should be
> done
> > > > using
> > > > > > the
> > > > > > > > > same
> > > > > > > > > > > API. Traffic Portal Import should use the standard
> API
> > > > > > endpoint.
> > > > > > > > > > >
> > > > > > > > > > > Import/export (shown below) has this form which
> includes
> > the
> > > > > > > > "profile"
> > > > > > > > > > key
> > > > > > > > > > > at the top level. The "secure" option in the
> > "parameters"
> > > > secion
> > > > > > > > uses
> > > > > > > > > a
> > > > > > > > > > > 0/1 rather than a boolean true/false. These 2 things
> > make it
> > > > > > > > > > inconsistent.
> > > > > > > > > > >
> > > > > > > > > > > Opinions are welcome...
> > > > > > > > > > >
> > > > > > > > > > > Dan
> > > > > > > > > > >
> > > > > > > > > > > {
> > > > > > > > > > > "profile": {
> > > > > > > > > > > "name": "myname",
> > > > > > > > > > > ...
> > > > > > > > > > > },
> > > > > > > > > > > "parameters": [
> > > > > > > > > > > {
> > > > > > > > > > > "name": "foo",
> > > > > > > > > > > "secure": 0,
> > > > > > > > > > > ...
> > > > > > > > > > > ]
> > > > > > > > > > > }
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > >
> > > >
> >
>
