Re: [Mesa-dev] Moving amdgpu/addrlib into a git submodule

2016-08-13 Thread Mathias Fröhlich
Hi,

On Friday, 12 August 2016 12:15:52 CEST Nicolai Hähnle wrote:
> I take it this means `git bisect` becomes unusable? At least it looks 
> like it after a bit of playing around with it. That seems like a 
> show-stopper :(

git bisect skip does this job.
But it's pretty surprising if you step on that and dont' know and being even 
more far off to understand whats going on.
That's what I meant with 'needs special care'.

You may be able to get asistance for a hand driven process of merging by 
subtree merge together with producing patches that get applied with git am 
without that surprising second merge parent.
May be an addrlib upstream repository that generates the required mesa side
upstream patch files already with the changed file paths by hook scripts helps 
somehow? And for the other direction as well using hooks in the mesa upstream?
No automatic pushes, but at least nothing gets lost and diverges too far.
... just to trigger ideas.

best
Mathias

___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev


Re: [Mesa-dev] Moving amdgpu/addrlib into a git submodule

2016-08-12 Thread Nicolai Hähnle

Hi Mathias,

thanks for the input!

On 11.08.2016 06:49, Mathias Fröhlich wrote:

For subtree-merge:

There is a pair of visualisation teams that used to work for the same
company until they both got sold to individual vendors but still wanted
to share their basic framwork code. Those two applications done by these
teams are somewhere beyond - may be far beyond 1e6 loc each. The shared
part is at least 2e5 loc, probably more. I do not recall the exact
numbers. The shared part is communicated via a common upstream
repository managed by git-subtree. There happens branching merging,
rebasing, the usual git things. The git version that is used is ancient,
the one from redhat 6.

The git repository is based on an import from svn with non standard
branch names in svn and that svn used to be imported from cvs. All
together more than 10 years of history.

Observed problems: It needs special care to be handled. There were not
so nice things that happened like disconnected git object trees. After
installing a git hook in the subtree managed upstream repository that
refuses such a push, disconnected object trees never happened again.
Also checking out commits from the subtree side of a subtree merge gives
interresting results. Well, plausible once you think about it, but
probably surprising for the average git user. You will see a completely
different filesystem structure, the one of the subtree you have.


I take it this means `git bisect` becomes unusable? At least it looks 
like it after a bit of playing around with it. That seems like a 
show-stopper :(


Cheers,
Nicolai



The success side: A hand full years sharing > 2e5 loc where normal
development happens. Probably some orders of magnitude more changes than
I observed with addrlib since its creation. It basically does what you
expect when you think about the problem to be solved. So, beside the
problems there has been loads of expected and good behavior. Without
subtree merge that undertaken would have been much more difficult to
infeasible.



I do not think that I can give a recomendation. The basic constraints
like the age of git used and the history of the repositry may introduce
problems that you never observe with something simpler.



best

Mathias


___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev


Re: [Mesa-dev] Moving amdgpu/addrlib into a git submodule

2016-08-12 Thread Nicolai Hähnle

On 11.08.2016 13:12, Enrico Weigelt, metux IT consult wrote:

On 10.08.2016 15:19, Nicolai Hähnle wrote:

It would be nice to have something more concrete to go on, like what are
the problems _really_ :)


Actually, I haven't understood who the other users of addrlib actually
are - is it just AMDs proprietary driver ?


ROCm, i.e. compute, will want to use it at some point.



What pain do submodules give you that wouldn't be even _worse_ with a
separate script for fetching external sources?


In the end - from dist perspective - it's the same kind of problem:
you still have to do extra steps to get a full source tree, and you'll
need to do extra downloads within the build process (unless you mirror/
merge it into your own repo, to create a full tree for the source
package) ... in either ways: really ugly.


The "extra steps" are `git clone --recursive` instead of `git clone` and 
`git pull && git submodule update` instead of `git pull`. If you're 
building from source releases, you don't have to change anything at all 
(the idea is to include addrlib in those).




Look, there are a bunch of different use cases here. Distro packagers;
Mesa contributors and bleeding-edge users; and us (which is slightly
separate because we're not _just_ Mesa contributors but have parts of
the stack elsewhere, which is why this is coming up in the first place).


ACK.


Most of the complaints seem to come from distro packaging, though I have
yet to understand what the exact problems are.


We need a 100% automatic build process - from the original source.
And we need an *easy* way to apply/deapply our local patches ontop of
the source tree - or even better: just have a git repo with just the
full tree.

If upstream doesn't provide that, all the distros out there have to invent
their own mechanisms to transform upstream's mess into sth useful - and
that needs to support easy rebasing.


Rebasing, seriously?

Honestly, when distributions go crazy with local patches, they just tend 
to make everybody's life worse. Suddenly you have users appearing with 
weird bug reports that you do not understand because their distribution 
changed something for whatever reason.


I can understand that a tweak in the build scripts is occasionally 
necessary, but when you start applying your own patches to the _drivers_ 
(which _is_ what you're implying here...), you should re-think your life 
choices.


At the very least, you shouldn't come whining to upstream when you're 
having a hard time doing something stupid.


I'll admit though that the rebasing situation sucks. Given Git's 
underlying data model, there's really no reason why rebase should be any 
harder with submodules than in the alternative world where you keep 
addrlib completely separately, but right now the command-line interface 
is supremely unhelpful for that use case.


Seems like a classic chicken-and-egg situation where submodules have a 
bad reputation because some of the tooling is not up to the usual git 
standard, and then it doesn't get fixed because of that bad reputation. 
Oh well.




For us, in addition to avoiding the same hassles there's the question of
maintainability, and perhaps (with a huge load of optimism) getting the
closed source folks a bit more involved in the fact that there's another
world out here as well.


The hassle comes from the fact that source tree's (instead of having
libraries) directly shared between projects in the first place. So, the
root is having a bad sw architecture.


I actually disagree about the architecture part. The pro and con 
arguments are fundamentally the same as in the mono-repo vs. multi-repo 
debate. If there were no legal (closed vs. open) and social obstacles, 
I'd say a monorepo is the technically correct solution for this 
particular case -- for basically the same reason(s) that the Linux 
kernel does that, by the way.




Why can't we just solve the root problem instead of burning our time
w/ workarounds ?


By the way: just had a quick look into addrlib and found several
defines which aren't used anywhere (in mesa):

https://github.com/metux/mesa/commit/a38def1c9c6ac09c151399ec1f91b92a5fac3eb0

Maybe there's another consumer for that, but then it would be private
to him and therefore should be defined there.


Yes, there are some other things as well. I'm actually trying to get a 
process worked out here where we're better at that stuff. This thread is 
part of that :-)


Cheers,
Nicolai





--mtx

___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev


___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev


Re: [Mesa-dev] Moving amdgpu/addrlib into a git submodule

2016-08-11 Thread Enrico Weigelt, metux IT consult
On 10.08.2016 15:19, Nicolai Hähnle wrote:
> It would be nice to have something more concrete to go on, like what are
> the problems _really_ :)

Actually, I haven't understood who the other users of addrlib actually
are - is it just AMDs proprietary driver ?

> What pain do submodules give you that wouldn't be even _worse_ with a
> separate script for fetching external sources?

In the end - from dist perspective - it's the same kind of problem:
you still have to do extra steps to get a full source tree, and you'll
need to do extra downloads within the build process (unless you mirror/
merge it into your own repo, to create a full tree for the source
package) ... in either ways: really ugly.

> Look, there are a bunch of different use cases here. Distro packagers;
> Mesa contributors and bleeding-edge users; and us (which is slightly
> separate because we're not _just_ Mesa contributors but have parts of
> the stack elsewhere, which is why this is coming up in the first place).

ACK.

> Most of the complaints seem to come from distro packaging, though I have
> yet to understand what the exact problems are.

We need a 100% automatic build process - from the original source.
And we need an *easy* way to apply/deapply our local patches ontop of
the source tree - or even better: just have a git repo with just the
full tree.

If upstream doesn't provide that, all the distros out there have to invent
their own mechanisms to transform upstream's mess into sth useful - and
that needs to support easy rebasing.

> For us, in addition to avoiding the same hassles there's the question of
> maintainability, and perhaps (with a huge load of optimism) getting the
> closed source folks a bit more involved in the fact that there's another
> world out here as well.

The hassle comes from the fact that source tree's (instead of having
libraries) directly shared between projects in the first place. So, the
root is having a bad sw architecture.

Why can't we just solve the root problem instead of burning our time
w/ workarounds ?


By the way: just had a quick look into addrlib and found several
defines which aren't used anywhere (in mesa):

https://github.com/metux/mesa/commit/a38def1c9c6ac09c151399ec1f91b92a5fac3eb0

Maybe there's another consumer for that, but then it would be private
to him and therefore should be defined there.


--mtx

___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev


Re: [Mesa-dev] Moving amdgpu/addrlib into a git submodule

2016-08-10 Thread Mathias Fröhlich
Hi,

On Wednesday, 10 August 2016 15:19:06 CEST Nicolai Hähnle wrote:
> On 09.08.2016 22:12, Dave Airlie wrote:
> >>
> >> tbh, git submodules are more annoying than they need to be, and I'm
> >> not really terribly excited to use that for something that is a build
> >> dependency.  Maybe just move it into libdrm instead?
> >>
> >
> > I've only had to use git submodules once with spice project, and it
> > was a nightmare. It makes packaging etc a real pita.
> 
> It would be nice to have something more concrete to go on, like what are 
> the problems _really_ :)
> 
> 
> > Alternatives are something like a fetch external sources script,
> > that does git submodules but does it better, you'll see Vulkan-CTS
> > etc use something like that, it would have to be integrated with
> > the build system a bit better though.
> 
> What pain do submodules give you that wouldn't be even _worse_ with a 
> separate script for fetching external sources?
> 
> 
> Look, there are a bunch of different use cases here. Distro packagers; 
> Mesa contributors and bleeding-edge users; and us (which is slightly 
> separate because we're not _just_ Mesa contributors but have parts of 
> the stack elsewhere, which is why this is coming up in the first place).
> 
> Most of the complaints seem to come from distro packaging, though I have 
> yet to understand what the exact problems are.
> 
> For Mesa contributors and users, I'd really like to avoid the hassle of 
> having to get another repository manually.
> 
> For us, in addition to avoiding the same hassles there's the question of 
> maintainability, and perhaps (with a huge load of optimism) getting the 
> closed source folks a bit more involved in the fact that there's another 
> world out here as well.
> 
> On those axes, I've seen one alternative suggestion that makes sense to 
> me, and that is subtree-merges. I have to play with them a bit, but the 
> initial impression is good. They seem like a more git-like solution. 
> That would also be something new for Mesa, though, since it would mean 
> more exceptions to the otherwise linear history.

For subtree-merge:
There is a pair of visualisation teams that used to work for the same company 
until 
they both got sold to individual vendors but still wanted to share their basic 
framwork 
code. Those two applications done by these teams are somewhere beyond - may be 
far beyond 1e6 loc each. The shared part is at least 2e5 loc, probably more. I 
do not 
recall the exact numbers. The shared part is communicated via a common upstream 
repository managed by git-subtree. There happens branching merging, rebasing, 
the 
usual git things. The git version that is used is ancient, the one from redhat 
6.
The git repository is based on an import from svn with non standard branch 
names in 
svn and that svn used to be imported from cvs. All together more than 10 years 
of 
history.
Observed problems: It needs special care to be handled. There were not so nice 
things 
that happened like disconnected git object trees. After installing a git hook 
in the 
subtree managed upstream repository that refuses such a push, disconnected 
object 
trees never happened again. Also checking out commits from the subtree side of 
a 
subtree merge gives interresting results. Well, plausible once you think about 
it, but 
probably surprising for the average git user. You will see a completely 
different 
filesystem structure, the one of the subtree you have.
The success side: A hand full years sharing > 2e5 loc where normal development 
happens. Probably some orders of magnitude more changes than I observed with 
addrlib since its creation. It basically does what you expect when you think 
about the 
problem to be solved. So, beside the problems there has been loads of expected 
and 
good behavior. Without subtree merge that undertaken would have been much more 
difficult to infeasible.

I do not think that I can give a recomendation. The basic constraints like the 
age of git 
used and the history of the repositry may introduce problems that you never 
observe 
with something simpler.

best
Mathias
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev


Re: [Mesa-dev] Moving amdgpu/addrlib into a git submodule

2016-08-10 Thread Nicolai Hähnle

On 10.08.2016 06:22, Edward O'Callaghan wrote:

Well taking a step back here for a second. My initial thoughts are -
what, conceptually, is the primary user of addrlib from an extremely
high level view?

If the case is mesa3d then perhaps mesa should be treated as the
upstream from which other users pull.


Mesa really doesn't make logical sense as a dependency for the HSA stack.



If the case is that it is indeed
truly agnostic/general then perhaps deferring to the packaging
expertise/infrastructure of the distro's (aka pkgconfig, etc..) is
perhaps the best mechanism.  In the latter case, addrlib would be
released with versioning which should be fairly maintainable as it is a
slowly moving target right?


Please no. Those things are a fragile mess. I appreciate it the thought, 
but there's a reason why major version upgrades of shared libraries are 
feared.


Nicolai





Just my two cent, hope its helpful!
Kind Regards,
Edward.

On 08/10/2016 01:39 AM, Christian König wrote:

Am 09.08.2016 um 15:47 schrieb Nicolai Hähnle:

Hi everybody,

addrlib is the addressing and alignment calculator which is used by
radeonsi. It's developed (and also used) internally at AMD, and so far
we've had one open source copy living in the Mesa repository at
src/gallium/winsys/amdgpu/drm/addrlib.

The question of using addrlib in non-Mesa parts of our open-source
stack has come up, in particular in relation to compute. We'd
obviously like to share the code rather than having multiple copies
flying around. Since the interface of addrlib is slow-moving but
unstable, shared linking is not an option.

I think the best way forward is to create a dedicated repository for
addrlib which is then integrated into Mesa as a git submodule.

The point of this email is to gather feedback from the Mesa community
on this plan, which is explicitly:

(0) Create an addrlib repository, say amd/addrlib on fd.o.
(1) Add it as a git submodule to the Mesa repository.
(2) Fix up whatever aspects of the build system that may be affected
(perhaps for building source tarballs).
(3) Go back to mostly ignoring addrlib, except for trying to get
better at syncing with the internal closed-source version.

From initial experiments, the impact on users interested in radeon is
that they will have to run `git submodule init` and then occasionally
`git submodule update`. Users who do not build radeonsi should be able
to ignore the change completely.

There are alternatives. For example, ROCm uses Google's repo tool
already. But for Mesa, git submodule looks like a lightweight, well
supported and overall conservative option that everybody should
already have installed. If there are good arguments for something
else, let's hear them!

Another point: if we proceed with this plan, I think we should
consider moving addrlib into src/amd/addrlib. There are two reasons:
First, transitioning to a submodule *without* changing the directory
is probably more fragile, i.e. what happens when you switch between
checkouts before and after the transition. Second, if/when radv ends
up being merged into Mesa master, it makes sense to have addrlib there
anyway.

Thoughts? Complaints? Praise?


Well using git submodule is a possibility and we had rather good
experience with that in GStreamer.

But it would remove one major argument to beating the addrlib guys
towards a stable interface and/or proper library version handling.

Moving it into libdrm is clearly not an option because then you would
need to use versioning for the whole libdrm_amdgpu library which we
don't want.

Christian.


Nicolai
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev



___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev



___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev


Re: [Mesa-dev] Moving amdgpu/addrlib into a git submodule

2016-08-10 Thread Nicolai Hähnle

On 09.08.2016 22:12, Dave Airlie wrote:


tbh, git submodules are more annoying than they need to be, and I'm
not really terribly excited to use that for something that is a build
dependency.  Maybe just move it into libdrm instead?



I've only had to use git submodules once with spice project, and it
was a nightmare. It makes packaging etc a real pita.


It would be nice to have something more concrete to go on, like what are 
the problems _really_ :)




Alternatives are something like a fetch external sources script,
that does git submodules but does it better, you'll see Vulkan-CTS
etc use something like that, it would have to be integrated with
the build system a bit better though.


What pain do submodules give you that wouldn't be even _worse_ with a 
separate script for fetching external sources?



Look, there are a bunch of different use cases here. Distro packagers; 
Mesa contributors and bleeding-edge users; and us (which is slightly 
separate because we're not _just_ Mesa contributors but have parts of 
the stack elsewhere, which is why this is coming up in the first place).


Most of the complaints seem to come from distro packaging, though I have 
yet to understand what the exact problems are.


For Mesa contributors and users, I'd really like to avoid the hassle of 
having to get another repository manually.


For us, in addition to avoiding the same hassles there's the question of 
maintainability, and perhaps (with a huge load of optimism) getting the 
closed source folks a bit more involved in the fact that there's another 
world out here as well.


On those axes, I've seen one alternative suggestion that makes sense to 
me, and that is subtree-merges. I have to play with them a bit, but the 
initial impression is good. They seem like a more git-like solution. 
That would also be something new for Mesa, though, since it would mean 
more exceptions to the otherwise linear history.


Nicolai
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev


Re: [Mesa-dev] Moving amdgpu/addrlib into a git submodule

2016-08-09 Thread Edward O'Callaghan
Hey guys,

Well taking a step back here for a second. My initial thoughts are -
what, conceptually, is the primary user of addrlib from an extremely
high level view?

If the case is mesa3d then perhaps mesa should be treated as the
upstream from which other users pull. If the case is that it is indeed
truly agnostic/general then perhaps deferring to the packaging
expertise/infrastructure of the distro's (aka pkgconfig, etc..) is
perhaps the best mechanism. In the latter case, addrlib would be
released with versioning which should be fairly maintainable as it is a
slowly moving target right?

Just my two cent, hope its helpful!
Kind Regards,
Edward.

On 08/10/2016 01:39 AM, Christian König wrote:
> Am 09.08.2016 um 15:47 schrieb Nicolai Hähnle:
>> Hi everybody,
>>
>> addrlib is the addressing and alignment calculator which is used by
>> radeonsi. It's developed (and also used) internally at AMD, and so far
>> we've had one open source copy living in the Mesa repository at
>> src/gallium/winsys/amdgpu/drm/addrlib.
>>
>> The question of using addrlib in non-Mesa parts of our open-source
>> stack has come up, in particular in relation to compute. We'd
>> obviously like to share the code rather than having multiple copies
>> flying around. Since the interface of addrlib is slow-moving but
>> unstable, shared linking is not an option.
>>
>> I think the best way forward is to create a dedicated repository for
>> addrlib which is then integrated into Mesa as a git submodule.
>>
>> The point of this email is to gather feedback from the Mesa community
>> on this plan, which is explicitly:
>>
>> (0) Create an addrlib repository, say amd/addrlib on fd.o.
>> (1) Add it as a git submodule to the Mesa repository.
>> (2) Fix up whatever aspects of the build system that may be affected
>> (perhaps for building source tarballs).
>> (3) Go back to mostly ignoring addrlib, except for trying to get
>> better at syncing with the internal closed-source version.
>>
>> From initial experiments, the impact on users interested in radeon is
>> that they will have to run `git submodule init` and then occasionally
>> `git submodule update`. Users who do not build radeonsi should be able
>> to ignore the change completely.
>>
>> There are alternatives. For example, ROCm uses Google's repo tool
>> already. But for Mesa, git submodule looks like a lightweight, well
>> supported and overall conservative option that everybody should
>> already have installed. If there are good arguments for something
>> else, let's hear them!
>>
>> Another point: if we proceed with this plan, I think we should
>> consider moving addrlib into src/amd/addrlib. There are two reasons:
>> First, transitioning to a submodule *without* changing the directory
>> is probably more fragile, i.e. what happens when you switch between
>> checkouts before and after the transition. Second, if/when radv ends
>> up being merged into Mesa master, it makes sense to have addrlib there
>> anyway.
>>
>> Thoughts? Complaints? Praise?
> 
> Well using git submodule is a possibility and we had rather good
> experience with that in GStreamer.
> 
> But it would remove one major argument to beating the addrlib guys
> towards a stable interface and/or proper library version handling.
> 
> Moving it into libdrm is clearly not an option because then you would
> need to use versioning for the whole libdrm_amdgpu library which we
> don't want.
> 
> Christian.
> 
>> Nicolai
>> ___
>> mesa-dev mailing list
>> mesa-dev@lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/mesa-dev
> 
> 
> ___
> mesa-dev mailing list
> mesa-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev



signature.asc
Description: OpenPGP digital signature
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev


Re: [Mesa-dev] Moving amdgpu/addrlib into a git submodule

2016-08-09 Thread Jason Ekstrand
On Tue, Aug 9, 2016 at 1:12 PM, Dave Airlie  wrote:

> >
> > tbh, git submodules are more annoying than they need to be, and I'm
> > not really terribly excited to use that for something that is a build
> > dependency.  Maybe just move it into libdrm instead?
> >
>
> I've only had to use git submodules once with spice project, and it
> was a nightmare. It makes packaging etc a real pita.
>
> Alternatives are something like a fetch external sources script,
> that does git submodules but does it better, you'll see Vulkan-CTS
> etc use something like that, it would have to be integrated with
> the build system a bit better though.
>

This isn't a plug for submodules, but *please* don't base anything on the
Vulkan CTS fetch_sources script.  We've had no end of trouble with their
git hacks.
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev


Re: [Mesa-dev] Moving amdgpu/addrlib into a git submodule

2016-08-09 Thread Dave Airlie
>
> tbh, git submodules are more annoying than they need to be, and I'm
> not really terribly excited to use that for something that is a build
> dependency.  Maybe just move it into libdrm instead?
>

I've only had to use git submodules once with spice project, and it
was a nightmare. It makes packaging etc a real pita.

Alternatives are something like a fetch external sources script,
that does git submodules but does it better, you'll see Vulkan-CTS
etc use something like that, it would have to be integrated with
the build system a bit better though.

Dave.
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev


Re: [Mesa-dev] Moving amdgpu/addrlib into a git submodule

2016-08-09 Thread Erik Faye-Lund
On Tue, Aug 9, 2016 at 7:24 PM, Nicolai Hähnle  wrote:
>
> On 09.08.2016 18:22, Erik Faye-Lund wrote:
>>
>> On Tue, Aug 9, 2016 at 4:59 PM, Nicolai Hähnle  wrote:
>>>
>>> On 09.08.2016 15:58, Rob Clark wrote:


 tbh, git submodules are more annoying than they need to be, and I'm
 not really terribly excited to use that for something that is a build
 dependency.  Maybe just move it into libdrm instead?
>>>
>>>
>>>
>>> I know. That's what I would have proposed if the addrlib interface were
>>> stable. Unfortunately it isn't, and realistically speaking, that's not
>>> going
>>> to change.
>>>
>>> So shared linking is right out.
>>>
>>> Static linking or just including source files from a separate repository
>>> could be considered, but then what's the process for ensuring you have
>>> the
>>> right version?
>>>
>>> The nice aspect of submodules is that every commit of the Mesa repository
>>> "knows" what the corresponding right version of addrlib is, and so git
>>> can
>>> update the submodule to the correct version for you automatically.
>>
>>
>> I'm not a huge fan of submodules either. They just don't deal with
>> distributed development properly, which should be a non-starter for
>> OSS IMO. You either set the submodule to point to an absolute URL, in
>> which case it's awkward to work with if you need to change the code,
>> or you use a relative URL, which forces everyone who has a fork to
>> fork the submodule also. Yuck. As a formerly active Git developer, my
>> impression is that nobody of the core-git developers really loved the
>> idea of git-submodule, it was mostly introduced into Git to help KDE
>> transition their gigantic SVN-external based source tree to Git.
>>
>> IMO, a much better alternative would be to have addrlib live in its
>> own repository, and periodically do a git subtree-merge into mesa and
>> other dependent system. That means that nobody really needs to deal
>> with the fact that the upstream is in a different repo, except when
>> submitting patches for upstream. This is what git.git itself does for
>> some of its subsystems.
>
>
> That looks interesting. Are people using subtree-merge with the kind of
> linear history that Mesa uses?
>

I work mostly with branch-heavy work-flows these days, so I don't
really know. The subtree-merges themselves will obviously appear as
merges, but you can always keep the development in the upstream linear
if you want...
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev


Re: [Mesa-dev] Moving amdgpu/addrlib into a git submodule

2016-08-09 Thread Nicolai Hähnle



On 09.08.2016 18:22, Erik Faye-Lund wrote:

On Tue, Aug 9, 2016 at 4:59 PM, Nicolai Hähnle  wrote:

On 09.08.2016 15:58, Rob Clark wrote:


tbh, git submodules are more annoying than they need to be, and I'm
not really terribly excited to use that for something that is a build
dependency.  Maybe just move it into libdrm instead?



I know. That's what I would have proposed if the addrlib interface were
stable. Unfortunately it isn't, and realistically speaking, that's not going
to change.

So shared linking is right out.

Static linking or just including source files from a separate repository
could be considered, but then what's the process for ensuring you have the
right version?

The nice aspect of submodules is that every commit of the Mesa repository
"knows" what the corresponding right version of addrlib is, and so git can
update the submodule to the correct version for you automatically.


I'm not a huge fan of submodules either. They just don't deal with
distributed development properly, which should be a non-starter for
OSS IMO. You either set the submodule to point to an absolute URL, in
which case it's awkward to work with if you need to change the code,
or you use a relative URL, which forces everyone who has a fork to
fork the submodule also. Yuck. As a formerly active Git developer, my
impression is that nobody of the core-git developers really loved the
idea of git-submodule, it was mostly introduced into Git to help KDE
transition their gigantic SVN-external based source tree to Git.

IMO, a much better alternative would be to have addrlib live in its
own repository, and periodically do a git subtree-merge into mesa and
other dependent system. That means that nobody really needs to deal
with the fact that the upstream is in a different repo, except when
submitting patches for upstream. This is what git.git itself does for
some of its subsystems.


That looks interesting. Are people using subtree-merge with the kind of 
linear history that Mesa uses?


Nicolai




___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev


Re: [Mesa-dev] Moving amdgpu/addrlib into a git submodule

2016-08-09 Thread Nicolai Hähnle

On 09.08.2016 19:18, Marek Olšák wrote:

On Tue, Aug 9, 2016 at 5:35 PM, Nicolai Hähnle  wrote:

On 09.08.2016 17:21, Marek Olšák wrote:


On Tue, Aug 9, 2016 at 3:47 PM, Nicolai Hähnle  wrote:


Hi everybody,

addrlib is the addressing and alignment calculator which is used by
radeonsi. It's developed (and also used) internally at AMD, and so far
we've
had one open source copy living in the Mesa repository at
src/gallium/winsys/amdgpu/drm/addrlib.

The question of using addrlib in non-Mesa parts of our open-source stack
has
come up, in particular in relation to compute. We'd obviously like to
share
the code rather than having multiple copies flying around. Since the
interface of addrlib is slow-moving but unstable, shared linking is not
an
option.

I think the best way forward is to create a dedicated repository for
addrlib
which is then integrated into Mesa as a git submodule.

The point of this email is to gather feedback from the Mesa community on
this plan, which is explicitly:

(0) Create an addrlib repository, say amd/addrlib on fd.o.
(1) Add it as a git submodule to the Mesa repository.
(2) Fix up whatever aspects of the build system that may be affected
(perhaps for building source tarballs).
(3) Go back to mostly ignoring addrlib, except for trying to get better
at
syncing with the internal closed-source version.

From initial experiments, the impact on users interested in radeon is
that
they will have to run `git submodule init` and then occasionally `git
submodule update`. Users who do not build radeonsi should be able to
ignore
the change completely.

There are alternatives. For example, ROCm uses Google's repo tool
already.
But for Mesa, git submodule looks like a lightweight, well supported and
overall conservative option that everybody should already have installed.
If
there are good arguments for something else, let's hear them!

Another point: if we proceed with this plan, I think we should consider
moving addrlib into src/amd/addrlib. There are two reasons: First,
transitioning to a submodule *without* changing the directory is probably
more fragile, i.e. what happens when you switch between checkouts before
and
after the transition. Second, if/when radv ends up being merged into Mesa
master, it makes sense to have addrlib there anyway.

Thoughts? Complaints? Praise?



I don't know.

How does this ensure that Mesa and ROCm addrlib copies won't diverge?



They won't really be different copies, because both "copies" are really
checkouts from the same repository. They will occasionally be checkouts of
_different versions_ from the same repository -- usually that would happen
after a sync with the internal copy, when one driver updates their pointer
before the other does. But that's easiy to reconcile. Usually it should just
mean changing the version pointer in whichever driver uses the older
version.



What issues can we expect if Mesa and ROCm addrlib copies diverge?



This is about software maintenance. If we _do_ have separate copies, and
someone applies a bug fix in one copy, they may forget to apply it to the
other. When we want to sync with the internal copy, that has to be done
twice. Basically, all the usual frictions that go with having the same (or
almost the same) piece of code in more than one place.


Instead of introducing a new repo, can ROCm simply copy addrlib
directly from the Mesa tree?

If I understand it correctly, the only thing the git submodule would
allow is that ROCm developers wouldn't have to clone Mesa.


They certainly can copy addrlib from the Mesa tree. The question is 
whether that's a good basis for staying synchronized in the future.


Nicolai



Marek


___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev


Re: [Mesa-dev] Moving amdgpu/addrlib into a git submodule

2016-08-09 Thread Marek Olšák
On Tue, Aug 9, 2016 at 5:35 PM, Nicolai Hähnle  wrote:
> On 09.08.2016 17:21, Marek Olšák wrote:
>>
>> On Tue, Aug 9, 2016 at 3:47 PM, Nicolai Hähnle  wrote:
>>>
>>> Hi everybody,
>>>
>>> addrlib is the addressing and alignment calculator which is used by
>>> radeonsi. It's developed (and also used) internally at AMD, and so far
>>> we've
>>> had one open source copy living in the Mesa repository at
>>> src/gallium/winsys/amdgpu/drm/addrlib.
>>>
>>> The question of using addrlib in non-Mesa parts of our open-source stack
>>> has
>>> come up, in particular in relation to compute. We'd obviously like to
>>> share
>>> the code rather than having multiple copies flying around. Since the
>>> interface of addrlib is slow-moving but unstable, shared linking is not
>>> an
>>> option.
>>>
>>> I think the best way forward is to create a dedicated repository for
>>> addrlib
>>> which is then integrated into Mesa as a git submodule.
>>>
>>> The point of this email is to gather feedback from the Mesa community on
>>> this plan, which is explicitly:
>>>
>>> (0) Create an addrlib repository, say amd/addrlib on fd.o.
>>> (1) Add it as a git submodule to the Mesa repository.
>>> (2) Fix up whatever aspects of the build system that may be affected
>>> (perhaps for building source tarballs).
>>> (3) Go back to mostly ignoring addrlib, except for trying to get better
>>> at
>>> syncing with the internal closed-source version.
>>>
>>> From initial experiments, the impact on users interested in radeon is
>>> that
>>> they will have to run `git submodule init` and then occasionally `git
>>> submodule update`. Users who do not build radeonsi should be able to
>>> ignore
>>> the change completely.
>>>
>>> There are alternatives. For example, ROCm uses Google's repo tool
>>> already.
>>> But for Mesa, git submodule looks like a lightweight, well supported and
>>> overall conservative option that everybody should already have installed.
>>> If
>>> there are good arguments for something else, let's hear them!
>>>
>>> Another point: if we proceed with this plan, I think we should consider
>>> moving addrlib into src/amd/addrlib. There are two reasons: First,
>>> transitioning to a submodule *without* changing the directory is probably
>>> more fragile, i.e. what happens when you switch between checkouts before
>>> and
>>> after the transition. Second, if/when radv ends up being merged into Mesa
>>> master, it makes sense to have addrlib there anyway.
>>>
>>> Thoughts? Complaints? Praise?
>>
>>
>> I don't know.
>>
>> How does this ensure that Mesa and ROCm addrlib copies won't diverge?
>
>
> They won't really be different copies, because both "copies" are really
> checkouts from the same repository. They will occasionally be checkouts of
> _different versions_ from the same repository -- usually that would happen
> after a sync with the internal copy, when one driver updates their pointer
> before the other does. But that's easiy to reconcile. Usually it should just
> mean changing the version pointer in whichever driver uses the older
> version.
>
>
>> What issues can we expect if Mesa and ROCm addrlib copies diverge?
>
>
> This is about software maintenance. If we _do_ have separate copies, and
> someone applies a bug fix in one copy, they may forget to apply it to the
> other. When we want to sync with the internal copy, that has to be done
> twice. Basically, all the usual frictions that go with having the same (or
> almost the same) piece of code in more than one place.

Instead of introducing a new repo, can ROCm simply copy addrlib
directly from the Mesa tree?

If I understand it correctly, the only thing the git submodule would
allow is that ROCm developers wouldn't have to clone Mesa.

Marek
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev


Re: [Mesa-dev] Moving amdgpu/addrlib into a git submodule

2016-08-09 Thread Erik Faye-Lund
On Tue, Aug 9, 2016 at 4:59 PM, Nicolai Hähnle  wrote:
> On 09.08.2016 15:58, Rob Clark wrote:
>>
>> tbh, git submodules are more annoying than they need to be, and I'm
>> not really terribly excited to use that for something that is a build
>> dependency.  Maybe just move it into libdrm instead?
>
>
> I know. That's what I would have proposed if the addrlib interface were
> stable. Unfortunately it isn't, and realistically speaking, that's not going
> to change.
>
> So shared linking is right out.
>
> Static linking or just including source files from a separate repository
> could be considered, but then what's the process for ensuring you have the
> right version?
>
> The nice aspect of submodules is that every commit of the Mesa repository
> "knows" what the corresponding right version of addrlib is, and so git can
> update the submodule to the correct version for you automatically.

I'm not a huge fan of submodules either. They just don't deal with
distributed development properly, which should be a non-starter for
OSS IMO. You either set the submodule to point to an absolute URL, in
which case it's awkward to work with if you need to change the code,
or you use a relative URL, which forces everyone who has a fork to
fork the submodule also. Yuck. As a formerly active Git developer, my
impression is that nobody of the core-git developers really loved the
idea of git-submodule, it was mostly introduced into Git to help KDE
transition their gigantic SVN-external based source tree to Git.

IMO, a much better alternative would be to have addrlib live in its
own repository, and periodically do a git subtree-merge into mesa and
other dependent system. That means that nobody really needs to deal
with the fact that the upstream is in a different repo, except when
submitting patches for upstream. This is what git.git itself does for
some of its subsystems.
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev


Re: [Mesa-dev] Moving amdgpu/addrlib into a git submodule

2016-08-09 Thread Marek Olšák
On Tue, Aug 9, 2016 at 3:47 PM, Nicolai Hähnle  wrote:
> Hi everybody,
>
> addrlib is the addressing and alignment calculator which is used by
> radeonsi. It's developed (and also used) internally at AMD, and so far we've
> had one open source copy living in the Mesa repository at
> src/gallium/winsys/amdgpu/drm/addrlib.
>
> The question of using addrlib in non-Mesa parts of our open-source stack has
> come up, in particular in relation to compute. We'd obviously like to share
> the code rather than having multiple copies flying around. Since the
> interface of addrlib is slow-moving but unstable, shared linking is not an
> option.
>
> I think the best way forward is to create a dedicated repository for addrlib
> which is then integrated into Mesa as a git submodule.
>
> The point of this email is to gather feedback from the Mesa community on
> this plan, which is explicitly:
>
> (0) Create an addrlib repository, say amd/addrlib on fd.o.
> (1) Add it as a git submodule to the Mesa repository.
> (2) Fix up whatever aspects of the build system that may be affected
> (perhaps for building source tarballs).
> (3) Go back to mostly ignoring addrlib, except for trying to get better at
> syncing with the internal closed-source version.
>
> From initial experiments, the impact on users interested in radeon is that
> they will have to run `git submodule init` and then occasionally `git
> submodule update`. Users who do not build radeonsi should be able to ignore
> the change completely.
>
> There are alternatives. For example, ROCm uses Google's repo tool already.
> But for Mesa, git submodule looks like a lightweight, well supported and
> overall conservative option that everybody should already have installed. If
> there are good arguments for something else, let's hear them!
>
> Another point: if we proceed with this plan, I think we should consider
> moving addrlib into src/amd/addrlib. There are two reasons: First,
> transitioning to a submodule *without* changing the directory is probably
> more fragile, i.e. what happens when you switch between checkouts before and
> after the transition. Second, if/when radv ends up being merged into Mesa
> master, it makes sense to have addrlib there anyway.
>
> Thoughts? Complaints? Praise?

I don't know.

How does this ensure that Mesa and ROCm addrlib copies won't diverge?

What issues can we expect if Mesa and ROCm addrlib copies diverge?

For texture sharing, the buffer metadata is set in a way that doesn't
leave any room for interpretation. I think it's possible to bypass
addrlib in this case.

Marek
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev


Re: [Mesa-dev] Moving amdgpu/addrlib into a git submodule

2016-08-09 Thread Christian König

Am 09.08.2016 um 15:47 schrieb Nicolai Hähnle:

Hi everybody,

addrlib is the addressing and alignment calculator which is used by 
radeonsi. It's developed (and also used) internally at AMD, and so far 
we've had one open source copy living in the Mesa repository at 
src/gallium/winsys/amdgpu/drm/addrlib.


The question of using addrlib in non-Mesa parts of our open-source 
stack has come up, in particular in relation to compute. We'd 
obviously like to share the code rather than having multiple copies 
flying around. Since the interface of addrlib is slow-moving but 
unstable, shared linking is not an option.


I think the best way forward is to create a dedicated repository for 
addrlib which is then integrated into Mesa as a git submodule.


The point of this email is to gather feedback from the Mesa community 
on this plan, which is explicitly:


(0) Create an addrlib repository, say amd/addrlib on fd.o.
(1) Add it as a git submodule to the Mesa repository.
(2) Fix up whatever aspects of the build system that may be affected 
(perhaps for building source tarballs).
(3) Go back to mostly ignoring addrlib, except for trying to get 
better at syncing with the internal closed-source version.


From initial experiments, the impact on users interested in radeon is 
that they will have to run `git submodule init` and then occasionally 
`git submodule update`. Users who do not build radeonsi should be able 
to ignore the change completely.


There are alternatives. For example, ROCm uses Google's repo tool 
already. But for Mesa, git submodule looks like a lightweight, well 
supported and overall conservative option that everybody should 
already have installed. If there are good arguments for something 
else, let's hear them!


Another point: if we proceed with this plan, I think we should 
consider moving addrlib into src/amd/addrlib. There are two reasons: 
First, transitioning to a submodule *without* changing the directory 
is probably more fragile, i.e. what happens when you switch between 
checkouts before and after the transition. Second, if/when radv ends 
up being merged into Mesa master, it makes sense to have addrlib there 
anyway.


Thoughts? Complaints? Praise?


Well using git submodule is a possibility and we had rather good 
experience with that in GStreamer.


But it would remove one major argument to beating the addrlib guys 
towards a stable interface and/or proper library version handling.


Moving it into libdrm is clearly not an option because then you would 
need to use versioning for the whole libdrm_amdgpu library which we 
don't want.


Christian.


Nicolai
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev



___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev


Re: [Mesa-dev] Moving amdgpu/addrlib into a git submodule

2016-08-09 Thread Nicolai Hähnle

On 09.08.2016 17:21, Marek Olšák wrote:

On Tue, Aug 9, 2016 at 3:47 PM, Nicolai Hähnle  wrote:

Hi everybody,

addrlib is the addressing and alignment calculator which is used by
radeonsi. It's developed (and also used) internally at AMD, and so far we've
had one open source copy living in the Mesa repository at
src/gallium/winsys/amdgpu/drm/addrlib.

The question of using addrlib in non-Mesa parts of our open-source stack has
come up, in particular in relation to compute. We'd obviously like to share
the code rather than having multiple copies flying around. Since the
interface of addrlib is slow-moving but unstable, shared linking is not an
option.

I think the best way forward is to create a dedicated repository for addrlib
which is then integrated into Mesa as a git submodule.

The point of this email is to gather feedback from the Mesa community on
this plan, which is explicitly:

(0) Create an addrlib repository, say amd/addrlib on fd.o.
(1) Add it as a git submodule to the Mesa repository.
(2) Fix up whatever aspects of the build system that may be affected
(perhaps for building source tarballs).
(3) Go back to mostly ignoring addrlib, except for trying to get better at
syncing with the internal closed-source version.

From initial experiments, the impact on users interested in radeon is that
they will have to run `git submodule init` and then occasionally `git
submodule update`. Users who do not build radeonsi should be able to ignore
the change completely.

There are alternatives. For example, ROCm uses Google's repo tool already.
But for Mesa, git submodule looks like a lightweight, well supported and
overall conservative option that everybody should already have installed. If
there are good arguments for something else, let's hear them!

Another point: if we proceed with this plan, I think we should consider
moving addrlib into src/amd/addrlib. There are two reasons: First,
transitioning to a submodule *without* changing the directory is probably
more fragile, i.e. what happens when you switch between checkouts before and
after the transition. Second, if/when radv ends up being merged into Mesa
master, it makes sense to have addrlib there anyway.

Thoughts? Complaints? Praise?


I don't know.

How does this ensure that Mesa and ROCm addrlib copies won't diverge?


They won't really be different copies, because both "copies" are really 
checkouts from the same repository. They will occasionally be checkouts 
of _different versions_ from the same repository -- usually that would 
happen after a sync with the internal copy, when one driver updates 
their pointer before the other does. But that's easiy to reconcile. 
Usually it should just mean changing the version pointer in whichever 
driver uses the older version.




What issues can we expect if Mesa and ROCm addrlib copies diverge?


This is about software maintenance. If we _do_ have separate copies, and 
someone applies a bug fix in one copy, they may forget to apply it to 
the other. When we want to sync with the internal copy, that has to be 
done twice. Basically, all the usual frictions that go with having the 
same (or almost the same) piece of code in more than one place.




For texture sharing, the buffer metadata is set in a way that doesn't
leave any room for interpretation. I think it's possible to bypass
addrlib in this case.


Right, this is orthogonal to interoperability. Multiple drivers with 
different versions of addrlib can coexist in a system.


Nicolai



Marek


___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev


Re: [Mesa-dev] Moving amdgpu/addrlib into a git submodule

2016-08-09 Thread Enrico Weigelt, metux IT consult
On 09.08.2016 16:59, Nicolai Hähnle wrote:

> So shared linking is right out.

Not exactly. Just everything needs to be linked against the matching
versions. More a dist-layer problem.

addrlibs folks should learn to introduce a proper versioning and
provide MVCC-capable build rules. That really isn't hard.
> Static linking or just including source files from a separate repository
> could be considered, but then what's the process for ensuring you have
> the right version?

pkgconfig ?

> The nice aspect of submodules is that every commit of the Mesa
> repository "knows" what the corresponding right version of addrlib is,
> and so git can update the submodule to the correct version for you
> automatically.

No, it can only checkout the ref'ed commit or anywhere else the user
tells it to. Just jumping to the head does exactly *not* jump to
anything like an correct version. And that's all that git can do
for you automatically.


--mtx

___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev


Re: [Mesa-dev] Moving amdgpu/addrlib into a git submodule

2016-08-09 Thread Enrico Weigelt, metux IT consult
On 09.08.2016 15:47, Nicolai Hähnle wrote:

> I think the best way forward is to create a dedicated repository for
> addrlib which is then integrated into Mesa as a git submodule.

If you really wanna make a lot of people, especially dist-maintainers
very unhappy ...

> From initial experiments, the impact on users interested in radeon is
> that they will have to run `git submodule init` and then occasionally
> `git submodule update`.

Which requires additional, package specific, logic in build/rule files
of all the distros out there.

> There are alternatives. For example, ROCm uses Google's repo tool
> already. 

Even worse.


--mtx

___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev


Re: [Mesa-dev] Moving amdgpu/addrlib into a git submodule

2016-08-09 Thread Nicolai Hähnle

On 09.08.2016 15:58, Rob Clark wrote:

On Tue, Aug 9, 2016 at 9:47 AM, Nicolai Hähnle  wrote:

Hi everybody,

addrlib is the addressing and alignment calculator which is used by
radeonsi. It's developed (and also used) internally at AMD, and so far we've
had one open source copy living in the Mesa repository at
src/gallium/winsys/amdgpu/drm/addrlib.

The question of using addrlib in non-Mesa parts of our open-source stack has
come up, in particular in relation to compute. We'd obviously like to share
the code rather than having multiple copies flying around. Since the
interface of addrlib is slow-moving but unstable, shared linking is not an
option.

I think the best way forward is to create a dedicated repository for addrlib
which is then integrated into Mesa as a git submodule.

The point of this email is to gather feedback from the Mesa community on
this plan, which is explicitly:

(0) Create an addrlib repository, say amd/addrlib on fd.o.
(1) Add it as a git submodule to the Mesa repository.
(2) Fix up whatever aspects of the build system that may be affected
(perhaps for building source tarballs).
(3) Go back to mostly ignoring addrlib, except for trying to get better at
syncing with the internal closed-source version.

From initial experiments, the impact on users interested in radeon is that
they will have to run `git submodule init` and then occasionally `git
submodule update`. Users who do not build radeonsi should be able to ignore
the change completely.


tbh, git submodules are more annoying than they need to be, and I'm
not really terribly excited to use that for something that is a build
dependency.  Maybe just move it into libdrm instead?


I know. That's what I would have proposed if the addrlib interface were 
stable. Unfortunately it isn't, and realistically speaking, that's not 
going to change.


So shared linking is right out.

Static linking or just including source files from a separate repository 
could be considered, but then what's the process for ensuring you have 
the right version?


The nice aspect of submodules is that every commit of the Mesa 
repository "knows" what the corresponding right version of addrlib is, 
and so git can update the submodule to the correct version for you 
automatically.


Cheers,
Nicolai


BR,
-R


There are alternatives. For example, ROCm uses Google's repo tool already.
But for Mesa, git submodule looks like a lightweight, well supported and
overall conservative option that everybody should already have installed. If
there are good arguments for something else, let's hear them!

Another point: if we proceed with this plan, I think we should consider
moving addrlib into src/amd/addrlib. There are two reasons: First,
transitioning to a submodule *without* changing the directory is probably
more fragile, i.e. what happens when you switch between checkouts before and
after the transition. Second, if/when radv ends up being merged into Mesa
master, it makes sense to have addrlib there anyway.

Thoughts? Complaints? Praise?
Nicolai
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev


Re: [Mesa-dev] Moving amdgpu/addrlib into a git submodule

2016-08-09 Thread Rob Clark
On Tue, Aug 9, 2016 at 9:47 AM, Nicolai Hähnle  wrote:
> Hi everybody,
>
> addrlib is the addressing and alignment calculator which is used by
> radeonsi. It's developed (and also used) internally at AMD, and so far we've
> had one open source copy living in the Mesa repository at
> src/gallium/winsys/amdgpu/drm/addrlib.
>
> The question of using addrlib in non-Mesa parts of our open-source stack has
> come up, in particular in relation to compute. We'd obviously like to share
> the code rather than having multiple copies flying around. Since the
> interface of addrlib is slow-moving but unstable, shared linking is not an
> option.
>
> I think the best way forward is to create a dedicated repository for addrlib
> which is then integrated into Mesa as a git submodule.
>
> The point of this email is to gather feedback from the Mesa community on
> this plan, which is explicitly:
>
> (0) Create an addrlib repository, say amd/addrlib on fd.o.
> (1) Add it as a git submodule to the Mesa repository.
> (2) Fix up whatever aspects of the build system that may be affected
> (perhaps for building source tarballs).
> (3) Go back to mostly ignoring addrlib, except for trying to get better at
> syncing with the internal closed-source version.
>
> From initial experiments, the impact on users interested in radeon is that
> they will have to run `git submodule init` and then occasionally `git
> submodule update`. Users who do not build radeonsi should be able to ignore
> the change completely.

tbh, git submodules are more annoying than they need to be, and I'm
not really terribly excited to use that for something that is a build
dependency.  Maybe just move it into libdrm instead?

BR,
-R

> There are alternatives. For example, ROCm uses Google's repo tool already.
> But for Mesa, git submodule looks like a lightweight, well supported and
> overall conservative option that everybody should already have installed. If
> there are good arguments for something else, let's hear them!
>
> Another point: if we proceed with this plan, I think we should consider
> moving addrlib into src/amd/addrlib. There are two reasons: First,
> transitioning to a submodule *without* changing the directory is probably
> more fragile, i.e. what happens when you switch between checkouts before and
> after the transition. Second, if/when radv ends up being merged into Mesa
> master, it makes sense to have addrlib there anyway.
>
> Thoughts? Complaints? Praise?
> Nicolai
> ___
> mesa-dev mailing list
> mesa-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev


[Mesa-dev] Moving amdgpu/addrlib into a git submodule

2016-08-09 Thread Nicolai Hähnle

Hi everybody,

addrlib is the addressing and alignment calculator which is used by 
radeonsi. It's developed (and also used) internally at AMD, and so far 
we've had one open source copy living in the Mesa repository at 
src/gallium/winsys/amdgpu/drm/addrlib.


The question of using addrlib in non-Mesa parts of our open-source stack 
has come up, in particular in relation to compute. We'd obviously like 
to share the code rather than having multiple copies flying around. 
Since the interface of addrlib is slow-moving but unstable, shared 
linking is not an option.


I think the best way forward is to create a dedicated repository for 
addrlib which is then integrated into Mesa as a git submodule.


The point of this email is to gather feedback from the Mesa community on 
this plan, which is explicitly:


(0) Create an addrlib repository, say amd/addrlib on fd.o.
(1) Add it as a git submodule to the Mesa repository.
(2) Fix up whatever aspects of the build system that may be affected 
(perhaps for building source tarballs).
(3) Go back to mostly ignoring addrlib, except for trying to get better 
at syncing with the internal closed-source version.


From initial experiments, the impact on users interested in radeon is 
that they will have to run `git submodule init` and then occasionally 
`git submodule update`. Users who do not build radeonsi should be able 
to ignore the change completely.


There are alternatives. For example, ROCm uses Google's repo tool 
already. But for Mesa, git submodule looks like a lightweight, well 
supported and overall conservative option that everybody should already 
have installed. If there are good arguments for something else, let's 
hear them!


Another point: if we proceed with this plan, I think we should consider 
moving addrlib into src/amd/addrlib. There are two reasons: First, 
transitioning to a submodule *without* changing the directory is 
probably more fragile, i.e. what happens when you switch between 
checkouts before and after the transition. Second, if/when radv ends up 
being merged into Mesa master, it makes sense to have addrlib there anyway.


Thoughts? Complaints? Praise?
Nicolai
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev