Re: [OSGeoLive] [Incubator] The Open Data Cube as a OSGeo Community Project

[Alex Leith]

Hey Folks

Finally making some progress here.

I'm using this tool, which works well:
https://github.com/johann-petrak/licenseheaders

And I'm planning to use this text. Any issues with my proposed wording?

#
# Copyright (c) 2015-2020 ODC Contributors
#
# This file is part of the Open Data Cube
# See https://opendatacube.org for more information
#
# SPDX-License-Identifier: Apache-2.0
#

On Tue, 8 Sep 2020 at 17:20, Jody Garnett  wrote:

> Yeah this kind of thing can be quick (many projects keep track of code
> from day one) or take years (for geoserver we eventually sat down with a
> code sprint and git history).
>
> Here is a script I made, that produced a CSV file, that I then open up in
> a spreadsheet:
>
> #!/bin/bash
> for file in `git ls-files -x src/*`
> do
> HASH=`git log --diff-filter=A --format="%H" -- $file | tail -n 1`
> AUTHOR=`git show -s --format="%aN" --date=short $HASH --`
> CREATED=`git show -s --format="%cd" --date=short $HASH --`
> MODIFIED=`git log --format="%cd" --date=short -n 1 -- $file`
> printf "%s,%s,%s,%s,%s\n" $file "$AUTHOR" "${CREATED:0:4}"
> "${MODIFIED:0:4}" $HASH
> done
>
> Reference: https://bugs.eclipse.org/bugs/show_bug.cgi?id=442206
>
> Keep in mind some of the fun of this exercise is to double check where
> your code actually came from and confirm you have permission to distribute
> it as open source. While a script can provide a running start, it is your
> confidence in what you have done as a team that we are really going for
> here :)
>
> Also note that code providence audits like this are performed when
> evaluating software projects for use. Performing this check yourself lowers
> the bar for organizations wishing to use your software.
> --
> Jody Garnett
>
>
> On Thu, 3 Sep 2020 at 16:56, Alex Leith  wrote:
>
>> Thanks Markus!
>>
>> I'll have a read of it, but I haven't got much experience running perl
>> and we are probably better off whipping up something custom.
>>
>> Always nice to see the history!
>>
>> On Fri, 4 Sep 2020 at 05:08, Markus Neteler  wrote:
>>
>>> Hi,
>>>
>>> I searched around for a while,
>>>
>>> found
>>> https://wiki.osgeo.org/wiki/GRASS_Incubation_Progress
>>> --> https://wiki.osgeo.org/wiki/GRASS_Provenance_Review
>>>
>>> with the hint
>>> "Kudos to Schuyler Erle for writing the PERL script which quite helped
>>> (tools/copywrite.pl). The expansion of CVS names was done by reading
>>> in contributors.csv."
>>>
>>> and voilà:
>>> https://github.com/OSGeo/grass/blob/master/tools/copywrite.pl
>>>
>>> (originally added on April 27, 2006:
>>>
>>> https://github.com/OSGeo/grass-legacy/blob/develbranch_6/tools/copywrite.pl
>>> )
>>>
>>> Dunno if it helps in your case but nothing is lost ;-)
>>>
>>> Markus
>>>
>>> On Thu, Sep 3, 2020 at 12:38 AM Alex Leith  wrote:
>>> >
>>> > I've been procrastinating on this for a long time... I will get to it,
>>> but if there is an existing script, that would help!
>>> >
>>> > On Thu, 3 Sep 2020 at 02:02, Markus Neteler  wrote:
>>> >>
>>> >> Hi Alex,
>>> >>
>>> >> On Wed, Sep 2, 2020 at 12:55 AM Alex Leith 
>>> wrote:
>>> >> >
>>> >> > Oh nice!
>>> >> >
>>> >> > I'm working on ODC as an OSGeo project... I just need to find a day
>>> to add headers to all our Python files to progress to the next stage.
>>> >>
>>> >> I did much of this job back then for GRASS GIS (in 2006 or so): we
>>> >> used a script to generate and inject draft headers with author names
>>> >> extracted from the repo log.
>>> >> Don't know if I can still find this tool but others here may have done
>>> >> something similar.
>>> >>
>>> >> Cheers,
>>> >> Markus
>>> >
>>> >
>>> >
>>> > --
>>> > Alex Leith
>>> > m: 0419189050
>>>
>>> --
>>> Markus Neteler, PhD
>>> https://www.mundialis.de - free data with free software
>>> https://grass.osgeo.org
>>> https://courses.neteler.org/blog
>>>
>>
>>
>> --
>> Alex Leith
>> m: 0419189050
>> ___
>> Incubator mailing list
>> incuba...@lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/incubator
>
>

-- 
Alex Leith
m: 0419189050
___
osgeolive mailing list
osgeolive@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/osgeolive

Re: [OSGeoLive] [Incubator] The Open Data Cube as a OSGeo Community Project

[Jody Garnett]

Just a few comments inline:

> We ask that the project demonstrate collaboration, perhaps with a
> history of bug report or pull requests
>
>- Both in github and in OSGeo website, we show the team of
>   contributors, and the steering committee members. But I got to say that 
> we
>   are just starting in this respect. There are currently few external
>   collaborators. Most of the development is still done by the core team. 
> We
>   are however open for collaboration, and we hope that becoming an OSGeo
>   community project will bring more collaboration. We need to give time to
>   the community to join the project and start collaborating.
>
> And that is fine, this is one reason why we start slow with a
CONTRIBUTING.md file, and the PRs and notes you provided below are great.

I also got a chance 1st hand to see your group join the Bolsena 2020 code
sprint :)

aside: It can be quite a big step for some organizations to think through
how to accept outside contributions, and notice they are no longer the sole
"owner" of a codebase. Indeed many groups just use an open source license
to publish because they can never get legal approval to work with others.

Please consider this message as an *Application of MobilityDB for OSGeo
> Community Project*.
>

Great, I appreciate your work thus far. It will take me some time to double
check and make a motion (so if any other committee member wants to have a
go please step up).

>
___
osgeolive mailing list
osgeolive@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/osgeolive

Re: [OSGeoLive] [Incubator] The Open Data Cube as a OSGeo Community Project

[Mahmoud SAKR]

Hi Jody,
It means that we, as far, can become an OSGeo community project, which is
indeed the goal of this thread.
I see that we fulfill the checklist for a community project:

"""
In addition to being listed on the website, projects applying to the *OSGeo
Community* program are asked to:
Be geospatial
Confirm by checking README or project description (done)
project page on the osgeo website (done)
We ask projects have some user documentation, for example an OSGeo
Live quickstart:

   - a detailed user manual for the release in master branch:
  https://docs.mobilitydb.com/MobilityDB/master/
  - a detailed user manual for the develop branch:
  https://docs.mobilitydb.com/MobilityDB/develop/
  - multiple use cases in a workshop style:
  https://github.com/MobilityDB/MobilityDB-workshop
  - a data generator and a query tutorial/workshop:
  https://github.com/MobilityDB/MobilityDB-BerlinMOD

Have a free license or an open source license.
The license must be OSI approved (done, PostgreSQL license)
We ask that the project team check the file headers and double
check the license has been appropriately applied (done)
Welcome participation and new contributors.
We look for a clear contribution policy (done, CONTRIBUTING.md
)
We ask that the project demonstrate collaboration, perhaps with a
history of bug report or pull requests

   - Both in github and in OSGeo website, we show the team of contributors,
  and the steering committee members. But I got to say that we are just
  starting in this respect. There are currently few external collaborators.
  Most of the development is still done by the core team. We are
however open
  for collaboration, and we hope that becoming an OSGeo community project
  will bring more collaboration. We need to give time to the community to
  join the project and start collaborating.
  - Examples of collaborations:
 - Krishna Chaitanya has implemented an SQL alchemy driver of
 MobilityDB: https://github.com/adonmo/mobilitydb-sqlalchemy
 - He is currently implementing MEOS, an extension of GEOS, to
 support moving object data management. The goal is that MEOS
will become
 the processing engine of MobilityDB, taking care of spatiotemporal
 functions.
 - Ongoing discussions with Nyall Dawson, from the QGIS project, to
 support mobility data visualization in QGIS.
 - Collaboration with the OGC moving feature standard working
 group, using MobilityDb as a reference implementation.
 - Collaborating with the pgRouting project, in the BerlinMOD data
 generator: https://github.com/MobilityDB/MobilityDB-BerlinMO

Projects are required to have a code of conduct (done,
code-of-conduct.md)


Please consider this message as an *Application of MobilityDB for OSGeo
Community Project*.

It is understood that becoming an OSGeo project is a bigger step that needs
much more work. We need to allow more time for people to use MobilityDB,
report bugs, contribute ideas and code, build packages, and so on. It is a
strategic goal for the future. This first step of becoming a community
project is expected to help in this direction.


Best regards,
Mahmoud



On Tue, Sep 8, 2020 at 9:20 AM Jody Garnett  wrote:

> Yeah this kind of thing can be quick (many projects keep track of code
> from day one) or take years (for geoserver we eventually sat down with a
> code sprint and git history).
>
> Here is a script I made, that produced a CSV file, that I then open up in
> a spreadsheet:
>
> #!/bin/bash
> for file in `git ls-files -x src/*`
> do
> HASH=`git log --diff-filter=A --format="%H" -- $file | tail -n 1`
> AUTHOR=`git show -s --format="%aN" --date=short $HASH --`
> CREATED=`git show -s --format="%cd" --date=short $HASH --`
> MODIFIED=`git log --format="%cd" --date=short -n 1 -- $file`
> printf "%s,%s,%s,%s,%s\n" $file "$AUTHOR" "${CREATED:0:4}"
> "${MODIFIED:0:4}" $HASH
> done
>
> Reference: https://bugs.eclipse.org/bugs/show_bug.cgi?id=442206
>
> Keep in mind some of the fun of this exercise is to double check where
> your code actually came from and confirm you have permission to distribute
> it as open source. While a script can provide a running start, it is your
> confidence in what you have done as a team that we are really going for
> here :)
>
> Also note that code providence audits like this are performed when
> evaluating software projects for use. Performing this check yourself lowers
> the bar for organizations wishing to use your software.
> --
> Jody Garnett
>
>
> On Thu, 3 Sep 2020 at 16:56, Alex Leith  wrote:
>
>> Thanks Markus!
>>
>> I'll have a read of it, but I haven't got

Re: [OSGeoLive] [Incubator] The Open Data Cube as a OSGeo Community Project

[Jody Garnett]

Yeah this kind of thing can be quick (many projects keep track of code from
day one) or take years (for geoserver we eventually sat down with a code
sprint and git history).

Here is a script I made, that produced a CSV file, that I then open up in a
spreadsheet:

#!/bin/bash
for file in `git ls-files -x src/*`
do
HASH=`git log --diff-filter=A --format="%H" -- $file | tail -n 1`
AUTHOR=`git show -s --format="%aN" --date=short $HASH --`
CREATED=`git show -s --format="%cd" --date=short $HASH --`
MODIFIED=`git log --format="%cd" --date=short -n 1 -- $file`
printf "%s,%s,%s,%s,%s\n" $file "$AUTHOR" "${CREATED:0:4}"
"${MODIFIED:0:4}" $HASH
done

Reference: https://bugs.eclipse.org/bugs/show_bug.cgi?id=442206

Keep in mind some of the fun of this exercise is to double check where your
code actually came from and confirm you have permission to distribute it as
open source. While a script can provide a running start, it is your
confidence in what you have done as a team that we are really going for
here :)

Also note that code providence audits like this are performed when
evaluating software projects for use. Performing this check yourself lowers
the bar for organizations wishing to use your software.
--
Jody Garnett


On Thu, 3 Sep 2020 at 16:56, Alex Leith  wrote:

> Thanks Markus!
>
> I'll have a read of it, but I haven't got much experience running perl and
> we are probably better off whipping up something custom.
>
> Always nice to see the history!
>
> On Fri, 4 Sep 2020 at 05:08, Markus Neteler  wrote:
>
>> Hi,
>>
>> I searched around for a while,
>>
>> found
>> https://wiki.osgeo.org/wiki/GRASS_Incubation_Progress
>> --> https://wiki.osgeo.org/wiki/GRASS_Provenance_Review
>>
>> with the hint
>> "Kudos to Schuyler Erle for writing the PERL script which quite helped
>> (tools/copywrite.pl). The expansion of CVS names was done by reading
>> in contributors.csv."
>>
>> and voilà:
>> https://github.com/OSGeo/grass/blob/master/tools/copywrite.pl
>>
>> (originally added on April 27, 2006:
>>
>> https://github.com/OSGeo/grass-legacy/blob/develbranch_6/tools/copywrite.pl
>> )
>>
>> Dunno if it helps in your case but nothing is lost ;-)
>>
>> Markus
>>
>> On Thu, Sep 3, 2020 at 12:38 AM Alex Leith  wrote:
>> >
>> > I've been procrastinating on this for a long time... I will get to it,
>> but if there is an existing script, that would help!
>> >
>> > On Thu, 3 Sep 2020 at 02:02, Markus Neteler  wrote:
>> >>
>> >> Hi Alex,
>> >>
>> >> On Wed, Sep 2, 2020 at 12:55 AM Alex Leith 
>> wrote:
>> >> >
>> >> > Oh nice!
>> >> >
>> >> > I'm working on ODC as an OSGeo project... I just need to find a day
>> to add headers to all our Python files to progress to the next stage.
>> >>
>> >> I did much of this job back then for GRASS GIS (in 2006 or so): we
>> >> used a script to generate and inject draft headers with author names
>> >> extracted from the repo log.
>> >> Don't know if I can still find this tool but others here may have done
>> >> something similar.
>> >>
>> >> Cheers,
>> >> Markus
>> >
>> >
>> >
>> > --
>> > Alex Leith
>> > m: 0419189050
>>
>> --
>> Markus Neteler, PhD
>> https://www.mundialis.de - free data with free software
>> https://grass.osgeo.org
>> https://courses.neteler.org/blog
>>
>
>
> --
> Alex Leith
> m: 0419189050
> ___
> Incubator mailing list
> incuba...@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/incubator
___
osgeolive mailing list
osgeolive@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/osgeolive

Re: [OSGeoLive] [Incubator] The Open Data Cube as a OSGeo Community Project

[Alex Leith]

Thanks Markus!

I'll have a read of it, but I haven't got much experience running perl and
we are probably better off whipping up something custom.

Always nice to see the history!

On Fri, 4 Sep 2020 at 05:08, Markus Neteler  wrote:

> Hi,
>
> I searched around for a while,
>
> found
> https://wiki.osgeo.org/wiki/GRASS_Incubation_Progress
> --> https://wiki.osgeo.org/wiki/GRASS_Provenance_Review
>
> with the hint
> "Kudos to Schuyler Erle for writing the PERL script which quite helped
> (tools/copywrite.pl). The expansion of CVS names was done by reading
> in contributors.csv."
>
> and voilà:
> https://github.com/OSGeo/grass/blob/master/tools/copywrite.pl
>
> (originally added on April 27, 2006:
>
> https://github.com/OSGeo/grass-legacy/blob/develbranch_6/tools/copywrite.pl
> )
>
> Dunno if it helps in your case but nothing is lost ;-)
>
> Markus
>
> On Thu, Sep 3, 2020 at 12:38 AM Alex Leith  wrote:
> >
> > I've been procrastinating on this for a long time... I will get to it,
> but if there is an existing script, that would help!
> >
> > On Thu, 3 Sep 2020 at 02:02, Markus Neteler  wrote:
> >>
> >> Hi Alex,
> >>
> >> On Wed, Sep 2, 2020 at 12:55 AM Alex Leith 
> wrote:
> >> >
> >> > Oh nice!
> >> >
> >> > I'm working on ODC as an OSGeo project... I just need to find a day
> to add headers to all our Python files to progress to the next stage.
> >>
> >> I did much of this job back then for GRASS GIS (in 2006 or so): we
> >> used a script to generate and inject draft headers with author names
> >> extracted from the repo log.
> >> Don't know if I can still find this tool but others here may have done
> >> something similar.
> >>
> >> Cheers,
> >> Markus
> >
> >
> >
> > --
> > Alex Leith
> > m: 0419189050
>
> --
> Markus Neteler, PhD
> https://www.mundialis.de - free data with free software
> https://grass.osgeo.org
> https://courses.neteler.org/blog
>


-- 
Alex Leith
m: 0419189050
___
osgeolive mailing list
osgeolive@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/osgeolive

Re: [OSGeoLive] [Incubator] The Open Data Cube as a OSGeo Community Project

[Markus Neteler]

Hi,

I searched around for a while,

found
https://wiki.osgeo.org/wiki/GRASS_Incubation_Progress
--> https://wiki.osgeo.org/wiki/GRASS_Provenance_Review

with the hint
"Kudos to Schuyler Erle for writing the PERL script which quite helped
(tools/copywrite.pl). The expansion of CVS names was done by reading
in contributors.csv."

and voilà:
https://github.com/OSGeo/grass/blob/master/tools/copywrite.pl

(originally added on April 27, 2006:
 https://github.com/OSGeo/grass-legacy/blob/develbranch_6/tools/copywrite.pl
)

Dunno if it helps in your case but nothing is lost ;-)

Markus

On Thu, Sep 3, 2020 at 12:38 AM Alex Leith  wrote:
>
> I've been procrastinating on this for a long time... I will get to it, but if 
> there is an existing script, that would help!
>
> On Thu, 3 Sep 2020 at 02:02, Markus Neteler  wrote:
>>
>> Hi Alex,
>>
>> On Wed, Sep 2, 2020 at 12:55 AM Alex Leith  wrote:
>> >
>> > Oh nice!
>> >
>> > I'm working on ODC as an OSGeo project... I just need to find a day to add 
>> > headers to all our Python files to progress to the next stage.
>>
>> I did much of this job back then for GRASS GIS (in 2006 or so): we
>> used a script to generate and inject draft headers with author names
>> extracted from the repo log.
>> Don't know if I can still find this tool but others here may have done
>> something similar.
>>
>> Cheers,
>> Markus
>
>
>
> --
> Alex Leith
> m: 0419189050

-- 
Markus Neteler, PhD
https://www.mundialis.de - free data with free software
https://grass.osgeo.org
https://courses.neteler.org/blog
___
osgeolive mailing list
osgeolive@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/osgeolive

Re: [OSGeoLive] [Incubator] The Open Data Cube as a OSGeo Community Project

[Alex Leith]

I've been procrastinating on this for a long time... I will get to it, but
if there is an existing script, that would help!

On Thu, 3 Sep 2020 at 02:02, Markus Neteler  wrote:

> Hi Alex,
>
> On Wed, Sep 2, 2020 at 12:55 AM Alex Leith  wrote:
> >
> > Oh nice!
> >
> > I'm working on ODC as an OSGeo project... I just need to find a day to
> add headers to all our Python files to progress to the next stage.
>
> I did much of this job back then for GRASS GIS (in 2006 or so): we
> used a script to generate and inject draft headers with author names
> extracted from the repo log.
> Don't know if I can still find this tool but others here may have done
> something similar.
>
> Cheers,
> Markus
>


-- 
Alex Leith
m: 0419189050
___
osgeolive mailing list
osgeolive@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/osgeolive

Re: [OSGeoLive] [Incubator] The Open Data Cube as a OSGeo Community Project

[Markus Neteler]

Hi Alex,

On Wed, Sep 2, 2020 at 12:55 AM Alex Leith  wrote:
>
> Oh nice!
>
> I'm working on ODC as an OSGeo project... I just need to find a day to add 
> headers to all our Python files to progress to the next stage.

I did much of this job back then for GRASS GIS (in 2006 or so): we
used a script to generate and inject draft headers with author names
extracted from the repo log.
Don't know if I can still find this tool but others here may have done
something similar.

Cheers,
Markus
___
osgeolive mailing list
osgeolive@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/osgeolive

Re: [OSGeoLive] [Incubator] The Open Data Cube as a OSGeo Community Project

[Cameron Shorter]

replying to the correct osgeolive email list this time...

On Wed, 2 Sep 2020 at 08:55, Alex Leith  wrote:

> Oh nice!
>
> I'm working on ODC as an OSGeo project... I just need to find a day to add
> headers to all our Python files to progress to the next stage.
>
> And yeah, we're talking about the software being included in the OSGeo
> Live project, though it's difficult, since the ODC Python Library is
> closely coupled to a Postgres DB and to data, really... it's a complex app.
> More complex than something like Geoserver.
>
> On Wed, 2 Sep 2020 at 06:15, Cameron Shorter 
> wrote:
>
>> Hey Alex,
>> I've just been sitting in on the osgeolive fortnightly meeting.
>> opendatacube was getting discussed.
>> http://irclogs.geoapt.com/osgeolive/%23osgeolive.2020-09-01.log
>> Projects which are to be included on the next OSGeoLive release are
>> getting worked out now. It would be great to see opendatacube included.
>>
>> 19:59:38 darkblue_b: opendatacube AU is enthusiastic last week
>> 19:59:41 kalxas: motion to include them in the next version
>> 20:00:01 kalxas: darkblue_b, yes I have reached out to them months ago
>> to become Community Project
>> 20:05:21 cameronshorter: I can vouch for the people behind Datacube.
>> Alex is very active in the Australian OSGeo community. And the project
>> comes out of combined government/university and has great roots.
>> 20:05:23 darkblue_b: phma we can chat about this
>> 20:05:33 kalxas: phma, we can help in chat outside the meeting
>> 20:05:34 phma: ok
>> 20:05:50 kalxas: cameronshorter, push them to apply :)
>>
>> On Sat, 16 May 2020 at 07:29, Alex Leith  wrote:
>>
>>> Hey Jody and others
>>>
>>> I work at Geoscience Australia, and Kirill is on my team. Most of the
>>> contributions to the ODC Core codebase are from GA, though there some minor
>>> contributions from external individuals and some significant work from
>>> CSIRO.
>>>
>>> We talked about the code headers and the team in GA feel confident that
>>> there is no significant code in there that is not original to the ODC,
>>> which is one of the reasons for auditing and adding headers, as I
>>> understand it.
>>>
>>> I think we'll write some script that goes and adds Even's suggested
>>> addition into all the appreciate files. I'll report back when we've merged
>>> that in, which will be soonish!
>>>
>>> Thanks again for your support here.
>>>
>>> Cheers,
>>>
>>> On Fri, 15 May 2020 at 21:48, Angelos Tzotsos 
>>> wrote:
>>>
 Big +1 to have opendatacube onboard.

 On 5/15/20 9:02 AM, Jody Garnett wrote:

 I went back to check the website (https://www.opendatacube.org) to see who
 is actually distributing opendatacube. This really is a great example of
 open source being used as the glue to bind a partner ship of a wide range
 of organizations. In this case I assume the players (Geoscience Australia,
 NASA, CSIRO, USGS,Catapult,Analytical Mechanics Associates) are putting in
 work, which is distributed by a public repository 
 (https://github.com/opendatacube).

 1. Checking the history of a random 
 file
  
 
 shows
 it was created by Kirill888  
  from Canberra
 Australia.
 2. That is a unique name so I have a good chance of finding him on LinkedIn
 krill-kouzoubov  
 
 3. LinkedIn shows his employer is Geoscience Australia
 4. If I assume he is operating as an employee, and not as an individual,
 GeoScience Australia the legal entity distributing at least part 
 ofopendatacube.org as open source.

 I think if I find another file we could find a different organization; this
 really is a shared work.

 Notes:
 - The kind of research I just did above is a bother, one of the things we
 are addressing here is getting that detail out of the way so that any
 potential user or contributor the the project can tell who they are working
 with (and evaluate risk accordingly).
 - This kind of thing where multiple organizations are distributing a shared
 work are exactly where an open source foundation like OSGeo thrive. In some
 cases groups find it easier to use a CLA to donate the code to OSGeo which
 operates as neutral party to distribute the code. OSGeo is willing to do
 so, but asks that the project set up a committee (usually with
 representation from the different partners) to manage things.

 I am really impressed with opendatacube, if you are happy using Even
 Rouault's approach you should run with it. The other questions you can save
 for later in your open source journey.
 --
 Jody Garnett
>>>