Re: [OSM-talk] Testing torrents for the planet dump

[Yuri Astrakhan]

Christian, I would like to add torrent support to the download-osm tool
[1].  While I could try to scrape
https://osm.cquest.org/torrents/ , I would obviously rather use the
structured xml file (or if you could provide a JSON file, even better).

Proposed logic:
* get the catalog file (xml/json)
* download the latest torrent
* (TBD) possibly use some magical aria2c options to stop the download if it
is not progressing and fallback to regular http, or possibly pick a
slightly older torrent (?).  Suggestions are welcome -- see full list of
aria2c options [2]

[1]
https://github.com/openmaptiles/openmaptiles-tools#multi-streamed-osm-data-downloader
[2] https://aria2.github.io/manual/en/html/aria2c.html

On Sun, Feb 9, 2020 at 12:44 PM Christian Quest 
wrote:

>
> Le 09/02/2020 à 18:26, Maarten Deen a écrit :
> > On 2020-02-09 16:17, Christian Quest wrote:
> >> A couple of weeks ago, I've (re) started sharing planet dump files
> >> using Bittorrent to test an alternative way to distribute our planet
> >> dumps to reduce the bandwidth load on the OSMF servers.
> >>
> >> Here is a short summary after 2 weeks tests...
> >>
> >> The torrents are generated a few hours after the planet dump
> >> availability on planet.openstreetmap.org server (time needed to
> >> download the original file to generate the torrent).
> >
> > A question about updates of the torrent. The planet changes weekly.
> > Suppose I download the plannet using the torrent and then also seed
> > it, what happens when the next planet comes available? Do I need to
> > use a different torrent to download it again or will the one I have be
> > updated?
> Each planet file has its own torrent. A new planet file means using a
> new torrent to download it.
>
> To simplify downloading the lastest planet, you'll find a
> planet-latest.pbf.torrent similar to the planet-latest.pbf (it's a
> symlink to the last planet available).
>
> If you want to automate downloading new planet thru torrents, for
> example to seed them, there's a rss.xml file you can use. Many
> bittorrent clients support RSS to automate downloads.
>
> --
> Christian Quest - OpenStreetMap France
>
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Testing torrents for the planet dump

[John Whelan]

My concern would be more the volume of data we have whirling round.

Smaller torrents of each country might be more useful and I'd have to 
get my head around picking up the new one each week.  I'm running 
qBittorrent if that is of any use.


Cheerio John

Christian Quest wrote on 2020-02-09 12:42 PM:


Le 09/02/2020 à 18:26, Maarten Deen a écrit :

On 2020-02-09 16:17, Christian Quest wrote:

A couple of weeks ago, I've (re) started sharing planet dump files
using Bittorrent to test an alternative way to distribute our planet
dumps to reduce the bandwidth load on the OSMF servers.

Here is a short summary after 2 weeks tests...

The torrents are generated a few hours after the planet dump
availability on planet.openstreetmap.org server (time needed to
download the original file to generate the torrent).


A question about updates of the torrent. The planet changes weekly. 
Suppose I download the plannet using the torrent and then also seed 
it, what happens when the next planet comes available? Do I need to 
use a different torrent to download it again or will the one I have 
be updated?
Each planet file has its own torrent. A new planet file means using a 
new torrent to download it.


To simplify downloading the lastest planet, you'll find a 
planet-latest.pbf.torrent similar to the planet-latest.pbf (it's a 
symlink to the last planet available).


If you want to automate downloading new planet thru torrents, for 
example to seed them, there's a rss.xml file you can use. Many 
bittorrent clients support RSS to automate downloads.




--
Sent from Postbox 
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Testing torrents for the planet dump

[Christian Quest]

Le 09/02/2020 à 18:26, Maarten Deen a écrit :

On 2020-02-09 16:17, Christian Quest wrote:

A couple of weeks ago, I've (re) started sharing planet dump files
using Bittorrent to test an alternative way to distribute our planet
dumps to reduce the bandwidth load on the OSMF servers.

Here is a short summary after 2 weeks tests...

The torrents are generated a few hours after the planet dump
availability on planet.openstreetmap.org server (time needed to
download the original file to generate the torrent).


A question about updates of the torrent. The planet changes weekly. 
Suppose I download the plannet using the torrent and then also seed 
it, what happens when the next planet comes available? Do I need to 
use a different torrent to download it again or will the one I have be 
updated?
Each planet file has its own torrent. A new planet file means using a 
new torrent to download it.


To simplify downloading the lastest planet, you'll find a 
planet-latest.pbf.torrent similar to the planet-latest.pbf (it's a 
symlink to the last planet available).


If you want to automate downloading new planet thru torrents, for 
example to seed them, there's a rss.xml file you can use. Many 
bittorrent clients support RSS to automate downloads.


--
Christian Quest - OpenStreetMap France


___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Testing torrents for the planet dump

[Maarten Deen]

On 2020-02-09 16:17, Christian Quest wrote:

A couple of weeks ago, I've (re) started sharing planet dump files
using Bittorrent to test an alternative way to distribute our planet
dumps to reduce the bandwidth load on the OSMF servers.

Here is a short summary after 2 weeks tests...

The torrents are generated a few hours after the planet dump
availability on planet.openstreetmap.org server (time needed to
download the original file to generate the torrent).


A question about updates of the torrent. The planet changes weekly. 
Suppose I download the plannet using the torrent and then also seed it, 
what happens when the next planet comes available? Do I need to use a 
different torrent to download it again or will the one I have be 
updated?


Regards,
Maarten

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

[OSM-talk] Testing torrents for the planet dump

[Christian Quest]

A couple of weeks ago, I've (re) started sharing planet dump files using 
Bittorrent to test an alternative way to distribute our planet dumps to 
reduce the bandwidth load on the OSMF servers.


Here is a short summary after 2 weeks tests...

The torrents are generated a few hours after the planet dump 
availability on planet.openstreetmap.org server (time needed to download 
the original file to generate the torrent).


I'm seeding it from a couple of servers which allows to spread the dump 
file faster. Seeding is also done thru the original https link and main 
mirrors (this is the "web seed" feature supported by some bittorrent 
clients).


After the initial seed, downloading the planet file can be as short as 
30/40mn if you have a good downlink. After 3 days, there's currently 15 
seeds, allowing event faster or more balanced downloads (just did one in 
only 15mn).


If you have downloaded the planet dump using the torrent or plan to do 
so or want to test it, please share your results on the wiki discussion: 
https://wiki.openstreetmap.org/wiki/Talk:Planet.osm#Torrent


The torrents are located at https://osm.cquest.org/torrents/

--
Christian Quest - OpenStreetMap France


___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

[OSM-talk] weeklyOSM #498 2020-01-28-2020-02-03

[weeklyteam]

The weekly round-up of OSM news, issue # 498,
is now available online in English, giving as always a summary of a lot of 
things happening in the openstreetmap world:

 http://www.weeklyosm.eu/en/archives/12843/

Enjoy! 

Did you know that you can also submit messages for the weeklyOSM? Just log in 
to https://osmbc.openstreetmap.de/login with your OSM account. Read more about 
how to write a post here: 
http://www.weeklyosm.eu/this-news-should-be-in-weeklyosm 

weeklyOSM? 
who: https://wiki.openstreetmap.org/wiki/WeeklyOSM#Available_Languages 
where?: 
https://umap.openstreetmap.fr/en/map/weeklyosm-is-currently-produced-in_56718#2/8.6/108.3
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [Talk-GB] Panorama Mapping Party with TrekView - May 23 - Ashurst, New Forest, UK

[Nick Whitelegg]

Hello Jez,

I think it probably would, as it would be of interest to open mapping 
enthusiasts, aims to collect open panoramic data, and OSM is used to connect 
the panoramas together - so I don't see why not.

I'll put it up when I have a chance.

Nick


From: Jez Nicholson 
Sent: 08 February 2020 11:39
To: Nick Whitelegg 
Cc: osm-talk ; Talk-GB 
Subject: Re: [Talk-GB] Panorama Mapping Party with TrekView - May 23 - Ashurst, 
New Forest, UK

Nice hookup with Trek Viewdoes this warrant adding to 
https://wiki.openstreetmap.org/wiki/Current_events as a mapping party? I always 
like seeing UK events on there.

On Fri, Feb 7, 2020 at 5:37 PM Nick Whitelegg 
mailto:nick.whitel...@solent.ac.uk>> wrote:
Hello everyone,

As some of you may know I am developing OpenTrailView 
(https://opentrailview.org), a pure 100% FOSS StreetView-like application for 
off-road routes such as hiking trails which uses OpenStreetMap data to 
auto-connect the panoramas together.

Recently I've been working with Trek View (trekview.org), 
an organisation which aims to capture panoramas of the natural world.
In their words: TrekView is a not-for-profit organisation using the power of 
panoramic photography to help educate and protect against further destruction 
of our beautiful planet. In 2020, they're launching Trekker Camp. Think virtual 
field trips. Trekker Camp will design and deliver immersive learning 
experiences to give students (7-11) the necessary understanding and skills to 
tackle the world's most pressing issues, from ocean health to climate change.

TrekView loan 360 camera packs (using the GoPro Fusion) to allow people to 
capture imagery of the natural world, from off-road routes including hiking 
routes and rivers. As well as Google Street View, TrekView's software now 
allows contributors to upload to OpenTrailView.

On to the most important aspect of this post. On May 23rd, and inspired by OSM 
mapping parties, we're organising a Panorama Mapping Party at Ashurst, New 
Forest, Hampshire, UK, with the aim of intensively capturing panoramic imagery 
of the paths and trails in the area which will then be uploaded to 
OpenTrailView. As OSM coverage in the area is exceptionally good, this should 
then result in the creation of extensive walk-through tours of the area.

The form will be similar to mapping parties. The plan is to meet at 11:00 
(there's a train which arrives from London at the local station, Ashurst New 
Forest, at around 10:45), plan, capture imagery and then get together in the 
pub afterwards.

More details (with OSM map showing location): 
https://www.trekview.org/blog/2020/pano-party-new-forest-uk-may-23-2020/


So if you're interested in 360 photography and OSM, then do come along! 360 
camera packs will be available to borrow and use on the day, or if you have 
your own device, please bring it along.

Thanks,
Nick

___
Talk-GB mailing list
talk...@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-gb
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] OTG rule, borders & mountains existing | Re: Crimea situation - on the ground

[Colin Smale]

On 2020-02-09 04:26, Joseph Eisenberg wrote:

>> Re: "on a government map, by legal / statutory decree, from data 
>> authoritatively published on a website"
> 
> These examples are not "good practice" sources for openstreetmap.
> While many mappers import data from such sources, there is no "value
> added" in the case that mappers are unable to confirm if the
> government or "authoritative" data is accurate or inaccurate. Since
> the data in Openstreetmap can be changed at any time, and often by
> mistakes caused by new mappers, the authoritative database or source
> will always be better for database users to consult directly, unless
> openstreetmap can improve the originally imported data by checking it
> against reality.

I beg to differ. Importing positions of accurately and authoritatively
surveyed objects gives us calibration points for our more manual work.
We are all warned about distortions and offsets in aerial imagery, and
99% of our on-the-ground mappers will be using consumer-grade GPS. If
the location of an admin boundary has been surveyed to centimetre
accuracy as lat X / lon Y, the presence of this in the OSM database,
plus an indication of its authoritative source, gives an invaluable
frame of reference. If Joe Bloggs comes along with his smartphone and
locates it at X+dX,Y+dY he needs to understand that it is he who has the
inferior data, and he should refrain from "improving" OSM by changing
the location of the boundary. If other objects like rivers, highways etc
should probably coincide with the admin boundary but don't, Joe Bloggs
needs to consider that the professionally surveyed data is more likely
to be correct before moving the admin boundary in OSM to fit his
imperfect data. 

Besides, OSM strives not only to be "complete" but also "useful". If
imports can increase the usefulness of OSM, it is likely to positively
impact its adoption. So what's not to like? 

A subject often ignored in OSM is defining what me mean by "data
quality." Quality is always relative to some definition of perfection.
Is a point entered by an "OTG mapper" with a smartphone, of higher
quality than a definitive, authoritative survey? 

> Remember, this is the "good practice" page we are talking about
> editing, not the "how things really are done" page: we want to focuse
> on the "Gold Standard", best practices.

Irrespective of the discussion above, Best Practises and Gold Standards
can often usefully be illustrated by negative examples. Standards have
quality too! A good standard will be unambiguous; one that is vague and
open to a lot of interpretation is not a good standard.___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk