> To put OSM data live to xmpp ist very simple and I don't think it's
> expensive.
Coming back to this a bit older topic. XMPP is server-based solution, so you
will overload some server. Why not use good old and free "Kazaa" network, in
its Skype groupchat re-incarnation, so the delivery channel w
This is an implementation of this for Live Journal:
http://updates.sixapart.com/
Lets you connect to a TCP port and get live XML feed of all updates on
Livejournal.. Has some cool features, such as discarding data from the
stream when you can't keep up.
/Erik
___
On Wed, May 13, 2009 at 1:48 PM, Matt Amos wrote:
> its better to get this done without the main db and the rails_port
> code diverging too much, so i'm looking for methods which are as
> un-invasive as possible.
I agree. Since it seems like a huge amount of work to augment the current
infrastr
On Wed, May 13, 2009 at 7:36 PM, Ian Dees wrote:
> On Wed, May 13, 2009 at 1:33 PM, Matt Amos wrote:
>> On Wed, May 13, 2009 at 7:30 PM, Ian Dees wrote:
>> > On Wed, May 13, 2009 at 1:27 PM, Matt Amos wrote:
>> >> sorry, i wasn't clear in my question: why triggers in particular,
>> >> rather th
On Wed, May 13, 2009 at 1:33 PM, Matt Amos wrote:
> On Wed, May 13, 2009 at 7:30 PM, Ian Dees wrote:
> > On Wed, May 13, 2009 at 1:27 PM, Matt Amos wrote:
> >> sorry, i wasn't clear in my question: why triggers in particular,
> >> rather than one of the many other features that the DB provides
On Wed, May 13, 2009 at 7:30 PM, Ian Dees wrote:
> On Wed, May 13, 2009 at 1:27 PM, Matt Amos wrote:
>> sorry, i wasn't clear in my question: why triggers in particular,
>> rather than one of the many other features that the DB provides for
>> doing this?
>
> Mostly because it would allow us to u
On Wed, May 13, 2009 at 1:27 PM, Matt Amos wrote:
> On Wed, May 13, 2009 at 7:13 PM, Ian Dees wrote:
> > On Wed, May 13, 2009 at 1:09 PM, Matt Amos wrote:
> >>
> >> why via triggers?
> >
> > Because the database is the only aggregation point for the data. There
> are
> > many API servers (which
On Wed, May 13, 2009 at 6:30 PM, Frederik Ramm wrote:
> Matt Amos wrote:
>>
>> i think if we can get the delay on the diffs down from 5 mins to under
>> 2 mins then there's no reason why streaming can't be built on top of
>> the diffs and be able to support all the things people want to do with
>>
On Wed, May 13, 2009 at 7:13 PM, Ian Dees wrote:
> On Wed, May 13, 2009 at 1:09 PM, Matt Amos wrote:
>>
>> why via triggers?
>
> Because the database is the only aggregation point for the data. There are
> many API servers (which would be the ideal spot for creating this data
> feed), but my init
On Wed, May 13, 2009 at 1:09 PM, Matt Amos wrote:
> why via triggers?
>
Because the database is the only aggregation point for the data. There are
many API servers (which would be the ideal spot for creating this data
feed), but my initial thought was that it was quite cumbersome to try and
aggr
On Wed, May 13, 2009 at 7:05 PM, Ian Dees wrote:
> On Wed, May 13, 2009 at 12:18 PM, Matt Amos wrote:
>> On Wed, May 13, 2009 at 5:15 PM, Tom Hughes wrote:
>> > Ian Dees wrote:
>> >> The whole argument I'm making is that after the initial
>> >> implementation**, streaming the data is a lot less
On Wed, May 13, 2009 at 12:30 PM, Frederik Ramm wrote:
> can any of today's
> hip & trendy messaging protocols be used to painlessly notify anyone who
> is interested that "there's a new diff ready", instead of having
> over-eager scripts poll the directory every 10 seconds?
The server would ne
On Wed, May 13, 2009 at 12:18 PM, Matt Amos wrote:
> On Wed, May 13, 2009 at 5:15 PM, Tom Hughes wrote:
> > Ian Dees wrote:
> >> The whole argument I'm making is that after the initial
> >> implementation**, streaming the data is a lot less resource intensive
> >> than what we are currently doin
Hi,
Matt Amos wrote:
> i think if we can get the delay on the diffs down from 5 mins to under
> 2 mins then there's no reason why streaming can't be built on top of
> the diffs and be able to support all the things people want to do with
> streaming.
What you are talking about is "simulated strea
On Wed, May 13, 2009 at 5:15 PM, Tom Hughes wrote:
> Ian Dees wrote:
>> The whole argument I'm making is that after the initial
>> implementation**, streaming the data is a lot less resource intensive
>> than what we are currently doing. Perhaps I don't have the whole picture
>> of what goes on in
On Wed, May 13, 2009 at 4:40 PM, andrzej zaborowski wrote:
> In a different mail you said:
>> Ian Dees wrote:
>>> OSM isn't about the geodata, it's about the data. That includes the fact
>>> that it is in the geographic domain, but it also means that we can
>>> manipulate it or store it however we
Frederik Ramm wrote:
> I fully agree that streaming is probably a niche thing, a nice-to-have
> and not a must-have, and I have no problem if the idea is treated as a
> small priority. But dismissing it just because your imagination is too
> limited...?
It's fine for people to discuss it. What
Ian Dees wrote:
> The whole argument I'm making is that after the initial
> implementation**, streaming the data is a lot less resource intensive
> than what we are currently doing. Perhaps I don't have the whole picture
> of what goes on in the backend, but at some point the changeset XML
> f
Bernhard zwischenbrugger wrote:
> To put OSM data live to xmpp ist very simple and I don't think it's
> expensive.
>
> An easy way would be to post it to a xmpp groupchat:
>
>
> geodata here
>
>
> After login it's just a copy to a tcp socket port 5222.
> Everybody who wants the data can log
Jonathan Bennett schrieb:
> andrzej zaborowski wrote:
> > Cool visualisation tools don't have to comply with a) or b), they just
>
>> need to be cool :)
>>
>
> So cool you're prepared to pay for the infrastructure to support it?
>
>
>
To put OSM data live to xmpp ist very simple and I d
On Wed, May 13, 2009 at 10:33 AM, Jonathan Bennett <
openstreet...@jonno.cix.co.uk> wrote:
> andrzej zaborowski wrote:
> > Cool visualisation tools don't have to comply with a) or b), they just
> > need to be cool :)
>
> So cool you're prepared to pay for the infrastructure to support it?
>
I th
Hi,
Jonathan Bennett wrote:
> andrzej zaborowski wrote:
>> You might be missing out on a cool visualisation tool though (maybe
>> what Bernhard is trying doing is similar), but that's the only use
>> case I can think of right now.
>
> How does that help anyone a) use the data, or b) improve the d
On Wed, May 13, 2009 at 10:28 AM, Jonathan Bennett <
openstreet...@jonno.cix.co.uk> wrote:
> Ian Dees wrote:
> > Woah! Since when can OSM tell me what sort of applications I can and
> > can't write with the open source data that OSM is providing**?
>
> You're not being told what to do with the dat
2009/5/13 Jonathan Bennett :
> andrzej zaborowski wrote:
> > Cool visualisation tools don't have to comply with a) or b), they just
>> need to be cool :)
>
> So cool you're prepared to pay for the infrastructure to support it?
I didn't say that. I said there *are* things you're missing out on.
andrzej zaborowski wrote:
> Cool visualisation tools don't have to comply with a) or b), they just
> need to be cool :)
So cool you're prepared to pay for the infrastructure to support it?
--
Jonathan (Jonobennett)
___
talk mailing list
talk@openstr
2009/5/13 Jonathan Bennett :
> andrzej zaborowski wrote:
>> You might be missing out on a cool visualisation tool though (maybe
>> what Bernhard is trying doing is similar), but that's the only use
>> case I can think of right now.
>
> How does that help anyone a) use the data, or b) improve the da
Ian Dees wrote:
> Woah! Since when can OSM tell me what sort of applications I can and
> can't write with the open source data that OSM is providing**?
You're not being told what to do with the data, but it's being suggested
to you that you can't have it in a particular, resource-intensive format
On Wed, May 13, 2009 at 10:11 AM, Jonathan Bennett <
openstreet...@jonno.cix.co.uk> wrote:
> Frederik Ramm wrote:
> > But saying: "We don't intend to support this because we cannot think of
> > an application that absolutely requires it", is quite un-OSM, is it not?
>
> Qualify "application" as "a
2009/5/13 Ian Dees :
> 2009/5/13 Iván Sánchez Ortega
>>
>> As a wise man once said, "all problems in computer science can be solved
>> by
>> adding another indirection layer".
>>
>> If you really really want a stream, I'm positive it can be hacked with a
>> couple of scripts and the minutely diffs
andrzej zaborowski wrote:
> You might be missing out on a cool visualisation tool though (maybe
> what Bernhard is trying doing is similar), but that's the only use
> case I can think of right now.
How does that help anyone a) use the data, or b) improve the data? See
ITO's OSM Mapper if you want
2009/5/13 Jonathan Bennett :
> Ian Dees wrote:
>>> I don't think anybody has ever given a use case which requires such
>>> a stream and can't work with the diffs.
>>
>>
>> I agree, but the point is that minutely-diffs are a minute old. At some
>> point in the future someone will want to see
Frederik Ramm wrote:
> But saying: "We don't intend to support this because we cannot think of
> an application that absolutely requires it", is quite un-OSM, is it not?
Qualify "application" as "application which actually uses the geodata",
and it's not so far off the mark. We don't need a milli
2009/5/13 Iván Sánchez Ortega
> As a wise man once said, "all problems in computer science can be solved by
> adding another indirection layer".
>
> If you really really want a stream, I'm positive it can be hacked with a
> couple of scripts and the minutely diffs.
You have discovered one of my
El Miércoles, 13 de Mayo de 2009, Ian Dees escribió:
> [...] the point is that minutely-diffs are a minute old. At some point in
> the future someone will want to see the data in real time as a stream.
If you can't wait *one* minute to see the data, you have a very acute case of
OSMOCD, and you
Hi,
Peter Childs wrote:
> The Problem is that you can't rebuild the map from a continuing
> stream, This is the problem with Database Replication in general.
True, but maybe the stream use cases don't require that? Maybe it is
more important for an application to know in an instant where somethi
2009/5/13 Ian Dees :
> Ok, this I'll agree on. My original post was just to talk about it... not
> really to do it. But it sounds like we should take "baby" steps. Let's work
> on the minutely diffs first and if some crazy person comes up with a good
> use case for streaming, we can talk about it
Ian Dees wrote:
>> I don't think anybody has ever given a use case which requires such
>>a stream and can't work with the diffs.
>
>
> I agree, but the point is that minutely-diffs are a minute old. At some
> point in the future someone will want to see the data in real time as a
> stream
On Wed, May 13, 2009 at 2:41 AM, Tom Hughes wrote:
> Ian Dees wrote:
>
> I'd like to continue this part of the thread. As was discussed by
>> Frederik, I think the end goal should be a real-time OSM stream of what's
>> getting applied to the database. Doing that in a performant way is
>> relativ
On Wed, May 13, 2009 at 8:43 AM, Lennard wrote:
> Matt Amos wrote:
>
>> these might be of interest:
>>
>> http://matt.sandbox.cloudmade.com/
>
> Which would have been fine and dandy in the past, but somebody needs to
> nudge that one into life again, /me thinks.
yeah, sorry. its on my todo list ;
Hi
Maybe you like this:
http://datenkueche.com/osmlive/
If I get nice feedback I will make it zoomable.
Bernhard
___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk
Frederik Ramm wrote:
> Tom Hughes wrote:
>> It's a completely insane solution though. It we want to do it we
>> should just do it properly in the database not fart around with stupid
>> hacks in the rails code that break as soon as any updates are not done
>> via rails.
>
> Assuming for a mome
Matt Amos wrote:
> these might be of interest:
>
> http://matt.sandbox.cloudmade.com/
Which would have been fine and dandy in the past, but somebody needs to
nudge that one into life again, /me thinks.
--
Lennard
___
talk mailing list
talk@openstre
Ian Dees wrote:
> I'd like to continue this part of the thread. As was discussed by
> Frederik, I think the end goal should be a real-time OSM stream of
> what's getting applied to the database. Doing that in a performant way
> is relatively difficult (which is why we're using Osmosis and minut
Sorry, I lost the thread in Gmail here, but:
On Tue, May 12, 2009 at 7:53 PM, Matt Amos wrote:
> >> unless, of course, you're talking about twittering the updates. that
> >> would be teh moar ;-)
> >
>
I'd like to continue this part of the thread. As was discussed by Frederik,
I think the end go
Iván Sánchez Ortega wrote:
> El Martes, 12 de Mayo de 2009, Bernhard zwischenbrugger escribió:
>> Is there a possibility to get all new data entered to OSM in realtime?
>
> No, AFAIK. The closest you can get is the minutely diffs (all the changes
> done
> in the last minute).
It would be cool t
On Wed, May 13, 2009 at 1:27 AM, Frederik Ramm wrote:
> Matt,
>
>> the least invasive way is to use the minutely diffs, as it doesn't
>> touch the API or DB servers at all.
>
> Sure, but they are (a) delayed by 5 minutes and (b) broken ;-)
we're working on both (a) and (b) at the moment... we'll
Hi,
Shaun McDonald wrote:
> I really don't want to be attempting to try and collate the edits from
> the api server logs. For a start they don't contain all the information
> that you would need.
I was not talking about the web server logs, but special log files
created solely for the purpose
Matt,
> the least invasive way is to use the minutely diffs, as it doesn't
> touch the API or DB servers at all.
Sure, but they are (a) delayed by 5 minutes and (b) broken ;-)
I was initially opposed to the concept of diffs. I remember a developer
meeting in Essen in 2007 where I rather violent
Frederik,
On 13 May 2009, at 01:01, Frederik Ramm wrote:
> Hi,
>
> Tom Hughes wrote:
>> It's a completely insane solution though. It we want to do it we
>> should
>> just do it properly in the database not fart around with stupid
>> hacks in
>> the rails code that break as soon as any updates
On Wed, May 13, 2009 at 1:10 AM, Bernhard zwischenbrugger
wrote:
> Hi
>> http://planet.openstreetmap.org/minute/
>>
> That's perfect!!!
>
> Is there also the a file with the *newest* data?
> Or do I have to read the timestamp file?
reading the timestamp.txt is the best way to do it.
> I don't wa
Hi,
Bernhard zwischenbrugger wrote:
> I don't want to synchronize a database. The thing I'm thinking
> about is a visualization of the current activity.
Google for "OSMAware" for some inspiration!
Bye
Frederik
--
Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33"
___
Hi
> http://planet.openstreetmap.org/minute/
>
That's perfect!!!
Is there also the a file with the *newest* data?
Or do I have to read the timestamp file?
I don't want to synchronize a database. The thing I'm thinking
about is a visualization of the current activity.
Bernhard
__
Hi,
Tom Hughes wrote:
> It's a completely insane solution though. It we want to do it we should
> just do it properly in the database not fart around with stupid hacks in
> the rails code that break as soon as any updates are not done via rails.
Assuming for a moment that the database was our b
On Wed, May 13, 2009 at 12:36 AM, Frederik Ramm wrote:
> To be just slightly more constructive, the least invasive way of
> querying the API for new data only without changing the code would be to
> make multi-GETs for batches of object IDs just above the highest known
> object ID. That would prob
Frederik Ramm wrote:
> Probably the best way to have a live feed - and a technique that has
> been discussed on dev about two years ago - would be to have the rails
> code log all successful database operations into a file which could then
> be retrieved by an independent daemon and fed into wh
Hi,
Tom Hughes wrote:
> Anybody trying such a stunt will be liable to summary blocking when
> caught however.
I was waiting for that ;-)
To be just slightly more constructive, the least invasive way of
querying the API for new data only without changing the code would be to
make multi-GETs fo
2009/5/13 Iván Sánchez Ortega :
> El Miércoles, 13 de Mayo de 2009, andrzej zaborowski escribió:
>> From the minutely diffs if a new way is created and deleted in the same
>> minute, you would never know about it
>
> Can't you get the changeset IDs from the diff, then query the API to know the
> ex
andrzej zaborowski wrote:
> You can in theory extract all edits, at higher than 1 minute
> granularity, from http://www.openstreetmap.org/browse/changesets
> together with all history. (From the minutely diffs if a new way is
> created and deleted in the same minute, you would never know about it
On Tue, May 12, 2009 at 11:50 PM, andrzej zaborowski wrote:
> You can in theory extract all edits, at higher than 1 minute
> granularity, from http://www.openstreetmap.org/browse/changesets
> together with all history. (From the minutely diffs if a new way is
> created and deleted in the same min
El Miércoles, 13 de Mayo de 2009, andrzej zaborowski escribió:
> From the minutely diffs if a new way is created and deleted in the same
> minute, you would never know about it
Can't you get the changeset IDs from the diff, then query the API to know the
exact time of the changeset?
--
--
2009/5/13 Ian Dees :
> On Tue, May 12, 2009 at 4:56 PM, Bernhard zwischenbrugger
> wrote:
>>
>> Hi all
>>
>> Is there a possibility to get all new data entered to OSM in realtime?
>>
>> If someone adds a new road, building, restaurant,... I would like to
>> have this data.
>>
>> There was talks to
El Martes, 12 de Mayo de 2009, Bernhard zwischenbrugger escribió:
> Is there a possibility to get all new data entered to OSM in realtime?
No, AFAIK. The closest you can get is the minutely diffs (all the changes done
in the last minute).
> If someone adds a new road, building, restaurant,... I
On Tue, May 12, 2009 at 4:56 PM, Bernhard zwischenbrugger <
b...@datenkueche.com> wrote:
> Hi all
>
> Is there a possibility to get all new data entered to OSM in realtime?
>
> If someone adds a new road, building, restaurant,... I would like to
> have this data.
>
> There was talks to put this ki
Hi all
Is there a possibility to get all new data entered to OSM in realtime?
If someone adds a new road, building, restaurant,... I would like to
have this data.
There was talks to put this kind of data to the jabber network.
Is this already available?
Bernhard
__
64 matches
Mail list logo
