Re: [OSM-dev] UTF8 problem with last night's daily .osc

2008-08-30 Thread Brett Henderson
Frederik, I just saw your emails but I'm about to head out and can't 
look at it now, I'll try to take a look tomorrow but it sounds like 
you've already diagnosed the problem.

If you can organise for the problematic way to be deleted or fixed in 
the database I'll re-create all changesets since that point (I just have 
to modify the timestamp file and kick off the extract again).  A DB 
update will presumably require TomH, I don't have write access (and 
would rather not touch it if I did).

Frederik Ramm wrote:
 Hi,

 Frederik Ramm wrote:
   
 Closer inspection reveals that this is a tag value that has been 
 truncated at character #255, which happens to be in the MIDST of an 
 UTF-8 sequence. Ouch! Who truncates tags to 255 characters?
 

 It's a bit embarassing to keep talking to myself here but in case anyone 
 else is interested:

 The culprit is way #26604650 which was newly created with Potlatch 
 0.10b, apparently with the tag value being truncated in the middle of an 
 UTF-8 sequence, which makes any Osmosis processing of the resulting diff 
 files (and probably also planet files?) impossible (the parser aborts).

 It also seems to create intermittent problems with the API. Just now 
 (check email headers for exact time) about one in five requests of

 $ wget -O- http://www.openstreetmap.org/api/0.5/way/26604650/history

 fails with an internal server error, while the same number of requests 
 against randomly selected other ways work fine all the time. Is this 
 strange or what?

 Bye
 Frederik

 ___
 dev mailing list
 dev@openstreetmap.org
 http://lists.openstreetmap.org/listinfo/dev
   


___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] [Tilesathome] [EMAIL PROTECTED] API still turn off...

2008-08-30 Thread 80n
XAPI has been serving an increased number of ?map style requests in the last
few hours.

It's probably not as fast as the main API for this, but it should scale well
under load.

I hope the new XAPI server will be on-line before the end of the weekend as
well, so that will provide some more capacity.

Now, don't all rush at once...

80n

On Sat, Aug 30, 2008 at 6:29 AM, spaetz [EMAIL PROTECTED] wrote:

 The queue length is at 0 for all API instances but hasn't been turned on
 again yet, I guess we can't deny TomH some sleep.

 It's extremly frustrating for me too to see the [EMAIL PROTECTED] API to be 
 given the
 lowest prioriy. neither of the other [EMAIL PROTECTED] infrastructure runs on 
 OSM-owned
 or controlled stuff, so the API is really all we need and use from OSM...

 And I am looking forward to a performant read-only api that does the map
 call. I know there are at least 2 attempts by Ian dees and blarson to get
 one and if other's want to chime in and help that would be great. There's at
 least 2 modular tasks: 1) answer queries efficiently and 2) populate the
 database with initial osm files b)keep database synched with .osc diffs.

 If you want to render a few tilesets inbetween, you can turn on the XAPI
 fallbackin your configuration file as this is still working.

 I'll take a break and get to my real life until the API works again
 spaetz

 ___
 Tilesathome mailing list
 [EMAIL PROTECTED]
 http://lists.openstreetmap.org/listinfo/tilesathome

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] UTF8 problem with last night's daily .osc

2008-08-30 Thread Richard Fairhurst
Frederik Ramm wrote:

 Frederik Ramm wrote:
 Closer inspection reveals that this is a tag value that has been
 truncated at character #255, which happens to be in the MIDST of an
 UTF-8 sequence. Ouch! Who truncates tags to 255 characters?

 It's a bit embarassing to keep talking to myself here but in case  
 anyone
 else is interested:

 The culprit is way #26604650 which was newly created with Potlatch
 0.10b, apparently with the tag value being truncated in the middle  
 of an
 UTF-8 sequence

Well, the relevant bit of the migration is

 create_table current_way_tags, myisam_table do |t|
   t.column id, :bigint, :limit = 64
   t.column k,  :string,:default = , :null  
= false
   t.column v,  :string,:default = , :null  
= false
 end

and a :string means a MySQL 255-character VARCHAR (http:// 
www.orthogonalthought.com/blog/index.php/2007/06/mysql-and-ruby-on- 
rails-datatypes/)... so yes, that'll be why it's happening.

So I guess the solution is either for Osmosis to conform to Postel's  
Law; or to change the datatype (presumably breaks indexing?); or for  
Potlatch/amf_controller, which don't currently have any limit on key/ 
value lengths (well, 64k :) ), to preprocess keys/values by  
truncating at the nearest UTF-8 boundary before 255 bytes.  
Suggestions welcome as to how this should be done.

cheers
Richard

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] openstreetbugs to the main-side

2008-08-30 Thread Pierre-André Jacquod
Ulf Lamping a écrit :
 If the one running the openstreetbug server is willing to face the 
 additional traffic, I'm all for adding a link (or maybe even better a 
 tab) to the main site.
Actually a tab would be great, I think more intuitive and with less 
inhibition for the user. But would need some text near the main map 
saying that there is the possibility to announce an error...
regards
Pierre-André


___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] Icons for graveyards

2008-08-30 Thread Patrick Kilian
Hi all,

is this the wrong mailing list to propose a change like that? Did I miss
some important feedback? Is somebody important (like the maintainer of
the stylesheet) on holiday?
Or did I just waste two days on a change which is listed as priority one
in the wiki but nobody cares about?


Patrick Petschge Kilian


___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] UTF8 problem with last night's daily .osc

2008-08-30 Thread Frederik Ramm
Hi,

Richard Fairhurst wrote:
 Well, the relevant bit of the migration is
[...]

I looked that up myself but didn't go the extra length to find out that 
string actually means 255 characters only... I always assumed that we 
allowed longer values. There goes the freedom of stuffing your sales 
brochure into a tag value ;-)

(Nodes currently do allow longer values but that's because they are 
still all lumped together and not broken out into their own node_tags 
table, right?)

 So I guess the solution is either for Osmosis to conform to Postel's  
 Law; 

I know it is not Osmosis' fault because Osmosis just uses standard XML 
parsing but I'm a bit unhappy about the gigantic waste of processing 
power incurred by all this parsing stuff - it seems anyone touching the 
XML file with a proper library has to actually decipher the UTF8 
sequences, whereas if I just want to use Osmosis to split out a section 
of the data or apply a patch, I would be perfectly happy if it would 
process whatever data is there *without* trying to make sense of it more 
than it absolutely has to. (I don't think there can ever be a double or 
single quote anywhere in bytes 2-n of a n-byte UTF-8 sequence, can there?)

I guess I'll have to resort to dirty regex parsingif I want that.

 or for  
 Potlatch/amf_controller, which don't currently have any limit on key/ 
 value lengths (well, 64k :) ), to preprocess keys/values by  
 truncating at the nearest UTF-8 boundary before 255 bytes.  

I have verified that the regular API suffers from the same problem. 
Upload something with an UTF-8 character at position 255 and you get 
into trouble. So actually the API needs to do something as well, either 
truncate the data properly, or at least return an error to the client if 
you try to set a value of more than 255 bytes.

I think it could be done in the add_tag_keyval method of the 
way/relation controllers. I am actually in favour of returning an error 
instead of silently truncating data because I think we should only 
return ok if we have stored all data that was sent.

Returning an error should be no more difficult than throwing an 
exception there if v.length exceeds 255, only thing I am not sure about 
is whether Ruby will try to be smart and return the length not in bytes 
but in characters...?

If one wanted to silently truncate, then I believe the rule is: You may 
cut a string between neighbouring characters a and b if b is in 
the range 0..127 (0x00..0x7f - regular one-byte character) or 192..253 
(0xc0..0xfd - first byte of UTF-8 sequence). Anything outside these 
ranges may be a follow-on byte of an UTF-8 sequence.

Bye
Frederik

-- 
Frederik Ramm  ##  eMail [EMAIL PROTECTED]  ##  N49°00'09 E008°23'33

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] UTF8 problem with last night's daily .osc

2008-08-30 Thread Martijn van Oosterhout
On Sat, Aug 30, 2008 at 11:51 AM, Frederik Ramm [EMAIL PROTECTED] wrote:
 I know it is not Osmosis' fault because Osmosis just uses standard XML
 parsing but I'm a bit unhappy about the gigantic waste of processing
 power incurred by all this parsing stuff - it seems anyone touching the
 XML file with a proper library has to actually decipher the UTF8
 sequences, whereas if I just want to use Osmosis to split out a section
 of the data or apply a patch, I would be perfectly happy if it would
 process whatever data is there *without* trying to make sense of it more
 than it absolutely has to. (I don't think there can ever be a double or
 single quote anywhere in bytes 2-n of a n-byte UTF-8 sequence, can there?)

UTF-8 parsing is trivial. Read first byte, lookup table tells you the
length. Copy that many extra bytes. No, a double quote can't appear
anywhere in a sequence, that's why it complained. Otherwise we would
just have seen mysterious data corruption.

Sure, your regex parsing wouldn't have failed on this particular test,
but the moment you want to actually use the result to load into JOSM,
osm2pgsql, coastline checker, it will barf anyway, so you may as well
get it right the first time.

 I have verified that the regular API suffers from the same problem.
 Upload something with an UTF-8 character at position 255 and you get
 into trouble. So actually the API needs to do something as well, either
 truncate the data properly, or at least return an error to the client if
 you try to set a value of more than 255 bytes.

Can you set mysql to strict mode so it throws an error instead of
silently truncating?

 If one wanted to silently truncate, then I believe the rule is: You may
 cut a string between neighbouring characters a and b if b is in
 the range 0..127 (0x00..0x7f - regular one-byte character) or 192..253
 (0xc0..0xfd - first byte of UTF-8 sequence). Anything outside these
 ranges may be a follow-on byte of an UTF-8 sequence.

That's about right.

Have a nice day,
-- 
Martijn van Oosterhout [EMAIL PROTECTED] http://svana.org/kleptog/

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev


[OSM-dev] correcting relation after 412 error

2008-08-30 Thread ivom
Hi,

I am trying to find out the reason why uploading modifications to relation 
(id=5286) through josm consistently results in the following error:

upload to: http://www.openstreetmap.org/api/0.5/relation/5286...connected
got return: 412 with id 5286

Suggestions to fix these kind of errors are appriated.

Kind regards,
Ivom

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] Icons for graveyards

2008-08-30 Thread Patrick Kilian
Hi,

 yeah, try posting to talk -- i don't think Steve Chiltern reads dev.
Thanks, I'l do that. I just thought that improvements like that are 
development.

 Oh and make it clear in the subject line that you've attached a patch
 -- that always gets people's attention :-)
Hehe. Thanks for the hint.

Patrick Petschge Kilian

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] UTF8 problem with last night's daily .osc

2008-08-30 Thread Karl Newman
On Sat, Aug 30, 2008 at 2:51 AM, Frederik Ramm [EMAIL PROTECTED] wrote:

 Hi,

 Richard Fairhurst wrote:
  Well, the relevant bit of the migration is
 [...]

 I looked that up myself but didn't go the extra length to find out that
 string actually means 255 characters only... I always assumed that we
 allowed longer values. There goes the freedom of stuffing your sales
 brochure into a tag value ;-)

 (Nodes currently do allow longer values but that's because they are
 still all lumped together and not broken out into their own node_tags
 table, right?)

  So I guess the solution is either for Osmosis to conform to Postel's
  Law;

 I know it is not Osmosis' fault because Osmosis just uses standard XML
 parsing but I'm a bit unhappy about the gigantic waste of processing
 power incurred by all this parsing stuff - it seems anyone touching the
 XML file with a proper library has to actually decipher the UTF8
 sequences, whereas if I just want to use Osmosis to split out a section
 of the data or apply a patch, I would be perfectly happy if it would
 process whatever data is there *without* trying to make sense of it more
 than it absolutely has to. (I don't think there can ever be a double or
 single quote anywhere in bytes 2-n of a n-byte UTF-8 sequence, can there?)

 I guess I'll have to resort to dirty regex parsingif I want that.

  or for
  Potlatch/amf_controller, which don't currently have any limit on key/
  value lengths (well, 64k :) ), to preprocess keys/values by
  truncating at the nearest UTF-8 boundary before 255 bytes.

 I have verified that the regular API suffers from the same problem.
 Upload something with an UTF-8 character at position 255 and you get
 into trouble. So actually the API needs to do something as well, either
 truncate the data properly, or at least return an error to the client if
 you try to set a value of more than 255 bytes.

 I think it could be done in the add_tag_keyval method of the
 way/relation controllers. I am actually in favour of returning an error
 instead of silently truncating data because I think we should only
 return ok if we have stored all data that was sent.

 Returning an error should be no more difficult than throwing an
 exception there if v.length exceeds 255, only thing I am not sure about
 is whether Ruby will try to be smart and return the length not in bytes
 but in characters...?

 If one wanted to silently truncate, then I believe the rule is: You may
 cut a string between neighbouring characters a and b if b is in
 the range 0..127 (0x00..0x7f - regular one-byte character) or 192..253
 (0xc0..0xfd - first byte of UTF-8 sequence). Anything outside these
 ranges may be a follow-on byte of an UTF-8 sequence.

 Bye
 Frederik


If I recall correctly, the database column is not actually set for UTF-8
(but is double-encoded to return actual UTF-8 to the client...). Wouldn't it
be a better long-term fix to change the database to UTF-8 (or whatever),
then presumably MySql wouldn't allow invalid sequences to be stored? Still
would be a good idea to raise an error if the length was too long, though.

Karl
___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] correcting relation after 412 error

2008-08-30 Thread ivom
Thanks for the reference Steven.  I tried to remove the way from the
relation in josm, but when I try and upload the same error gets logged. As
this relation is broken it does not seem possible to remove members from
it.

From the history of the relation
http://api.openstreetmap.org/api/0.5/relation/5286/history I would like to
ry to get the data from 2008-06-28T07:40:04+01:00 back and leave the edit
of 2008-07-26T20:08:37+01:00 as invalid. Is this a possible path to
follow? And if it is a reasonable solution how can I get back the
relation from 2008-06-28T07:40:04+01:00?

Starting from scratch and create a new relation is in this case to much
since this relation is huge when considering the number of members (it's a
long-distance bicycling path)

Regards,
Ivom


On Sat, 30 Aug 2008, Steven te Brinke wrote:

 Probably because way 23374110 is in there twice (one time with role backward 
 and one time without a role). Removing one of them using josm should fix the 
 problem.
 Currently there are some broken relations in the database and because the api 
 does not accept these, changes can only be made if they are fixed first.

 Greetings,
 Steven


 ivom schreef:
 Hi,
 
 I am trying to find out the reason why uploading modifications to relation 
 (id=5286) through josm consistently results in the following error:
 
 upload to: http://www.openstreetmap.org/api/0.5/relation/5286...connected
 got return: 412 with id 5286
 
 Suggestions to fix these kind of errors are appriated.
 
 Kind regards,
 Ivom
 
 ___
 dev mailing list
 dev@openstreetmap.org
 http://lists.openstreetmap.org/listinfo/dev
 



___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] Icons for graveyards

2008-08-30 Thread Lauri Hahne
I can currently see one problem. The buddhists, taoist, sikh, and some
else to a lesser extent look much darker than some others such as
christian and pastafarian. This could be overcome by adjusting the
darker green colour in the dark ones to a lighter one. Another option
would be to make the icons somehow lighter or make them tile less
often.

2008/8/28 Patrick Kilian [EMAIL PROTECTED]:
 Hello,

 Finally I was able to get a working mapnik installation. Using that I
 found some bugs in the stylesheet. Those are fixed in the updated patch
 which is attached to this mail.

 Also attached are the osm file which I use for testing. The bounding box  to
 render the file is ll = (9.833, 49.780, 9.846, 49.790). The resulting png is
 too big for this list and can be seen at http://www.petschge.de/image.png

 Patrick Petschge Kilian



 ___
 dev mailing list
 dev@openstreetmap.org
 http://lists.openstreetmap.org/listinfo/dev





-- 
Lauri Hahne

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] Icons for graveyards

2008-08-30 Thread Robert Funnell
Alternatively the Christian crosses could perhaps be made bigger.

Presumably the Pastafarian version won't be used often; I would in 
fact suggest omitting it to avoid pointlessly upsetting someone.

I might also suggest using X's instead of crosses for Unknown.

How to handle mixed? non-denominational? military? communist?

- Robert

On Sat, 30 Aug 2008, Lauri Hahne wrote:

 I can currently see one problem. The buddhists, taoist, sikh, and some
 else to a lesser extent look much darker than some others such as
 christian and pastafarian. This could be overcome by adjusting the
 darker green colour in the dark ones to a lighter one. Another option
 would be to make the icons somehow lighter or make them tile less
 often.

 2008/8/28 Patrick Kilian [EMAIL PROTECTED]:
 Hello,

 Finally I was able to get a working mapnik installation. Using that I
 found some bugs in the stylesheet. Those are fixed in the updated patch
 which is attached to this mail.

 Also attached are the osm file which I use for testing. The bounding box  to
 render the file is ll = (9.833, 49.780, 9.846, 49.790). The resulting png is
 too big for this list and can be seen at http://www.petschge.de/image.png

 Patrick Petschge Kilian


___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] correcting relation after 412 error

2008-08-30 Thread Frederik Ramm
Hi,

 I am trying to find out the reason why uploading modifications to relation 
 (id=5286) through josm consistently results in the following error:

Steven has already mentioned one of the problems for you, but as you 
rightly say there is another problem. I'll give you (and others) a quick 
run-down of what I tend to do in these cases.

1. Download the relation into a file.
2. Download every way referenced, and check whether all ways actually 
exist. (They do in this case.)
3. Download all nodes referenced by all these ways and check whether 
they actually exist. (They do in this case, but I had situations where 
one of the ways that make up the relation were broken because one of 
their nodes had been deleted and you'll see 412 on relation upload in 
these cases.)
4. If all this doesn't help, then create a new relation and find the 
culprit through binary search, i.e. first upload the relation with all 
its members. On 412, comment out (using !-- and --) the first half of 
members in the relation XML, re-upload. If this works, then the problem 
is in the first half; now comment out only the first quarter, and so on, 
until you have found the one member where upload works if it is 
commented out, and upload doesn't work if you have it in.

In this case, the offending way seems to be way #22949135. Remove it 
from the relation and it will upload. Why? That way looks perfectly 
normal...

?xml version=1.0 encoding=UTF-8?
osm version=0.5 generator=OpenStreetMap server
   way id=22949135 visible=true 
timestamp=2008-08-05T15:14:49+01:00 user=Djohnnie
 nd ref=247267461/
 nd ref=247267463/
 tag k=highway v=residential/
 tag k=created_by v=Potlatch 0.10b/
 tag k=name v=Elisabethplein/
   /way
/osm

... UNTIL you look at its history:

osm version=0.5 generator=OpenStreetMap server
   way id=22949135 visible=true 
timestamp=2008-02-11T20:53:59+00:00 user=ivom
 nd ref=247267461/
 nd ref=247267462/
 nd ref=247267463/
 tag k=created_by v=JOSM/
 tag k=highway v=residential/
   /way
   way id=22949135 visible=true 
timestamp=2008-05-04T20:42:39+01:00 user=ivom
 nd ref=247267461/
 nd ref=247267462/
 nd ref=247267463/
 tag k=name v=Elisabethplein/
 tag k=created_by v=JOSM/
 tag k=highway v=residential/
   /way
   way id=22949135 visible=true 
timestamp=2008-05-05T21:40:24+01:00 user=ivom
 nd ref=247267461/
 nd ref=247267462/
 nd ref=262395126/
 nd ref=247267463/
 tag k=name v=Elisabethplein/
 tag k=created_by v=JOSM/
 tag k=highway v=residential/
   /way
   way id=22949135 visible=true 
timestamp=2008-08-05T15:14:49+01:00 user=Djohnnie
 nd ref=283641193/
 nd ref=247267461/
 nd ref=247267462/
 nd ref=28364119/
 nd ref=262395126/
 nd ref=247267463/
 tag k=name v=Elisabethplein/
 tag k=created_by v=Potlatch 0.10b/
 tag k=highway v=residential/
   /way
/osm

This is one of those unfortunate ways where the current version from 
history is not identical to the real current version. This would not 
happen in a transactional database environment but we don't yet have 
such, so discrepancies are possible.

Some of the nodes referenced by the latest history version of this way 
have been deleted, and the current table has been updated to reflect 
this, but the history table still links to non-existent nodes.

And that's why you cannot upload your relation. Fix the way - actually 
if you just re-upload it in its current state this will fix it - or 
remove it from the relation.

Bye
Frederik

-- 
Frederik Ramm  ##  eMail [EMAIL PROTECTED]  ##  N49°00'09 E008°23'33

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] Icons for graveyards

2008-08-30 Thread Robert Funnell
On Sat, 30 Aug 2008, Patrick Kilian wrote:

 ...
 Presumably the Pastafarian version won't be used often; I would in fact 
 suggest omitting it to avoid pointlessly upsetting someone.
 Not including one religion because other religions might not like it is a 
 very slippery slope.

Do you really mean Rastafarian and not Pastafarian? I can't make 
out what the symbol is and assumed that it is spaghetti.

 I might also suggest using X's instead of crosses for Unknown.
 Sorry, image.png was an old version. In the updated version unknown shows as 
 headstones.

Perhaps the headstones could be a bit bigger? It would make them more 
recognizable, and also avoid the lightness problem.
Are there any religions which frown on headstones, as opposed to 
plaques, mausoleums, etc? An 'X' would certainly be neutral, and very 
recognizable.

 How to handle mixed? non-denominational? military? communist?
 Mixed: If one denomination dominates use that symbol, otherwise use the 
 General/Unknown one.
 Non-denominational: same as General/Unknown
 Military: I could make an icon with crossed swords. Although that might 
 collide some day with the rendering of battlegrounds.

Both your suggestion and your reservation about it are good. From a 
religious point of view, they are often non-denominational, which was 
actually my point. They could also be General/Unknown. The important 
thing would be to document these usages.

- Robert


___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] Icons for graveyards

2008-08-30 Thread Lauri Hahne
2008/8/30 Marc Schütz [EMAIL PROTECTED]:
  Presumably the Pastafarian version won't be used often; I would in
  fact suggest omitting it to avoid pointlessly upsetting someone.
 Not including one religion because other religions might not like it is
 a very slippery slope.

 Yes, but as there are probably no pastafarian cemeteries (yet?), the only 
 applications would obviously be vandalism and bad jokes.

But honestly, if somebody gets upset from including a symbol like that
then it really is their own fault. It's incredible how people want to
omit stuff because somebody potentially could get upset even if there
was no real reason for anybody to get upset. Sensitivity is one thing
and paranoia or fascistic politically correctness are another thing.



-- 
Lauri Hahne

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] [Tilesathome] [EMAIL PROTECTED] API still turn off...

2008-08-30 Thread Ian Dees
On Sat, Aug 30, 2008 at 12:29 AM, spaetz [EMAIL PROTECTED] wrote:


 And I am looking forward to a performant read-only api that does the map
 call. I know there are at least 2 attempts by Ian dees and blarson to get
 one and if other's want to chime in and help that would be great. There's at
 least 2 modular tasks: 1) answer queries efficiently and 2) populate the
 database with initial osm files b)keep database synched with .osc diffs.


I'm currently importing Wednesday's planet into my DB. I'm at 12.3M nodes
(out of 262M) so far, running at 2200 nodes/sec.

The map call works and is pretty speedy (but could definitely be faster).

I'll let you know when it's ready to use...
___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] UTF8 problem with last night's daily .osc

2008-08-30 Thread Joachim Zobel
Am Samstag, den 30.08.2008, 08:39 -0700 schrieb Karl Newman:
 If I recall correctly, the database column is not actually set for
 UTF-8 (but is double-encoded to return actual UTF-8 to the client...).
 Wouldn't it be a better long-term fix to change the database to UTF-8
 (or whatever), then presumably MySql wouldn't allow invalid sequences
 to be stored? Still would be a good idea to raise an error if the
 length was too long, though.

This means you are storing utf8 as raw in latin1 database tables, right?
That is the root of the problem and what should be fixed. If the
database would consider each utf8 character a single character, it
couldn't truncate them.

Sincerely,
Joachim



___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] UTF8 problem with last night's daily .osc

2008-08-30 Thread Grant Slater
Karl Newman wrote:
 If I recall correctly, the database column is not actually set for 
 UTF-8 (but is double-encoded to return actual UTF-8 to the client...). 
 Wouldn't it be a better long-term fix to change the database to UTF-8 
 (or whatever), then presumably MySql wouldn't allow invalid sequences 
 to be stored? Still would be a good idea to raise an error if the 
 length was too long, though.


Yes, this is the case.

It's on the list of planned fixes for the 0.6 API. More on the wiki soon.

/ Grant


___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev


[josm-dev] how to create icons?

2008-08-30 Thread A Morris
I have implemented a new 'action' in josm, but it needs an icon in the
menu. What is the preferred way to create these icons?

Cheers, Aled.

___
josm-dev mailing list
josm-dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/josm-dev


Re: [josm-dev] how to create icons?

2008-08-30 Thread Dirk Stöcker
On Sat, 30 Aug 2008, A Morris wrote:

 I have implemented a new 'action' in josm, but it needs an icon in the
 menu. What is the preferred way to create these icons?

a) You don't need an icon. Pass null and it works without.
b) I use gimp to create icons.

Ciao
-- 
http://www.dstoecker.eu/ (PGP key available)

___
josm-dev mailing list
josm-dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/josm-dev


Re: [josm-dev] how to create icons?

2008-08-30 Thread Christoph Eckert
Hi,

 I have implemented a new 'action' in josm, but it needs an icon in the
 menu. What is the preferred way to create these icons?

instead of Dirk, I try to get  modify some SVG in Inkscape. Then I export it 
as png.

Best regards,

ce


___
josm-dev mailing list
josm-dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/josm-dev


Re: [josm-dev] how to create icons?

2008-08-30 Thread A Morris
Thanks for the suggestions - I've cobbled something together in gimp,
though the quality reflects my artistic talent, I'm afraid.

Aled.

___
josm-dev mailing list
josm-dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/josm-dev


[josm-dev] Patch for Align nodes straight and parallel action

2008-08-30 Thread A Morris
This is a patch against the latest svn, please try it out. Bug reports welcome.

Basically it adds a new action Align nodes straight and parallel. It
requires that two or more ways are selected (this is required, as
otherwise the algorithm wouldn't know which nodes were in which line).
The nodes will be moved so that they are straight and parallel. You
can optionally select a subset of nodes in the way, and only the
selected nodes will be moved.

I intend to make a 'Align nodes curved and parallel' next, but I
thought I'd start with an easier one :-)

Aled.
=== added file 'images/alignparallel.png'
Binary files images/alignparallel.png   1970-01-01 00:00:00 + and 
images/alignparallel.png  2008-08-30 23:01:27 + differ
=== added file 
'src/org/openstreetmap/josm/actions/AlignNodesStraightAndParallel.java'
--- src/org/openstreetmap/josm/actions/AlignNodesStraightAndParallel.java   
1970-01-01 00:00:00 +
+++ src/org/openstreetmap/josm/actions/AlignNodesStraightAndParallel.java   
2008-08-30 23:46:11 +
@@ -0,0 +1,193 @@
+// AlignWaysStraightAndParallel.java
+
+//Copyright (C) 2008 Aled Morris [EMAIL PROTECTED]
+//
+//This program is free software: you can redistribute it and/or modify
+//it under the terms of the GNU General Public License as published by
+//the Free Software Foundation, either version 2 of the License, or
+//(at your option) any later version.
+//
+//This program is distributed in the hope that it will be useful,
+//but WITHOUT ANY WARRANTY; without even the implied warranty of
+//MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+//GNU General Public License for more details.
+//
+//You should have received a copy of the GNU General Public License
+//along with this program.  If not, see http://www.gnu.org/licenses/.
+
+package org.openstreetmap.josm.actions;
+
+import static org.openstreetmap.josm.tools.I18n.tr;
+
+import java.awt.event.ActionEvent;
+import java.awt.event.KeyEvent;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.HashSet;
+import java.util.LinkedList;
+
+import javax.swing.JOptionPane;
+
+import org.openstreetmap.josm.Main;
+import org.openstreetmap.josm.command.Command;
+import org.openstreetmap.josm.command.MoveCommand;
+import org.openstreetmap.josm.command.SequenceCommand;
+import org.openstreetmap.josm.data.Bearing;
+import org.openstreetmap.josm.data.osm.Node;
+import org.openstreetmap.josm.data.osm.OsmPrimitive;
+import org.openstreetmap.josm.data.osm.Way;
+
+public final class AlignNodesStraightAndParallel extends JosmAction {
+
+   public AlignNodesStraightAndParallel() {
+   super(tr(Align Nodes in Parallel Lines), alignparallel, 
tr(Move the selected nodes to make them straight and parallel), 
KeyEvent.VK_EQUALS, 0, true);
+   }
+
+   /**
+* User must select two or more ways. The algorithm moves all the nodes 
in the ways so they are straight and parallel.
+* The user may also optionally select a subset of nodes within the 
way(s); only the selected nodes will be moved.
+*/
+   public void actionPerformed(ActionEvent e) {
+   CollectionOsmPrimitive sel = Main.ds.getSelected();
+   HashSetNode selectedNodes = new HashSetNode();
+   CollectionWay selectedWays = new LinkedListWay();
+   
+   for (OsmPrimitive osm : sel)
+   {
+   if (osm instanceof Node) {
+   selectedNodes.add((Node)osm);
+   }
+   else if (osm instanceof Way) {
+   selectedWays.add(((Way)osm));
+   }
+   }
+   
+   if (selectedWays.size()  2) {
+   JOptionPane.showMessageDialog(Main.parent, tr(Please 
select at least two ways (and optionally, a subset of the way's nodes).));
+   return;
+   }
+   
+   ArrayListArrayListNode nodeChains = new 
ArrayListArrayListNode();
+   
+   // Go through each way that was selected, and create a node 
chain containing either: a) all the nodes in the way
+   // or b) just the nodes that were selected in the way.
+   for (Way way : selectedWays) {
+   ArrayListNode nodeChain = new ArrayListNode();
+   
+   // see if any of the nodes in the way were selected; if 
so, add them to the chain.
+   for (Node node : way.nodes) {
+   if (selectedNodes.contains(node)) {
+   nodeChain.add(node);
+   selectedNodes.remove(node); // Ensure 
no node is added twice (can happen in circular ways)
+   }
+   }
+   
+