Re: [mapserver-users] Performance in regular expressions or an alternative way to select a list of features
Hello David,
I wasn't aware of the "IN" syntax. It looks cleaner, and hopefully will
be better. I think in the end I will try all sollutions and see which
one is the fastest for a given (large) expression.
Thangs again,
Adrian
Fawcett, David wrote:
Adrian,
One method that I have used is to use an IN statement in my expression
and then use a variable to populate the list of IDs in the statement.
I create a class like this:
CLASS
NAME "Low"
EXPRESSION ('[COUNTY_FIP]' in '%group1%')
OUTLINECOLOR 0 0 0
COLOR 255 204 204
END
And then in the URL calling the map, I include &group1=27001,27003,27005
I actually use this with five classes (five different URL vars) to
create a thematic map of counties based entirely on data passed in
through the URL.
I have no idea on how the performance of this method compares to what
you have done, but it might be worth a try. There are only 87
counties in Minnesota, so that is the largest number it gets, but the
performance isn't bad.
David.
-Original Message-
*From:* mapserver-users-boun...@lists.osgeo.org
[mailto:mapserver-users-boun...@lists.osgeo.org] *On Behalf Of
*Adrian Popa
*Sent:* Tuesday, July 14, 2009 1:15 AM
*To:* mapserver-users@lists.osgeo.org
*Subject:* [mapserver-users] Performance in regular expressions or
an alternative way to select a list of features
Hello everyone,
Here's my problem: I'm trying to highlight segments from a line
layer by using an expression in a specific class. This portion of
the mapfile is dynamically generated and when it is done, it is
sent to mapserver for rendering.
My problem is that I have to select between 10 - 400 features at a
time and I noticed when I have a lot of features there is a severe
performance degradation in mapserver (takes a lot of time to
render or even times out).
Right now, my expression is built using regular expressions:
something like:
*EXPRESSION /^ITEM1$|^ITEM2$|^ITEM3$|^ITEM4$/*
This works ok, but as I said has a performance penalty when you
reach ~400 items. My data is selected from a shapefile layer
which has about 5500 items.
Since I wouldn't be using the regular expressions at full capacity
(I'm matching the full name), I might rewrite the expression using
something like:
*EXPRESSION ( ([NAME]=="ITEM1") OR ([NAME]=="ITEM2") OR
([NAME]=="ITEM3") OR ([NAME]=="ITEM4") )*
From the documentation I see that:
/Regular expression with MapServer work similarly to string
comparison, but allow more complex operation. They are slower than
pure string comparisons, but might be still faster than logical
expression. As with the string comparison use regular expressions,
a FILTERITEM or a CLASSITEM has to defined, respectively.
/I would like to know if there is an efficient way of selecting a
list of elements from a layer, or what are your recommendations.
Also - have there been significant changes in performance for this
issue from mapserver 4.10 (I am now migrating to mapserver 5.4)?
Thanks,
Adrian
___
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users
RE: [mapserver-users] Performance in regular expressions or an alternative way to select a list of features
Adrian,
One method that I have used is to use an IN statement in my expression
and then use a variable to populate the list of IDs in the statement.
I create a class like this:
CLASS
NAME "Low"
EXPRESSION ('[COUNTY_FIP]' in '%group1%')
OUTLINECOLOR 0 0 0
COLOR 255 204 204
END
And then in the URL calling the map, I include &group1=27001,27003,27005
I actually use this with five classes (five different URL vars) to
create a thematic map of counties based entirely on data passed in
through the URL.
I have no idea on how the performance of this method compares to what
you have done, but it might be worth a try. There are only 87 counties
in Minnesota, so that is the largest number it gets, but the performance
isn't bad.
David.
-Original Message-
From: mapserver-users-boun...@lists.osgeo.org
[mailto:mapserver-users-boun...@lists.osgeo.org] On Behalf Of Adrian
Popa
Sent: Tuesday, July 14, 2009 1:15 AM
To: mapserver-users@lists.osgeo.org
Subject: [mapserver-users] Performance in regular expressions or
an alternative way to select a list of features
Hello everyone,
Here's my problem: I'm trying to highlight segments from a line
layer by using an expression in a specific class. This portion of the
mapfile is dynamically generated and when it is done, it is sent to
mapserver for rendering.
My problem is that I have to select between 10 - 400 features at
a time and I noticed when I have a lot of features there is a severe
performance degradation in mapserver (takes a lot of time to render or
even times out).
Right now, my expression is built using regular expressions:
something like:
EXPRESSION /^ITEM1$|^ITEM2$|^ITEM3$|^ITEM4$/
This works ok, but as I said has a performance penalty when you
reach ~400 items. My data is selected from a shapefile layer which has
about 5500 items.
Since I wouldn't be using the regular expressions at full
capacity (I'm matching the full name), I might rewrite the expression
using something like:
EXPRESSION ( ([NAME]=="ITEM1") OR ([NAME]=="ITEM2") OR
([NAME]=="ITEM3") OR ([NAME]=="ITEM4") )
From the documentation I see that:
Regular expression with MapServer work similarly to string
comparison, but allow more complex operation. They are slower than pure
string comparisons, but might be still faster than logical expression.
As with the string comparison use regular expressions, a FILTERITEM or a
CLASSITEM has to defined, respectively.
I would like to know if there is an efficient way of selecting a
list of elements from a layer, or what are your recommendations.
Also - have there been significant changes in performance for
this issue from mapserver 4.10 (I am now migrating to mapserver 5.4)?
Thanks,
Adrian
___
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Performance in regular expressions or an alternative way to select a list of features
Hello Nikolai, As far as you know - the Layers/Classes are processed in paralel (threads)? Would it even be more efficient not to use expressions, but to use simple string matchings and dynamically generate 10-400 Classes in a layer (one class per item)? What would be the best/most scalable approach? Thank you, Adrian Nikolai Nikolov wrote: Hi Adrian, I have used long EXPRESSION list in the past, but my data are “static”, so eventually I modified the map data and removed the polygons I didn’t want to render. I could suggest to you to use in your map file several LAYERs or CLASSes to “spread around” long EXPRESSION lists. Those LAYERs or CLASSes would be identical but for the EXPRESSION lists. You could put a limit of let’s say 10 selected features per CLASS definition. Best regards, Nick *From:* mapserver-users-boun...@lists.osgeo.org [mailto:mapserver-users-boun...@lists.osgeo.org] *On Behalf Of *Adrian Popa *Sent:* 14 July 2009 07:15 *To:* mapserver-users@lists.osgeo.org *Subject:* [mapserver-users] Performance in regular expressions or an alternative way to select a list of features Hello everyone, Here's my problem: I'm trying to highlight segments from a line layer by using an expression in a specific class. This portion of the mapfile is dynamically generated and when it is done, it is sent to mapserver for rendering. My problem is that I have to select between 10 - 400 features at a time and I noticed when I have a lot of features there is a severe performance degradation in mapserver (takes a lot of time to render or even times out). Right now, my expression is built using regular expressions: something like: *EXPRESSION /^ITEM1$|^ITEM2$|^ITEM3$|^ITEM4$/* This works ok, but as I said has a performance penalty when you reach ~400 items. My data is selected from a shapefile layer which has about 5500 items. Since I wouldn't be using the regular expressions at full capacity (I'm matching the full name), I might rewrite the expression using something like: *EXPRESSION ( ([NAME]=="ITEM1") OR ([NAME]=="ITEM2") OR ([NAME]=="ITEM3") OR ([NAME]=="ITEM4") )* >From the documentation I see that: /Regular expression with MapServer work similarly to string comparison, but allow more complex operation. They are slower than pure string comparisons, but might be still faster than logical expression. As with the string comparison use regular expressions, a FILTERITEM or a CLASSITEM has to defined, respectively. /I would like to know if there is an efficient way of selecting a list of elements from a layer, or what are your recommendations. Also - have there been significant changes in performance for this issue from mapserver 4.10 (I am now migrating to mapserver 5.4)? Thanks, Adrian ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
RE: [mapserver-users] Performance in regular expressions or an alternative way to select a list of features
Hi Adrian, I have used long EXPRESSION list in the past, but my data are "static", so eventually I modified the map data and removed the polygons I didn't want to render. I could suggest to you to use in your map file several LAYERs or CLASSes to "spread around" long EXPRESSION lists. Those LAYERs or CLASSes would be identical but for the EXPRESSION lists. You could put a limit of let's say 10 selected features per CLASS definition. Best regards, Nick _ From: mapserver-users-boun...@lists.osgeo.org [mailto:mapserver-users-boun...@lists.osgeo.org] On Behalf Of Adrian Popa Sent: 14 July 2009 07:15 To: mapserver-users@lists.osgeo.org Subject: [mapserver-users] Performance in regular expressions or an alternative way to select a list of features Hello everyone, Here's my problem: I'm trying to highlight segments from a line layer by using an expression in a specific class. This portion of the mapfile is dynamically generated and when it is done, it is sent to mapserver for rendering. My problem is that I have to select between 10 - 400 features at a time and I noticed when I have a lot of features there is a severe performance degradation in mapserver (takes a lot of time to render or even times out). Right now, my expression is built using regular expressions: something like: EXPRESSION /^ITEM1$|^ITEM2$|^ITEM3$|^ITEM4$/ This works ok, but as I said has a performance penalty when you reach ~400 items. My data is selected from a shapefile layer which has about 5500 items. Since I wouldn't be using the regular expressions at full capacity (I'm matching the full name), I might rewrite the expression using something like: EXPRESSION ( ([NAME]=="ITEM1") OR ([NAME]=="ITEM2") OR ([NAME]=="ITEM3") OR ([NAME]=="ITEM4") ) >From the documentation I see that: Regular expression with MapServer work similarly to string comparison, but allow more complex operation. They are slower than pure string comparisons, but might be still faster than logical expression. As with the string comparison use regular expressions, a FILTERITEM or a CLASSITEM has to defined, respectively. I would like to know if there is an efficient way of selecting a list of elements from a layer, or what are your recommendations. Also - have there been significant changes in performance for this issue from mapserver 4.10 (I am now migrating to mapserver 5.4)? Thanks, Adrian ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
[mapserver-users] Performance in regular expressions or an alternative way to select a list of features
Hello everyone, Here's my problem: I'm trying to highlight segments from a line layer by using an expression in a specific class. This portion of the mapfile is dynamically generated and when it is done, it is sent to mapserver for rendering. My problem is that I have to select between 10 - 400 features at a time and I noticed when I have a lot of features there is a severe performance degradation in mapserver (takes a lot of time to render or even times out). Right now, my expression is built using regular expressions: something like: *EXPRESSION /^ITEM1$|^ITEM2$|^ITEM3$|^ITEM4$/* This works ok, but as I said has a performance penalty when you reach ~400 items. My data is selected from a shapefile layer which has about 5500 items. Since I wouldn't be using the regular expressions at full capacity (I'm matching the full name), I might rewrite the expression using something like: *EXPRESSION ( ([NAME]=="ITEM1") OR ([NAME]=="ITEM2") OR ([NAME]=="ITEM3") OR ([NAME]=="ITEM4") )* From the documentation I see that: /Regular expression with MapServer work similarly to string comparison, but allow more complex operation. They are slower than pure string comparisons, but might be still faster than logical expression. As with the string comparison use regular expressions, a FILTERITEM or a CLASSITEM has to defined, respectively. /I would like to know if there is an efficient way of selecting a list of elements from a layer, or what are your recommendations. Also - have there been significant changes in performance for this issue from mapserver 4.10 (I am now migrating to mapserver 5.4)? Thanks, Adrian ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
