Re: [PATCH] wurfl device detection build fixes and dummy library
Hi Christopher, thanks for all suggestions : I'll be working on a patch to address the various issues. Yes, the callback should be called (once with an existing header, once with a non existing one) as that is code to be tested as well : I'll include this issue in the patch. Best -Paul On Wed, Apr 24, 2019 at 6:08 PM Christopher Faulet wrote: > Le 23/04/2019 à 11:10, Willy Tarreau a écrit : > > Hi Paul, > > > > On Fri, Apr 19, 2019 at 06:45:22PM +0200, Paul Stephen Borile wrote: > >> Hi Willy, > >> > >> fine for me, thanks for the adjustments and no problem backporting this > to > >> 1.9. > >> I also confirm that the contact email address is working correctly. > > > > Fine thank you. I could finish the polishing (add USE_WURFL to the list > > of known and reported build options in the makefile) and I've > reintegrated > > the code now. > > > > You probably don't see the value in having the dummy library, but for > > developers it's invaluable. I have now updated my build script to build > > with USE_WURFL=1 by default so that I'll now see if anything causes > > warnings or errors there if we touch structures that are used by your > > code. > > > > It would be really awesome if Device Atlas and 51Degrees could do the > > same, as the build coverage becomes much better with very little effort > > for everyone. David, Ben, if you read this, please have a look at > > contrib/wurfl to get an idea of what is sufficient to have your code > > always built by anyone. Patches welcome :-) > > > Hi Paul, > > I quickly reviewed the wurfl integration. I tested it with the dummy > library. It really made my tests easier, many thanks. > > First, I have a segfault when I use the sample fetch "wurfl-get" at line > 521. It happens when I try to retrieve an unknown data (I mean not > listed in wurfl-information-list). Here is my config: > > global > ... > wurfl-data-file /usr/share/wurfl/wurfl.zip > wurfl-information-list wurfl_id model_name > ... > > frontend http > ... > http-request set-header X-WURFL-Properties > %[wurfl-get(wurfl_id,is_tablet)] # is_tablet is not in the list > > > Then, at the beginning of the wurfl sample fetches, the channel validity > must be checked calling the macro CHECK_HTTP_MESSAGE_FIRST(). Otherwise, > some processing can be performed on empty buffers or uninitialized data. > > Finally, the function ha_wurfl_retrieve_header() is not HTX aware. Take > a look at others HTTP sample fetches in src/http_fetch.c. > > Just a suggestion. It could be cool to call it from the dummy library, > in wurfl_lookup(). This way, we will be able to test this part. > > Regards, > -- > Christopher Faulet > -- Paul Stephen Borile Director, WURFL InFuze Product Development - Italy Scientiamobile Inc - paul.bor...@scientiamobile.com +39 348 7108474 Milano Office : +39 02 620227260 skype: paulstephenborile "Reducing complexity and size must be the goal in every step." Niklaus Wirth
Re: [PATCH] wurfl device detection build fixes and dummy library
Hi Willy, fine for me, thanks for the adjustments and no problem backporting this to 1.9. I also confirm that the contact email address is working correctly. Thanks! -Paul On Fri, Apr 19, 2019 at 4:46 PM Willy Tarreau wrote: > Hi Paul, > > On Thu, Apr 18, 2019 at 02:46:17PM +0200, Paul Stephen Borile wrote: > > please find attached to this email the 6 patches that cover various areas > > of restyling of > > the WURFL device detection feature for HAProxy. All patches can be back > > ported to 1.9 if necessary. > > Last patch is a dummy WURFL library that can be used to build/run haproxy > > compiled with the USE_WURFL option to make easier checking for any build > > problem in the future. > > We'll try to do the same and make sure that the module does not break > > builds again as happened in the past. > > So I gave a look to this patch set and had to perform a few adjustments > to make it work but now it looks OK. I'm attaching the changes I made so > that you can review them, they're all related to the dummy lib in order > to 1) fix its build and 2) ease the testing without having to modify the > build environment (since adding non-standard stuff into /usr/include or > /usr/lib is a no-go on most development environments). > > I figured that it was much simpler to build a ".a" from the file so that > it can naturally be loaded by the regular build process. I added the > ability to report the libwurfl version in "haproxy -vv" and when the > dummy lib is detected, it's explicitly mentioned "dummy library" there > so that you don't have to deal with false positives when users report > issues. I also added a little bit of doc explaining to haproxy devs > how to build with wurfl. This way I think it could be added by default > to any developer's build script so that it never breaks in the future. > > I'm attaching my changes. I'm fine with retrofitting them into your > patches if they look OK to you. Please just let me know if you're OK to > go with this (and if you're OK with me backporting this to 1.9 so that > we can fix 1.9 once for all). > > Thanks! > Willy > > PS: I've CCed the contact address in the maintainers file just to verify > that there is no typo there, please confirm that it was properly > received. > > -- Paul Stephen Borile Director, WURFL InFuze Product Development - Italy Scientiamobile Inc - paul.bor...@scientiamobile.com +39 348 7108474 Milano Office : +39 02 620227260 skype: paulstephenborile "Reducing complexity and size must be the goal in every step." Niklaus Wirth
[PATCH] wurfl device detection build fixes and dummy library
Hi All, Willy,
please find attached to this email the 6 patches that cover various areas
of restyling of
the WURFL device detection feature for HAProxy. All patches can be back
ported to 1.9 if necessary.
Last patch is a dummy WURFL library that can be used to build/run haproxy
compiled with the USE_WURFL option to make easier checking for any build
problem in the future.
We'll try to do the same and make sure that the module does not break
builds again as happened in the past.
-Paul
--
Paul Stephen Borile
Director, WURFL Product Development - Italy
Scientiamobile Inc - paul.bor...@scientiamobile.com +39 348 7108474
Milano Office : +39 02 620227260
skype: paulstephenborile
"Reducing complexity and size must be the goal in every step." Niklaus
Wirth
From d04f6a5d94f59c043a3819e7626981c41c6c2779 Mon Sep 17 00:00:00 2001
From: paulborile
Date: Thu, 18 Apr 2019 11:57:04 +0200
Subject: [PATCH 3/6] CLEANUP: wurfl: removed deprecated methods
last 2 major releases of libwurfl included a complete review of engine options with
the result of deprecating many features. The patch removes unecessary code and fixes
the documentation.
Can be backported on any version of haproxy.
---
doc/WURFL-device-detection.txt | 9 +--
doc/configuration.txt | 37 ++--
examples/wurfl-example.cfg | 10 +---
src/wurfl.c| 106 +++--
4 files changed, 15 insertions(+), 147 deletions(-)
diff --git a/doc/WURFL-device-detection.txt b/doc/WURFL-device-detection.txt
index c9686151..0d61db0a 100644
--- a/doc/WURFL-device-detection.txt
+++ b/doc/WURFL-device-detection.txt
@@ -23,25 +23,18 @@ These are the supported WURFL directives (see doc/configuration.txt) :
virtual capabilities, property names we plan to use in injected headers)
- wurfl-information-list-separator (character that will be
used to separate values in a response header, ',' by default).
-- wurfl-engine-mode (Sets the WURFL engine target. You can choose
- between "accuracy" and "performance","performance" by default)
- wurfl-cache-size (Sets the WURFL caching strategy)
- wurfl-patch-file [] (Sets the paths to custom WURFL patch files)
Sample configuration :
global
- wurfl-data-file /usr/share/wurfl/wurfl-eval.xml
+ wurfl-data-file /usr/share/wurfl/wurfl.zip
wurfl-information-list wurfl_id model_name
#wurfl-information-list-separator |
- wurfl-engine-mode performance
- #wurfl-engine-mode accuracy
-
- ## double LRU cache
- wurfl-cache-size 10,3
## single LRU cache
#wurfl-cache-size 10
## no cache
diff --git a/doc/configuration.txt b/doc/configuration.txt
index ccf614fd..5b99da47 100644
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -599,9 +599,7 @@ The following keywords are supported in the "global" section :
- wurfl-data-file
- wurfl-information-list
- wurfl-information-list-separator
- - wurfl-engine-mode
- wurfl-cache-size
- - wurfl-useragent-priority
* Performance tuning
- max-spread-checks
@@ -1246,10 +1244,6 @@ wurfl-information-list []*
- wurfl_api_version Contains a string representing the currently
used Libwurfl API version.
- - wurfl_engine_target Contains a string representing the currently
-set WURFL Engine Target. Possible values are
-"HIGH_ACCURACY", "HIGH_PERFORMANCE", "INVALID".
-
- wurfl_info A string containing information on the parsed
wurfl.xml and its full path.
@@ -1258,8 +1252,6 @@ wurfl-information-list []*
- wurfl_normalized_useragent The normalized useragent.
- - wurfl_useragent_priorityThe user agent priority used by WURFL.
-
Please note that this option is only available when haproxy has been compiled
with USE_WURFL=1.
@@ -1277,36 +1269,15 @@ wurfl-patch-file []
Please note that this option is only available when haproxy has been compiled
with USE_WURFL=1.
-wurfl-engine-mode { accuracy | performance }
- Sets the WURFL engine target. You can choose between 'accuracy' or
- 'performance' targets. In performance mode, desktop web browser detection is
- done programmatically without referencing the WURFL data. As a result, most
- desktop web browsers are returned as generic_web_browser WURFL ID for
- performance. If either performance or accuracy are not defined, performance
- mode is enabled by default.
-
- Please note that this option is only available when haproxy has been compiled
- with USE_WURFL=1.
-
-wurfl-cache-size [,]
- Sets the WURFL caching strategy. Here is the Useragent cache size, and
- is the internal device cache size. There are three possibilities here :
+wurfl-cache-size
+ Sets the WURFL Useragent cache size. For faster lookups, already processed user
+ agents are kept in
[PATCH] BUILD/MINOR : WURFL fix for build problems
Hi,
in attach patch for :
- build fix for 1.9/2.0 code base (0001)
- removed deprecated methods (0001)
- enabled multithreading mode (0001)
- added point of contact in MAINTAINERS file (0004)
Module is now tested in MT mode internally.
Thanks
-Paul
--
Paul Stephen Borile
Director, WURFL InFuze Product Development - Italy
Scientiamobile Inc - paul.bor...@scientiamobile.com +39 348 7108474
Milano Office : +39 02 620227260
skype: paulstephenborile
"Reducing complexity and size must be the goal in every step." Niklaus
Wirth
From 1951335fe4782a778caa8d504dcc55aaa40db653 Mon Sep 17 00:00:00 2001
From: paulborile
Date: Thu, 7 Mar 2019 14:47:46 +0100
Subject: [PATCH 1/4] wurfl module porting to 1.9
---
doc/WURFL-device-detection.txt | 10 +--
doc/configuration.txt | 36 +---
examples/wurfl-example.cfg | 11 +--
src/wurfl.c| 157 -
4 files changed, 44 insertions(+), 170 deletions(-)
diff --git a/doc/WURFL-device-detection.txt b/doc/WURFL-device-detection.txt
index c968615..908fb66 100644
--- a/doc/WURFL-device-detection.txt
+++ b/doc/WURFL-device-detection.txt
@@ -23,26 +23,18 @@ These are the supported WURFL directives (see doc/configuration.txt) :
virtual capabilities, property names we plan to use in injected headers)
- wurfl-information-list-separator (character that will be
used to separate values in a response header, ',' by default).
-- wurfl-engine-mode (Sets the WURFL engine target. You can choose
- between "accuracy" and "performance","performance" by default)
- wurfl-cache-size (Sets the WURFL caching strategy)
- wurfl-patch-file [] (Sets the paths to custom WURFL patch files)
Sample configuration :
global
- wurfl-data-file /usr/share/wurfl/wurfl-eval.xml
+ wurfl-data-file /usr/share/wurfl/wurfl.zip
wurfl-information-list wurfl_id model_name
#wurfl-information-list-separator |
- wurfl-engine-mode performance
- #wurfl-engine-mode accuracy
-
- ## double LRU cache
- wurfl-cache-size 10,3
- ## single LRU cache
#wurfl-cache-size 10
## no cache
#wurfl-cache-size 0
diff --git a/doc/configuration.txt b/doc/configuration.txt
index 8a23c0a..7c75483 100644
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -599,9 +599,7 @@ The following keywords are supported in the "global" section :
- wurfl-data-file
- wurfl-information-list
- wurfl-information-list-separator
- - wurfl-engine-mode
- wurfl-cache-size
- - wurfl-useragent-priority
* Performance tuning
- max-spread-checks
@@ -1236,10 +1234,6 @@ wurfl-information-list []*
- wurfl_api_version Contains a string representing the currently
used Libwurfl API version.
- - wurfl_engine_target Contains a string representing the currently
-set WURFL Engine Target. Possible values are
-"HIGH_ACCURACY", "HIGH_PERFORMANCE", "INVALID".
-
- wurfl_info A string containing information on the parsed
wurfl.xml and its full path.
@@ -1248,7 +1242,6 @@ wurfl-information-list []*
- wurfl_normalized_useragent The normalized useragent.
- - wurfl_useragent_priorityThe user agent priority used by WURFL.
Please note that this option is only available when haproxy has been compiled
with USE_WURFL=1.
@@ -1267,36 +1260,15 @@ wurfl-patch-file []
Please note that this option is only available when haproxy has been compiled
with USE_WURFL=1.
-wurfl-engine-mode { accuracy | performance }
- Sets the WURFL engine target. You can choose between 'accuracy' or
- 'performance' targets. In performance mode, desktop web browser detection is
- done programmatically without referencing the WURFL data. As a result, most
- desktop web browsers are returned as generic_web_browser WURFL ID for
- performance. If either performance or accuracy are not defined, performance
- mode is enabled by default.
-
- Please note that this option is only available when haproxy has been compiled
- with USE_WURFL=1.
-
-wurfl-cache-size [,]
- Sets the WURFL caching strategy. Here is the Useragent cache size, and
- is the internal device cache size. There are three possibilities here :
+wurfl-cache-size
+ Sets the WURFL Useragent cache size. For faster lookups, already processed user
+ agents are kept in a LRU cache :
- "0" : no cache is used.
- - : the Single LRU cache is used, the size is expressed in elements.
- - , : the Double LRU cache is used, both sizes are in elements. This is
- the highest performing option.
+ - : size of lru cache in elements.
Please note that this option is only available when haproxy has been compiled
with USE_WURFL=1.
-wurfl-useragent-priority { plain | sideloaded_browser }
- Tel
