Just to update on this discussion, if someone is going to read it in the future...
Apparently the support for projection was added in the mongo-c-driver library v1.5.0. With older versions it proved that using projection to specify the list of returned columns triggered errors. The master branch has a fix for working with older versions than 1.5.0, it will be backported. Cheers, Daniel On 01.03.18 15:21, Stephane wrote: > Thanks very much Daniel ! > This is working now, with both mongo server versions. > Kind regards, > Stephane > > On 1 March 2018 at 14:31, Daniel-Constantin Mierla <mico...@gmail.com > <mailto:mico...@gmail.com>> wrote: > > Hello, > > indeed, it was a similar case in another function. Can you fetch > again master and try? > > Cheers, > Daniel > > > On 01.03.18 13:32, Stephane wrote: >> Hello Daniel, >> >> Maybe a similar issue somewhere else in the code? >> >> 0(5462) ERROR: db_mongodb [mongodb_dbase.c:557]: >> db_mongodb_convert_bson(): field [projection] not found in result >> iterator >> >> 0(5462) ERROR: db_mongodb [mongodb_dbase.c:708]: >> db_mongodb_convert_result(): failed to convert bson at pos 0 >> >> 0(5462) ERROR: db_mongodb [mongodb_dbase.c:776]: >> db_mongodb_store_result(): failed to set the rows in result >> >> 0(5462) ERROR: db_mongodb [mongodb_dbase.c:965]: >> db_mongodb_query(): failed to store result >> >> 0(5462) ERROR: db_mongodb [mongodb_dbase.c:974]: >> db_mongodb_query(): failed to do the query >> >> >> Kind regards, >> Stephane >> >> On 1 March 2018 at 12:58, Daniel-Constantin Mierla >> <mico...@gmail.com <mailto:mico...@gmail.com>> wrote: >> >> Hello, >> >> thanks for testing -- the new error was on matching returned >> columns, as now they are stored in the projection field document. >> >> I just pushed a new patch, can you try again with latest master? >> >> Cheers, >> Daniel >> >> >> On 01.03.18 11:14, Stephane wrote: >>> Hello Daniel, >>> >>> The format of the mongo request is better I think. >>> But there is the following new error (with both mongo server >>> versions) >>> >>> 0(30473) ERROR: db_mongodb [mongodb_dbase.c:386]: >>> db_mongodb_get_columns(): field [projection] not found in >>> result iterator >>> >>> 0(30473) ERROR: db_mongodb [mongodb_dbase.c:752]: >>> db_mongodb_store_result(): failed to set the columns >>> >>> 0(30473) ERROR: db_mongodb [mongodb_dbase.c:945]: >>> db_mongodb_query(): failed to store result >>> >>> 0(30473) ERROR: db_mongodb [mongodb_dbase.c:954]: >>> db_mongodb_query(): failed to do the query >>> >>> >>> The pcap traces are in attachment. >>> Thanks ! >>> Stéphane >>> >>> >>> >>> On 1 March 2018 at 08:34, Daniel-Constantin Mierla >>> <mico...@gmail.com <mailto:mico...@gmail.com>> wrote: >>> >>> Hello, >>> >>> can you try with latest master branch or apply the patch >>> from the next commit to your branch? >>> >>> * >>> >>> https://github.com/kamailio/kamailio/commit/d50498cf483cd22ac4e62af5aaa86fc27654f640 >>> >>> <https://github.com/kamailio/kamailio/commit/d50498cf483cd22ac4e62af5aaa86fc27654f640> >>> >>> If possible, test it with both mongo server versions and >>> if works fine with both, then I am going to backport. >>> >>> Cheers, >>> Daniel >>> >>> >>> On 28.02.18 18:49, Stephane wrote: >>>> Hello Daniel, >>>> >>>> I tried with mongoc 1.8.0 (the trace I have sent) and >>>> also mongoc 1.9.2. The result is the same : a "failed >>>> to parse" with as indication "unrecognized field >>>> 'table_version'" when using mongo server version 3.6.2. >>>> >>>> Both mongoc versions are running fine with a mongo >>>> server version 3.0.15. >>>> >>>> Kind regards, >>>> Stephane >>>> >>>> >>>> >>>> >>>> On 28 February 2018 at 15:25, Daniel-Constantin Mierla >>>> <mico...@gmail.com <mailto:mico...@gmail.com>> wrote: >>>> >>>> Hello, >>>> >>>> I see in the pcap that the returned error contains: >>>> >>>> Failed to parse: { find: "version", filter: { >>>> table_name: "location" }, table_version: 1, >>>> $readPreference: { mode: "secondaryPreferred" }, >>>> $db: "kamailio" }. Unrecognized field 'table_version'. >>>> >>>> What is the version of mongoc library? I tried to >>>> quick search on google and found a similar report : >>>> >>>> - >>>> https://github.com/mongodb/mongo-php-driver/issues/516 >>>> <https://github.com/mongodb/mongo-php-driver/issues/516> >>>> >>>> On the other hand, it works with same lib but for >>>> different versions of mongo server ... so might be >>>> something different. >>>> >>>> Cheers, >>>> Daniel >>>> >>>> >>>> On 27.02.18 11:26, Stephane wrote: >>>>> Hello Daniel, >>>>> >>>>> You will find two pcap files in attachment. I am >>>>> using the same mongoc library version in both cases. >>>>> Let me know if you need more info. >>>>> >>>>> Kind regards, >>>>> Stephane >>>>> >>>>> On 27 February 2018 at 09:01, Daniel-Constantin >>>>> Mierla <mico...@gmail.com >>>>> <mailto:mico...@gmail.com>> wrote: >>>>> >>>>> Hello, >>>>> >>>>> can you send here the requests sent by >>>>> kamailio for the two mongodb versions? To >>>>> compare and see where it can be fixed. >>>>> >>>>> Also, were you using the same mongoc library >>>>> version in both cases? >>>>> >>>>> Cheers, >>>>> Daniel >>>>> >>>>> >>>>> On 26.02.18 12:57, Stephane wrote: >>>>>> Hello, >>>>>> I made some additional tests. >>>>>> The problem is only present when kamailio is >>>>>> facing mongo release 3.2 (or higher). >>>>>> With mongo version 3.0 (or lower), the >>>>>> request sent by kamailio is different: it >>>>>> contains $query and $table_version elements >>>>>> and mongo accepts it. >>>>>> Kind regards, >>>>>> Stephane >>>>>> >>>>>> On 23 February 2018 at 10:48, Stephane >>>>>> <mandalav...@gmail.com >>>>>> <mailto:mandalav...@gmail.com>> wrote: >>>>>> >>>>>> Hello Daniel, >>>>>> >>>>>> I took a pcap trace. And I see the >>>>>> following in the answer sent by mongo >>>>>> >>>>>> Failed to parse: { find: "version", >>>>>> filter: { table_name: "location" }, >>>>>> table_version: 1, $db: "kamailio", >>>>>> $readPreference: { mode: >>>>>> "primaryPreferred" }, $clusterTime: { >>>>>> clusterTime: Timestamp(1519379980, 1), >>>>>> signature: { hash: BinData(0, >>>>>> 0000000000000000000000000000000000000000), >>>>>> keyId: 0 } } }. Unrecognized field >>>>>> >>>>>> 'table_version'.codecodeNameFailedToParseoperationTimeZ$clusterTimeXclusterTimeZsignature3hashkeyId >>>>>> >>>>>> Kind regards, >>>>>> Stephane >>>>>> >>>>>> On 23 February 2018 at 09:39, >>>>>> Daniel-Constantin Mierla >>>>>> <mico...@gmail.com >>>>>> <mailto:mico...@gmail.com>> wrote: >>>>>> >>>>>> Hello, >>>>>> >>>>>> I don't know if you can see the >>>>>> result sent back by mongodb in its >>>>>> logs, it doesn't show it there. Maybe >>>>>> you can sniff the traffic on the >>>>>> network and see if mongodb sends the >>>>>> proper response back. Kamailio logs >>>>>> like nothing was retrieved. >>>>>> >>>>>> Cheers, >>>>>> Daniel >>>>>> >>>>>> >>>>>> On 22.02.18 18:02, Stephane Focant wrote: >>>>>>> Hello, >>>>>>> I have an issue when I try to use >>>>>>> kamailio 5.1 with usrloc and >>>>>>> db_mongdb modules. >>>>>>> When I try to start kamailio, I get >>>>>>> the following error: >>>>>>> >>>>>>> 0(22216) DEBUG: <core> [db.c:314]: >>>>>>> db_do_init2(): connection >>>>>>> 0x7f735b5c4100 not found in pool >>>>>>> 0(22216) DEBUG: db_mongodb >>>>>>> [mongodb_connection.c:55]: >>>>>>> db_mongodb_new_connection(): >>>>>>> connection open to: >>>>>>> mongodb://10.0.12.149 >>>>>>> >>>>>>> <http://10.0.12.149>,10.0.12.150,10.0.12.151/kamailio?replicaSet=sre_location&slaveOk=true&readPreference=primaryPreferred >>>>>>> >>>>>>> <http://10.0.12.151/kamailio?replicaSet=sre_location&slaveOk=true&readPreference=primaryPreferred> >>>>>>> 0(22216) DEBUG: db_mongodb >>>>>>> [mongodb_dbase.c:864]: >>>>>>> db_mongodb_query(): query to >>>>>>> collection [version] >>>>>>> 0(22216) DEBUG: db_mongodb >>>>>>> [mongodb_dbase.c:898]: >>>>>>> db_mongodb_query(): query filter: { >>>>>>> "table_name" : "location" } >>>>>>> 0(22216) DEBUG: db_mongodb >>>>>>> [mongodb_dbase.c:917]: >>>>>>> db_mongodb_query(): columns filter: >>>>>>> { "table_version" : 1 } >>>>>>> 0(22216) DEBUG: <core> >>>>>>> [db_res.c:120]: db_new_result(): >>>>>>> allocate 56 bytes for result set at >>>>>>> 0x7f735b5c4450 >>>>>>> 0(22216) DEBUG: db_mongodb >>>>>>> [mongodb_dbase.c:746]: >>>>>>> db_mongodb_store_result(): no result >>>>>>> from mongodb >>>>>>> 0(22216) DEBUG: <core> [db.c:404]: >>>>>>> db_table_version(): no row for table >>>>>>> location found >>>>>>> 0(22216) ERROR: <core> [db.c:450]: >>>>>>> db_check_table_version(): invalid >>>>>>> version 0 for table location found, >>>>>>> expected 9 (check table structure >>>>>>> and table "version") >>>>>>> 0(22216) ERROR: usrloc >>>>>>> [dlist.c:659]: register_udomain(): >>>>>>> error during table version check. >>>>>>> 0(22216) DEBUG: <core> >>>>>>> [db_pool.c:100]: pool_remove(): >>>>>>> removing connection from the pool >>>>>>> 0(22216) ERROR: registrar >>>>>>> [registrar.c:651]: domain_fixup(): >>>>>>> failed to register domain >>>>>>> 0(22216) ERROR: <core> >>>>>>> [core/route.c:1154]: fix_actions(): >>>>>>> fixing failed (code=-1) at >>>>>>> >>>>>>> cfg:/usr/local/kamailio-5.1/etc/kamailio/kamailio.cfg:660 >>>>>>> 0(22216) ERROR: <core> >>>>>>> [core/rvalue.c:3860]: >>>>>>> fix_rval_expr(): failure in cfg at >>>>>>> line: 660 col: 22 >>>>>>> 0(22216) ERROR: <core> >>>>>>> [core/rvalue.c:3860]: >>>>>>> fix_rval_expr(): failure in cfg at >>>>>>> line: 660 col: 22 >>>>>>> 0(22216) ERROR: <core> >>>>>>> [core/route.c:1154]: fix_actions(): >>>>>>> fixing failed (code=-1) at >>>>>>> >>>>>>> cfg:/usr/local/kamailio-5.1/etc/kamailio/kamailio.cfg:663 >>>>>>> ERROR: error -1 while trying to fix >>>>>>> configuration >>>>>>> 0(22216) DEBUG: tm [t_funcs.c:85]: >>>>>>> tm_shutdown(): start >>>>>>> 0(22216) DEBUG: tm [t_funcs.c:88]: >>>>>>> tm_shutdown(): emptying hash table >>>>>>> 0(22216) DEBUG: tm [t_funcs.c:90]: >>>>>>> tm_shutdown(): removing semaphores >>>>>>> 0(22216) DEBUG: tm [t_funcs.c:92]: >>>>>>> tm_shutdown(): destroying tmcb lists >>>>>>> 0(22216) DEBUG: tm [t_funcs.c:95]: >>>>>>> tm_shutdown(): done >>>>>>> 0(22216) INFO: <core> >>>>>>> [core/sctp_core.c:53]: >>>>>>> sctp_core_destroy(): SCTP API not >>>>>>> initialized >>>>>>> 0(22216) DEBUG: <core> >>>>>>> [core/mem/shm.c:261]: >>>>>>> shm_destroy_manager(): destroying >>>>>>> memory manager: q_malloc >>>>>>> 0(22216) DEBUG: <core> >>>>>>> [core/mem/q_malloc.c:1165]: >>>>>>> qm_shm_lock_destroy(): destroying >>>>>>> the shared memory lock >>>>>>> 0(22216) DEBUG: <core> >>>>>>> [core/mem/pkg.c:91]: >>>>>>> pkg_destroy_manager(): destroying >>>>>>> memory manager: q_malloc >>>>>>> >>>>>>> >>>>>>> In mongoDB, I have the collection >>>>>>> "version" in the db kamailio, and >>>>>>> the entries needed for usrloc >>>>>>> (location and location_attrs): >>>>>>> sre_location:PRIMARY> use kamailio >>>>>>> switched to db kamailio >>>>>>> sre_location:PRIMARY> >>>>>>> db.getCollection("version").find() >>>>>>> { "_id" : >>>>>>> ObjectId("5a8ec43c601b262bef82a595"), >>>>>>> "table_name" : "location", >>>>>>> "table_version" : 9 } >>>>>>> { "_id" : >>>>>>> ObjectId("5a8edb72a3ec40427577b8d0"), >>>>>>> "table_name" : "location_attrs", >>>>>>> "table_version" : 1 } >>>>>>> >>>>>>> >>>>>>> And logs in mongo show the following >>>>>>> info when kamailio starts: >>>>>>> >>>>>>> 2018-02-22T17:13:02.604+0100 I >>>>>>> NETWORK [conn90] received client >>>>>>> metadata from 10.0.12.180:56320 >>>>>>> <http://10.0.12.180:56320> conn: { >>>>>>> driver: { name: "mongoc", version: >>>>>>> "1.9.2" }, os: { type: "Linux", >>>>>>> name: "CentOS", version: "6.9 >>>>>>> (Final)", architecture: "x86_64" }, >>>>>>> platform: "cfg=0x1563000 >>>>>>> posix=200809 CC=GCC 4.4.7 20120313 >>>>>>> (Red Hat 4.4.7-18) CFLAGS="" >>>>>>> LDFLAGS=""" } >>>>>>> 2018-02-22T17:13:02.604+0100 I >>>>>>> COMMAND [conn90] command admin.$cmd >>>>>>> command: isMaster { isMaster: 1, >>>>>>> client: { driver: { name: "mongoc", >>>>>>> version: "1.9.2" }, os: { type: >>>>>>> "Linux", name: "CentOS", version: >>>>>>> "6.9 (Final)", architecture: >>>>>>> "x86_64" }, platform: "cfg=0x1563000 >>>>>>> posix=200809 CC=GCC 4.4.7 20120313 >>>>>>> (Red Hat 4.4.7-18) CFLAGS="" >>>>>>> LDFLAGS=""" }, compression: [], >>>>>>> $readPreference: { mode: >>>>>>> "secondaryPreferred" }, $db: "admin" >>>>>>> } numYields:0 reslen:716 locks:{} >>>>>>> protocol:op_query 0ms >>>>>>> 2018-02-22T17:13:02.604+0100 I >>>>>>> COMMAND [conn90] command >>>>>>> kamailio.$cmd command: find { find: >>>>>>> "version", filter: { table_name: >>>>>>> "location" }, table_version: 1, $db: >>>>>>> "kamailio", $readPreference: { mode: >>>>>>> "primaryPreferred" }, $clusterTime: >>>>>>> { clusterTime: Timestamp(1519315979, >>>>>>> 1), signature: { hash: BinData(0, >>>>>>> 0000000000000000000000000000000000000000), >>>>>>> keyId: 0 } } } numYields:0 >>>>>>> reslen:531 locks:{} protocol:op_msg 0ms >>>>>>> 2018-02-22T17:13:02.605+0100 I >>>>>>> NETWORK [conn90] end connection >>>>>>> 10.0.12.180:56320 >>>>>>> <http://10.0.12.180:56320> (9 >>>>>>> connections now open) >>>>>>> >>>>>>> I probably miss something: it is not >>>>>>> clear to me why kamailio is not able >>>>>>> to retrieve the value of >>>>>>> table_version for the table_name >>>>>>> location. Any idea would be very >>>>>>> welcome ! >>>>>>> >>>>>>> I use mongo-c-driver 1.9.2 and >>>>>>> mongodb 3.6 >>>>>>> >>>>>>> Thank you. >>>>>>> Stephane >>>>>>> >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Kamailio (SER) - Users Mailing List >>>>>>> sr-users@lists.kamailio.org >>>>>>> <mailto:sr-users@lists.kamailio.org> >>>>>>> >>>>>>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users >>>>>>> >>>>>>> <https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users> >>>>>> >>>>>> -- >>>>>> Daniel-Constantin Mierla >>>>>> www.twitter.com/miconda >>>>>> <http://www.twitter.com/miconda> -- >>>>>> www.linkedin.com/in/miconda >>>>>> <http://www.linkedin.com/in/miconda> >>>>>> Kamailio Advanced Training - March 5-7, >>>>>> 2018, Berlin - www.asipto.com <http://www.asipto.com> >>>>>> Kamailio World Conference - May 14-16, 2018 >>>>>> - www.kamailioworld.com >>>>>> <http://www.kamailioworld.com> >>>>>> >>>>>> >>>>>> >>>>> >>>>> -- >>>>> Daniel-Constantin Mierla >>>>> www.twitter.com/miconda >>>>> <http://www.twitter.com/miconda> -- >>>>> www.linkedin.com/in/miconda >>>>> <http://www.linkedin.com/in/miconda> >>>>> Kamailio Advanced Training - March 5-7, 2018, Berlin >>>>> - www.asipto.com <http://www.asipto.com> >>>>> Kamailio World Conference - May 14-16, 2018 - >>>>> www.kamailioworld.com >>>>> <http://www.kamailioworld.com> >>>>> >>>>> >>>> >>>> -- >>>> Daniel-Constantin Mierla >>>> www.twitter.com/miconda >>>> <http://www.twitter.com/miconda> -- >>>> www.linkedin.com/in/miconda >>>> <http://www.linkedin.com/in/miconda> >>>> Kamailio Advanced Training - March 5-7, 2018, Berlin - >>>> www.asipto.com <http://www.asipto.com> >>>> Kamailio World Conference - May 14-16, 2018 - >>>> www.kamailioworld.com <http://www.kamailioworld.com> >>>> >>>> >>> >>> -- >>> Daniel-Constantin Mierla >>> www.twitter.com/miconda <http://www.twitter.com/miconda> -- >>> www.linkedin.com/in/miconda >>> <http://www.linkedin.com/in/miconda> >>> Kamailio Advanced Training - March 5-7, 2018, Berlin - >>> www.asipto.com <http://www.asipto.com> >>> Kamailio World Conference - May 14-16, 2018 - >>> www.kamailioworld.com <http://www.kamailioworld.com> >>> >>> >> >> -- >> Daniel-Constantin Mierla >> www.twitter.com/miconda <http://www.twitter.com/miconda> -- >> www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda> >> Kamailio Advanced Training - March 5-7, 2018, Berlin - >> www.asipto.com <http://www.asipto.com> >> Kamailio World Conference - May 14-16, 2018 - www.kamailioworld.com >> <http://www.kamailioworld.com> >> >> > > -- > Daniel-Constantin Mierla > www.twitter.com/miconda <http://www.twitter.com/miconda> -- > www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda> > Kamailio Advanced Training - March 5-7, 2018, Berlin - www.asipto.com > <http://www.asipto.com> > Kamailio World Conference - May 14-16, 2018 - www.kamailioworld.com > <http://www.kamailioworld.com> > > -- Daniel-Constantin Mierla www.twitter.com/miconda -- www.linkedin.com/in/miconda Kamailio Advanced Training - April 16-18, 2018, Berlin - www.asipto.com Kamailio World Conference - May 14-16, 2018 - www.kamailioworld.com
_______________________________________________ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users