> This sort of "concat" operation is a bad idea, because it is prone to
> collisions...
Those were just examples to discuss a point. You can find similar types of
concatenations in multiple guides
written for setting up postfix with a mysql backend. For example refer to
'virtual_alias_domains.cf'
mentioned in arch linux's wiki page:
https://wiki.archlinux.org/title/Virtual_user_mail_system_with_Postfix,_Dovecot_and_Roundcube
I was just trying to understand if these type operations (concat, etc.) need to
be supported in the
projection. Am I correct in understanding they are not?
> You're working too hard. This sort of thing just encourages bad
> design choices. Just implement "result_attribute" (typically just
> one), and "result_format" (ala LDAP) to pick this apart and inject
> it into some template if need be. Then combine multiple results
> (if need be) by separating with commas.
If the result_attribute + result_format design is the best practice, I'm all
for that.
need to go look at the result_format and understand how to use it with mongo..
>> which would return:
>> maadani,ha...@dexo.tech/,dukhovni,vik...@postfix.org/
>
> which makes no sense.
This is honestly confusing to me. This was meant to show we are printing
multiple multi-valued
results as one comma separated string. When you say this makes no sense, are
you referring to this
result not being useful to postfix because of multiple mail-paths in it? or the
comma separated string part!?
> You do have to decide how mailing lists are modeled in MongoDB. Are
> they one row per member? Is it a list of "_id" values? Or a list of
> email addresses? If the former, how does list expansion work? Can
> MongoDB do joins as well as projections? ...
I imagine each list as a JSON object with an array of addresses inside of it.
Something like:
{ "createdAt": ISODate("<some date>"), "active": 1, "addresses": [
"ha...@dexo.tech",
"vik...@postfix.org" ] }
Would that work?
MongoDB supports joins, but through "aggregation pipelines":
https://www.mongodb.com/docs/manual/aggregation
here, we are using 'mongoc_collection_find_with_opts' which runs a 'find'
operation. If support for
joins are necessary, we should switch to 'mongoc_database_aggregate' and
require 'filter' to be in the
pipeline format:
http://mongoc.org/libmongoc/current/mongoc_database_aggregate.html
One more question, what's the policy regarding multiple databases? the way that
the module is now, it
supports multiple collections (tables) in only one database. Should I put any
effort in supporting multiple?
For example, if mailboxes are in cluster 1 and mail lists in cluster 2
(separate URIs basically)?
Regards
Hamid Maadani
> June 21, 2022 6:29 PM, "Viktor Dukhovni" <postfix-us...@dukhovni.org> wrote:
>
>> On Tue, Jun 21, 2022 at 07:26:53PM +0000, Hamid Maadani wrote:
>>
>>> Want to discuss the 'result_attribute' before I go ahead with the
>>> implementation though.
>>> It is kind of the same story as with 'filter' and 'search_key' attributes.
>>
>> Not exactly, there are enough differences to warrant specific treatment.
>>
>>> If we are to return multiple fields, each of which might need a custom
>>> projection.
>>
>> Whatever the returned fields are, it only makes sense to combine values
>> of the same semantic "type". Postfix dictionary lookups return either a
>> single value or a list of comma-separated values with the list eleemnts
>> interpreted in a uniform way.
>>
>> Therefore, in the typical case where the result attribute are returned
>> "as-is", the "projection" is trivial (identity function) and the values
>> (if more than one) are returned comma-separated.
>>
>>> So, We would need to have multiple result_attributes, and each one is
>>> either set to '1' (matching exact field in the document),
>>> or an optional JSON document for projection.
>>
>> No. The "result_attribute" list is a list of attributes whose values
>> are either used as-is or perhaps (in harmony with e.g. the LDAP table)
>> are subject to a "result_format" template, with each value expanded
>> into the specified template.
>>
>> If instead the desired result elements are built from multiple MongoDB
>> columns, via some MongoDB "projection" template, then "result_attribute"
>> should not be used, and the user can specify the relevant projection
>> instead.
>>
>>> Consider this data set:
>>> { "first": "hamid", "last": "maadani", "domain": "dexo.tech", active: 1,
>>> "other": "some other field
>>> 1"}
>>> { "first": "viktor", "last": "dukhovni", "domain": "postfix.org", active:
>>> 1, "other": "some other
>>> field 2"}
>>> { "first": "wietse", "last": "venema", "domain": "postfix.org", active: 0,
>>> "other": "some other
>>> field 3"}
>>
>> In practive you'd always want a mailbox address, and often also the
>> associated primary email address (for canonical_maps):
>>
>> filter = { "address": "ha...@dexo.tech"
>> , "mail": "hamid.maad...@dexo.tech"
>> , "maildrop": "ha...@imap1.dexo.tech"
>> , "maildir": "dexo.tex/hamid/"
>> , active: 1 }
>>
>> So that mail addressed to "address" is canonicalised to "mail" in
>> headers, but rewritten to "maildrop" for delivery to the right server,
>> where (on that server) "maildir" is the relative path to the deliver to.
>>
>> Each user would have one or more input "address" values that
>> canonicalise to the same primary address, and route to the same
>> maildrop and ultimately maildir.
>>
>>> Let's say I need mail path for all active users, as well as their last name.
>>
>> This makes no sense and is not useful. The result elements are not of
>> the same semantic type. The "last name" is not a maildir path. You'd
>> never do this.
>>
>> Furthermore, Postfix never needs/wants the whole database, it asks for
>> data matching a specific lookup key (recipient or sender address, a
>> domain or IP address for access(5) lookups ...) and wants just the
>> matching row(s), not a database dump.
>>
>>> The format for mail path is 'first@domain/'. The filter would be
>>> simple: { "active": 1 }
>>
>> This is not a useful filter. Instead you'd use, for example:
>>
>> { "active": 1, "address": "%s" }
>>
>>> We need to build this projection for the desired result:
>>> { "_id": 0, "last": 1, "mail_path": {"$$concat": ["$$first", "@",
>>> "$$domain", "/"]} }
>>
>> A sensible "projection" would then be "maildrop" for virtual_alias_maps,
>> and "maildir" for "virtual_mailbox_maps", ...
>>
>>> { "_id": 0, "last": 1, "mail_path": {"$$concat": ["$$first", "@",
>>> "$$domain", "/"]} }
>>
>> This sort of "concat" operation is a bad idea, because it is prone to
>> collisions, and works poorly when the user's name changes, but the
>> mailbox stays the same, ... It is just bad design. Store the mailbox
>> directly as an explicit fixed string.
>>
>>> which would return:
>>> maadani,ha...@dexo.tech/,dukhovni,vik...@postfix.org/
>>
>> which makes no sense.
>>
>>> 'result_attributes' would be need to be set to :
>>> result_attributes = last, mail_path
>>
>> See above.
>>
>>> and I guess the 'projection' attribute would need to be set to something
>>> like:
>>> projection = 1, {"$$concat": ["$$first", "@", "$$domain", "/"]}
>>
>> You're working too hard. This sort of thing just encourages bad
>> design choices. Just implement "result_attribute" (typically just
>> one), and "result_format" (ala LDAP) to pick this apart and inject
>> it into some template if need be. Then combine multiple results
>> (if need be) by separating with commas.
>>
>> You do have to decide how mailing lists are modeled in MongoDB. Are
>> they one row per member? Is it a list of "_id" values? Or a list of
>> email addresses? If the former, how does list expansion work? Can
>> MongoDB do joins as well as projections? ...
>>
>>> or something similar to this.
>>> Would it not be more beneficial/easier to leave the projection as a JSON
>>> object and up to the user,
>>> instead of having 'result_attribute'?
>>
>> No.
>>
>> --
>> Viktor.
