Hi Don,
>> I suppose to put a finer point on it - what is your usecase here? What
>> are you trying to achieve, and can we help you do that in a way that
>> requires smaller changes to Patchwork, and is less fragile? (For example
>> this is going to break if someone mis-spells the keyword you're looking
>> for in the subject_match.)
>
> So here is our use case. Internally at Red Hat we use one mailing list to
> post all of our kernel patches. However, being a distro company, patches
> can be applied to either RHEL-6,7,X. For the last 15 years we have always
> used the method:
>
> [RHEL-7 PATCH] instead of [PATCH].
Ah yes. I work for Canonical (although I do Patchwork in a private
capacity only) and our kernel team does something very similar.
> The project inside the []s has been what we filter through our regex. Is it
> prone to human typos? Yes. Most folks have stuck this in the .git/config
> subjectprefix option. That limited the typos. It isn't perfect.
>
> We have been using a hacked up PatchWork1 for the last 7 or so years. This
> is one of those features we need (or something to solve our problem) if we
> want to migrate to a PatchWork2 instance.
>
> I hope that adds a little bit of context on our thinking. Thoughts?
That's a compelling use-case and I'm happy to look at supporting it. I
will review the patch more closely in the coming days.
Regards,
Daniel
> Cheers,
> Don
>
>>
>> Regards,
>> Daniel
>>
>> >
>> > Signed-off-by: Veronika Kabatova <vkaba...@redhat.com>
>> > ---
>> > docs/api.yaml | 3 +++
>> > docs/deployment/management.rst | 4 +--
>> > patchwork/api/project.py | 5 ++--
>> > .../0021_add_subject_match_to_project.py | 29
>> > ++++++++++++++++++++++
>> > patchwork/models.py | 9 ++++++-
>> > patchwork/parser.py | 19 +++++++++++++-
>> > .../notes/list-filtering-4643d98b4064367a.yaml | 10 ++++++++
>> > 7 files changed, 73 insertions(+), 6 deletions(-)
>> > create mode 100644
>> > patchwork/migrations/0021_add_subject_match_to_project.py
>> > create mode 100644 releasenotes/notes/list-filtering-4643d98b4064367a.yaml
>> >
>> > diff --git a/docs/api.yaml b/docs/api.yaml
>> > index 3e79f0b..3373226 100644
>> > --- a/docs/api.yaml
>> > +++ b/docs/api.yaml
>> > @@ -374,6 +374,9 @@ definitions:
>> > list_id:
>> > type: string
>> > description: Mailing list identifier for project.
>> > + subject_match:
>> > + type: string
>> > + description: Regex used for email filtering.
>> > list_email:
>> > type: string
>> > description: Mailing list email address for project.
>> > diff --git a/docs/deployment/management.rst
>> > b/docs/deployment/management.rst
>> > index c50b7b6..870d7ee 100644
>> > --- a/docs/deployment/management.rst
>> > +++ b/docs/deployment/management.rst
>> > @@ -63,7 +63,7 @@ due to, for example, an outage.
>> > .. option:: --list-id <list-id>
>> >
>> > mailing list ID. If not supplied, this will be extracted from the mail
>> > - headers.
>> > + headers. Don't use this option if you require filtering based on
>> > subjects.
>> >
>> > .. option:: infile
>> >
>> > @@ -88,7 +88,7 @@ the :ref:`deployment installation guide
>> > <deployment-parsemail>`.
>> > .. option:: --list-id <list-id>
>> >
>> > mailing list ID. If not supplied, this will be extracted from the mail
>> > - headers.
>> > + headers. Don't use this option if you require filtering based on
>> > subjects.
>> >
>> > .. option:: infile
>> >
>> > diff --git a/patchwork/api/project.py b/patchwork/api/project.py
>> > index 446c473..597f605 100644
>> > --- a/patchwork/api/project.py
>> > +++ b/patchwork/api/project.py
>> > @@ -39,8 +39,9 @@ class ProjectSerializer(HyperlinkedModelSerializer):
>> > class Meta:
>> > model = Project
>> > fields = ('id', 'url', 'name', 'link_name', 'list_id',
>> > 'list_email',
>> > - 'web_url', 'scm_url', 'webscm_url', 'maintainers')
>> > - read_only_fields = ('name', 'maintainers')
>> > + 'web_url', 'scm_url', 'webscm_url', 'maintainers',
>> > + 'subject_match')
>> > + read_only_fields = ('name', 'maintainers', 'subject_match')
>> > extra_kwargs = {
>> > 'url': {'view_name': 'api-project-detail'},
>> > }
>> > diff --git a/patchwork/migrations/0021_add_subject_match_to_project.py
>> > b/patchwork/migrations/0021_add_subject_match_to_project.py
>> > new file mode 100644
>> > index 0000000..6066266
>> > --- /dev/null
>> > +++ b/patchwork/migrations/0021_add_subject_match_to_project.py
>> > @@ -0,0 +1,29 @@
>> > +# -*- coding: utf-8 -*-
>> > +# Generated by Django 1.10.8 on 2018-01-19 18:16
>> > +from __future__ import unicode_literals
>> > +
>> > +from django.db import migrations, models
>> > +
>> > +
>> > +class Migration(migrations.Migration):
>> > +
>> > + dependencies = [
>> > + ('patchwork', '0020_tag_show_column'),
>> > + ]
>> > +
>> > + operations = [
>> > + migrations.AddField(
>> > + model_name='project',
>> > + name='subject_match',
>> > + field=models.CharField(blank=True, default=b'',
>> > help_text=b'Regex to match the subject against if only part of emails sent
>> > to the list belongs to this project. Will be used with IGNORECASE and
>> > MULTILINE flags. If rules for more projects match the first one returned
>> > from DB is chosen.', max_length=64),
>> > + ),
>> > + migrations.AlterField(
>> > + model_name='project',
>> > + name='listid',
>> > + field=models.CharField(max_length=255),
>> > + ),
>> > + migrations.AlterUniqueTogether(
>> > + name='project',
>> > + unique_together=set([('listid', 'subject_match')]),
>> > + ),
>> > + ]
>> > diff --git a/patchwork/models.py b/patchwork/models.py
>> > index 11886f1..907707f 100644
>> > --- a/patchwork/models.py
>> > +++ b/patchwork/models.py
>> > @@ -71,8 +71,14 @@ class Project(models.Model):
>> >
>> > linkname = models.CharField(max_length=255, unique=True)
>> > name = models.CharField(max_length=255, unique=True)
>> > - listid = models.CharField(max_length=255, unique=True)
>> > + listid = models.CharField(max_length=255)
>> > listemail = models.CharField(max_length=200)
>> > + subject_match = models.CharField(
>> > + max_length=64, blank=True, default='', help_text='Regex to match
>> > the '
>> > + 'subject against if only part of emails sent to the list belongs
>> > to '
>> > + 'this project. Will be used with IGNORECASE and MULTILINE flags.
>> > If '
>> > + 'rules for more projects match the first one returned from DB is '
>> > + 'chosen.')
>> >
>> > # url metadata
>> >
>> > @@ -100,6 +106,7 @@ class Project(models.Model):
>> > return self.name
>> >
>> > class Meta:
>> > + unique_together = (('listid', 'subject_match'),)
>> > ordering = ['linkname']
>> >
>> >
>> > diff --git a/patchwork/parser.py b/patchwork/parser.py
>> > index ac7dc5f..015f709 100644
>> > --- a/patchwork/parser.py
>> > +++ b/patchwork/parser.py
>> > @@ -157,9 +157,25 @@ def find_project_by_id(list_id):
>> > project = Project.objects.get(listid=list_id)
>> > except Project.DoesNotExist:
>> > logger.debug("'%s' if not a valid project list-id", list_id)
>> > + except Project.MultipleObjectsReturned:
>> > + logger.debug("Multiple projects with list-id '%s' found", list_id)
>> > return project
>> >
>> >
>> > +def find_project_by_id_and_subject(list_id, subject):
>> > + """Find a `project` object based on `list_id` and subject prefix
>> > match."""
>> > + projects = Project.objects.filter(listid=list_id)
>> > + for project in projects:
>> > + if (not project.subject_match or
>> > + re.search(project.subject_match, subject,
>> > + re.MULTILINE | re.IGNORECASE)):
>> > + return project
>> > +
>> > + logger.debug("No project to match email with list-id '%s', subject
>> > '%s' "
>> > + "found", list_id, subject)
>> > + return None
>> > +
>> > +
>> > def find_project_by_header(mail):
>> > project = None
>> > listid_res = [re.compile(r'.*<([^>]+)>.*', re.S),
>> > @@ -181,7 +197,8 @@ def find_project_by_header(mail):
>> >
>> > listid = match.group(1)
>> >
>> > - project = find_project_by_id(listid)
>> > + project = find_project_by_id_and_subject(listid,
>> > + mail.get('Subject'))
>> > if project:
>> > break
>> >
>> > diff --git a/releasenotes/notes/list-filtering-4643d98b4064367a.yaml
>> > b/releasenotes/notes/list-filtering-4643d98b4064367a.yaml
>> > new file mode 100644
>> > index 0000000..21c1680
>> > --- /dev/null
>> > +++ b/releasenotes/notes/list-filtering-4643d98b4064367a.yaml
>> > @@ -0,0 +1,10 @@
>> > +---
>> > +features:
>> > + - |
>> > + Allow list filtering into multiple projects (and email dropping)
>> > based on
>> > + subject prefixes. Disabled by default, enable by specifying a regular
>> > + expression which needs to be matched in the subject on a per-project
>> > basis
>> > + (field `subject_match`).
>> > +api:
>> > + - |
>> > + Expose `subject_match` in REST API.
>> > --
>> > 2.13.6
>> >
>> > _______________________________________________
>> > Patchwork mailing list
>> > Patchwork@lists.ozlabs.org
>> > https://lists.ozlabs.org/listinfo/patchwork
>> _______________________________________________
>> Patchwork mailing list
>> Patchwork@lists.ozlabs.org
>> https://lists.ozlabs.org/listinfo/patchwork
_______________________________________________
Patchwork mailing list
Patchwork@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/patchwork
