Andrew Donnellan <andrew.donnel...@au1.ibm.com> writes: > On 08/01/18 18:46, Andrew Donnellan wrote: >> When using Python 2, an incoming email that uses CRLF line endings won't >> have the line endings converted by the email parser. This means that the >> regex to detect pull request emails will fail to match. >> >> Clean up the line endings when we extract the email body to fix this. (On >> Python 3, the email parser policy fixes this for us at the initial email >> parsing stage.) >> >> Add a test pull request mbox with CRLF line endings to ensure we don't >> regress. > > Hmm, sudden realisation that git handles line endings for us, so perhaps > we don't have an easy way of including a test here.
If it's just git-send-email, I'm happy to pull a signed tag. Otherwise I suggest you look back through history and perhaps consider uuencode. :P Regards, Daniel > > >> >> Closes: #148 ("Parsing pull request emails broken on Python 2") >> Signed-off-by: Andrew Donnellan <andrew.donnel...@au1.ibm.com> >> --- >> patchwork/parser.py | 4 + >> .../mail/0017-git-pull-request-crlf-newlines.mbox | 348 >> +++++++++++++++++++++ >> patchwork/tests/test_parser.py | 3 + >> 3 files changed, 355 insertions(+) >> create mode 100644 >> patchwork/tests/mail/0017-git-pull-request-crlf-newlines.mbox >> >> diff --git a/patchwork/parser.py b/patchwork/parser.py >> index 1568bc4..7c677db 100644 >> --- a/patchwork/parser.py >> +++ b/patchwork/parser.py >> @@ -666,9 +666,13 @@ def clean_content(content): >> """Remove cruft from the email message. >> >> Catch signature (-- ) and list footer (_____) cruft. >> + >> + Change to Unix line endings (the Python 3 email module does this for us, >> + but not Python 2). >> """ >> sig_re = re.compile(r'^(-- |_+)\n.*', re.S | re.M) >> content = sig_re.sub('', content) >> + content = content.replace('\r\n', '\n') >> >> return content.strip() >> >> diff --git a/patchwork/tests/mail/0017-git-pull-request-crlf-newlines.mbox >> b/patchwork/tests/mail/0017-git-pull-request-crlf-newlines.mbox >> new file mode 100644 >> index 0000000..bad78ae >> --- /dev/null >> +++ b/patchwork/tests/mail/0017-git-pull-request-crlf-newlines.mbox >> @@ -0,0 +1,348 @@ >> +From b...@kernel.crashing.org Fri Oct 22 11:51:02 2010 >> +Return-Path: <linuxppc-dev-bounces+jk=ozlabs....@lists.ozlabs.org> >> +X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on bilbo.ozlabs.org >> +X-Spam-Level: >> +X-Spam-Status: No, score=0.0 required=3.0 tests=none autolearn=disabled >> + version=3.3.1 >> +X-Original-To: j...@ozlabs.org >> +Delivered-To: j...@ozlabs.org >> +Received: from bilbo.ozlabs.org (localhost [127.0.0.1]) >> + by ozlabs.org (Postfix) with ESMTP id ED4B3100937 >> + for <j...@ozlabs.org>; Fri, 22 Oct 2010 14:51:54 +1100 (EST) >> +Received: by ozlabs.org (Postfix) >> + id BF799B70CB; Fri, 22 Oct 2010 14:51:50 +1100 (EST) >> +Delivered-To: linuxppc-...@ozlabs.org >> +Received: from gate.crashing.org (gate.crashing.org [63.228.1.57]) >> + (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) >> + (Client did not present a certificate) >> + by ozlabs.org (Postfix) with ESMTPS id 94629B7043 >> + for <linuxppc-...@ozlabs.org>; Fri, 22 Oct 2010 14:51:49 +1100 (EST) >> +Received: from [IPv6:::1] (localhost.localdomain [127.0.0.1]) >> + by gate.crashing.org (8.14.1/8.13.8) with ESMTP id o9M3p3SP018234; >> + Thu, 21 Oct 2010 22:51:04 -0500 >> +Subject: [git pull] Please pull powerpc.git next branch >> +From: Benjamin Herrenschmidt <b...@kernel.crashing.org> >> +To: Linus Torvalds <torva...@linux-foundation.org> >> +Date: Fri, 22 Oct 2010 14:51:02 +1100 >> +Message-ID: <1287719462.2198.37.camel@pasglop> >> +Mime-Version: 1.0 >> +X-Mailer: Evolution 2.30.3 >> +Cc: linuxppc-dev list <linuxppc-...@ozlabs.org>, >> + Andrew Morton <a...@linux-foundation.org>, >> + Linux Kernel list <linux-ker...@vger.kernel.org> >> +X-BeenThere: linuxppc-...@lists.ozlabs.org >> +X-Mailman-Version: 2.1.13 >> +Precedence: list >> +List-Id: Linux on PowerPC Developers Mail List <cbe-oss-dev.ozlabs.org> >> +List-Unsubscribe: <https://lists.ozlabs.org/options/linuxppc-dev>, >> + <mailto:linuxppc-dev-requ...@lists.ozlabs.org?subject=unsubscribe> >> +List-Archive: <http://lists.ozlabs.org/pipermail/linuxppc-dev> >> +List-Post: <mailto:linuxppc-...@lists.ozlabs.org> >> +List-Help: <mailto:linuxppc-dev-requ...@lists.ozlabs.org?subject=help> >> +List-Subscribe: <https://lists.ozlabs.org/listinfo/linuxppc-dev>, >> + <mailto:linuxppc-dev-requ...@lists.ozlabs.org?subject=subscribe> >> +Content-Type: text/plain; >> + charset="us-ascii" >> +Content-Transfer-Encoding: 7bit >> +Sender: linuxppc-dev-bounces+jk=ozlabs....@lists.ozlabs.org >> +Errors-To: linuxppc-dev-bounces+jk=ozlabs....@lists.ozlabs.org >> +X-UID: 11446 >> +X-Length: 16781 >> +Status: R >> +X-Status: N >> +X-KMail-EncryptionState: >> +X-KMail-SignatureState: >> +X-KMail-MDN-Sent: >> + >> +Hi Linus ! >> + >> +Here's powerpc's batch for this merge window. Mostly bits and pieces, >> +such as Anton doing some performance tuning left and right, and the >> +usual churn. One hilight is the support for the new Freescale e5500 core >> +(64-bit BookE). Another one is that we now wire up the whole lot of >> +socket calls as direct syscalls in addition to the old style indirect >> +method. >> + >> +Cheers, >> +Ben. >> + >> +The following changes since commit e10117d36ef758da0690c95ecffc09d5dd7da479: >> + Linus Torvalds (1): >> + Merge branch 'upstream-linus' of >> git://git.kernel.org/.../jgarzik/libata-dev >> + >> +are available in the git repository at: >> + >> + git://git.kernel.org/pub/scm/linux/kernel/git/benh/powerpc.git next >> + >> +Andreas Schwab (1): >> + powerpc: Remove fpscr use from [kvm_]cvt_{fd,df} >> + >> +Anton Blanchard (5): >> + powerpc: Optimise 64bit csum_partial >> + powerpc: Optimise 64bit csum_partial_copy_generic and add >> csum_and_copy_from_user >> + powerpc: Add 64bit csum_and_copy_to_user >> + powerpc: Feature nop out reservation clear when stcx checks address >> + powerpc: Check end of stack canary at oops time >> + >> +Arnd Bergmann (1): >> + powerpc/spufs: Use llseek in all file operations >> + >> +Benjamin Herrenschmidt (4): >> + powerpc/dma: Add optional platform override of dma_set_mask() >> + powerpc/dart_iommu: Support for 64-bit iommu bypass window on PCIe >> + Merge remote branch 'kumar/merge' into next >> + Merge remote branch 'jwb/next' into next >> + >> +Denis Kirjanov (1): >> + powerpc: Use is_32bit_task() helper to test 32-bit binary >> + >> +Harninder Rai (1): >> + powerpc/85xx: add cache-sram support >> + >> +Ian Munsie (1): >> + powerpc: Wire up direct socket system calls >> + >> +Ilya Yanok (1): >> + powerpc/mpc83xx: Support for MPC8308 P1M board >> + >> +Joe Perches (2): >> + powerpc: Use static const char arrays >> + powerpc: Remove pr_<level> uses of KERN_<level> >> + >> +Josh Boyer (1): >> + powerpc/44x: Update ppc44x_defconfig >> + >> +Julia Lawall (7): >> + powerpc/via-pmu-led.c: Add of_node_put to avoid memory leak >> + powerpc/maple: Add of_node_put to avoid memory leak >> + powerpc/powermac/pfunc_core.c: Add of_node_put to avoid memory leak >> + powerpc/cell: Add of_node_put to avoid memory leak >> + powerpc/chrp/nvram.c: Add of_node_put to avoid memory leak >> + powerpc/irq.c: Add of_node_put to avoid memory leak >> + i2c/i2c-pasemi.c: Fix unsigned return type >> + >> +Kumar Gala (11): >> + powerpc/ppc64e: Fix link problem when building ppc64e_defconfig >> + powerpc/fsl-pci: Fix MSI support on 83xx platforms >> + powerpc/mpc8xxx_gpio: Add support for 'qoriq-gpio' controllers >> + powerpc/fsl-booke: Add PCI device ids for P2040/P3041/P5010/P5020 >> QoirQ chips >> + powerpc/fsl-booke: Add p3041 DS board support >> + powerpc: Fix compile error with paca code on ppc64e >> + powerpc/fsl-booke: Add support for FSL 64-bit e5500 core >> + powerpc/fsl-booke: Add support for FSL Arch v1.0 MMU in >> setup_page_sizes >> + powerpc/fsl-booke64: Use TLB CAMs to cover linear mapping on FSL >> 64-bit chips >> + powerpc/fsl-booke: Add p5020 DS board support >> + powerpc/fsl-booke: Add e55xx (64-bit) smp defconfig >> + >> +Matthew McClintock (7): >> + powerpc/mm: Assume first cpu is boot_cpuid not 0 >> + powerpc/kexec: make masking/disabling interrupts generic >> + powerpc/85xx: Remove call to mpic_teardown_this_cpu in kexec >> + powerpc/85xx: Minor fixups for kexec on 85xx >> + powerpc/85xx: flush dcache before resetting cores >> + powerpc/fsl_soc: Search all global-utilities nodes for rstccr >> + powerpc/fsl_booke: Add support to boot from core other than 0 >> + >> +Michael Neuling (1): >> + powerpc: Move arch_sd_sibling_asym_packing() to smp.c >> + >> +Nathan Fontenot (3): >> + powerpc/pseries: Export device tree updating routines >> + powerpc/pseries: Export rtas_ibm_suspend_me() >> + powerpc/pseries: Partition migration in the kernel >> + >> +Nishanth Aravamudan (8): >> + powerpc/pci: Fix return type of BUID_{HI,LO} macros >> + powerpc/dma: Fix dma_iommu_dma_supported compare >> + powerpc/dma: Fix check for direct DMA support >> + powerpc/vio: Use put_device() on device_register failure >> + powerpc/viobus: Free TCE table on device release >> + powerpc/pseries: Use kmemdup >> + powerpc/pci: Cleanup device dma setup code >> + powerpc/pseries/xics: Use cpu_possible_mask rather than cpu_all_mask >> + >> +Paul Gortmaker (1): >> + powerpc: Fix invalid page flags in create TLB CAM path for PTE_64BIT >> + >> +Paul Mackerras (5): >> + powerpc: Abstract indexing of lppaca structs >> + powerpc: Dynamically allocate most lppaca structs >> + powerpc: Account time using timebase rather than PURR >> + powerpc/pseries: Re-enable dispatch trace log userspace interface >> + powerpc/perf: Fix sampling enable for PPC970 >> + >> +Scott Wood (1): >> + oprofile/fsl emb: Don't set MSR[PMM] until after clearing the >> interrupt. >> + >> +Sean MacLennan (2): >> + powerpc: Fix incorrect .stabs entry for copy_32.S >> + powerpc: mtmsrd not defined >> + >> +Shaohui Xie (1): >> + fsl_rio: Add comments for sRIO registers. >> + >> +Stephen Rothwell (1): >> + powerpc: define a compat_sys_recv cond_syscall >> + >> +Timur Tabi (5): >> + powerpc: export ppc_proc_freq and ppc_tb_freq as GPL symbols >> + powerpc/watchdog: Allow the Book-E driver to be compiled as a module >> + powerpc/p1022: Add probing for individual DMA channels >> + powerpc/85xx: add ngPIXIS FPGA device tree node to the P1022DS board >> + powerpc/watchdog: Make default timeout for Book-E watchdog a Kconfig >> option >> + >> +Tirumala Marri (1): >> + powerpc/44x: Add support for the AMCC APM821xx SoC >> + >> +matt mooney (1): >> + powerpc/Makefiles: Change to new flag variables >> + >> + arch/powerpc/boot/addnote.c | 4 +- >> + arch/powerpc/boot/dts/bluestone.dts | 254 +++++++++++++ >> + arch/powerpc/boot/dts/mpc8308_p1m.dts | 332 ++++++++++++++++ >> + arch/powerpc/boot/dts/p1022ds.dts | 11 + >> + arch/powerpc/configs/44x/bluestone_defconfig | 68 ++++ >> + arch/powerpc/configs/e55xx_smp_defconfig | 84 ++++ >> + arch/powerpc/configs/ppc44x_defconfig | 9 +- >> + arch/powerpc/configs/ppc64e_defconfig | 4 +- >> + arch/powerpc/include/asm/checksum.h | 10 + >> + arch/powerpc/include/asm/compat.h | 4 +- >> + arch/powerpc/include/asm/cputable.h | 14 +- >> + arch/powerpc/include/asm/dma-mapping.h | 14 +- >> + arch/powerpc/include/asm/elf.h | 2 +- >> + arch/powerpc/include/asm/exception-64s.h | 3 +- >> + arch/powerpc/include/asm/fsl_85xx_cache_sram.h | 48 +++ >> + arch/powerpc/include/asm/kexec.h | 1 + >> + arch/powerpc/include/asm/kvm_fpu.h | 4 +- >> + arch/powerpc/include/asm/lppaca.h | 29 ++ >> + arch/powerpc/include/asm/machdep.h | 3 + >> + arch/powerpc/include/asm/mmu-book3e.h | 15 + >> + arch/powerpc/include/asm/paca.h | 10 +- >> + arch/powerpc/include/asm/page_64.h | 4 +- >> + arch/powerpc/include/asm/ppc-pci.h | 4 +- >> + arch/powerpc/include/asm/ppc_asm.h | 50 ++- >> + arch/powerpc/include/asm/processor.h | 4 +- >> + arch/powerpc/include/asm/pte-common.h | 7 + >> + arch/powerpc/include/asm/rtas.h | 1 + >> + arch/powerpc/include/asm/systbl.h | 19 + >> + arch/powerpc/include/asm/system.h | 4 +- >> + arch/powerpc/include/asm/time.h | 5 - >> + arch/powerpc/include/asm/unistd.h | 21 +- >> + arch/powerpc/kernel/Makefile | 4 +- >> + arch/powerpc/kernel/align.c | 4 +- >> + arch/powerpc/kernel/asm-offsets.c | 12 +- >> + arch/powerpc/kernel/cpu_setup_44x.S | 1 + >> + arch/powerpc/kernel/cpu_setup_fsl_booke.S | 15 + >> + arch/powerpc/kernel/cputable.c | 43 ++- >> + arch/powerpc/kernel/crash.c | 13 +- >> + arch/powerpc/kernel/dma-iommu.c | 21 +- >> + arch/powerpc/kernel/dma.c | 20 +- >> + arch/powerpc/kernel/entry_64.S | 40 ++ >> + arch/powerpc/kernel/fpu.S | 10 - >> + arch/powerpc/kernel/head_fsl_booke.S | 10 +- >> + arch/powerpc/kernel/irq.c | 6 +- >> + arch/powerpc/kernel/lparcfg.c | 14 +- >> + arch/powerpc/kernel/machine_kexec.c | 24 ++ >> + arch/powerpc/kernel/machine_kexec_32.c | 4 + >> + arch/powerpc/kernel/paca.c | 70 ++++- >> + arch/powerpc/kernel/pci-common.c | 4 +- >> + arch/powerpc/kernel/ppc970-pmu.c | 2 + >> + arch/powerpc/kernel/process.c | 12 - >> + arch/powerpc/kernel/ptrace.c | 2 +- >> + arch/powerpc/kernel/rtas.c | 4 +- >> + arch/powerpc/kernel/setup_32.c | 2 +- >> + arch/powerpc/kernel/smp.c | 14 +- >> + arch/powerpc/kernel/time.c | 275 +++++++------- >> + arch/powerpc/kernel/traps.c | 5 + >> + arch/powerpc/kernel/vdso.c | 6 +- >> + arch/powerpc/kernel/vdso32/Makefile | 6 +- >> + arch/powerpc/kernel/vdso64/Makefile | 6 +- >> + arch/powerpc/kernel/vio.c | 10 +- >> + arch/powerpc/kvm/Makefile | 2 +- >> + arch/powerpc/kvm/book3s_paired_singles.c | 44 +-- >> + arch/powerpc/kvm/emulate.c | 4 +- >> + arch/powerpc/kvm/fpu.S | 8 - >> + arch/powerpc/lib/Makefile | 7 +- >> + arch/powerpc/lib/checksum_64.S | 482 >> +++++++++++++++++------- >> + arch/powerpc/lib/checksum_wrappers_64.c | 102 +++++ >> + arch/powerpc/lib/copy_32.S | 2 +- >> + arch/powerpc/lib/ldstfp.S | 36 +- >> + arch/powerpc/lib/locks.c | 4 +- >> + arch/powerpc/lib/sstep.c | 8 + >> + arch/powerpc/math-emu/Makefile | 2 +- >> + arch/powerpc/mm/Makefile | 6 +- >> + arch/powerpc/mm/fault.c | 6 + >> + arch/powerpc/mm/fsl_booke_mmu.c | 15 +- >> + arch/powerpc/mm/mmu_context_nohash.c | 6 +- >> + arch/powerpc/mm/mmu_decl.h | 5 +- >> + arch/powerpc/mm/tlb_nohash.c | 56 +++- >> + arch/powerpc/mm/tlb_nohash_low.S | 2 +- >> + arch/powerpc/oprofile/Makefile | 4 +- >> + arch/powerpc/oprofile/backtrace.c | 2 +- >> + arch/powerpc/oprofile/op_model_fsl_emb.c | 15 +- >> + arch/powerpc/platforms/44x/Kconfig | 16 + >> + arch/powerpc/platforms/44x/ppc44x_simple.c | 1 + >> + arch/powerpc/platforms/83xx/Kconfig | 4 +- >> + arch/powerpc/platforms/83xx/mpc830x_rdb.c | 3 +- >> + arch/powerpc/platforms/85xx/Kconfig | 28 ++- >> + arch/powerpc/platforms/85xx/Makefile | 2 + >> + arch/powerpc/platforms/85xx/p1022_ds.c | 2 + >> + arch/powerpc/platforms/85xx/p3041_ds.c | 64 ++++ >> + arch/powerpc/platforms/85xx/p5020_ds.c | 69 ++++ >> + arch/powerpc/platforms/85xx/smp.c | 83 ++++- >> + arch/powerpc/platforms/Kconfig.cputype | 8 +- >> + arch/powerpc/platforms/cell/ras.c | 4 +- >> + arch/powerpc/platforms/cell/spider-pic.c | 4 +- >> + arch/powerpc/platforms/cell/spufs/file.c | 18 + >> + arch/powerpc/platforms/chrp/nvram.c | 4 +- >> + arch/powerpc/platforms/iseries/Makefile | 2 +- >> + arch/powerpc/platforms/iseries/dt.c | 4 +- >> + arch/powerpc/platforms/iseries/smp.c | 2 +- >> + arch/powerpc/platforms/maple/setup.c | 1 + >> + arch/powerpc/platforms/powermac/pfunc_core.c | 9 +- >> + arch/powerpc/platforms/pseries/Makefile | 13 +- >> + arch/powerpc/platforms/pseries/dlpar.c | 7 +- >> + arch/powerpc/platforms/pseries/dtl.c | 224 +++++++++--- >> + arch/powerpc/platforms/pseries/lpar.c | 25 ++- >> + arch/powerpc/platforms/pseries/mobility.c | 362 ++++++++++++++++++ >> + arch/powerpc/platforms/pseries/pseries.h | 9 + >> + arch/powerpc/platforms/pseries/setup.c | 52 +++ >> + arch/powerpc/platforms/pseries/xics.c | 2 +- >> + arch/powerpc/sysdev/Makefile | 5 +- >> + arch/powerpc/sysdev/dart_iommu.c | 74 ++++- >> + arch/powerpc/sysdev/fsl_85xx_cache_ctlr.h | 101 +++++ >> + arch/powerpc/sysdev/fsl_85xx_cache_sram.c | 159 ++++++++ >> + arch/powerpc/sysdev/fsl_85xx_l2ctlr.c | 231 +++++++++++ >> + arch/powerpc/sysdev/fsl_msi.c | 9 +- >> + arch/powerpc/sysdev/fsl_pci.c | 60 +++- >> + arch/powerpc/sysdev/fsl_pci.h | 1 + >> + arch/powerpc/sysdev/fsl_rio.c | 65 ++-- >> + arch/powerpc/sysdev/fsl_soc.c | 20 +- >> + arch/powerpc/sysdev/mpc8xxx_gpio.c | 3 + >> + arch/powerpc/sysdev/pmi.c | 2 +- >> + arch/powerpc/xmon/Makefile | 4 +- >> + drivers/i2c/busses/i2c-pasemi.c | 2 +- >> + drivers/macintosh/via-pmu-led.c | 4 +- >> + drivers/watchdog/Kconfig | 22 +- >> + drivers/watchdog/booke_wdt.c | 47 ++- >> + include/linux/pci_ids.h | 8 + >> + kernel/sys_ni.c | 1 + >> + 130 files changed, 3676 insertions(+), 683 deletions(-) >> + create mode 100644 arch/powerpc/boot/dts/bluestone.dts >> + create mode 100644 arch/powerpc/boot/dts/mpc8308_p1m.dts >> + create mode 100644 arch/powerpc/configs/44x/bluestone_defconfig >> + create mode 100644 arch/powerpc/configs/e55xx_smp_defconfig >> + create mode 100644 arch/powerpc/include/asm/fsl_85xx_cache_sram.h >> + create mode 100644 arch/powerpc/lib/checksum_wrappers_64.c >> + create mode 100644 arch/powerpc/platforms/85xx/p3041_ds.c >> + create mode 100644 arch/powerpc/platforms/85xx/p5020_ds.c >> + create mode 100644 arch/powerpc/platforms/pseries/mobility.c >> + create mode 100644 arch/powerpc/sysdev/fsl_85xx_cache_ctlr.h >> + create mode 100644 arch/powerpc/sysdev/fsl_85xx_cache_sram.c >> + create mode 100644 arch/powerpc/sysdev/fsl_85xx_l2ctlr.c >> + >> + >> +_______________________________________________ >> +Linuxppc-dev mailing list >> +linuxppc-...@lists.ozlabs.org >> +https://lists.ozlabs.org/listinfo/linuxppc-dev >> diff --git a/patchwork/tests/test_parser.py b/patchwork/tests/test_parser.py >> index c21040e..9dca54a 100644 >> --- a/patchwork/tests/test_parser.py >> +++ b/patchwork/tests/test_parser.py >> @@ -545,6 +545,9 @@ class PatchParseTest(PatchTest): >> def test_git_pull_request(self): >> self._test_pull_request_parse('0001-git-pull-request.mbox') >> >> + def test_git_pull_request_crlf_newlines(self): >> + >> self._test_pull_request_parse('0017-git-pull-request-crlf-newlines.mbox') >> + >> def test_git_pull_wrapped_request(self): >> self._test_pull_request_parse('0002-git-pull-request-wrapped.mbox') >> >> > > -- > Andrew Donnellan OzLabs, ADL Canberra > andrew.donnel...@au1.ibm.com IBM Australia Limited > > _______________________________________________ > Patchwork mailing list > Patchwork@lists.ozlabs.org > https://lists.ozlabs.org/listinfo/patchwork _______________________________________________ Patchwork mailing list Patchwork@lists.ozlabs.org https://lists.ozlabs.org/listinfo/patchwork