Santhosh has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/277480

Change subject: Reduce the frequency of AbuseFilter Validations
......................................................................

Reduce the frequency of AbuseFilter Validations

The frequence fragment validation need Restbase and eventually parsoid
to do the validations. Validating in every autosave will not be smart
as per recent observations in the increase in requests to Restbase.

Following approach is attempted:

1. In translation units that are sent to cxsave API for saving
   have a boolean typed validate flag. If true, it will be validated,
   otherwise skipped
2. The translation storage module will be responsible for setting
   that flag true or false
3. Do not set that flag true for sections other than paragraphs
   and target title.
4. Set the validate flag true in every 10th autosave
5. Set the validate flag true if the section has a validation error

Bug: T129314
Change-Id: I8660527a891b77e272e9af1d6d3b0bbe3ee5ebf2
---
M api/ApiContentTranslationSave.php
M includes/TranslationUnit.php
M modules/translation/ext.cx.translation.storage.js
3 files changed, 36 insertions(+), 3 deletions(-)


  git pull 
ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/ContentTranslation 
refs/changes/80/277480/1

diff --git a/api/ApiContentTranslationSave.php 
b/api/ApiContentTranslationSave.php
index 3f31a97..e1ed70e 100644
--- a/api/ApiContentTranslationSave.php
+++ b/api/ApiContentTranslationSave.php
@@ -77,6 +77,9 @@
                $checker = new AbuseFilterCheck( $this->getUser(), $title );
 
                foreach ( $translationUnits as $translationUnit ) {
+                       if ( !$translationUnit->getValidate() ) {
+                               continue;
+                       }
                        $sectionId = $translationUnit->getSectionId();
                        if ( $sectionId === 'mwcx-source-title' ) {
                                $validationResults[$sectionId] =
@@ -190,7 +193,9 @@
                                // Content can be null in case translator clear 
the section.
                                $tuData['content'] = null;
                        }
-
+                       if ( !isset( $tuData['valdiate'] ) ) {
+                               $tuData['valdiate'] = false;
+                       }
                        $tuData['translationId'] = 
$this->translation->getTranslationId();
                        $translationUnits[] = new TranslationUnit( $tuData );
                }
diff --git a/includes/TranslationUnit.php b/includes/TranslationUnit.php
index 7a529b3..a4b72bd 100644
--- a/includes/TranslationUnit.php
+++ b/includes/TranslationUnit.php
@@ -16,6 +16,7 @@
        protected $sequenceId;
        protected $content;
        protected $timestamp;
+       protected $validate;
 
        public function __construct( array $params ) {
                if ( isset( $params['translationId'] ) ) {
@@ -30,6 +31,7 @@
                } else {
                        $this->timestamp = wfTimestamp();
                }
+               $this->validate = (string)$params['validate'];
        }
 
        /**
@@ -72,4 +74,8 @@
        public function getContent() {
                return $this->content;
        }
+
+       public function getValidate() {
+               return $this->validate;
+       }
 }
diff --git a/modules/translation/ext.cx.translation.storage.js 
b/modules/translation/ext.cx.translation.storage.js
index 74ead77..365d103 100644
--- a/modules/translation/ext.cx.translation.storage.js
+++ b/modules/translation/ext.cx.translation.storage.js
@@ -18,6 +18,7 @@
 
        ContentTranslationStorage.prototype.init = function () {
                this.sections = {};
+               this.validationTracker = {};
                this.listen();
        };
 
@@ -163,8 +164,10 @@
                        if ( validations[ sectionId ] && Object.keys( 
validations[ sectionId ] ).length ) {
                                $targetSection.data( 'errors', validations[ 
sectionId ] );
                                mw.hook( 'mw.cx.translation.validation.error' 
).fire( $targetSection );
+                               this.validationTracker[ sectionId ].error = 
true;
                        } else {
                                $targetSection.removeData( 'errors' );
+                               this.validationTracker[ sectionId ].error = 
false;
                                mw.hook( 'mw.cx.translation.validation.success' 
).fire( $targetSection );
                        }
                }
@@ -201,7 +204,8 @@
        };
 
        ContentTranslationStorage.prototype.markForSave = function ( 
$targetSection ) {
-               var $sourceSection, sourceSectionId, targetSectionId, 
sequenceId, state, origin;
+               var $sourceSection, sourceSectionId, targetSectionId, 
sequenceId, state, origin,
+                       validate;
 
                targetSectionId = $targetSection.attr( 'id' );
                state = $targetSection.data( 'cx-state' );
@@ -214,19 +218,37 @@
                        origin = 'user';
                }
                sequenceId = $sourceSection.data( 'seqid' );
+
+               // To avoid large number of validations, we set validation flag 
in every 10th change of
+               // section or if the section has error. Or if the section has 
validation error.
+               this.validationTracker[ sourceSectionId ] = 
this.validationTracker[ sourceSectionId ] || {
+                       count: 1,
+                       error: false
+               };
+               validate = this.validationTracker[ sourceSectionId ].count % 10 
=== 0 ||
+                       this.validationTracker[ sourceSectionId ].error ||
+                       state === 'mt';
+
+               if ( !$targetSection.is( 'p, #cxmwcx-source-title' ) ) {
+                       // Avoid validating sections that are not paragraphs or 
target title.
+                       validate = false;
+               }
+               this.validationTracker[ sourceSectionId ].count++;
+
                this.sections[ targetSectionId ] = {
                        content: this.getContent( $targetSection ),
                        sectionId: sourceSectionId, // source section id is the 
canonical section id.
                        saved: false,
+                       validate: validate,
                        sequenceId: sequenceId,
                        origin: origin
                };
-
                // Source sections are saved only once.
                this.sections[ sourceSectionId ] = this.sections[ 
sourceSectionId ] || {
                        content: this.getContent( $sourceSection ),
                        sectionId: sourceSectionId,
                        saved: false,
+                       validate: false,
                        sequenceId: sequenceId,
                        origin: 'source'
                };

-- 
To view, visit https://gerrit.wikimedia.org/r/277480
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I8660527a891b77e272e9af1d6d3b0bbe3ee5ebf2
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/extensions/ContentTranslation
Gerrit-Branch: master
Gerrit-Owner: Santhosh <santhosh.thottin...@gmail.com>

_______________________________________________
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

Reply via email to