[MediaWiki-commits] [Gerrit] wikimedia...crm[master]: Update Omnimail GET to add rml fields

2017-07-26 Thread jenkins-bot (Code Review) 
jenkins-bot has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/365543 )

Change subject: Update Omnimail GET to add rml fields
..


Update Omnimail GET to add rml fields

I think these fields are specific to our DB so have treated them like custom 
data,
rather than adding getLanguage() etc functions.

Add Omnimmembergroup load functions

Perhaps I'm just Silverpopped out but this felt a bit harder to keep generic &
keep the lines between what should do what. I've cut a few corners, trying to 
do more
wmf-like handling in the main load function. Hopefully I've mitigated that 
somewhat
with tests & comments.

This should allow us to get our group of contacts with no id & to create into 
our DB with

drush cvapi 'Omnigroupmember', 'load', array('mail_provider' => 'Silverpop', 
'group_identifier' => 18468760, 'group_id' => 777);

Whre group_identifier is the id in the phab & group_id is the id of an 
as-yet-to-be-created
group in CiviCRM.

Data brought in is
- email
- language (if we can figure out what it is)
- country (if matching DB countries)
- source is a combo of the fact it is silverpop + the source in the rml form + 
the submit date
I thought about putting submit date in created_date but it's arguable whether 
that is a good idea as there are 2 separate events. That can't be done through 
the api on create if we do wish to.
- is_opt_out  - see notes below

Is OPT out
in the data set I got some contacts were already opted out. I need to dig a 
little
more here as there is opt in and opt out and I'm not sure if opt in is the 
absence of opt
out or an active opt in. There are some flags that can be set when fetching 
data that
can manipulate what is fetched to I need to answer that here.

Also, although I was purposely not setting the 'skip ones already downloaded' 
some of
my later downloads seemed a little light - I need to do a little more testing 
to check that
out.

I think clarifying these opt-out questions is probably the last step in the 
silverpop chunk of
work

Bug T160949

Change-Id: Ia0dc97b84603ca3cd32aba17094593cfad95900f
---
M composer.lock
M 
sites/default/civicrm/extensions/org.wikimedia.omnimail/api/v3/Omnigroupmember/Get.php
A 
sites/default/civicrm/extensions/org.wikimedia.omnimail/api/v3/Omnigroupmember/Load.php
M 
sites/default/civicrm/extensions/org.wikimedia.omnimail/tests/phpunit/OmnigroupmemberGetTest.php
A 
sites/default/civicrm/extensions/org.wikimedia.omnimail/tests/phpunit/OmnigroupmemberLoadTest.php
M 
sites/default/civicrm/extensions/org.wikimedia.omnimail/tests/phpunit/Responses/20170509_noCID
 - All - Jul 5 2017 06-27-45 AM.csv
6 files changed, 252 insertions(+), 4 deletions(-)

Approvals:
  jenkins-bot: Verified
  Ejegg: Looks good to me, approved



diff --git a/composer.lock b/composer.lock
index eb004f2..b6be6ff 100644
--- a/composer.lock
+++ b/composer.lock
@@ -1842,7 +1842,7 @@
 "source": {
 "type": "git",
 "url": 
"https://github.com/eileenmcnaughton/omnimail-silverpop.git;,
-"reference": "c48de4450ec495a8fd1cb26ba5e656876321e025"
+"reference": "cd3933b8ee7b263d0472cc44d047b82f3ec4029c"
 },
 "require": {
 "league/csv": "^8.0",
diff --git 
a/sites/default/civicrm/extensions/org.wikimedia.omnimail/api/v3/Omnigroupmember/Get.php
 
b/sites/default/civicrm/extensions/org.wikimedia.omnimail/api/v3/Omnigroupmember/Get.php
index e813920..0f179f6 100644
--- 
a/sites/default/civicrm/extensions/org.wikimedia.omnimail/api/v3/Omnigroupmember/Get.php
+++ 
b/sites/default/civicrm/extensions/org.wikimedia.omnimail/api/v3/Omnigroupmember/Get.php
@@ -19,7 +19,7 @@
   $options = _civicrm_api3_get_options_from_params($params);
   $values = array();
   foreach ($result as $groupMember) {
-$values[] = array(
+$value = array(
   'email' => (string) $groupMember->getEmail(),
   'is_opt_out' => (string) $groupMember->isOptOut(),
   'opt_in_date' => (string) $groupMember->getOptInIsoDateTime(),
@@ -28,6 +28,10 @@
   'opt_out_date' => (string) $groupMember->getOptOutIsoDateTime(),
   'contact_id' => (string) $groupMember->getContactReference(),
 );
+foreach ($params['custom_data_map'] as $fieldName => $dataKey) {
+  $value[$fieldName] = (string) $groupMember->getCustomData($dataKey);
+}
+$values[] = $value;
 if ($options['limit'] > 0 && count($values) === (int) $options['limit']) {
   break;
 }
@@ -68,5 +72,16 @@
   $params['retrieval_parameters'] = array(
 'title' => ts('Additional information for retrieval of pre-stored 
requests'),
   );
+  $params['custom_data_map'] = array(
+'type' => CRM_Utils_Type::T_STRING,
+'title' => ts('Custom fields map'),
+'description' => array('custom mappings pertaining to the mail provider 
fields'),
+'api.default' => array(
+  'language' => 'rml_language',
+

[MediaWiki-commits] [Gerrit] wikimedia...crm[master]: Update Omnimail GET to add rml fields

2017-07-16 Thread Eileen (Code Review) 
Eileen has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/365542 )

Change subject: Update Omnimail GET to add rml fields
..

Update Omnimail GET to add rml fields

I think these fields are specific to our DB so have treated them like custom 
data,
rather than adding getLanguage() etc functions.

Bug: T160949
Change-Id: I91320432dd8902484510a8db117717b3e643858d
---
M 
sites/default/civicrm/extensions/org.wikimedia.omnimail/api/v3/Omnigroupmember/Get.php
M 
sites/default/civicrm/extensions/org.wikimedia.omnimail/tests/phpunit/OmnigroupmemberGetTest.php
M 
sites/default/civicrm/extensions/org.wikimedia.omnimail/tests/phpunit/Responses/20170509_noCID
 - All - Jul 5 2017 06-27-45 AM.csv
3 files changed, 21 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm 
refs/changes/42/365542/1

diff --git 
a/sites/default/civicrm/extensions/org.wikimedia.omnimail/api/v3/Omnigroupmember/Get.php
 
b/sites/default/civicrm/extensions/org.wikimedia.omnimail/api/v3/Omnigroupmember/Get.php
index e813920..0f179f6 100644
--- 
a/sites/default/civicrm/extensions/org.wikimedia.omnimail/api/v3/Omnigroupmember/Get.php
+++ 
b/sites/default/civicrm/extensions/org.wikimedia.omnimail/api/v3/Omnigroupmember/Get.php
@@ -19,7 +19,7 @@
   $options = _civicrm_api3_get_options_from_params($params);
   $values = array();
   foreach ($result as $groupMember) {
-$values[] = array(
+$value = array(
   'email' => (string) $groupMember->getEmail(),
   'is_opt_out' => (string) $groupMember->isOptOut(),
   'opt_in_date' => (string) $groupMember->getOptInIsoDateTime(),
@@ -28,6 +28,10 @@
   'opt_out_date' => (string) $groupMember->getOptOutIsoDateTime(),
   'contact_id' => (string) $groupMember->getContactReference(),
 );
+foreach ($params['custom_data_map'] as $fieldName => $dataKey) {
+  $value[$fieldName] = (string) $groupMember->getCustomData($dataKey);
+}
+$values[] = $value;
 if ($options['limit'] > 0 && count($values) === (int) $options['limit']) {
   break;
 }
@@ -68,5 +72,16 @@
   $params['retrieval_parameters'] = array(
 'title' => ts('Additional information for retrieval of pre-stored 
requests'),
   );
+  $params['custom_data_map'] = array(
+'type' => CRM_Utils_Type::T_STRING,
+'title' => ts('Custom fields map'),
+'description' => array('custom mappings pertaining to the mail provider 
fields'),
+'api.default' => array(
+  'language' => 'rml_language',
+  'source' => 'rml_source',
+  'created_date' => 'rml_submitdate',
+  'country' => 'rml_country',
+),
+  );
 
 }
diff --git 
a/sites/default/civicrm/extensions/org.wikimedia.omnimail/tests/phpunit/OmnigroupmemberGetTest.php
 
b/sites/default/civicrm/extensions/org.wikimedia.omnimail/tests/phpunit/OmnigroupmemberGetTest.php
index 086029c..0509181 100644
--- 
a/sites/default/civicrm/extensions/org.wikimedia.omnimail/tests/phpunit/OmnigroupmemberGetTest.php
+++ 
b/sites/default/civicrm/extensions/org.wikimedia.omnimail/tests/phpunit/OmnigroupmemberGetTest.php
@@ -52,7 +52,10 @@
 $this->assertEquals('2017-07-04 11:11:00', 
$result['values'][0]['opt_out_date']);
 $this->assertEquals('Added by WebForms', 
$result['values'][0]['opt_in_source']);
 $this->assertEquals('Opt out via email opt out.', 
$result['values'][0]['opt_out_source']);
-
+$this->assertEquals('clever place', $result['values'][2]['source']);
+$this->assertEquals('US', $result['values'][2]['country']);
+$this->assertEquals('en', $result['values'][2]['language']);
+$this->assertEquals('07/04/17', $result['values'][2]['created_date']);
   }
 
 
diff --git 
a/sites/default/civicrm/extensions/org.wikimedia.omnimail/tests/phpunit/Responses/20170509_noCID
 - All - Jul 5 2017 06-27-45 AM.csv 
b/sites/default/civicrm/extensions/org.wikimedia.omnimail/tests/phpunit/Responses/20170509_noCID
 - All - Jul 5 2017 06-27-45 AM.csv
index 0f37c0d..b3e791d 100644
--- 
a/sites/default/civicrm/extensions/org.wikimedia.omnimail/tests/phpunit/Responses/20170509_noCID
 - All - Jul 5 2017 06-27-45 AM.csv
+++ 
b/sites/default/civicrm/extensions/org.wikimedia.omnimail/tests/phpunit/Responses/20170509_noCID
 - All - Jul 5 2017 06-27-45 AM.csv
@@ -1,4 +1,4 @@
 Email,Opt In Date,Opted Out,Opt In Details,Email Type,Opted Out Date,Opt Out 
Details,ContactID,IsoLang,LastClickDate,LastOpenDate,LastSentDate,Segment,Segment_AOL,Segment_Newsletter,Segment_Newsletter2,SendHour,firstname,lastname,postal_code,rml_country,rml_device,rml_language,rml_phone,rml_segment,rml_source,rml_submitDate,state,timezone
 e...@example.com,10/18/16 08:01 PM,T,Added by WebForms,0,07/04/17 11:11 AM,Opt 
out via email opt out.12/04/16,10/22/16,0,0,0,0,9FR,,fr,,35,,10/18/16,,
 sa...@example.com,12/13/16 12:27 AM,T,Added by WebForms,0,07/03/17 03:34 
PM,Optout by WebForms,12/24/16,0,0,0,0,0US,,en,,83,,12/12/16,,