Eileen has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/365543 )
Change subject: Add Omnimmembergroup load functions ...................................................................... Add Omnimmembergroup load functions Perhaps I'm just Silverpopped out but this felt a bit harder to keep generic & keep the lines between what should do what. I've cut a few corners, trying to do more wmf-like handling in the main load function. Hopefully I've mitigated that somewhat with tests & comments. This should allow us to get our group of contacts with no id & to create into our DB with drush cvapi 'Omnigroupmember', 'load', array('mail_provider' => 'Silverpop', 'group_identifier' => 18468760, 'group_id' => 777); Whre group_identifier is the id in the phab & group_id is the id of an as-yet-to-be-created group in CiviCRM. Data brought in is - email - language (if we can figure out what it is) - country (if matching DB countries) - source is a combo of the fact it is silverpop + the source in the rml form + the submit date I thought about putting submit date in created_date but it's arguable whether that is a good idea as there are 2 separate events. That can't be done through the api on create if we do wish to. - is_opt_out - see notes below Is OPT out in the data set I got some contacts were already opted out. I need to dig a little more here as there is opt in and opt out and I'm not sure if opt in is the absence of opt out or an active opt in. There are some flags that can be set when fetching data that can manipulate what is fetched to I need to answer that here. Also, although I was purposely not setting the 'skip ones already downloaded' some of my later downloads seemed a little light - I need to do a little more testing to check that out. I think clarifying these opt-out questions is probably the last step in the silverpop chunk of work Bug T160949 Change-Id: Ia0dc97b84603ca3cd32aba17094593cfad95900f --- A sites/default/civicrm/extensions/org.wikimedia.omnimail/api/v3/Omnigroupmember/Load.php A sites/default/civicrm/extensions/org.wikimedia.omnimail/tests/phpunit/OmnigroupmemberLoadTest.php 2 files changed, 230 insertions(+), 0 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm refs/changes/43/365543/1 diff --git a/sites/default/civicrm/extensions/org.wikimedia.omnimail/api/v3/Omnigroupmember/Load.php b/sites/default/civicrm/extensions/org.wikimedia.omnimail/api/v3/Omnigroupmember/Load.php new file mode 100644 index 0000000..5fcc26f --- /dev/null +++ b/sites/default/civicrm/extensions/org.wikimedia.omnimail/api/v3/Omnigroupmember/Load.php @@ -0,0 +1,145 @@ +<?php +/** + * Created by IntelliJ IDEA. + * User: emcnaughton + * Date: 5/3/17 + * Time: 12:46 PM + */ + +/** + * Get details about Omnimails. + * + * @param $params + * + * @return array + */ +function civicrm_api3_omnigroupmember_load($params) { + $contacts = civicrm_api3('Omnigroupmember', 'get', $params); + $defaultLocationType = CRM_Core_BAO_LocationType::getDefault(); + $locationTypeID = $defaultLocationType->id; + + foreach ($contacts['values'] as $groupMember) { + if (!empty($groupMember['email']) && !civicrm_api3('email', 'getcount', array('email' => $groupMember['email']))) { + // If there is already a contact with this email we will skip for now. + // It might that we want to create duplicates, update contacts or do other actions later + // but let's re-assess when we see that happening. Spot checks only found emails not + // otherwise in the DB. + $source = (empty($params['mail_provider']) ? ts('Mail Provider') : $params['mail_provider']) . ' ' . (!empty($groupMember['source']) ? $groupMember['source'] : $groupMember['opt_in_source']); + $source .= ' ' . $groupMember['created_date']; + + $contactParams = array( + 'contact_type' => 'Individual', + 'email' => $groupMember['email'], + 'is_opt_out' => $groupMember['is_opt_out'], + 'source' => $source, + 'preferred_language' => _civicrm_api3_omnigroupmember_get_language($groupMember), + ); + + if (!empty($groupMember['country']) && _civicrm_api3_omnigroupmember_is_country_valid($groupMember['country'])) { + $contactParams['api.address.create'] = array( + 'country_id' => $groupMember['country'], + 'location_type_id' => $locationTypeID, + ); + } + + $contact = civicrm_api3('Contact', 'create', $contactParams); + if (!empty($params['group_id'])) { + civicrm_api3('GroupContact', 'create', array( + 'group_id' => $params['group_id'], + 'contact_id' => $contact['id'], + )); + } + $values[$contact['id']] = reset($contact['values']); + } + } + return civicrm_api3_create_success($values); +} + +/** + * Get the contact's language. + * + * This is a place in the code where I am struggling to keep wmf-specific coding out + * of a generic extension. The wmf-way would be to call the wmf contact_insert function. + * + * That is not so appropriate from an extension, but we have language/country data that + * needs some wmf specific handling as it might or might not add up to a legit language. + * + * At this stage I'm compromising on containing the handling within the extension, + * ensuring test covering and splitting out & documenting the path taken /issue. + * Later maybe a more listener/hook type approach is the go. + * + * It's worth noting this is probably the least important part of the omnimail work + * from wmf POV. + * + * @param array $params + * + * @return string|null + */ +function _civicrm_api3_omnigroupmember_get_language($params) { + static $languages = NULL; + if (!$languages) { + $languages = civicrm_api3('Contact', 'getoptions', array('field' => 'preferred_language', 'limit' => 0)); + $languages = $languages['values']; + } + $attempts = array( + $params['language'] . '_' . strtoupper($params['country']), + $params['language'], + ); + foreach ($attempts as $attempt) { + if (isset($languages[$attempt])) { + return $attempt; + } + } + return NULL; +} + +/** + * Check if the country is valid. + * + * @param string $country + * + * @return bool + */ +function _civicrm_api3_omnigroupmember_is_country_valid($country) { + static $countries = NULL; + if (!$countries) { + $countries = CRM_Core_PseudoConstant::countryIsoCode(); + } + return array_search($country, $countries) ? $country : FALSE; +} + +/** + * Get details about Omnimails. + * + * @param $params + */ +function _civicrm_api3_omnigroupmember_load_spec(&$params) { + $params['username'] = array( + 'title' => ts('User name'), + ); + $params['password'] = array( + 'title' => ts('Password'), + ); + $params['mail_provider'] = array( + 'title' => ts('Name of Mailer'), + 'api.required' => TRUE, + ); + $params['start_date'] = array( + 'title' => ts('Date to fetch from'), + 'api.default' => '3 days ago', + 'type' => CRM_Utils_Type::T_TIMESTAMP, + ); + $params['end_date'] = array( + 'title' => ts('Date to fetch to'), + 'type' => CRM_Utils_Type::T_TIMESTAMP, + ); + $params['group_identifier'] = array( + 'title' => ts('Identifier for the group'), + 'type' => CRM_Utils_Type::T_STRING, + 'api.required' => TRUE, + ); + $params['retrieval_parameters'] = array( + 'title' => ts('Additional information for retrieval of pre-stored requests'), + ); + +} diff --git a/sites/default/civicrm/extensions/org.wikimedia.omnimail/tests/phpunit/OmnigroupmemberLoadTest.php b/sites/default/civicrm/extensions/org.wikimedia.omnimail/tests/phpunit/OmnigroupmemberLoadTest.php new file mode 100644 index 0000000..27c9c94 --- /dev/null +++ b/sites/default/civicrm/extensions/org.wikimedia.omnimail/tests/phpunit/OmnigroupmemberLoadTest.php @@ -0,0 +1,85 @@ +<?php + +use Civi\Test\EndToEndInterface; +use Civi\Test\HookInterface; +use Civi\Test\TransactionalInterface; +use GuzzleHttp\Client; +use GuzzleHttp\Handler\MockHandler; +use GuzzleHttp\HandlerStack; +use GuzzleHttp\Psr7\Response; +require_once __DIR__ . '/OmnimailBaseTestClass.php'; + +/** + * FIXME - Add test description. + * + * Tips: + * - With HookInterface, you may implement CiviCRM hooks directly in the test class. + * Simply create corresponding functions (e.g. "hook_civicrm_post(...)" or similar). + * - With TransactionalInterface, any data changes made by setUp() or test****() functions will + * rollback automatically -- as long as you don't manipulate schema or truncate tables. + * If this test needs to manipulate schema or truncate tables, then either: + * a. Do all that using setupHeadless() and Civi\Test. + * b. Disable TransactionalInterface, and handle all setup/teardown yourself. + * + * @group e2e + */ +class OmnigroupmemberLoadTest extends OmnimailBaseTestClass implements EndToEndInterface, TransactionalInterface { + + public function setUpHeadless() { + // Civi\Test has many helpers, like install(), uninstall(), sql(), and sqlFile(). + // See: https://github.com/civicrm/org.civicrm.testapalooza/blob/master/civi-test.md + return \Civi\Test::e2e() + ->installMe(__DIR__) + ->apply(); + } + + public function tearDown() { + parent::tearDown(); + } + + /** + * Example: Test that a version is returned. + */ + public function testOmnigroupmemberLoad() { + $client = $this->setupSuccessfulDownloadClient(); + $group = civicrm_api3('Group', 'create', array('name' => 'Omnimailers', 'title' => 'Omni')); + + civicrm_api3('Omnigroupmember', 'load', array('mail_provider' => 'Silverpop', 'username' => 'Shrek', 'password' => 'Fiona', 'options' => array('limit' => 3), 'client' => $client, 'group_identifier' => 123, 'group_id' => $group['id'])); + $groupMembers = civicrm_api3('GroupContact', 'get', array('group_id' => $group['id'])); + $this->assertEquals(3, $groupMembers['count']); + $contactIDs = array('IN' => array()); + foreach ($groupMembers['values'] as $groupMember) { + $contactIDs['IN'][] = $groupMember['contact_id']; + } + $contacts = civicrm_api3('Contact', 'get', array( + 'contact_id' => $contactIDs, + 'sequential' => 1, + 'return' => array('contact_source', 'email', 'country', 'created_date', 'preferred_language', 'is_opt_out') + )); + $this->assertEquals('fr_FR', $contacts['values'][0]['preferred_language']); + $this->assertEquals('e...@example.com', $contacts['values'][0]['email']); + $this->assertEquals('France', $contacts['values'][0]['country']); + $this->assertEquals(1, $contacts['values'][0]['is_opt_out']); + $this->assertEquals('Silverpop Added by WebForms 10/18/16', $contacts['values'][0]['contact_source']); + + $this->assertEquals('Silverpop clever place 07/04/17', $contacts['values'][2]['contact_source']); + + } + + + /** + * @return \GuzzleHttp\Client + */ + protected function setupSuccessfulDownloadClient() { + $responses = array( + file_get_contents(__DIR__ . '/Responses/ExportListResponse.txt'), + file_get_contents(__DIR__ . '/Responses/JobStatusCompleteResponse.txt'), + ); + copy(__DIR__ . '/Responses/20170509_noCID - All - Jul 5 2017 06-27-45 AM.csv', sys_get_temp_dir() . '/20170509_noCID - All - Jul 5 2017 06-27-45 AM.csv'); + fopen(sys_get_temp_dir() . '/20170509_noCID - All - Jul 5 2017 06-27-45 AM.csv.complete', 'c'); + $this->createSetting('omnimail_omnigroupmembers_load', array('Silverpop' => array('last_timestamp' => '1487890800'))); + + $client = $this->getMockRequest($responses); + return $client; + } +} -- To view, visit https://gerrit.wikimedia.org/r/365543 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ia0dc97b84603ca3cd32aba17094593cfad95900f Gerrit-PatchSet: 1 Gerrit-Project: wikimedia/fundraising/crm Gerrit-Branch: master Gerrit-Owner: Eileen <emcnaugh...@wikimedia.org> _______________________________________________ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits