Spring ’19 (V3.28) Release Notes – February 18th 2019
- Merge Engine. Attribute Group Sub Groups. This new capability allows Data Sharing Consent flags to be taken from other matched records within the group that share common attributes with the Attribute Group Winning Record. For example, divisional (or channel-specific) email consent flags could be taken from records that provide a trusted source for a given division and have the same Email address as the winning record for the Attribute Group (from which all other attributes on the Master Record would be populated).
- Merge Engine. Attribute Groups. It is now possible for the merge engine to blank Source Record field values when populating the master – this is typically required where a unique constraint exists on the field.
- Matching Engine. Previously it was possible for a Matched status Source Record flagged as a standalone Master Record to be reverted to No Match status where no existing matches for the record are activated for a given matching cycle. This limitation has now been addressed.
v3.2 Release Notes – January 31st 2019 >
- Individual Object. The standard object, ‘Individual’ was introduced for data protection and privacy within Salesforce. Its purpose is to store a person’s data preferences with regards to how their data is stored, used and shared. clearMDM supports this feature by applying settings within the Target Object, which will allow the individual record to be updated to the Master Record after Matching and Merging based upon the following factors:
Data Privacy Record Retention Policy (Newest, Oldest, Keep) determined by the Data Privacy Record Retention Date Field (Created Date, Birth Date, Last Modified Date, System Modstamp, Last viewed date).
- Blocking Key Process Request (BKPR). Within Application Settings, Blocking Key Process Settings, there is now the ability to be able to set the BKPR to run up to 10 processors in parallel. Each processor executes in 2 phases with the Match, Merge and Synchronisation as phase 1 followed by Reparenting. Reparenting enables identified matched records, determined by the Blocking Key, that have transactions recorded to be updated to the Master Record. The Master Record then has all the transactional history from the Merge Source Record(s).
- Data Quality Rules. It is now possible to add a negative value as a Data Quality Rule as well as a positive value. This can be done through Settings, Data Quality Rule Set Settings. The value field now contains some negative values which can be added to the Quality Rule setup and which will help determine the Quality Score Metric in the Target Object.
- Cross Field Merge. When setting up an Attribute Group Template, you can now specify whether unique values only are used during the Merge process. An example of this could be a duplicate address. If Records have the same address, when the Master Record is created a duplicate value of the same address will not be merged to the Master Record.
- Multi Select Picklist. Attribute Groups now support Merging to the Master Record of all values from Multi-Select Picklist fields for all valid Source Records. When Records are Matched, and they contain a Multi-Select Picklist with multiple values, when the Master Record is created, these values will be Merged to display within the Master Record.
- Settings API. The ability to be able to setup the configuration settings within clearMDM from a source Salesforce org and deploy to one or more target Salesforce org (s). Reducing the time to setup configuration manually and also alleviating any human errors. Further details about this and how to implement can be found here.
- MDM Reset Reparenting. After the MDM operations have completed and where transactions (for example) have been reparented to the Master Record from the Source Record, if the Master Record is reset using the MDM Reset button, this will now remove the reparented transactions and return to the source record(s) in question.
v3.0 Release Notes – August 10th 2018 >
- Data Services. A new feature within clearMDM that provides the ability for record cleanse, verification or improved quality services via the use of external API connectors and supports the Google Translation API connector using a separate package. This enables translation from the source language to a target language with over 100 languages at various levels available. Some of the significant features of Google Translation API connector are:
- Provides a speedy and easy interface for different language conversation/translation.
- Converts about 100+ languages using Translation API.
- Identifies the local language in your native language without any errors.
A significant step for clearMDM and will prove to be a useful API connector.
2. Normalisation. It is now possible to update fields where there has been an abbreviation entered, so an example could be Street’ abbreviated with ‘Str’ or ‘St’. By setting up a custom setting list of match and replace values and referencing that via a lookup rule, the value of the field will be replaced with Street (in this example) where these abbreviations are used. Within Develop, Custom Settings there is ‘Replace Values’ where you can add a list of values to be replaced.
3. Merge. Within the Attribute Group Settings there is a field value for Priority Type. This can be Newest, Oldest or Dynamic which is based upon the operational source system timestamp. We are introducing a new priority which focuses specifically on date fields, so you will be able to pick the Newest or Oldest and the field this needs to relate to.
4. BKPR. A new feature has been added called Blocking Key Process Requests. Within the new tab, you can setup a request based upon the Blocking Key Value. Any record with that blocking key will then enter a process queue if any changes are made, based upon the MDM status. This in turn means any external systems will be updated with the new information thereby creating a realtime MDM environment.
Spring ’18 (v2.26) Release Notes – April 14th 2018 >
1) Cross Field Merge. The Merge operation now supports cross-field merge. A generic Attribute Group Template can be defined (e.g. Address, Phone or
any other grouping) – with placeholder fields. Attribute Groups (e.g. Billing Address, Shipping Address) can then be mapped to the template (field-by-field) and the population order set; i.e. the order in which the Attribute Group fields will be populated on the Master Record.
The Merge engine will gather all mapped values in priority order. The
highest priority values are then added to the Master Record fields within the
first Attribute Group (by population order) and so on.
2) E164 Normalisation Rule Type. Phone number values can now be normalised to the E164 format (e.g. +441912583886) using country specific prefixes driven by the record-level Country ISO Code value. Beyond standardisation and matching this capability will be particularly useful for CTI integrations such as NewVoiceMedia that require the E164 format.
3) Create Master for New Groups. Previously the Merge operation would designate an available Source Record of the correct type (within a matched group) as the Master Record. Now it is possible to specify that a new Master Record must always be created for new groups. This setting allows all Source Records to remain intact which can be critical where records are integrated with external systems.
v2.22 Release Notes – February 6th 2018 >
1) Asynchronous Actions. clearMDM supports transactional MDM processing models via packaged Apex Actions that can be incorporated into Process Builder or Flow (Visual Workflow) process automations. The packaged Apex Actions now support asynchronous processing which can reduce the risk of exceeding platform execution limits (in complex environments) by moving the MDM processing to a separate transaction.
2) Batch MDM Reset. The clearMDM package now includes a batch-based mechanism for the reset of MDM related fields and the removal of matched record pair and blocking key statistic records. This new capability introduces additional efficiency to testing and validation cycles.
3) Predictive Partitioning. The initial MDM processing of a dataset using parallel batch operations previously imposed a 1M record limit, record volumes above this required a single batch operation approach with a consequential impact on processing duration. With clearMDM 2.22 this limit is removed via a new predictive partitioning setting that enables massive data volumes to be processed using parallel operations.
v2.15 Release Notes – November 27th 2017 >
1) Winter ’18 Lightning Experience UI Compatibility. The clearMDM Lightning UI has been updated to align with the Winter ’18 Lightning Experience UI.
2) Job Monitoring. On rare occasion where 2 parallel batch job processes complete at the same time, the overall job status can stick at “In Progress” rather than moving to “Completed”. To address this a new automatic job monitoring capability has been added which periodically checks for this scenario and applies the correct job completion logic.
v2.12 Release Notes – October 15th 2017 >
1) Data Quality RuleSet Update Action. A new packaged Action has been added to support scenarios where control is required over the specific Data Quality RuleSets that are processed by the Data Quality MDM operation. Process automations can now be implemented (Process Builder, Flow etc.) that include steps which de-activate or re-activate specific RuleSets before/after the invocation of Data Quality operations.
2) Submit Batch Job Run Action – Deferred Start Hour. MDM jobs invoked via the [Batch Job Run] Action from within Process Builder or Flow can now specify the start hour for the job. If the start hour has passed for the current day, execution will be deferred to the specified hour on the next calendar day. Previously job invocation via Action has been limited to immediate execution only.
v2.11 Release Notes – October 9th 2017 >
1) Blocking Key Update Action. A new packaged Action has been added to support scenarios where multiple matching passes are required across different Blocking Key constructs. Process automations can now be implemented (Process Builder, Flow etc.) that blend job control functions (i.e. Job Submission) with updates to the Blocking Key configuration.
v2.9 Release Notes – September 9th 2017 >
1) Matching Check Notifications. Salesforce Classic only. Notifications shown in the UI when duplicates are found on record-save now include the calculated Match Score and View links to enable easy navigation.
2) QuickMatching API. The QuickMatching API operation now supports a new parameter that allows the blocking key to be overridden. The use case here is where the inputs required to construct a valid blocking key are unavailable at the point-of-entry and an alternative field (such as email) should be used instead.
3) QuickStart, Implementation Guide and API Guide documentation update.
v2.8 Release Notes – August 5th 2017 >
1) Advanced Merge. A key engineering focus over recent releases has been to deliver an advanced matching engine (cross-field matching being one such capability). Version 2.8 marks a change in direction toward the delivery of an advanced merge engine. Attribute Groups are the first feature in this regard and represent the most powerful capability added to the clearMDM product to date.
Attribute Groups allow groupings of fields to be defined that are Merged (or Synchronised) together. All fields in an Attribute Group populate to the Master Record from one Source Record only. A common example is address data where the best quality, or most recent (and entire) address should be utilised. Attribute Groups provide additional advanced merge capabilities:
1.1) Existing Record Evaluation. Attribute Group evaluation considers Matched Records, related Master records and also any existing Source Records related to those Master Records. This latter inclusion is new and exclusive to Attribute Groups.
1.2) Priority Types. Attribute Groups support Newest, Oldest and Dynamic priority types. For time-based priority types a Datetime field must be specified to enable record evaluation. For the dynamic priority type a Numeric field must be specified; this could be a field populated by the Data Quality MDM operation (e.g. Address Quality Score) – in a model where the highest quality address is merged to the Master Record. Dynamic priority is a radical shift from static (Data Source specific) field-priorities; the dynamic approach enables record field values to drive the prioritisation of Master Record field population.
2) Record Reparenting Action. A new Action has been added to enable the Reparenting MDM operation to be executed from Process Builder, Flow or the Rest API. A complete cycle of MDM processing can now be implemented in a transactional model, i.e. Source Records match, merge and initiate Child Record reparenting in near-real time.
3) Check-only Data Sources. clearMDM now supports cross-object matching between standard objects. The main use case here is checking between Leads, Contacts and/or Person Accounts – however the approach is generic enough to allow Opportunity checking against Case should this requirement ever arise! More typically, Lead creation (file import, UI data entry etc.) can be blocked where the Lead matches to an existing Contact.
v2.4 Release Notes – June 11th 2017 >
1) Target Object Settings UI. The new streamlined Target Object Settings UI enables fields to be selected and added as rules, rather than displaying all fields defined on the object by default. This optimisation addresses performance issues where the custom field count is high.
2) Synchronisation Operation Update. The Synchronisation MDM operation now checks the existing field population state on the Master Record and preserves a non-blank field value unless a Source Record with a higher merge priority exists. This update aligns the Synchronisation and Merge MDM operations.
v2.2 Release Notes – May 17th 2017 >
1) QuickStart Documentation. The new QuickStart guides provide a comprehensive overview of the clearMDM product (and key use cases) plus step-by-step implementation guidance for each MDM operation (including the new Data Quality feature-set).
2) Adhoc Merge. The “Matching Test” feature now supports record merge from the Compare step. This new capability allows related records to be found through field filters and directly merged. A common Blocking Key is not necessary for adhoc merge.
3) Re-merge. The “MDM Reset” feature now allows previously merged record groups to be re-merged. A new “Re-merge” button appears when a record with MDM Status equal to “Merge Master” is to be reset. The new button reverts the related group of records back to the Matched Record Group stage allowing records to be removed, a new master record selection to be applied plus modifications to individual field population selections.
4) Last Synchronisation Date. The Synchronisation MDM operation now updates a distinct “Last Synchronisation Date” field, previously the “Last Merged Date” field was shared with the Merge MDM operation.
5) Lightning Experience UI updates. Enhancements have been applied across the User Interface to align with the latest version of the Salesforce Lightning Design system (SLDS).
v2.0 Release Notes – April 11th 2017 >
1) Introducing clearMDM Editions; Basic, Mid and Enterprise.
2) Data Quality Rule Sets, Rules and Actions.
v1.28 Release Notes – April 3rd 2017 >
1) Cross-field matching. Check for field value matches across multiple fields. This new capability is primarily intended for matching Phone numbers, Email addresses and Address data where the matching values may exist in any one of a number of potential fields (e.g. Phone, Home Phone or Mobile).
2) Matching rules support Date and Datetime fields. Enable matching rules on Person Date-of-birth etc.
3) Exact Matching. A new Match Rule Type “Exact” requires that field values are identical.
4) Matched Record Group Splits. When candidate matches are rejected, it is possible that matched record group becomes logically split into sub-groups, i.e the chain of matches is broken. The Matched Record Group page recognises this situation and allows the sub-groups to be saved as distinct groups. In such cases a group column appears in the table and a “Save & Split” button appears.
v1.26 Release Notes – February 20th 2017 >
1) Point-of-Entry Default Search Field. Previously the default search field on the Search-before-Create page was the first field in alphabetic sort order. Now the most relevant field can be selected via the “Default?” checkbox option displayed against each active field on the Target Object setting page.
2) Override Matching Check. For many reasons it can be desirable to apply conditions as to when Matching Checks are applied at the time of record create or edit. Version 1.26 allows a checkbox formula field to be referenced that returns a True value where matching checks should not applied. The formula expression can encode the required conditions (specific user, profiles, field value permutations etc.).
3) Find Matches. Previously the clearMDM real-time matching UI returned all match candidates for the Blocking Key irrespective of whether the candidate pair included the record from which matching was invoked. With 1.26, the Find Matches page offers “Find” and “Find All” options; the former restricts the results to the current record, the latter returns all matches for the blocking key.
4) Manual Merge. The Manual Merge page has been enhanced to provide better support for different data types (dates, email addresses, numbers etc.) and null values within the merge override area.
5) Record-Level Flag Management. Previously management of the [Is Normalised?] and [Is Active for Matching?] record-level flags has been a function of data integration processing or applied via on-platform workflow/process. With 1.26, clearMDM can manage both flags automatically by detecting changes to significant fields as they happen.
v1.25 Release Notes – February 2nd 2017 >
1) Selective Re-parenting. A new Custom Setting [Manual Merge Settings] provides control (org, profile and user level) over the reparenting option visibility and default behaviour on the Manual Merge page. Additionally, where the reparenting is invoked from the Manual Merge page it runs in a selective, non-blocking mode meaning only the source records involved in the merge are processed and multiple instances can run concurrently. This new functionality provides support for implementations where manual matching and merge functions are de-centralised across business users, rather than being centralised to one or more data stewards.
2) Matched Record Group Size Visibility. A new Application Setting [Max UI Visible MRG Size] has been added to provide specific control over the maximum size of a Matched Record Group visible via the clearMDM UI. Previously this was controlled by batch job related settings. The new setting allows larger groups to be opened and matches accepted/rejected prior to Merge.
3) QuickMatching API operation. The new API operation supports ad-hoc use cases such as distributed MDM record matching checks – where data modifications should not be persisted. A JSON record representation is passed to the operation which performs in-memory normalisation and matching MDM operations. The returned results can include a requested list of field values (for example the SAP Id) to enable downstream business logic to be applied in the source system.
v1.22 Release Notes – November 26th 2016 >
1) MDM Reset. A new UI feature that resets MDM fields and relationships across Source and Master Records. For example, if a Merge Master record is reset all related Source Records will be reset also. This convenient functionality allows inadvertent matches or incorrectly merged records to be reset to the start of the MDM record life-cycle.
v1.19 Release Notes – 18th October 2016 >
1) Blocking Key Statistics. A new MDM operation “Blocking Key Statistics” allows pre-calculation of statistics relating to the defined Blocking Key configuration. Calculated statistics can be used to fine-tune the configuration and are also referenced to optimise the performance of Matching MDM operations.
2) Custom Merge Logic. Prior to the Winter ’17 release, the algorithm employed by Merge MDM operations to select the master record could not be customised. With the new release custom business rules can be implemented to prioritise (and de-prioritise) individual records and thereby allow fine-grained control over how master records are selected.
v1.17 Release Notes – 25th September 2016 >
1) Real-time MDM Actions. clearMDM now supports transactional MDM processing for individual records (or groups) in real-time via Process Builder or the REST API. Custom business logic can be implemented via Visual Workflow and exposed as REST API resources.
v1.16 Release Notes – 9th September 2016 >
1) Field Priority Logic. Master Records now retain a memory of how individual fields were last populated – the memory is used to prevent overwrite of manual changes or values populated from higher priority data sources.
2) Synchronisation MDM operation. A new one-step operation that allows Source Record updates to be applied to directly to Master Records. Synchronisation respects field priority logic. An efficient, scalable implementation model becomes possible where Source Records linked to Master Records synchronise change and only new Source Records are matched and merged.
v1.15 Release Notes – 15th August 2016 >
1) Lightning Ready Certified. clearMDM is now fully Lightning Experience compatible. The app user interface dynamically switches between Salesforce Classic and Lightning Experience based on the current view selected by the user.
v1.12 Release Notes – 17th April 2016 >
1) Real-time Matching. Prior to this release the clearMDM real-time matching features have respected the record-level [Is Active for Matching?] flag by default (where configured). As of v1.12 real time matching features do not require this flag to be set, enabling candidate matching records to be identified irrespective of their matching flag status.
2) Custom Rollups for Text fields. Custom Rollups can now be configured to concatenate text fields from child records (transactional data) to a text field on the master record. A useful feature for bringing note information together on the master record to provide a complete at-a-glance view.
v1.10 Release Notes – 8th March 2016 >
1) Blocking Key Padding. Blocking key inputs can now be padded up to the required blocking key length. The Blocking Key Padding Flag is set per Target Object.
2) Custom Rollup Incremental Addition Mode. Custom Rollups can now be configured to process only records created since the last time of rollup processing. With the 1.10 release only the SUM function is supported for incremental processing.
v1.8 Release Notes – 7th February 2016 >
1) Blocking Key Auto-adjustment. With larger data sets it is possible for blocking key match values to cover large volumes of records. The Matching MDM operation can now be configured to dynamically adjust the length of the overall blocking key used in creating matching groups. This fine-tuning prevents blocking key match values being skipped due to the size of records (Max Records per Group setting).
2) Master Record Inclusion. Master Record inclusion allows Master Records to be selectively exposed to the Matching MDM operation for blocking keys found within the Source Record data set. With version 1.8 matches between Master Records exposed by this logic are prevented; only Source Record to Master Record matching paths are supported.
3) Strict Blocking Key Population. In Strict mode blocking keys are populated only where each input is of sufficient length to match the required input length. This feature avoids unusable blocking keys being created and enables early reporting.
v1.0 Release Notes – 30th December 2015 >
clearMDM has passed the Salesforce AppExchange Security Review process and will be publicly listed on the AppExchange from the 31st December 2015.
RC1 Release Notes – 28th October 2015 >
1) Data Source Specific Operations. The Matching, Conversion and Re-parenting jobs now support a data source list parameter. This enables operations to run for specific data sources only. For large data volume scenarios partition data sources can be defined and processed independently. Where compatible jobs are chained the data sources parameter is passed to the next job in the sequence.
Beta 9 Release Notes – 19th October 2015 >
1) Custom Action Support. clearMDM batch jobs can now be invoked via Custom Actions. This capability enables MDM operations to be invoked from Process Builder or via the REST API. In the latter case, invocation of MDM operations can now be incorporated into data integration tasks.
Beta 8 Release Notes – 13th October 2015 >
1) Audit Log Control. Log types provide control over the logging behaviour of the clearMDM package, valid values are (None, Normal, Debug).
Beta 7 Release Notes – 21st September 2015 >
1) Formula Field Support. Field-level matching rules can now employ formula fields. In combination with Key and Deterministic matching rule types, matching efficiency can be increased as fuzzy matching processing overhead is reduced.
2) Job Chaining. MDM operations run in a logical order (normalisation, match, merge etc.) Jobs can now be chained in sequence to enabled automated, efficient batch processing.
3) Custom Rollups. Rollup summary fields over lookup relationships. This new feature enables child record reparenting to be employed without losing the key benefits of rollup summary fields which are limited to master detail relationships. Note, numeric fields and the following functions are support for the first release:
SUM, AVERAGE, MIN, MAX, COUNT
4) Internal matching. Where all records to be matched exist in the same object, internal matching provides an efficient, highly scalable matching operation.
Beta 6 Release Notes – 10th September 2015 >
1) New Normalisation rules types.
Title : Each word in the Source Field Value is Title-cased.
Use case = Standardisation of last names, e.g. smith, Smith (Smith).
Title Hyphen : Each word in the Source Field Value is Title-cased, words are hyphen separated.
Use case = Standardisation of two-part first names, e.g. Anne marie, Anne Marie (Anne-Marie).
2) The Merge MDM Operation can now reset Master Record Identifier fields (populated from specified Source Record fields), or concatenate new fields to the existing data.
This behaviour is controlled by the Target Object Merge setting [Reset Identifier Target Fields?].