de en


Clear lists: Duplicate matching

Duplicate matching combines subscribers with identical content but different subscriber IDs under a single subscriber ID.

This way, you not only avoid multiple deliveries to the same recipient, but also collect all activities (clicks, opens) and interest values in a single subscriber.

Duplicate matching does not lose any subscriber. The subscriber's membership in subscriber lists also remains unchanged, i.e. the merged subscriber can belong to several subscriber lists without any problem.

What are duplicates?

Every subscriber has a unique subscriber ID. In practice, however, the same subscriber sometimes subscribes several times or is imported several times.

As a result, subscribers with identical content receive different subscriber IDs and are therefore treated by emma as different recipients. If these duplicates are not cleaned up, this leads to unnecessary disadvantages.

Duplicates are created if you

  • ... create the same recipient several times in the 'Subscriber management', e.g. in different subscriber lists. Solution: Edit the existing subscriber and assign another list to them ('List membership' tab).
  • ... allow the same subscriber to subscribe several times in the 'Opt-in setup'. Solution: Select a duplicate matching setup, so that the duplicate is cleaned up directly after the subscription.
  • ... in the subscriber 'Import', selected the import type 'Add' or 'Replace'. Solution: Select the import type 'Update and add'. This updates existing subscribers with the new import data instead of creating new duplicates with every import.

New duplicate matching

On the 'New search' tab, select the subscriber list(s) to be searched for duplicates in the 'Match lists' area. If you select more than one list, the search runs across all selected lists.

Then define the 'Match fields' and the 'Follow-up action' you want, or simply select a 'Duplicate matching setup' in which you have already saved the settings you want.

Match fields

You define which subscribers count as "identical in content" by selecting the 'Match fields'. Typical examples:

  • Sending by email: same email address
  • Sending by SMS: same phone number
  • Sending by postcard: same first name + last name + street + house number + postal code + city + country

If several fields are selected, as in the postcard example, a duplicate is only detected if all match fields match (AND operator). Upper/lower case and spaces are ignored, i.e. "Hans" and "hans " count as duplicates.

You decide how "strict" your duplicate search should be. An example: you advertise that subscribing to your customer newsletter is rewarded with a personal welcome voucher. Now Mr. and Mrs. Mustermann subscribe separately, but both with the email address mustermann@webhoster.de. To make sure that both receive their personal voucher and are not treated as duplicates, select email address and first name and last name as match fields. Another popular option is to match the fields 'Email' and 'Customer number' (if available).

Please note, however, that this stricter check generally leaves more duplicates in place, e.g. if the same recipient fills in the subscription form with a customer number today but without one tomorrow.

Follow-up action

The duplicates found can be automatically 'Merged' or 'Exported' using a follow-up action. If no follow-up action is selected, the search results page is displayed afterwards and you can start the action you want manually:

'Export' creates a CSV file that you can save on your computer and edit manually.

'Merge' merges the duplicates found under the oldest subscriber ID in each case, so that its valuable activity data (clicks, opens) and interest values are always kept. The activity data and interest values of the newer subscriber IDs are discarded.

You decide yourself whose subscriber fields are taken over using the 'Merge mode':
  • Subscriber with latest change
    The subscriber with the latest modification date is searched for among the duplicates and their field values are taken over.
  • Subscriber with oldest change
    As above, but with priority on the subscriber with the oldest change.

The option Only take over filled fields (recommended) should usually remain activated. This prevents empty fields from being taken over. If, for example, you selected 'Subscriber with latest change', the following happens: if the field of the subscriber with the latest change is empty, the subscriber with the second-latest change is checked, and so on.  This fills as many fields of the merged subscriber as possible with values that are as recent as possible, without emptying fields that are already filled.

After you click 'Start', emma starts the duplicate matching and switches to the 'Search results' tab. Depending on the number of subscribers and match fields, the matching can take a while to complete. When it is done, the number of 'Duplicates' and 'Groups' is shown. If, for example, you match by email address and three subscribers with the email address info@emma.de are found, the result shows three duplicates in the group info@emma.de.

The actions available here are: 'Results', 'Merge', 'Export' and 'Delete'. Note that 'Delete' really removes all(!) subscribers found, not only the duplicate part of the duplicates. With 'Export' you save a CSV file on your computer, for example to edit the duplicates found in Excel. 'Merge' shows you the choice of merge mode described above and merges the duplicates into one record according to your choice. Click 'Results' to get a detailed view of the duplicates found.

Exception: if only two or three duplicates are found per group, you can decide manually for each individual subscriber field whether the values of subscriber 1 or 2 are taken over.

Click the trash can icon to discard the search.

The 'Merging' and 'Export' tabs each show the current status of the actions you selected after the duplicate search.