1. Contact

Contact for technical questions and support: subscription.center.dev@toyota-europe.com

Version: 1.1.0-RC70

Built on: 2018-10-19T17:02:37+0200

Tip
Impatient? Then start browsing and using the API using the built-in HAL Browser here

2. Subscription Centre

2.1. Glossary

Noun Meaning

SC

The Subscription Centre that is provided by TME.

Subscription Data

A group name for any customer-related data expressing the customer’s consents and preferred channels. It allows for future extensions like newsletter preferences and preferred dealership. The SC basically deals with subscription data, and syncing it between source systems.

Context

Refers to which logical consumer/customer-base or -scope that the change of subscription data is restricted. Examples are Toyota, Lexus, TFS or TIM.

NMSC

A unique name for an NMSC organisation, for example: TGB, TMI, etc. An organisation can have multiple Source Systems and data for multiple contexts.

Source System

A system that is a source of subscription data. A source system is a logical concept, its name is unique within a single organisation. If the same physical source system is used for multiple contexts, for example Toyota and Lexus, then there will be 2 logical source systems and both must be integrated with SC.

Warning
When registering a Source System in SC, its name must match exactly with the one used in Consumer Domain.

Destination

A resource that needs to be registered by the Source System, and that basically contains the actual webhook URI and the version of the data (definition-wise) that SC needs to post to this URI. The Destination will accept messages containing new validated Subscription Data (see Validation).

Webhook

A URI that is exposed by the Source System. It is called by the SC whenever new validated subscription data has arrived for that particular Source System or a new cluster containing the customer’s data appears in Consumer Domain.

Note
In order to guarantee confidentiality of the exchanged subscription data, this URI needs to support TLS

Source Customer Id

The unique key of a customer (or related concept like test drive or car config request) in a particular Source System.

Validation

The process of informing the customer of consent changes and allowing him to confirm the changes. In case the customer wishes to reject the changes, no action is required.

Propagation

The process of notifying a source system of validated Subscription Data. When subscription data is changed by a source system, the SC will figure out which other systems contain data for that particular real life customer, and need to be notified about the change. The initial request to change Subscription Data is validated, and, if that is confirmed by the customer, subsequently propagated to the concerned Source Systems.

Cluster or Customer Cluster

In order to successfully mediate subscription data changes, the SC makes use of clustered customer data. A cluster will contain all source system’s customer records that are regarded as the same physical real life person at a certain moment in time. A cluster is a potentially volatile piece of data, it is provided by the Consumer Domain.

Published Language

The concepts and their definitions that are exposed by the API to its consumers (the source systems). Changes of the API are explicitly managed and agreed upon before they take place. Versioning is explicit.

Canonical consent category

Consent categories that have been defined at the European level.

Refer to the appendix for all possible values.

An NMSC can choose to hide one or more of these consent categories.

Additional consent category

Next to the canonical consent categories, an NMSC can apply for an additional consent category in case local legislation requires it.

Example: Italy has the additional consent category Profiling.

DPO

Data Protection Officer, this role needs to be assigned to at least 1 person within an NMSC. The NMSC itself has the possibility to appoint a different DPO per brand.

Apart from the NMSC level, the DPO can also be assigned at TME level.

TARS

Toyota Authentication / Registration Service.

2.2. Environments

SC is available in multiple environments, each with its own purpose:

Subscription Centre

Refresh rate

Tamper-proof Log

Consumer Domain

Salesforce Marketing Cloud

Landing Page (Toyota)

Landing Page (Lexus)

DEV

twice daily

Apache Distributed Log [1]

DEV

Sandbox

DEV

PREV

TEST

on demand

Apache Distributed Log [1]

DEV

Sandbox

DEV

PREV

UAT

on demand

Apache Distributed Log [1]

UAT

Sandbox

UAT

PREV

PRD

-

-

-

-

-

-

2.3. Business goals

Most of the business goals of the Subscription Centre (SC) are rooted in the GDPR regulation that will be imposed and verified by the EU from May 2018 onwards. GDPR is essentially a law to protect the privacy of users in such a way that the data they shared with Toyota, cannot be used against them, or used for purposes that they don’t agree with. Many of the requirements are about meeting GDPR with regards to the Data Subject's rights:

  • Right to receive transparent information

  • Right of access

  • Right to rectification

  • Right to erasure

  • Right to data portability

  • Right to object

  • Right to refuse to automated individual decision-making, including profiling

2.4. How SC supports these goals

The SC offers

  • a service for tracking and linking decentralized consent- and preference-data (Subscription Data) for/to the same physical person

  • a service to update and consume new Subscription Data

  • a mechanism to share Subscription Data changes with all impacted Source Systems

  • a way to automatically notify and validate Subscription Data changes to/with the customer

  • a single source of truth allowing traceability and auditability: we need to be able to explain and justify our decisions

Be aware that the SC in itself:

  • does not cover (completely) all of the business goals listed in previous paragraph.

  • is not guaranteeing GDPR-compliancy, but it is rather facilitating it; simply connecting your Source System to SC will not make it GDPR-compliant

  • is initially focused on direct marketing related consent categories and preferences; however, albeit the initial focus or genesis, the SC approach/architecture can be used for other non-marketing-related use cases

Apart from the Data Subject (the customer, the consumer), there are requirements of some other actors that need to be met: the GDPR Auditor and the Court Of Law whenever Toyota gets sued for alleged violation of the privacy law. SC focuses on having for each (potential) customer the latest consent to use his/her data for a number of distinct use cases (so called consent-categories like surveys, newsletters, etc).

Apart from pure consent, a number of other data are gathered by the SC:

  • communication preferences: currently only preferred communication channel(s))

  • the actual text (disclaimer, consent description, etc) that the customer consented to

  • (a reference to) the historical consumer data (more precisely, the cluster) that was used for propagation and decision taking in general

People can have accounts in multiple Toyota source systems, and, as they make use of these systems, give or reject their consent and preferences for various topics. Given that these so called source systems are integrated with the SC, when, for example, a user gives/rejects consent for a local consent category, then the source system will communicate this consent change to the SC. The source system does not update its local data yet, but awaits approval by the SC.

The SC will log this request in a tamper-proof log (also called ledger), and will, if indicated so by the Source System, validate with the customer whether (s)he actually initiated this change or not, and if the person still agrees with the change. When confirmed by the customer, the SC will propagate (push) this change to the initiating source system ├ánd all other source systems where that same person has an account (or is known in some way). All targeted systems should update their local data (consent, preference for customer X), and explicitly confirm this by sending a confirmation-message to the SC using the SC’s API.

In essence this means that the SC’s primary task is to synchronize and validate Subscription Data changes across Toyota-systems. Please note that, in all cases, this activity is scoped per NMSC: synchronization between systems belonging to one and the same NMSC, about customers that - directly or indirectly - belong to the same NMSC.

If the change is about consent, the source system needs to communicate in canonical terms, i.e. use one of the consent categories that were agreed upon on a European level. This means that, whatever local consent-category the source system was using, it needs to map it to the canonical category before communicating it to SC. Vice versa, when SC pushes a change to them, they need to map it back to their own local/proprietary consent category.

To do the validation with the customer of a recent Subscription Data change, the SC will use Salesforce Marketing Cloud (SFMC) to send a validation email to the Data Subject, asking for his/her confirmation. If the confirmation is not received within a reasonable amount of time, the consent change request is marked as invalid and the consent/preference/etc will not change. In this case, nothing is communicated towards any source system. On the contrary, when the consent change is confirmed, the source systems (incl. the initiating system) will be notified and must change their local data accordingly. So the SC will make sure that, once they are integrated, the source systems receive the most recent consent/preferences at all times. This syncing takes place among all source systems of the same NMSC, but not beyond.

2.6. How SC knows in which source systems a customer is known

Of course, a user X in system S1, and a user Y in system S2 can only be identified as one and the same physical person by the SC, if there is some system that links X and Y as being the same person. This system is called the Consumer Domain (CD), and it provides a deduplication service which results in so-called clusters (grouped personal data). Each cluster contains a number of records (each record referring to a customer/consumer/any record containing personal data, in a specific system), with every record typically originating in another source system. All records in the cluster are considered to actually refer to the same physical person in the real world. But this cluster can be volatile as its correctness highly depends on the quality of the data, and the accuracy of the grouping/clustering ruleset. A cluster is the result of some AI (provided by TAMR) grouping the records together based on a number of criteria and TAMR training. This means that some clusters can change overnight due to the arrival of new data in the CD, and when that happens, the source systems (referred in the affected clusters) need to be notified of the change and its impact on the consolidated Subscription Data of the affected local customer/consumer-accounts. The SC will make use of the same push mechanism mentioned above to propagate changes that originate from cluster changes.

2.7. Traceability and auditability

From the perspective of the auditor, the SC also needs to meet some requirements. SC will keep an immutable track record of all the Subscription Data of all customers, in time. By introducing the element of immutability and strict ordering in the ledgers (the logs), and by keeping the whole history and not only the latest state, Toyota can prove that the provided data is actually genuine, and give a complete picture of why decisions were made, by whom and at what point in time. In general we should start from the principle that the more types of consecutive events (request to change consent, validation over SFMC, etc) are being recorded in this log, the easier it is to assemble a plausible body of proof in a court of law. The above auditing-related elements constitute what we call superior auditability of the system.

3. Cookbook for integrating a Source System

3.1. Introduction

This cookbook serves as a guide for the NMSC on how to integrate one or more Source Systems with the SC.

3.2. Before you start

Identify Source Systems that contain Subscription Data that is valuable for the NMSC and/or Retailer network. The NMSC must make sure that all Subscription Data has been gathered in a GDPR-compliant way before uploading the data into SC. If needed, initiate a renewal campaign with your customers to collect Subscription Data in a GDPR-compliant way.

Each Source System probably has stored consent belonging to one or more local consent categories. As SC only supports canonical consent categories, a mapping needs to be defined between local and canonical consent categories. Please note that the actual mapping from local to canonical category (and vice versa) needs to be performed by the Source System that sends (resp receives) data from the SC.

Next step is to verify whether all Source Systems containing Subscription Data are integrated with Consumer Domain since the outcome of the deduplication process (clustering) is essential for SC to be able to propagate validated Subscription Data to the appropriate Source Systems where a person has personal data stored.

For each Source System, identify whether it contains customer records and Subscription Data for a specific Context (typically Toyota or Lexus) and Retailer. Some possible scenarios:

  • multiple Contexts: the Source System in question needs to register a Destination-resource within SC for each Context separately.

  • single Retailer, the Source System needs to register a Destination-resource at the Retailer level, this is not yet supported.

  • multiple Retailers, the Source System needs to register a Destination-resource at the NMSC level.

Every Source System needs to expose a so-called webhook (refer to How to create a Destination) in order to accept propagated consent. The NMSC needs to contact the Source System vendor in order to plan this activity.

Note
Some Source Systems like for example Lead Management (PAN-E web forms) are sources of customer preferences but don’t manage them. This means such a Source System will not have a destination registered with SC. Therefore, in this scenario, while propagating back the consent, SC will not be able to call this Source System’s Webhook to inform it of the validated consent. Please contact SC application team if you need to enable this.

3.3. Integrating with SC API

For every individual Source System, some configuration is required both from the source system and the SC application team. The source system needs to provide the following information to the SC application team:

  • The context of the source system

  • The NMSC of the source system

  • The id/name of the source system

Once those informations are provided, the SC application team will register a source system on the API and generate username and password in order to access the API.

Once the source system is registered correctly by the SC application team and the credentials are issued, the source system can configure the destination/webhook. Refer to Create a Destination for more information. This configuration is done by the source system and can be updated at any time by the source system.

After successful registration, the Source System can start sending subscription data to SC and receive subscription data from SC.

The diagram below showcases the interaction of the various components involved in processing consent. The most important phases are:

  1. Validation: obviously, this phase is only required in case SC has received unvalidated consent. The validation consists of sending out an email to the customer, asking him to confirm the consent. The customer is directed to a landing page which can be branded depending on the context. On this landing page, the customer has basically only one option: confirm the consent. When the customer doesn’t want to confirm, he doesn’t need to do anything. The confirmed consent is then captured by SC.

  2. Propagation: only executed in case the consent is validated. SC will then fetch the Consumer Domain cluster for the given customer. This cluster contains one or more Source Systems where this same customer is known. SC will then propagate the validated consent to all Source Systems of that cluster.

cookbook what happens with consent in sc

3.5. Uploading your initial consent

SC offers the possibility for a Source System to upload all of its consent at once. This procedure is meant to be run to bootstrap the system. After this so-called initial upload, regular consent changes have to be delivered as described in sending subscription data.

SC has decided to use a file-based upload with JSON format. The file must be uploaded in an AWS S3 bucket, contact SC application team to get the necessary S3 credentials. The JSON file contains a list of consents that need to be uploaded. The content of the individual consent is similar to the payload of sending subscription data attached with an envelope including context, nmsc, source system and sourceCustomerId.

General file specifications:

  • file in JSON format

  • one JSON object = one consent record

  • multiple consent records can be present in the same file with each consent record separated by a comma

  • only validated and GDPR compliant consent can be uploaded

File format:

The file format rules of these JSON key value pairs will be the same as the ones mentioned here: sending subscription data (path parameters)

JSON key

Meaning

context

Context

nmsc

NMSC code

sourceSystemName

Source System name

sourceCustomerId

The ID of the customer

The rules of the value object of this JSON key data will be the same as the ones mentioned here: sending subscription data (request fields)

data

The value of this key will hold the actual consent data

Sample file:

[
  {
    "context": "toyota",
    "nmsc": "tgb",
    "sourceSystemName": "crm",
    "sourceCustomerId": "cust-123",
    "data": {
      "commandType": "REQUESTED",
      "commandTimestamp": "2018-04-23T15:10:44.000+0100",
      "consent": {
        "nmsc": "tgb",
        "gdprCompliant" : true,
        "validated" : true,
        "notify" : false,
        "communicationAttributes": {
          "language": "en",
          "country": "GB",
          "email": "test@toyota.com"
        },
        "consentAttributes": [
          {
            "consentCode": "REMINDERS",
            "consentFlag": true,
            "consentDescription": "I give consent to receiving REMINDERS",
            "consentLongDescription": "",
            "validationReference": "valref-123",
            "originSourceSystem": "",
            "modifiedByOrganization": "",
            "modifiedByUser": "",
            "requestedTimestamp": "2017-03-20T15:10:44.000+0100",
            "validatedTimestamp": "2017-03-20T12:10:44.000+0100"
          },
          {
            "consentCode": "OFFERS",
            "consentFlag": true,
            "consentDescription": "I give consent to receiving OFFERS",
            "consentLongDescription": "",
            "validationReference": "valref-123",
            "originSourceSystem": "",
            "modifiedByOrganization": "",
            "modifiedByUser": "",
            "requestedTimestamp": "2017-03-20T15:10:44.000+0100",
            "validatedTimestamp": "2017-03-21T12:10:44.000+0100"
          }
        ]
      },
      "channel": {
        "nmsc": "tgb",
        "channelAttributes" : [
          {
            "channelCode": "SMS",
            "channelFlag": true,
            "requestedTimestamp": "2017-03-20T15:10:44.000+0100",
            "originSourceSystem": "",
            "modifiedByOrganization": "",
            "modifiedByUser": ""
          },
          {
            "channelCode": "EMAIL",
            "channelFlag": true,
            "requestedTimestamp": "2017-03-20T15:10:44.000+0100",
            "originSourceSystem": "",
            "modifiedByOrganization": "",
            "modifiedByUser": ""
          }
        ]
      }
    }
  },
  {
    "context": "lexus",
    "nmsc": "tmi",
    "sourceSystemName": "toshiko",
    "sourceCustomerId": "cust-456",
    "data": {
      "commandType": "REQUESTED",
      "commandTimestamp": "2018-04-23T15:10:44.000+0100",
      "consent": {
        "gdprCompliant" : true,
        "validated" : true,
        "notify" : false,
        "communicationAttributes": {
          "language": "it",
          "country": "IT",
          "email": "test1@toyota.com"
        },
        "consentAttributes": [
          {
            "consentCode": "REMINDERS",
            "consentFlag": true,
            "consentDescription": "I give consent to receiving REMINDERS",
            "consentLongDescription": "",
            "validationReference": "valref-123",
            "originSourceSystem": "",
            "modifiedByOrganization": "",
            "modifiedByUser": "",
            "requestedTimestamp": "2017-03-20T15:10:44.000+0100",
            "validatedTimestamp": "2017-03-20T12:10:44.000+0100"
          },
          {
            "consentCode": "SURVEYS",
            "consentFlag": false,
            "consentDescription": "I do not give consent to receiving SURVEYS",
            "consentLongDescription": "",
            "validationReference": "valref-123",
            "originSourceSystem": "",
            "modifiedByOrganization": "",
            "modifiedByUser": "",
            "requestedTimestamp": "2017-03-20T15:10:44.000+0100",
            "validatedTimestamp": "2017-03-21T12:10:44.000+0100"
          }
        ]
      },
      "channel": {
        "channelAttributes" : [
          {
            "channelCode": "MAIL",
            "channelFlag": true,
            "requestedTimestamp": "2017-03-20T15:10:44.000+0100",
            "originSourceSystem": "",
            "modifiedByOrganization": "",
            "modifiedByUser": ""
          },
          {
            "channelCode": "PHONE",
            "channelFlag": true,
            "requestedTimestamp": "2017-03-20T15:10:44.000+0100",
            "originSourceSystem": "",
            "modifiedByOrganization": "",
            "modifiedByUser": ""
          }
        ]
      }
    }
  }
]

3.6. Bulk export of consents collected via PAN-E web forms

SC provides a solution to bulk extract validated consents collected via PAN-E web forms as long as no other Local Systems are integrated with SC.

  • This is only available at NMSC and TME levels.

  • Only a person acting as DPO for a specific NMSC, will be allowed to request a bulk export of validated consent within its NMSC.

  • Only a person acting as DPO for TME, will be allowed to request a bulk export of validated consent for all NMSC’s for the context his role is defined for.

  • Credentials can be provided to every person acting as the DPO in the NMSC. It is advised to have at least 2 persons appointed with the DPO role in order to assure business continuity.

  • The result of bulk export will be put in a downloadable file and returned via browser.

File format: the downloadable file will be in JSON format. It is identical to the format of the consolidated consent.

3.6.1. How will the DPO access the bulk export endpoint?

The person acting as a DPO should have a TARS account assigned to the appropriate Organisation (NMSC/TME). He then needs to file a Service Now request to attach the appropriate DPO role to his account with the following details:

Application Group (TARS)

Application

Role

Organisation

Notes/Description

Sales/Marketing

Subscription Centre

SC NMSC DPO TOYOTA

NMSC

Role to be assigned to the DPO at the NMSC specific for Toyota

SC NMSC DPO LEXUS

NMSC

Role to be assigned to the DPO at the NMSC specific for Lexus

SC TME DPO TOYOTA

TME

Role to be assigned to the DPO at TME specific for Toyota.

This DPO user will see the data over all NMSCs

SC TME DPO LEXUS

TME

Role to be assigned to the DPO at TME specific for Lexus.

This DPO user will see the data over all NMSCs

An NMSC DPO user will only have access to the validated consent within his/her organisation.

3.6.2. Example bulk export request

DPO user browses to the bulk export login page https://subscription-centre.toyota-europe.com/bulk-export/login.

He logs in with his TARS credentials assigned to him for his DPO role to access bulk export in SC. Once successfully logged in, the DPO user will see a landing page. The landing page will have a list of all bulk export URLs for the DPO’s NMSC configured in SC within a specific context (for which his role is defined).

Note
For a TME DPO user, the landing page will list bulk export URLs for all NMSCs configured in SC within the context for which the DPO’s role is defined.

To initiate bulk export, the DPO user clicks the appropriate link and the bulk export json file is generated and offered for download.

Warning
As this downloadable file contains sensitive personal data, it needs to be treated according the GDPR rules.

3.6.3. Example response file

[
  {
    "context": "toyota",
    "nmsc": "tgb",
    "sourceSystemName": "crm",
    "sourceCustomerId": "cust-123",
    "data": {
      "commandType": "REQUESTED",
      "commandTimestamp": "2018-04-23T15:10:44.000+0100",
      "consent": {
        "nmsc": "tgb",
        "gdprCompliant" : true,
        "validated" : true,
        "notify" : false,
        "communicationAttributes": {
          "language": "en",
          "country": "GB",
          "email": "test@toyota.com"
        },
        "consentAttributes": [
          {
            "consentCode": "REMINDERS",
            "consentFlag": true,
            "consentDescription": "I give consent to receiving REMINDERS",
            "consentLongDescription": "",
            "validationReference": "valref-123",
            "originSourceSystem": "",
            "modifiedByOrganization": "",
            "modifiedByUser": "",
            "requestedTimestamp": "2017-03-20T15:10:44.000+0100",
            "validatedTimestamp": "2017-03-20T12:10:44.000+0100"
          },
          {
            "consentCode": "OFFERS",
            "consentFlag": true,
            "consentDescription": "I give consent to receiving OFFERS",
            "consentLongDescription": "",
            "validationReference": "valref-123",
            "originSourceSystem": "",
            "modifiedByOrganization": "",
            "modifiedByUser": "",
            "requestedTimestamp": "2017-03-20T15:10:44.000+0100",
            "validatedTimestamp": "2017-03-21T12:10:44.000+0100"
          }
        ]
      },
      "channel": {
        "nmsc": "tgb",
        "channelAttributes" : [
          {
            "channelCode": "SMS",
            "channelFlag": true,
            "requestedTimestamp": "2017-03-20T15:10:44.000+0100",
            "originSourceSystem": "",
            "modifiedByOrganization": "",
            "modifiedByUser": ""
          },
          {
            "channelCode": "EMAIL",
            "channelFlag": true,
            "requestedTimestamp": "2017-03-20T15:10:44.000+0100",
            "originSourceSystem": "",
            "modifiedByOrganization": "",
            "modifiedByUser": ""
          }
        ]
      }
    }
  },
  {
    "context": "lexus",
    "nmsc": "tmi",
    "sourceSystemName": "toshiko",
    "sourceCustomerId": "cust-456",
    "data": {
      "commandType": "REQUESTED",
      "commandTimestamp": "2018-04-23T15:10:44.000+0100",
      "consent": {
        "gdprCompliant" : true,
        "validated" : true,
        "notify" : false,
        "communicationAttributes": {
          "language": "it",
          "country": "IT",
          "email": "test1@toyota.com"
        },
        "consentAttributes": [
          {
            "consentCode": "REMINDERS",
            "consentFlag": true,
            "consentDescription": "I give consent to receiving REMINDERS",
            "consentLongDescription": "",
            "validationReference": "valref-123",
            "originSourceSystem": "",
            "modifiedByOrganization": "",
            "modifiedByUser": "",
            "requestedTimestamp": "2017-03-20T15:10:44.000+0100",
            "validatedTimestamp": "2017-03-20T12:10:44.000+0100"
          },
          {
            "consentCode": "SURVEYS",
            "consentFlag": false,
            "consentDescription": "I do not give consent to receiving SURVEYS",
            "consentLongDescription": "",
            "validationReference": "valref-123",
            "originSourceSystem": "",
            "modifiedByOrganization": "",
            "modifiedByUser": "",
            "requestedTimestamp": "2017-03-20T15:10:44.000+0100",
            "validatedTimestamp": "2017-03-21T12:10:44.000+0100"
          }
        ]
      },
      "channel": {
        "channelAttributes" : [
          {
            "channelCode": "MAIL",
            "channelFlag": true,
            "requestedTimestamp": "2017-03-20T15:10:44.000+0100",
            "originSourceSystem": "",
            "modifiedByOrganization": "",
            "modifiedByUser": ""
          },
          {
            "channelCode": "PHONE",
            "channelFlag": true,
            "requestedTimestamp": "2017-03-20T15:10:44.000+0100",
            "originSourceSystem": "",
            "modifiedByOrganization": "",
            "modifiedByUser": ""
          }
        ]
      }
    }
  }
]

4. Frequently Asked Questions

This section provides answers on typical questions an NMSC could have when integrating with SC. In case your question is not listed below, please contact the SC application team.

  1. How can I protect the webhook URI on Source System side?

    During registration of a Source System in SC, the webhook URI is specified. SC will generate a unique and hard-to-guess API key and make it available to the Source System. Whenever SC needs to progagate Subscription Data changes to the Source System, this API key will be included in the HTTP headers. The webhook URI on Source System's side needs to verify whether the correct API key is present in the HTTP headers or not. The approach explained in above mechanism, and the fact that we are relying on very common industry standards, is intended to lower as much as possible the barrier for a Source System to integrate with the SC:

    • be the least possible invasive for the Source System

    • be implementable by almost any system, no matter the technology that it is based on (Java, .Net, something else)

  2. How does SC notify other Source Systems?

    Whenever a consent/preference is changed and sent to SC, SC will notify all the Source Systems which know about the same person. This information is extracted from the cluster, which is a product of the Consumer Domain clustering process. Each Source System will be called on its registered webhook URI (Destination resource).

  3. How does SC handle the deletion of a customer?

    This is not foreseen in the current scope.

  4. How much time is given to the customer to validate his Subscription Data change?

    This is not yet defined, although a 24-hour windows seems advisable. A reminder will be sent out in case the customer did not react on the first message.

  5. What happens when the customer doesn’t react at all on the validation message?

    If the time for the validation process runs out, SC will send a reminder to the customer. If the customer still doesn’t validate, then nothing will happen. This means the consent must not be taken into account by any Source System. This is the reason why a Source System must not take into account a consent change as long as it isn’t validated.

  6. Does SC accept partial preferences from a Source System?

    Yes, SC can accept partial preferences/consents from a Source System. All Subscription Data that is part of a single request, will be validated at the same time.

  7. How does SC communicate any updated preference information to the end customer? Does it send just a subset of the preferences or all preferences?

    After the validation/notification procedure, SC will only send the subset that is changed to the Source Systems - potentially only one consent or preference.

  8. When SC propagates validated Subscription Data to other Source Systems, does it include the originating Source System?

    Yes, it does.

  9. What happens if an individual rejects preferences sent in email by SC?

    There will be no reject button in the validation email. There will only be one button labelled confirm. If the customer wants to reject the preferences, he simply ignores the email.

  10. What will be the approach if an individual wants to approve partial preferences?

    This is not possible. The email contains all preferences and has a single confirm button.

  11. Does SC have a mechanism to send preferences to the Source System again if there is failure/error while communicating from SC to Source System?

    Yes, SC has foreseen failures during communication with the Source System. SC will keep sending the message until the Source System has received it. If for example, the Source System is down and comes back again after a while, it will receive all propagated consents from SC as soon as possible.

  12. If an individual’s preference is to receive information via Telephone, how does SC communicate with the customer?

    Not for phase 1. Validation is only supported by email.

  13. Does SC register which user on Source System side initiated the consent?

    This is supported by the modifiedByOrganization and modifiedByUser attributes, refer to this section for more info.

  14. Does SC manage multiple email addresses for the same customer?

    No, SC does not manage personal data, only Subscription Data. It is the Source System's responsibility to include the correct email address when requesting a consent- or preference-change which needs to be validated.

  15. Does SC act as a Source System for Consumer Domain?

    No, SC only captures the customer’s consent. It doesn’t feed personal data into CD. SC only asks for the minimal set of personal data that it needs to support its own functionality, for example email address for validation.

5. General API considerations

This document describes the HTTP/REST-based API of the Subscription Centre Service. This means that:

  • the API is kept as comprehensible and stable as possible

  • the API is versioned

  • older versions of the API will be maintained and supported for some time, but will be phased out following a predictable schedule that will be agreed and communicated up-front to the consumers of the API

We primarily make use of the technique of webhooks to inform the source system about subscription data updates that it is interested in. We think this is the most simple, interoperable technique to implement notification-style callbacks. However, for those source systems that - for some reason - cannot or don’t want to expose an HTTPS-based endpoint at their side, we think of exposing an API that the source system can poll. Which technique (eg ATOM feeds, ODATA, etc) will be used for this polling-based API, has not been decided yet.

5.1. Versioning

  • SC makes use of standard Semantic Versioning, discriminating between MAJOR, MINOR and PATCH versions

    • a PATCH is used for bugfixes; usually all API consumers always immediately need to upgrade to the patch-version once it gets released

    • when the MINOR version is bumped, some functionality has been added, maybe some attributes have been added to the payload of the JSON-objects, but we expect that the API consumer does not have to update his code; lenient parsing of message payloads should save the day here

    • when the MAJOR version is bumped, this means that there is a breaking change for the consumer of the API; i.e. the consumer needs to update his code, and cannot rely on lenient parsing etc to let their system accommodate automatically the change

  • API version needs to be provided as query parameter version on the source system’s HTTP requests; if no version is specified, we will direct the source system’s HTTP request to the latest version of the API

  • when registering a webhook destination, the source system needs to specify which version of the message (data structure definition) is to be POST-ed to this webhook URI

5.2. HAL & HATEOAS

SC adheres as much as possible to the general HAL standard, whenever we deal with REST resources expressing hyperlinks between each other. HAL is a simple format that gives a consistent and easy way to hyperlink between resources in an API. HAL has a media type for both the JSON and XML variants, but currently we only use the JSON-variant application/hal+json. More precisely, we make use of a popular constraint of the REST architecture, called HATEOAS. HATEOAS satisfies the HAL format and goes even further. This should yield an even better, self-explanatory, unambiguous and predictable API.

5.3. HTTP verbs

Verb Usage

GET

Used to retrieve a resource (idempotent)

POST

Used to create a new resource

PATCH

Used to update a resource

PUT

Used to update an existing resource (idempotent)

DELETE

Used to delete an existing resource

5.4. HTTP Status Codes

SC tries to adhere as closely as possible to standard HTTP and REST conventions in its use of HTTP status codes.

Status code Usage

200 OK

Used to retrieve a resource (idempotent)

201 Created

A new resource has been created successfully, the resource’s URI is available from the response’s Location header

204 No content

An update to an existing resource has been applied successfully

400 Bad Request

The request was malformed, the response body will include an error providing further information

404 Not Found

The requested resource did not exist

409 Conflict

The requested resource already exists (for non-idempotent operations like POST)

5.5. Authentication and Authorization

Authentication and authorization is based on OAuth2.

Every API call between a Source System and the SC application needs to be accompanied by an access token. This token is issued by the SC application itself after authenticating with the appropriate username and password credentials. Once issued, the same token can be reused as long it is not expired. The validity of the access token will start from the time it is issued.

Steps to follow:

Request a token

Issue an HTTP POST request to /oauth/token containing

Header/Body

Key

Value

Remarks

Header

Authorization

Basic dG1lOnNlY3JldA==

This is the base64 encoded value of client-id and client-secret

Body

grant_type

password

Fixed value

Body

username

<your username here>

Body

password

<your password here>

A successful response looks like

{
  "access_token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzb3VyY2Vfc3lzdGVtIjpudWxsLCJ1c2VyX25hbWUiOiJwYXRyaWNrLTIwMTgtMDMtMjMiLCJzY29wZSI6WyJyZWFkIiwid3JpdGUiXSwiZXhwIjoxNTIxODUyNjk3LCJubXNjIjoicGgtbm1zYy0yMDE4LTAzLTIzIiwiYXV0aG9yaXRpZXMiOlsiTk1TQ19ST0xFIl0sImp0aSI6Ijg0NzZhZTcxLTE3ZmYtNDJmNS1iZTU2LWU4YjJlMzRkNDBmNiIsImNsaWVudF9pZCI6InRtZSJ9.UecGgy6yHh2Aq8AHEJoY9EKHbJ1oG0OGySyTR3GAy1U",
  "token_type": "bearer",
  "expires_in": 43199,
  "scope": "read write",
  "source_system": null,
  "nmsc": "ph-nmsc-2018-03-23",
  "jti": "8476ae71-17ff-42f5-be56-e8b2e34d40f6"
}

Use the token

For every API request, include your token in the HTTP Authorization header with the value

Bearer <your token here>

An attempt to connect to SC with an expired token will result to an authorization error. An unsuccessful response looks like

HTTP/1.1 401 Unauthorized

{
  "error": "invalid_token",
  "error_description": "Access token expired: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzb3VyY2Vfc3lzdGVtIjpudWxsLCJ1c2VyX25hbWUiOiJsb2NhbC1hZG1pbiIsInNjb3BlIjpbInJlYWQiLCJ3cml0ZSJdLCJleHAiOjE1MjI5NTQ4NDksIm5tc2MiOm51bGwsImF1dGhvcml0aWVzIjpbIkFETUlOX1JPTEUiXSwianRpIjoiNTVmMGRhMGQtM2Y0My00MjljLWIxZTAtMDI4YmNhNzA5ZWEyIiwiY2xpZW50X2lkIjoidG1lIn0.Ptl9ukr4hCGqWAeCZO7lxf4XqYItRmIBF93zvDFA39k"
}

6. Exploring the API

The technical API documentation can be found at here

7. Receive propagation

Example payloads during the propagation from Subscription Center:

{
  "commandType" : "PROPAGATED",
  "commandTimestamp" : "2017-01-01T02:01:00.000Z",
  "sourceCustomerId" : "12345",
  "consent" : {
    "gdprCompliant" : true,
    "consentAttributes" : [ {
      "consentCode" : "REMINDERS",
      "consentFlag" : true,
      "consentDescription" : "TEST",
      "consentLongDescription" : "TEST",
      "requestedTimestamp" : "2017-01-01T02:01:00.000Z",
      "validatedTimestamp" : "2017-01-01T02:01:00.000Z",
      "validationReference" : "TEST",
      "originSourceSystem" : null,
      "modifiedByOrganization" : null,
      "modifiedByUser" : null
    } ],
    "validated" : true,
    "notify" : false,
    "communicationAttributes" : null
  },
  "channel" : {
    "channelAttributes" : [ {
      "channelCode" : "EMAIL",
      "channelFlag" : true,
      "requestedTimestamp" : "2017-01-01T02:01:00.000Z",
      "originSourceSystem" : null,
      "modifiedByOrganization" : null,
      "modifiedByUser" : null
    } ]
  }
}
{
  "commandType" : "PROPAGATED",
  "commandTimestamp" : null,
  "sourceCustomerId" : "12345",
  "consent" : null,
  "channel" : {
    "channelAttributes" : [ {
      "channelCode" : "EMAIL",
      "channelFlag" : true,
      "requestedTimestamp" : "2017-01-01T02:01:00.000Z",
      "originSourceSystem" : null,
      "modifiedByOrganization" : null,
      "modifiedByUser" : null
    } ]
  }
}
{
  "commandType" : "PROPAGATED",
  "commandTimestamp" : null,
  "sourceCustomerId" : "12345",
  "consent" : {
    "gdprCompliant" : true,
    "consentAttributes" : [ {
      "consentCode" : "REMINDERS",
      "consentFlag" : true,
      "consentDescription" : "TEST",
      "consentLongDescription" : "TEST",
      "requestedTimestamp" : "2017-01-01T02:01:00.000Z",
      "validatedTimestamp" : "2017-01-01T02:01:00.000Z",
      "validationReference" : "TEST",
      "originSourceSystem" : null,
      "modifiedByOrganization" : null,
      "modifiedByUser" : null
    } ],
    "validated" : true,
    "notify" : false,
    "communicationAttributes" : null
  },
  "channel" : null
}
{
  "commandType" : "PROPAGATED",
  "commandTimestamp" : null,
  "sourceCustomerId" : "12345",
  "consent" : {
    "gdprCompliant" : true,
    "consentAttributes" : [ {
      "consentCode" : "REMINDERS",
      "consentFlag" : true,
      "consentDescription" : "TEST",
      "consentLongDescription" : "TEST",
      "requestedTimestamp" : "2017-01-01T02:01:00.000Z",
      "validatedTimestamp" : "2017-01-01T02:01:00.000Z",
      "validationReference" : "TEST",
      "originSourceSystem" : null,
      "modifiedByOrganization" : null,
      "modifiedByUser" : null
    }, {
      "consentCode" : "OFFERS",
      "consentFlag" : false,
      "consentDescription" : "TEST",
      "consentLongDescription" : "TEST",
      "requestedTimestamp" : "2017-01-01T02:01:00.000Z",
      "validatedTimestamp" : "2017-01-01T02:01:00.000Z",
      "validationReference" : "TEST",
      "originSourceSystem" : null,
      "modifiedByOrganization" : null,
      "modifiedByUser" : null
    }, {
      "consentCode" : "SURVEYS",
      "consentFlag" : false,
      "consentDescription" : "TEST",
      "consentLongDescription" : "TEST",
      "requestedTimestamp" : "2017-01-01T02:01:00.000Z",
      "validatedTimestamp" : "2017-01-01T02:01:00.000Z",
      "validationReference" : "TEST",
      "originSourceSystem" : null,
      "modifiedByOrganization" : null,
      "modifiedByUser" : null
    } ],
    "validated" : true,
    "notify" : false,
    "communicationAttributes" : null
  },
  "channel" : {
    "channelAttributes" : [ {
      "channelCode" : "EMAIL",
      "channelFlag" : true,
      "requestedTimestamp" : "2017-01-01T02:01:00.000Z",
      "originSourceSystem" : null,
      "modifiedByOrganization" : null,
      "modifiedByUser" : null
    }, {
      "channelCode" : "PHONE",
      "channelFlag" : false,
      "requestedTimestamp" : "2017-01-01T02:01:00.000Z",
      "originSourceSystem" : null,
      "modifiedByOrganization" : null,
      "modifiedByUser" : null
    }, {
      "channelCode" : "SMS",
      "channelFlag" : true,
      "requestedTimestamp" : "2017-01-01T02:01:00.000Z",
      "originSourceSystem" : null,
      "modifiedByOrganization" : null,
      "modifiedByUser" : null
    } ]
  }
}

8. Appendix

Following Consent Category codes are defined:

Consent Category Code Description

REMINDERS

Pan-European Canonical Consent Category

OFFERS

Pan-European Canonical Consent Category

SURVEYS

Pan-European Canonical Consent Category

EVENTS

Pan-European Canonical Consent Category

EVENTS_SURVEYS

Combined Consent Category

OFFERS_SURVEYS

Combined Consent Category

OFFERS_EVENTS

Combined Consent Category

REMINDERS_SURVEYS

Combined Consent Category

REMINDERS_EVENTS

Combined Consent Category

REMINDERS_OFFERS

Combined Consent Category

OFFERS_EVENTS_SURVEYS

Combined Consent Category

REMINDERS_EVENTS_SURVEYS

Combined Consent Category

REMINDERS_OFFERS_SURVEYS

Combined Consent Category

REMINDERS_OFFERS_EVENTS

Combined Consent Category

REMINDERS_OFFERS_EVENTS_SURVEYS

Combined Consent Category

The 4 Pan-European Canonical Consent Categories are defined: REMINDERS, OFFERS, SURVEYS and EVENTS.

The API will also allow to combine the Canonical Consent Categories into a single Combined Consent Category.

Note
The Combined Consent Categories are reserved for exceptional use and should only be considered after alignment with TME.

Implementation Rules:

  • A maximum of 4 consent categories can be selected

  • In case of Combined Consent Categories, overlap of Consent Categories is not allowed

Examples:

  • SURVEYS, EVENTS and REMINDERS

  • REMINDERS and EVENTS_SURVEYS

  • REMINDERS_EVENTS_SURVEYS

8.2. Command Types

Following Command Type codes are defined:

Command Type Description

REQUESTED

Use this code when a Source System sends subscription data to SC.

To be used for new consent.

PROPAGATED

When the Source System receives subscription data from SC, the value will be set to PROPAGATED.

PROCESSED

Use this code when a Source System sends subscription data to SC.

To indicate successful processing.

Usage: sending subscription data, initial upload (only the value REQUESTED is allowed)

9. Release notes

9.2. Version 0.18.0 (June 25, 2018)

9.4. Version 0.16.0 (May 28, 2018)

  • Validation and Notification procedure using Salesforce Marketing Cloud

  • Procedure for bulk export

9.6. Version 0.14.0 (April 30, 2018)

  • Updated data structure of sending subscription data sending subscription data

  • Updated documentation related to the new data structure

9.7. Version 0.13.0 (April 06, 2018)

9.8. Version 0.12.0 (March 30, 2018)

9.9. Version 0.11.0 (March 16, 2018)

  • Add paragraph cookbook describing the initial upload

9.10. Version 0.10.0 (March 02, 2018)


1. Refer to http://bookkeeper.apache.org/distributedlog/ for more information