2 - Technical Guidelines

2.1 Rules 

• The Match-Connect technical specifications consist of an OpenAPI specification and the documents listed below. When developing a Match-Connect implementation, all content of the following documents is relevant and needs to be considered:
  • The Semantics of Match-Connect Messages.
  • The Match-Connect Data Dictionary.
  • Automatically generated and manually curated Swagger (2.4.6).
• A Donor block is embedded in every result message, i.e., TYP_RES, IDM_RES, SMP_RES, RSV_RES, or NO_RES message, as the donor record changes in its source.
  • More generally: every time data on a match list provided by donor upload changes, a DIFF upload becomes necessary to update the master data.
• The term "donor" in this document usually refers to "adult donor", "cord blood unit", or "adult donor cryopreserved unit." 
• The paying institution (INST_PAY) has to be of the institution type (INST_TYPE) FIN.
• Empty/blank/missing fields in the payload mean deletion. 
  • Consequently, empty/blank integer or float values must be provided as null values.
• The WMDA nomenclature files (https://hla.alleles.org/pages/wmda/wmda/) must be used for all HLA fields. The central hub and/or the receiving registry may reject messages with invalid values. The details are described in the WMDA Guidelines for use of HLA Nomenclature (https://www.nature.com/articles/bmt201393; follow the link" Supplementary information").
• The Match-Connect semantics for the WMDA-approved additional codes are defined as follows:

• HLA data changes of a patient must not cancel requests for the patient. The requests remain under the full responsibility of the patient registry.
• A donor can have only one open request at a time.
  • Multiple requests of the same type for the same patient-donor pair are invalid and must be rejected by the receiving registry.
  • Duplicate requests on the same day: This issue becomes complicated if Sample Requests are concerned - sometimes users want to "correct" their previous request (i.e., forgot to request quantity and product).
    • The correct way to do this is to cancel the erroneous first request and send the second one later.
    • However, this procedure might also confuse if not carried out on the same working day. If in doubt, a phone call helps to sort things out.
• Multiple responses for the same request (i.e., the same message type) are regarded as updates and should therefore differ from the previously sent one.
• Denial, Warning, and Remark content from the message response as well as the Text message content have to be presented to the responsible user (search coordinator) by the local Match-Connect administrator in an appropriate manner, unless already presented by the registry software.
• Don't send information multiple times (via Match-Connect and fax/email) - if it happens Message Responses of type "Warning" are to be used to find errors.
• To establish worldwide uniqueness of the IDs, do not create new IDs for patients or institutions: use your organization's unique id prefixed by your unique registry code (in EMDIS known as hub code) as Match-Connect IDs.
  • Donor IDs (GRID and CB_ID) are defined elsewhere and must already be worldwide unique
• Return original data in responses even if you had to change incoming data due to national rules.
• Announce changes in your national system, for example, sending/accepting more Match-Connect messages.
• As a patient registry provide Sample Response or Request Rejection for all samples received, even if the donor is reported as unavailable in the meantime. Closing a request formally by sending an appropriate response message is essential (for result reminders and invoicing) because the donor often becomes available again.

2.2 How to read the new semantics if you know EMDIS

As previously indicated in Section 2.1, the OpenAPI specification forms an integral component of our updated documentation set. Henceforth, we will refer to this API documentation as "Swagger." You can access it via the following publicly available URL: https://apispecs.wmda.info/

At this address, both Search & Match API specifications and Connect API specifications can be found:

Since the communication is now split to communication with WMDA's Search & Match and messaging with other registries, each API request in this semantics document may be represented in two styles:

1. Communication with S&M
   
   Most of the Patient API endpoints and all Search API endpoints consist of such messages.
   
   
2. Messaging with other registries
   
   One Patient API and all Connect APIs are represented in this style.
   
   Certain API endpoints include only the patient ID field, while others include the Patient Embedded Block. Similarly for donors: donor ID vs. Donor Embedded Block. The embedded block is used in those cases where the details of the person are important to transmit to the partner registry. For example, when the donor status changes in the work on some request, the Donor Embedded Block is sent to inform of this change.
   
   
3. Required fields
   Required fields are indicated with a red asterisk (*). Fields without (*) are considered optional. 
   
   

   ---

   Example required fields
4. Nested required fields
   Information for any "nested" required fields only need to be provided if another related piece of information is sent. Specifically, when a required field is the child of a parent-field that is not mandatory. This implies that the required child-field only becomes mandatory if the parent-field is sent.
   This design allows flexibility by only making certain fields mandatory in the context of other related information being sent.
   
   

   ---

   Example nested required fields
   
   
   

Language note: Original EMDIS language uses Request/Result terminology when speaking about donor requests. In API terminology there are also similar words. A combination was made in creation of API endpoint. For example, when sending an Extended Typing Request as a donor request, the registry must make an API request to an endpoint that is called extendedTypingRequestRequest. To receive that response, the receiving registry must make a call to an API endpoint called extendedTypingRequestRetrieve. Then, for the donor registry to send the message Extended Typing Result, it must send an API request to an endpoint that is called extendedTypingResponseRequest. Respectively, the patient registry retrieves the response (result) from the endpoint extendedTypingResponseRetrieve.

Wherever suitable, we included EMDIS names of the messages in parentheses, e.g. 5.1 - Typing Request (TYP_REQ).

In contrast to the flat FML messages used in the EMDIS communication, the REST API based Match-Connect communication uses nested (multi-level) data structures. Furthermore, the field designations have been improved, e.g. the EMDIS field REQ_DATE is named as requestDate in the API model classes. 

EMDIS rules from the EMDIS Semantics are modified accordingly to the REST API based approach and are added to each message, where applicable.

A table showing recommended sets of API endpoints to be implemented is provided in appendix A. The set of APIs depends on your use of services.

2.3 API authentication

curl --location --request POST 'https://login.microsoftonline.com/c3ab1869-1472-4577-b669-0d64c732e75c/oauth2/token' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'grant_type=client_credentials' \
--data-urlencode 'client_id=<YOURCLIENTID>' \
--data-urlencode 'client_secret=<YOURCLIENTSECRET>' \
--data-urlencode 'resource=<RESOURCE_ID_FOR_APPLICATION_YOU_WANT_TO_ACCESS>'

tenant: c3ab1869-1472-4577-b669-0d64c732e75c
resource (sandbox Patient and search API): f6d78a69-c840-46a2-941b-a0e08f9296e7
resource (production Patient and search API): 5df05a6d-8ef0-49fe-9441-62cd9b540376

tenant: c3ab1869-1472-4577-b669-0d64c732e75c
resource (sandbox connect API): fd05059b-9c9e-443c-a7e7-bc4ddfe29f5c
resource (production Connect API): 00f6f281-2c36-45a4-97ae-79d745265090

tenant: c3ab1869-1472-4577-b669-0d64c732e75c
resource (sandbox data manager API): 4ff9e943-41d7-4430-b1cf-5502b9440601
resource (production data manager API): 577a3074-a64c-44e5-b394-bf40eb53413a

{
    "token_type": "Bearer",
    "expires_in": "3599",
    "ext_expires_in": "3599",
    "expires_on": "1642763674",
    "not_before": "1642759774",
    "resource": "1da84816-4337-44d3-a782-b6abebaf6710",
    "access_token": "<Obfuscated for clarity and security>"
}

Bearer <access token> 

2.4 General technical design

2.4.1 Message Ordering/Sequencing

The general reasons behind message ordering are:

• preservation of message order
  • controlling the correct order is important to stay in sync between sending and receiving registries
  • some messages only make sense if processed in the correct order
• making transparent (no) messages are missing
  • in a perfect world, this would not be needed and the current timestamps + reference codes would be sufficient. However e.g. in the case that the receiving registry collects messages from the queue, but loses those messages afterward in a fully automated system, they will not know (in all cases) that messages are missing, and even if they do, they might not know which ones. 
  • proper sequencing will make troubleshooting easier

Message ordering will be realized by:

• A central sequence number curated by SMC. This sequence number will be drawn from a separate sequence per receiving registry (sequence of integers without leaps).

For the message ordering to work properly, there are different responsibilities of the different parties involved:

• Sender responsibility:  

The sender is responsible for ensuring the desired order of calls to the Connect APIs.

If API calls are to be scheduled/batched, it is suggested that the sender maintains a queue of outgoing calls/messages so that the message sequence is persistent in the event of system failure.

Make sure the next message is sent only when the status of the previous one has been marked "OK" by SMC.

• Central system responsibility: 

The central system provides a sequence number which is contained in the response body (general information) to each connect message and in the message retrieved by the receiving registry (meta block).

• Receiver responsibility: 

The sequence number is part of the meta block. The receiver has to check that the sequence number is correct (no numbers missing) and only process the messages if that is ok. Otherwise, go back to the central system and claim the missing message.

Pro:

• • avoid peer-to-peer problem
  • least effort needed on registry side
  • receiver can re-request "missing" message
    • can be automated, no help needed by WMDA staff

2.4.2 Message collection - general flow

Messages can be collected via individual API calls against the different "retrieve" API endpoints, which can be described as a "pull model".

The pull model is described in the following steps:

1. Retrieving an overview of all messages that are ready for retrieval by the receiving registry

• Using the endpoint:  availableMessages - Retrieve an array with an overview of all messages that are ready for retrieval, including sequence numbers
  
  This endpoint allows end users to retrieve an overview of all messages ready to be downloaded, including their corresponding sequence numbers. This information can be used to retrieve the messages from their respective *retrieve endpoints. If desired, the messages can be pulled in chronological order, given by the sequence numbers. Each "retrieve" endpoint will have an optional parameter "sequenceNumber", that allows for an individual collection of the corresponding message(s). If that parameter is left empty all available messages are retrieved (according to the limit set in the parameters). 
  
  
• Using the endpoint:  availableMessagesAll - Retrieve an array of all the messages that are ready for retrieval, including message content (OPTIONAL, MAY NOT BE IMPLEMENTED)
  
  This endpoint allows end users to retrieve an array of all messages ready to be downloaded, including their content. This would allow a registry to call one single endpoint to collect all messages.
  
  Limitations:
  This option does not work for all applications/tooling, like defining custom API connectors in the 365 PowerPlatform. In those cases the same static structure for all call responses on a specific endpoint is not only implementing best practices (IBP), but also required.
  
  

2. Pulling available messages from corresponding "retrieve" endpoints

After identifying which specific message(s) should be retrieved by using the availableMessages endpoint, the messages can be retrieved from the "retrieve" endpoints that correspond to the message type. In the request body, users can specify parameters such as the limit (number of messages to retrieve), "shouldPeek" and sequenceNumber.

Calling to a "retrieve" endpoint results in the retrieval of the requested messages. Users can process and display these messages in their application based on their type and content.

This pull model allows users to have control over the message type they retrieve, specifying limits and retrieving messages from a particular point in the sequence using "sequenceNumber"

3. Messages are (soft)deleted after retrieval

After retrieval messages are soft-deleted. This means that after retrieval, messages are marked as deleted for the end-user and no longer available in the "availableMessages" and their corresponding "Retrieve" endpoint. 

After soft-deletion, messages will be stored for an additional temporary period of 72 hours. During this timeframe, the data is still restorable. 

4. Restoring messages using the "recoverMessages" endpoint

To recover or "restore" a soft-deleted message, users can call to the "recoverMessages" endpoint. This endpoint allows users to bring back messages that were soft-deleted within a 72 hour timeframe.

This approach provides an additional safety measure for end users. In case a message was mistakenly deleted during the retrieval process, or if there is a need to recover soft-deleted data, this mechanism allows for data recovery without permanent loss.

Users should be aware that the functionality to restore messages is limited to 72 hours. After this window, the soft-deleted messages are permanently removed from the system. Also see: Chapter 2.4.7 Message storage retention policies.

2.4.3 "shouldPeek" function

The "shouldPeek" function, which is specified as a parameter for every "Retrieve" endpoint, allows users to preview messages without marking them as retrieved or soft-deleted. It serves as a testing utility to inspect messages without altering their status within the system.

This function is valuable for debugging and ensuring the proper functionality of the message retrieval process.

It is crucial to emphasize that the "shouldPeek" function should only be used for testing purposes to ensure proper functioning of the intended message-flow.

2.4.4 RESTful best practices

In the development of Match Connect, we have adhered to RESTful best practices to ensure optimal performance and maintainability. 

The design-choices for Match Connect prioritize simplicity, scalability and standardization. 

Justification for some of these decisions are further explained below:

• Using 'POST' instead of 'GET' for message retrieval
  
  A 'POST' API call for retrieval of messages aligns with HTTP semantics, especially considering the nature of Match Connect. In line with HTTP semantics and RESTful best practices, 'GET' requests should per definition be cacheable. In Match Connect message retrieval triggers a mutation, because messages are no longer available after retrieval. Therefore, the information in Match Connect is non-cacheable. 
  
  The 'POST' method better reflects the intention of performing a server-side operation and helps avoid unintended caching of mutable operations, enhancing the overall robustness and correctness of Match Connect.
  
  Additional information about RESTful best practices can be found here.
  Additional information about HTTP semantics can be found here. 

2.4.5 Generic message flow for Connect messages

For all user generated connect messages from a patient registry to a donor registry or vice versa the generic message flow consists of the message sent (e.g. a Sample Request) and a corresponding Message Response (6.2). The detailed message flow for such a message including the generic response message is shown below. The generic response message will be omitted in request message flows in chapter 5, for better readability. 

For further details on the message response itself see chapter 6.2.

The message flow below also shows the direct answer to an API call (for simplicity only 200 and 400 are shown).

The flow includes the routine, asychronous check for pending messages that has to be implemented by each participating registry in the Connect system, see 2.4.2.

2.4.6 Automatically generated and manually curated Swagger

The use of both an automatically generated and manually curated Swagger serves a dual purpose.

The automatically generated Swagger (https://connect-api.wmda.info/swagger/index.html) reflects the implementation of the API endpoints in its current state with accurate details on the sub-schemas. This representation ensures that implementers have access to the most up-to-date information.

The manually curated Swagger (https://apispecs.wmda.info/?urls.primaryName=Connect%20API) plays a crucial role in communication to stakeholders. This representation reflects a broader overview of the relevant endpoints and provides insights into planned future endpoints. This curated version therefore also acts as a guide to provide a clearer understanding of the future development of Match Connect.

Together, these two approaches provide important information about technical accuracy and communication about the development of Match Connect.

WMDA is currently releasing changes to both Production (https://connect-api.wmda.info/swagger/index.html) and Sandbox environments (https://sandbox-connect-api.wmda.info/swagger/index.html) of Match-Connect.

2.4.7 Message storage retention policies

Messages sent using the Connect API have the following retention policies applied: 

Messages in the MatchConnect system are stored in encrypted format. Only administrators with access to the message storage and with the proper decryption key are able to read the contents of messages. Additional infrastructure level encryption is also applied. 

The MatchConnect system has redundancies and backups in place that decrease the likelihood of the system going down or messages getting lost. If the system would undergo a fatal crash, the system can be restored from a backup. Message sequence numbers would be restored to the state of the last backup. This may be 1 hour at most.  

2.4.8 Document storage retention policies

Documents (attachments) using the Connect API have the following retention policies applied: 

Document can be uploaded using "uploadAttachment" endpoint by sending organisation. Receiving organisation can download uploaded document using "downloadAttachment" endpoint.

A document is stored in the WMDA blob storage for a maximum of 90 days after upload. If it was not downloaded up after 90 days, it will be automatically deleted and must be uploaded again, if still needed to be sent. 

After the first successful download, the document can be downloaded multiple times within the next 72 hours. Once this period expires, the document is deleted from WMDA storage and must be requested again if needed.