Versions Compared

Old Version 8

changes.mady.by.user Hans-Peter Eberhard

Saved on

compared with

New Version Current

changes.mady.by.user Alicia Venter

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents
absoluteUrltrue

2.1 Rules 

  • The Match-Connect technical specifications consist of the API documentation an OpenAPI specification and the documents listed below. When developing a Match-Connect implementation, all content of the following documents is relevant and needs to be considered:
    • The Semantics of Match-Connect Messages
    • The Match-Connect Data Dictionary
    The reference code (REF_CODE) must be unique over all types of messages at the side of the sending partner.
  • A Donor _DIFF block is embedded in every TYPevery result message, i.e., TYP_RES, IDM_RES, SMP_RES, RSV_RES, or NO_RES message, as the donor record is changing changes in its source.
    • More generally: every time data on a match list that was provided by donor upload changes, a DIFF upload becomes necessary to update the master data.
  • The term ”donor” " donor" in this document usually means ”adult donor” or ”cord blood unit” refers to "adult donor", "cord blood unit", or "adult donor cryopreserved unit.". 
  • The paying institution paying (INST_PAY) has to be of the institution type (INST_TYPE) FIN.
  • It is recommended to transmit every known field value every time. 
  • Empty/blank/missing fields in the payload mean deletion. 
    • Consequently, empty/blank integer or float values must be provided as null values.
  • The WMDA nomenclature files (http://hla.alleles.org/wmda/index.html) must be used for all HLA fields. The recipient may central hub and/or the receiving registry may reject messages with invalid values. The details are described in the WMDA Guidelines for use of HLA Nomenclature (https://www.nature.com/articles/bmt201393; follow the link ”Supplementary information”" Supplementary information").
  • The Match-Connect semantics for the WMDA-approved additional codes are defined as follows:
  • Image Removed
  • Match-Connect patient IDs should be defined in a way that a change of patient data such as name, birth date or transplant centre does not result in a changed Match-Connect ID. Once a patient ID has been assigned for a given patient, the patient ID cannot be changed.
  • Detailed information about the Global Registration Identifier (GRID) can be found on: https://share.wmda.info/display/OptimisingSearchMatchConnect/Project%3A+GRID.
Message\ValueNEWXXXXNNNN
Patient block (see 6 - Administration v1.0)YDRB3/4/5DRB3/4/5
Donor block (see 6 - Administration v1.0)YDRB3/4/5DRB3/4/5
Typing responseYDRB3/4/5DRB3/4/5
Sample responseYDRB3/4/5DRB3/4/5


  • HLA data changes of a patient HLA changes of a recipient must not cancel requests for the recipientpatient. The requests remain under the full responsibility of the transplant centrethe patient registry.
  • A donor can have only one open request at a time.
    • Multiple requests of the same type for the same patient-donor pair are invalid and must be rejected by the receiving registry.
    • Duplicate requests on the same day: This issue becomes
    particularly difficult if SMP_REQs
    • complicated if Sample Requests are concerned - sometimes users want to
    ”correct”
    • " correct" their previous request (i.e., forgot to request quantity and product).
      • The correct way
    of doing
      • to do this is to cancel the erroneous first request
    first
      • and send the second one later.
      • However, this procedure might also confuse if not carried out on the same working day.
    In
      • If in doubt, a phone call helps
    sorting things out.
      • to sort things out.
  • Multiple responses for the same request (i.e., the same message type) are regarded as updates and should therefore differ from the previously sent one.
  • Denial, Warning, and Remark MSG_DEN, WARNING and REMARK content from the message response messages as well as the TXT_MSG messages Text message content have to be presented to the responsible user (search coordinator) by the local Match-Connect administrator in an appropriate manner.
  • If there are results that cannot be delivered via Match-Connect for whatever reason, the usage of any Match-Connect message (e.g. to give the result in the remark field) is discouraged - fax or email should be used instead.
  • It’s not only useless but forbidden to give a postbox (POB) in a LAB address since this address is used for delivery of samples. For the same reason ZIP code, City and Country are required in the NEW_ADD message.
  • Generally faxes and emails should be avoided in an electronic communication system. If they become necessary, it’s recommended not to mix channels in message groups, for example, if the TYP_REQ was in Match-Connect, the result should be in Match-Connect as well and not by fax/email (and vice versa).
  • Don’t send information multiple times - WARNINGs are to be used if it happens in order to find errors.
  • Multiple *_RES or *_ARR messages are regarded as updates and therefore should be different from the previously sent one.
  • Do not initialize integer and float values with zero but use ”null”.
  • The correct action if a request cannot be accepted is to send a NO_RES followed by a DONOR_CB. The NO_RES.REASON shall be given as ”OT”. 
  • Do not change IDs of patients, donors or addresses: use the primary keys of your database tables prefixed by your hub code as Match-Connect IDs.
  • Only use the specified (see Data Dictionary) character set – a character set translation routine might be necessary.
  • Check the MSG_DEN and WARNING messages you receive from your partners.
  • Return the data as you’ve received it, for example, if you had to change incoming data due to national rules, return the original data with the answer.
  • Resolution of patient HLA typing: provide the highest resolution of HLA-A,B and DR possible. DNA low (XX) resolution on HLA-A,B and DR seriously hampers DNA based matching and mismatching.
  • Announce changes in your national system, for example, sending/accepting more Match-Connect messages.
  • Try to improve your national Match-Connect interface: automated LAB/TC address updates (danger of lost samples for verification typing (CT) and work-up), recognition of duplicates (patients, requests), and usage of the correct reference code (i.e. of the recipient’s side) in result messages (to allow correct assignment of results).
  • Institution addresses registered in the Match-Connect network must be kept up to date. Any updates (and updates only) of Match-Connect institutions must be transmitted at least once a year.
  • Provide SMP_RES or NO_RES for all samples received even if the donor is reported as unavailable in the meantime. Closing a request formally by sending an appropriate result message is important (for result reminders, invoicing) because often the donor later shows up as AV again.
    • If a new hub does not want to implement the whole set of available message types, reasonable groups of message types should be implemented together.Support of typing requests by a donor registry:
  • IN: TYP_REQ, REQ_CAN
    • OUT: TYP_RES, MSG_ACK, NO_RES
    plus associated administrative messages like MSG_DEN, WARNING, NEW_ADD
  • , unless already presented by the registry software.
  • Don't send information multiple times (via Match-Connect and fax/email) - if it happens Message Responses of type "Warning" are to be used to find errors.
  • To establish worldwide uniqueness of the IDs, do not create new IDs for patients or addresses: use the primary keys of your database tables prefixed by your unique registry code (in EMDIS known as hub code) as Match-Connect IDs.
    • Donor IDs (GRID and CB_ID) are defined elsewhere and must already be worldwide unique
  • Return original data in responses even if you had to change incoming data due to national rules.
  • Announce changes in your national system, for example, sending/accepting more Match-Connect messages.
  • Provide Sample Response or Request Rejection for all samples received, even if the donor is reported as unavailable in the meantime. Closing a request formally by sending an appropriate response message is essential (for result reminders and invoicing) because the donor often becomes available again.

2.2 How to read the new semantics if you know EMDIS

As previously indicated in Section 2.1, the OpenAPI specification forms an integral component of our updated documentation set. Henceforth, we will refer to this API documentation as "Swagger." You can access it via the following publicly available URL: https://apispecs.wmda.info/

At this address, both Search & Match API specifications and Connect API specifications can be found:

Image Added

Since the communication is now split to communication with WMDA's Search & Match and messaging with other registries, each API request in this semantics document may be represented in two styles:

  1. Communication with S&M
    Image Added
    Most of the Patient API endpoints and all Search API endpoints consist of such messages.

  2. Messaging with other registries
    Image Added
    One Patient API and all Connect APIs are represented in this style.

    Certain API endpoints include only the patient ID field, while others include the Patient Embedded Block. Similarly for donors: donor ID vs. Donor Embedded Block. The embedded block is used in those cases where the details of the person are important to transmit to the partner registry. For example, when the donor status changes in the work on some request, the Donor Embedded Block is sent to inform of this change.

  3. Required fields
    Required fields are indicated with a red asterisk (*). Fields without (*) are considered optional. 
    Imagefloat
    captionExample required fields

    Image Added

  4. Nested required fields
    Information for any "nested" required fields only need to be provided if another related piece of information is sent. Specifically, when a required field is the child of a parent-field that is not mandatory. This implies that the required child-field only becomes mandatory if the parent-field is sent.
    This design allows flexibility by only making certain fields mandatory in the context of other related information being sent.
    Imagefloat
    captionExample nested required fields

    Image Added



Language note: Original EMDIS language uses Request/Result terminology when speaking about donor requests. In API terminology there are also similar words. A combination was made in creation of API endpoint. For example, when sending an Extended Typing Request as a donor request, the registry must make an API request to an endpoint that is called extendedTypingRequestRequest. To receive that response, the receiving registry must make a call to an API endpoint called extendedTypingRequestRetrieve. Then, for the donor registry to send the message Extended Typing Result, it must send an API request to an endpoint that is called extendedTypingResponseRequest. Respectively, the patient registry retrieves the response (result) from the endpoint extendedTypingResponseRetrieve.

Wherever suitable, we included EMDIS names of the messages in parentheses, e.g. 5.1 - Typing Request (TYP_REQ).

In contrast to the flat FML messages used in the EMDIS communication, the REST API based Match-Connect communication uses nested (multi-level) data structures. Furthermore, the field designations have been improved, e.g. the EMDIS field REQ_DATE is named as requestDate in the API model classes. 

EMDIS rules from the EMDIS Semantics are modified accordingly to the REST API based approach and are added to each message, where applicable.

A table showing recommended sets of API endpoints to be implemented is provided in appendix A. The set of APIs depends on your use of services.

2.3 API authentication

Include Page
API authentication
API authentication


2.4 General technical design

2.4.1 Message Ordering/Sequencing

The general reasons behind message ordering are:

  • preservation of message order
    • controlling the correct order is important to stay in sync between sending and receiving registries
    • some messages only make sense if processed in the correct order
  • making transparent (no) messages are missing
    • in a perfect world, this would not be needed and the current timestamps + reference codes would be sufficient. However e.g. in the case that the receiving registry collects messages from the queue, but loses those messages afterward in a fully automated system, they will not know (in all cases) that messages are missing, and even if they do, they might not know which ones. 
    • proper sequencing will make troubleshooting easier

Message ordering will be realized by:

  • A central sequence number curated by SMC. This sequence number will be drawn from a separate sequence per receiving registry (sequence of integers without leaps).

For the message ordering to work properly, there are different responsibilities of the different parties involved:

  • Sender responsibility:  

The sender is responsible for ensuring the desired order of calls to the Connect APIs.

If API calls are to be scheduled/batched, it is suggested that the sender maintains a queue of outgoing calls/messages so that the message sequence is persistent in the event of system failure.

Make sure the next message is sent only when the status of the previous one has been marked "OK" by SMC.

  • Central system responsibility: 

The central system provides a sequence number which is contained in the response body (wmda response) to each connect message and in the message retrieved by the receiving registry (meta block).

  • Receiver responsibility: 

The sequence number is part of the meta block. The receiver has to check that the sequence number is correct (no numbers missing) and only process the messages if that is ok. Otherwise, go back to the central system and claim the missing message.

Pro:

    • avoid peer-to-peer problem
    • least effort needed on registry side
    • receiver can re-request "missing" message
      • can be automated, no help needed by WMDA staff

2.4.2 Message collection - general flow

Messages can be collected via individual API calls against the different "retrieve" API endpoints, which can be described as a "pull model".

The pull model is described in the following steps:

1. Retrieving an overview of all messages that are ready for retrieval by the receiving registry

  • Using the endpoint:  availableMessages - Retrieve an array with an overview of all messages that are ready for retrieval, including sequence numbers

    This endpoint allows end users to retrieve an overview of all messages ready to be downloaded, including their corresponding sequence numbers. This information can be used to retrieve the messages from their respective *retrieve endpoints. If desired, the messages can be pulled in chronological order, given by the sequence numbers. Each "retrieve" endpoint will have an optional parameter "sequenceNumber", that allows for an individual collection of the corresponding message(s). If that parameter is left empty all available messages are retrieved (according to the limit set in the parameters). 

  • Using the endpoint:  availableMessagesAll - Retrieve an array of all the messages that are ready for retrieval, including message content (OPTIONAL, MAY NOT BE IMPLEMENTED)

    This endpoint allows end users to retrieve an array of all messages ready to be downloaded, including their content. This would allow a registry to call one single endpoint to collect all messages.

    Limitations:
    This option does not work for all applications/tooling, like defining custom API connectors in the 365 PowerPlatform. In those cases the same static structure for all call responses on a specific endpoint is not only implementing best practices (IBP), but also required.

2. Pulling available messages from corresponding "retrieve" endpoints

After identifying which specific message(s) should be retrieved by using the availableMessages endpoint, the messages can be retrieved from the "retrieve" endpoints that correspond to the message type. In the request body, users can specify parameters such as the limit (number of messages to retrieve), "shouldPeek" and sequenceNumber.

Calling to a "retrieve" endpoint results in the retrieval of the requested messages. Users can process and display these messages in their application based on their type and content.

This pull model allows users to have control over the message type they retrieve, specifying limits and retrieving messages from a particular point in the sequence using "sequenceNumber"

3. Messages are (soft)deleted after retrieval

After retrieval messages are soft-deleted. This means that after retrieval, messages are marked as deleted for the end-user and no longer available in the "availableMessages" and their corresponding "Retrieve" endpoint. 

After soft-deletion, messages will be stored for an additional temporary period of 72 hours. During this timeframe, the data is still restorable. 


4. Restoring messages using the "recoverMessages" endpoint

To recover or "restore" a soft-deleted message, users can call to the "recoverMessages" endpoint. This endpoint allows users to bring back messages that were soft-deleted within a 72 hour timeframe.

This approach provides an additional safety measure for end users. In case a message was mistakenly deleted during the retrieval process, or if there is a need to recover soft-deleted data, this mechanism allows for data recovery without permanent loss.

Users should be aware that the functionality to restore messages is limited to 72 hours. After this window, the soft-deleted messages are permanently removed from the system. Also see: Chapter 2.4.7 Message storage retention policies.

2.4.3 "shouldPeek" function

The "shouldPeek" function, which is specified as a parameter for every "Retrieve" endpoint, allows users to preview messages without marking them as retrieved or soft-deleted. It serves as a testing utility to inspect messages without altering their status within the system.

This function is valuable for debugging and ensuring the proper functionality of the message retrieval process.

It is crucial to emphasize that the "shouldPeek" function should only be used for testing purposes to ensure proper functioning of the intended message-flow.

2.4.4 RESTful best practices

In the development of Match Connect, we have adhered to RESTful best practices to ensure optimal performance and maintainability. 

The design-choices for Match Connect prioritize simplicity, scalability and standardization. 

Justification for some of these decisions are further explained below:


  • Using 'POST' instead of 'GET' for message retrieval

    A 'POST' API call for retrieval of messages aligns with HTTP semantics, especially considering the nature of Match Connect. In line with HTTP semantics and RESTful best practices, 'GET' requests should per definition be cacheable. In Match Connect message retrieval triggers a mutation, because messages are no longer available after retrieval. Therefore, the information in Match Connect is non-cacheable. 

    The 'POST' method better reflects the intention of performing a server-side operation and helps avoid unintended caching of mutable operations, enhancing the overall robustness and correctness of Match Connect.

    Additional information about RESTful best practices can be found here.
    Additional information about HTTP semantics can be found here. 


2.4.5 Generic message flow for Connect messages

For all user generated connect messages from a patient registry to a donor registry or vice versa the generic message flow consists of the message sent (e.g. a Sample Request) and a corresponding Message Response (6.2). The detailed message flow for such a message including the generic response message is shown below. The generic response message will be omitted in request message flows in chapter 5, for better readability. 

For further details on the message response itself see chapter 6.2.

The message flow below also shows the direct answer to an API call (for simplicity only 200 and 400 are shown).

The flow includes the routine, asychronous check for pending messages that has to be implemented by each participating registry in the Connect system, see 2.4.2.


PlantUML Macro
dropshadowfalse
separationfalse
formatPNG
alignleft
titleGeneric message flow
participant "Patient Registry" as PR

control "Match-Connect" as MC

control "Search & Match\nService" as SM

participant "Donor Registry" as DR

skinparam SequenceMessageAlign center

skinparam dpi 92

== Sending a user generated message ==
PR -[#dodgerblue]> DR : Send 'Message A'

group details
note left DR: routine asychronous check for pending messages
DR -[#green]> MC : Call /availableMessages to check for not-picked up messages
activate MC #FFBBBB
MC -> DR : 200 OK - Array of messages not yet delivered (**emtpy**)
MC -> DR : 400 Bad Request
deactivate MC

PR -[#green]> MC : Call respective Request-API to send 'Message A'
activate MC #FFBBBB
MC -> PR : 200 OK
MC -> PR : 400 Bad Request
deactivate MC
 
DR -[#green]> MC : Call /availableMessages to check for not-picked up messages
activate MC #FFBBBB
MC -> DR : 200 OK - Array of messages not yet delivered (**'Message A'**)
MC -> DR : 400 Bad Request
deactivate MC
 
DR -[#green]> MC : Call respective Response-API to picked up 'Message A'
activate MC #FFBBBB
MC -> DR : 200 OK
MC -> DR : 400 Bad Request
deactivate MC 
end
== Providing the generic response ==
DR -[#dodgerblue]> PR : Message response (ack, reject, warning to 'Message A') \nshould always include messageId of referring message

group details
PR -[#green]> MC : Call /availableMessages to check for not-picked up messages
activate MC #FFBBBB
MC -> PR : 200 OK - Array of messages not yet delivered (**emtpy**)
MC -> PR : 400 Bad Request
deactivate MC

DR -[#green]> MC : Call /messageResponse API to send the generic response to 'Message A'
activate MC #FFBBBB
MC -> DR : 200 OK
MC -> DR : 400 Bad Request
deactivate MC
 
PR -[#green]> MC : Call /availableMessages to check for not-picked up messages
activate MC #FFBBBB
MC -> PR : 200 OK - Array of messages not yet delivered (**generic response to 'Message A'**)
MC -> PR : 400 Bad Request
deactivate MC
 
PR -[#green]> MC : Call /messageResponseRetrieve API to pick up the generic response to 'Message A'
activate MC #FFBBBB
MC -> PR : 200 OK
MC -> PR : 400 Bad Request
deactivate MC
 
end


2.4.6 Automatically generated and manually curated Swagger

The use of both an automatically generated and manually curated Swagger serves a dual purpose.

The automatically generated Swagger (https://connect-api.wmda.info/swagger/index.html) reflects the implementation of the API endpoints in its current state with accurate details on the sub-schemas. This representation ensures that implementers have access to the most up-to-date information.

The manually curated Swagger (https://apispecs.wmda.info/?urls.primaryName=Connect%20API) plays a crucial role in communication to stakeholders. This representation reflects a broader overview of the relevant endpoints and provides insights into planned future endpoints. This curated version therefore also acts as a guide to provide a clearer understanding of the future development of Match Connect.

Together, these two approaches provide important information about technical accuracy and communication about the development of Match Connect.

2.4.7 Message storage retention policies

Messages sent using the Connect API have the following retention policies applied: 

Message stateRetention time
In MatchConnect system, but not yet retrieved90 days after being received by sending organisation
In MatchConnect system, retrieved72 hours after retrieval. During this time the message can be recovered and retrieved again. 

Messages in the MatchConnect system are stored in encrypted format. Only administrators with access to the message storage and with the proper decryption key are able to read the contents of messages. Additional infrastructure level encryption is also applied. 

The MatchConnect system has redundancies and backups in place that decrease the likelihood of the system going down or messages getting lost. If the system would undergo a fatal crash, the system can be restored from a backup. Message sequence numbers would be restored to the state of the last backup. This may be 1 hour at most.