2.0 Message Ordering

The general reasons behind a message ordering are:

• guaranteeing the order of messages 
  • controlling the correct order is important to stay in sync between sending and receiving registry
  • some messages only make sense if processed in the correct order
• making transparent (no) messages are missing
  • in a perfect world this would not be needed and the current timestamps + reference codes would be sufficient. However e.g. in the case that the receiving registry collects messages from the queue, but looses those messages afterwards in a fully automated system, they will not know (in all cases) messages are missing, and even if they do, they might not know which ones. 
  • proper sequencing will make error handling and issue tracing easier

Message ordering will be realized by:

• A central sequence number provided by SMC. This sequence number will be drawn from a separate sequence per receiving registry.

For the message ordering to work properly there are different responsibilities with the different parties invovled:

• Sender responsibility:  

The sender is responsible for ensuring the desired order of calls to the Connect APIs.  If API calls are to be scheduled / batched, it is suggested that the sender maintain a queue of outgoing calls / messages so that message sequence is persistent in the event of system failure. Make sure the next message is sent only when the status of the previous one has been marked OK by SMC.

• Central system responsibility: 

Central system provides the sequence number as part of the meta block as answer in each connect message (messages to another Registry).
Central system provides the sequence number as part of the meta block in the message retrieved by the recieving registry.

• Receiver responsibility: 

The sequence number would be part of the meta block. The receiver has to check that the sequence number is correct (no numbers missing) and only process the messages if that is ok. Otherwise go back to the central system and claim the missing message.

Pro:

• • avoid peer-to peer problem
  • least effort needed on registry side
  • receiver can re-request "missing" message
    • can be automated, no help needed by WMDA staff

2.0.1

How to collect all messages at keep the sequence:

Options:

to allow external registries to access those messages there are several options:

A) 

"the current suggested situation":

• 1) call /availableMessages endpoint and then collect the messages from the endpoints with waiting messages
• 2) or collect messages from all endpoints and accept that some are empty

To keep the ordering (aside from timestamp) the above suggested sequence per receiving registry should be introduced.

The registries has to separate collecting from processing in order to keep the sequence of processing right.

B)

"meta-queue":

• 1) introduce the sequence per receiving registry to /availableMessages endpoint and list all messages per specific queue, not only give the sum
• 2) collect those messages based on the sequence number by going round the specific endpoints

This would allow a registry to collect and process messages in the right order without storing them locally and sort them there

C)

"All-in-One Queue":

• provide a new endpoint that makes use of the "oneOf" tag and provides all messages for external registries
• here the sequence is not needed for ordering anymore but still needed to realize missing messages

This would allow a registry to call just one endpoint to collect all messages.

Limitations:

Option C) doesn't work for all applications/tooling, like defining custom API connectors in the 365 PowerPlatform. In those cases the same static structure for all call responses on a specific endpoint is not only IBP but also required.

As on WMDA side (propably) all messages are stored in one queue/store anyhow (check with Mark) options A-C) will only effect the exposure of the messages to the registries and not the internal workings on WMDA side.

So implementing the options next to each other might give the registries the choice to pick there prefered solution and might still be fine for WMDA?!?

Recommendation:

implement B and C

As a consequence of implementing B) all individual endpoint will have an optional parameter "sequenceNumber" that will function as a filter so that you only get the message with that sequence from the endpoint, without that parameter the endpoints will function as is.

2.1 Rules

• The Match-Connect technical specifications consist of an OpenAPI specification and the documents listed below. When developing a Match-Connect implementation, all content of the following documents is relevant and needs to be considered:
  • The Semantics of Match-Connect Messages
  • The Match-Connect Data Dictionary
• A Donor block is embedded in every result message, i.e., TYP_RES, IDM_RES, SMP_RES, RSV_RES, or NO_RES message, as the donor record changes in its source.
  • More generally: every time data on a match list provided by donor upload changes, a DIFF upload becomes necessary to update the master data.
• The term" donor" in this document usually refers to "adult donor", "cord blood unit", or "adult donor cryopreserved unit." 
• The paying institution (INST_PAY) has to be of the institution type (INST_TYPE) FIN.
• Empty/blank/missing fields in the payload mean deletion. 
  • Consequently, empty/blank integer or float values must be provided as null values.
• The WMDA nomenclature files (http://hla.alleles.org/wmda/index.html) must be used for all HLA fields. The central hub and/or the receiving registry may reject messages with invalid values. The details are described in the WMDA Guidelines for use of HLA Nomenclature (https://www.nature.com/articles/bmt201393; follow the link" Supplementary information").
• The Match-Connect semantics for the WMDA-approved additional codes are defined as follows: 

• HLA data changes of a patient must not cancel requests for the patient. The requests remain under the full responsibility of the patient registry.
• Duplicate requests on the same day: This issue becomes complicated if SMP_REQs are concerned - sometimes users want to" correct" their previous request (i.e., forgot to request quantity and product). The correct way to do this is to cancel the erroneous first request and send the second one later. However, this procedure might also confuse if not carried out on the same working day. In doubt, a phone call helps to sort things out.
• MSG_DEN, WARNING, and REMARK content from the message response messages as well as the TXT_MSG messages, have to be presented to the responsible user (search coordinator) by the local Match-Connect administrator in an appropriate manner.
• Don't send information multiple times - WARNINGs are to be used if it happens in order to find errors.
• A donor can have only one open request at a time. Multiple *_REQ messages for the same patient-donor pair are invalid and must be rejected by the receiving registry. If a request has to be corrected, it has to be canceled (REQ_CAN) and subsequently re-requested ( *_REQ)
• Multiple *_RES or *_ARR messages for the same request (i.e., the same message ID) are regarded as updates and should therefore differ from the previously sent one.
• Do not create new IDs of patients, donors, or addresses: use the primary keys of your database tables prefixed by your unique registry code (in EMDIS, known as hub code) as Match-Connect IDs.
• Return original data in responses even if you had to change incoming data due to national rules.
• Announce changes in your national system, for example, sending/accepting more Match-Connect messages.
• Provide SMP_RES or NO_RES for all samples received, even if the donor is reported as unavailable in the meantime. Closing a request formally by sending an appropriate result message is essential (for result reminders and invoicing) because the donor often shows up as AV again.

2.2 How to read the new semantics if you know EMDIS

As previously indicated in Section 2.1, the OpenAPI specification forms an integral component of our updated documentation set. Henceforth, we will refer to this API documentation as "Swagger." You can access it via the following publicly available URL: https://apispecs.wmda.info/

At this address one can find both Search & Match API specifications and Connect API specifications:

Since the communication is now split to communication with WMDA's Search & Match and messaging with other registries, each API request in this semantics document may be represented in two styles:

1. Communication with S&M
   
   Most of the Patient API endpoints and all Search API endpoints consist of such messages.
   
   
2. Messaging with other registries
   
   One Patient API and all Connect APIs are represented in this style.
   
   

Wherever suitable, we included EMDIS names of the messages in parentheses, e.g. 5.1 - Extended Typing Request (TYP_REQ).

In contrast to the flat FML messages used in the EMDIS communication, the REST API based Match-Connect communication uses nested (multi-level) data structures. Furthermore, the field designations have been improved, e.g. the EMDIS field REQ_DATE is named as requestDate in the API model classes. 

EMDIS rules from the EMDIS Semantics are modified accordingly to the REST API based approach and are added to each message, where applicable.

The table below shows recommended sets of API endpoints to be implemented depending on your use of services:

Service in use API endpoint \	Comment	Recommended for Patient & Search	Recommended for Patient & Search, download Registry	Mandatory for Connect	Mandatory for Connect, Download Registry	Comment
SEARCH & MATCH
PATIENTS
​/patients [POST]	Create a patient	yes	no	yes (PR)	no
​/patients [PUT]	Update an existing patient	no	no	yes (PR)	no
​/patients [GET]	Retrieved patients list	no	no	no	no	Search and Match Service Graphical User Interface (GUI) content, optional for participating organisations
​/patients​/{wmdaId} [GET]	Retrieve	no	no	no	no	Search and Match Service GUI content, optional for participating organisations
/patients/status [PUT]		yes	no	no	no
​/patients​/user [PUT]		no	no	no	no	Search and Match Service GUI content, optional for participating organisations
SEARCH
/searches [POST]		yes	no	yes (PR)	no	for recommendations see semantics
/searches/patientSearches/{wmdaId} [GET]		yes	no	yes (PR)	no
/searches/refreshAllPatientSearches [POST]		no	no	no	no	Search and Match Service GUI content, optional for participating organisations
/searches/{searchId} [GET]		no	no	no	no	highly recommended
/searches/refreshAllPatientSearches [POST]		no	no	no	no
/searches/searchResults/donors [POST]		yes	no	yes (PR)	no	at least one of those is mandatory,
/searches/searchResults/cbus [POST]
/searches/{searchId}/searchResults/adcu [GET]
​/searches​/{searchId}​/searchResults​/registries [GET]		no	no	no	no	Search and Match Service GUI content, optional for participating organisations
/searches/{searchId}/searchResults/cbbs [GET]		no	no	no	no	Search and Match Service GUI content, optional for participating organisations
/searches/selected/donors [POST]		no	no	no	no	Search and Match Service GUI content, optional for participating organisations
​/searches​/selected​/cbus [POST]		no	no	no	no	Search and Match Service GUI content, optional for participating organisations
/searches/donors/{searchResultsId} [GET]		no	no	no	no	Search and Match Service GUI content, optional for participating organisations
​/searches​/cbus​/{searchResultsId} [GET]		no	no	no	no	Search and Match Service GUI content, optional for participating organisations
DASHBOARD
for all		no	no	no	no	Search and Match Service GUI content, optional for participating organisations
CONNECT API
ATTACHMENT
​/api​/v1​/attachmentTicket [POST]				no	no	Optional for participating organisations
/api/v1/attachmentDownloadURL [POST]				no	no	Optional for participating organisations
​/api​/v1​/attachmentDownloadedNotification [POST]				no	no	Optional for participating organisations
RETRIEVE
​/api​/v1​/availableMessages [POST]				yes	yes	check with Mark
ADMIN
/api/v1/alertRetrieve [GET]				no	no	recommended
/api/v1/alertUpdateRetrieve [GET]				no	no	recommended
PATIENT
/api/v1/updateRegisteredPatient [POST]				yes (PR)	yes (PR)
​/api​/v1​/updateRegisteredPatientRetrieve [GET]				yes (DR)	yes (DR)
PING
​/api​/v1​/pingRequest [POST]				no	no	Optional for participating organisations
/api/v1/pingRetrieve [POST]				no	no	Optional for participating organisations
TEXT MESSAGE
/api/v1/textMessageRequest [POST]				no	no	Optional for participating organisations
​/api​/v1​/TextMessageRetrieve [GET]				no	no	Optional for participating organisations
REQUEST
/api/v1/genericRequestRequest [POST]				no	no
/api/v1/genericRequestRetrieve [GET]				no	no
/api/v1/extendedTypingRequestRequest [POST]				yes (PR)	yes (PR)
/api/v1/extendedTypingRequestRetrieve [POST]				no	no	recommended for DCs with incompletely typed donors
/api/v1/extendedTypingResponseRequest [POST]				no	no	recommended for DCs with incompletely typed donors
/api/v1/extendedTypingResponseRetrieve [POST]				yes (PR)	yes (PR)
​/api​/v1​/sampleRequestRequest [POST]				yes (PR)	yes (PR)
​/api​/v1​/sampleRequestRetrieve [POST]				yes (DR, CBB)	yes (DR, CBB)
​/api​/v1​/sampleInfoRequest [POST]				yes (DR, CBB)	yes (DR, CBB)
​/api​/v1​/sampleInfoRetrieve [POST]				yes (PR)	yes (PR)
​/api​/v1​/sampleArrivalRequest [POST]				yes (DR, CBB)	yes (DR, CBB)
​/api​/v1​/sampleArrivalRetrieve [POST]				yes (PR)	yes (PR)
/api/v1/sampleResponseRequest [POST]				yes (PR)	yes (PR)
/api/v1/sampleResponseRetrieve [POST]				yes (DR, CBB)	yes (DR, CBB)
/api/v1/infectiousDiseaseMarkerRequest [POST]				yes (PR)	yes (PR)
/api/v1/infectiousDiseaseMarkerRequestRetrieve [POST]				yes (DR)	yes (DR)
​/api​/v1​/infectiousDiseaseMarkerResultRequest [POST]				yes (DR)	yes (DR)
/api/v1/infectiousDiseaseMarkerResultRetrieve [POST]				yes (PR)	yes (PR)
​/api​/v1​/reservationRequestRequest [POST]				yes (PR)	yes (PR)	under discussion in user group
​/api​/v1​/reservationRequestRetrieve [POST]				yes (DR)	yes (DR)
​/api​/v1​/reservationResponseRequest [POST]				yes (DR)	yes (DR)
/api/v1/reservationResponseRetrieve [POST]				yes (PR)	yes (PR)
​/api​/v1​/reservationReleaseRequest [POST]				yes (PR)	yes (PR)
​/api​/v1​/reservationReleaseRetrieve [POST]				yes (DR)	yes (DR)
/api/v1/requestCancellationRequest [POST]				yes (PR)	yes (PR)
/api/v1/requestCancellationRetrieve [POST]				yes (DR)	yes (DR)
​/api​/v1​/requestRejectedRequest [POST]				yes (DR)	yes (DR)
/api/v1/requestRejectedRetrieve [POST]				yes (PR)	yes (PR)
​/api​/v1​/resultReminderRequest [POST]				no	no	recommended
/api/v1/resultReminderRetrieve [POST]				no	no	recommended
GENERAL
/api/v1/messageResponse [POST]				yes	yes
​/api​/v1​/messageResponseRetrieve [POST]				yes	yes
/api/v1/messageAcknowledgementRequest [POST]						all replaced by the messageResponse
/api/v1/messageAcknowledgementRetrieve [POST]
​/api​/v1​/warningRequest [POST]
/api/v1/warningRetrieve [POST]
​/api​/v1​/messageDenialRequest [POST]
/api/v1/messageDenialRetrieve [POST]
CBU
/api/v1/cordBloodUnitReportRequestRequest	on hold			no	no	should be generalised to CBU and ADCUs, etc...
/api/v1/cordBloodUnitReportRequestRetrieve	on hold			yes (CBB)	yes (CBB)
/api/v1/cordBloodUnitReportResponseRequest	on hold			no	no
​/api​/v1​/cordBloodUnitReportResponseRetrieve	on hold			yes (CBB)	yes (CBB)

2.3 API authentication