⚠Due to planned maintenance you will experience short (<30 min) downtime between 08:00 - 08:30 CET.
2.0 Message Ordering
The general reasons behind the need for message ordering are:
- guaranteeing the order of messages
- controlling the correct order is important to stay in sync between sending and receiving registry
- some messages only make sense if processed in the correct order
- making transparent (no) messages are missing
- in a perfect world this would not be needed and the current timestamps + reference codes would be sufficient. However e.g. in the case that the receiving registry collects messages from the queue, but looses those messages afterwards in a fully automated system, they will not know (in all cases) messages are missing, and even if they do, they might not know which ones.
- proper sequencing will make error handling and issue tracing easier
There are several possibilities to realize this:
Option A):
intrinsic ordering only:
Sender Responsibility: The sender is responsible for ensuring the desired order of calls to the Connect APIs. If api calls are to be scheduled / batched, it is suggested that the sender maintain a queue of outgoing calls / messages so that message sequence is persistent in the event of system failure.
Receiver Responsibility: The receiver is responsible for monitoring the availableMessages queue and process retrieval calls to the Connect APIs in the order specified.
Pro:
no additional information needed (ordering based on timestamps and incoming order SMC service)
Con:
only implicit orderingno explicit indicator on missing messages on receiver side
Option B):
ordering via local sequence number:
Sender Responsibility: The sender has to add a receiver specific sequence to each and every message.
Receiver Responsibility: The sequence number would be part of each message itself . The receiver has to check that the sequence number is correct (no number missing) and only process the messages if that is ok. Otherwise go back to the central system and claim the missing message.
Pro:
most detailed information as the number is part of the whole communication proess
Con:
alot of overhead in the communicationextra work at each registryeach registry has to maintain a sequence number for each new registry (peer to peer behaviour)extra work for each registry when a new registry is added to the system
error prone as independent implementations are needed and no control by the centralized system
Option C):
central sequence number provided by SMC:
option C1: one sequence per receiving registry
option C2: one number per sending / receiving registry combination
Sender Responsibility:
The sender is responsible for ensuring the desired order of calls to the Connect APIs. If api calls are to be scheduled / batched, it is suggested that the sender maintain a queue of outgoing calls / messages so that message sequence is persistent in the event of system failure. Make sure the next message is sent only when the status of the previous one has been marked OK by SMC.
Central system:
Central system provides the sequence number as part of the meta block as answer in each connect message (messages to another Registry).
Option: not add it to the message_response (and others)?
Receiver Responsibility:
The sequence number would be part of the meta block. The receiver has to check that the sequence number is correct (no numbers missing) and only process the messages if that is ok. Otherwise go back to the central system and claim the missing message.
Pro:
- avoid peer-to peer problem
- least effort needed on registry side
- receiver can re-request "missing" message
- can be automated, no help needed by WMDA staff
- allows for different levels of details (C1 vs. C2)
Con:
- additional effort on WMDA side
- new endpoint for re-request specific message
- handling of the sequences
- storing registry to registry messages centralized for a time to make re-request possible.
2.0.1
How to collect all messages at keep the sequence:
Options:
to allow external registries to access those messages there are several options:
A)
"the current suggested situation":
- 1) call /availableMessages endpoint and then collect the messages from the endpoints with waiting messages
- 2) or collect messages from all endpoints and accept that some are empty
To keep the ordering (aside from timestamp) the above suggested sequence per receiving registry should be introduced.
The registries has to separate collecting from processing in order to keep the sequence of processing right.
B)
"meta-queue":
- 1) introduce the sequence per receiving registry to /availableMessages endpoint and list all messages per specific queue, not only give the sum
- 2) collect those messages based on the sequence number by going round the specific endpoints
This would allow a registry to collect and process messages in the right order without storing them locally and sort them there
C)
"All-in-One Queue":
- provide a new endpoint that makes use of the "oneOf" tag and provides all messages for external registries
- here the sequence is not needed for ordering anymore but still needed to realize missing messages
This would allow a registry to call just one endpoint to collect all messages.
Limitations:
Option C) doesn't work for all applications/tooling, like defining custom API connectors in the 365 PowerPlatform. In those cases the same static structure for all call responses on a specific endpoint is not only IBP but also required.
As on WMDA side (propably) all messages are stored in one queue/store anyhow (check with Mark) options A-C) will only effect the exposure of the messages to the registries and not the internal workings on WMDA side.
So implementing the options next to each other might give the registries the choice to pick there prefered solution and might still be fine for WMDA?!?
Recommendation:
implement B and C
As a consequence of implementing B) all individual endpoint will have an optional parameter "sequenceNumber" that will function as a filter so that you only get the message with that sequence from the endpoint, without that parameter the endpoints will function as is.
2.1 Rules
- The Match-Connect technical specifications consist of the API documentation and the documents listed below. When developing a Match-Connect implementation, all content of the following documents is relevant and needs to be considered:
- The Semantics of Match-Connect Messages
- Match-Connect Data Dictionary
- A Donor_DIFF block is embedded in every result message, i.e., TYP_RES, IDM_RES, SMP_RES, RSV_RES, or NO_RES message, as the donor record changes in its source.
- More generally: every time data on a match list provided by donor upload changes, a DIFF upload becomes necessary to update the master data.
- The term" donor" in this document usually means "adult donor", "cord blood unit", or "adult donor cryopreserved unit."
- The institution paying (INST_PAY) has to be of the institution type (INST_TYPE) FIN.
- Empty/blank/missing fields in the payload mean deletion.
- Consequently, empty/blank integer or float values must be provided as null values.
- The WMDA nomenclature files (http://hla.alleles.org/wmda/index.html) must be used for all HLA fields. The central hub and/or the receiving registry may reject messages with invalid values. The details are described in the WMDA Guidelines for use of HLA Nomenclature (https://www.nature.com/articles/bmt201393; follow the link" Supplementary information").
- The Match-Connect semantics for the WMDA-approved additional codes are defined as follows:
Message | NEW | XXXX | NNNN |
---|---|---|---|
PAT_UPD | Y | DRB3/4/5 | DRB3/4/5 |
Donor_DIFF | Y | DRB3/4/5 | DRB3/4/5 |
TYP_RES | Y | DRB3/4/5 | DRB3/4/5 |
SMP_RES | Y | DRB3/4/5 | DRB3/4/5 |
- HLA changes of a patient must not cancel requests for the patient. The requests remain under the full responsibility of the patient registry.
- Duplicate requests on the same day: This issue becomes complicated if SMP_REQs are concerned - sometimes users want to" correct" their previous request (i.e., forgot to request quantity and product). The correct way to do this is to cancel the erroneous first request and send the second one later. However, this procedure might also confuse if not carried out on the same working day. In doubt, a phone call helps to sort things out.
- MSG_DEN, WARNING, and REMARK content from the message response messages as well as the TXT_MSG messages, have to be presented to the responsible user (search coordinator) by the local Match-Connect administrator in an appropriate manner.
- Don't send information multiple times - WARNINGs are to be used if it happens in order to find errors.
- A donor can have only one open request at a time. Multiple *_REQ messages for the same patient-donor pair are invalid and must be rejected by the receiving registry. If a request has to be corrected, it has to be canceled (REQ_CAN) and subsequently re-requested ( *_REQ)
- Multiple *_RES or *_ARR messages for the same request (i.e., the same message ID) are regarded as updates and should therefore differ from the previously sent one.
- Do not change the IDs of patients, donors, or addresses: use the primary keys of your database tables prefixed by your unique registry code (in EMDIS, known as hub code) as Match-Connect IDs.
- Return the data as you've received it. For example, return the original data with the answer if you had to change incoming data due to national rules.
- Announce changes in your national system, for example, sending/accepting more Match-Connect messages.
- Provide SMP_RES or NO_RES for all samples received, even if the donor is reported as unavailable in the meantime. Closing a request formally by sending an appropriate result message is essential (for result reminders and invoicing) because the donor often shows up as AV again.
2.2 How to read the new semantics if you know EMDIS
As one can see from the first sentence above, the API documentation is part of the new documentation set. This API documentation is called "swagger" for shortness. It is located at this publicly available address: https://apispecs.wmda.info/
At this address one can find both Search & Match API specifications and Connect API specifications:
Since the communication is now split to communication with WMDA's Search & Match and messaging with other registries, each API request in the Semantics may be represented in two styles:
- Communication with S&M
- Most of the Patient API endpoints and all Search API endpoints consist of such messages.
- Messaging with other registries
- One Patient API and all Connect APIs are represented in this style.
Wherever suitable, we included EMDIS names of the messages in parentheses, e.g. 5.1 - Extended Typing Request (TYP_REQ).
In many cases, the current implementation of EMDIS messages will require a conversion of the flat EMDIS message structure into the nested (multi-level) structure of the API messages. All the fields in API have better names: e.g., REQ_DATE becomes requestDate.
EMDIS rules from the EMDIS Semantics are modified accordingly to the new reality and are added to each message, where applicable.
The table below shows recommended sets of API endpoints to be implemented depending on your use of services:
Service in use API endpoint \ | Comment | Recommended for Patient & Search | Recommended for Patient & Search, download Registry | Mandatory for Connect | Mandatory for Connect, Download Registry | Comment |
---|---|---|---|---|---|---|
SEARCH & MATCH | ||||||
PATIENTS | ||||||
/patients [POST] | Create a patient | yes | no | yes (PR) | no | |
/patients [PUT] | Update an existing patient | no | no | yes (PR) | no | |
/patients [GET] | Retrieved patients list | no | no | no | no | Search and Match Service Graphical User Interface (GUI) content, optional for participating organisations |
/patients/{wmdaId} [GET] | Retrieve | no | no | no | no | Search and Match Service GUI content, optional for participating organisations |
/patients/status [PUT] | yes | no | no | no | ||
/patients/user [PUT] | no | no | no | no | Search and Match Service GUI content, optional for participating organisations | |
SEARCH | ||||||
/searches [POST] | yes | no | yes (PR) | no | for recommendations see semantics | |
/searches/patientSearches/{wmdaId} [GET] | yes | no | yes (PR) | no | ||
/searches/refreshAllPatientSearches [POST] | no | no | no | no | Search and Match Service GUI content, optional for participating organisations | |
/searches/{searchId} [GET] | no | no | no | no | highly recommended | |
/searches/refreshAllPatientSearches [POST] | no | no | no | no | ||
/searches/searchResults/donors [POST] | yes | no | yes (PR) | no | at least one of those is mandatory, | |
/searches/searchResults/cbus [POST] | ||||||
/searches/{searchId}/searchResults/adcu [GET] | ||||||
/searches/{searchId}/searchResults/registries [GET] | no | no | no | no | Search and Match Service GUI content, optional for participating organisations | |
/searches/{searchId}/searchResults/cbbs [GET] | no | no | no | no | Search and Match Service GUI content, optional for participating organisations | |
/searches/selected/donors [POST] | no | no | no | no | Search and Match Service GUI content, optional for participating organisations | |
/searches/selected/cbus [POST] | no | no | no | no | Search and Match Service GUI content, optional for participating organisations | |
/searches/donors/{searchResultsId} [GET] | no | no | no | no | Search and Match Service GUI content, optional for participating organisations | |
/searches/cbus/{searchResultsId} [GET] | no | no | no | no | Search and Match Service GUI content, optional for participating organisations | |
DASHBOARD | ||||||
for all | no | no | no | no | Search and Match Service GUI content, optional for participating organisations | |
CONNECT API | ||||||
ATTACHMENT | ||||||
/api/v1/attachmentTicket [POST] | no | no | Optional for participating organisations | |||
/api/v1/attachmentDownloadURL [POST] | no | no | Optional for participating organisations | |||
/api/v1/attachmentDownloadedNotification [POST] | no | no | Optional for participating organisations | |||
RETRIEVE | ||||||
/api/v1/availableMessages [POST] | yes | yes | ||||
ADMIN | ||||||
/api/v1/alertRetrieve [GET] | no | no | recommended | |||
/api/v1/alertUpdateRetrieve [GET] | no | no | recommended | |||
PATIENT | ||||||
/api/v1/updateRegisteredPatient [POST] | yes (PR) | yes (PR) | ||||
/api/v1/updateRegisteredPatientRetrieve [GET] | yes (DR) | yes (DR) | ||||
PING | ||||||
/api/v1/pingRequest [POST] | no | no | Optional for participating organisations | |||
/api/v1/pingRetrieve [POST] | no | no | Optional for participating organisations | |||
TEXT MESSAGE | ||||||
/api/v1/textMessageRequest [POST] | no | no | Optional for participating organisations | |||
/api/v1/TextMessageRetrieve [GET] | no | no | Optional for participating organisations | |||
REQUEST | ||||||
/api/v1/genericRequestRequest [POST] | no | no | ||||
/api/v1/genericRequestRetrieve [GET] | no | no | ||||
/api/v1/extendedTypingRequestRequest [POST] | yes (PR) | yes (PR) | ||||
/api/v1/extendedTypingRequestRetrieve [POST] | no | no | recommended for DCs with incompletely typed donors | |||
/api/v1/extendedTypingResponseRequest [POST] | no | no | recommended for DCs with incompletely typed donors | |||
/api/v1/extendedTypingResponseRetrieve [POST] | yes (PR) | yes (PR) | ||||
/api/v1/sampleRequestRequest [POST] | yes (PR) | yes (PR) | ||||
/api/v1/sampleRequestRetrieve [POST] | yes (DR, CBB) | yes (DR, CBB) | ||||
/api/v1/sampleInfoRequest [POST] | yes (DR, CBB) | yes (DR, CBB) | ||||
/api/v1/sampleInfoRetrieve [POST] | yes (PR) | yes (PR) | ||||
/api/v1/sampleArrivalRequest [POST] | yes (DR, CBB) | yes (DR, CBB) | ||||
/api/v1/sampleArrivalRetrieve [POST] | yes (PR) | yes (PR) | ||||
/api/v1/sampleResponseRequest [POST] | yes (PR) | yes (PR) | ||||
/api/v1/sampleResponseRetrieve [POST] | yes (DR, CBB) | yes (DR, CBB) | ||||
/api/v1/infectiousDiseaseMarkerRequest [POST] | yes (PR) | yes (PR) | ||||
/api/v1/infectiousDiseaseMarkerRequestRetrieve [POST] | yes (DR) | yes (DR) | ||||
/api/v1/infectiousDiseaseMarkerResultRequest [POST] | yes (DR) | yes (DR) | ||||
/api/v1/infectiousDiseaseMarkerResultRetrieve [POST] | yes (PR) | yes (PR) | ||||
/api/v1/reservationRequestRequest [POST] | yes (PR) | yes (PR) | ||||
/api/v1/reservationRequestRetrieve [POST] | yes (DR) | yes (DR) | ||||
/api/v1/reservationResponseRequest [POST] | yes (DR) | yes (DR) | ||||
/api/v1/reservationResponseRetrieve [POST] | yes (PR) | yes (PR) | ||||
/api/v1/reservationReleaseRequest [POST] | yes (PR) | yes (PR) | ||||
/api/v1/reservationReleaseRetrieve [POST] | yes (DR) | yes (DR) | ||||
/api/v1/requestCancellationRequest [POST] | yes (PR) | yes (PR) | ||||
/api/v1/requestCancellationRetrieve [POST] | yes (DR) | yes (DR) | ||||
/api/v1/requestRejectedRequest [POST] | yes (DR) | yes (DR) | ||||
/api/v1/requestRejectedRetrieve [POST] | yes (PR) | yes (PR) | ||||
/api/v1/resultReminderRequest [POST] | no | no | recommended | |||
/api/v1/resultReminderRetrieve [POST] | no | no | recommended | |||
GENERAL | ||||||
/api/v1/messageResponse [POST] | yes | yes | ||||
/api/v1/messageResponseRetrieve [POST] | yes | yes | ||||
/api/v1/messageAcknowledgementRequest [POST] | all replaced by the messageResponse | |||||
/api/v1/messageAcknowledgementRetrieve [POST] | ||||||
/api/v1/warningRequest [POST] | ||||||
/api/v1/warningRetrieve [POST] | ||||||
/api/v1/messageDenialRequest [POST] | ||||||
/api/v1/messageDenialRetrieve [POST] | ||||||
CBU | ||||||
/api/v1/cordBloodUnitReportRequestRequest | on hold | no | no | should be generalised to CBU and ADCUs, etc... | ||
/api/v1/cordBloodUnitReportRequestRetrieve | on hold | yes (CBB) | yes (CBB) | |||
/api/v1/cordBloodUnitReportResponseRequest | on hold | no | no | |||
/api/v1/cordBloodUnitReportResponseRetrieve | on hold | yes (CBB) | yes (CBB) | |||
2.3 API authentication
Unable to render {include} The included page could not be found.