There are two ways to upload data for listing in the Search & Match Service:
If the data are received they are processed and available for search coordinators and transplant physicians who use the Search & Match Service for search.
This is a technical work instruction, feel free to contact us if you need any support or if you do not understand the work instruction. Our e-mail address is: support@wmda.info
Definitions
PGP | Pretty Good Privacy (PGP) is an encryption programme that provides cryptographic privacy and authentication for data communication |
XML | Extensible Markup Language, a structued file format |
API | Application Programming Interface (API): a set of clearly defined methods of communication between various software components |
ION | Issuing Organisation Number (ION) which is provided by ICCBBA to organisations listing in the Search & Match Service You may lookup IONS in the list of organisations maintained by WMDA |
XSD | XSD defines what elements and attributes may appear in an XML document; annually a new version of the XSD will be released; the current version is: 2.1 |
Purpose
This work instruction describes the manual upload method of an PgP/GpG encrypted XML file through the submission platform.
General concepts:
File requirements:
We have 2 variants of the data submission platform :
You can access the data submission platform through the web. You can access the web address (or URL) by entering the address: https://staging-dataupload.wmda.info/.
PLEASE NOTE: This platform is still under development. Therefore, you may notice outages, changes to your data, or even find some functions broken and giving errors. These potentially issues are either due to new functional deployments taking place or their being a bug in the software. If you do come across any bugs or defects please do report these as part of any feedback you may have by sending an email to support@wmda.info.
The premise of giving our listing organisations access to the test site is to allow them to familiarise themselves in advance of this being deployed to production and also start to implement and test their XML files and provide us feedback on any functions that they might wish to enhance or see added to the system.
Before you can access the data upload service, you need to be an authorized user in our system. Currently, the following users are added to the system:
All of these people will receive a welcome email from the data upload service system with instructions how they can access the service.
If you are not a current user and would like to have access to upload your test XML file, please send us an email to support@wmda.info
Forgot password:
Forgot registered email address: If you might have forgotten your e-mail address that was registered for your data upload account, please contact the WMDA office by email via support@wmda.info
Change password: You can also change your password after you logged into the system.
Tip: If you haven’t changed your password for a long time, we recommend that you do so to ensure that your profile is secure. We recommend that you change your password every 6 months.
If you attempt to login more than 3 times with incorrect credentials the system for security reasons will ask you to enter a code that appears in an image as shown in Figure 3b.
To close and exit click on the Settings button next to your name on the top left side (Figure 4). This should give you a drop-down menu with 'log out'. Click on 'Log out' and you will be redirected to the login screen.
PLEASE NOTE: Do not simply close your browser; otherwise your unsaved data may be lost.
Settings button |
After logging in, you get the screen as shown in Figure 5. On your home page, you can perform the following functions:
PLEASE NOTE: On the left hand side you can see the Workspace for your registry that you have access to. Most organisations will have access to just one space even if you are submitting data on behalf of other registries.
Setting button: when you click on the settings button, you can seen a drop-down menu (Figure 4) with
Workspace: Within your Workspace you can find the list of uploaded files
When you login for the first time, the Workspace will be empty (Figure 6).
|
Once you have used your Workspace to upload file(s), a new block will appear with your latest "Workspace activity" on the right hand menu. This menu shows the processes that your files have been through and also at what stage of the processing procedure the file is.
If you wish to upload a new PgP/GpG encrypted XML file into your Workspace in order to process the file, please follow the next steps:
An alternative method to upload your file to your Workspace is by dragging your file from your desktop to the main middle pane of your Workspace
PLEASE NOTE: The time to upload your file to the system depends on the file size and your internet upload speed.
You are only allowed to upload files that are encrypted. If you try and upload files that are non PgP/GpG encrypted the system will give you an upload error as shown in figure 8b.
|
After uploading your file, you can still delete the file if it hasn't picked-up by the processing pipeline. Once the file is picked-up, you cannot delete the file anymore or cancel the processing. The system will check for new files every 10 minutes. Therefore, the time that you have left to delete the file can vary.
|
PLEASE NOTE: Once the file has been added to the processing pipeline service the file will then move to the Archive folder. As a user you only have read/write access to your Workspace.
This folder contains all of the files that have moved to the pipeline for processing. There is a 10 minutes lag before a file that's been uploaded onto your workspace is picked by the pipeline for processing. Files in the Archive folder can't be deleted. You only have read access to the Archive folder. The file name is changed to indicate the date/time the file was placed in this folder for processing.
PLEASE NOTE: These files are deleted from the folder after 6 months of them being processed. There associated reports too will be deleted from the report folder.
This folder has all of the processed report outcomes of the uploaded file. Once the processing pipeline picked-up the file, it will start with initial validations in a sequence. The sequence is: Decrypting the file > Unzipping the file > XML file structure validation > Data content validation > HLA validation. Only once all of these validations have been completed, the system will provide all the users from the organisation a notification and a report that will indicate the outcome of the file processing. Processing messages are generated and stored in a file in your Reports folder. All messages are collected in one file for each file you uploaded to the system. This is clearly indicated using the same file name with the date/time stamp so users can easily identify the file the report relates to.
PLEASE NOTE: Just like the Archive folder you only have read access to this folder. Also all reports are deleted after 6 months of them being placed in this folder.
Below are the list of messages that users can expect and the reasons for these messages
***** STILL BEING DRAFTED***
Whilst the uploaded files are being processed users should be reported for the following stages
Users should be reported for the following scenarios on file level validations
Scenario | Event stage | Description | Type of message |
File with wrong filename | The system will reject the file if the file name doesn't conform to the agreed format which is : ION-1234-D.PgP, ION-1234-D.xml.Pgp, ION-1234-D.zip.PgP, ION-1234-C.PgP, ION-1234-C.xml.Pgp, ION-1234-C.zip.PgP. | Rejected - ADD MESSAGE AS PART OF THE REPORT | |
If a user uploads two files consecutively with same file name. Eg ION-1234-D.PgP and ION-1234-D.PgP | In such a scenario the most recently uploaded file is taken through the pipeline, the previous file is deleted and not processed. | ||
If a user uploads two files consecutively with different file names. E.g ION-1234-D.PgP and ION-1234-C.PgP | In such a scenario the system will process the two files as separate files. | ||
If a user uploads a PgP encrypted file with a invalid key. | Decrypted | In such a scenario the system will not be able to decrypt the file and the pipeline will not be able to process the file. | Rejected - ADD MESSAGE AS PART OF THE REPORT |
If a user has uploaded a zipped file which isn't an XML | Decompressed | In such a scenario once the system has decrypted the file and finds the file isn't an XML the pipeline will not be able to process the file. | Rejected - ADD MESSAGE AS PART OF THE REPORT |
If a user has uploaded a file with the file creation time set in the future | XSD file validation | In such a scenario once the system has decrypted the file and finds that the file creation date/time is in the future then the pipeline will not be able to process the file. | Rejected - ADD MESSAGE AS PART OF THE REPORT |
If a user has uploaded a file with the file creation date and/or time in an incorrect format | XSD file validation | In such a scenario once the system has decrypted the file and finds that the file creation date/time is in an incorrect format then the pipeline will not be able to process the file. | Rejected - ADD MESSAGE AS PART OF THE REPORT |
If a user has uploaded a file where the listing organisation doesn't match the ION number in the file name | XSD file validation | Rejected - ADD MESSAGE AS PART OF THE REPORT | |
If a user has uploaded a file where the update mode is other than full | XSD file validation | Rejected - ADD MESSAGE AS PART OF THE REPORT | |
If a user has uploaded a file where the file name and content type do not match. | XSD file validation | In such a scenario once the system has decrypted the file and finds that there is a discrepancy in the file name content type and the file content type then the pipeline will not be able to process the file. E.g File name = ION-1234-D.PgP and the Content type = C | Rejected - ADD MESSAGE AS PART OF THE REPORT |
If a user has uploaded a file that has content type of both donor and cord | XSD file validation | Rejected - ADD MESSAGE AS PART OF THE REPORT | |
If a user has uploaded a file where the file version isn't correct | XSD file validation | Rejected - ADD MESSAGE AS PART OF THE REPORT |
[1] The specification is available at https://pydio.com/en/docs/references/api/v2#!/File/uploadStream
STANDARD OPERATING PROCEDURE | |||
| |||
TITLE: | Data submission protocol | ||
VERSION No: | 0.3 | ||
DATE OF ORIGINAL ISSUE: | March 2017 | ||
REVIEW INTERVAL: | 12 months (This includes the testing process) | ||
ORIGINAL PREPARED BY: | Manish Kakkar | ||
QM REVIEWED BY: | Quality Team | ||
AUTHORISED BY: | Chief Executive | ||
COPY and LOCATION | Master Copy – WMDA Share – DOCxxx | ||
Document Version History | |||
Version | Author | Changes | |
0.1 | Manish Kakkar | SOP generated | |
0.2 | Beer Holties | Update (Sprint 3/4) | |
0.3 | Jorine Koenderman | Release staging version (28-07-2017) | |
0.4 | Manish Kakkar | Updated post release to staging version (07-08-2017) |