BEING UPDATED

Introduction

The service is called  Data Upload before, and is changed to Data Manager at beginning of 2024. The reason for changing name of this service is because extra feature related to data will be added gradually to this service, and Data Upload can not interpret it well.

There are two ways to upload data for listing in the Search & Match Service:

• manual upload (described below) and
• automatic upload where a connection is established between computer system, so called REST API. The manual for the automatic upload can be found here.
  
  

Data mentioned here is actually the encrypted files which can be files in XML or JSON format. Currently, only file in XML format is supported, when JSON format is supported, WMDA will send out a notification.

If the data is received, it will be processed,  and accepted data will be available for Search & Match. Search coordinators and transplant physicians who use the Search & Match Service will be able to search in these data. The data will also be used for Match-Connect service.

For the API usage, it is a technical work instruction, feel free to contact us if you need any support or if you do not understand the work instruction clearly. Our e-mail address is: support@wmda.info

Definitions

Purpose

This work instruction describes the manual upload method of an PGP/GPG encrypted file through the web application Data Manager.

General concepts:

• Only organisations that have an ION (listing organisations) will be able to upload, ION can be requested before you can list in WMDA DataManager Service.
• User must login the web application using MFA with specific email, no group email is allowed anymore, and please follow WMDA MFA user guide here to set it up.
• The platform is organised in so called 'Workspaces'. This Workspace is a private space for each organisation where files can be uploaded and processing reports become available.
• If your organisation submits data on behalf of other registries/CBBs then as an uploading organisation you will be able to see Workspace where you can upload files for other registry(ies) that you are affiliated with.
• Uploaded XML files should follow the XML guidelines as described in the Data submission information, and JSON format in the page  TBC. JSON format will be supported from data schema version 2.4.
  
  

File requirements:
Please check the detail user guide of XML generation and requirements in Search & Match Service Data submission information.
Some key requirements are as below:

• Files must following the file naming rules as described in the Search & Match Service Data submission information
• Only files with the .pgp and .gpg extension will be accepted for upload.
• XML files should be encrypted with a public key (How to encrypt a file can be found here)
• XML files larger than 800MB is rejected. If your file reach this limitation, contact support team at support@wmda.info. 

Getting started

Data Manager Service URLs

We have 2 data submission platform :

• sandbox: https://sandbox-datamanager.wmda.info : here organisations can upload their data for testing purposes and check their dataquality, before moving to production.
• production: https://datamanager.wmda.info : here production data is uploaded, which is permitted only after QA inspection by WMDA.

Setup MFA first time to get access for Data Manager

Please follow steps below if you have not setup MFA with any other WMDA services like Search & Match or SPEAR:

1. After you get permission to list data in WMDA Data Manager service, please contact support team at support@wmda.info to add you in our CRM system, mention it clearly you want to get access for Data Manager web application.
2. After you are added in our system, normally after 30 mins, you will get a Welcome email, follow steps inside to setup the MFA. For your convenience, we also added the steps below:
   
   1. Click on the following link: WMDA Service password reset
   2. Confirm your e-mail address and set your password
   3. When you are done with setting up your password go to: https://portal.wmda.info/
   4. Click on “Partner Sign in” in top right corner
   5. Enter your email address as username
   6. Enter the password you set up earlier
   7. Click “sign in”
   8. Follow the steps to add this account to your authentication app on your mobile device
   9. Once complete, confirm that your MFA credentials allows you access to https://datamanager.wmda.info/

To access the system 

1. You successfully setup your MFA.
2. To access the sandbox system to test your files, please go to https://sanbox-datamanager.wmda.info/. To access the production system, please got to https://datamanager.wmda.info/. You will see the Login screen as in Figure 1.
3. If you are a normal (B2C) user, click on the Login button in Figure 1.  If you are a B2B user, please follow the user guide here to login. 
4. Follow the steps for MFA to log in Data Manager web application.
5. After login, the system defaults to your personalized Workspace, where you will find your organisation Workspace and all the folders you need. 
   
   

---

Figure 1. Login screen WMDA Data Manager service

Home pages and workspace

After logging in, you will get the screen based on whether you have access to one ION or multiple IONs.

If you have access to only one ION, then you will be redirect to your Workspace folders directly as show in Figure 2.
In Figure 2, it shows the example of the workspace for ION-9998, and all the folders user have access to. We will give more guidance to use this Workspace page below. 

---

Figure 2. Home page for uses have access to one ION

If you have access to multiple IONs, then you will be redirected to a web page with the list of ION as show in Figure 3. You can click button to go into workspace for each ION. And you can click on the tab "ION List" to be back to the ION list page. Statistic information for each ION will be added for user to monitor the activities in the future.

---

Figure 3. Home page for uses have access to multiple IONs

Workspace 

Workspace is the place where user can upload, list, download, view the files and processing reports.

When you login for the first time, the Workspace will be empty  for each folder (Figure 4). You will see folders: Upload, processing, Processed, Rejected, Failed, Report Dqreport.

 

---

Figure 4: Content of workspace screen

Upload folder

You will only see button "Upload file"  in the Upload folder. Please check the user guide below to see how to upload a file.

Processing folder

This folder contains the files are picked up for processing. 

Processed folder

This folder contains all the files are processed and accepted by the Data Manager Service.

Rejected folder

This folder contains all the files are processed and rejected on file level by the Data Manager Service.

Failed folder

This folder contains all the files are failed to be processed by the Data Manager Service.

Report folder

This folder has all of the processed report outcomes of the uploaded file. Once the processing pipeline picked-up the file, it will start with initial validations in a sequence. The sequence is: Decrypting the file > Unzipping the file (optional) > XML file structure validation > Data content validation > HLA validation. Only once all of these validations have been completed, the system will provide all the users have access to the folder an email notification and a report that indicates the outcome of the file processing will be generated. Processing messages are generated and stored in a .txt file in the Reports folder of the ION. All messages are collected in one file for each file you uploaded to the system. This is clearly indicated using the same file name with the date/time stamp so users can easily identify the file the report relates to.

Check more details of the message in the report in the page here. 

PLEASE NOTE: All files included the report will be deleted from the folder after 6 months of them being processed. 

Dqreport folder

TBA

Upload a file

If you want to upload a new PGP/GPG encrypted XML file into your Workspace to be processed, please follow the next steps:

1. Click on the Upload folder of your organisation/correct ION.
2. Within the Upload folder click on icon "Upload file" at right upper corner in the file list area. And a window to guide to upload a file will pop-up. (Figure 5). 
3. Option to select a file:
   1). Click on button "Pick file" and a browser window will open and you can select the file on your computer.
   2). You can also select a file by dragging  and dropping the file in the proper area.
4. After a file is selected,  then the selected file will be listed, and the "Upload" button will become available. (Figure 6)
5. Confirm the file selected is the correct file based on the name. 
6. Click on button Upload.
7. The file then uploaded and becomes visible in the file list area in the Upload folder. If the file is big, it may take some time to upload.

PLEASE NOTE: You are only allowed to upload files that are encrypted. If you try and upload files that are non PGP/GPG encrypted, the system will give you an upload error. (Figure 7)

After a file is uploaded completed to the Upload folder, then it can be picked up.  After the An alternative method to upload your file to your Workspace is by dragging your file from your desktop to the main middle pane of your Workspace

•  Once the upload of the file is completed, you receive a message.
• The file is now ready to be picked-up by the Search & Match Service for further processing. When the file is picked-up by the processing pipeline, a copy of the file is stored in your archives folder. All messages concerning the processing of your file can be found in the Reports folder. The copy of the file and the file with messages have a filename related to the name of the original file in combination with the upload time and user

PLEASE NOTE: The time to upload your file to the system depends on the file size and your internet upload speed.

---

Figure 5: Upload new file

---

Figure 6: Select file for upload screens

Update existing uploaded files 

After uploading your file, you can still delete the file if it hasn't picked-up by the processing pipeline. Once the file is picked-up, you cannot delete the file anymore or cancel the processing. The system will check for new files every 10 minutes. Therefore, the time that you have left to delete the file can vary.

1. Within your Workspace go to the Upload folder, go to the file you wish to update/delete and click on the 'Delete' button at the end of the file (Figure 8). 
2. The system will warn you to ensure if you really want to delete this file (Figure 9).
3. If you click OK, the file is deleted and will no longer be processed.

PLEASE NOTE: Once the file has been added to the processing pipeline service the file will then be moved to the Processing folder. As a user you only have read access to all other folders except Upload folder.

 

---

Figure 8: How to delete an uploaded file

 

---

Figure 9: How to delete an uploaded file

Logout from the service

To close and exit, click on the dropdown arrow next to your name on the top left side (Figure 10). This should give you a drop-down menu with 'Logout'. Click on 'Logout' and you will be get a pop-up window to pick an account to logout (Figure 11). If you have multiple accounts listed here, please select the correct one. And after that you will see a notification window to mention you now safely logout.

Manage and transfer MFA

When you setup your MFA, or after your MFA is setup successfully, you may encounter issues. Go to the WMDA MFA troubleshooting guide to check whether your issue can be fixed. After your MFA is setup, the

Processing report messages

Below are the list of messages that users can expect and the reasons for these messages

Whilst the uploaded files are being processed users should be reported for the following stages

• Uploaded file successfully - This is a message that users get as part of the standard file upload successfully. Please see section upload a file as part of uploading your file.
• Uploaded file is picked up by pipeline for processing 
• Uploaded file processed and report available by the pipeline

Users should be reported for the following scenarios on file level validations