⚠Due to planned maintenance you will experience short (<30 min) downtime between 08:00 - 08:30 CET.
Status | ||||
---|---|---|---|---|
|
Table of Contents |
---|
Introduction
Data Manager, previously known as Data Upload, is WMDA's enhanced platform designed for efficient and secure data management within the global donor registry network. Renamed in early 2024, the platform's name change reflects its expanded functionality beyond simple data uploads, accommodating a growing range of data-related features.
Data Manager supports both manual and automatic data uploads, allowing registries to list donor information There are two ways to upload data for listing in the Search & Match Service:
...
. Manual uploads involve uploading encrypted XML files, while automatic uploads utilize a REST API for seamless integration between systems. Though XML is currently the primary format, support for JSON files is forthcoming, with notifications to be provided when this becomes available.
Once data is uploaded and processed, it becomes accessible to
...
If the data are received, they are processed and available for search coordinators and transplant physicians who use through the Search & Match Service for search, and is also used for integrated into the Match-Connect service. For those using the API usage, it is a technical work instruction, feel free to contact us if you need any support or if you do not understand the work instruction clearly. Our e-mail address is: support@wmda.infois readily available to ensure smooth operation and understanding.
Data Manager is designed to meet the evolving needs of donor registries, offering robust and real-time data processing capabilities that ensure accurate and up-to-date information is always available. For more information and support, you can contact us directly by emailing support@wmda.infoThe service is called Data Manager before, and now changed to Data Manager.
Definitions
PGP/GPG | Pretty Good Privacy (PGP) is an encryption programme that provides cryptographic privacy and authentication for data communication |
XML | Extensible Markup Language, a structured file format |
API | Application Programming Interface (API): a set of clearly defined methods of communication between various software components |
ION | Issuing Organisation Number (ION) which is provided by ICCBBA to organisations listing in the Search & Match Service You may lookup IONS in the list of organisations maintained by WMDA |
XSD | XSD defines what elements and attributes may appear in an XML document; annually a new version of the XSD will be released; the current version is: 2.1 |
JSON | JavaScript Object Notation, a structured data format |
Purpose
...
General concepts:
Only
organisations that haveorganizations with an ION (
listing organisations) will be able to upload- The usernames are your email address that you have requested to be added in our database as Data Managerers
- Every organisation that is listing only donors or only cord blood units is allowed to have two dedicated accounts. If an organisation is listing both donors and cord blood units, they are allowed to have a maximum of four accounts. Multiple accounts of one organisation share one Workspace, but every user has its own login details.
- The platform is organised in so called 'Workspaces'. This Workspace is a private space for each organisation where files can be uploaded and processing reports become available.
International Organization Number) are eligible to upload data to WMDA's Data Manager. If your organization does not have an ION, you will need to request one before listing data.
To access the Data Manager platform, users must log in using Multi-Factor Authentication (MFA) with a personal email address (group emails are not permitted). For instructions on setting up MFA, please refer to the WMDA MFA User Guide.
The Data Manager platform is organized into "Workspaces," which are private areas designated for each organization. Within your Workspace, you can upload files and view processing reports. If your organization
If your organisationsubmits data on behalf of other registries
then as an uploading organisation you will only see one Workspace where you can upload files for other registry(ies) that you are affiliated with.or cord blood banks (CBBs), you will also have access to their Workspaces to upload files for them.
When uploading XML files, please ensure they comply with the XML guidelines outlined in the Data Submission Information. JSON format will also be supported with the release of data schema version 2.4, and guidelines will be provided accordingly
Uploaded files should follow the XML guidelines as described in the Data submission information.
File requirements:
For detailed instructions on generating XML files and the specific requirements, please consult the Search & Match Service Data submission information. Some key points to remember include:
- Files must adhere to the file naming conventions specified Files should following the file naming rules as described in the Search & Match Service Data submission information
- XML files larger than 200Mb should be compressed using ZIP (How to compress your file with ZIP can be found here)
- .
- Only files with the a .pgp and or .gpg extension will be accepted for upload.
- XML files should must be encrypted with using a public key (How public key. Instructions on how to encrypt a file can be found here)[here].
- XML files larger than 800MB will be rejected. If your file exceeds this limit, please contact support at support@wmda.info.
Getting started
Accessing the data submission platform
We have 2 variants of the data submission platform :
- staging-dataupload.wmda.info : here organisations can upload their data for testing purposes and check their dataquality, before moving to
- dataupload.wmda.info : here production data is uploaded, which is permitted only after QA inspection by WMDA.
You can access the data submission platform through the web. You can access the web address (or URL) by entering the address: https://staging-dataupload.wmda.info/.
PLEASE NOTE: This platform is still under development. Therefore, you may notice outages, changes to your data, or even find some functions broken and giving errors. These potentially issues are either due to new functional deployments taking place or their being a bug in the software. If you do come across any bugs or defects please do report these as part of any feedback you may have by sending an email to support@wmda.info.
The premise of giving our listing organisations access to the test site is to allow them to familiarise themselves in advance of this being deployed to production and also start to implement and test their XML files and provide us feedback on any functions that they might wish to enhance or see added to the system.
Logging in for the first time
Before you can access the Data Manager service, you need to be an authorized user in our system. Currently, the following users are added to the system:
- All users from the old test upload system who helped with the proof of concept consultation
- All people who previously sent a test file by email
- All people who requested access by email
All of these people will receive a welcome email from the Data Manager service system with instructions how they can access the service.
If you are not a current user and would like to have access to upload your test XML file, please send us an email to support@wmda.info
To access the system for the first time
...
Data Manager Service URLs
There are two environments for data submission:
- Sandbox: https://sandbox-datamanager.wmda.info - This environment is for testing purposes, allowing organizations to upload data and check its quality before moving to production.
- Production: https://datamanager.wmda.info - This environment is for uploading data to be used in the Search & Match Service and Match-Connect. Production uploads are only permitted after a quality assurance inspection by WMDA.
Setting Up MFA for Data Manager Accesset
Setup MFA for the first time
If you have not set up MFA for other WMDA services (such as Search & Match or SPEAR), please follow these steps:
- After receiving permission to list data in WMDA Data Manager, contact support at support@wmda.info to be added to the CRM system. Be sure to mention that you need access to the Data Manager web application, and provide the personal based email which can be used to setup MFA.
- Once added to WMDA system (typically within 30 minutes), you will receive a Welcome email with instructions for setting up MFA. The Welcome email will give clear information you are a B2C user or B2B user.
- If you do not receive the Welcome email or encounter any issues during setup, please contact support at support@wmda.info.
Troubleshooting MFA
If you experience issues with MFA during or after setup, consult the WMDA MFA troubleshooting guide for solutions.
Managing and Transferring MFA to a New Mobile Device
If you need to transfer your MFA to a new device after setting it up, follow these steps:
- Log in to the Data Manager service.
- Click the dropdown arrow next to your name in the top left corner.
- Select "Manage and Transfer My MFA."
- Follow the detailed guide to manage your MFA method.
Imagefloat | ||
---|---|---|
| ||
Accessing the system
Once MFA is set up, you can access the system:
- To test to upload your files, go to https://sandbox-datamanager.wmda.info.
- To access the production system, go to https://datamanager.wmda.info.
At the login screen, click the "Login" button if you are a B2C user. B2B users should follow the specific user guide for login instructions. After logging in, you will be directed to your personalized Workspace, where you can access your organization’s Workspace and folders.
Imagefloat | ||
---|---|---|
| ||
Home Page and Workspace
Upon logging in, your access will depend on whether you have access to one or multiple IONs:
- If you have access to only one ION, you will be redirected to your Workspace folders directly.
- If you have access to multiple IONs, you will be directed to a list of IONs, where you can choose the appropriate Workspace.
Imagefloat | ||
---|---|---|
| ||
Imagefloat | ||
---|---|---|
| ||
Workspace Folders
Your Workspace contains several folders where you can upload, view, and manage your files:
- Upload Folder: Where you can upload new files.
- Processing Folder: Contains files that are being processed.
- Processed Folder: Contains files that have been successfully processed.
- Rejected Folder: Contains files that were rejected during processing.
- Failed Folder: Contains files that failed to process.
- Report Folder: Stores reports generated after processing. These reports include the outcome of the file validation and processing steps. Reports are retained for six months.
Imagefloat | ||
---|---|---|
| ||
|
Uploading a File
To upload an encrypted XML file to your Workspace:
- Click on the "Upload" folder in your Workspace.
- Click the "Upload file" icon in the upper right corner.
- Select a file by either clicking "Pick file" or dragging and dropping it into the designated area.
- Once the file is selected, click "Upload" to begin the process.
Note: Only encrypted files (.pgp or .gpg) are allowed. Non-encrypted files will trigger an upload error.
...
Imagefloat | ||
---|---|---|
| ||
Change password or forgot password
Forgot password:
- Click on the 'Forgot your password?' link and fill out your email address on the password reset page (Figure 2) and click on the 'OK' button. You will then receive an email with a link to reset your password.
- Click on the link and choose your own password, confirm this password and click on the OK button.
Forgot registered email address: If you might have forgotten your e-mail address that was registered for your Data Manager account, please contact the WMDA office by email via support@wmda.info
Change password: You can also change your password after you logged into the system.
- To do this, click on your home page on the settings button.
- A drop-down menu will appear and then click on 'My Account'.
- In the next window, click on the button 'change password' in the left bottom corner
- A new smaller window will open. Fill out your old password and your new chosen password and click on OK.
Tip: If you haven’t changed your password for a long time, we recommend that you do so to ensure that your profile is secure. We recommend that you change your password every 6 months.
Imagefloat | ||
---|---|---|
| ||
Imagefloat | ||
---|---|---|
| ||
If you attempt to login more than 3 times with incorrect credentials the system for security reasons will ask you to enter a code that appears in an image as shown in Figure 3b.
Imagefloat | ||
---|---|---|
| ||
Logging off from the service
To close and exit click on the Settings button next to your name on the top left side (Figure 4). This should give you a drop-down menu with 'log out'. Click on 'Log out' and you will be redirected to the login screen.
PLEASE NOTE: Do not simply close your browser; otherwise your unsaved data may be lost.
Imagefloat | ||
---|---|---|
| ||
Settings button |
Home
After logging in, you get the screen as shown in Figure 5. On your home page, you can perform the following functions:
Imagefloat | ||
---|---|---|
| ||
- Your name: On the top left corner. This is the name of the user that logged into the system
- Notification indicator: This is the bell that will highlight if you have any new notifications around the status of your uploaded file
- Setting button : This is the menu for functions like - Logout and Updating account details
- Search box: Useful when trying to find files, for example using a specific date
- My Workspace: Your own Workspace named with the ION of your organisation. In conjunction with your Workspace you also see two separate folders called Archives and Reports. For further details on the purpose of these folders please see section around Archive/Reports.
- Recent history of submissions in the middle main pane
PLEASE NOTE: On the left hand side you can see the Workspace for your registry that you have access to. Most organisations will have access to just one space even if you are submitting data on behalf of other registries.
Setting button: when you click on the settings button, you can seen a drop-down menu (Figure 4) with
- My account: In this section you can set an avatar (picture) or your organisations logo and change your password. You can only review the Name and Email address but cant change it. If the name of email address shown within the account section is wrong please contact WMDA via email support@wmda.info so we can update those details in out CRM database.
- Share and About: These two functions at the moment aren't developed so please refrain from using this.
- Log out: Exit system
Workspace: Within your Workspace you can find the list of uploaded files
When you login for the first time, the Workspace will be empty (Figure 6).
Imagefloat | ||
---|---|---|
| ||
|
Once you have used your Workspace to upload file(s), a new block will appear with your latest "Workspace activity" on the right hand menu. This menu shows the processes that your files have been through and also at what stage of the processing procedure the file is.
Upload a file
If you wish to upload a new PgP/GpG encrypted XML file into your Workspace in order to process the file, please follow the next steps:
- Click on the Workspace of your organisation.
- Within your Workspace click on the red tab with 'New' and then on the 'Upload' icon at the bottom of the drop-down menu (Figure 7).
- A new screen will open where you can select your file for upload (Figure 8a).
- Click on the red button with 'Pick File'.
- A browser window will open and you can select the file on your computer you would like to upload. Click on Open.
- The file becomes then visible in the browser window and when it is the correct file, click on OK to upload the file to your Workspace.
An alternative method to upload your file to your Workspace is by dragging your file from your desktop to the main middle pane of your Workspace
- Once the upload of the file is completed, you receive a message.
- The file is now ready to be picked-up by the Search & Match Service for further processing. When the file is picked-up by the processing pipeline, a copy of the file is stored in your archives folder. All messages concerning the processing of your file can be found in the Reports folder. The copy of the file and the file with messages have a filename related to the name of the original file in combination with the upload time and user
PLEASE NOTE: The time to upload your file to the system depends on the file size and your internet upload speed.
Once uploaded, the file will be picked up by the processing pipeline for further processing.
Imagefloat | ||
---|---|---|
| ||
Imagefloat | ||
---|---|---|
| ||
You are only allowed to upload files that are encrypted. If you try and upload files that are non PgP/GpG encrypted the system will give you an upload error as shown in figure 8b.
Imagefloat | ||
---|---|---|
| ||
|
Manage/Update existing uploaded files
Updating Existing Uploaded Files
If you need to delete a file after uploading but before it is processed:
- Go to the "Upload" folder and locate the file.
- Click the "Delete" button next to the file.
- Confirm the deletion when prompted.
Note: Once a file is picked up for processing, it can no longer be deletedAfter uploading your file, you can still delete the file if it hasn't picked-up by the processing pipeline. Once the file is picked-up, you cannot delete the file anymore or cancel the processing. The system will check for new files every 10 minutes. Therefore, the time that you have left to delete the file can vary.
...
As a user you only have read access to all other folders except Upload folder.
Imagefloat | ||
---|---|---|
| ||
|
...
Imagefloat | ||
---|---|---|
| ||
|
PLEASE NOTE: Once the file has been added to the processing pipeline service the file will then move to the Archive folder. As a user you only have read/write access to your Workspace.
Archive folder
This folder contains all of the files that have moved to the pipeline for processing. There is a 10 minutes lag before a file that's been uploaded onto your workspace is picked by the pipeline for processing. Files in the Archive folder can't be deleted. You only have read access to the Archive folder. The file name is changed to indicate the date/time the file was placed in this folder for processing.
PLEASE NOTE: These files are deleted from the folder after 6 months of them being processed. There associated reports too will be deleted from the report folder.
Report folder
This folder has all of the processed report outcomes of the uploaded file. Once the processing pipeline picked-up the file, it will start with initial validations in a sequence. The sequence is: Decrypting the file > Unzipping the file > XML file structure validation > Data content validation > HLA validation. Only once all of these validations have been completed, the system will provide all the users from the organisation a notification and a report that will indicate the outcome of the file processing. Processing messages are generated and stored in a file in your Reports folder. All messages are collected in one file for each file you uploaded to the system. This is clearly indicated using the same file name with the date/time stamp so users can easily identify the file the report relates to.
PLEASE NOTE: Just like the Archive folder you only have read access to this folder. Also all reports are deleted after 6 months of them being placed in this folder.
Processing report messages
Below are the list of messages that users can expect and the reasons for these messages
***** STILL BEING DRAFTED***
Whilst the uploaded files are being processed users should be reported for the following stages
- Uploaded file successfully - This is a message that users get as part of the standard file upload successfully. Please see section upload a file as part of uploading your file.
- Uploaded file is picked up by pipeline for processing
- Uploaded file processed and report available by the pipeline
Users should be reported for the following scenarios on file level validations
...
|
Downloading a File
To download a file from your Workspace:
- Locate the file and click the download button.
- The file will appear in your browser's download list.
Imagefloat | ||
---|---|---|
| ||
Logging Out
To log out:
- Click the dropdown arrow next to your name in the top left corner.
- Select "Logout" from the menu.
- Choose the account you wish to log out of and confirm.
Imagefloat | ||
---|---|---|
| ||
Imagefloat | ||
---|---|---|
| ||
|
Other Information
Session Timeout
For security reasons, your session will time out after one hour of inactivity. If this happens, you can reactivate the page by clicking the pop-up window. If more than two hours have passed, you will need to log in again.
Imagefloat | ||
---|---|---|
| ||
...
[1] The specification is available at https://pydio.com/en/docs/references/api/v2#!/File/uploadStream
STANDARD OPERATING PROCEDURE
TITLE:
Data submission protocol
VERSION No:
0.3
DATE OF ORIGINAL ISSUE:
March 2017
REVIEW INTERVAL:
12 months (This includes the testing process)
ORIGINAL PREPARED BY:
Manish Kakkar
QM REVIEWED BY:
Quality Team
AUTHORISED BY:
Chief Executive
COPY and LOCATION
Master Copy – WMDA Share – DOCxxx
Document Version History
Version
Author
Changes
0.1
Manish Kakkar
SOP generated
0.2
Beer Holties
Update (Sprint 3/4)
0.3
Jorine Koenderman
Release staging version (28-07-2017)