Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Status
colourYellow
titlebeing updated

Table of Contents

Introduction

There are two ways to upload data for listing in the Search & Match Service:

  • manual upload (described below) and
  • automatic upload where a connection is established between computer system, so called REST API. The manual for the automatic upload can be found here.

If the data are received they are processed and available for search coordinators and transplant physicians who use the Search & Match Service for search.

...

  • Only organisations that have an ION (listing organisations) will be able to upload
  • The usernames are your email address that you have requested to be added in our database as Data Uploaders
  • Every organisation that is listing only donors or only cord blood units is allowed to have two dedicated accounts. If an organisation is listing both donors and cord blood units, they are allowed to have a maximum of four accounts. Multiple accounts of one organisation share one Workspace, but every user has its own login details.
  • The platform is organised in so called 'Workspaces'. This Workspace is a private space for each organisation where files can be uploaded and processing reports become available.
  • If your organisation submits data on behalf of other registries then as an uploading organisation you will only see one Workspace where you can upload files for other registry(ies) that you are affiliated with.
  • Uploaded files should follow the XML guidelines as described in the Data submission information.

File requirements:

  • Files should following the file naming rules as described in the Search & Match Service Data submission information
  • XML files larger than 200Mb should be compressed using ZIP (How to compress your file with ZIP can be found here)
  • Only files with the .pgp and .gpg extension will be accepted for upload.
  • XML files should be encrypted with a public key (How to encrypt a file can be found here)

Getting started

Accessing the data submission platform

We have 2 variants of the data submission platform :

...

The premise of giving our listing organisations access to the test site is to allow them to familiarise themselves in advance of this being deployed to production and also start to implement and test their XML files and provide us feedback on any functions that they might wish to enhance or see added to the system.

Logging in for the first time

Before you can access the data upload service, you need to be an authorized user in our system. Currently, the following users are added to the system:

...

If you are not a current user and would like to have access to upload your test XML file, please send us an email to support@bmdw.org 

To access the system for the first time

  1. As a new user, you should have received a welcome email that includes a link to the data upload service and instructions to set your password. 
  2. Click on the link and then on the 'Forgot your password?' link.
  3. Fill out your email address and click on OK. You will now receive an email with a link to reset your password. 
  4. Click on the link, choose your own password, confirm this password and click on the OK button.  
  5. Go to https://staging-dataupload.wmda.info/ to login (Figure 1) If you just have set your password, the system should return you automatically to the login page. 
  6. Within the login page you can optionally change your language by clicking on the flag just besides 'Enter login/password': 
  7. You can also save a bookmark in your browser, e.g. for the data upload service, so that you are immediately directed to the login page rather than having to enter the URL.
  8. Enter your email address and your password.
  9. The system defaults to your personalized Workspace, where you will find your organisation Workspace and all other folders like notification and archived folder.

...

Imagefloat
captionLogin screen BMDW data upload service

 


Change password or forgot password

Forgot password:

  1. Click on the 'Forgot your password?' link and fill out your email address on the password reset page (Figure 2) and click on the 'OK' button. You will then receive an email with a link to reset your password.
  2. Click on the link and choose your own password, confirm this password and click on the OK button. 

...

Imagefloat
captionFigure 3b: Screen after 3 login attempts

 


Logging off from the service

To close and exit click on the Settings button next to your name on the top left side (Figure 4). This should give you a drop-down menu with 'log out'. Click on 'Log out' and you will be redirected to the login screen.

...

Imagefloat
captionFigure 4: Logging out of the BMDW data upload service: setting button and dropdown menu

Settings button


Home

After logging in, you get the screen as shown in Figure 5. On your home page, you can perform the following functions:

...

Once you have used your Workspace to upload file(s), a new block will appear with your latest "Workspace activity" on the right hand menu. This menu shows the processes that your files have been through and also at what stage of the processing procedure the file is.


Upload a file

If you wish to upload a new PgP/GpG encrypted XML file into your Workspace in order to process the file, please follow the next steps:

...

Imagefloat
captionFigure 8b: Only PgP encrypted files allowed

 


Manage/Update existing uploaded files 

After uploading your file, you can still delete the file if it hasn't picked-up by the processing pipeline. Once the file is picked-up, you cannot delete the file anymore or cancel the processing. The system will check for new files every 10 minutes. Therefore, the time that you have left to delete the file can vary.

...

PLEASE NOTE: Once the file has been added to the processing pipeline service the file will then move to the Archive folder. As a user you only have read/write access to your Workspace.


Archive folder

This folder contains all of the files that have moved to the pipeline for processing. There is a 10 minutes lag before a file that's been uploaded onto your workspace is picked by the pipeline for processing.  Files in the Archive folder can't be deleted. You only have read access to the Archive folder. The file name is changed to indicate the date/time the file was placed in this folder for processing.

PLEASE NOTE: These files are deleted from the folder after 6 months of them being processed. There associated reports too will be deleted from the report folder. 

Report folder

This folder has all of the processed report outcomes of the uploaded file. Once the processing pipeline picked-up the file, it will start with initial validations in a sequence. The sequence is: Decrypting the file > Unzipping the file > XML file structure validation > Data content validation > HLA validation. Only once all of these validations have been completed, the system will provide all the users from the organisation a notification and a report that will indicate the outcome of the file processing. Processing messages are generated and stored in a file in your Reports folder. All messages are collected in one file for each file you uploaded to the system. This is clearly indicated using the same file name with the date/time stamp so users can easily identify the file the report relates to. 

PLEASE NOTE: Just like the Archive folder you only have read access to this folder. Also all reports are deleted after 6 months of them being placed in this folder.

Processing report messages

Below are the list of messages that users can expect and the reasons for these messages

...

                                                                                   

ScenarioEvent stageDescriptionType of message
File with wrong filename The system will reject the file if the file name doesn't conform to the agreed format which is : ION-1234-D.PgP, ION-1234-D.xml.Pgp, ION-1234-D.zip.PgP, ION-1234-C.PgP, ION-1234-C.xml.Pgp, ION-1234-C.zip.PgP.Rejected - ADD MESSAGE AS PART OF THE REPORT
If a user uploads two files consecutively with same file name. Eg ION-1234-D.PgP and ION-1234-D.PgP In such a scenario the most recently uploaded file is taken through the pipeline, the previous file is deleted and not processed.  
If a user uploads two files consecutively with different file names. E.g ION-1234-D.PgP and ION-1234-C.PgP In such a scenario the system will process the two files as separate files.  
If a user uploads a PgP encrypted file with a invalid key. DecryptedIn such a scenario the system will not be able to decrypt the file and the pipeline will not be able to process the file. Rejected - ADD MESSAGE AS PART OF THE REPORT
If a user has uploaded a zipped file which isn't an XMLDecompressedIn such a scenario once the system has decrypted the file and finds the file isn't an XML the pipeline will not be able to process the file.Rejected - ADD MESSAGE AS PART OF THE REPORT
If a user has uploaded a file with the file creation time set in the future XSD file validationIn such a scenario once the system has decrypted the file and finds that the file creation date/time is in the future then the pipeline will not be able to process the file.Rejected - ADD MESSAGE AS PART OF THE REPORT
If a user has uploaded a file with the file creation date and/or time in an incorrect format XSD file validationIn such a scenario once the system has decrypted the file and finds that the file creation date/time is in an incorrect format then the pipeline will not be able to process the file.Rejected - ADD MESSAGE AS PART OF THE REPORT
If a user has uploaded a file where the listing organisation doesn't match the ION number in the file nameXSD file validation Rejected - ADD MESSAGE AS PART OF THE REPORT
If a user has uploaded a file where the update mode is other than fullXSD file validation Rejected - ADD MESSAGE AS PART OF THE REPORT
If a user has uploaded a file where the file name and content type do not match. XSD file validationIn such a scenario once the system has decrypted the file and finds that there is a discrepancy in the file name content type and the file content type  then the pipeline will not be able to process the file. E.g File name = ION-1234-D.PgP and the Content type = CRejected - ADD MESSAGE AS PART OF THE REPORT
If a user has uploaded a file that has content type of both donor and cord XSD file validation Rejected - ADD MESSAGE AS PART OF THE REPORT
If a user has uploaded a file where the file version isn't correctXSD file validation Rejected - ADD MESSAGE AS PART OF THE REPORT

 


 

STANDARD OPERATING PROCEDURE

 

TITLE:

Data submission protocol

VERSION No:

0.3

DATE OF ORIGINAL ISSUE:

March 2017

REVIEW INTERVAL:

12 months (This includes the testing process)

ORIGINAL PREPARED BY:

Manish Kakkar

QM REVIEWED BY:

Quality Team

AUTHORISED BY:

Chief Executive

COPY and LOCATION

Master Copy – WMDA Share – DOCxxx

Document Version History

Version

Author

Changes

0.1

Manish Kakkar

SOP generated

0.2

Beer Holties

Update (Sprint 3/4)

0.3

Jorine Koenderman

Release staging version (28-07-2017)

0.4Manish Kakkar Updated post release to staging version (07-08-2017)