Table of Contents

Purpose

Upload and download files in Data Manager can be done using various mechanisms : 

• Web-based interface at https://sandbox-datamanager.wmda.info (please note the URL for production is https://datamanager.wmda.info )
• REST API, using commands like cURL, or libraries like Restsharp

This document describes the Data Manager mechanism via the REST API method, where users can upload a PGP encrypted file.

Please do read https://ec.haxx.se/usingcurl-netrc.html on the use of Bearer Token if you plan to automate these procedures.

# example use:
curl --netrc-file mycredentialsfile https://xyz.com

# the mycredentialsfile contains :
machine <xyz.com> login <johndoe@xyz.com> password <uf9873o^9ufwa>

Using the REST API

...

Introduction

Data Manager, previously known as Data Upload, is WMDA's enhanced platform designed for efficient and secure data management within the global donor registry network. Renamed in early 2024, the platform's name change reflects its expanded functionality beyond simple data uploads, accommodating a growing range of data-related features.

Data Manager supports both manual and automatic data uploads, allowing registries to list donor information in the Search & Match Service. Manual uploads involve uploading encrypted XML files, while automatic uploads utilize a REST API for seamless integration between systems. Though XML is currently the primary format, support for JSON files is forthcoming, with notifications to be provided when this becomes available.

Once data is uploaded and processed, it becomes accessible to search coordinators and transplant physicians through the Search & Match Service, and is also integrated into the Match-Connect service. For those using the API, technical support is readily available to ensure smooth operation and understanding.

Data Manager is designed to meet the evolving needs of donor registries, offering robust and real-time data processing capabilities that ensure accurate and up-to-date information is always available. For more information and support, you can contact us directly by emailing support@wmda.info

This document describes the mechanism via the REST API method using curl commands. You can use what programming language you are familiar with to implement the API usage.

Request and implement credential

First time request

If you are not an API user yet, and plan to automate your file handling process. Then please request an API credential from WMDA support team to email support@wmda.info. And please follow steps below:

1. Send email to support@wmda.info with a PGP/GPG public key.  Or if you have other secure way to communicate the credential, please identify that in the email. Below we provide guide how can you generate PGP public key
2. WMDA will provide you the credential and send email to notify you.
3. You test the credential is working and can call endpoint in sandbox system. You can follow the steps below at "Implementation of Generating Bear Token" and  "Using the REST API" to test the credential:
   1. Implement the credential.
   2. Refresh Bear upload user guide below.

Guide to generate PGP/GPG key:

• Use command-line:
  https://www.gnupg.org/gph/en/manual/c14.html

• Use windows based tool Kleopatra
  
  

Implementation of Generating Bear Token

Data Manager uses Bear Token for the authorization as all WMDA API services, please follow user guide for API authentication to implement https://share.wmda.info/x/8I7RFw.

Update credential because of expiration

WMDA API credential is only available for one year currently. If your credential is expired, please send to WMDA support team to request a new one. 
Currently, WMDA is implementing a new feature to let API user to request a new credential by themselves before the credential expires, we will send announcement when this feature is available.

Using the REST API

To get more knowledge for usage of curl, we recommend member to read the doc for curl   https://ec.haxx.se/usingcurl-netrc.html. 

There are 2 steps 

Step 1: Request the credential which can be used to generate the Bear Token 

Step 2: Test to use the API in WMDA Data Manager swaggerData Manager API specification is in the swagger doc: https://brave-forest-04a379103.azurestaticapps.netapispecs.wmda.info/?urls.primaryName=Data%20Manager%20Data%20Exchange%20API#/

Step 3: Implement the API.

If you are already familiar with the API  implementation, then you can skip Step 2, and test from your own implementation.

Data%20Manager%20Data%20Exchange%20API

Below is an example to guide how to list ion(s) that you have access:

Step 1: Set up the credential by following the instruction above in the "Implementation of Generating Bear Token" 

Step 2: Test to use the API in WMDA Data Manager sandbox.
            Below is an example to guide how to get the information to upload a file:In case the upload succeeded, an XML response with meta information of the upload is generated. You may use this for own logging purposes. For more info you can read the full API specification.

no request body needed

Content-Type

application/

• Server name should be https://staging-dataupload.wmda.info
• For production you will need https://dataupload.wmda.info

SAMPLE Request

First test the connection by using the right path and fetching the metadata : it should result in a response 200 OK.

• Look carefully at the path : here we use ion0999, that should be replaced by your ION.
• You may notice that we add the filename to the path: depending on the library you may need to add that

Image Removed

Upload

Now we know the path is correct and we can do an upload using cURL: 

Below is the invalid to deploy, currently it is still available in PRODUCTION till production maintenance. But not available any more in STAGING dataupload. 

json

Step 3:

Run curl call as below to get the ion(s) you have access.  You may get one or multiple ions based on your access permission.
"-i" option can be used to check the details of the communication if something go wrong.

curl -i 'https://sandbox-datamanager-api

curl -i -X PUT -H "Authorization:Basic d21.....uZT=="   -T "./test.pgp"  'https://staging-dataupload.wmda.info/api/v2v1/io/ION0999/'

Below are suggestions with more secure headers:

Suggestion 1:

...

ions/list' \
--header 'Authorization: Bearer ••••••' \
--header 'Content-Type: application/json'

For windows user, please use double quote instead single quote for the reqest:

curl -i -X PUT -H"https://sandbox-datamanager-api.wmda.info/api/v1/ions/list" \
--header "Authorization:Basic d21.....uZT== Bearer ••••••" \
-H-header "Content-Type: multipartapplication/form-data" --data-binary "@/path/to/file/ION-0999-D.gpg"  json"

Upload

Below is a suggestion for curl call based on the Data Manager swagger doc to upload a file for an ION. 

curl 'https://sandbox-datamanager-api"https://staging-dataupload.wmda.info/api/v2v1/io/ION0999/" -H "X-File-Name:ION-0999-D.gpg"

curl -i -X PUT -H "Authorization:Basic d21.....uZT==" -H "files/upload' \
--header 'Authorization: Bearer ••••••' \
--header 'Content-Type: multipart/form-data"' \
--data-binary "@c:form 'file=@/path/to/file/ION-0999-D.gpg"' \
--form "https://staging-dataupload.wmda.info/api/v2/io/ION0999/" -H "X-File-Name:ION-0999-D.gpg"

Suggestion 2:

For files that encrypted and identified as PGP/GPG MESSAGE (open the encrypted file and you can see this info), then -F and -H options also work. -F already included "Content-Type: multipart/form-data", so no need to provide it any more.

For binary file, extra header information will be added to the content of the file when file is uploaded, and then it can not be decrypted any more, so please use suggestion 1 to upload. 

curl -i -X PUT -H "Authorization:Basic d21.....uZT==" -F "file="@/path/to/file/ION-0999-D.gpg" "https://staging-dataupload.wmda.info/api/v2/io/ION0999/" -H "X-File-Name:ION-0999-D.gpg"

'ion="9999"' \
--form 'folder="upload"'

Something to be aware of for windows based cmd:

1. please use double quote instead single quote for the request.
2.  Line break "\" dose not work, so use the curl script in one line, or use specific break that works for your script.
3. Use "\" to escape the double quote for the value.
4. The path separator for the file dose not matter, both Linux used "/" and windows used  "\" works.

curl "https://sandbox-datamanager-api.wmda.info/api/v1/files/upload" --header "Authorization: Bearer ••••••" --header "Content-Type: multipart/form-data" --form "file=@C

curl -i -X PUT -H "Authorization:Basic d21.....uZT==" -F "file="@c:/path/to/file/ION-0999-D.gpg" "https://staging-dataupload.wmda.info/api/v2/io/ION0999/" -H "X-File-Name:ION-0999-D.gpg"

FAQ

Question : My report states : File could not be processed due to a file decryption error, when using the API upload? Why?

Answer :

-form "ion=\"9999\"" --form "folder=\"upload\""

Download

Download reports

You may use the API also to fetch your reports . Unfortunately, it is not one line cmd, and need And 2 steps are needed. 
For windows based cmd syntax, please see the example in the "Upload" part.

1.Fetch the files list and get the file names.

Use the following curl and endpoint file list API to fetch an array with filedescriptors: url should use double quote or no quote in windows env:of file list for the folder "report":

curl https://sandbox-datamanager-api

curl -H "Authorization:Basic d21.....uZT==" https://staging-dataupload.wmda.info/api/v2v1/fs/reports-ion0999/?children=f

You will get the result in xml format, and you can get json format,  use the cmd  below:

curl -H "Authorization:Basic d21.....uZT==" https://staging-dataupload.wmda.info/api/v2/fs/reports-ion0999/?children=f&format=jsonfiles/list \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer ••••••' \
--data '{
  "ion": 9999,
  "folder": "report"
}'

2.  Fetch all the reports or the one you need.

Note, you need to use "io" instead of "fs" in the path.

Use the file download API to download the file you needed. You can only download one file a time. 

curl https://sandbox-datamanager-api

curl -O GET -H "Authorization:Basic d21.....uZT==" 'https://staging-dataupload.wmda.info/api/v2v1/io/reports-ion0999/$filename' -o '$filename.txt'

In windows cmd, the -o dose not work, and the cmd is as below:

curl -H "Authorization:Basic d21.....uZT==" https://staging-dataupload.wmda.info/api/v2/io/reports-ion0999/$filename > $filename.txt

Download archived files

Download archived files will be the same as download reports. And 2 steps as above are needed.

The example URL for archive folder is as below:

https://staging-dataupload.wmda.info/api/v2/fs/archive-ion0999/?children=f

...

files/download \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer ••••••' \
--data '{
  "ion": 9999,
  "folder": "report",
  "fileName": "<filename>.txt"
}' \
--output <filename>.txt

Image Removed

Response should be 200:

Image Removed

Download full dataset

For those registries that are permitted to use the full dataset the statement below will fetch that. Please change the ION workspace to match your ION.You can download the file in the folder "download"

curl https://sandbox-datamanager-api

curl -O GET -H "Authorization:Basic .............." 'https://dataupload.wmda.info/api/v2v1/io/downloads/ION1804/bmdw4data.zip.gpg' -o 'wmda_data_v22.zip.gpg'

In windows cmd, the -o dose not work, and the cmd is as below:

curl -H "Authorization:Basic .............." "https://dataupload.wmda.info/api/v2/io/downloads/ION1804/bmdw4data.zip.gpg" >files/download \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer ••••••' \
--data '{
  "ion": 9999,
  "folder": "download",
  "fileName": "wmda_data_v22.zip.gpg"
} \
--output wmda_data_v22.zip.gpg 

We noticed sometimes the download maybe terminated because of some temp network limitation, and the "-C -" option can be used to continue the download.
-v is for more details of the track info

...

Guide for troubleshooting of API usage

Q: I get 401 error when using endpoint?

A: This means you do not have access to the endpoint or you did no provide the token correctly. Please double check whether you have extra space or miss space for the parameter "Authorization: Bearer <token>". And if you confirm that and still get 401, please contact support@wmda.info.

Q: I see 403 error for endpoint upload/download, what is next step?

A:  If you can call the endpoint of ions/list, and then get 403 error for upload/download, most possible is because the request body is invalid. Please double check whether any single or double quote or options are wrongly used.