You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 31 Next »

Purpose

Data upload (and download) can be done using various mechanisms : 

This document describes the data upload mechanism via the REST API method, where users can upload a PGP encrypted XML file.

Important note on security

The examples below are for educational purposes:

Please do read https://ec.haxx.se/usingcurl-netrc.html on the use of passwords and usernames if you plan to automate these procedures.


# example use:
curl --netrc-file mycredentialsfile https://xyz.com

# the mycredentialsfile contains :
machine <xyz.com> login <johndoe@xyz.com> password <uf9873o^9ufwa>
      

    



Using the REST API

To test the API you may use a tool like Restlet, a plugin for Chrome browsers to test API calls before you deploy. The server responds with a HTTP code 200 in when succesful, but the response body may contain detailed error messages.

In case the upload succeeded, an XML response with meta information of the upload is generated. You may use this for own logging purposes. For more info you can read the full API specification.


URL
SERVER URLhttps://staging-dataupload.wmda.info
URI

/api/v2/io/ION0999/

Please refer to the file naming convention.

This  is stated as : For XML format:The files are marked with ION numbers ( ION-1234-D or ION-1234-C)

The ION number is an unique number of an organisation sending the file.

The D give the information that it is a donor file and the C give the information that it is a cord blood file

METHODPUT
HEADERonly Authorization is required, the other ones are optional
Authorization

basic

Content-Typeapplication/x-www-form-urlencoded
X-Rename-If-Existstrue

SAMPLE Request

First test the connection by using the right path and fetching the metadata : it should result in a response 200 OK.

  • Look carefully at the path : here we use ion0999, that should be replaced by your ION.
  • You may notice that we add the filename to the path: depending on the library you may need to add that



Upload

Now we know the path is correct and we can do an upload using cURL: 


curl -X PUT -H "Authorization:Basic d21.....uZT=="   -T "./test.pgp"  'https://staging-dataupload.wmda.info/api/v2/io/ION0999/'


In case you need a testfile you may generate one (in windows) issueing "FSUtil File CreateNew temp.pgp 0x2500000"



FAQ


Question : My report states : File could not be processed due to a file decryption error, when using the API

Answer :

The Dataupload's API endpoint /io streams file(s) directly to the workspace. We have detected that premature file pickups may occur and attempt to move/delete files while content is still being appended or processed. We have taken actions to reduce this behaviour by decreasing the frequency for file pickup which in turn does lower the chance for premature pick. We have also implemented gpg error validation to check integrity based on its messages.

Note: Files are picked from the workspaces for processing every 10 minutes.

Some key messages:

  • Known internal gpg messages for corrupted files during API uploads: "invalid packet", "invalid encoding" and "failed". 
  • Report message that uploader users would receive in case of corrupted gpg file upload:   File could not be processed due to a file decryption error. Please make sure file is properly encrypted. 

For more information please contact WMDA support team.



Download

You may use the API also to fetch your reports . Unfortunately, it is not one line cmd, and need 2 steps. 

1.Fetch the files list and get the file names.

Use the following curl and endpoint to fetch an array with filedescriptors:

curl -H "Authorization:Basic d21.....uZT==" 'https://staging-dataupload.wmda.info/api/v2/fs/reports-ion0999/?children=f'

You will get the result in xml format, and you can get json format,  use the cmd  below:

curl -H "Authorization:Basic d21.....uZT==" 'https://staging-dataupload.wmda.info/api/v2/fs/reports-ion0999/?children=f&format=json'

2.  Fetch all the reports or the one you need.

Note, you need to use "io" instead of "fs" in the path.

curl -O GET -H "Authorization:Basic d21.....uZT==" 'https://staging-dataupload.wmda.info/api/v2/io/reports-ion0999/$filename'



In RESTLET :


                       

Click on Code to get raw CURL syntax:


Response should be 200:


Download full dataset

For those registries that are permitted to use the full dataset the statement below will fetch that. Please change the ION workspace to match your ION.

curl -O GET -H "Authorization:Basic .............." 'https://dataupload.wmda.info/api/v2/io/downloads/ION1804/bmdw4data.zip.gpg' -o 'bmdw4data.zip.gpg'





  • No labels