⚠Due to planned maintenance you will experience short (<30 min) downtime between 08:00 - 08:30 CET.
Purpose
Data upload (and download) can be done using various mechanisms :
- Web-based interface at https://staging-dataupload.wmda.info (please note the URL for production is dataupload.wmda.info )
- Desktop applications and Sync Clients for Mac, Linux, Windows.
- REST API, using commands like cURL, or libraries like Restsharp
This document describes the data upload mechanism via the REST API method, where users can upload a PGP encrypted XML file.
Important note on security
The examples below are for educational purposes: WMDA
Please do read https://ec.haxx.se/usingcurl-netrc.html on the use of passwords and usernames if you plan to automate these procedures.
# example use: curl --netrc-file mycredentialsfile https://xyz.com # the mycredentialsfile contains : machine <xyz.com> login <johndoe@xyz.com> password <uf9873o^9ufwa>
WMDA dataupload production plan to do a maintenance and update on 8th, Nov, 2023, which will introduce more security control.
From staging update we know this will impact the current method used in the REST API.
Below you can find some suggestions to improve current script to upload.
Using the REST API
To test the API you may use a tool like Restlet, a plugin for Chrome browsers to test API calls before you deploy. The server responds with a HTTP code 200 in when succesful, but the response body may contain detailed error messages.
In case the upload succeeded, an XML response with meta information of the upload is generated. You may use this for own logging purposes. For more info you can read the full API specification.
URL | |
---|---|
SERVER URL | https://staging-dataupload.wmda.info |
URI | /api/v2/io/ION1234/ Explanations:
|
METHOD | PUT |
Full URL path example | For donor for organization with ION-1234, the full path is as below: https://staging-dataupload.wmda.info/api/v2/io/ION1234/ION-1234-D.gpg |
HEADER | Authorization and Content-Type are required, the other ones are optional |
Authorization | basic |
Content-Type | application/x-www-form-urlencoded Currently, suggest to use -F (--form) which will by default have a Content-Type multipart/form-data, and can provide the file path. |
X-Rename-If-Exists | true |
- Server name should be https://staging-dataupload.wmda.info
- For production you will need https://dataupload.wmda.info
SAMPLE Request
First test the connection by using the right path and fetching the metadata : it should result in a response 200 OK.
- Look carefully at the path : here we use ion0999, that should be replaced by your ION.
- You may notice that we add the filename to the path: depending on the library you may need to add that
Upload
Now we know the path is correct and we can do an upload using cURL:
This is a old way to deploy, currently it is still available in production till production maintenance before Nov, 7th, 2023. But not available any more in staging dataupload.
Old curl for linux
curl -i -X PUT -H "Authorization:Basic d21.....uZT==" -T "./test.pgp" 'https://staging-dataupload.wmda.info/api/v2/io/ION0999/'
Below are suggested:
Curl for linux
curl -i -X PUT -H "Authorization:Basic d21.....uZT==" -F "file=@/path/to/file/ION-0999-D.gpg" https://staging-dataupload.wmda.info/api/v2/io/ION0999/ -H "X-File-Name:ION-0999-D.gpg"
Curl for windows
curl -i -X PUT -H "Authorization:Basic d21.....uZT==" -F "file=@c:/path/to/file/ION-0999-D.gpg" https://staging-dataupload.wmda.info/api/v2/io/ION0999/ -H "X-File-Name:ION-0999-D.gpg"
In case you need a testfile you may generate one (in windows) issueing "FSUtil File CreateNew temp.pgp 0x2500000"
FAQ
Question : My report states : File could not be processed due to a file decryption error, when using the API
Answer :
Download
You may use the API also to fetch your reports . Unfortunately, it is not one line cmd, and need 2 steps.
1.Fetch the files list and get the file names.
Use the following curl and endpoint to fetch an array with filedescriptors:
curl -H "Authorization:Basic d21.....uZT==" 'https://staging-dataupload.wmda.info/api/v2/fs/reports-ion0999/?children=f'
You will get the result in xml format, and you can get json format, use the cmd below:
curl -H "Authorization:Basic d21.....uZT==" 'https://staging-dataupload.wmda.info/api/v2/fs/reports-ion0999/?children=f&format=json'
2. Fetch all the reports or the one you need.
Note, you need to use "io" instead of "fs" in the path.
curl -O GET -H "Authorization:Basic d21.....uZT==" 'https://staging-dataupload.wmda.info/api/v2/io/reports-ion0999/$filename' -o '$filename.txt'
In RESTLET :
Click on Code to get raw CURL syntax:
Response should be 200:
Download full dataset
For those registries that are permitted to use the full dataset the statement below will fetch that. Please change the ION workspace to match your ION.
curl -O GET -H "Authorization:Basic .............." 'https://dataupload.wmda.info/api/v2/io/downloads/ION1804/bmdw4data.zip.gpg' -o 'bmdw4data.zip.gpg'