Versions Compared

Old Version 51

changes.mady.by.user Zhihong He

Saved on

compared with

New Version Current

changes.mady.by.user Zhihong He

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Expand

The Dataupload's API endpoint /io streams file(s) directly to the workspace. We have detected that premature file pickups may occur and attempt to move/delete files while content is still being appended or processed. We have taken actions to reduce this behaviour by decreasing the frequency for file pickup which in turn does lower the chance for premature pick. We have also implemented gpg error validation to check integrity based on its messages.

Note: Files are picked from the workspaces for processing every 10 minutes.

Some key messages:

  • Known internal gpg messages for corrupted files during API uploads: "invalid packet", "invalid encoding" and "failed". 
  • Report message that uploader users would receive in case of corrupted gpg file upload:   File could not be processed due to a file decryption error. Please make sure file is properly encrypted. 

For more information please contact WMDA support team.



Download

Download reports

You may use the API also to fetch your reports . Unfortunately, it is not one line cmd, and need 2 steps. 

...

Use the following curl and endpoint to fetch an array with filedescriptors: url should use double quote or no quote in windows env:

Code Block
curl -H "Authorization:Basic d21.....uZT==" 'https://staging-dataupload.wmda.info/api/v2/fs/reports-ion0999/?children=f'

You will get the result in xml format, and you can get json format,  use the cmd  below:

Code Block
curl -H "Authorization:Basic d21.....uZT==" 'https://staging-dataupload.wmda.info/api/v2/fs/reports-ion0999/?children=f&format=json'

2.  Fetch all the reports or the one you need.

...

Code Block
curl -O GET -H "Authorization:Basic d21.....uZT==" 'https://staging-dataupload.wmda.info/api/v2/io/reports-ion0999/$filename' -o '$filename.txt'

In windows cmd, the -o dose not work, and the cmd is as below:

Code Block
curl -H "Authorization:Basic d21.....uZT==" https://staging-dataupload.wmda.info/api/v2/io/reports-ion0999/$filename > $filename.txt

Download archived files

Download archived files will be the same as download reports. And 2 steps as above are needed.

The example URL for archive folder is as below:

Code Block
https://staging-dataupload.wmda.info/api/v2/fs/archive-ion0999/?children=f


In RESTLET :


                       

Click on Code to get raw CURL syntax:

...

Code Block
curl -O GET -H "Authorization:Basic .............." 'https://dataupload.wmda.info/api/v2/io/downloads/ION1804/bmdw4data.zip.gpg' -o 'wmda_data_v22.zip.gpg'

In windows cmd, the -o dose not work, and the cmd is as below:

Code Block
curl -H "Authorization:Basic .............." "https://dataupload.wmda.info/api/v2/io/downloads/ION1804/bmdw4data.zip.gpg" > wmda_data_v22.zip.gpg


We noticed sometimes the download maybe terminated because of some temp network limitation, and the "-C -" option can be used to continue the download.
-v is for more details of the track info

Code Block
curl -v -H "Authorization:Basic .............." -o 'wmda_data_v22.zip.gpg' -C - 'https://dataupload.wmda.info/api/v2/io/downloads/ION1804/wmda_data_v22.zip.gpg'