Blog

We have finished a new release of the Search & Match Service, where a new infrastructure is built to process larger data files.

The release includes:

• Donor report: In donor search results, you can find a full donor report when you are expanding the donor details. Additional data are available if the listing organization submits these data to WMDA.
• Cord Blood Unit report: In the cord blood search results, you can find a cord blood unit report when you are expanding the cord blood unit details. Additional data are available if the listing organisation submits these data to WMDA. 
• Accreditation status of cord blood banks: You can see if a cord blood bank is accredited by FACT or AABB.

For more information on the new features and the usage, visit our userguide.

Keep in mind the following:

• Search results can be different than before. Because the validation of the submitted donor and cord blood data is done stricter, it may be that donors and cord blood units could be rejected during the new file processing and not be available anymore on your match list.
• No data shown in all fields of the donor/cord blood unit reports. From today till June 1, 2018 organisations will be switched to the new infrastructure. This means that you will see full data sets from organisations that already switched, and minimal data fields from other organizations. On the following link you can see the transition dates for each organisation. On the following link you see the number of donors and cord blood units visible for each organization. 
• Donor / Cord blood unit report error: Occasionally, you might get an error when opening the report. We are working on a fix, the reason is probably the caching of the web application. If you experience this problem, please log-out and login again.
• Match runs appear to be slow since this release. All searches need to be repeated resulting in more load on the servers. This will last for approximately 48 hours.

If you are experiencing any problems after the maintenance, please contact us through support@wmda.info.

Thank you for your patience and we hope that the new release will improve search activities for your patients.

We are very delighted that we received already many test XML files in our staging environment from our listing organisations. From tomorrow on, we will start again with reviewing the files and send you feedback for improvements or next steps. Because we have to review many files, it might take somewhat longer before you receive feedback. We ask your understanding for this situation.

If you haven't uploaded your file in the staging environment, please do this as soon as possible. If you and your organisation are not able to upload a test file on short notice, please contact us as soon as possible by sending an email to support@wmda.info . We can then look together to options to support you and your organisation in this transition to XML submission. Also for all questions or other assistance, please do not hesitate to contact us.

We will also start tomorrow our pilot testing period with a selection of 8 listing organisations as well as their affiliated organisations. These tests will be performed in our production system to have an as much as possible comparable situation of the live system and processing procedures. This testing period will take two weeks. Results will be shared after the closure of this period.

Days till switch dot20->XML for LIVE system

UPDATE

The countdown has begun to the implementation on April 8^th of the new data submission process for listing donors and cord blood units in the Search & Match Service (BMDW).

We elaborated the steps and the activities necessary to allow acceptance of files in the new XML format from your organisation. 

Timeline

We have updated the timetable to illustrate the key dates and activities for the migration from the old data format to the new XML format .

Please visit the Updated Transition information & Timelines page for more information.

The last couple of days, you might have received a processing report with more HLA errors than expected which resulted in more rejected records as well. Some organisations didn't even receive their processing report as the email was too large to receive.

The HLA reference tables were not properly created last weekend or only partially created, which resulted in wrong tables and lot of non-recognized HLA in the processed files. The tables have been updated again and we started to manually process again all files that have been received after the onset of these issues. You will therefore receive a new processing report from a file you have been sending some days ago.

After all files have been processed again, the data will be sent to the matching service OptiMatch to get the most recent data in the Search & Match Service. After that, we will restart the automatic processing and we hope it will be functioning as expected.

We are sorry for the inconvenience it might have caused. If you keep having problems or experience strange or unexpected errors in your processing reports, please inform us as soon as possible.

As you might know, WMDA is striving to have more and high quality data and insight available to inform effective and efficient patient/donor matching. In August, a letter has been sent out to inform you about the change to move to a new Extensible Markup Language (XML) file format.

Not all organisations are ready to go to the XML format and asked us to delay the transition to the first quarter of 2018.

The infrastructure of accepting and validating files is nearly ready. At the moment, we are working on performance and penetration tests of the system. This is important in order to ensure the security and authenticity of all the data. However, the testing period requires more time than initially anticipated.

Based on these observations, the decision has been taken to extend the transition date to move to XML for data submission to April 1, 2018.

Find below the revised timelines in order to migrate from the old data format to the new (XML) data format.

We published the revised timelines for transition to XML file data submission to BMDW here. We also sent this information by email to the CEOs of all listing organisations and by email to the people who are responsible for the data submission of the DOT20 files and all test users. You can also take a look at a copy of the letter here.

If you are involved in the data submission to BMDW for your organisation, please make sure that you take notice of this information. If you haven't started to implement the changes to create an XML file in your organisation, please start scheduling the development as soon as possible.

On November 9^th, you can also meet the team at the WMDA Fall meeting in Minneapolis. They are available from 09:00 AM till 11:00 AM in Director’s Row 3 in the Hilton Hotel for answering any questions.

Our developers resolved the problems with file processing and reports are generated as usual. You can upload your files again on the staging BMDW data upload service for testing purposes. Please keep in mind that we still have the limit of 1000 records per file. We hope to remove this restriction next week.

Thank you for your patience.

The developers have finished the update of our Search & Match Service and the matching service OptiMatch.

Since the initial launch back in November 2016, the BMDW Search & Match Service has constantly been enriched with enhancements requested by BMDW users. These enhancements are to ensure that BMDW remains very intuitive and easy to navigate, whilst providing BMDW users with all the functions they need to do their day-to-day search requests for patients in need of a transplant.

The latest release to production includes the following new features:

• Additional donor fields in search results when expanding the donor details: Status and Last contact date are added.
• Additional CBU fields in search results when expanding the CBU details: Volume, Status, amount of Mononuclear cells (MN), Viability and number of attached segments are added.
• Optional grouping/sorting: standard search results are grouped/sorted by number of matching values on allele level and within the groups sorted by probability. When you have many potential matched donors with low probabilities, the standard grouping/sorting prevented you from seeing the mismatched donors which can be better options in these cases. Therefore, we now introduce an alternative grouping/sorting option where search results can be grouped together and sorted by the sum of probabilities of potential matched and mismatched donors when selecting one or two mismatches. For example: When haplotype frequencies are selected and 10 loci are considered (n=10) with 1 mismatch, and sorted by sum of probabilities – include 0 and 1 mismatches, the sum values of the probabilities for potential 10/10 matched donors and 9/10 potential matched donors are used for primary sorting. So if potential 9/10 matched donors are available with higher probabilities than 10/10 potential matched donors, the 9/10 donors will appear on top of your match list with header +9/10 (potential allele matches).
• Introduction of new haplotype frequency sets to make probability calculations more accurate: This feature is not directly visible for you as a user, but the probabilities that are calculated and used for matching your patient with donors will be more accurate than before when we only used 1 haplotype frequency set for all probability calculations. The new frequency sets have been calculated and based upon geographical and available data of the donors of an organisation.

 For more information on the new features and the usage, please visit our userguide.

Thank you for your patience and we hope that the new features will improve search activities for your patients.

Unfortunately, we are experiencing problems on our staging data upload service. Most files that have been uploaded since last Friday afternoon have not been processed completely and also no processing reports have been generated. In your archive folder, the uploaded file keeps showing the status “processing” even though the file processing already stopped.
Our developers are trying to identify why the processing of the files stops prematurely and no reports are created and we hope that they can resolve it very soon.

We will notify you when the file processing is working properly again.

We are sorry for the inconvenience this might have caused.

We previously published instructions how you can manually upload your XML files with data to the BMDW data upload service.

We now published also instructions for automatic data submission to the service via REST API mechanisms. You can find the instructions on the following page: How to automate data upload/download with the REST API. Please be aware that currently the instructions point to the staging server/ environment of the data upload service. Once we switch to the production environment, you need to make changes to direct to the correct server.

In the XSD files, you can already find a lot of field requirements and we also validate your data according to these definitions. However, BMDW also has set some additional rules for validation. These rules are called BMDW business validation rules. You can now find these rules in the tab  'Data Validation: Errors and warnings in processing report' on the page with Search & Match Service Data submission information.

If you have any questions, please don't hesitate to contact us by sending an email to support@bmdw.org 

Our developers worked very hard to implement all our requirements and validation rules. The first version of the pipeline that can process all the data is now ready and we are currently performing many tests to identify as many problems before this procedure will be tested with the three large organization, NMDP/Be the Match, ZKRD and Anthony Nolan.

Until further notice, we would like to ask you to only upload test files with a maximum of 1000 records. Larger files will negatively influence the velocity of our testing period.

Once we will be focusing more on performance during processing of larger files, we will approach you again to help with uploading of larger files.

Thank you for your cooperation and if you experience any problems or receive error/warning messages that you do not understand or didn’t expect, please send an email to support@bmdw.org

The last weeks, the WMDA office tried to contact all BMDW listing organisations that 

• have NOT responded to the survey "Moving to XML"  last May
• have NOT requested an account for our BMDW data upload service for uploading their test XML file
• provided a test XML file before, but the quality/ structure of the file was NOT good enough

Unfortunately, we were not able to reach all organisations.

If you haven't been in contact with the WMDA office before on your status of moving to the XML format for data submission to BMDW, please contact us at your earliest convenience by sending an email to support@bmdw.org.

You can also contact us if you would like to get explanation on the requirements for an XML file or support from our ICT specialists. All information and requirements are described on the BMDW data submission information pages.

The end of our development work is approaching!!

The last month, BMDW and our developers worked very hard to implement all the validation rules that are necessary to properly validate all data fields that can be provided in the XML files.

The big testing phase will start tomorrow. We will perform first many tests ourselves. During this period, you can also still upload your test files. We expect that the reports that will be generated upon uploading your file, will be much more accurate than before. However, if you experience any problems or find strange or unexpected error/warning messages, please contact us by sending an email to support@bmdw.org and we can determine together whether a mistake was made during implementation of a certain validation rule or that there was an inconsistency in your data file.

Together we can test and improve our new data processing pipeline!!

From October on, we will start testing with three larger organisations: Be the Match/NMDP, ZKRD, and Anthony Nolan. If all tests succeed and the system performs as expected, we will start the production system at the beginning of November.

If you have created also a test XML file and want to join the BMDW data upload service, please send us email to support@bmdw.org to get your account details.

We have been testing XML files from several organisations for the last months and we would like to share the common mistakes found in the files.

On file level:

• File name incorrect: The correct file naming is ION-1234-D.xml.pgp or ION-1234-C.xml.pgp
• Wrong public key used for encryption: For the XML files you have to use the new BMDW public key.
• File is signed: The file should not be signed; only encrypted with the BMDW public key.
• Encoding in the top line: we recommend to add the encoding in the top line: <?xml version=”1.0” encoding=”utf-8”?>  

• We would recommend to create the XML file not as one line of data. For better readability, you should use line endings and indentations.

On record level:

• CREATION_TIIME and SNAPSHOT_TIME: These values should look like 2016-08-23T13:16:48Z; so no fractions of seconds and a 'Z' to reflect UTC time.
• SCHEMA_VERSION is missing: This element was added in the last version of the XSD files. We prefer to add this already.
• BIRTH_DATE: Correct format should be YYYY-MM-DD and this is a mandatory field.
• GRID: GRID is only applicable for DONORs and nor for CBUs.
• POOL: POOL is no longer an element within <DONOR> or <CBU>, but should be within <INVENTORIES>
• HLA values: DNA values should not be added to the <SER> (serological) value fields.
• Order of the elements: The data should be delivered in the correct order of the elements. Otherwise you will receive messages that a certain elements is not expected.
• VOL versus VOL_FRZN: VOL represent the volume of a cord at time of the collection. VOL_FRZN is the volume after processing and before freezing. VOL_FRZN is comparable with NVC in DOT20 files.
• TNC versus TNC_FRZN: TNC represent the total number of nucleated cells of a cord at time of the collection. TNC_FRZN is the number of cells after processing and before freezing. TNC_FRZN is comparable with TNC in DOT20 files. Also be aware that the units of these elements are different in DOT20 and XML. For DOT20 for example you used 300 and this was in units E7; for XML you should use the real number, e.g. 3.0E9
• CD34PC versus CD34PC_FRZN: CD34PC represent the number of CD34+ cells of a cord at time of the collection. CD34PC_FRZN is the number of CD34+ cells after processing and before freezing. CD34PC_FRZN is comparable with CD34P in DOT20 files. Also be aware here that the units are different.

If you haven't started to create your XML test file, please take the above mentioned mistakes into account. If you have any questions, please contact us by sending an email to support@bmdw.org 

Is your organisation ready to implement the new BMDW file format with extended data on November 1, 2017? Many organisations have sent their test files and are ready to go.

However, some organisations have not contacted the WMDA to inform us when they will sent the test file in the new XML format.

Let us know when you are able to send the test file, you can read all details here.

If you are not able to send a test file before November 1, 2017 contact our ICT specialists at support@bmdw.org. During an hour free consultation, they can investigate the likely effort and costs that are required to implement the changes in your organisation.