In many ways, the strategic planning required to comply with EMA Policy 0070―a requirement to publish anonymized versions of clinical study reports (CSRs) and other submission documents―is similar to a secret agent’s mission: sponsors must keep the identities of people named or referred to in those documents top-secret while maintaining the document’s clinical utility. In this blog post, I’ll discuss how transparency and disclosure (T&D) regulations like Policy 0070 will affect your organization and how to facilitate compliance―a seeming “Mission Impossible.”
The rules of engagement: The rapidly evolving T&D landscape
Policy 0070 is not the only regulation or requirement driving publication of regulatory documents. EU Policy 0043 predates Policy 0070 and does not require placing documents in the public domain. Under Policy 0043 (The EU’s Freedom of Information Act), individuals can request documents utilized in the EMA decision making process. These documents are redacted for privacy information and potential company confidential information (CCI) by the EMA and provided to sponsors for feedback. Sponsors can request changes to the EMA’s proposed redactions. But in our experience, the EMA has been unwilling to accept most redactions requested by sponsors.
The Clinical Trial Regulation, another EU regulation, was published in April 2014 and is expected to go into effect in October 2018. Under this regulation, redacted CSRs and results will need to be posted on the EU Clinical Trial Registry.
The US Final Rule was published in September 2016 and applied to trials registered or results posted as of January 18, 2017. Under this regulation, protocols and statistical analysis plans (SAPs) must be posted with trial results on ClinicalTrials.gov. The NIH has not mandated any redaction requirements for these documents. They have stated that they do not intend to review or comment on the documents in a systematic fashion, but they will provide feedback if they feel the sponsors have over- or under-redacted.
Publication of medical research in scientific journals also places regulatory documents in the public domain. Many medical journals require sponsors to include a redacted protocol and SAP with any submitted publication. The timelines for these submissions do not necessarily correspond with regulatory timelines. The International Committee of Medical Journal Editors (ICMJE) transparency policy and the World Health Organization’s guidance on when trials should be submitted for publication in medical journals can be prior to regulatory submission. This leads to sponsors potentially creating multiple anonymized versions of the same documents.
Your mission, should you choose to accept it: Updates to the EMA Policy 0070 guidance
Updates to the EMA Policy 0070 Guidance occurring last December included, in our opinion, three significant changes.
The first change required using specific colors for redactions and their overlay text. Protected personal data (PPD) redactions now must be in blue with black overlay stating “PPD,” and CCI redactions must be in black with red overlay stating “CCI”. This requirement arose during the consultation process for several of our sponsors prior to the guidance being published. We were able to incorporate this change into our process prior to our sponsors submitting their final redacted versions.
Our redaction system utilizes a back end artificial intelligence tool and a front end redaction tool created for pharma regulatory anonymization. Technology cannot be altered at the flip of a switch. We would like to see EMA incorporate phase-in timeframes to requirements going forward, or industry will find it difficult to incorporate technology into their processes.
The second major change contained in the guidance was an increase in scope. Initially the only CSRs required for anonymization and submission under Policy 0070 were those included in the initial regulatory submission. However, the EMA updated its definition of “in-scope” to encompass CSRs referenced in the submission, but not necessarily submitted as a part of the package.
This definition change has complicated preparation and timelines. Sponsors don’t know which documents are in-scope until the consultation. Obviously, the core set of submitted documents will be in scope. Sponsors can start anonymizing these documents prior to initiating the process with the EMA. However for questionable documents, the sponsor must either prepare them in advance to protect timelines, but potentially waste money, OR wait until the EMA process commences at which time the timelines will be compressed. Neither way is ideal.
The third major change to the guidance restricts which listings can be removed. The new definition states that only per patient per visit listings in which all patients for the trial are included can be removed. All other listings must be redacted.
Keeping company confidential information under wraps
The justification table outlines all CCI requested for redaction. CCI is any information that would cause financial harm to the company should it be made public. The EMA can accept, reject or request clarification for each CCI redaction request. This document is not made public.
Sponsors struggle with the justification table for several reasons. First, the EMA requires a unique rationale for each proposed CCI redaction. The second complication is any information proposed as a CCI redaction cannot exist in the public domain. To date, we aren’t aware of any instances in which EMA has validated whether the CCI being requested for removal is in the public domain. However, the EMA has conducted this research under Policy 0043. In one instance, our client wanted to redact an investigator’s name. EMA researchers found that investigator’s name on a conference poster that was posted online. The EMA rejected this redaction request.
If the EMA allocates more resources to handle this regulation, they may take more time to validate CCI redactions. Additionally, submitting the justification table document requires confirming that any requested CCI redactions are not in the public domain.
Another complication with justification table development is incorporating the many individuals involved in the review process. Staff from the legal, clinical, and medical writing departments identify CCI. Meeting tight timelines with multiple stakeholders complicates this process.
In an October 2016 press conference, EMA personnel revealed that of the 260,000+ pages published within the first two Policy 0070 submissions, only two pages containing CCI redactions were accepted.
The last justification table complication concerns logistics. The most utilized software package for redacting documents is Adobe. Creating redactions requires Adobe Pro, not the free Adobe reader. Adobe Pro is not a standard software package in pharma companies. Given that multiple stakeholders across the organization may be involved in CCI review and that the stakeholders may change with each submission, sponsors typically do not want to invest in purchasing Adobe Pro and training staff on its redaction capabilities.
Streamlining CCI redaction
No secret agent can complete his mission without the right tech gadgets. After all, James Bond depends heavily on his Q branch issued poison pens, X-ray enabled eyeglasses, and exploding keychains to defeat various villains. For our T&D “agents”, we have created a solution in our artificial intelligence system. Stakeholders review and highlight the CCI with a specific color. Then, our system converts the highlighted text to redactions. Additionally, sponsors can add the justification and other required text in the comment associated with the highlighted proposed CCI redactions. Our system pulls out this text to populate a draft justification table.
Keeping personal protected data “classified”
While the justification table lists CCI requested for redaction, the anonymization report provides an overall philosophy towards anonymizing the submission package.
Additionally, this document describes the sponsor’s efforts to maintain data utility. The EMA initially stated in stakeholder meetings that they only intended to validate that the philosophies in the anonymization report accurately represent the proposed redactions. However in our experience, the EMA has been reviewing select documents in great detail within each submission.
The options to anonymize documents include:
- Masking (redaction) removes personal data and replaces it with black boxes or, in the case of the EMA, blue boxes.
- Randomization techniques, such as noise addition in which numbers, like height, might be randomly adjusted up or down within a range.
- Generalization techniques, such as aggregation, in which an age of 27 might be changed to 20-30.
These options have different impacts on the risk of re-identification and data utility.
Most sponsors prefer a redaction solution. To date, six of the eight published submission packets under Policy 0070 utilized redaction. The other two had few redactions for study administrators and no patient PPD.
Preventing study administrator and patient re-identification
What is involved in the anonymizing submission documents and removing CCI? First, let’s understand anonymization. Submission documents contain personal data. Anonymizing data requires stripping it of sufficient elements such that the data subject can no longer be identified by all reasonable means. To properly anonymize the submission documents, sponsors have a few options. The first option is to ensure that re-identification cannot occur using linkability, singling out, or inference. Completely preventing linkability, singling out and inference is extremely difficult and significantly impacts data utility. The EMA’s alternative method allows sponsors to assess the risk of their anonymization methods. To date, no sponsor has utilized the technique of fully preventing against linkability, singling out and inference. We do not believe that the EMA wants sponsors to utilize this technique because the ultimate result would severely limit data utility.
The anonymization report has complexities as well. Sponsors must choose either a qualitative or quantitative method to assess the risk of re-identification for each submission. Then, they determine their desired risk threshold. Corresponding with this decision, sponsors must consider different attacker scenarios and determine which ones are most appropriate. Next, they assess the risk of re-identification against the pre-determined threshold and address how data utility was maintained. Lastly, sponsors make global redaction decisions as well as decisions specific to each submission and each trial.
Our clients employ a range of risk assessment approaches. Some sponsors consider all information associated with an individual to pose a high risk of re-identification. Others place a much lower risk with various categories of information associated with individuals. Some sponsors are only willing to share age demographic information for patients below 89. Other sponsors will share almost all demographic information. The real rather than theoretical risk actually remains to be seen. To our knowledge, only research-based attacks have been committed against health information to broaden our knowledge of re-identification risks. No sponsors have had a specific patient or study administrator face a negative impact due to their personal information being shared, their identity discovered, and a law suit resulting. The extent to which sponsors feel that this is possible drives their philosophy towards redaction or retention of personal information.
The flow of drug development information: Mitigating the risk of accidental disclosure
The entire T&D process is connected. Initial protocol development or later CSR development often impacts not only down-stream redaction efforts, but also plain language summary creation and registry protocol/results postings.
Some sponsors are just starting to create centralized T&D teams. While others have their T&D team in place, but are working to incorporate their T&D team’s knowledge in upstream activities.
Sponsor controlled sources of clinical trial data
Sponsors influence many types of regulatory information in the public domain―the company website, scientific publications, public registries like ClinicalTrials.gov, and more.
The growth of sources of trial information not controlled by the sponsor is an emerging industry trend. Various organizations are making regulatory information public either through press campaigns, aggregating regulatory information, or assessing individual organizations compliance with regulatory and ethical transparency requirements. Some organizations perform these assessments with sponsor input while others are seemingly adversarial and do not provide sponsors with an opportunity to explain or become compliant.
For instance, the COMpare project assesses public clinical trial outcome reporting. COMPare reviews protocol information in public registries and then compares the intended outcome measures with the results reported in those same registries and in medical journals. The COMPare project does not give sponsors an opportunity to speak to the findings. Rather the COMPare project researchers send letters to the medical journals explaining the inconsistencies found.
As a different example, the Good Pharma Scorecard assesses compliance to both regulatory and ethical standards for trial registration (protocols and results) and publication. This assessment― conducted by BioEthics International― does allow sponsors to review their results prior to publishing the results. Sponsors can also provide feedback to BioEthics International on the Scorecard’s findings.
The risks of non-compliance with T&D regulations
Running afoul of T&D regulatory and ethical guidelines has numerous serious business implications. Any regulatory non-compliance can cause increased scrutiny from regulatory agencies. Under new regulations, fines are a greater possibility for those not meeting these mandates. In addition to fines, the EU is maintaining a public list of all sponsors found to be non-compliant with Policy 0070. Sponsors who miss EMA Policy 0070 deadlines could also be placed on this list.
Besides the regulatory implications of non-compliance, there are also market implications to non-compliance. Negative assessments from external groups like COMPare and Good Pharma Scorecard can harm a sponsor’s brand and reputation. Lastly, mishandling PPD anonymization increases the chance of potential law suits from patients and/or study administrators.
We have the regulatory science expertise and technology to help T&D agents succeed in their critical missions. To learn more about the challenges of meeting ethical and regulatory T&D requirements, please watch a recent webinar I gave on this topic.