In 2016, TACT was endorsed by International Rare Diseases Research Consortium (IRDiRC) as a recognised resource. The vision of IRDiRC is to enable all people living with a rare disease to receive an accurate diagnosis, care, available therapy within one year of coming to medical attention and to develop methodologies to assess the impact of diagnoses and therapies on rare disease patients. IRDiRC aims to contribute to the development of 1000 new approved therapies for rare diseases, focussing on diseases without approved options.

IRDiRC gives recognition to resources which contribute to this aim by accelerating the pace of translating discoveries into clinical applications. TACT contributes to the aim and vision of IRDiRC by giving comprehensive expert advice to researchers and industry on drug development programs for rare neuromuscular diseases.

At time of production of this toolkit, 81 applications for advice have been reviewed by TACT in disease areas including:

• Duchenne muscular dystrophy
• Becker muscular dystrophy
• Spinal muscular atrophy
• Limb girdle muscular dystrophy
• Myotonic dystrophy
• Congenital myopathies
• Pompe disease
• Facioscapulohumeral muscular dystrophy
• X-linked myotubular myopathy
• Mitochondrial disease

It is important to emphasise that the success of TACT should not just be measured by those reviewed programs still in development but by the systematic approach to identify the realistic prospects for a particular compound which, positive or negative, can help patients, the field and the applicant.

Members of TACT have produced a TACT 10th anniversary paper which demonstrates the impact of TACT, advice given and the progress made by previous applicants. In preparation for this paper, a survey was sent out to past applicants who reported that TACT had helped them to prepare for discussions with regulators, inform study design including selection of endpoints and go/no- go decisions, enrolment and dosing decisions. Previous applicants have shared how they have benefited from TACT’s multidisciplinary approach and from having a wide array of experts available during one meeting:

“The expertise TACT assembled in one room was unparalleled and greatly appreciated. The review provided us with an outside consensus regarding how to proceed in this uncharted territory as the first company working in this disease (Congenital Muscular Dystrophy).” – Jodi Wolff, Santhera

“TACT provided a very accurate review of the protocol and useful suggestions on both the preclinical and clinical parts.” – Emilio Clementi, University of Milano

This diagram illustrates the main areas of expertise that are required to form a TACT review panel.

Although there will be common areas of expertise required to form an ACT review panel in a particular rare disease area, your core committee should form and agree consensus on what type of expertise will be required, as a minimum, for each review meeting. The core committee attend every meeting so it is important that the committee includes key areas of expertise required for each meeting.

The type of expertise required can differ between applications, which is why we recommend that you form a large and varied extended review committee. Members of the extended committee do not have to attend every meeting, they can be called upon to attend any meeting depending on the applications to be reviewed and the expertise required.

It is essential to the success of an ACT that your new reviewers are adequately prepared to review an application for the first time. This will help to ensure that the applicant receives the best advice available to them. Each new reviewer should attend a short briefing before the review meeting begins.

We have provided you with detailed guidance on the different tasks a reviewer is expected to carry out. To complement this guidance, the next few pages include representative views from TACT members, including how they approach a review, common pitfalls and what the benefits are for becoming a reviewer.

1. Pre-clinical

   Approach to reviewing an application:

   • Assess whether wild types were included to measure the deficit and control of a treatment.
   • Assess if the work has been carried out in two separate laboratories.
   • Have standardized operating procedures been applied?
   • Has the applicant assessed multiple parameters, with a primary endpoint pre-defined?
   • Have proof of concept and further preclinical studies been carried out (e.g. on dosing, timing, regimen etc.)?

   
   Common pitfalls to look out for:

   • No wild type controls used.
   • Studies carried out in one laboratory only.
   • Only few outcome measures included (sometimes handpicked).
   • Only proof of concept shown and no pre-clinical optimization.

2. Regulatory

   Approach to reviewing an application:

   • Has the applicant had contact with regulators?
   • What endpoints will the applicants use in the trial? Are the end-points acceptable to regulators?
   • Is there enough data to design a trial?

   
   Common pitfalls to look out for:

   • If the applicant has had no contact with regulators.

3. Clinical

   Approach to reviewing an application:

   • Evaluate the clinical trial eligibility criteria.
   • Assess the suitability of the proposed outcome measures.
   • Review the overall clinical trial design.

   
   Common pitfalls to look out for:

   • Insufficient sample size.
   • Inappropriate outcome measures.
   • Lack of biological rationale or supporting preclinical data.

4. Physiotherapy

   Approach to reviewing an application:

   • Assess what the mechanism of action of the drug is. This is crucial to understanding if the right outcome measures will be used.
   • Does the applicant understand the disease area and the nature of that condition, such as its complexity and components of progression (muscle weakness, respiratory, learning issues etc.)?
   • Is the mechanism of action reflected in their choice of inclusion and exclusion criteria? Do these make sense both clinically and practically or will they make recruitment difficult?
   • Are the clinical endpoints feasible, fatiguing, appropriate and are they measurable?
   • Are the outcome measures valid and reliable?
   • What is the frequency of assessments?
   • Is the length of the clinical trial appropriate for the mechanism of action?
   • Have they included Patient Reported Outcome Measures (PROMS) and Quality of Life (QoL) measures that are appropriate?

   
   Common pitfalls to look out for:

   • Poor understanding of outcome measures.
   • Not linking outcome measures with mechanism of action.
   • Too many outcome measures proposed.
   • Problematic inclusion and exclusion criteria.
   • No draft schedule of events.

5. Statistics

   Approach to reviewing an application:

   Pre-clinical:

   • If only pre-clinical data is provided, assess that the data was analysed and used properly. It is crucial that the applicant provides summary statistics and the statistical methods used, rather than only text statements of the findings, which cannot be critically reviewed.

   Clinical:

   • If preliminary clinical data is provided, assess if the data is adequately described in order to review the statistical methods used and the conclusion made.
   • Has a power or sample size analysis been carried out on the proposed clinical study, to ensure that there will be adequate data to test the hypotheses proposed?
   • If provided, review the statistical analysis plan for the proposed clinical study. Critically thinking about and writing the analysis plan helps to define the primary hypothesis of the study.

   
   Common pitfalls to look out for:

   • Inadequate or incorrect analysis of pre-clinical data. Pre-clinical data can be rather complex in certain experiments, however, the statistical analysis of that data is often very rudimentary or incorrect.
   • Including the conclusions from non-published pre-clinical data but not providing details of the data within the application.
   • Proposing a very complex clinical study (i.e. multiple drug doses, multiple time points, multiple endpoints) without consideration of how complex the statistical analysis will be, often leading to a proposed analysis that cannot test the primary hypothesis proposed.
   • Lack of appreciation of the sample size required for complicated studies. This is particularly true if the proposed study is pre-clinical in nature. Often pre-clinical studies are planned with a sample size of 10 or smaller, without regard to whether or not that sample size is adequate.
   • Lack of appreciation for the difficulty in patient recruitment for rare diseases.
   • Lack of appreciation for the patient burden for clinical studies. Clinical studies can often propose to collect large amounts of data from patients without considering the burden of a long clinical visit on a patient.

6. Patient Perspective

   Approach to reviewing an application:

   • Understand if the rationale is solid where the application is at the pre-clinical stage. It is important at this stage to evaluate if the product is relevant to patients – what will the landscape look like when this product is marketed? Will the product still be relevant?
   • Review trial feasibility.
   • Assess if the patient burden is too high.
   • Review aspects that matter to the patient including travel burden, assessment schedule, invasive procedures such as biopsies, use of placebo, availability of open label extension, inclusion/exclusion criteria, etc. This information helps to answer the question of whether the trial will be able to recruit or not.
   • Has the applicant provided a regulatory plan and how do they plan to demonstrate evidence of benefit to risk?
   • Has the applicant talked to patients/families and/or patient advocacy organizations to understand the disease, the burden, the needs and the landscape overall?

   
   Common pitfalls to look out for:

   • If it is not clear what the product will do, how it will be used and administered and who it is for.
   • If an applicant has not discussed their project with a patient group, or does not have a strong understanding of the community and how the patient voice can help.
   • If aggregate data summaries are used rather than the individual level data.
   • If the trial design is hard to follow and not enough detail is provided. It is essential to include detailed information on the patient journey – how will the patient flow through the trial from screening to end?

What are the benefits of becoming an ACT reviewer? 

The diagram below includes feedback from TACT reviewers about what the benefits are to them for participating in a TACT review:

To ensure that applicants provide reviewers with adequate, clear and concise information, they should be asked to consider including the following information in their full application, where possible, thus allowing reviewers to give a well informed review of their application.

Hypothesis:

• What is the biological support for the drug to be considered as a potential therapy?
• If a clinical study is proposed, a clear primary hypothesis is to be tested. This hypothesis must be specific about the intended outcome (i.e. “strength will increase with treatment” is not a clear hypothesis).

Preclinical:

• Provide detailed information on preclinical studies including the number of mice, how they were treated and how they were evaluated.
• Evidence that the sample size proposed is adequate.
• Have preclinical data been reproduced in more than one laboratory?
• Are the preclinical studies supportive of the route and dosage of administration required for a clinical trial?

Drug Substance: 

• What is it? Is it available? Could it be commercialized?

Pharmacology: 

• Drug potency, selectivity, proof of concept, advantages over competitive approaches.
• Pharmacokinetics (time course upon administration) and pharmacodynamics (intensity of effect).

Formulation and administration:

• Provide information on the relationship between dosing in the preclinical studies and subsequent clinical trials. Is there appropriate justification for the scaling up of dosing?
• Is the route of administration compatible with clinical trials?

Evidence of safety:

• Where the target population is paediatric, what previous paediatric regimes have been used? If there is perspective for long term use, do previous studies reflect this or would further studies be needed to extrapolate to different populations or duration of dosage?

Plans for clinical trials:

• Have the applicants considered all relevant aspects of trial design for the disease under consideration?
• Would patients be available and recruitable based on actual estimates at intended sites and data available from established international patient registries?
• Are outcome measures well understood and have they been used in previous studies?
• How would a trial for this drug fit with other ongoing or planned studies in the disease under consideration?

Patient Perspective:

• Do patients/patient representatives agree that a clinical trial with this drug would be an acceptable burden on the target population?
• Have the researchers fully addressed the risk/benefit balance for these plans?
• If marketed, would the drug make a clinically meaningful contribution to the existing standards of care for the condition?

Regulatory:

• Does the applicant understand the process for seeking advice (such as orphan drug products) and registration? e.g. FDA and EMA.
• Have regulatory issues in different countries been considered, relevant to this treatment?

A potential applicant will contact the ACT secretariat to discuss their proposed application and to gain an understanding of the ACT process.

An applicant will be invited to complete a pre-application form, which will be reviewed by the core committee, secretariat and ACT Chair for suitability.

If deemed appropriate, the applicant will be invited to submit a full application to the ACT Secretariat, using the form provided.

The secretariat, advised by the ACT Chair and core committee, will invite reviewers with suitable expertise to review the application and attend the meeting.

The secretariat will secure a suitable venue (if held in person) to hold the meeting and will arrange all logistics relating to the meeting.

The full application is due to be submitted to the ACT secretariat by the applicant.

The secretariat will circulate the full applications to the review panel. Reviewers will provide their comments and feedback to the secretariat. A lead reviewer will be selected for each application. The secretariat will compile all feedback and circulate to the lead reviewer and review panel.

The meeting will convene and each application will be allocated half a day for review. Each application will be discussed for 1 hour by the panel, before the applicant will be invited to join the meeting for discussion and questions, which will last 1 hour and 30 minutes. The applicant will leave and the panel will discuss their recommendations for 1 hour.

The lead reviewer will write a report to the applicant, using a template provided by the secretariat. The report will include an overview of the proposal, strengths and limitations as agreed by the panel and responses to the questions asked by the applicant in their application.

The report must be sent out to the applicant six weeks after the meeting. A PDF version of the report will be sent to the applicant with the disclaimer that if they chose to share the report with anyone e.g. patient organisations, funders, regulatory authorities it must be sent out in its entirety. The ACT committee reserve the right to investigate if a report has been altered.

The applicant and the secretariat will agree a non-confidential summary to be made public on the network’s website. Examples of non-confidential summaries can be found on the TREAT-NMD website here.

The full report will be circulated to reviewers for their reference. Under the terms of each reviewer’s CDA, the report is solely for their own viewing and must not be shared.

To make a case for the need for an ACT in your disease area, you may consider collating the following information:

• The number of treatments available in your disease area: It is estimated that only around 5% of rare diseases have a treatment. For more information about rare diseases and orphan medicine designation, see the EMA Orphan Medicines pages here. EURORDIS also highlights recent orphan medicine approvals here. The lack of treatments in a disease area is good evidence of need.

• The number of trials currently available in your disease area: ClinicalTrials.gov and Clincaltrialregister.eu have search tools that allow you to search for trials by disease. The website includes information about trials that have been terminated, which can generate statistics about the number of unsuccessful trials. This information could support the need to have a resource which provides independent and objective advice on the development of therapeutic programs in your disease area.

• The current drug pipeline: Having a knowledge of potential drugs in development is important. Companies with therapies in development are your potential applicants. Regular horizon scanning opportunities can help you keep up to date with your field (horizon scanning involves actively looking to see what the future might look like). In the field of medicine and healthcare, this is about investigating to see what medicines are in the “pipeline,” that is, in development in clinical trials but also in pre-clinical research. By systemically investigating the evidence, a clearer picture can be presented, as to what is in development in a given disease or sub-group. Earlier identification of potential medicines can facilitate greater collaboration between industry, academics, patient organisations and disease networks.

There are various ways to conduct this sort of horizon scanning. For instance, many patient organisations publish drug pipelines on their websites. In terms of pre-clinical research it is best to check company websites, such as their pipeline sections, press releases and investor presentations (as this is early stage, it is unlikely to be included on clinical trial registers but note pre-clinical research may be published in an academic journal).There are also websites available to carry out pipeline searches.

The user community of an ACT will include:

• The applicant: An ACT review provides the applicant with advice, which is independent of any funding stream, to inform their clinical trial design including recommendations on go/no go decisions, selection of endpoints and outcome measures. A subsequent ACT report could help an applicant to secure additional funding from other sources. Whilst industry applicants will be asked to make a suitable donation towards meeting costs, academic applicants can have access to this resource free of charge.

• Reviewers: To be able to offer such a high-level of multidisciplinary advice, it is essential to engage with the key opinion leaders in your field. Their support and input is imperative to the success of an ACT. There are many benefits to reviewers participating in such a review panel including learning from experts in different areas of the same disease field. Reviewers can also be best placed to identify applicants who can benefit from an ACT review.

• Patients: The overall aim of an ACT review is to examine the realistic prospects of a compound, and provide objective advice to support the development of a compound through to eventual registration, which will benefit patients. Patients and/or patient representatives will always be included on each review panel and will have the opportunity to give the patient perspective on the risk/benefit and potential burden of a clinical trial, as well as advising on the relevance and significance of expected outcomes of a therapeutic to patients.

• Patient organisations: Whilst an ACT report is confidential, interested parties can contact an applicant to ask to view a copy of the report they received from ACT. This report could help patient organisations (and other funding bodies) to prioritise requests for funding.

An ACT review meeting is significantly different to an advisory board used by industry. The ACT secretariat will form a bespoke, independent, multidisciplinary, objective panel of drug development experts, especially for each application under review. The independent aspect of an ACT is key to ensuring that each applicant receives objective advice.

Each reviewer completes a conflict of interest declaration prior to being selected. The applicant does not have any influence over the selection of reviewers. At the face-to-face meeting, the applicants are the guests of the reviewers, rather than the other way round.

To form a panel of this standard would be significantly costly, as well as logistically difficult to bring together a group of highly valued and key opinion leaders. For academics, such an advisory board is not typically a resource available to them.

An ACT can provide academics with multi-disciplinary advice free of charge. An ACT is a unique resource available to both academics and industry to utilise twice a year, every year. Thus providing a long term resource to support the development of therapeutics in rare diseases, where there are few or no treatments available.

In order to support ACT review meetings, you may wish to consider the following examples of sources for funding:

• Patient Organisations

Support and funding from patient organisations has been crucial to the TACT model, particularly for providing funding for the TACT project manager role. Many neuromuscular patient organisations have financially supported TACT meetings, where only academic applications have been submitted. Without their support it would be difficult to cover the costs of academic applications, which are free for the applicant.

Patient organisations understand the crucial role that patients can play in improving the design of clinical trials. The ACT model allows patient representatives to be involved at the earliest stages of clinical trial design in an independent and structured environment.

The ACT model can bring additional advantages; some patient organisations who have worked with TACT now require that applicants for research grants must first complete a TACT review. Supporting a multi-disciplinary committee, which objectively reviews developing therapeutic programs, can be of huge benefit to patient organisations when making funding decisions. In addition, when patient organisations see the value of the process to translational research and the benefit to the patients that they represent, they are often keen to offer support.

• Industry applicants

In the TACT model, industry applicants are asked to pay a contribution towards meeting costs depending on the size of their company. TACT operates on a not-for-profit basis and all funding supports meeting costs. TACT uses the boundaries below to decide a contribution amount to ask of industry applicants:

• • ≤9 employees
  • ≤50 employees
  • ≤250 employees
  • ≥250 employees
• Utilisation of other meetings and remote meetings

Meeting venues, travel and accommodation may account for the majority of expenditure in your ACT budget. Explore the opportunity of utilising a meeting room at an upcoming conference or large meeting, at which many of your potential reviewers will already be present.

Prior to the COVID-19 pandemic, all TACT meetings were held in person. During the pandemic, all meetings were held remotely and these meetings ran smoothly. TACT currently holds one meeting a year in person and one meeting remotely. In person meetings can often better facilitate collaboration and networking between the multi-disciplinary panel so we do recommend that you do try to hold some in person meetings. However, you may wish to consider holding some meetings remotely when you first establish your ACT to reduce costs.

For in person meetings, a suitable venue will need to be identified to hold the meeting. Consider utilising university/hospital meeting rooms which will likely be free or little cost to hire. If these type of rooms are unavailable to you, consider contacting patient organisations with whom you may be engaged to ask if they have available meeting space. If none of these options are available, you will need to budget for a hotel or conference-centre venue. Patient participation is an essential consideration when forming an ACT panel of reviewers so you may need to consider accessibility, and any specialist equipment required, to ensure their participation at any chosen venue.

You will need to create a secretariat in order to run and organise ACT meetings. It is advised that a project manager (full-time), one project assistant (part-time) and the time of one academic lead will be required to support and run an ACT. In addition to organising and running the review meetings the secretariat must promote their ACT, recruit potential applications and support the committee.

In advance of the meeting you must decide on and budget for reviewers’ costs. Depending on the applications being reviewed, between 12-20 reviewers will be invited to attend the meeting. For each reviewer’s participation, depending on your available budget you may consider offering the following:

• Honorarium for attending the meeting and reviewing applications: It is advised that the honorarium is a fixed amount for all reviewers, including patient representatives. In the TACT model, an honorium is offered for both in person and remote meetings.
• Reimbursement: for reasonable travel costs, if applicable.
• Accommodation: to control costs it would be advisable to make a block booking at a hotel and you will likely receive a preferential rate by doing this.
• An evening meal for all reviewers: this is also important for bringing the panel together as a team in a social setting.

You may wish to budget for costs for the design and printing of leaflets, brochures and posters for promoting your ACT to potential applicants at conferences. You may also need to consider travel to attend such conferences.

When establishing your ACT, it is important to have buy-in from the key opinion leaders within your disease community. You should involve them when forming your ACT and core committee. Your core committee will need to establish consensus on the minimum types of expertise needed, to provide an objective, transparent and multi-disciplinary review of applications for advice. For example, within the neuromuscular model, the core committee includes members with expertise in pre-clinical, clinical, toxicology, statistics and regulatory aspects of drug development.

Ideally, an ACT should have access to a specialty network such as TREAT-NMD, where they can identify potential reviewers. ERNs are perfectly placed to know who the experts are within different domains. We recommend that you hold a meeting, either virtual or in person, to introduce the concept of an ACT and discuss the types of expertise needed to be able to review a drug development programme within your disease area. This will help you to gain support for your ACT and identify potential members for your core and extended committees. Once you have a core committee in place, you will need to agree terms of reference for your ACT. An example of terms can be found on the TREAT-NMD website.

Once you have established your core committee, you need to identify who your potential applicants could be from both industry or academia. There are a number of ways to reach out to potential applicants including horizon scanning, attending conferences and working with patient organisations. Further suggestions for identifying applicants have been made within the sustainability section of this toolkit. If you decide to have a kick-off meeting for your ACT, you could schedule time to invite representatives from pharmaceutical companies working in your disease area. This will allow you to explain what an ACT is and gauge whether this would be useful to your potential applicants.

You will need to have an application process in place before inviting applicants. Within the ACT model, potential applicants must complete a pre-application form and a full application form. The purpose of the completion of the pre-application is for your ACT Chair and core committee to review for suitability e.g. if the applicant has not yet identified a lead compound, it would be too early for an ACT review. If the ACT Chair and core committee agree that the application is at a suitable stage for an ACT review, the applicant will be invited to submit a full application. In the neuromuscular field, the application form was devised based on the process of drug development and to facilitate a comprehensive appraisal. The information provided in both the pre-application and full application are very important, particularly because the applications will be used to identify potential reviewers, who have the right expertise required to review a particular application. The information provided in the application will help reviewers to make a fully informed review of the proposed development programme. Within this toolkit, we have provided you with template application forms for you to use and adapt to meet the needs of your ACT.

An ACT Chair with experience of drug development and knowledge of your disease field is required to oversee the strategic direction of your ACT. A core committee, which is representative of the different disciplines required, is needed to drive and oversee the work of your ACT. In order to support the work of your core committee and your ACT in general, you will need to establish a secretariat. The secretariat provides logistical support for each meeting and elicits applications for review. The secretariat will manage all contracting, funding and payments. We suggest that you include a project manager (full-time) or a project manager part-time with the support of a part-time project assistant. You should include an academic lead who can contribute time to review applications and select reviewers. Whilst this suggested structure may not be feasible when you first set-up your ACT, it is really important to ensure that you have someone in place who has overall responsibility to set-up and manage your ACT, along with the guidance of the ACT Chair and Core Committee.

If you are able to secure applications from industry for your first ACT review meeting, you will be able to charge them a fee for their attendance. This funding will help to initiate your ACT but you will likely need some additional funding for your first meeting. When planning your first meeting, you need to create a realistic budget. Within this toolkit, we have provided an idea of costs to consider e.g. venue hire, catering, reviewer’s costs. We have also included within the sustainability section suggested sources of initial ‘seed funding’. You may not be able to fund a project manager to oversee your act initially but it is really important to appoint someone to oversee your ACT whilst it is in its infancy.