1. A traffic light system of green (go), amber (amend) and red (stop) might be preferable to a simple stop/go approach when specifying progression criteria for internal pilot studies;
2. Pre-specified progression criteria agreed with funders need to strike a careful balance between being firm enough to promote ambition in the trial team yet being flexible enough to allow opportunities to remedy early problems;
3. Recruitment progression criteria should be based on rates per centre per unit time (eg, per month) that can be easily extrapolated, rather than specifying that an absolute number should be reached by a specific date, due to the unpredictability of opening sites;
4. When recruitment falls behind, it is essential to explore screening logs to determine whether insufficient participants were approached, insufficient participants passed eligibility criteria or insufficient eligible participants agreed to randomisation;
5. The assessment of intervention adherence, cross-over and outcome event rates should take into account the duration from randomisation to timing of primary outcomes if sufficient data are to be gleaned in time to inform progression;
6. When assessing missing data, it is important to explore the degree of missing data within key outcomes as well as the percentage of participants with missing data;
7. Trial teams should involve both their funders and their Trial Steering Committee in assessing their progression criteria;
8. Pilot study recruitment sites should be representative of sites that recruit into the main study;
9. Triallists may be able to take the opportunity to assess whether changes to existing technologies have occurred since the original study was planned, so that new technologies can be considered with funders, such as using an adaptive design;
10. Pilot studies need to be reported fully. An extension to CONSORT guidelines for pilot and feasibility studies is now available.
Don’t over-complicate it - use people to get a second opinion, help you, critique your application
Ensure PPI activities are not treated merely as a tick box exercise, but have a clear purpose/role in improving the research study
Use a range of engagement techniques - focus groups, individual conversations, surveys etc
Respect the expertise patients bring - value their time (inc. payment) and give equal voice
Improve plain English summary (PES)
Read the INVOLVE and NIHR guidelines about how to write a plain PES
Make sure PES explains the research, not just the reason for it
PES tend to suffer from the same problems of language, layout, etc
Use lay people to write PES. Often if PPI comes from academic profession, the PES remains over complicated despite their help
Description of public involvement in grant applications
Weave PPI description throughout the application, if you just put it at the end then reviewers have to piece it all together - make their job easier!
Make sure your application ‘fully’ costs PPI involvement
Ensure early stage researchers have adequate mentoring and strong co-applicants
Use INVOLVE guidelines on how to work out accurate and fair payment for public members involved in the research
Value of PPI to research design
Make sure the difference PPI has made to the application is explained
Good PPIE makes the research better - it’s not just about giving ‘lip-service’ to a requirement.
Get lay person with good communication skills to check Plain English Summary even before initial submission.
Don’t use the words ‘we will invite people’ - involve them from the start.
Good PPIE doesn’t happen overnight. Demonstrate that you've built those relationships?
Clarity of public co-applicants
Ensure application has clear role described if PPI co-applicant present
Mandatory use of PPI payment guide from INVOLVE. Only give choice of vouchers if it will affect public members on benefits.
Having the right people involved
Ensure PPIE is reflective of the communities you are working with
Make sure the application is studying a diverse enough group to be applicable more widely
Think about how to balance, including people, the research is intended for with the tasks/skills you need for PPIE
One person’s view is not enough - you need to talk to lots of patients
Involve relevant members of the community to help identifying the specific terms of the project proposed
Beware patient group endorsement - they will always support research into their topic
Ensure researchers think about participants belonging to patient groups
PPI can help you identify compounding factors that might influence your intervention, e.g. in a randomised study how many people might access intervention (especially when exercise / online support) independently.
Ensure participants are involved in dissemination - get them tickets to conferences, etc.
How can we develop more fundable NIHR digital healthcare proposals?
Created on Wednesday, 18 Jul 2018.
Great blog post from Jeremy Wyatt, Professor od Digital Healthcare - here are his top 10 tips!
Start with a real health or social care problem that matters – not an obscure condition or rare disease that only affects 50 people in the UK
Build a research team that includes all relevant expertise including patients or service users, clinicians and methodologists (statisticians, health economists and / or health psychologists), as well as digital health experts
Review previous work on and theories relevant to your chosen problem area (eg. Michie’s COM-B model of behaviour change) and ensure that the digital technology you choose is justified by reference to user characteristics, the scenario of use (“use case”) and our theoretical understanding of why the problem occurs.
For your impact study, ensure that study participants, outcome measures and the control are clearly defined and justified with respect to the main study question. Add a process evaluation and any other measures that may be necessary to understand why the study failed to show the expected result and how the intervention can be improved.
Consider how your new technology supports existing care pathways or models, and what changes might be needed in these to maximise the benefits of your innovation. Are there implications for patients, carers, clinical skill mix or education that could ultimately make your new approach infeasible? Are you evaluating a new digital healthcare technology or a new clinical servicesupported by that technology? Sometimes, drawing a logic model showing how your proposed intervention or activity achieves its objectives can be a helpful way to identify intermediate outcomes, leading indicators or process measure .
Consider how the intervention might be supported and improved once NIHR funding stops. One practical question here is, what data would NHS organisations need to write a successful business plan for this digital pathway, and can you collect this data during the NIHR funded project? Another is, does this digital intervention come under the revised EU regulations for medical devices and if so, at what stage to seek the CE marking this requires?
10 ways to illustrate how you used public involvment in your funding aplication
This is a great article from Health Research Authority on ways to use public involvement to inform your ethical review. You can use the same principles within your funding bid to show what effects the public have had on shaping your application. Public involvement needs to be woven through applications and it is imperative that you show the changes you've made due to the involvement - it's no longer acceptable just to state that you've consulted the public during the design process, details are what will make your application stand out!
How patients shaped the research question or why patients thought the research important (not merely stating that patients thought it important).
How patients shaped the intervention and decided which outcome measures to use in clinical trials.
How patients’ input was used to minimise the burden on participants.
How patients influenced the ethical design of a trial - e.g. whether use of placebo would be acceptable.
Where patients identified that participants might potentially experience distress and what appropriate changes had been made in response.
How practical arrangements were changed to better meet the needs of participants e.g. follow-up clinics at more appropriate times.
How recruitment processes were changed to be sensitive to the emotional and practical needs of potential participants.
How patients were involved in deciding what questions to ask in interviews/ focus groups, rather than only being asked comment on the wording of questions written by researchers.
How patients were involved in designing the protocol and patient facing information from the start, the responses they gave and the changes made as a result.
How patients would continue to be involved in the project at different stages, with a clear explanation of what input was expected and how it might shape future decisions.
Guidance on co-producing a research project - Design Tips
Going the Extra Mile [NIHR 2015] envisaged a population actively involved in research to improve health and wellbeing, stating that the “most successful collaborations will be those where knowledge is shared in a mutual partnership between researchers, the public and health professionals.”
The application of co-production within applied health research varies, some argue that co-production in research is just ‘really good PPI’. The recent publication Guidance on co-producing a research project [INVOLVE, NIHR 2018] aims to provide clarity through 5 key principles and features of co-production.
Sharing of Power Co-production requires the research to be jointly owned with people actively working together to achieve a joint understanding. Sharing of power does not mean that everybody is involved in every decision and every part of the project, people working on a project will still have different roles. With shared power and ownership of key decisions comes responsibility. There needs to be defined roles for everyone with each team member holding real responsibility.
Including all perspectives and skills Make sure the research team includes all those who can make a contribution, this will ensure all necessary views, experiences, skills and knowledge are included. Co-production involves embracing diversity and enabling involvement of all those people required for a particular project, including underrepresented groups. Inclusivity requires the research to be accessible. This includes ensuring physical access to meetings and accessible information; documents, for example, are in an appropriate format and language.
Respecting and valuing the knowledge of all those working together on the research Everyone working together on a research project is of equal importance. Everybody on the team is recognised as an asset. Co-production acknowledges the different knowledge bases, experiences and perspectives of all involved and each member of the research team is afforded equal respect and value.
Reciprocity All contributions to the research should be recognised. Everybody working together on a research project should get something back from contributing to that project. This could take many forms, not just financial rewards. For example, the development of social networks, increased confidence, new knowledge and skills and access to courses and training.
Building and maintaining relationships The evolving relationships between the various people working together in research are key to co-producing research. In order for trust to develop, individuals need to reflect on the knowledge, assumptions, preconceptions and biases that they bring to a research project. There needs to be acknowledgement and understanding of the complexity involved in ‘power differentials’.
Establish Ground Rules Set out expectations, in terms of the roles, responsibilities and behaviours of all at the start of the project. This will create an environment of respect.
Joint Ownership of Key Decisions Joint ownership of key decisions differentiates co-producing from collaborating. Not everyone needs to be involved in every decision or every aspect of a piece of research. The group should decide and agree who should be involved and when, in terms of the management, governance and undertaking of the research.
A commitment to relationship building Addressing power differences and developing relationships requires the development of open, honest, trusting and reciprocal relationships. Organisations and researchers need to be proactive in encouraging and facilitating public involvement and developing relationships beyond the research community.
Opportunities for personal growth and development Project leads need to facilitate the involvement of the public effectively and manage the flexibility and uncertainty that are often involved in co-produced research projects. Members of the research team need to be willing to relinquish power and accept reciprocity of experience and expertise. This may require a cultural change in the research team and/or the organisation hosting the team.
Flexibility A co-produced research project should provide opportunities for an iterative, fluid, open ended, experimental and interactive process; there should be opportunity for solutions and innovations to emerge from the relationships developed.
Valuing and evaluating the impact of co-producing research Research findings or outputs, from working co-productively, will produce knowledge and end results which may be different from those produced by a conventional academic process. These include new relationships, expanded social networks and increased confidence of members of the public.
Continuous reflection The process of continuous reflection gives the research team opportunities to look at how they are working together, how they might be using their particular expertise and perspective in the project and how this might impact on the research process and findings/outcomes. Creating safe and supportive spaces which enable team members to openly and honestly reflect on challenging issues such as power dynamics and inequalities is an integral part of co-producing research.
You need to be thinking about costing from the earliest stages of your project design. You should aim to contact the relevant Research Support Office(s), and Trust R&D offices, several weeks before the deadline and to have familiarised yourself with the current research costing guidance.
Consider the questions:
Are the costs realistic?
Is your research project value for money?
Are you paying patients or public involved in your project?
Will you need to pay for any other assistance or do you have enough staff within your research project?
Do members of the research team need training and does this have cost implications?
Have you considered costs for equipment procurement, software licences, travel expenses, office supplies (printing, photocopying, postage, stationery, advertising), literature reviews, access to available publications, dissemination costs etc?
Do you have CTU involvement in your project - as this will need to be costed?
Have you taken into account the three main types of cost - Research costs, NHS support costs, and treatment costs in line with AcoRD?
Have you allowed sufficient time for the costings to be verified by the host institution finance office and the finance office of any collaborating partners?
Attributing the costs of health and social care Research and Development (AcoRD) The AcoRD guidance categorises the costs associated with research based on the nature of the activity and whether activities contribute to NHS patient care. Activities are divided into three categories, to which all research and development activities can be attributed:
NHS Treatment Costs: are patient care costs, which would continue to be incurred if the patient care service in question continued to be provided after the R&D study had stopped. These costs might be higher or lower than the cost of providing the current standard treatment in the NHS. Where the costs are higher, these are known as Excess Treatment Costs. These are met through the normal commissioning process.
NHS Service Support Costs: are additional patient care costs associated with the research, which would end once the R&D study in question had stopped, even if the patient care involved continued to be provided. Examples of such costs are processing patient records to identify NHS patients eligible to participate in the study and obtaining informed consent from participants. These costs are met from the R&D budget of the Department of Health and are provided through the NIHR Clinical Research Network (CRN) with the activities carried out by CRN funded staff. To ensure your study can be delivered successfully contact the CRN by emailing firstname.lastname@example.org
Research Costs: are costs of the R&D itself that end when the research ends. They relate to activities that are being undertaken to answer the research questions. Research costs are split into two parts; Part A must always be met by the funder, and Part B which will be met by the DH where the funder is an Association of Medical Research Charities (AMRC) member and NIHR Non-commercial partner (www.nihr.ac.uk/research-and-impact/documents/NIHR-CRN-Portfolio/NIHR%20Partner%20List.pdf).
Local study trial co-ordination and management
Data collection needed to answer the questions that the research study is addressing
Regulatory preparation and compliance including obtaining ethical approval and complying with the Medicine for Human Use (Clinical Trials) Regulations 2004
The time taken by chief and principal investigators (CI and PI to explain the study to professional colleagues, and to understand, the research elements of a study.
There is a need for the authors of research reports to be able to communicate their work clearly and effectively to readers who are not familiar with the research area. Because many researchers write using technical, specialised language, particularly in scientific reports, writing Plain English summaries (PESs) can be challenging. This study looked at how to improve the quality of PESs.
The quality of the three versions (original summary, rewritten summary and edited summary) were assessed in two ways. First, a group of people who were not specialists in the subject area read and rated how easy the summaries were to understand. Secondly, a well-known measure called the Flesch reading ease score to assess how easy the PESs were to read.
The results showed that the following methods resulted in the best readable PESs:
to employ a specialist to do the task
to offer more explicit information to researchers on how to write it.
The full article can be read online in the BioMed Central Research Involvement and Engagement Journal
Intellectual Property: What is it and why does it matter?
What is intellectual property? Intellectual Property (IP) is used to define the products of human imagination, creativity and inventiveness. It includes literary and artistic works, inventions, symbols, names, images and designs. In healthcare IP may come from research or clinical practice. It includes: medical devices; computer software; training materials; service delivery or pathways; project or patient management; tools, scales or instruments; diagnostics; pharmaceuticals; and biotechnology.
How can you protect your idea? Identifying where Intellectual Property exists is the first step, as unless it is identified, it cannot be captured, protected and exploited. It’s most important keep your ideas confidential from the very beginning. It is very easy to lose IP early on before it has been protected, by publishing research papers, presentations at conferences or meetings and unguarded conversations. Most universities or NHS trusts will have a business or enterprise unit who will help you with protecting and developing your idea. Your research office should be able to advise you who to contact.
What about the confidentiality of your idea when you’re applying for funding?
Discussion with RDS staff: Any discussion between RDS staff and researchers will be considered as confidential by the RDS and not related to a third party without your consent.
Funding schemes: Funding scheme managers also understand the issues of confidentiality and welcome contact from the researcher at an early stage when completing the application form to discuss any issues. You should be clear about the conditions of the funding scheme you are applying to. Funders will want you to publish the results of the research they funded, whereas you may want to keep the results confidential. Commissioned calls may present a particular problem. Again discussion with the funding scheme prior to application may clarify this.
Who owns the IP? Legislation in the United Kingdom generally provides that Intellectual Property created by you as an employee in the course of your employment is owned by your employer. If a project has been funded by the NIHR who owns the Intellectual Property Rights then? “The Department of Health still expects that where possible and appropriate, the Contractor owns IP arising from research. However, this is not always the case in research with collaborators”. (NIHR FAQs on intellectual property)
Does your employer have an IP policy? The NHS trust or university you work for should have an IP policy which sets out the principles of the organization with regard to Intellectual Property. This will cover issues about the ownership, protection and use of materials and inventions created by employees.
How can you protect your IP? Methods of protecting IP will depend on what it is. They include:
Patents: Patents protect inventions, i.e. the technical aspects of products or processes.
Registered rights must be applied for to the Patent Office: It gives a monopoly right for up to 20 years (from when the patent is granted not when an invention comes to market). To be patentable a product must be novel or include an innovative step. Obtaining a patent is a costly and slow process. A Patent Agent will be needed to assist you. NB methods of treatment and computer software are not patentable
Registered Trademarks: Registered trademarks protect brand names logos etc. You may see one of these two symbols ™ or ®. You must apply for Registered Rights. Registration requirements are distinctiveness, not descriptive, and having no conflict with other marks.
Registered Designs: Registered Designs protect the appearance of a product e.g. shape or surface designs, Registered design rights must be applied for. Unregistered design right protection is also possible. Registration requirements are novelty and individual character.
Know-how: Know-how is knowledge which may not fall under more rigorous forms of protection, but which has commercial value. This confidential information may also be referred to as a ‘trade secret’. Commonly know-how is information which identifies how an industrial process or a key technical step can be implemented.
Developing your idea
Licences: When you are developing your idea you may want to use existing technology to help with that development. If this is the case you will need permission from the IP owner and for them agree the terms under which you can use it. This may be in the form of a licence.
Non-Disclosure Agreements: Collaborations with commercial or other partners may be required during the development process. A non-disclosure or confidentiality agreement should be signed between the IP holder and the third party. This should also be discussed with Funders so that they can help with the process if necessary and understand what is going on. Discussions will be confidential.
HRA Approval is the new process for the NHS in England that brings together the assessment of governance and legal compliance, undertaken by dedicated HRA staff, with the independent REC opinion provided through the UK Health Departments’ Research Ethics Service. It replaces the need for local checks of legal compliance and related matters by each participating organisation in England.
Using Information in your Research Proposal
Created on Wednesday, 12 Apr 2017.
Information is critical throughout your research
Your research idea
Has anyone done it before? Perform a scoping search of PubMed. Check for systematic reviews.
Is anyone working on it now? Check ongoing/current databases such as the UK Clinical Trials Gateway, Europe PubMed Central Grant Finder Tool, Clinical trials.gov
Establish whether the topic has been flagged as important to the NHS and relevant to clinical practice/national policy. Look for relevant guidance and policy or priority-setting documents. Look at NICE guidance, clinical guidelines, and the James Lind Alliance
Writing the background Set your study in a practical and/or theoretical context, making it clear how much is known already and what difference your research will make. Justify your research proposal using evidence from the literature, particularly any systematic reviews or recent guidelines (NICE). Support your case with:
Recent data describing the size of the problem, the cost to the NHS and implications for patients
Recent articles/primary research – show awareness of current ‘players’ in the field
Systematic reviews/meta analyses whenever possible
Methods section Information can also be useful in the methods section to:
Explore how other researchers have tackled similar questions
Demonstrate the feasibility of your proposed research design
Other uses of information Your searches can also help you to:
Identify funding for your research by noting how other similar studies were supported
Referencing Remember to cite all information sources, including any supporting statistics.
Getting help Make use of:
Health librarians - contact your local NHS trust or university librarians
Online tutorials – available for searching techniques and about individual databases
The RDS has Information Specialists who can help you with your searches. They can also give advice on undertaking systematic reviews, search techniques, sources of information and referencing (Mary Edmunds Otter, email@example.com; and Christine Keen, firstname.lastname@example.org)
Sign up to RDS EM research and funding news alerts
By signing up you are giving us your consent to contact you with information about health related research and funding news as well as RDS EM workshops, services and events. We use MailChimp to distribute all of our ebulletins and updates.