Month: October 2018

The Grant Model Canvas for Developing Great Grants

A LEAN CANVAS-STYLE APPROACH TO DEVELOPING GREAT GRANTS AND PROJECTS

 

Writing a great grant or designing a transformative new project is hard work and many issues need to be considered carefully. An idea for grant application/project will typically go through many, many iterations before something great emerges. During this process, many different ideas and issues need to be explored and addressed and researched. If you sat down and wrote a full grant application for every idea you have, it would be far too much work. As an assistant prof starting out, I wrote way too many grants on half-baked ideas and needed a better way. The problem was, a simple framework for iterating and working up grant/science project ideas is lacking. Here, I try to introduce a solution.

 

Over the past few years, I became familiar with the LEAN CANVAS system for creating start up companies. It is simple and useful and I thought that it could be easily modified to help with grant writing. The idea behind the LEAN CANVAS approach is that writing a full finalized business plan for a start up company is silly because you need to work through so many different ideas and issues. The start up process is very dynamic and creative.  Experts in the area determined that one should be able to draft a more business plan easily to work through the issues and evaluate different ideas – and easily change directions. To learn about LEAN CANVAS for startups go here (https://strategyzer.com/). The platform is simple for iterating and exploring ideas.

 

Writing a finalized business plan is similar to writing a finalized grant application, and every new research project idea is similar to a new start up company idea. A lot of work and iteration is required before a new research project idea is worthy of a full grant application. Thus, the process of developing new research projects for grants fits well into the LEAN CANVAS-style framework.

 

Great grants provide outstanding solutions to important problems. They clearly explain to the target audience why the problem is important and your solution is great.

 

Here I share a LEAN CANVAS-style framework for working up grant ideas that are great solutions to important problems. For lack of a better name, I call this THE GRANT MODEL CANVAS.

 

THE GRANT MODEL CANVAS is available in Word (link), Keynote (link) and PowerPoint (link) formats. Please download and use as you like! The reason I share it in these formats is that Keynote and Powerpoint make it easy to create one file for grant ideas and then you simply duplicate the THE GRANT MODEL CANVAS slide as needed to explore a new project idea. I entered fields for you to type into easily to address the necessary issues needed to vet a new project idea. If you want a pdf version, go here (link).

 

Here is how it works:

 

Enter a title for the proposal at the top of the slide. The title should be short and capture the big idea. Now fill each indicated field:

Step 1: Problem – Clearly state the real world problem your project will address. What are the top 3 elements of the problem?

 

Step 2: Who is your target audience? Who is seeking a solution to this problem? Who would buy into your solution by funding your grant? What type of colleague/expert are you selling to?

 

Step 3: Solution – State your proposed solution to the stated problem. What is your big idea? What are the top 3 features of your solution.

 

Step 4: What is known?  Indicate the key knowledge and competing technology that already exists. What is the state of the field?

 

Step 5: What is unknown?  Indicate the critical knowledge/technology that is lacking, but needed to solve the stated problem.

 

Step 6: Value Propositions – Clearly state the value of your proposed solution. What is the #1 contribution that your grant will deliver? How does it address the problem and critical gaps in knowledge. Your value proposition clearly states how the knowledge/technology your grant will create solves your target audience’s problem or improves their situation (relevancy to your target audience). Your value proposition must indicate specific benefits (value) and explain to a grant reviewer why they should buy into your grant and not the ones you are competing against (unique differentiation).

 

Step 7: Unfair advantage – What advantages do you have that others do not?This includes proven expertise in the area, preliminary findings that support your case, novel technology or resources.

 

Step 8: Expected Impact – Clearly indicate the expected impact of your solution. What will the impact be? Which fields will be transformed?Why will it be transformative? How far reaching could the impact be on research, society and healthcare?

 

Step 9: Key Resources – What key resources/collaborators do you need to be successful? Indicate them here.

 

Step 10: Novelty – What is the evidence that your solution is novel?State what is novel about your solution and do some reading to make sure you are right!

 

If you go through these steps using THE GRANT CANVAS MODEL and fill in the different sections in the Keynote or Powerpoint versions, you should be able to iterate your ideas more systematically, rigorously and efficiently. This should help you identify an excellent topic for your grant/thesis project/postdoc project. Share the CANVAS with colleagues for feedback. Once you are on solid ground, you can begin to flush out specific aims and workout the details of the grant storyline and how the study should be performed.

 

If you have suggestions from your experiences, please share them and I will improve this.

 

 

Chris Gregg

 

 

 

Peer Review and the Dangers of Cognitive Easing

Getting Away With Murder

Murder is a serious, but alarmingly prevalent phenomenon in academic science. Every minute, someone somewhere kills another person’s idea. After years of labor, a researcher puts together a paper detailing a question, data gathered and the interpretation of the results. Each paper is a partially correct story presenting an idea and supporting evidence. Now the paper is handed off to peer reviewers, who will decide the fate of the paper (and the scientists involved). These reviewers are stressed, busy, get no credit for their effort and have intrinsic biases, whether they know it or not.

 

A relevant and famous example of irrational judgements is the finding that parole application reviews for convicts are more frequently approved if the judge reviews the application after lunch, rather than before lunch. Like a helpless prisoner, an academic scientist can only hope their peer reviewer(s) had lunch before reviewing their paper. In short, peer review is far less rational than we think and it may help to know some of the issues.

 

Daniel Kahneman’s incredible book, “Thinking, Fast and Slow”, systematically summarizes decades of psychology research that shows most decision-making is highly irrational and the experience of rationality is frequently an illusion. The brain will rationalize almost any outcome. Thus, even the most data driven members of society are irrational most of the time. One of my favorite examples is the finding that none of the best Wall Street stock traders do better than chance – despite deep training, oceans of data and state-of-the-art algorithms. Nonetheless, they are hailed as experts and draw massive salaries for services that a coin toss could achieve. The implications of this research for expert peer review are substantial. Kahneman doesn’t touch on it much, but I lay out some warnings derived from the material in his book on COGNITIVE EASING, which is a major factor that shapes favorable choices:

 

Figure. Cognitive Easing Controls Decision Making

One experiences cognitive easing when things are “perceived” as going well, because incoming information is familiar and fits with the individual’s existing knowledge of the world (See Figure). Thus, there is no need to mobilize effort to form a new understanding and no changes exist that are potentially indicative of a threat. In contrast, cognitive strain arises when one is challenged with unfamiliar information and must strain to understand the new information and fit it into their current view. Cognitive ease leads to a positive feeling and cognitive strain leads to a negative feeling and dislike. However, the experience of cognitive ease is not rational – it is a subconscious experience.

 

Research shows that favorable decisions are frequently based on simple things like preconceived expectations, font or language structures that promote cognitive easing and feelings of familiarity. This is a big problem when the decisions for rejection or revision should be based on the data and conclusions, and it is horrible for people trying to publish new ideas…

 

CONSIDER THESE POINTS:

 

  1. Cognitive easing and novelty: Truly novel directions and ideas that break from the current fashion of the field will be aversive to a reviewer only because they are unfamiliar and cause cognitive strain. In contrast, a new study that presents results that fit with the reviewer’s expectations and existing knowledge will be viewed more favorably. This experience is not rational – it is a feeling that manifests immediately and unconsciously and will increase acceptance of familiar lines of investigation, not novel ones.
  2. Cognitive easing and asking for more experiments: Peer review sometimes involves less of an evaluation of the evidence for the conclusions drawn and more of a laundry list of the experiments the reviewer feels would be make the paper more interesting/believable to them. How frequently are the experiments actually suggested by the reviewer because they are seeking a feeling of cognitive ease and the suggested experiments will provide it for them? This is different from requesting experiments that relate to the actual conclusions drawn by the authors.
  1. Cognitive easing and misreading: If the peer reviewer is stressed and struggling to understand your work, they will experience cognitive strain and be more likely reject the study independent of the data or conclusions. Frequently, they may extract conclusions that you never made or misread because they are looking for patterns that feel familiar. Their decision will ultimately be made on some sort of gut feeling rather than a rational evaluation of the content.

 

Rationality Is Frequently An Illusion: So what to do?

Kahneman shows that these cognitive biases are never going away and are deep in our biology. Amazingly, even if you teach people all the issues and solutions, they will convince themselves “this time is different”. However, you don’t want to be the goof that kills a big thing (or career)? (eg. initial grants to use CRISPR for mammalian genome editing were rejected). So, what can you do to fight cognitive ease?

 

AUTHORS:

  1. Accept that if you are really working on novel ideas, your colleagues won’t like them until they become familiar. 
  1. Accept that Nature, Cell and Science are often publishing important papers that fit with the momentum of the field. 
  1. Be persistent and don’t give up on your novel ideas! Present them widely so they become familiar to others.
  1. Present your work as clearly as possible and ground the novel ideas in familiar concepts.

 

REVIEWERS:

  1. Commit to signing the review with your name. If you can’t do this, it is probably because you don’t have the time to do a good job. Take responsibility for your work.
  2. Rationalize your arguments. This is your best defense against cognitive easing and straining biases. Generic statements that the study isn’t novel/important/high impact/rigorous are not constructive and probably the result of cognitive strain. What is the evidence for your claim(s)? Can you cite papers that directly undermine the novelty/impact? Can you defend your case with clear arguments based in fact, like a lawyer? If so, you are less likely to have been a victim of cognitive bias and the authors can consider your points carefully.
  3. Be constructive. Help the authors carve out a useful “knowledge product” (aka. useable knowledge). Often the language and cognitive strain is actually at the heart of your negative reaction. Realize that language is easy to change and may be causing your aversion. If their claim is too strong, what can they claim in your view? If no substantive claim is supported, then reject with a clear rational. Every paper gets better with peer review. No one wants to publish a shitty paper and they are depending on you to think of things that they didn’t – and save their bacon. It is OK to kill someone’s idea….just make sure it is clear why you did it and how to move forward.

 

Christopher Gregg, PhD

Kill Ideas Quickly – Natural selection is why you need to work so hard as a scientist

Creative science involves obsessive information gathering and thinking to generate a lot of ideas. Most ideas are partially or completely bad/wrong in their initial form. It takes years of sculpting to chisel out a great knowledge product. You do not want to spend years pursuing bad/wrong ideas, but in most cases it is not immediately obvious that your idea is bad/wrong. The faster you figure out whether your idea holds water, the better. Science takes enormous courage. The reason is that you are risking your time and there are many factors beyond your control. You can always earn back lost money or restore your pride – but time is lost forever. Imagine working 70 hours a week for years on a wrong idea and then be in the 5th year of your graduate studies, postdoc or assistant professor position only to have nothing to show for a lot of time and effort. Terrifying.

 

This means that you must carefully and fanatically work to generate and kill off ideas. Kill weak ideas as fast as possible to get one that puts you on firm ground. This short article includes some tips to kill off the weak.

 

How to kill ideas quickly:

Step 1. The 48 hour rule. The formation of an idea is exciting and rewarding. Give it 48 hours or a week and see if the enthusiasm persists after a couple of sleeps. I have ideas that have persisted for years – they are the ones that we typically end up doing or coming back to at some point. Keep a record in evernote.

 

Step 2. The next step to kill an idea is determine the impact of the idea if it worked. How big? Is it worth your time? How does the impact compare to other ideas you have? How feasible is it compared to your other ideas? What is the estimated timeline to success? Will technology likely have dramatically changed by that future point in a way that undermines your potential success dramatically?

 

Step 3. The second fastest way to kill an idea is to find a study that has already done it or done something that informs you about the novelty and feasibility. I personally find the experience of reading literature that undermines my favorite idea aversive – but it is essential to plow through it. This is your essential second step. Build a folder on your computer, fill it full of papers that are relevant and then go through them and answer the following:

  1. Is this novel?
  2. Are there signs that this question has never been asked or tested, at least with modern methods?
  3. Is the answer already known and well established? Are there reasons to challenge the dogma?
  4. Assume others have the same idea, because they will. Now ask: “Do I have any unique advantages that will allow me to succeed?”. What are they? Make a list.

 

Step 4. Answer this: What is the simplest experiment I can do that would kill this idea? Then do it. I recommend talking with trusted colleagues to discuss this and looking at available resources – don’t build anything new if you can help it. Often colleagues will point out weaknesses in your idea, but if you have gone through step 3, then you can weight these comments in an informed manner. Don’t be discouraged by negative feedback, just filter as needed. Finally, go as far as possible with available data and computation. Wetlab work is often slow and hard – do that when it is essential to nail the idea down.

 

If your idea is still standing after these four steps, you have got the beginnings of your PhD thesis, postdoc or big new story. Now focus like a laser and get it done. Mock up figures every week, and constantly storyboard to work out the right study designs, controls and statistical methods. Storyboarding improves thinking. We do it all the time.

Most ideas will not survive this four-step process in their initial form, which means you need to work like a highly organized and intelligent maniac to kill the weak and select out the strong – gradually chiseling out a masterpiece along the way. The key is to ask (and attempt to answer) clear, fundamental questions as you go.

 

Chris Gregg, PhD.