How to Discover

For myself and my future trainees, I am using this blog entry to reflect on the principle components of the path to discovery in bench science. We have a limited lifetime to discover things and make an impact in our careers, so it is worth thinking carefully about the questions and directions one chooses to focus on. Discovery is at its best when it arrives in the form of an unexpected observation, but there is an art to discovery and a correct approach that will position an individual to make unexpected observations. Intertwined with that approach is the right attitude and the art of phrasing an interesting question. I want to comment on both aspects of discovery from my experiences. It is important to note that discovery leads to more problems/questions, as much as it does to solutions. An important part of the scientific process is solving problems and I think of this as separate to discovery. A full scientific research program in a lab balances a spectrum of projects focused on discovery, problem solving, and the generation of new technology within a focused area of expertise. Research projects within the lab should complement each other to promote synergy, rather than dilute expertise and interest across too many poorly related questions. A central bread-and-butter theme is needed to unite the lab environment, especially at early stages.

I start with an outline of the scientific process and end with some personal perspectives.

The scientific process is a series of deductive and inductive steps (see Nisbet, Elder and Miner. Handbook of Statistical Analysis and Data Mining Applications,.2009).

1. Define the problem and central question to be answered

2. Gather existing information about the phenomenon

3. Form one or more hypotheses

4. Collect new experimental data

5. Analyze the information in the new data set

6. Interpret the results.

7. Synthesize conclusions, based on the old data, new data, and intuition.

8. Form new hypotheses for further testing

9. Do it again.

Before beginning a major new body of work, it is useful to reflect on aspects of this process in the design of your project. Consider the following checklist:

1. What is the question you asking? Refine this question so it is clear and simple.

2. What have others found with regard to this question? On a scale of from 1 to 10, what are the opportunities for discovery? Is this a crowded, old field?

3. How can you improve your question to make it more interesting? To ask something that has not been asked previously?

4. How does your question fit into a story or conceptual framework? What stories could you tell?Scientific publishing is about the story.  This is simply how human beings understand information.

5. What is interesting or important about your question? Where could it take you in your dreams? Rank it on a scale of 1 to 10.

6. Can you optimize your question to maximize your chances of discovery?

7. What are the different approaches you could take to address the question? Balance Risk vs Reward in the decision for the approach.

8. What controls will you need to interpret your experimental results?

9. How will you analyze your results?! What statistical approaches will you use. Think of this before, so you don’t forget controls that prevent interpretation of the data.

10. What preliminary tests could you do to optimize technical aspects of your approach to improve the quality of your data? (Always take the time to optimize so that your results will be clear and interpretable….do not base conclusions on weak results!)

11. Write out each of the steps in your experiment, so you don’t need to think while you are doing it!

12. What will contribute to variability in your experiment? How can you control that variability?

13. What will the final figure look like that will present your results? What are the possible comparisons and different ways of looking at the data you will get?

14. What degree of effect do you expect in your study? Do you expect a lot of variability? How many replicates will you need to overcome that variability and detect a real effect?

15. Do you have all the reagents and/or samples (animals) you need for your experiment? Figure this out ahead of time.

16. Plan everything out in day planner schedule to determine how long it will take and when certain milestones will be achieved.

17.  Do not under power your study.  Plan for an appropriately high n, so you can make solid conclusions at the end.

18. Order everything you need and FINISH your experiment! A well conceived experiment is always worth finishing to the END! (always finish your experiments even if you loose heart half way through)

1. When you think about your experiment from a technical perspective, think about efficiency. At each step you capture a certain percentage of the effect and with each step you will introduce a certain amount of noise. How do you optimize your signal to noise? How big of an effect are you seeking?

2. Make sure your initial results are truly rock SOLID. These are the foundations of your project and if they are flimsy, you will reach a stage where you will feel pressure to bend your data to fit those early observations so you can get that paper together under extreme stress in 3-4 years time.    The pressure is coming….make sure the foundations of your study…your first experiments….are rock solid, so you are set on the right path from the beginning. BE PATIENT if your observations are ambiguous and keep working to NAIL THAT RESULT!

3. Thinking about the final figure that will represent your data is essential at the very beginning, so you know where you are headed. Design your figures mentally before and during an experiment. (I constantly pencil out figures to think about results)

4. FINISH YOUR EXPERIMENT AND IMMEDIATELY MAKE A PUBLICATION QUALITY FIGURE. This ensures you are making progress and on top of your data, even if the results don’t make sense at that time. They may make perfect sense years later!

Science is a balance of risk and reward. The existing funding system forces you to balance your risk carefully. You must show productivity and at the same time push into new frontiers. Though recipes for discovery are limiting in themselves, I tend to favor characterization projects to get going. Simple approaches that involve just “looking and learning”, like observing a behavior pattern or staining for several marker proteins to visualize the organization of neural circuit, can expose lots of new questions. Mouse genetic approaches to characterizing a system can also give great and elegant insights as well, but carry more risk and time investment. An unbiased characterization and careful systematic analysis of the results will foster ideas. Look for new technical approaches to characterizing the system associated with your question. A new angle can change everything!! Always think of new technical approaches to ask questions in ways that could not be done before, you will always learn something. Characterization projects lead to papers and useful knowledge with relatively little risk and they set you up for discovery. Everyone should have a component of their research that simply involves looking and describing.

1. Make an unbiased list of the features you are seeing in your data. For example, if you are looking at the expression pattern of several genes of interest, where are they expressed? what types of cells? Where are the cells? What do we know about their functions?

2. Design some specific questions (or hypotheses) from your initial observations. Make a list of several different ideas and questions and use your gut to judge the best place to start.

3. Think of methods to rigorously quantitate and statistically analyze your characterization of the system. For example, quantitate cell numbers, measure dendritic projection patterns, monitor feeding patterns, etc.

Characterization projects generate new knowledge and I think it is important to distinguish knowledge generation from innovation. Innovation involves solving a problem for the first time or in some novel manner, or following a crazy idea to see where it might lead. Innovation is about following your gut and trying a high risk, high reward idea. You must be comfortable and accept regular failure in the road to innovation and that is why it is important to balance your research program by having two classes of projects. In the optimal circumstances, characterization projects and innovation projects complement each other.

Big discoveries are made by:

(1) Doing something others cannot do, because they don’t have access to the knowledge, resources and tools necessary – aka. build new tools/resources that only you have and learn new fields whenever possible.

(2) Doing something others won’t do, because it is very difficult – no replacement for hardwork

(3) Fortunate insight or chance discovery that is capitalized on (the prepared mind!) – pay attention, do carefully controlled experiments, think deeply about your results

(4) Taking risks. You must take some risks in your career and recognize that the path you start down is rarely headed where you think it is…hold weak opinions.

(5) Reading and talking. You must learn broadly in order to understand the impact of your results and observations and connections to other fields. Something that seems mundane might be huge when cast in the right light.

(1) Characterization projects are powerful ways to develop hypotheses and break new ground, but you must use them strategically. Think differently. Look for untracked territory and ways to bring together different fields.

(2) If you are uncomfortable and unsure of where your work is leading, but you find it very interesting….that is normal and that is life on the front of innovation. Just keep asking good questions.

(3) The genius is in the details. Think carefully about the what, why, where and when details of your observations.

JUST TRY! JUST KEEP TRYING! Never fear failure or a new technique. Dust yourself off and try again. Science is mostly about just trying….you often won’t know if you are on to a good thing until late in the game.