From the four prompts provided, pick one (e.g., the Mozart effect).
Then, in one single document, write your responses to part 1 and part 2.
QDAFI method TLDR
Why is such a method necessary?
Rationale: Memory drift will – over time – lead you to remember a highly simplified but
grossly distorted caricature of the paper unless we get ahead of this process and distill the
pithy, yet relevant essence (PYRE) of the paper up front. An added benefit of this process is
that it will allow you to also understand the paper better right now.
We can exploit the canonical structure of papers to facilitate active and profitable reading.
This structure is mirrored in the QDAFI method, laid out as such:
Q: Each paper starts with a question that the authors set out to answer. State this question
explicitly, in your own words, so we can gauge understanding.
D: What did the authors do to answer the question? This should be on the level of what did
they measure (y/DV) as a function of what (they) varied (x/IV), not more detailed. If they
did a lot of stuff, what was the most important such x/y or IV/DV pairing regarding the
question? Much of what is reported in papers are controls, internal replications or tangents
that were obviously requested by reviewers.
A: The rationale links the two – what was varied and what was measured. This is usually
the trickiest part because it is often not obvious to students scientists would do such things.
For the dress paper (2017), the rationale is that if the lighting of the image is ambiguous so
assumptions become important and if people assume lighting that they have experienced
more often, then we should look for an independent variable that would alter the
proportion of long wavelength exposure. Chronotype seems a good candidate, as that is
stable in adult life, and everything else being equal, we can assume that owls are exposed to
more artificial – long wavelength – light than larks, so we would expect/predict them to see
the dress as black/blue.
F: Literally, what is the big finding, that was set up by D and A?
I: How did the authors answer the original question, given these findings? Sometimes, there
are issues like confounds (not just limitations, which all studies have) that seriously
challenge the interpretation by the authors, yielding a deviating – your interpretation. This
is – by the way – why explicitly stating the rationale is so important. There, the logical chain
of the study will become apparent. If some of the assumptions don’t hold, or other causal
links haven’t been conclusively ruled out, other interpretations become possible, if not
more likely.
Honestly, whereas the full exposition of the methods states this should fit on a single page,
with 1-3 sentences per item, all of this can – and needs to be, for skilled operators – fit on
less than half of a page, here is how:
Q, D and F are literally a single, well-crafted sentence each.
A can be a sentence, but is usually two: One for assumptions and one for predictions
I is a single sentence if the interpretation is legit, and two if it is not. The second sentence is
for the statement of the issue (i.e. “SES is fundamentally confounded with trust”) and the
alternative interpretation (i.e. “thus, we can’t rule out that we are simply measuring trust,
not willpower”).
Parting words: None of this will be easy, if you have never done it before. But if it is hard for
you, that very fact demonstrates the need for you to learn the method. Unlike newspaper
articles, the meaning of most papers is not immediately obvious; you have to work for it.
Happily, skills can be learned by deliberate practice. It will be well worth it in the long run.
Gathering and digesting
background information
Gathering Background Information
● Two cautions
○
Don’t assume a scientific literature search is a short task or the same as a casual Internet search
■
○
There may be hundreds, or thousands of research articles related to your topic
Popular media outlets, such as the news websites of major broadcast networks and online or print
versions of newspapers and news magazines, are not scientific sources
■
This includes networks (e.g., NBC, CBS, CNN, BBC, CBC, etc.)
■
Online or print versions of newspapers or magazines (Time, Newsweek, etc.)
Gathering Background Information
Searching Scientific Databases
● Google – Just ‘cause you can Google something doesn’t mean that you know
how to perform an efficient, thorough scientific literature search
● Scientific Databases
○
Online databases from APA – PsycINFO and PsycARTICLES
○
Databases in related disciplines
○
General search engines
○
Google Scholar
○
Elicit
General Search Engines and Google Scholar
● Google is great for everyday browsing,
but insufficient for most scientific literature searches
● In contrast, Google Scholar is awesome!
○
Dedicated to searching scholarly articles
○
It’s possible to interface this with NYU’s library system for good results!
Gathering Background Information
Obtaining articles
■
These are almost always available in .pdf form electronically
Let’s demo!
● Did anyone try to find the QDAFI articles for today?
○
Woodworth, M., & Porter, S. (2002). In cold blood: Characteristics of criminal homicides as a
function of psychopathy. Journal of Abnormal Psychology, 111(3), 436-445.
doi:10.1037/0021843x.111.3.436
○
Sternberg Task
Something to try out: elicit.org
Gathering and digesting
background information
General structure of a research article
https://psych242.class.uic.edu/Article.html
Color code for 3
key elements:
Context
Content
Conclusion
Mensh and Kording, 2017
PDF link
Papers are full of details that are necessary for replication and useful
for the experts, but that don’t contribute much to understanding.
Why is QDAFI useful?
● Will help you to understand main aspects of paper better by
distilling the gist of what was done
● Will help you remember the key aspects of a paper, when
memory for other details may fade
● We can exploit the typical format of scientific papers to
facilitate active and profitable reading
● This structure is mirrored in the QDAFI method
QDAFI components
● Q: What was the authors’ main QUESTION?
● D: What did the authors DO to answer their question?
● A: What was the AUTHORS’ rationale?
● F: What did the authors FIND?
● I: What was the authors’ INTERPRETATION of the findings with regard
to the initial question?
Q (Question)
● Each paper starts with a question that the authors set out to answer. State
this question explicitly, in your own words, so we can gauge understanding.
○
If there are multiple questions, focus on main question
● Single sentence
○
Doesn’t have to be a complete sentence
○
(also: only use a period at end of a complete sentence)
Q: Does taking class notes by hand yield better academic
performance than taking them on a laptop?
D (Do)
● What did the authors do to answer the question? (Method)
○ What was the main predictor variable?
○ What was the main outcome variable?
○ No need for further detail
● If they used many methods, report the most important/central method
○ Much of what is reported in papers are controls, internal replications or tangents that
were requested by reviewers
● One or two sentences max
D: Participants were asked to either take notes on a laptop or by hand
while watching TED talks. They were then asked to answer both factual
and conceptual questions about the material presented in the talks.
A (Authors’ Rationale)
● Links the research question (Q) to the methods that the authors used (D)
○ How did they think this method would allow them to address their question?
● Sets up what the authors can conclude from any possible outcome, given their methods and question
○ One form: “If [outcome] were found to depend on [independent variable] in [under certain condition], then this
would demonstrate that [answer to Q], because…”
○ Another form: “If the answer to [Q] were …, then we would expect [outcome] to depend on [IV] in [this way],
because….”
○ Variant of that: “They reasoned that if their [hypothesized answer to Q] is correct, then [the predictor] should be
correlated with [the outcome]”
● Can be tricky to track down, but usually in Introduction and Method
● One or two sentences
A (Authors’ Rationale)
● Links the research question (Q) to the methods that the authors used (D)
○ How did they think this method would allow them to address their question?
● Sets up what the authors can conclude from any possible outcome, given their methods and question
○ One form: “If [outcome] were found to depend on [independent variable] in [under certain condition], then this
would demonstrate that [answer to Q], because…”
A: It is believed that using laptops to take notes yields poor performance
○ Another form: “If the answer to [Q] were …, then we would expect [outcome] to depend on [IV] in [this way],
because
of distractions. However, even without distractions, if laptop use leads
because….”
to shallower
processing,
it could
detrimentally
impact
on should be
○ Variant of that:
“They reasoned
that if their
[hypothesized answer
to Q] isperformance
correct, then [the predictor]
correlated
with [the outcome]”
conceptual
questions.
● Can be tricky to track down, but usually in Introduction and Method
● One or two sentences
Other examples of A
● “The authors reasoned that if autistic individuals suffer from a deficit in inhibitory
neurotransmitters, then autistic participants show less spatial suppression, which is
believed to rely on inhibition, and therefore be quicker to see large stimuli.”
F (Finding)
● What is the main finding of the paper?
○
This should directly follow from D and A
● If there are multiple findings, just describe the main finding
that relates to what you described in Q, D, and A
● Single sentence
F: Students who typed notes on a computer performed worse on conceptual
but not factual questions compared to those who wrote them by hand.
I (Interpretation)
● How did the authors’ results address their original question?
○ Single sentence if the interpretation makes sense and seems valid
○ Two sentences if you have concerns about validity/correctness of the authors’
interpretation
● If you question the validity, you can describe this in the second
sentence and, if possible, offer your alternative explanation
I: Taking notes on a laptop negatively affects performance in response to
conceptual questions compared to taking notes by hand, which could be due to
the fact that taking notes on a laptop seems to encourage shallow processing,
such as copying the material verbatim.
Does taking class notes by hand lead to better academic performance than
taking them on a laptop? To address this question, Mueller and Oppenheimer
conducted a study asking participants to take notes either on a laptop or by hand
while watching TED talks. They were then asked to answer both factual and
conceptual questions about the talks. It is commonly thought that using laptops to
take notes yields poor performance because of distractions. However, even
without distractions, laptop use may lead to shallower processing. If this is true,
the authors reasoned, this would detrimentally impact performance on
conceptual questions especially. Indeed, participants who typed notes on a
computer performed worse on conceptual but not factual questions compared to
those who wrote them by hand. The authors suggest that this is because taking
notes on a laptop may specifically encourage shallow processing, like copying
the material verbatim instead of reframing it in one’s own words.
QDAFI – further remarks
● The shorter the better!
● Use your own words if possible.
● Pascal recommends a 140 character limit for each QDAFI component, but
this is not a requirement.
● Missing from QDAFI (and sometimes from paper): further questions and
future research directions
● “It provides an effective template for structuring abstracts of original
research articles.”
Abstract + figures/tables reading
•
Much of the info you will need can be found in the
abstract and figures!
•
The abstract should lay out the main question,
approach, and findings. You can then confirm it when
you read through the paper
•
Figures should also show the main findings. They may
also depict the main method or task.
Asking questions:
Translating personally relevant questions
into societally relevant questions
Empiricism: Knowledge Based on Experience
● Empirical knowledge – knowledge
based on these senses and
experiences with the world
● Empiricism – the process of
acquiring knowledge directly
through observation and experience
○
The central building block of science
Thank you Prof. Reed!
Generating Research Ideas
● Four common ways that research
ideas come about
○
Personal experience and daily events
○
Prior research and theory*
○
Serendipity
○
Real-world problems
Thank you Prof. Reed!
Personal Experience and Daily Events
● John Darley and Bibb Latane’s work
on “Bystander Intervention”
○
This came from the infamous 1964 murder
of Kitty Genovese in New York City
○
Media reported that Genovese’s neighbors
heard her cries for help for over 30
minutes and didn’t call for police or
otherwise intervene
https://en.wikipedia.org/wiki/Murder_of_Kitty_Genovese
Thank you Prof. Reed!
Prior Research and Theory
● Previous Research
○
Testing a new question based on findings of a previous study
Does the bystander effect also occur in the animal kingdom?
○
Conducting an in-depth examination on a portion of previous research
How big does the crowd need to be for the bystander effect to occur/dissipate?
● Theory
○
Scientists test a theory by deriving predictions from it
What outcome does the theory predict should occur in this particular situation?
Thank you Prof. Reed!
Serendipity
●
Stumbling across something by chance –
often while looking for something else – and
having the wisdom and curiosity to recognize
that you may be on to an important discovery
●
Serendipity – the process of using an existing
theory to form a new research question
○
Lithium
■
Used to treat individuals with bipolar
disorder
■
The “calming” effect of lithium occurred
by chance when it was administered to
guinea pigs for another purpose
Thank you Prof. Reed!
Real-World Problems
● Widespread problems (e.g., addiction, depression, other psychological disorders,
family dysfunction, prejudice, crime, school and workplace conflicts, etc.)
○
What causes it?
○
How prevalent is it?
○
What characteristics and consequences are associated with it?
○
How do we best treat or prevent it?
● Specific situations
Reflect on how the changes in our everyday lives
due to the pandemic have affected you.
What have you learned about yourself or your
friends? Did you learn anything about society?
How might you go about
studying or learning more?
Can you ask the research question in a
way that makes it generalizable,
i.e., still “valid” after the pandemic?
SCIENCE is focused on exploring the natural world
through a process of discovery, exploration, investigation.
●
Scientists search for patterns (“laws”) of
nature in their experiments/observations.
○ How does this work when there are
sudden shifts in conditions around
these patterns, like a pandemic?
○ What might our behavior during such
rapid changes teach us about ‘normal’
situations?
○ What can we learn from past research
on how to act in changing conditions?
Scroll through the projects in the Project Tracker Spreadsheet above and…
1.
Pick one or two research questions that seem particularly
interesting/important to you. Why do they speak to you?
2. What does “under IRB review” mean?
3. This list was started early on in the pandemic; do any projects feel outdated? If
yes: how could they be adapted?
4. Which research questions do YOU think we should be asking
now?
Woodworth, M., & Porter, S. (2002). In cold blood: Characteristics of criminal homicides as a function of
psychopathy. Journal of Abnormal Psychology, 111(3), 436-445. doi:10.1037/0021843x.111.3.436
Q:
Are homicides committed by psychopaths more likely to be instrumental than homicides
committed by non-psychopaths?
D:
The researchers had the criminals take a PCL-R and categorized their homicides as purely
reactive, reactive/instrumental, instrumental/reactive, or purely instrumental.
A:
Psychopaths are often described as cold, calculating and unemotional. Therefore, it would be
reasonable to assume that they would commit premeditated homicides as opposed to non-psychopaths,
who would commit “crimes of passion.”
F:
The researchers found that nearly all homicides committed by psychopaths were instrumental,
compared to about half of homicides committed by non-psychopaths.
I:
When psychopaths commit homicide, they are almost twice as likely to commit an instrumental
homicide (rather than a reactive one) as non-psychopaths are to do so.
Q: How is information retrieved from short-term memory?
D: Participants were asked whether a test number was a member of a previously memorized
set of numbers while measuring how long they take to answer this question.
A: If the scanning process to retrieve information from memory is serial, reaction time should
increase with set size, whereas it should be independent of set size if this process is parallel.
F: Reaction time increases with each number that is added to the memorized set, at a constant
rate of about 40 ms per item.
I: Memory retrieval happens via a serial information scanning process and this process appears
to be linear and quite rapid, over a large range of set sizes.