Bias, priming, and salience are the main psychological factors that influence our ability to estimate. Knowing what happens psychologically when we estimate, and using techniques from psychology, help us to deal with those factors so that we can improve our estimations, argued Joseph Pelrine, social complexity scientist and PhD researcher in psychology.
At the Agile and Software Architecture Symposium 2016 Pelrine gave the opening keynote in which he talked about the psychological aspects of estimating. InfoQ is covering this conference with Q&As, summaries, and articles.
The language we use primes us when doing estimations, Pelrine stated. The words used when asking for an estimate can result in different figures. He referred to the "reconstruction of automobile destruction" experiment that was done by Loftus and Palmer (1974) where using different verbs like "smashed" or "hit" when asking teams about the speed of the cars led to different estimates.
The way most people calculating velocity by using a moving window average is wrong, said Pelrine. You have to question if all data points are equal in importance when for instance the team composition has changed or if the team has improved their way of working. Solutions that would give better velocity estimates are to reduce the number of iterations for calculating the average or use an exponentially weighted formula.
InfoQ interviewed Pelrine about the cognitive processes that happen in the brains of people when they are doing estimations, why people keep using practices for which there is no evidence that they work, psychological factors which influence people's ability to work, techniques from psychology for doing estimation, why people don’t like to do estimations, and approaches for estimating work.
InfoQ: Which cognitive processes happen in the brain of people doing estimations?
Joseph Pelrine: A full answer would require delving into neuroscience, i.e. the "hardware" of the brain, which is not my field of speciality.
Cognitive processes are the "software" that run on the "hardware" of our brain. They can be roughly divided into two types: the subconscious operating system, and the conscious application layer, although the distinction is often flowing and the interaction seamless.
Estimation is a creative decision-making process that takes place in the subconscious based on a first-fit pattern-matching process which evolved over millennia (Klein, 1999). External inputs go through the conscious application layer and are passed over to the pattern-matching engine, which compares them to our database of experience, makes a decision, and passes that back up to be rationalised and expressed. This process is heavily influenced by three factors: cognitive bias, priming, in which exposure to a stimulus moderates a response, and salience, which is how we consider something to be relevant or important.
InfoQ: Sometimes people continue using certain practices with no evidence of working, or even evidence that they don’t work. What is it that makes people keep on using such practices?
Pelrine: There are many possible reasons. Force of habit. Fear of the unknown. The time required to become accustomed to new ways of working. Just try to get a developer to change his code formatting style, let alone his IDE! "Sharpen your axe!" said the traveller to the farmer slowly chopping wood in the forest. "I don’t have time," the farmer replied, "I need to cut wood."
Humans develop rituals as a way to bring order and structure into their lives. Rituals give a feeling of comfort, a feeling of familiarity, and help to reduce cognitive load, thus freeing up cognitive resources to deal with more urgent challenges.
InfoQ: Which psychological factors influence people's ability to estimate?
Pelrine: Here too, there are many factors. First off the whole realm of memory, which touches on both the neurological and psychological sides of our cognitive processes. Then the three factors mentioned before: bias, priming, and salience.
For me, though, one of the most important factors is the Kahneman and Tversky’s Planning Fallacy (1973), a theory so groundbreaking that it won Daniel Kahneman the Nobel Prize in Economics.
"The planning fallacy is a consequence of the tendency to neglect distributional data, and to adopt what may be termed an ’internal approach’ to prediction, where one focuses on the constituents of the specific problem rather than on the distribution of outcomes in similar cases. The internal approach to the evaluation of plans is likely to produce underestimation." (Kahneman and Tversky, 1977)
InfoQ: How can you deal with those factors?
Pelrine: A first step is simply creating and raising awareness of these cognitive processes, so that they don’t happen purely subconsciously. Be aware and accept that the brain is trying to act in your best interest! A next step would be to use new technical and psychological methods to help increase the quality of the estimation process.
InfoQ: Which techniques from psychology do you recommend for doing estimation?
Pelrine: Most of us are already using one psychologically based technique every day. Known as unpacking (Kruger and Evans, 2004), we know it as task breakdown. Unpacking help raise the quality of estimates through the analysis of all steps necessary to achieve a goal, which helps ensure that nothing gets overseen or let out.
Some other simple techniques are:
- Avoid distraction while estimating (Day et al., 2009)
- Consider the time of day – estimation sessions in the morning tend to be of higher quality (Blatter and Cajochen, 2007)
- Keep estimation sessions short – no longer than 90 minutes. (Broughton, 1975)
A number of techniques which were recommended in Danziger et al.’s (2011) study on ego depletion and decision fatigue are:
- Take frequent short rests (Tyler and Burns, 2008)
- Keep a positive attitude and relax during the breaks (Tice et al., 2007)
- Consume glucose to counter-act the effects of ego depletion (Gailliot et al., 2007)
InfoQ: Many people don’t like to do estimation. Can you explain why?
Pelrine: Estimation is commonly viewed as a chore that must be done, and many question the value of it. Estimates are viewed as commitments to which developers are held by management. The tools and techniques many developers use are often lacking. And in the end you have to accept the phenomena of retrospective coherence, the fact that in a complex system you can’t know things in advance.
InfoQ: Which approaches do you recommend for estimating work?
Pelrine: A very good method is reference class forecasting (Flyvbjerg, 2008), which was developed as a way to apply Kahneman and Tversky’s research to software estimation.
For task-level estimation I’ve developed chronobiological method, nicknamed the "Quattro Stagioni" method, which is based on Kleitman’s research into the Basic Rest Activity Cycle, or BRAC (Kleitman, 1957). Simply explained, the unit of estimation is not an artificial unit such as a story point, but the natural cyclic period of 90 minutes. Essentially, you break the day into 4 quarters of roughly 90 minutes each – start of day to coffee break, after coffee break to lunch, after lunch to coffee break, and after coffee break to end of day – and estimate in these units.
InfoQ: Do you have references of theories or studies that dive more into cognitive and social psychological aspects?
Pelrine: In addition to the studies mentioned above, I highly recommend Daniel Kahneman’s "Thinking fast and slow" (2011) and Dan Ariely’s "Predictably Irrational" (2009), both of which are more easily accessible to the layperson than most psychological papers are.
References:
Ariely, D. (2009) Predictably irrational. Harper Collins.
Blatter, K. & Cajochen, C. (2007) Circadian rhythms in cognitive performance: Methodological constraints, protocols, theoretical underpinnings. Physiology & Behavior, 90, 196-208.
Broughton, R. (1975) Biorhythmic variations in consciousness and psychological functions. Canadian Psychological Review/Psychologie canadienne, 16, 217.
Danziger, S., Levav, J. & Avnaim-Pesso, L. (2011) Extraneous factors in judicial decisions. Proc Natl Acad Sci U S A, 108, 6889-6892.
Day, R.-F., Lin, C.-H., Huang, W.-H. & Chuang, S.-H. (2009) Effects of music tempo and task difficulty on multi-attribute decision-making: An eye-tracking approach. Computers in Human Behavior, 25, 130-143.
Flyvbjerg, B. (2008) Curbing Optimism Bias and Strategic Misrepresentation in Planning: Reference Class Forecasting in Practice. European Planning Studies, 16, 3-21.
Gailliot, M.T. et al. (2007) Self-control relies on glucose as a limited energy source: willpower is more than a metaphor. Journal of personality and social psychology, 92, 325-336.
Kahneman, D. (2011) Thinking, Fast and Slow. Penguin.
Kahneman, D. & Tversky, A. (1973) On the psychology of prediction. Psychological Review, 80, 237-251.
Kahneman, D. & Tversky, A. (1977) Intuitive prediction: Biases and corrective procedures.
Klein, G. (1999) Sources of Power: How People Make Decisions. Cambridge: The MIT Press.
Kleitman, N. (1957) Sleep, wakefulness, and consciousness. Psychological Bulletin, 54, 354-359.
Kruger, J. & Evans, M. (2004) If you don’t want to be late, enumerate: Unpacking reduces the planning fallacy. Journal of Experimental Social Psychology, 40, 586-598.
Loftus, E.F. & Palmer, J.C. (1974) Reconstruction of automobile destruction: An example of the interaction between language and memory. Journal of Verbal Learning and Verbal Behavior, 13, 585-589.
Tice, D.M., Baumeister, R.F., Shmueli, D. & Muraven, M. (2007) Restoring the self: Positive affect helps improve self-regulation following ego depletion. Journal of Experimental Social Psychology, 43, 379-384.
Tyler, J.M. & Burns, K.C. (2008) After Depletion: The Replenishment of the Self’s Regulatory Resources. Self and Identity, 7, 305-321.
