Advertisement

Open Access

Strategies for cellular decision‐making

Theodore J Perkins, Peter S Swain

Author Affiliations

  1. Theodore J Perkins1 and
  2. Peter S Swain*,2
  1. 1 Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
  2. 2 Centre for Systems Biology at Edinburgh, University of Edinburgh, Edinburgh, UK
  1. *Corresponding author. Centre for Systems Biology at Edinburgh, University of Edinburgh, Mayfield Road, Edinburgh, Scotland EH9 3JD, UK. Tel.: +44 131 650 5451; Fax: +44 131 651 9068; E-mail: peter.swain{at}ed.ac.uk

Abstract

Stochasticity pervades life at the cellular level. Cells receive stochastic signals, perform detection and transduction with stochastic biochemistry, and grow and die in stochastic environments. Here we review progress in going from the molecular details to the information‐processing strategies cells use in their decision‐making. Such strategies are fundamentally influenced by stochasticity. We argue that the cellular decision‐making can only be probabilistic and occurs at three levels. First, cells must infer from noisy signals the probable current and anticipated future state of their environment. Second, they must weigh the costs and benefits of each potential response, given that future. Third, cells must decide in the presence of other, potentially competitive, decision‐makers. In this context, we discuss cooperative responses where some individuals can appear to sacrifice for the common good. We believe that decision‐making strategies will be conserved, with comparatively few strategies being implemented by different biochemical mechanisms in many organisms. Determining the strategy of a decision‐making network provides a potentially powerful coarse‐graining that links systems and evolutionary biology to understand biological design.

Introduction

Life at the cellular level is stochastic. Diffusion, gene expression, signal transduction, the cell cycle, and the extracellular environment are all stochastic processes that change in time in ways that can be difficult to predict (Raj and van Oudenaarden, 2008; Shahrezaei and Swain, 2008). While a cell's environment determines its response, information on the environment comes from different, fluctuating, and perhaps contradictory, signals. This information is processed using biochemical networks whose components themselves fluctuate in concentration and intracellular location. By coming together into a multicellular organism, cells can reduce stochastic effects in their immediate environment, but even in humans signals and the cellular response to signals can be substantially stochastic (Geva‐Zatorsky et al, 2006; Sigal et al, 2006; Feinerman et al, 2008).

In our opinion, such conditions imply that the cell's internal model of its environment can only be probabilistic. We propose that a biochemical network performing decision‐making has three main tasks: it should infer from noisy, incoming stimuli the probable state or states of the extracellular environment and, potentially, the probable future states; given the most probable states, it must decide an appropriate response through weighing the advantages and disadvantages of each potential response; and it must implement these functions using a strategy that is evolutionarily stable and so allow a population of cells to outcompete their rivals and survive environmental catastrophes (Figure 1). Such a division has been made in other fields, from economics to artificial intelligence and neuroscience. Statistical inference is the discipline concerned with inferring a quantity we cannot observe directly (the quantity is hidden) from a quantity we can observe, but which is only correlated with the quantity of interest. Decision theory provides a means to find the optimum response given uncertain information by weighing appropriately the costs and benefits of each potential response. Finally, evolutionary theory considers scenarios where decisions are not made in isolation but with other competing decision‐makers.

Figure 1.

Factors influencing cellular decision‐making. A cell senses signals generated by a change in the environment and must decide an appropriate response. This decision‐making can depend on the cell's predictions for the current and future state of the environment based on the signals it has sensed, the short‐term history of the cell, the expected benefits and the costs of each potential response, the actions of other cells, which may be competitive or cooperative, and the time taken to both decide and generate the response. The response may be a change in internal state or an action that changes the environment itself.

Here we survey recent work showing that techniques from these fields can explain not just qualitatively but quantitatively the behaviour of cellular networks, suggesting that cells may have evolved to biochemically implement such methods. We will try to place into one framework the strategies adopted by cells to detect, process, and respond to extracellular changes. By strategy we mean how a particular signalling network detects and analyses information not in terms of the details of biochemistry, but in terms of the functions of information processing that biochemistry performs. We believe that it is at this level of information processing that we shall discover evolutionarily conserved principles, whether we consider a stem cell deciding between different fates or a bacterium deciding between expressing and not expressing a particular operon. We will begin by investigating the strategies adopted to benefit individual cells before discussing strategies that are best understood at the level of populations of cells.

How do cells interpret noisy signals?

Cells are confronted with a fundamental problem: their biochemical decision‐making machinery is intracellular, but their behaviour should be determined by the extracellular environment. The environment may contain, for example, energy resources or a mating partner or predator. Signals detected on the cell surface and transduced intracellularly, however, are stochastic and can never present a complete picture of the environment. What strategy should cells adopt to interpret and make use of such noisy extracellular signals?

One possibility is statistical inference: the cell may use extracellular signals to explicitly estimate, or infer, the state of the extracellular environment. To a human reasoner, estimating states is natural. When a doctor diagnoses a patient, she will have several possible physiological states of the patient in mind and will use observations and tests to determine which state is most likely. Similarly, a cell might be interested in the state of its environment, even though it cannot observe the state directly, because knowing the probable state can be much more beneficial than knowing several environmental parameters. For example, a rise in temperature might mean that a bacterium has become exposed to the sun or that it has entered a host organism—two different environmental states that require very different responses. From measuring extracellular signals, such as the local concentration of metabolites or hormones, cells ought to estimate the most likely state of their environment before deciding an appropriate response.

Cells that do estimate the state of their environment must infer the state or the likely future state from signals that are only correlated with the state. The optimum way to perform such inference is Bayesian inference, at least it can be proved to be so if we accept a set of axioms that any form of inference ought to obey (Cox, 1946). We conjecture, then, that cells compute the likelihood of different possible environmental states, E, based on signals they sense, S, according to Bayes's rule:

Embedded Image

This computation assumes that several forms of ‘prior knowledge’ are available to the cell. First, it assumes knowledge of the possible environments, E, and their relative likelihoods, or prior probabilities, P(E). This prior knowledge may be uninformative—for example, that a mating partner is equally likely to be in any direction before pheromone is detected—or more restrictive—for example, that concentrations of an extracellular sugar should be in one of two states, either high or low, with a low state twice as likely as a high state. Second, it requires the probabilities of observing different signals in different environments, P(SE). The third term, P(S), describes the overall likelihood of sensing a signal S for all possible states of the environment. The result of the computation is the posterior probability, P(ES)—an inference about the likelihood of different environmental states given the prior knowledge and the signals that have been sensed. In Box 1, we give an example of using Bayes's rule. The posterior probability, P(ES), is a function of the magnitude of the signal sensed, S, and we next discuss the common shapes that this function takes.

Box 1 Inferring changes in the environment—Bayes's rule

Bayes's rule is a probability theorem, which allows a quantitative description of inference. It enables the computation of the probability that an unobserved random variable, say X, takes different values based on the observed value of a related random variable, say Y. The prior probability of X, P(X), is updated via Bayes's rule to the posterior probability of X given that we have observed Y, P(XY). Bayes's rule can be derived from the definition of conditional probability: P(X and Y)=P(XY)P(Y)=P(YX)P(X). Consequently, we have

Embedded Image

which is Bayes's rule relating the prior and posterior probabilities of X.

As an example, suppose you are leaving the cinema after seeing a movie, and you wonder whether or not it is raining outside. Let X=Rain denote that it is raining and X=NoRain denote that is not. Knowing nothing more, you believe it is equally likely to be raining as not: the prior probabilities of X are P(X=Rain)=P(X=NoRain)=0.5. While you cannot yet see for certain whether or not it is raining, suppose that you can see whether or not people that enter the cinema are holding umbrellas. We denote this observation to be Y, and we write Y=Umbrellas if some people have umbrellas and Y=NoUmbrellas otherwise. We expect a relationship between whether or not people are carrying umbrellas and whether or not it is raining. We can, therefore, infer the probability it is raining from observing whether or not the people that enter do have umbrellas. For example, you might believe that if it is raining, then some people will surely enter with umbrellas: P(Y=Umbrellas∣X=Rain)=1 and P(Y=NoUmbrellas∣X=Rain)=0. If it is not raining, however, then perhaps some people will still have umbrellas, but the probability is lower: P(Y=Umbrellas∣X=NoRain)=0.2 and P(Y=NoUmbrellas∣X=NoRain)=0.8.

Now, suppose you do observe people enter with umbrellas, Bayes's rule allows you to quantify how your prior belief in rain changes. Specifically, you can compute the probability that it is raining given your observation of umbrellas:

Embedded Image

We already know the two terms in the numerator of the right‐hand side of the equation. We do not know, but can compute, P(Y=Umbrellas):

Embedded Image

using conditional probability. Returning to Bayes's rule, we thus conclude:

Embedded Image

Your posterior probability of rain has increased compared to the prior probability, as expected given that you saw people enter the cinema with umbrellas.

Cells may infer the state of their environment

Many signal transduction and genetic networks with very different biochemistry have dose–response functions that are sigmoidal. A sigmoidal function is often considered advantageous because it prevents fluctuations in the input signal affecting the response if the input is below a threshold value, at which the response increases sharply. Near the threshold value, however, a sigmoidal response can amplify fluctuations because a small change in input generates a large change in output. If the signal S is continuous and the environment can only be in two states, then equation (1) describes the posterior probability that the environment is in one of these states. Viewed as a function of S, such a posterior probability is often smooth and sigmoidal raising the possibility that biochemical networks generate sigmoidal responses because they are solving inference problems (Libby et al, 2007). In the simplest case, the output of a decision‐making network could be proportional to the posterior probability of the extracellular environment being in a particular state. This inference about the probable state of the environment can then be processed by downstream networks to decide an appropriate response.

For example, Libby et al (2007) asked whether it is possible to design a genetic network that can infer the state of the environment from noisy concentrations of an intracellular signal—in essence, implementing a Bayesian computation using genetic‐regulatory machinery. They considered a bacterium in an environment with just two states: one rich in a metabolite, say a sugar, and one poor in sugar. These states could correspond to the gut of a host organism and the soil. To regulate the genes for metabolism of sugars, many bacteria employ transcription factors directly as sensors: sugar enters the cell, interacts with a transcription factor, and consequently influences gene expression. Libby et al, therefore, treat intracellular sugar as the environmental signal. Each state of the environment implies a different amount of intracellular sugar, although this amount is stochastic because of fluctuations in the transport of sugar, its consumption in the cell, and other factors (Figure 2). We write P(S∣high) for the distribution of intracellular sugar, S, when the environment has a high concentration of extracellular sugar and P(S∣low) for the distribution of intracellular sugar when the environment has a low concentration of extracellular sugar. Bayes's rule then states that the posterior probability of the state high in sugar depends on the concentration of intracellular sugar through

Figure 2.

Biochemical networks may use statistical inference to infer the probable state of the extracellular environment. (A) Inference in an environment with two states corresponding to low and high amounts of extracellular sugar S. The state low in sugar generates the blue distribution of intracellular sugar; the state high in sugar generates the red intracellular distribution. The environmental state is ambiguous for intracellular concentrations of sugar lying in the overlap between the two distributions. The Bayesian posterior probability of the state high in sugar given intracellular levels of S is the black, sigmoid‐like curve. (B) The response function of the lac operon measured by its rate of transcription in populations of E. coli as a function of the chemical IPTG, a non‐hydrolysable version of the sugar lactose, and cyclic AMP (cAMP), whose concentration in vivo is inversely proportional to the concentration of glucose (Makman and Sutherland, 1965). Data taken from Setty et al (2003). In the interpretation of Libby et al, the extracellular environment has two states: one high in lactose (IPTG) and low in glucose (high cAMP), and the other low in lactose and high in glucose (low cAMP). (C) An example of the probability distributions for lactose and cAMP in the two extracellular states (Libby et al, 2007). If the extracellular environment has two states with these distributions, then the response function measured in panel B is similar to the posterior probability of the state high in lactose and low in glucose (high in cAMP). (D) The posterior probability of the state high in lactose and low in glucose given the two distributions in panel C. Compare with the measured response function in panel B.

Embedded Image

where we have expanded P(S) over the two states of the environment. Analysing models of single‐component gene regulation, Libby et al showed that even networks consisting of just one gene controlled by an allosteric transcription factor can transcribe at a rate that tightly matches the posterior probability of the state high in sugar for many distributions of intracellular sugar.

This interpretation of a biochemical network as a network that performs inference is consistent with measurements of the regulatory response in vivo. We can use an experimentally measured response to determine the underlying distributions for the input stimulus—the equivalent of P(S∣high) and P(S∣low) in equation (2)—that would give rise to the measured response if this response is proportional to the posterior probability of an environmental state high in the input stimulus (Figure 2B–D). These distributions are part of the organism's internal model of its environment. They describe what the organism expects in different environmental states and, as a consequence, underlay its decision‐making strategies.

Improving inference over time

The inference described by Libby et al depends only on the steady‐state concentration of sugar. It, therefore, requires the network to reach steady state within the lifetime of a fluctuation in extracellular sugar if the network is not to average fluctuations in sugar. In situations where the signal fluctuates substantially over time, the cell might be expected to continually update its beliefs. Andrews et al (2006) have proposed that the network generating bacterial chemotaxis performs such real‐time inference. To chemotax along a gradient of a signal, Escherichia coli estimates a time derivative of the signal (Berg and Brown, 1972). The signal is detected by its binding to receptors at the plasma membrane, which is a stochastic process (Korobkova et al, 2004).Andrews et al assume that, before estimating the time derivative, the cell first infers the concentration of the signal at the cell membrane from the concentration of receptors bound by signal. Using simulation, they show that the inference implemented by the chemotactic network strongly resembles a Kalman filter (Kalman, 1960; Kalman and Bucy, 1961), an inference technique in control theory to track the dynamics of a hidden variable (here the concentration of the signal) from noisy measurements of a correlated variable (the concentration of receptors bound by the signal). A Kalman filter falls within the Bayesian framework. It performs updating through a sequential application of Bayes's rule: the current posterior probability of the extracellular state becomes the prior probability of the extracellular state at the next time step, and Bayes's rule is then applied again to find the updated posterior probability (Barker et al, 1995). Intuitively, sequential updating allows a cell to base its decisions not just on the current signals it is receiving, but also on their recent history. In chemotaxis, such inference leads to optimum low‐pass filtering of the concentration of the signal, reducing the effects of stochastic biochemistry and rotational diffusion of the chemotaxing cell, while maintaining a response sufficiently fast to allow the bacterium to detect changes in the gradient of the signal in real‐time (Andrews et al, 2006).

Similar real‐time inference may also occur in the system for sugar metabolism described above. For example, once exposed to a high extracellular state of sugar, another state high in sugar is perhaps more likely, at least over some period of time, because the bacteria are probably in the human gut. Such memory naturally fits into Bayesian inference through the prior probabilities of the states high and low in sugar, P(high) and P(low). After exposure to a state high in sugar, P(high) could increase and P(low) will correspondingly decrease. With this new prior probability, the posterior probability of the state high in sugar will still be a sigmoidal function of S, but will be larger at low concentrations of sugar. The change in the prior probability, P(high), could be biochemically implemented in E. coli through the concentration at the plasma membrane of the lactose permease, LacY, which is known to remain at an elevated concentration for generations after an initial exposure to lactose (Novick and Weiner, 1957). Increasing the concentration of the permease will increase the rate of the transcriptional response in a manner similar to the change in the posterior probability because more lactose will be transported into the cell for the same concentration of extracellular lactose. In the eukaryote Saccharomyces cerevisiae, a similar epigenetic memory of prior exposure to galactose is created through concentrations of the cytosolic enzyme Gal1p (Zacharioudakis et al, 2007). This increase in concentration also has the effect of enhancing the transcriptional response of the GAL regulon to low concentrations of galactose (Kundu et al, 2007). Chromatin modification is another eukaryotic epigenetic mechanism that has the potential to biochemically implement changes in prior probabilities of environmental states (Houseley et al, 2008). Such learning is often referred to as adaptive sensitization (Ginsburg and Jablonka, 2009).

These examples show that cells have the potential to implement sophisticated statistical calculations to infer changes in their environment despite stochastic signals and stochastic sensing networks. Over evolutionary time scales, the signalling and decision‐making networks should evolve to encode the properties of the different possible environmental states. If environmental characteristics change, then the networks should alter to match this change (Tagkopoulos et al, 2008; Mitchell et al, 2009).

Cells anticipate changes in the state of the environment

Cells are continually sensing signals from a multitude of sources. Integrating this information has the potential to improve inference and consequently the fitness of the organism. While inferring the current environmental state can be advantageous, equally so is anticipating future changes. Tagkopoulos et al (2008) have shown that E. coli appears to infer from a sudden increase in temperature that it has left the soil and is now in a host organism. Consequently, as the bacteria pass into the gut of the host, they will experience a reduction in available oxygen. Using microarrays, Tagkopoulos et al demonstrated that the transcriptional response to an increase in temperature overlaps with the response to a loss of oxygen even if the temperature change occurs at maximal oxygen levels. Having inferred from the increase in temperature that they are now in a host, the bacteria predict an imminent loss of oxygen and respond appropriately in advance (Tagkopoulos et al, 2008). Such anticipation is learnt over evolutionary time scales. Using microevolution experiments in which increase in temperature was unnaturally followed by increase in oxygen, Tagkopoulos et al evolved bacteria in which the association between oxygen and temperature was substantially reduced. Another example can be found in the expression of the sugar operons of E. coli. During passage along the human gut, lactose appears earlier than maltose, and, indeed, anticipating future exposure to maltose, E. coli expresses the genes for metabolizing maltose upon exposure to lactose (Mitchell et al, 2009). This response is adaptive: activation of the maltose operon is lost if bacteria are grown in an environment where lactose is not followed by maltose and alternative sugars cannot substitute for lactose and induce expression. Similar anticipatory responses also occur in S. cerevisiae (Mitchell et al, 2009).

Biochemical networks have also been proposed that learn on the time scale of the lifetime of the organism (Gandhi et al, 2007; Ginsburg and Jablonka, 2009; Fernando et al, 2009). In such an associative learning framework, learning requires both memory and recall. Upon responding to a stimulus, an organism must record the aspects of the stimulus and its response. When the stimulus stops, the organism should also stop responding, but, through recall of its previous exposure, the threshold of stimulus at which future responses occur will change (Ginsburg and Jablonka, 2009). A classic example is Pavlov's dog, which learnt to associate a bell chime with feeding by simultaneous occurrence of the chime and sight of food. Genetic and signal transduction networks have been designed in silico, which, although they initially respond only to stimulus A and not to stimulus B, learn upon simultaneous exposure to both stimuli to associate the stimuli and then respond to stimulus B when it is applied alone (Gandhi et al, 2007; Fernando et al, 2009). Both networks work through a molecule that enhances the response to stimulus B, but is only synthesized when both stimuli simultaneously occur. Such associative learning, despite its adaptive potential, has not yet been discovered in cells.

Weighing costs and benefits

Once a cell has inferred the most probable state of its environment, it needs to decide an appropriate response. The anticipated costs and benefits of each potential response, given the probable environmental state and the probable future environmental states, must be compared to choose both the most advantageous response and the level at which to respond. For new gene expression, for example, one expected cost is the expenditure of cellular energy in the synthesis of RNA and proteins; the expected benefits will depend on the environment and the properties and quantities of the proteins synthesized. These costs and benefits will be biochemically encoded into decision‐making networks over evolutionary time‐scales.

Cost and benefit in terms of fitness

In many situations, it may be hard to quantify or even identify the various costs and benefits to a cell of a particular response, particularly for cells in multicellular organisms. For unicellular organisms, however, the situation is simpler because much of their physiology appears optimized to allow as rapid a reproduction as possible, at least for laboratory strains. An appropriate measure of fitness, therefore, is cellular growth rate, an experimentally accessible quantity. Perhaps the simplest cellular decision is when and at what level a cell should express a particular set of genes. Dekel and Alon (2005) elegantly studied precisely this decision in the bacterium E. coli by measuring the effects on cellular growth rate of expressing the lac operon in different extracellular concentrations of the sugar lactose. The lac operon encodes enzymes to metabolize lactose, and we will use Z to denote their intracellular concentration. By inducing the operon to different extents in an environment without lactose and measuring the reduction in growth rate of a population of bacteria as compared with a control population that do not express the operon, Dekel and Alon estimated the cost of this decision (Figure 3A). They found that the reduction in growth rate increased more than linearly with the amount of enzymes produced because, they argued, high synthesis rates of some proteins can deplete cellular resources and so impact cell growth super‐linearly (Dekel and Alon, 2005)—a form of opportunity cost where one decision precludes another. In this environment low in sugar, they found empirically that growth rate glow is reduced from the growth rate of the control population, gc, by

Figure 3.

Cells may evolve to make optimal decisions. (A) The cost of expressing the lac operon in E. coli is measured by the reduction in relative growth rate when the operon is expressed in environments without lactose. The red curve is given by a fit of equation (4). (B) Once the cost has been found, the benefit of expressing the operon can be obtained by measuring the increase in the relative growth rate when the operon is fully expressed in environments with different amounts of extracellular lactose. The red curve is given by a fit of equation (5) with equation (6). Data in panels A and B are from Dekel and Alon (2005). (C) The level of the expression of the lac operon, Z, under conditions of zero glucose. ZWT is the level of expression of the operon when fully induced. Data are from Kalisky et al (2007). The red curve is the predicted level of expression of the operon by Kalisky et al found by maximizing the benefit minus the cost as a function of the extracellular concentration of lactose (equation (7)). Bars indicate standard errors throughout.

Embedded Image

where cost of expression is calculated as

Embedded Image

for positive constants η0 and η0. Cost is a quadratic function of the quantity of enzymes synthesized, Z, at least for the range of Z tested. Given this cost, they estimated the benefit of expression in different extracellular concentrations of lactose by measuring the increase in growth rate for cells fully expressing the operon as compared with control cells that did not express the operon. Any increase in growth rate is determined by surplus energy gained by the metabolism of lactose despite synthesis of the enzymes required (Figure 3B). In this environment where concentration of extracellular sugar can be high, the growth rate is

Embedded Image

where increase in growth rate from sugar metabolism can be described by

Embedded Image

for positive constants δ and KY. Dekel and Alon (2005) postulate that this Michaelis–Menten form arises from the action of LacY permeases, which import lactose into the cell.

Decisions to optimize fitness

To make a decision, a cell should compare the fitness of each potential response given the expected extracellular environment. We define the fitness of a response as the expected benefit to the growth rate minus the expected cost. Such comparisons happen often in our own reasoning. To decide between one treatment and another, a doctor weighs the cost and efficacy of each treatment with the seriousness of the disease. Dekel and Alon (2005) showed that the level of expression of the lac operon appears to have evolved to optimize a similar trade‐off. Given their measured costs and benefits of expression, they used decision theory to ask what particular concentration of enzymes, Z, should E. coli synthesize to optimize its fitness. By assuming an extracellular environment in just one state with a constant concentration of extracellular lactose, they argue, and show with microevolution experiments, that bacteria maximize their growth rate as a function of Z. The optimum concentration of Z, Zopt, satisfies

Embedded Image

for a fixed concentration of the sugar lactose, S, and for all other concentrations of Z. We assume that the concentration of intracellular lactose is proportional to the extracellular concentration. The optimum Z is sigmoidal in S (Figure 3C). Below a critical concentration of sugar, the cost of expression outweighs the benefit, and the optimal expression level is zero. Above this concentration, the optimal expression increases with S, although it eventually saturates because of both diminishing benefit and increasing cost.

Surprisingly, considering a two‐state environment—one state low and one state high in sugar—with each state producing some distribution of intracellular sugar S, we can use decision theory and Dekel and Alon's measurements to optimize the expected growth rate and derive Bayes's rule (Box 2).

Deciding by optimizing fitness—a derivation of Bayes's rule as an optimal response

We can use decision theory to determine the optimal response for an E. coli in an environment that has two possible states, one low and one high in sugar. Using the measurements of Dekel and Alon (2005) of the cost and benefit to the growth rate of expressing an operon to catabolize sugar we can show, perhaps surprisingly, that the growth rate of the bacterium is optimized if its response follows Bayes's rule. Let Z(S) denote the level at which the bacterium expresses the operon when the intracellular concentration of sugar is S. We then use glow(Z) to denote the growth rate when the operon is expressed at level Z and the environment is in the state low in sugar, and ghigh(Z) for the growth rate when the environment is in the state high in sugar. The expected growth rate of the bacterium, Embedded Image, can be obtained by integrating the growth rate over all possible states of the environment and all possible intracellular concentrations of sugar, S, weighted by the probabilities of the environmental states and the concentrations of sugar:

Embedded Image

To find the function Z(S) that maximizes this expected growth rate, we use the calculus of variations. The optimal level of expression of the operon satisfies:

Embedded Image

We use equations (3) and (5) for the growth rates in the two environmental states, and by doing so assume that the intracellular concentration of sugar is proportional to the extracellular concentration, following Dekel and Alon (2005). In equation (3), we have further assumed that the concentration of sugar is so low in the state low in sugar that any benefit of synthesizing Z is substantially outweighed by its cost. From Figure 3B, this assumption implies that, for the sugar lactose, the probable concentrations of extracellular lactose in the state low in sugar are less than 0.1 mM. Differentiating equations (4) and (6) with respect to Z, and substituting above, we find that the optimal value of Z, which we denote Zopt, satisfies:

Embedded Image

or

Embedded Image

if we re‐arrange. Dekel and Alon found that

Embedded Image

where ZWT is the wild‐type level of expression of the lac operon, and we will ignore this term. If the most probable concentrations of extracellular sugar in the state high in sugar are greater than KY, which is estimated to be 0.4 mM for lactose (Dekel and Alon, 2005), then the permeases importing sugar are saturated, and Embedded Image. Consequently, Zopt(S) approximately satisfies

Embedded Image

which is Bayes's rule. Although we have made several assumptions, optimizing the expected growth rate also optimizes inference of the extracellular state.

Considering benefit minus cost as a measure of fitness may, however, be too simple. The expression for the lac operon predicted by Dekel and Alon from equation (7) does not match in detail the measured level of expression (for bacteria grown in the absence of glucose; Kalisky et al, 2007). The predicted optimal curve rises higher than that for wild‐type expression, although with a gentler slope (Figure 3C). By allowing an environment with a probability distribution for concentration of sugar, Kalisky et al have improved the prediction by averaging over this distribution. For their best comparisons, they use a bimodal distribution similar to a superposition of the two distributions generated by the two environmental states proposed by Libby et al (2007). They improve their fit further by considering stochastic fluctuations in the concentration of the transcription factor controlling the response (LacI). Such stochasticity reduces fitness, but only if the regulatory proteins have concentrations near those that optimize the growth rate. Otherwise, fluctuations can be beneficial because cells that by chance happen to grow faster will dominate the population (Tanase‐Nicola and ten Wolde, 2008). Near the optimal growth rate, the deleterious effect of fluctuations in the concentration of the transcription factor can be minimized if the cellular response is saturated at those concentrations of sugar that are most frequent (Kalisky et al, 2007). The DNA‐binding site of the transcription factor is consequently either always occupied or always free. Typical fluctuations in the concentration of free transcription factor are buffered either by high concentration of the inducer lactose or by a large number of active transcription factors (Elowitz et al, 2002).

Other definitions of cost and benefit

That cells may do more than optimizing their growth rate is also well known in ecology. There, a distinction is drawn between r and K selection (r and K are variables in the logistic equation, which models the growth of populations: r is the maximum possible growth rate and K is the carrying capacity or maximum size of the population; MacArthur and Wilson, 1967). A typical organism undergoing r selection grows quickly and usually lives in stochastic environments where extensive environmental calamities can occur, but there is little competition. A typical organism undergoing K selection lives in a competitive environment and maximizes its competitive abilities rather than its growth rate (Pianka, 1970). An r strategy, therefore, maximizes the expected growth rate, whereas a K strategy could minimize the extinction rate or perhaps the variance in the growth rate. Such issues become more complex when we consider the effects of decisions made by other cells.

Although reproduction ultimately decides fitness, we can also examine the effectiveness of biochemical networks that do not directly affect growth. Chemotaxis is an attractive system because its goal—to chemotax towards or away from a source of a chemical—can be identified. Stochasticity affects the diffusion of the signal, binding of the signal to any receptors at the cell surface, signal transduction, and potentially the motion of the chemotaxing cell itself. Some organisms, such as E. coli, move by swimming at constant speed with abrupt stops where they re‐orient in a random direction—a process known as tumbling—and then begin swimming again. To swim up a chemical gradient, the cell senses the current concentration of the chemical and compares it to the concentration sensed earlier (Schnitzer et al, 1990). If the concentration is increasing, the cell is swimming in the right direction and tumbling is suppressed. If the concentration is decreasing, the cell is swimming in the wrong direction and tumbling happens more often. In principle, cells could sense concentrations by allowing the chemical to enter the cytosol and interact with signalling molecules or transcription factors, as in the examples of sugar metabolism discussed previously. However, a more accurate strategy is for cells to degrade the signal at their surface and so prevent re‐measurement of previously observed molecules (Endres and Wingreen, 2008). Cells may also use stochasticity to improve their chemotaxis: bacteria swimming in the wrong direction may re‐orient faster by rotational diffusion rather than by actively changing their motion (Strong et al, 1998).

The accuracy of a decision can also be used to quantify cost. Andrews and Iglesias (2007) have modelled decision‐making and the chemotactic response of slime moulds. In their model, the state of the environment is the true angle of a chemical gradient, θs, up which a slime mould wishes to chemotax. A chemotaxing cell senses and responds stochastically with a movement angle θr. If θr does not equal θs, the cell does not chemotax towards the source and receives a cost in fitness, which Andrews and Iglesias suggest obeys the equation

Embedded Image

Equation (8) is minimal when θsr and maximal when θs and θr are 180 degrees apart. Using a Bayesian approach, they calculate the expected cost as

Embedded Image

where Pr∣θs) is a probability distribution describing the stochastic behaviour of the chemotactic network—the tighter this distribution is around θs, the better the chemotaxis—and the distribution Ps) is the cell's prior knowledge of the location of the source of the signal. Andrews and Iglesias asked how accurately does θr need to reflect θs if the expected cost is to be less than some threshold D, a standard information‐theoretic calculation (Cover and Thomas, 2006). To predict behaviour, they use the distribution Pr∣θs) that has the maximum allowed cost of D and so minimizes the correlation (or, more correctly, the mutual information) required between θr and θs. Interpreting the degree of polarization of the cell's morphology as proportional to the cell's degree of prior knowledge, their predictions are quantitatively consistent with observations with the slime mould Dictyostelium discoideum. Unpolarized cells respond as if they have no a priori assumptions and, for example, change directions more frequently than polarized cells (Andrews and Iglesias, 2007).

Decisions at the level of populations

So far we have looked at decision‐making strategies as they benefit isolated individuals, but cells and organisms typically exist in populations. Interactions between organisms or between organisms and their environment can change the fitness of different strategies over time. Decision theories with assumptions of a single decision‐maker and fixed costs and benefits are no longer appropriate (Nowak and Sigmund, 2004). Although we have argued that decision‐making strategies can be understood as maximizing or near‐maximizing the reproductive success of the individual, competition may force organisms to use strategies that appear suboptimal. For example, Pfeiffer et al (2001) have argued that a trade‐off exists between the yield of ATP and its rate of production during the metabolism of sugars. Fermentation can produce ATP at a faster rate than respiration because it produces fewer ATP molecules per sugar molecule. In situations where organisms are competing for a common, extracellular resource, they should, therefore, use fermentation. When metabolizing internal resources, they should use respiration. This prediction is borne out for some microorganisms, such as S. cerevisiae, which use fermentation to produce ATP while decomposing organic matter even in the presence of oxygen (Pfeiffer et al, 2001). A strategy with lower fitness in environments without competition—fermentation is inefficient use of a resource—can become successful in environments with competition.

Optimizing inclusive fitness

Such phenomena, where fitness of a strategy depends on the strategies adopted by the rest of the population, are best analysed using ideas from evolutionary theory. Natural selection can be viewed as maximizing not the fitness of an organism, but its inclusive fitness (Hamilton, 1964). The reproductive success of an organism need not only come through the individual organism's own reproduction, but also through reproduction of related organisms because they share the genes of the individual. Inclusive fitness includes the direct fitness of an organism, offspring generated by the organism's own behaviour, and its indirect fitness, the offspring of neighbours, which survive because of the actions of the organism, but their contribution to inclusive fitness is weighted by the degree of relatedness such offspring have with the organism. Decision‐making strategies that appear suboptimal because of a suboptimal direct fitness of the individual can be understood as optimal because of their contribution to increasing the individual's indirect fitness. Such cooperative strategies, which benefit other cells in the population, but are possibly detrimental to the decision‐maker, can be described by Hamilton's rule (Hamilton, 1964; Box 3). If b is the benefit to the fitness of the cooperation's recipient, c is the cost to the fitness of the cooperator, and r is a measure of the genetic relatedness of the recipient and the cooperator, then a cooperative strategy will be favoured by selection if

Decisions in populations—Hamilton's rule

Hamilton's rule determines whether natural selection favours cooperation. Here we follow the derivation of Queller (1992) and Frank (1998). For an organism, i, in a population, we let zi be the probability that the organism will try to cooperate when meeting another organism. Let us suppose that N genes encode the cooperative behaviour and that organism i has xij copies of gene j. Our first assumption is that zi can be written as a linear function of the xij:

Embedded Image

for constant bj. This equation for zi is usually interpreted as a linear fit across a population of z to the number of copies of all the required genes. Then δi is the fitting error, or residual, for organism i, and the average of δ over the population is then zero—as we might expect because the probability of cooperation should also be zero if all the xij are zero and the individual has no genes for cooperation. If we write Embedded Image then Embedded Image=Embedded Image with averages taken over the population. The equation of Price (1970) describes natural selection. In the simplest scenario, the difference in the mean value of a character such as g between an ancestral and a descendant population, Embedded Image, obeys

Embedded Image

with wi the fitness of organism i and Cov denoting covariance. For cooperativity to be selected, we require the probability of trying cooperation to increase in the descendant population and Embedded Image, but Embedded Image because Embedded Image, and so we need only

Embedded Image

Our second assumption is that the fitness of organism i can be written as a linear function of gi and of the average value of gi for the local group of organisms with which organism i interacts. We denote this average value as Gi. We assume that

Embedded Image

with, as a third assumption, α and β being constant across the population. This equation is also often interpreted as a linear fit of w to g and G with εi as a residual. The residual is not expected to co‐vary with either g or G. If we insert this expression in the Cov(w, g), we can see that cooperative behaviour and the genes for cooperation will be selected if

Embedded Image

Now α describes how the fitness of organism i directly changes because of its cooperative behaviour to others. We expect this cooperative behaviour to be costly to organism i, and so write α=−c, with c the cost to fitness. Organism i gains in fitness from the cooperative actions of others, however, and so we can write β=b, with b this benefit to fitness. Consequently, we have

Embedded Image

as our condition for selection. By interpreting the ratio of covariances as the coefficient of relatedness, r, we have Hamilton's rule. The history of this approach is described by Gardner and Foster (2008).

Embedded Image

A cooperative strategy can only be selected if there is genetic relatedness (r>0) and if the benefit is sufficiently high and the cost is sufficiently low.

Equally influential is the concept of an evolutionarily stable strategy (Maynard Smith and Price, 1973), which formalizes what we mean by an optimal strategy. A population implementing an evolutionarily stable strategy is optimal in that it cannot be invaded by a small number of organisms implementing an alternative strategy.

Research has focused on understanding the decision‐making strategies in populations of microorganisms (Keller and Surette, 2006; West et al, 2006). Experimentally determining the cost and benefits appearing in Hamilton's rule is difficult, but it is relatively straightforward to change the degree of relatedness in populations of microorganisms: for example, by seeding the population with either a single clone or two clones with opposing phenotypes, or by either preventing or allowing mixing of growing subpopulations. Such manipulations should, however, not alter the cost and benefit of a cooperative strategy. It is important, though, to be aware that generalizing from results obtained under laboratory conditions may not always be appropriate. The effects of stochasticity have also attracted attention, and we will begin by looking at such effects in bet‐hedging decisions.

Bet‐hedging strategies

A bet‐hedging strategy is usually one in which different individuals of an isogenic population persistently exhibit different phenotypes. It can be defined as a phenotypic polymorphism that reduces the variance in fitness of a population of cells while possibly increasing the variance in fitness for certain individuals within the population (Seger and Brockmann, 1987). How are such strategies implemented by the cell? Biochemically, the gene or protein network that determines the phenotype must be bi‐stable or, more generally, multi‐stable. It must have several distinct, heritable steady states. One example is phase variation in bacteria, where cells decide between expressing different phenotypes or ‘phases’. Although the biochemistry generating the phenotypes is diverse, ranging from site‐specific rearrangements of DNA to epigenetic mechanisms, the strategy of phase variation is an example of convergent evolution having been adopted by many bacterial species (Avery, 2006).

Stochastic fluctuations in a multi‐stable network can be both advantageous and disadvantageous. Too large, and they can undermine the dynamical stability of each steady state, causing cells to fluctuate too rapidly from one phenotype to another (Hasty et al, 2000; Acar et al, 2005). For example, much of the genetic regulation active in lysogenous phage lambda is believed to reduce stochastic fluctuations into the lysogenic state (Aurell et al, 2002; Santillan and Mackey, 2004). Yet, in general, decision theory predicts that random strategies can outperform deterministic strategies whenever some aspect of the environment is unobserved (Bertsekas, 2005). A cell can never accurately sense all relevant variables in the environment suggesting that the potential for stochastic behaviour is high, and not present under only special conditions. Indeed, without such fluctuations it may be impossible to generate different phenotypes within the population at all.

Although bacteria exploit stochastic fluctuations to generate phase variation and to determine the lifetime of each phase, how much stochasticity is necessary and how should this stochasticity relate to variation in the environment? Surprisingly, even fully stochastic switching with no sensing of the environment can be evolutionarily stable, but only if the environment changes infrequently (Kussell and Leibler, 2005). Assuming an alternative strategy that continuously senses the environment with a concomitant continuous cost in metabolic energy, Kussell and Leibler showed mathematically that stochastic switching without sensing is stable provided the state of the environment is not too uncertain, and related the extent of that uncertainty to the environment's entropy. The cost of sensing then outweighs the benefit because the environment changes rarely and most sensing is superfluous. In agreement with earlier predictions (Lachmann and Jablonka, 1996; Thattai and van Oudenaarden, 2004; Wolf et al, 2005), they proved that the optimal level of stochasticity or, more exactly, the optimal rate of switching is proportional to the probability of a change in the state of the environment and inversely proportional to the average lifetime of an environmental state (Kussell and Leibler, 2005). Such a choice balances the advantages of quickly switching to the optimum phenotype for the current environmental state and the disadvantages of quickly switching from this optimum phenotype before the state of the environment changes (Wolf et al, 2005). Wolf et al (2005) included stochastic sensing, allowing environmental transitions to be unobserved, observed only after long delays, or the environmental state to be incorrectly identified. When the costs of sensing are negligible, they found that the strategy of fully stochastically switching is only evolutionarily stable if the stochasticity impeding sensing is strong enough to effectively prevent sensing of environmental transitions. For example, if the delay in signal transduction is sufficiently long that the measured environmental state no longer corresponds to the current environmental state (Wolf et al, 2005).

Many of these predictions have been verified experimentally. Using a synthetic bi‐stable genetic network in E. coli, Kashiwagi et al (2006) showed that stochastic fluctuations can cause cells to switch into the state most favoured by the current environment. Acar et al measured the growth rate of a yeast strain engineered to switch stochastically between two states in an environment that periodically varies between two environmental states: one favouring the growth of one cellular state and the other favouring the growth of the other cellular state. As predicted, they found that fast switchers grow faster in rapidly varying environments and that slow switchers grow faster in slowly varying environments (Acar et al, 2008). Natural examples include the slow‐growing persister cells in isogenic bacterial colonies (Balaban et al, 2004; Kussell et al, 2005). Such cells are able to resist some antibiotics, and, after removal of the antibiotic, the surviving persisters give rise to a colony that again has a small fraction of persisters because stochastic transitions occur between the persister and the usual cellular state (Balaban et al, 2004). Another bet‐hedging strategy is followed by Bacillus subtilis. Under poor nutrient conditions, most cells commit to sporulation, but a small fraction instead become ‘competent’ (Maamar and Dubnau, 2005; Smits et al, 2005; Suel et al, 2006). They are then able to take up DNA from the environment betting that new DNA will enable growth despite the poor conditions. This decision to become competent is made stochastically: reducing intracellular stochasticity reduces the fraction of competent cells (Maamar et al, 2007; Suel et al, 2007).

Bet‐hedging can usually be understood as cooperative behaviour. Consider persister cells. Although while in the persister state, cells have the potential to survive some catastrophes, they grow only very slowly, and there is no guarantee that a suitable catastrophe will ever occur. Why should cells then enter the persister state? A strain with a lower percentage of persisters could potentially invade because its faster growth may generate a greater number of persisters at the next catastrophe. Bacterial cells are usually surrounded by relatives in a clonal group. Although their direct fitness is low in the persister state, unless a catastrophe occurs, they increase their indirect fitness by freeing resources for other cells (Gardner et al, 2007). Indeed, modelling predicts that the number of persister cells should increase as resources become scarce. The cost to persister cells of their decision becomes less because the growth rate of non‐persister cells decreases and the benefit to non‐persister cells increases because resources are limiting (Gardner et al, 2007).

The tragedy of the commons

Problems of cooperation occur most often when different organisms share a common resource (Hardin, 1968; Rankin et al, 2007). Organisms can ‘cheat’ by using the resource inefficiently or by not contributing as much to the resource as others and yet still receive almost as much benefit because the cost of their cheating is shared by all the organisms. Such cheaters can substantially lower the fitness of the population as compared with a population of cooperators (Figure 4). An example of this ‘tragedy of the commons’ is cancer.

Figure 4.

The tragedy of the commons. Blue cooperator cells secrete enzymes, shown by a pentagon, which hydrolyse an extracellular metabolite, shown as two joined circles, into a form that cells can import (two separated circles). The enzymatic reaction is highlighted within the dotted circle. Green cheater cells benefit from the cooperative action of synthesizing the enzyme by importing the hydrolysed molecules. They do not, however, pay the associated cost because they do not synthesize the enzyme themselves, and hence have a growth advantage. As the number of cheater cells grows, the resource is used less and less efficiently, and the fitness of the population of cells decreases.

Microorganisms often contribute to a common pool of molecules (West et al, 2006). For example, a cytosolic pool of viral proteins is created when many viruses infect a single host cell, but a mutant virus can evolve that sequesters proteins from the pool, but contributes little, leading to loss of fitness for all viruses (Turner and Chao, 1999). Many bacteria communicate by releasing small, diffusible, autoinducer molecules (Keller and Surette, 2006). Detection of autoinducers often leads to expression of exoproducts, such as extracellular enzymes, nutrient‐scavenging molecules, and toxins, and to further synthesis of the autoinducers. At high densities of cells, this positive feedback allows such quorum sensing to generate substantial production rates of exoproducts, a common resource, but quorum sensing too is vulnerable to mutants that avoid the cost of synthesizing the exoproducts, yet still benefit from them (Diggle et al, 2007). The fitness of such cheaters decreases with frequency because the common pool shrinks as fewer and fewer individuals contribute. Nevertheless, by competing a wild‐type and a cheater strain of Pseudomonas aeruginosa under different conditions of relatedness, Diggle et al (2007) showed that high relatedness favours the cooperative, quorum‐sensing strategy. Cheaters, intriguingly, reduce virulence in P. aeruginosa because they decrease the rate of production of virulence factors (Rumbaugh et al, 2009).

Both cheaters and cooperators can stably coexist. Greig and Travisano (2004) considered the strategy to express the SUC genes used by S. cerevisiae. These genes encode the enzyme invertase, which hydrolyses sucrose, but, unusually, this enzyme is secreted extracellularly and, therefore, potentially benefits all nearby cells. Greig and Travisano argue that the observed high degree of polymorphism both in the number of SUC genes and their activity arises because of selection for cheaters, whereby some cells with one polymorphism do not synthesize invertase, but benefit, instead, from its expression by others with a different polymorphism. Gore et al (2009) extended these ideas. By competing two strains of yeast, one, a cooperator that expresses invertase, and another, a cheater that does not, they demonstrated that small numbers of cooperating cells can invade a population of cheaters and that small numbers of cheaters can invade a population of cooperators. Both strategies can coexist: the evolutionarily stable strategy is a mixed strategy. Cooperators benefit slightly more from the invertase they express than nearby cheaters and can more than recover the cost of synthesizing the enzyme. As the number of cooperators grows, more sucrose is available to the cheaters whose growth rate overtakes that of the cooperators because the cheaters do not synthesize invertase. With many cheaters, however, their growth rate slows as compared with that of the cooperators because little hydrolysed sucrose is available. Invertase converts sucrose into glucose and fructose, and wild‐type yeast cells repress the expression of invertase when extracellular glucose levels are sufficiently high. Consequently, a wild‐type cell will cooperate in a population of cheaters and will cheat in a sufficiently large population of cooperators (Gore et al, 2009). A similar coexistence can occur with two strains of yeast competing for a common source of glucose. Following Pfeiffer et al (2001), MacLean and Gudelj (2006) competed a strain that was only able to respire against a strain that could both respire and ferment. Despite the fermenter strain expending the glucose faster, the cooperating respirer strain was not outcompeted because the fermenters are punished through the toxic by‐products (mainly ethanol with some acetate) they excrete. Although these by‐products diffuse away, they can accumulate rapidly when the density of fermenters is high (MacLean and Gudelj, 2006).

Cooperating with other cooperators: structured populations

A cooperative strategy is more likely to be evolutionarily stable if an organism is often surrounded by related organisms because this spatial structure increases r in Hamilton's rule. Ackermann et al (2008) considered self‐destructive cooperation where some cells decide stochastically on self‐destructive behaviour for the benefit of others. Such cooperation is in general not evolutionarily stable because cheaters that never act to benefit others can always invade and dominate. The situation changes, however, in a spatially structured environment. For example, pathogenic bacteria infect a population of hosts and each host is spatially isolated. Ackermann et al showed that if the number of cells infecting a host and the probability of cheating is small then cooperation is evolutionarily stable because cooperators are likely to find themselves with other cooperators. Cheater cells, if present, will dominate in any one host, but can then be invaded by cooperators in a new round of hosts. A possible example is Salmonella typhimurium, which must remove intestinal microflora as competitors. To do so, S. typhimurium triggers an inflammatory response in the human gut by invading gut tissue. Cells that invade gut tissue are, therefore, benefiting other cells and behaving cooperatively. This cooperation is also self‐destructive because those S. typhimurium that do invade are usually killed by the innate immune defences of the intestine (Ackermann et al, 2008).

Stochasticity can enhance cooperation in structured populations. If subpopulations of cells grow independently, the global proportion of cooperating cells can increase even though the number of cooperators within each subpopulation decreases (Chuang et al, 2009). This apparently paradoxical situation arises if those subpopulations with a higher proportion of cooperators grow faster than those with a lower proportion because fast‐growing subpopulations dominate global averages. Such a global increase in cooperators requires exponential growth and sufficient variance in the composition of the initial subpopulations (Chuang et al, 2009).

Although limited dispersal leading to structured populations where cells grow near their relatives can favour cooperation, it need not because scarce resources can cause related cells to compete and so reduce b in Hamilton's rule. Siderophores are molecules secreted by many microorganisms to scavenge iron. They form an extracellular, common pool, and mutant cheater cells can evolve that benefit from the secreted siderophores, but not synthesize their own. Griffin et al (2004) showed that cooperative production of siderophores by P. aeruginosa was favoured both by higher relatedness among neighbouring cells and by competition occurring globally rather than locally, which reduces competition between relatives.

Conclusion

We believe that ubiquitous stochasticity makes cellular decision‐making probabilistic. Here, we have reviewed recent work showing that cells can, in principle, biochemically implement statistical inference for estimating environmental states and that such an interpretation is both qualitatively and quantitatively consistent with measured responses of gene‐regulatory and signalling networks. Furthermore, cells can act with anticipation, making regulatory decisions that, although suboptimal for their current environment, are expected to be advantageous after an imminent environmental change. Key to decision‐making are the relative costs and benefits of different responses, which allow the optimality of responses to be tested experimentally. Finally, evolutionary theory shows how interactions within populations of organisms can lead to suboptimal behaviours, both for some individuals and for the entire population. Together, these examples demonstrate that human‐developed theories of decision‐making under uncertainty apply at the cellular level as well. This approach to understanding cellular behaviour is in its infancy, but we believe many discoveries are yet to come.

The conjecture that cellular networks have evolved to implement statistical and decision‐theoretic computations is challenging to verify experimentally. Rather than focussing on characterizing one strategy, it is better to compare different strategies, perhaps through competition experiments, but developing, for example, bacterial strains with rival strategies is difficult. Often we know little of the environmental statistics that held sway during the evolution of an organism and to which it expects to respond. One means to address this problem is microevolution experiments where, by controlling the environments that a population of cells experiences, we know the sensing and decision‐making challenges the cells face (Dekel and Alon, 2005; Tagkopoulos et al, 2008; Mitchell et al, 2009). The genomes of the evolved organisms can be sequenced to determine how the decision‐making network has evolved or predictions of decision‐making behaviours based on the presumed strategy of the cells can be verified (Dekel and Alon, 2005; Acar et al, 2008). We can investigate the potential strategies implemented by cells by determining what properties of time‐varying signals they measure using microfluidic devices (Bennett et al, 2008; Hersen et al, 2008; Mettetal et al, 2008). Synthetic biology is another approach by which our understanding of cellular decision‐making can be tested by synthesizing and analysing in vivo a biochemical network with a desired decision‐making strategy (Chuang et al, 2009).

Our approach to cellular decision‐making highlights the importance of determining as best as possible the native environment of an organism and of studying both individual cells and populations. The results of Tagkopoulos et al (2008) and Mitchell et al (2009) show that cells may not respond to the actual signal sensed, but may instead respond in anticipation of some event historically correlated with the signal. We need to investigate responses to signals of a magnitude that is appropriate to the cell's environment. Such magnitudes are usually lower than those applied in the laboratory and, as such, can mask cooperation, where some cells may not respond to low signal to allow others to do so, or cheating, where cells use the signal rapidly to outcompete others even though a rapid response reduces the benefit gained by all. Similarly, we may need to mimic the spatial structure of the native environment to understand why some cooperative strategies persist. The importance of stochasticity in cellular decision‐making highlights the importance of studying single cells. In general, random strategies can outperform deterministic strategies if some aspect of the environment is unobserved, even without competition (Bertsekas, 2005). Such exploitation of stochasticity is difficult to detect in populations of cells because stochastic effects are averaged at the level of the population. Alternatively, cells often regulate away stochasticity in the signals they sense. To understand how this regulation occurs biochemically, we need to measure the responses of individual cells to signals that fluctuate as cells have evolved to expect.

With a few exceptions (Vilar et al, 2003; Tanase‐Nicola and ten Wolde, 2008), an omission of present research is to connect sensing strategies at the molecular level to decision‐making strategies at the population level. Most studies on bet‐hedging and cooperativity, for example, do not even consider the role of sensing. Such a link is necessary to unite systems biology with evolutionary biology and to fully understand biological design (Loewe, 2009). Cellular sensing strategies, for example, have evolved in environments where interactions with other organisms are important: S. cerevisiae even though it does not secrete siderophores still expresses receptors for siderophores synthesized by other microorganisms. Analysing such inter‐organism interactions is a strength of evolutionary biology. Defining the limits of adaptation determined by biochemical networks and finding the functional form of the cost, benefit, and fitness of a decision‐making strategy are necessary for an understanding at an evolutionary level, yet are all strengths of systems biology.

A number of other areas have received little attention. Both deterministic dynamics and infinite populations are often incorrectly assumed when determining evolutionarily stable strategies. Opportunity costs, where one decision can preclude another by consuming resources, are usually ignored. So too is the ability of cells to influence the state of their environment—often viewed as the main purpose of decision‐making in artificial intelligence and control theory. Such abilities can generate systems with no evolutionarily stable strategy. For example, Kerr et al (2002) consider three strains of competing bacteria: a colicinogenic strain that can release a toxin, colicin, into the environment; a resister strain that has mutated the membrane proteins that translocate the toxin; and a sensitive strain. Under certain conditions, the evolutionary dynamics of this system oscillate with time. The resister strain can outgrow the colicinogenic strain because resisters do not carry the plasmid necessary to synthesize the toxin. The resister bacteria are themselves outgrown by the sensitive strain because this strain has fully functioning membrane proteins that, although sensitive to the toxin, also uptake nutrients. Finally, the sensitive strain can be outgrown by the colicinogenic strain because they are not resistant to colicin—a ‘rock‐paper‐scissors’ scenario (Kerr et al, 2002).

We also do not know the fidelity required of sensing. Often a cell can improve fidelity by, for example, increasing the number of receptors, but energy and resources must be expended to synthesize, operate, and maintain more complex signalling networks. Fidelity can also be increased by taking more time to detect and analyse stochastic signals, but in rapidly fluctuating or competitive environments such time may not be available. These trade‐offs have been little explored, although the physics of sensing at least determines a lower bound on what is achievable (Berg and Purcell, 1977; Bialek and Setayeshgar, 2005; Tostevin et al, 2007). A related line of research has focused on the reliability of a response by investigating its robustness or insensitivity to changes in the values of parameters (Rao et al, 2002; Stelling et al, 2004), particularly for chemotaxis (Barkai and Leibler, 1997; Alon et al, 1999; Yi et al, 2000; Kollmann et al, 2005), developmental networks (von Dassow et al, 2000; Eldar et al, 2002; Albert and Othmer, 2003; Manu Surkova et al, 2009), and the immune response (Feinerman et al, 2008). Such changes result from differences in the intracellular environment between cells and in individual cells over time. For example, many parameters in models are often implicit functions of the concentration of another intracellular species, which itself undergoes fluctuations with its own characteristic lifetime (Shahrezaei et al, 2008).

Not all decision‐making need be sophisticated. Cells include, for example, many intracellular homeostatic mechanisms (Alberts et al, 2007). In other cases, the need to respond quickly may be overriding. For example, we reflexively pull our hand away from a hot stove without careful contemplation of the temperature of the stove or the likely damage our hand will receive. The importance of minimizing injury trumps all other concerns. Cells may have similar ‘reflexes’ for dealing with potentially dangerous situations. A possible example is the response to osmotic shock in S. cerevisiae (Hohmann, 2002).

Nevertheless, because stochasticity and incomplete information are so pervasive at the cellular level, we predict that strategies from statistics, decision theory, and evolutionary theory should be widely observed when cellular networks are viewed at the level of information processing and as such should hold much explanatory power. Despite being implemented in different organisms and with different biochemistry, we believe that through functional conservation or convergent evolution, the number of such strategies will be comparatively small. Just as interactions between proteins and genes can be coarse‐grained to a level of interacting functional modules (Hartwell et al, 1999), with a limited number of functions performed by those modules, a yet higher level of coarse‐graining is to determine how these functional modules come together to create sensing and decision‐making strategies, and, higher still, how these strategies are linked to produce adaptable and evolvable organisms.

Conflict of Interest

The authors declare that they have no conflict of interest.

Acknowledgements

We thank Erez Dekel, Eldon Emberly, Kevin Foster, Andy Gardner, and Steve Michnick for critical comments on the paper, and Eric Libby for many interesting conversations. TJP is funded by NSERC and the Ottawa Hospital Research Institute. PSS is supported by a Scottish Universites Life Sciences Alliance (SULSA) chair in Systems Biology.

References

This is an open‐access article distributed under the terms of the Creative Commons Attribution License, which permits distribution, and reproduction in any medium, provided the original author and source are credited. This license does not permit commercial exploitation without specific permission.