Investment Timing

Solving Bellman Equation by Dynamic Programming

Given geometric Brownian motion $V$ in
\begin{equation}\label{eq:geometric}
dV=\alpha Vdt+\sigma VdW_t \tag{$\star$}
\end{equation}
we have the following theory.
$\newcommand\bbE{\mathbb{E}}$

Because the investment opportunity, $F(V)$, yields no cash flows up to the time $T$ that the investment is undertaken, the only return from holding it is its capital appreciation. Hence, the original Bellman equation for continuation region

$$\rho\Delta tF(x,t)=\max_u\left\lbrace\pi(x,u,t)+{1\over dt}\bbE[dF]\right\rbrace$$

turns into the following equation with $\pi(x,u,t)=0$ and the optimal stopping strategy $u^\ast$

$$\rho Fdt=\mathbb{E}[dF]$$

It means that over a time interval $dt$, the total expected return on the investment opportunity, $\rho Fdt$, is equal to its expected rate of capital appreciation.

We expand $dF$ using Ito’s Lemma, with the element $F_t(V)dt=0$ as $F$ is indifferent regarding to $t$

$$dF=F'(V)dV+{1\over 2}F''(V)(dV)^2$$

Substituting equation $\eqref{eq:geometric}$ for $dV$ into the expression, we get

$$\mathbb{E}[dF]=\alpha VF'(V)dt+{1\over 2}\sigma^2V^2F''(V)dt$$

Hence the Bellman equation becomes

$${1\over 2}\sigma^2 V^2 F''(V)+\alpha VF'(V)-\rho F(V)=0$$

i.e.

\begin{equation}\label{eq:differential}
{1\over 2}\sigma^2 V^2 F’’(V)+(\rho-\delta) VF’(V)-\rho F(V)=0 \tag{$\ddagger$}
\end{equation}

Note: For this problem to make sense, we must also assume that $\alpha<\rho$; otherwise the integral in $\eqref{eq:geometric}$ could be made indefinitely larger by choosing a larger $T$. Thus waiting longer would always be a better policy, and the optimum would not exist. Let $\delta$ denote the difference $\rho-\alpha$, which is greater than $0$.

In addition, $F(V)$ must satisfy the following boundary conditions:
\begin{equation}\label{eq:condition1}
F(0)=0 \
\end{equation}

\begin{equation}\label{eq:condition2}
F(V^\ast)=V^\ast-I \\
\end{equation}

\begin{equation}\label{eq:condition3}
F’(V^\ast)=1
\end{equation}

Condition $\eqref{eq:condition1}$ arises from the observation that if $V$ goes to zero, it will stay at zero. Therefore the option to invest $F(V)$ will be of no value when $V=0$.

Since $V^\ast$ is the critical value for optimal investment, condition $\eqref{eq:condition2}$ is a value-matching condition, which says that upon investing, the firm receives a net payoff $\Omega(V^\ast)=V^\ast-I$. As we have

$$\begin{align*} F(V)&=\max\left\lbrace\Omega(V),{1\over 1+\rho dt}\bbE[F(V+dV,t+dt)\mid V]\right\rbrace \\ &=\max\left\lbrace V-I,{1\over 1+\rho dt}\bbE[F(V+dV,t+dt)\mid V]\right\rbrace \end{align*}$$

Finally, condition $\eqref{eq:condition3}$ is the “smooth-pasting” condition that $F’(V^\ast)=\Omega’(V^\ast)$. If $F(V)$ were not continuous and smooth at the critical exercise point $V^\ast$, on could do better by exercising at a different point.

Note that equation $\eqref{eq:differential}$ is a second-order differential equation, but there are three boundary conditions that must be satisfied. The reason is that although the position of the first boundary ($V=0$) is known, the position of the second boundary is not. The “free boundary” $V^\ast$ must be determined as part of the solution, which needs the third condition.

Equation $\eqref{eq:condition2}$ has some intuitive interpretations. $V-F(V)$ means that the firm gets the project valued $V$ and yet gives up the opportunity or option to invest $F(V)$. Thus the gain net of the opportunity cost is $V-F(V)$. The critical value $V^\ast$ is where this net gain equals the direct or tangible cost of investment $I$. Equivalently, we could write the equation as $V^\ast=I+F(V^\ast)$, setting the value of the project equal to the full cost of making the investment.

Now we start solving $\eqref{eq:differential}$ using $\eqref{eq:condition1}$-$\eqref{eq:condition3}$.

To satisfy the boundary condition $\eqref{eq:condition1}$, the solution must take the form
\begin{equation}\label{eq:fv}
F(V)=AV^{\beta_1}
\end{equation}
where $A$ is to be determined and $\beta_1$ is a known constant depends on $\sigma$, $\rho$, and $\delta$.

Then utilizing the remaining conditions, we have
\begin{equation}\label{eq:Vstar}
V^\ast={\beta_1\over \beta_1-1}I
\end{equation}
and
\begin{equation}\label{eq:A}
A=(V^\ast-I)/(V^\ast)^{\beta_1}
\end{equation}

Since the second-order homogeneous differential equation $\eqref{eq:differential}$ is linear in the dependent variable $F$ and its derivatives, its general solution can be expressed as a linear combination of any two independent solutions. Substitute $AV^\beta$ in to $\eqref{eq:differential}$, and it provides $\beta$ as a root of the quadratic equation

$${1\over 2}\sigma^2\beta(\beta-1)+(\rho-\delta)\beta-\rho=0$$

which gives two roots $\beta_1>1$ and $\beta_2<0$. Hence, the general solution can be written as

$$F(V)=A_1V^{\beta_1}+A_2V^{\beta_2}$$

where $A_1$ and $A_2$ are to be determined.

In this problem, boundary condition $\eqref{eq:condition1}$ implies $A_2=0$, leaving the solution $\eqref{eq:fv}$.

Problem Setup

Consider the opportunity to invest in a project. The payoff process $X$ of the underlying project follows a geometric Brownian motion,

$${dX_t\over X_t}=bdt+\sigma dW_t$$

Investment in $X$ can be made at any time $t$ at cost $I$, which is known and fixed. The performance functional of the agent, whose time preferences are described by a weighted discount function $h$, which is obtained from some weighting distribution $F$, is given by

$$J(x;u)=\mathbb{E}\left[\int_0^\infty e^{-r\tau_u}dF(r)(X_{\tau_u}-I)\right]$$

The simple net present value rule is to invest as long as $V>I$, but as McDonald and Siegel demonstrated, this is incorrect. Because future values of $X$ are unknown, there is an opportunity cost to investing today.

To ensure this to be a well-posed problem, let $b<\inf\lbrace r\in[0,\infty):F(r)>0\rbrace$, otherwise $w$ can grow to infinity as $t$ goes to infinity.

Investment Behavior Under Weighted Discounting

We solve the Bellman system explicitly when $X$ follows a geometric Brownian Motion and when $G(X)=x-I$ for arbitrary weighted discount functions.

In this case, the Bellman equation is in the following form

$$V(x)=\max_u\left\lbrace G(x),\mathcal{A}V(x)-rV(x)\right\rbrace$$

which can be interpreted as
\begin{equation}\label{eq:equi}
\max\left\lbrace\mathcal{A}V(x)-\int_0^\infty rw(x;r)dF(r),G(x)-V(x)\right\rbrace=0 \tag{$\dagger$}
\end{equation}

The solving scheme is based on the following economic intuition. First, in equilibrium, each member of the group uses the same stopping rule. This yields the value matching condition on $w(x;r)$ for all $r\geq0$. Let denote the triggering boundary above which investment is optimal by $x_\ast$. For $x<x_\ast$ it holds that
\begin{equation}\label{eq:wpartial}
{1\over2}\sigma^2x^2w_{xx}(x;r)+bxw_x(x;r)-rw(x;r)=0
\end{equation}
with boundary conditions $w(0;r)=0$ and $w(x_\ast;r)=x_\ast-I$.

Note: In this setup, $w(x;r)\simeq F(V)$ in previous sector.

Because the smooth-pasting condition is not imposed on $w$ but on the value function $V$, the solution to the ODE $\eqref{eq:wpartial}$ is given by

$$w(x;r)=\left({x\over x_\ast}\right)^{\mu(r)}(x_\ast-I),\quad x<x_\ast$$

where $\mu(r)$ is the positive square root of the fundamental quadratic

$${1\over2}\sigma^2\mu^2+(b-{1\over2}\sigma^2)\mu-r=0$$

and $x_\ast$ is unknown for now.

Second, the equilibrium stopping rule maximizes the weighted average of the $w(x;r)$, i.e.,

$$V(x)=\int_0^\infty\left({x\over x_\ast}\right)^{\mu(r)}dF(r)(x_\ast-I)$$

Therefore, the smooth-pasting condition is given by $\int_0^\infty w_x(x_\ast;r)dF(r)=1$ and can be used to identify the triggering threshold $x_\ast$, i.e.,

$$\left.\int_0^\infty\left({x\over x_\ast}\right)^{\mu(r)-1}{\mu(r)\over x_\ast}dF(r)(x_\ast-I)\,\right|_{x=x_\ast}=1$$

It follows that

$$x_\ast={A\over A-1}I$$

where

$$A=\int_0^\infty\mu(r)dF(r)$$