 Research
 Open access
 Published:
New complexity analysis for primaldual interiorpoint methods for selfscaled optimization problems
Fixed Point Theory and Applications volume 2012, Article number: 213 (2012)
Abstract
A linear optimization problem over a symmetric cone, defined in a Euclidean Jordan algebra and called a selfscaled optimization problem (SOP), is considered. We formulate an algorithm for a largeupdate primaldual interiorpoint method (IPM) for the SOP by using a proximity function defined by a new kernel function, and we obtain the best known complexity results of the largeupdate IPM for the SOP by using the Euclidean Jordan algebra techniques.
MSC:90C51, 90C25, 65K05.
1 Introduction and preliminaries
Primal and dual interiorpoint methods (IPMs) have been well known as the most effective methods for solving wide classes of optimization problems, for example, the linear optimization (LO) problem, the quadratic optimization problem (QOP), the semidefinite optimization (SDO) problem, the secondorder cone optimization (SOCO) problem, and the convex optimization problem (CP).
The socalled barrier update parameter θ in algorithms for IPMs plays an important role in both theory and practice of IPMs. Usually, if θ is a constant independent of the dimension of the problem, then the algorithm is called a largeupdate method. If it depends on the dimension, then the algorithm is said to be a smallupdate method. Largeupdate methods are much more efficient than smallupdate methods in practice [1], but have a worstcase iteration bound. Such a gap between theory and practice has been referred to as irony of IPMs [2]. Recently, many authors have tried to reduce the gap of the worstcase iteration bound between the largeupdate IPM and the smallupdate IPM.
Using selfregular proximity functions instead of a classical logarithmic barrier function, Peng et al. [3–5] improved the complexity of largeupdate IPMs for the LO problem, the SDO problem, and the SOCO problem. Bai et al. [6] introduced a new class of eligible kernel functions. The class was defined by some simple conditions on the kernel function and its derivatives. The best iteration bound for the LO problem, which was given by Bai et al. [6], is \mathcal{O}(\sqrt{n}lognlog\frac{n}{\u03f5}). Recently, Wang et al. [7] obtained the complexity result \mathcal{O}(nlogn/\u03f5) for the SDO problem based on a simple kernel function. Bai and Wang [8] obtained the best known complexity result for the SOCO problem based on a parametric kernel function including the classical logarithmic function, the prototype regular kernel function, and the nonselfregular kernel function. Very recently, using the kernel function \varphi (t)=({t}^{2}1)/2+({e}^{{t}^{q}1}1)/q, Choi and Lee [9, 10] have obtained the complexity results of largeupdate primaldual IPMs for SDO and SOCO, \mathcal{O}(\sqrt{n}{(logn)}^{(q+1)/q}logn/\u03f5) and \mathcal{O}(\sqrt{N}{(logN)}^{(q+1)/q}logN/\u03f5), respectively.
In this paper, we consider a linear optimization problem over a symmetric cone which is defined in a Euclidean Jordan algebra. Nesterov and Todd [11] proposed first this kind of an optimization problem under the name of convex programming for selfscaled cones and established the polynomial complexity of the primaldual interior point method using the socalled NT (NesterovTodd) direction [12]. We call the linear optimization problem over the symmetric cone the selfscaled optimization problem (SOP).
Faybusovich first studied the SOP in view of a Euclidean Jordan algebra and gave a theoretical background for nondegeneracy assumptions and the uniqueness of solutions for Newton systems in IPMs for the SOP [13], presented a shortstep pathfollowing algorithm for a quadratic programming problem defined on the intersection of a symmetric cone with an affine subspace [14] and obtained complexity estimates for a longstep primaldual interiorpoint algorithm for the optimization problem of the minimization of a linear function on a feasible set obtained as the intersection of an affine subspace and a symmetric cone [15]. SOPs include linear optimization problems, semidefinite optimization problems, secondorder optimization problems, and various combinations of these types of problems as special cases. Schmieta and Alizadeh [16] extended primaldual interior point algorithms for LOs, SDOs, and SOCOs to SOPs by using logarithmic barrier functions.
Baes raised an open question in his monograph [17] as follows: The theory of selfregular functions has been created for linear programming by Jiming Peng, Cornelius Roos, and Tamás Terlaky [5]. They subsequently extended it to secondorder programming and semidefinite programming separately using implicitly the aforementioned construction. However, the unified treatment of this theory using the Jordan algebraic framework is not accomplished yet.
Choi and Lee [18] gave primaldual interior point algorithms by using a very simple selfregular function \psi (t)=\frac{1}{2}{(t\frac{1}{t})}^{2}, t>0 for the SOP and gave partial answers for the question of Baes. Very recently, Vieira [19, 20] gave complete answers for the open question of Baes by proving the econvexity property of eligible kernel functions and, in particular, he presented the iteration complexity results for ten eligible kernel functions. Among ten kernel functions in [19], the best iteration complexity for a largeupdate method was obtained for \psi (t)=\frac{{t}^{2}1}{2}+\frac{{t}^{1q}1}{q1} with q=logr, and its iteration complexity is \mathcal{O}(\sqrt{r}logrlog\frac{r}{\u03f5}), which is the best known one.
In this paper, we define a new eligible kernel function \psi (t)=\frac{{t}^{2}1}{2}+\frac{{e}^{p({t}^{q}1)}1}{pq}, p\geqq 1 and q\geqq 1 for t>0, which was modified from the one in [9, 10], and obtain the best known iteration complexity result for the largeupdate IPM of the SOP by using the analysis emphasized on the kernel function and the Euclidean Jordan algebra techniques. In our algorithm, we use the wellknown lemma for the upper bound of the μupdate (see Lemma 3.1) instead of using Theorem 5.4 in [20]. The lemma makes our analysis in the outer while loop easy. We refer to Theorem 4.9 and Proposition 5.6 in [20] for complexity analysis. But we use Proposition 3.1 in [18] obtained from the technique of Sun and Sun [21] instead of using Proposition 5.7 in [20].
This paper is organized as follows. In Section 2, we introduce our kernel functions, formulate the Newton system for the SOP, and present a useful inequality for our proximity function. In Section 3, we give an algorithm for the SOP and calculate an upper bound for the proximity function after μupdate. We calculate an upper bound for difference between proximity functions after one step in inner iterations and then determine our default step size for search directions. We present a worstcase iteration bound for our largeupdate primaldual interior point method for the SOP.
Now, we give definitions and preliminary properties for a Euclidean Jordan algebra which are found in [22] and will be used in the next sections.
Definition 1.1 ([22])
A finitedimensional real vector space V is called an algebra if a bilinear mapping (x,y)\to x\circ y from V\times V to V is defined.
An algebra V is called a Jordan algebra if the following hold:

(i)
commutativity: for all x,y\in V, x\circ y=y\circ x;

(ii)
Jordan’s axiom: for all x,y\in V, {x}^{2}\circ (x\circ y)=x\circ ({x}^{2}\circ y), where {x}^{2}=x\circ x.
A Jordan algebra V is said to be Euclidean if

(iii)
{x}^{2}+{y}^{2}=0\Rightarrow x=y=0, equivalently, there exists an inner product (\cdot \cdot ) on V such that (x\circ yz)=(yx\circ z).
A Jordan algebra V is simple if it does not contain any nontrivial ideal. The Jordan algebra may not be associative, but it is powerassociative, i.e., {x}^{p}\circ {x}^{q}={x}^{p+q}. We assume a Jordan algebra V has an identity element, i.e., there exists e such that x\circ e=e\circ x=x. Since V is finitedimensional, given x\in V, there exists a minimal positive integer k such that the vectors e,x,\dots ,{x}^{k} are linearly dependent. Denote this integer m(x). We define the rank of V as
An element x\in V is said to be invertible if there exists an element y\in \mathbb{R}[x] such that x\circ y=e, where \mathbb{R}[x] is the algebra over ℝ of polynomials in one variable with coefficients in ℝ. It is defined by {x}^{1}. An element v\in V is called idempotent if {v}^{2}=v. For an element x\in V, let L(x) be a linear map of V defined as L(x)y=x\circ y. The cone of squares
is a symmetric cone; the following conditions hold:

(i)
for every pair of x,y\in int\overline{\mathrm{\Omega}}, there is an invertible linear transformation L:V\to V such that L(\overline{\mathrm{\Omega}})=\overline{\mathrm{\Omega}} and L(x)=y;

(ii)
{\overline{\mathrm{\Omega}}}^{\ast}=\overline{\mathrm{\Omega}}, where {\overline{\mathrm{\Omega}}}^{\ast}:=\{y\in V\mid (x,y)\geqq 0,\text{for any}x\in \overline{\mathrm{\Omega}}\}.
Let \mathrm{\Omega}=int\overline{\mathrm{\Omega}}. Then \mathrm{\Omega}=\{{x}^{2}\mid x\in V\text{is invertible}\}=\{x\in V\mid L(x)\text{is positive definite}\}.
Definition 1.2 ([22])
Let {c}_{1},\dots ,{c}_{k}\in V. Then \{{c}_{1},\dots ,{c}_{k}\} is said to be a Jordan frame if {c}_{i}, i=1,\dots ,k are nonzero and cannot be written as a sum of other two idempotents, and the following properties hold:
Theorem 1.1 (Theorem III.1.2 in [22])
For every x\in V, there exist a Jordan frame \{{c}_{1}(x),\dots ,{c}_{r}(x)\} and real numbers {\lambda}_{1}(x),\dots ,{\lambda}_{r}(x) such that
The numbers {\lambda}_{i}(x), for all i=1,\dots ,r, are said to be the eigenvalues of x, and (1) is called the eigenvalue (or spectral) decomposition of x. Now, it is possible to extend the definition of any realvalued function \psi (\cdot ) to elements of the Euclidean Jordan algebra via their eigenvalues:
Particularly, we have some examples as follows:

(i)
Square root: {x}^{1/2}={\lambda}_{1}^{1/2}(x){c}_{1}(x)+\cdots +{\lambda}_{r}^{1/2}(x){c}_{r}(x) if all {\lambda}_{i}(x)\geqq 0.

(ii)
Inverse: {x}^{1}={\lambda}_{1}^{1}(x){c}_{1}(x)+\cdots +{\lambda}_{r}^{1}(x){c}_{r}(x) if all {\lambda}_{i}(x)\ne 0.

(iii)
Square: {x}^{2}={\lambda}_{1}^{2}(x){c}_{1}(x)+\cdots +{\lambda}_{r}^{2}(x){c}_{r}(x).
From the above examples, we know that for x\in \overline{\mathrm{\Omega}}, {\lambda}_{i}({x}^{1/2})={\lambda}_{i}^{1/2}(x) and for x\in \mathrm{\Omega}, {\lambda}_{i}({x}^{1})={\lambda}_{i}^{1}(x). Let us denote by {\psi}^{\prime}(x) the derivative of \psi (x) with respect to {\lambda}_{i}(x):
In the Jordan algebra, we define the determinant of x and the trace of x as follows:
Since V is a Euclidean Jordan algebra, \u3008x,y\u3009:=tr(x\circ y) is a scalar product on V (see Proposition III.1.5 in [22]). The following lemma is called the second Pierce decomposition theorem which will be used in Section 3.
Lemma 1.1 (Theorem IV.2.1 in [22], Theorem 2.6.6 (Second Pierce decomposition theorem) in [17])
Let \{{c}_{1},\dots ,{c}_{r}\} be a Jordan frame of V. If
we have

(i)
V={\u2a01}_{1\leqq i\leqq j\leqq r}{V}_{ij};

(ii)
{V}_{ij}\circ {V}_{kl}=\{0\}, if \{i,j\}\cap \{k,l\}=\mathrm{\varnothing};

(iii)
{V}_{ij}\circ {V}_{jk}\subset {V}_{ik}, if i\ne k;

(iv)
tr({v}_{ik})=0, for {v}_{ik}\in {V}_{ik} if i\ne k.
Consider the following selfscaled optimization problem (SOP):
and its dual problem:
where c,{a}_{1},\dots ,{a}_{m}\in V and b\in {\mathbb{R}}^{m} are given. We call x\in \overline{\mathrm{\Omega}} primal feasible if \u3008{a}_{i},x\u3009={b}_{i} for i=1,\dots ,m. Similarly, (y,s)\in {\mathbb{R}}^{m}\times \overline{\mathrm{\Omega}} is called dual feasible if {\sum}_{i=1}^{m}{y}_{i}{a}_{i}+s=c. Let Ax={(\u3008{a}_{1},x\u3009,\dots ,\u3008{a}_{m},x\u3009)}^{T} for any x\in V. Then A:V\to {\mathbb{R}}^{m} is a linear transformation. Throughout this paper, we assume that A is surjective. Then its adjoint {A}^{T} is injective and {A}^{T}y={\sum}_{i=1}^{m}{y}_{i}{a}_{i}, where y={({y}_{1},\dots ,{y}_{m})}^{T}\in {\mathbb{R}}^{m}. So, we can reformulate (P) and (D) as follows:
and its dual problem:
We can check that weak duality between (P) and (D) holds, that is, inf(\mathrm{P})\geqq sup(\mathrm{D}). From now on, we assume that both (P) and (D) satisfy the interiorpoint condition (IPC), that is, there exists ({x}^{0},{y}^{0},{s}^{0}) such that A{x}^{0}=b, {x}^{0}\in \mathrm{\Omega}, {A}^{T}{y}^{0}+{s}^{0}=c, {s}^{0}\in \mathrm{\Omega}. Then there exists a pair of optimal solutions (x,y,s) of (P) and (D), and inf(\mathrm{P})=sup(\mathrm{D}) [11, 23].
The following lemma is well known [13, 17, 22, 24].
Lemma 1.2 For x,s\in V, the following statements are equivalent:

(i)
x,s\in \overline{\mathrm{\Omega}} and \u3008x,s\u3009=0;

(ii)
x,s\in \overline{\mathrm{\Omega}} and x\circ s=0.
Using Lemma 1.2, we can check (see Proposition 2.1 in [13]) that finding a pair of optimal solutions (x,y,s) of (P) and (D) is equivalent to solving the following Newton system:
The basic idea of primaldual IPMs is to replace the third equation in (4), the socalled complementarity condition for the SOP, by the parameterized system with a positive parameter μ:
For each x\in V, we define the quadratic representation as follows:
Lemma 1.3 ([16])
Let x,s\in \mathrm{\Omega} and p be invertible. Then x\circ s=\mu e if and only if {Q}_{p}x\circ {Q}_{{p}^{1}}s=\mu e.
Proposition 1.1 (Proposition 18 in [16])
If x,s\in \mathrm{\Omega}, then {Q}_{x}s\in \mathrm{\Omega}.
Let x,s\in \mathrm{\Omega}. Then there uniquely exists p\in \mathrm{\Omega} such that {Q}_{{p}^{2}}x=s [25, 26]. So, we can choose p\in \mathrm{\Omega} such that {Q}_{p}x={Q}_{{p}^{1}}s. Such a choice exists and is unique, and leads to the NesterovTodd (NT) method.
From Lemma 1.3, the system (5) becomes
Then, for each \mu >0, the parameterized system (6) has a unique solution (x(\mu ),y(\mu ),s(\mu )) [11, 27], which is called a μcenter of (P) and (D). The set of μcenters, that is, \mathcal{C}=\{(x(\mu ),y(\mu ),s(\mu ))\mid \mu >0\}, is said to be the central path of (P) and (D). Therefore, as μ tends to zero, (x(\mu ),y(\mu ),s(\mu )) converges to a pair of optimal solutions of (P) and (D) [13, 28].
In general, IPMs for the SOP consist of two strategies. The first one, which is called the inner iteration scheme, is to keep the iterative sequence in a certain neighborhood of the central path or to keep the iterative sequence in a certain neighborhood of the μcenter. And the second one, called the outer iteration scheme, is to decrease the parameter μ to {\mu}_{+}:=(1\theta )\mu for some \theta \in (0,1).
2 Proximity functions and search directions
Newton’s method is a wellknown procedure to solve a system of nonlinear equations. Most IPMs for solving the SOP employ different search directions together with suitable strategies for following the central path appropriately.
Assume that a starting point ({x}^{0},{s}^{0}) in a certain neighborhood of the central path corresponding to \mu =1 is available. We then decrease μ to {\mu}_{+}:=(1\theta )\mu for some fixed \theta \in (0,1) and linearize the Newton system for (6) by replacing x, y, s with {x}_{+}:=x+\mathrm{\Delta}x, {y}_{+}:=y+\mathrm{\Delta}y, {s}_{+}:=s+\mathrm{\Delta}s, respectively. Then we get the following system in [16]:
To describe our new search direction, we need more notations:
In this case,
From Proposition 1.1, v\in \mathrm{\Omega}. Hence, L(v) is positive definite. Thus, the system (7) is equivalent to the following system:
We say that the above (dx,\mathrm{\Delta}y,ds) is called the NT search direction for the SOP. Furthermore, \u3008dx,ds\u3009=0, which is coming from the first and second equations of (10) or from the orthogonality of Δx and Δs.
For our IPM, we use the following new eligible kernel function:
Please see the definition of an eligible function in [6]. The new kernel function (11) satisfies
Note that \psi (1)={\psi}^{\prime}(1)=0. Then \psi (t) is determined:
The proximity function (measure) for (P) and (D) is
where \psi (v) is defined by (2). Note that \mathrm{\Psi}(v)=0, if v=e (i.e., x\circ s=\mu e) and \mathrm{\Psi}(v)>0, otherwise. Replacing the righthand side of the last equation in (10) by {\psi}^{\prime}(v), we have the following system from (10):
Let X=\{x\in V\mid \overline{A}x=0\}. Then {X}^{\perp}=\{{\overline{A}}^{T}y\mid y\in {\mathbb{R}}^{m}\}. Hence, the system (14) has a unique solution.We introduce the normbased proximity measure as follows:
The following lemma gives a lower bound of σ in terms of \mathrm{\Psi}(v).
Lemma 2.1 For any v\in \mathrm{\Omega},
Proof Since (11) satisfies 2\psi (t)\leqq {({\psi}^{\prime}(t))}^{2} and {\sigma}^{2}={\sum}_{i=1}^{r}{({\psi}^{\prime}({\lambda}_{i}(v)))}^{2},
This completes the proof. □
Also, our new kernel function (11) satisfies the following exponential convexity property.
Lemma 2.2 Let {t}_{1}>0 and {t}_{2}>0. Then
The following proposition can be found in [20], but for the completeness, we give its proof.
Proposition 2.1 (Theorem 4.9 in [20])
Let Ψ be the proximity function defined in (13), then for any x,s\in \mathrm{\Omega},
Proof Since {Q}_{{x}^{1/2}}s\in \mathrm{\Omega},
By Theorem 3.5 in [20],
and
Thus,
and
Let {\alpha}_{i}={\lambda}_{i}^{1/2}({Q}_{{x}^{1/2}}s) and {\beta}_{i}={\lambda}_{i}^{1/2}(x){\lambda}_{i}^{1/2}(s). Then {\alpha}_{i}>0 and {\beta}_{i}>0. Moreover, since these conditions satisfy the assumptions of Corollary 3.3.10 in [29] and (iii) in Corollary 3.3.10 in [29] with our kernel function (11),
By Lemma 2.2, we obtain the following result:
□
3 Algorithm and its complexity analysis
Now, we explain our algorithm for the largeupdate primaldual IPM for the SOP. Assuming that a starting point in a certain neighborhood of the central path is available, we can set out from this point. Then, we will go to the outer ‘while loop’. If μ satisfies r\mu \geqq \u03f5, then it is reduced by the factor 1\theta, where \theta \in (0,1). Then, we make use of the inner ‘while loop’, and we repeat the procedure until we find iterates that are ‘close’ to (x(\mu ),y(\mu ),s(\mu )), that is, the proximity \mathrm{\Phi}(x,s;\mu )<\tau. Here, we apply Newton’s method targeting at the new μcenters to decide a search direction (\mathrm{\Delta}x,\mathrm{\Delta}y,\mathrm{\Delta}s). We return to the outer ‘while loop’. The whole process is repeated until μ is small enough, say until r\mu <\u03f5.
The choice of the step size α is another crucial issue in the analysis of the algorithm. It has to be taken so that the closeness of the iterates to the current μcenter can improve by a sufficient amount. In the algorithm, the inner ‘while loop’ is called the inner iteration and the outer ‘while loop’ is called the outer iteration. Each outer iteration consists of an update of the parameter μ and a sequence of (one or more) inner iterations. The total number of inner iterations is the worstcase iteration bound for our algorithm.
The algorithm for our largeupdate primaldual IPM for the SOP is given as follows:
3.1 Bound of the proximity function after μupdate
We have \mathrm{\Psi}(v)\leqq \tau before the update of μ with the factor 1\theta at the start of each outer iteration. After updating μ in an outer iteration, the vector v is divided by the factor \sqrt{1\theta}, which in general leads to an increase in the value of \mathrm{\Psi}(v). Then during the inner iteration, the value of \mathrm{\Psi}(v) decreases until it passes the threshold τ.
As we mentioned, our kernel function (11) is eligible. To obtain an upper bound for a μupdated proximity function in each outer iteration in the algorithm, we use the wellknown Lemma 3.1, which can be induced from the decreasing part of the kernel function, instead of using theorems which can be obtained from some properties for eligible functions (for example, Theorem 3.2 in [6] and Theorem 5.4 in [20]). Both of the following lemmas make our analysis in the outer while loop easy. And we will show a theorem that an upper bound for \mathrm{\Psi}(\frac{1}{\sqrt{1\theta}}v) is expressed with \mathrm{\Psi}(v) by using the following two lemmas.
Lemma 3.1 Let \beta \geqq 1. Then
Proof Define {\psi}_{b}(t):=\frac{{e}^{p({t}^{q}1)}1}{pq}. Then {\psi}_{b}(t) is monotonically decreasing in t. So, we can easily obtain
□
Lemma 3.2 For any v\in \mathrm{\Omega}, then
Proof Since \frac{{e}^{p({t}^{q}1)}}{pq} is positive and pq\geqq 1, the kernel function (11) has a lower bound as follows:
This implies \frac{1}{2}{\sum}_{i=1}^{r}{\lambda}_{i}^{2}(v)\leqq \mathrm{\Psi}(v)+2r. □
Theorem 3.1 Let θ be such that 0<\theta <1. Then, for any v\in \mathrm{\Omega},
Proof From Lemma 3.1 with \beta =\frac{1}{\sqrt{1\theta}} and Lemma 3.2,
the last inequality comes from \theta \in (0,1). □
By the assumption \mathrm{\Psi}(v)\leqq \tau just before the update of μ,
We define
Since \tau =\mathcal{O}(r) and \theta =\mathrm{\Theta}(1),
3.2 Determining a default step size
In this section, we compute the feasible step size α such that the proximity function is decreasing and is bound for the decrease during inner iterations; then we give our default step size \overline{\alpha}; \overline{\alpha}={(3(1+3\sigma (1+pq+q){(1+{p}^{1}log3\sigma )}^{(q+1)/q}))}^{1}. We will show that the step size not only keeps the iterates feasible but also gives rise to a sufficiently large decrease in the barrier function \mathrm{\Psi}(v) in each inner iteration. Let us denote the difference between the proximity before and after one step by a function of the step size, that is,
The main task in the rest of this section is to study the decreasing behavior of f(\alpha ).
Now, in equation (16), {v}_{+} and {p}_{+} are determined by x, s in (9) and (8) replaced by {x}_{+}:=x+\alpha \mathrm{\Delta}x, {s}_{+}:=s+\alpha \mathrm{\Delta}s, respectively, which is as follows:
Lemma 3.3 (Proposition II.3.3 in [22])
Let x and s be elements in V. Then

(i)
{({Q}_{x}s)}^{1}={Q}_{{x}^{1}}{s}^{1} if x and s are invertible.

(ii)
{Q}_{{Q}_{s}x}={Q}_{s}{Q}_{x}{Q}_{s}.
Lemma 3.4 ([16])
Let x,s,p\in \mathrm{\Omega}. Then

(i)
{Q}_{{x}^{1/2}}s and {Q}_{{s}^{1/2}}x have the same eigenvalues.

(ii)
{Q}_{{x}^{1/2}}s and {Q}_{{({Q}_{p}x)}^{1/2}}({Q}_{{p}^{1}}s) have the same eigenvalues.
The following proposition was given by Vieira in [20] (see Proposition 5.6 in [20]), but we provide its proof using Lemma 3.3 and Lemma 3.4.
Proposition 3.1 Let Ψ be the proximity function defined in (13). Then we have
Proof From {Q}_{{p}^{1}}s={Q}_{p}x and (i) in Lemma 3.4, we know that {Q}_{p}x and {Q}_{{x}^{1/2}}{p}^{2} have the same eigenvalues. By the definition of p and (ii) in Lemma 3.3,
Then we can find {Q}_{{p}_{+}^{1}}{s}_{+} and {({Q}_{{x}_{+}^{1/2}}{s}_{+})}^{1/2} have the same eigenvalues. Here, \sqrt{\mu}{v}_{+}={Q}_{{p}_{+}^{1}}{s}_{+}. We know that {x}_{+}=\sqrt{\mu}{Q}_{{p}^{1}}(v+\alpha \phantom{\rule{0.2em}{0ex}}dx) and {s}_{+}=\sqrt{\mu}{Q}_{p}(v+\alpha \phantom{\rule{0.2em}{0ex}}ds), by the definition (8) and by (ii) in Lemma 3.4, then {({Q}_{{x}_{+}^{1/2}}{s}_{+})}^{1/2}=\sqrt{\mu}{(Q{({Q}_{{p}^{1}}(v+\alpha \phantom{\rule{0.2em}{0ex}}dx))}^{1/2}({Q}_{p}(v+\alpha \phantom{\rule{0.2em}{0ex}}ds)))}^{1/2} and \sqrt{\mu}{({Q}_{{(v+\alpha \phantom{\rule{0.2em}{0ex}}dx)}^{1/2}}(v+\alpha \phantom{\rule{0.2em}{0ex}}ds))}^{1/2} have the same eigenvalues. Therefore, the proximity function satisfies the equality. □
Then Proposition 2.1 and Proposition 3.1 imply the following inequality:
So, we can define {f}_{1}(\alpha ):
To facilitate the forthcoming analysis, we also define, for any x\in V,
The following lemma is obtained from Lemma 14 in [16] so that we can get the common lower bound of eigenvalues of v+\alpha \phantom{\rule{0.2em}{0ex}}dx and v+\alpha \phantom{\rule{0.2em}{0ex}}ds, where α satisfies v+\alpha \phantom{\rule{0.2em}{0ex}}dx\in \mathrm{\Omega} and v+\alpha \phantom{\rule{0.2em}{0ex}}ds\in \mathrm{\Omega}.
Lemma 3.5 For any \alpha \in (0,\frac{{\lambda}_{min}(v)}{\sigma}),
where σ is a number defined in (15).
Proof Let α be a fixed number in (0,\frac{{\lambda}_{min}(v)}{\sigma}). From Lemma 14 in [16],
Since \sigma \geqq \parallel dx\parallel, we have
Similarly, we obtain
□
The proof of the following proposition can be found in [18], but for the completeness, we give its detailed proof.
Proposition 3.2 ([18])
Suppose that the functions \psi (x) and \mathrm{\Psi}(x) are defined by (2) and (13), respectively. Then, for any \alpha \in (0,\frac{{\lambda}_{min}(v)}{\sigma}),
where
Proof Using Lemma 3.1 in [21], we have
Then we have
Then from Baes [17, 30] we know that tr({c}_{i}(v+\alpha \phantom{\rule{0.2em}{0ex}}dx))=1, and hence, from the definition (3), we get
Thus, we have
So, the first equality holds.
For the second inequality, we will use (17) by replacing ψ by {\psi}^{\prime}.
Here, let dx={\sum}_{j=1}^{r}{\lambda}_{j}(dx){c}_{j}(dx). Then we have
Since {c}_{i}(v+\alpha \phantom{\rule{0.2em}{0ex}}dx) and {c}_{j}(dx) are in \overline{\mathrm{\Omega}} which is a selfdual cone, then
Furthermore, {\sum}_{j=1}^{r}tr({c}_{i}(v+\alpha \phantom{\rule{0.2em}{0ex}}dx)\circ {c}_{j}(dx))=tr({c}_{i}(v+\alpha \phantom{\rule{0.2em}{0ex}}dx))=1. Then we have
Since {\sum}_{i=1}^{r}tr({c}_{i}(v+\alpha \phantom{\rule{0.2em}{0ex}}dx)\circ {c}_{j}(dx))=tr({c}_{j}(dx))=1, we have
Now, we decompose dx along Lemma 1.1 such as dx={\sum}_{1\leqq i\leqq k\leqq r}d{x}_{ik} for the system of idempotent \{{c}_{1}(v+\alpha \phantom{\rule{0.2em}{0ex}}dx),\dots ,{c}_{r}(v+\alpha \phantom{\rule{0.2em}{0ex}}dx)\}. Then, for j<l,
This means, for each j<l,
Moreover, we have,
Since for each i, {(tr({c}_{i}(v+\alpha \phantom{\rule{0.2em}{0ex}}dx)\circ dx))}^{2} are nonnegative and for each j, l with j<l, tr(({c}_{j}(v+\alpha \phantom{\rule{0.2em}{0ex}}dx)\circ dx)\circ ({c}_{l}(v+\alpha \phantom{\rule{0.2em}{0ex}}dx)\circ dx)) are nonnegative, we get from (18) and (19)
Similarly,
From the definition of {f}_{1}(\alpha ),
Thus, we have the conclusion. □
The next result presents an upper bound for the second derivative of {f}_{1}(\alpha ) which is usable for establishing the polynomial complexity of the algorithm.
Proposition 3.3 For any \alpha \in (0,\frac{{\lambda}_{min}(v)}{\sigma}),
Proof Since {\psi}^{\u2033}(t) is a decreasing function on t\in (0,\mathrm{\infty}), using Lemma 3.5 and the mean value theorem, we have
and
Thus, by Proposition 3.2,
□
We can easily check that {f}_{1}(0)=0 and {f}_{1}^{\prime}(0)=\frac{{\sigma}^{2}}{2}. By Proposition 3.3, we obtain an upper bound {f}_{2}(\alpha ) for {f}_{1}(\alpha ) as follows:
Note that {f}_{2}(0)=0. Furthermore, since {f}_{2}^{\prime}(\alpha )=\frac{{\sigma}^{2}}{2}+\frac{3\sigma}{2}({\psi}^{\prime}({\lambda}_{min}(v)){\psi}^{\prime}({\lambda}_{min}(v)\alpha \sigma )), we have {f}_{2}^{\prime}(0)=\frac{{\sigma}^{2}}{2} which is the same value of {f}_{1}^{\prime}(0), and {f}_{2}^{\u2033}(\alpha )=\frac{3{\sigma}^{2}}{2}{\psi}^{\u2033}({\lambda}_{min}(v)\alpha \sigma ) which is increasing on \alpha \in [0,\frac{{\lambda}_{min}(v)}{\sigma}). Using {f}_{1}^{\prime}(0)={f}_{2}^{\prime}(0) and {f}_{1}^{\u2033}(\alpha )\leqq {f}_{2}^{\u2033}(\alpha ), we can easily check that
This relation gives that
To compute the feasible step size α such that the proximity measure is decreasing when we take a new iterate for fixed μ, we want to calculate the step size α which satisfies that {f}_{2}^{\prime}(\alpha )\leqq 0 holds with α as large as possible. Since {f}_{2}^{\u2033}(\alpha )>0, that is, {f}_{2}^{\prime}(\alpha ) is monotonically increasing at α, the largest possible value at α satisfying {f}_{2}^{\prime}(\alpha )\leqq 0 occurs when {f}_{2}^{\prime}(\alpha )=0, that is,
Since {\psi}^{\u2033}(t) is monotonically decreasing, the derivative of the lefthand side in (20) with respect to {\lambda}_{min}(v) is
So, the lefthand side in (20) is decreasing at {\lambda}_{min}(v). This implies that if {\lambda}_{min}(v) becomes smaller, then α gets smaller with fixed σ. Note that
and the equality is true if and only if {\lambda}_{min}(v) is the only coordinate in ({\lambda}_{1}(v),\dots ,{\lambda}_{r}(v)) which is different from 1 and {\lambda}_{min}(v)<1, that is, {\psi}^{\prime}({\lambda}_{min}(v))<0. Hence, the worse situation for the largest step size occurs when {\lambda}_{min}(v) satisfies
In that case, the largest α satisfying (20) is minimal. For our purpose, we need to deal with the worse case, and so we assume that (21) holds.
From now on, we denote that \rho :[0,\mathrm{\infty})\to (0,1] is the inverse function of the restriction of {\psi}^{\prime}(t) in the interval (0,1]. Then (21) implies
By using (20) and (21), we immediately obtain
By the definition of ρ and (22), the largest step size α of the worse case is given as follows:
For the purpose of finding an upper bound of f(\alpha ), we need a default step size \overline{\alpha} that is the lower bound of the {\alpha}^{\ast} and consists of σ.
Lemma 3.6 Let \sigma \geqq 1. Then, for 0<t\leqq \rho (\frac{4}{3}\sigma ),
Proof From {\psi}^{\prime}(t)=t{t}^{q1}\cdot {e}^{p({t}^{q}1)}, let {\psi}_{b}^{\prime}(t)={t}^{q1}\cdot {e}^{p({t}^{q}1)} and let \underline{\rho}:[1,\mathrm{\infty})\to (0,1] denote the inverse function of the restriction of {\psi}_{b}^{\prime}(t) to the interval (0,1]. Let \rho (\frac{4}{3}\sigma )=\tilde{t}. Then 0<\tilde{t}\leqq 1 and \frac{4}{3}\sigma ={\psi}^{\prime}(\tilde{t})=\tilde{t}{\psi}_{b}^{\prime}(\tilde{t}). So, {\psi}_{b}^{\prime}(\tilde{t})=\tilde{t}+\frac{4}{3}\sigma \leqq 1+2\sigma \leqq 3\sigma. Since \underline{\rho} is a decreasing function, (\rho (\frac{4}{3}\sigma )=\tilde{t}=)\underline{\rho}({\psi}_{b}^{\prime}(\tilde{t}))\geqq \underline{\rho}(3\sigma ). Let \underline{\rho}(3\sigma )=\stackrel{\u02c6}{t}. Then
implies
the last inequality comes from \stackrel{\u02c6}{t}\in (0,1] and (25). □
Now, we present a lower bound of the value of {\alpha}^{\ast}.
Theorem 3.2 Let {\alpha}^{\ast} be as defined in (23). Then
Proof Since {\psi}^{\prime}(\rho (\sigma ))=\sigma, taking the derivative of σ at both sides, we get
Moreover, we have
where the inequality follows from \sigma \leqq \xi \leqq \frac{4}{3}\sigma and ρ and {\psi}^{\u2033} are monotonically decreasing. Also, by Lemma 3.6, we can complete the proof. □
For using \overline{\alpha} as the default step size in the algorithm for the SOP, define the \overline{\alpha} as follows:
We will use \overline{\alpha} as the default step size in our algorithm.
3.3 Decrease of the proximity function during an inner iteration
Now, we show that our proximity function Ψ with our default step size \overline{\alpha} is decreasing. It can be easily established by using the following result.
Lemma 3.7 ([4])
Let h(t) be a twice differentiable convex function with h(0)=0, {h}^{\prime}(0)<0 and let h(t) attain its (global) minimum at {t}^{\ast}>0. If {h}^{\u2033}(t) is increasing for t\in [0,{t}^{\ast}], then
Since {f}_{2}(\alpha ) satisfies assumptions of the above lemma,
Since {f}_{2}^{\prime}(0)=\frac{{\sigma}^{2}}{2}, we can obtain the upper bound for the decreasing value of the proximity in the inner iteration by Lemma 3.7.
Theorem 3.3 Let \overline{\alpha} be the default step size as defined in (26). Then we have
Proof Since {f}_{2}^{\prime}(0)=\frac{{\sigma}^{2}}{2} and \overline{\alpha}\in [0,{\alpha}^{\ast}], we have
This expresses the decrease in one inner iteration in terms of σ. Since the decrease depends monotonically on σ, we can express the decrease in terms of \mathrm{\Psi}=\mathrm{\Psi}(v) by Lemma 2.1 as follows:
where the inequality follows from {\mathrm{\Psi}}_{0}\geqq \mathrm{\Psi}\geqq \tau \geqq 1. The theorem is satisfied. □
3.4 Iteration bound
We need to count how many inner iterations are required to return to the situation where \mathrm{\Psi}(v)\leqq \tau after a μupdate. We denote the value of \mathrm{\Psi}(v) after μupdate as {\mathrm{\Psi}}_{0}; the subsequent values in the same outer iteration are denoted as {\mathrm{\Psi}}_{k}, k=1,\dots . If K denotes the total number of inner iterations in the outer iteration, then we have
and according to Theorem 3.3,
At this stage, we invoke Lemma 14 in [4].
Lemma 3.8 ([4])
Let {t}_{0},{t}_{1},\dots ,{t}_{K} be a sequence of positive numbers such that
where \beta >0 and 0<\gamma \leqq 1. Then
Letting {t}_{k}={\mathrm{\Psi}}_{k}, \beta =\frac{1}{6+18\sqrt{2}(1+pq+q){(1+\frac{1}{p}log3\sqrt{2{\mathrm{\Psi}}_{0}})}^{\frac{q+1}{q}}} and \gamma =\frac{1}{2}, we can get the following lemma from Lemma 3.8.
Lemma 3.9 Let K be the total number of inner iterations in the outer iteration. Then we have
where {\mathrm{\Psi}}_{0} is the value of \mathrm{\Psi}(v) after the μupdate in the outer iteration.
Now, we estimate the total number of iterations of our algorithm.
Theorem 3.4 If \tau \geqq 1 and 0<\theta <1, the total number of iterations is not more than
Proof In the algorithm, r\mu \geqq \u03f5, {\mu}_{k}:={(1\theta )}^{k}{\mu}_{0} and {\mu}_{0}=1. By simple computation, we have
Therefore, the number of outer iterations is bounded above by
Multiplication of this result by the number in the above lemma satisfies the theorem. □
Since {\mathrm{\Psi}}_{0}^{1/2}=\mathcal{O}(\sqrt{r}), if we take p=\mathcal{O}(logr) and q=1, then we can get the best known upper bound for the total number of inner iterations in the outer iteration is
Also, we take for θ a constant (not depending on r), namely \frac{1}{\theta}=\mathrm{\Theta}(1). With \tau =\mathcal{O}(r), the best complexity of the primaldual interiorpoint method for a linear optimization problem based on our new proximity function with p=logr and q=1 is given by
References
Andersen ED, Gondzio J, Mészáros C, Xu X: Implementation of interior point methods for large scale linear programming. In Interior Point Methods of Mathematical Programming. Edited by: Terlaky T. Kluwer Academic, Dordrecht; 1996:189–252.
Renegar J MPS/SIAM Ser. Optim. In A Mathematical View of InteriorPoint Methods in Convex Optimization. SIAM, Philadelphia; 2001.
Peng J, Roos C, Terlaky T: Primaldual interiorpoint methods for secondorder conic optimization based on selfregular proximities. SIAM J. Optim. 2002, 13: 179–203. 10.1137/S1052623401383236
Peng J, Roos C, Terlaky T: Selfregular functions and new search directions for linear and semidefinite optimization. Math. Program. 2002, 93: 129–171. 10.1007/s101070200296
Peng J, Roos C, Terlaky T: SelfRegularity: A New Paradigm for PrimalDual InteriorPoint Algorithms. Princeton University Press, Princeton; 2002.
Bai YQ, Ghami ME, Roos C: A comparative study of kernel functions for primaldual interiorpoint algorithms in linear optimization. SIAM J. Optim. 2004, 15: 101–128. 10.1137/S1052623403423114
Wang GQ, Bai YQ, Roos C: Primaldual interiorpoint algorithms for semidefinite optimization based on a simple kernel function. J. Math. Model. Algorithms 2005, 4: 409–433. 10.1007/s1085200535613
Bai YQ, Wang GQ: Primaldual interiorpoint algorithms for secondorder cone optimization based on a new parametric kernel function. Acta Math. Sin. Engl. Ser. 2007, 23: 2027–2042. 10.1007/s101140070967z
Choi BK, Lee GM: On complexity analysis of the primaldual interiorpoint methods for semidefinite optimization problem based on a new proximity function. Nonlinear Anal., Theory Methods Appl. 2009, 71: e2628e2640. 10.1016/j.na.2009.05.078
Choi BK, Lee GM: On complexity analysis of the primaldual interiorpoint method for secondorder cone optimization problem. J. Korean Soc. Ind. Appl. Math. 2010, 14: 93–111.
Nesterov YE, Tood M: Primaldual interiorpoint methods for selfscaled cones. SIAM J. Optim. 1998, 8: 324–364. 10.1137/S1052623495290209
Muramatsu M: On a commutative class of search directions for linear programming over symmetric cones. J. Optim. Theory Appl. 2002, 112: 595–625. 10.1023/A:1017920200889
Faybusovich L: Linear systems in Jordan algebras and primaldual interiorpoint algorithms. J. Comput. Appl. Math. 1997, 86: 149–175. 10.1016/S03770427(97)001532
Faybusovich L: Euclidean Jordan and interiorpoint algorithms. Positivity 1997, 1: 331–357. 10.1023/A:1009701824047
Faybusovich L, Arana R: A longstep primaldual algorithm for symmetric programming problems. Syst. Control Lett. 2001, 43: 3–7. 10.1016/S01676911(01)000925
Schmieta S, Alizadeh F: Extensions of primaldual interiorpoint algorithms to symmetric cones. Math. Program. 2003, 96: 409–438. 10.1007/s101070030380z
Baes, M: Optimization Methods for Convex Symmetric Problems. Monograph, April (2007)
Choi, BK, Lee, GM: Complexity analysis for primaldual interiorpoint methods for selfscaled optimization problems (submitted)
Vieira, MVC: Jordan algebraic approach to symmetric optimization. Electrical Engineering, Mathematics and Computer Science, Delft University of Technology, The Netherlands (2007)
Vieira MVC: Interiorpoint methods based on kernel functions for symmetric optimization. Optim. Methods Softw. 2011, 27: 513–537.
Sun D, Sun J: Lowner’s operator and spectral functions in Euclidean Jordan algebras. Math. Oper. Res. 2008, 33: 421–445. 10.1287/moor.1070.0300
Faraut J, Korányi A: Analysis on Symmetric Cones. Oxford University Press, London; 1994.
Nesterov YE, Nemirovskii A SIAM Stud. Appl. Math. 13. In Interior Point Polynomial Algorithms in Convex Programming. SIAM, Philadelphia; 1994.
Gowda MS, Szajder R, Tao J: Some Pproperties for linear transformations on Euclidean Jordan algebras. Linear Algebra Appl. 2004, 393: 203–232.
Faybusovich L: A Jordanalgebraic approach to potentialreduction algorithms. Math. Z. 2002, 239: 117–129. 10.1007/s002090100286
Lim Y: Applications of geometric means on symmetric cones. Math. Ann. 2001, 319: 457–468. 10.1007/PL00004442
Tunçel L: Potential reduction and primaldual methods. In Handbook of Semidefinite Programming Theory, Algorithms and Applications. Edited by: Wolkowicz H, Saigal R, Vandenberghe L. Kluwer Academic, Boston; 2000:235–265.
Alizadeh F, Schmieta S: Symmetric cones, potential reduction methods and wordbyword extensions. In Handbook of Semidefinite Programming, Theory, Algorithms and Applications. Edited by: Wolkowicz H, Saigal R, Vandenberghe L. Kluwer Academic, Boston; 2000:195–233.
Horn RA, Johnson CR: Topics in Matrix Analysis. Cambridge University Press, Cambridge; 1991.
Baes M: Convexity and differentiability properties of spectral functions and spectral mappings on Euclidean Jordan algebras. Linear Algebra Appl. 2007, 422: 664–700. 10.1016/j.laa.2006.11.025
Acknowledgements
This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MEST) (No. 20120006236).
Author information
Authors and Affiliations
Corresponding author
Additional information
Competing interests
The authors declare that they have no competing interests.
Authors’ contributions
The authors, together discussed and solved the problems in the manuscript. All authors read and approved the final manuscript.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License (https://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
About this article
Cite this article
Choi, B.K., Lee, G.M. New complexity analysis for primaldual interiorpoint methods for selfscaled optimization problems. Fixed Point Theory Appl 2012, 213 (2012). https://doi.org/10.1186/168718122012213
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/168718122012213