-
Projection, Degeneracy, and Singularity Degree for Spectrahedra
Authors:
Haesol Im,
Woosuk L. Jung,
Walaa M. Moursi,
David Torregrosa-Belin,
Henry Wolkowicz
Abstract:
Facial reduction, FR, is a regularization technique for convex programs where the strict feasibility constraint qualification, CQ, fails. Though this CQ holds generically, failure is pervasive in applications such as semidefinite relaxations of hard discrete optimization problems. In this paper we relate FR to the analysis of the convergence behaviour of a semi-smooth Newton root finding method fo…
▽ More
Facial reduction, FR, is a regularization technique for convex programs where the strict feasibility constraint qualification, CQ, fails. Though this CQ holds generically, failure is pervasive in applications such as semidefinite relaxations of hard discrete optimization problems. In this paper we relate FR to the analysis of the convergence behaviour of a semi-smooth Newton root finding method for the projection onto a spectrahedron, i.e., onto the intersection of a linear manifold and the semidefinite cone. In the process, we derive and use an elegant formula for the projection onto a face of the semidefinite cone. We show further that the ill-conditioning of the Jacobian of the Newton method near optimality characterizes the degeneracy of the nearest point in the spectrahedron. We apply the results, both theoretically and empirically, to the problem of finding nearest points to the sets of: (i) correlation matrices or the elliptope; and (ii) semidefinite relaxations of permutation matrices or the vontope, i.e., the feasible sets for the semidefinite relaxations of the max-cut and quadratic assignment problems, respectively.
△ Less
Submitted 8 July, 2024;
originally announced July 2024.
-
Single Element Error Correction/ in a Euclidean Distance Matrix
Authors:
Abdo Alfakih,
Woosuk L. Jung,
Henry Wolkowicz,
Tina Xu
Abstract:
We consider the \emph{exact} error correction of a noisy Euclidean distance matrix, EDM, where the elements are the squared distances between $n$ points in $R^d$. For our problem we are given two facts: (i) the embedding dimension, $d$, (ii) \emph{exactly one} distance in the data is corrupted by \emph{nonzero noise}. But we do \underline{not} know the magnitude nor position of the noise. Thus the…
▽ More
We consider the \emph{exact} error correction of a noisy Euclidean distance matrix, EDM, where the elements are the squared distances between $n$ points in $R^d$. For our problem we are given two facts: (i) the embedding dimension, $d$, (ii) \emph{exactly one} distance in the data is corrupted by \emph{nonzero noise}. But we do \underline{not} know the magnitude nor position of the noise. Thus there is a combinatorial element to the problem. We present three solution techniques. These use three divide and conquer strategies in combination with three versions of facial reduction that use: exposing vectors, facial vectors, and Gale transforms. This sheds light on the connections between the various forms of facial reduction related to Gale transforms. Our highly successful empirics confirm the success of these approaches as we can solve huge problems of the order of $100,000$ nodes in approximately one minute to machine precision. \\Our algorithm depends on identifying whether a principal submatrix of the \EDM contains the corrupted element. We provide a theorem for doing this that is related to the existing results for identifying \emph{yielding} elements, i.e.,~we provide a characterization for guaranteeing the perturbed EDM remains an EDM with embedding dimension $d$. The characterization is particularly simple in the $d=2$ case. \\In addition, we characterize when the intuitive approach of the nearest EDM problem, solves our problem. In fact, we show that this happens if, and only if, the original distance element is $0$, degenerate, and the perturbation is negative.
△ Less
Submitted 22 June, 2024;
originally announced June 2024.
-
Exact Solutions for the NP-hard Wasserstein Barycenter Problem using a Doubly Nonnegative Relaxation and a Splitting Method
Authors:
Abdo Alfakih,
Jeffery Cheng,
Woosuk L. Jung,
Walaa M. Moursi,
Henry Wolkowicz
Abstract:
The simplified Wasserstein barycenter problem consists in selecting one point from $k$ given sets, each set consisting of $n$ points, with the aim of minimizing the sum of distances to the barycenter of the $k$ points chosen. This problem is known to be NP-hard. We compute the Wasserstein barycenter by exploiting the Euclidean distance matrix structure to obtain a facially reduced doubly nonnegati…
▽ More
The simplified Wasserstein barycenter problem consists in selecting one point from $k$ given sets, each set consisting of $n$ points, with the aim of minimizing the sum of distances to the barycenter of the $k$ points chosen. This problem is known to be NP-hard. We compute the Wasserstein barycenter by exploiting the Euclidean distance matrix structure to obtain a facially reduced doubly nonnegative relaxation (DNN). The facial reduction provides a natural splitting for applying the symmetric alternating directions method of multipliers (sADMM) to the DNN relaxation. The sADMM method exploits structure in the subproblems to find strong upper and lower bounds.
The purpose of this paper is twofold. First we want to illustrate the strength of this DNN relaxation with a splitting approach. Our numerical tests then illustrate the surprising success on random problems, as we generally, efficiently, find the provable exact solution of this NP-hard problem. Comparisons with current commercial software illustrate this surprising efficiency. However, we demonstrate and prove that there is a duality gap for problems with enough multiple optimal solutions, and that this arises from problems with highly symmetrized structure.
△ Less
Submitted 8 November, 2023;
originally announced November 2023.
-
The $ω$-Condition Number for Optimal Preconditioning and Low Rank Generalized Jacobian Updating
Authors:
Woosuk L. Jung,
David Torregrosa-Belén,
Henry Wolkowicz
Abstract:
Preconditioning is essential in iterative methods for solving linear systems. It is also the implicit objective in updating approximations of Jacobians in optimization methods, e.g., in quasi-Newton methods. Motivated by the latter, we study a nonclassic matrix condition number, the $ω$-condition number. We do this in the context of optimal conditioning for: (i) our application to low rank updatin…
▽ More
Preconditioning is essential in iterative methods for solving linear systems. It is also the implicit objective in updating approximations of Jacobians in optimization methods, e.g., in quasi-Newton methods. Motivated by the latter, we study a nonclassic matrix condition number, the $ω$-condition number. We do this in the context of optimal conditioning for: (i) our application to low rank updating of generalized Jacobians; (ii) iterative methods for linear systems: (iia) clustering of eigenvalues and (iib) convergence rates.
For a positive definite matrix, the $ω$-condition measure is the ratio of the arithmetic and geometric means of the eigenvalues. In particular, our applications concentrate on linear systems with low rank updates of ill-conditioned positive definite matrices. These systems arise in the context of nonsmooth Newton methods using generalized Jacobians. We are able to use optimality conditions and derive explicit formulae for $ω$-optimal preconditioners and preconditioned updates. Connections to partial Cholesky sparse preconditioners are made.
Evaluating or estimating the classical condition number $κ$ can be expensive. We show that the $ω$-condition number can be evaluated explicitly following a Cholesky or LU factorization. Moreover, the simplicity of $ω$ allows for the derivation of formulae for optimal preconditioning in various scenarios, i.e., this avoids the need for expensive algorithmic calculations. Our empirics show that $ω$ estimates the actual condition of a linear system significantly better. Moreover, our empirical results show a significant decrease in the number of iterations required for a requested accuracy in the residual during an iterative method, i.e., these results confirm the efficacy of using the $ω$-condition number compared to the classical condition number.
△ Less
Submitted 23 June, 2024; v1 submitted 25 August, 2023;
originally announced August 2023.
-
Regularized Nonsmooth Newton Algorithms for Best Approximation
Authors:
Yair Censor,
Walaa M. Moursi,
Tyler Weames,
Henry Wolkowicz
Abstract:
We consider the problem of finding the best approximation point from a polyhedral set, and its applications, in particular to solving large-scale linear programs. The classical projection problem has many various and many applications. We study a regularized nonsmooth Newton type solution method where the Jacobian is singular; and we compare the computational performance to that of the classical p…
▽ More
We consider the problem of finding the best approximation point from a polyhedral set, and its applications, in particular to solving large-scale linear programs. The classical projection problem has many various and many applications. We study a regularized nonsmooth Newton type solution method where the Jacobian is singular; and we compare the computational performance to that of the classical projection method of Halperin-Lions-Wittmann-Bauschke (HLWB).
We observe empirically that the regularized nonsmooth method significantly outperforms the HLWB method. However, the HLWB has a convergence guarantee while the nonsmooth method is not monotonic and does not guarantee convergence due in part to singularity of the generalized Jacobian.
Our application to solving large-scale linear programs uses a parametrized projection problem. This leads to a \emph{stepping stone external path following} algorithm. Other applications are finding triangles from branch and bound methods, and generalized constrained linear least squares. We include scaling methods that improve the efficiency and robustness.
△ Less
Submitted 8 June, 2023; v1 submitted 26 December, 2022;
originally announced December 2022.
-
Singularity degree of non-facially exposed faces
Authors:
Fei Wang,
Henry Wolkowicz
Abstract:
In this paper, we study the facial structure of the linear image of a cone. We define the singularity degree of a face of a cone to be the minimum number of steps it takes to expose it using exposing vectors from the dual cone. We show that the singularity degree of the linear image of a cone is exactly the number of facial reduction steps to obtain the minimal face in a corresponding primal conic…
▽ More
In this paper, we study the facial structure of the linear image of a cone. We define the singularity degree of a face of a cone to be the minimum number of steps it takes to expose it using exposing vectors from the dual cone. We show that the singularity degree of the linear image of a cone is exactly the number of facial reduction steps to obtain the minimal face in a corresponding primal conic optimization problem. This result generalizes the relationship between the complexity of general facial reduction algorithms and facial exposedness of conic images under a linear transform by Drusvyatskiy, Pataki and Wolkowicz to arbitrary singularity degree. We present our results in the original form and also in its nullspace form. As a by-product, we show that frameworks underlying a chordal graph have at most one level of stress matrix.
△ Less
Submitted 28 December, 2022; v1 submitted 1 November, 2022;
originally announced November 2022.
-
Revisiting Degeneracy, Strict Feasibility, Stability, in Linear Programming
Authors:
Jiyoung Im,
Henry Wolkowicz
Abstract:
Currently, the simplex method and the interior point method are indisputably the most popular algorithms for solving linear programs, LPs. Unlike general conic programs, LPs with a finite optimal value do not require strict feasibility in order to establish strong duality. Hence strict feasibility is seldom a concern, even though strict feasibility is equivalent to stability and a compact dual opt…
▽ More
Currently, the simplex method and the interior point method are indisputably the most popular algorithms for solving linear programs, LPs. Unlike general conic programs, LPs with a finite optimal value do not require strict feasibility in order to establish strong duality. Hence strict feasibility is seldom a concern, even though strict feasibility is equivalent to stability and a compact dual optimal set. This lack of concern is also true for other types of degeneracy of basic feasible solutions in LP. In this paper we discuss that the specific degeneracy that arises from lack of strict feasibility necessarily causes difficulties in both simplex and interior point methods. In particular, we show that the lack of strict feasibility implies that every basic feasible solution, BFS, is degenerate; thus conversely, the existence of a nondegenerate BFS implies that strict feasibility (regularity) holds. We prove the results using facial reduction and simple linear algebra. In particular, the facially reduced system reveals the implicit non-surjectivity of the linear map of the equality constraint system. As a consequence, we emphasize that facial reduction involves two steps where, the first guarantees strict feasibility, and the second recovers full row rank of the constraint matrix. This illustrates the implicit singularity of problems where strict feasibility fails, and also helps in obtaining new efficient techniques for preproccessing. We include an efficient preprocessing method that can be performed as an extension of phase-I of the two-phase simplex method. We show that this can be used to avoid the loss of precision for many well known problem sets in the literature, e.g., the NETLIB problem set.
△ Less
Submitted 9 January, 2023; v1 submitted 5 March, 2022;
originally announced March 2022.
-
A Semismooth Newton-Type Method for the Nearest Doubly Stochastic Matrix Problem
Authors:
Hao Hu,
Haesol Im,
Xinxin Li,
Henry Wolkowicz
Abstract:
We study a semismooth Newton-type method for the nearest doubly stochastic matrix problem where both differentiability and nonsingularity of the Jacobian can fail. The optimality conditions for this problem are formulated as a system of strongly semismooth functions. We show that the so-called local error bound condition does not hold for this system. Thus the guaranteed convergence rate of Newton…
▽ More
We study a semismooth Newton-type method for the nearest doubly stochastic matrix problem where both differentiability and nonsingularity of the Jacobian can fail. The optimality conditions for this problem are formulated as a system of strongly semismooth functions. We show that the so-called local error bound condition does not hold for this system. Thus the guaranteed convergence rate of Newton-type methods is at most superlinear. By exploiting the problem structure, we construct a modified two step semismooth Newton method that guarantees a nonsingular Jacobian matrix at each iteration, and that converges to the nearest doubly stochastic matrix quadratically. To the best of our knowledge, this is the first Newton-type method which converges $Q$-quadratically in the absence of the local error bound condition.
△ Less
Submitted 20 July, 2021;
originally announced July 2021.
-
Robust Interior Point Method for Quantum Key Distribution Rate Computation
Authors:
Hao Hu,
Jiyoung Im,
Jie Lin,
Norbert Lütkenhaus,
Henry Wolkowicz
Abstract:
Security proof methods for quantum key distribution, QKD, that are based on the numerical key rate calculation problem, are powerful in principle. However, the practicality of the methods are limited by computational resources and the efficiency and accuracy of the underlying algorithms for convex optimization. We derive a stable reformulation of the convex nonlinear semidefinite programming, SDP,…
▽ More
Security proof methods for quantum key distribution, QKD, that are based on the numerical key rate calculation problem, are powerful in principle. However, the practicality of the methods are limited by computational resources and the efficiency and accuracy of the underlying algorithms for convex optimization. We derive a stable reformulation of the convex nonlinear semidefinite programming, SDP, model for the key rate calculation problems. We use this to develop an efficient, accurate algorithm. The stable reformulation is based on novel forms of facial reduction, FR, for both the linear constraints and nonlinear quantum relative entropy objective function. This allows for a Gauss-Newton type interior-point approach that avoids the need for perturbations to obtain strict feasibility, a technique currently used in the literature. The result is high accuracy solutions with theoretically proven lower bounds for the original QKD from the FR stable reformulation. This provides novel contributions for FR for general SDP. We report on empirical results that dramatically improve on speed and accuracy, as well as solving previously intractable problems.
△ Less
Submitted 1 September, 2022; v1 submitted 8 April, 2021;
originally announced April 2021.
-
A Peaceman-Rachford Splitting Method for the Protein Side-Chain Positioning Problem
Authors:
Forbes Burkowski,
Jiyoung Im,
Henry Wolkowicz
Abstract:
We formulate a doubly nonnegative (DNN) relaxation of the protein side-chain positioning (SCP) problem. We inherit the natural splitting of variables that stems from the facial reduction technique in the semidefinite relaxation. We solve the relaxation using a variant of the Peaceman-Rachford splitting method. Our numerical experiments show that we solve almost all instances of the NP-hard SCP pro…
▽ More
We formulate a doubly nonnegative (DNN) relaxation of the protein side-chain positioning (SCP) problem. We inherit the natural splitting of variables that stems from the facial reduction technique in the semidefinite relaxation. We solve the relaxation using a variant of the Peaceman-Rachford splitting method. Our numerical experiments show that we solve almost all instances of the NP-hard SCP problem to optimality.
△ Less
Submitted 24 March, 2023; v1 submitted 3 September, 2020;
originally announced September 2020.
-
A Restricted Dual Peaceman-Rachford Splitting Method for QAP
Authors:
Naomi Graham,
Hao Hu,
Haesol Im,
Xinxin Li,
Henry Wolkowicz
Abstract:
We revisit and strengthen splitting methods for solving doubly nonnegative, DNN, relaxations of the quadratic assignment problem, QAP. We use a modified restricted contractive splitting method, PRSM, approach. Our strengthened bounds and new dual multiplier estimates improve on the bounds and convergence results in the literature.
We revisit and strengthen splitting methods for solving doubly nonnegative, DNN, relaxations of the quadratic assignment problem, QAP. We use a modified restricted contractive splitting method, PRSM, approach. Our strengthened bounds and new dual multiplier estimates improve on the bounds and convergence results in the literature.
△ Less
Submitted 2 June, 2020;
originally announced June 2020.
-
Facial Reduction for Symmetry Reduced Semidefinite Doubly Nonnegative Programs
Authors:
Hao Hu,
Renata Sotirov,
Henry Wolkowicz
Abstract:
We consider both facial reduction, \FRp, and symmetry reduction, \SRp, techniques for semidefinite programming, \SDPp. We show that the two together fit surprisingly well in an alternating direction method of multipliers, \ADMMp, approach. In fact, this approach allows for simply adding on nonnegativity constraints, and solving the doubly nonnegative, \DNN, relaxation of many classes of hard combi…
▽ More
We consider both facial reduction, \FRp, and symmetry reduction, \SRp, techniques for semidefinite programming, \SDPp. We show that the two together fit surprisingly well in an alternating direction method of multipliers, \ADMMp, approach. In fact, this approach allows for simply adding on nonnegativity constraints, and solving the doubly nonnegative, \DNN, relaxation of many classes of hard combinatorial problems. We also show that the singularity degree remains the same after \SRp, and that the \DNN relaxations considered here have singularity degree one, that is reduced to zero after \FRp. The combination of \FR and \SR leads to a significant improvement in both numerical stability and running time for both the \ADMM and interior point approaches.
We test our method on various \DNN relaxations of hard combinatorial problems including quadratic assignment problems with sizes of more than $n=500$. This translates to a semidefinite constraint of order $250,000$ and $625\times 10^8$ nonnegative constrained variables, before applying the reduction techniques.
△ Less
Submitted 3 February, 2022; v1 submitted 21 December, 2019;
originally announced December 2019.
-
Error Bounds and Singularity Degree in Semidefinite Programming
Authors:
Stefan Sremac,
Hugo J. Woerdeman,
Henry Wolkowicz
Abstract:
In semidefinite programming a proposed optimal solution may be quite poor in spite of having sufficiently small residual in the optimality conditions. This issue may be framed in terms of the discrepancy between forward error (the unmeasurable `true error') and backward error (the measurable violation of optimality conditions). In his seminal work, Sturm provided an upper bound on forward error in…
▽ More
In semidefinite programming a proposed optimal solution may be quite poor in spite of having sufficiently small residual in the optimality conditions. This issue may be framed in terms of the discrepancy between forward error (the unmeasurable `true error') and backward error (the measurable violation of optimality conditions). In his seminal work, Sturm provided an upper bound on forward error in terms of backward error and singularity degree. In this paper we provide a method to bound the maximum rank over all solutions and use this result to obtain a lower bound on forward error for a class of convergent sequences. This lower bound complements the upper bound of Sturm. The results of Sturm imply that semidefinite programs with slow convergence necessarily have large singularity degree. Here we show that large singularity degree is, in some sense, also a sufficient condition for slow convergence for a family of external-type `central' paths. Our results are supported by numerical observations.
△ Less
Submitted 12 August, 2019;
originally announced August 2019.
-
Parametric Convex Quadratic Relaxation of the Quadratic Knapsac Problem
Authors:
Marcia Fampa,
Daniela Cristina Lubke,
Fei Wang,
Henry Wolkowicz
Abstract:
We consider a parametric convex quadratic programming, CQP, relaxation for the quadratic knapsack problem, QKP. This relaxation maintains partial quadratic information from the original QKP by perturbing the objective function to obtain a concave quadratic term. The nonconcave part generated by the perturbation is then linearized by a standard approach that lifts the problem to the matrix space. W…
▽ More
We consider a parametric convex quadratic programming, CQP, relaxation for the quadratic knapsack problem, QKP. This relaxation maintains partial quadratic information from the original QKP by perturbing the objective function to obtain a concave quadratic term. The nonconcave part generated by the perturbation is then linearized by a standard approach that lifts the problem to the matrix space. We present a primal-dual interior point method to optimize the perturbation of the quadratic function, in a search for the tightest upper bound for the \QKPp. We prove that the same perturbation approach, when applied in the context of semidefinite programming, SDP, relaxations of the QKP, cannot improve the upper bound given by the corresponding linear \SDP relaxation. The result also applies to more general integer quadratic problems. Finally, we propose new valid inequalities on the lifted matrix variable, derived from cover and knapsack inequalities for the QKP, and present the separation problems to generate cuts for the current solution of the CQP relaxation. Our best bounds are obtained from alternating between optimizing the parametric quadratic relaxation over the perturbation and adding cutting planes generated by the valid inequalities proposed.
△ Less
Submitted 10 June, 2019; v1 submitted 20 January, 2019;
originally announced January 2019.
-
Maximum determinant positive definite Toeplitz completions
Authors:
Stefan Sremac,
Hugo J. Woerdeman,
Henry Wolkowicz
Abstract:
We consider partial symmetric Toeplitz matrices where a positive definite completion exists. We characterize those patterns where the maximum determinant completion is itself Toeplitz. We then extend these results with positive definite replaced by positive semidefinite, and maximum determinant replaced by maximum rank. These results are used to determine the singularity degree of a family of semi…
▽ More
We consider partial symmetric Toeplitz matrices where a positive definite completion exists. We characterize those patterns where the maximum determinant completion is itself Toeplitz. We then extend these results with positive definite replaced by positive semidefinite, and maximum determinant replaced by maximum rank. These results are used to determine the singularity degree of a family of semidefinite optimization problems.
△ Less
Submitted 2 February, 2018;
originally announced February 2018.
-
Complete Facial Reduction in One Step for Spectrahedra
Authors:
Stefan Sremac,
Hugo Woerdeman,
Henry Wolkowicz
Abstract:
A spectrahedron is the feasible set of a semidefinite program, SDP, i.e., the intersection of an affine set with the positive semidefinite cone. While strict feasibility is a generic property for random problems, there are many classes of problems where strict feasibility fails and this means that strong duality can fail as well. If the minimal face containing the spectrahedron is known, the SDPca…
▽ More
A spectrahedron is the feasible set of a semidefinite program, SDP, i.e., the intersection of an affine set with the positive semidefinite cone. While strict feasibility is a generic property for random problems, there are many classes of problems where strict feasibility fails and this means that strong duality can fail as well. If the minimal face containing the spectrahedron is known, the SDPcan easily be transformed into an equivalent problem where strict feasibility holds and thus strong duality follows as well. The minimal face is fully characterized by the range or nullspace of any of the matrices in its relative interior. Obtaining such a matrix may require many facial reduction steps and is currently not known to be a tractable problem for spectrahedra with singularity degree greater than one. We propose a single parametric optimization problem with a resulting type of central path and prove that the optimal solution is unique and in the relative interior of the spectrahedron. Numerical tests illustrate the efficacy of our approach and its usefulness in regularizing SDPs.
△ Less
Submitted 20 October, 2017;
originally announced October 2017.
-
The many faces of degeneracy in conic optimization
Authors:
Dmitriy Drusvyatskiy,
Henry Wolkowicz
Abstract:
Slater's condition -- existence of a "strictly feasible solution" -- is a common assumption in conic optimization. Without strict feasibility, first-order optimality conditions may be meaningless, the dual problem may yield little information about the primal, and small changes in the data may render the problem infeasible. Hence, failure of strict feasibility can negatively impact off-the-shelf n…
▽ More
Slater's condition -- existence of a "strictly feasible solution" -- is a common assumption in conic optimization. Without strict feasibility, first-order optimality conditions may be meaningless, the dual problem may yield little information about the primal, and small changes in the data may render the problem infeasible. Hence, failure of strict feasibility can negatively impact off-the-shelf numerical methods, such as primal-dual interior point methods, in particular. New optimization modelling techniques and convex relaxations for hard nonconvex problems have shown that the loss of strict feasibility is a more pronounced phenomenon than has previously been realized. In this text, we describe various reasons for the loss of strict feasibility, whether due to poor modelling choices or (more interestingly) rich underlying structure, and discuss ways to cope with it and, in many pronounced cases, how to use it as an advantage. In large part, we emphasize the facial reduction preprocessing technique due to its mathematical elegance, geometric transparency, and computational potential.
△ Less
Submitted 12 June, 2017;
originally announced June 2017.
-
Rank Restricted Semidefinite Matrices and Image Closedness
Authors:
Ian Davidson,
Henry Wolkowicz
Abstract:
We study the closure of the projection of the (nonconvex) cone of rank restricted positive semidefinite matrices onto subsets of the matrix entries. This defines the feasible sets for semidefinite completion problems with restrictions on the ranks. Applications include conditions for low-rank completions using the nuclear norm heuristic.
We study the closure of the projection of the (nonconvex) cone of rank restricted positive semidefinite matrices onto subsets of the matrix entries. This defines the feasible sets for semidefinite completion problems with restrictions on the ranks. Applications include conditions for low-rank completions using the nuclear norm heuristic.
△ Less
Submitted 31 October, 2016;
originally announced October 2016.
-
Low-Rank Matrix Completion using Nuclear Norm with Facial Reduction
Authors:
Shimeng Huang,
Henry Wolkowicz
Abstract:
Minimization of the nuclear norm is often used as a surrogate, convex relaxation, for finding the minimum rank completion (recovery) of a partial matrix. The minimum nuclear norm problem can be solved as a trace minimization semidefinite programming problem, (SDP). The SDP and its dual are regular in the sense that they both satisfy strict feasibility. Interior point algorithms are the current met…
▽ More
Minimization of the nuclear norm is often used as a surrogate, convex relaxation, for finding the minimum rank completion (recovery) of a partial matrix. The minimum nuclear norm problem can be solved as a trace minimization semidefinite programming problem, (SDP). The SDP and its dual are regular in the sense that they both satisfy strict feasibility. Interior point algorithms are the current methods of choice for these problems. This means that it is difficult to solve large scale problems and difficult to get high accuracy solutions.
In this paper we take advantage of the structure at optimality for the minimum nuclear norm problem. We show that even though strict feasibility holds, the facial reduction framework can be successfully applied to obtain a proper face that contains the optimal set, and thus can dramatically reduce the size of the final nuclear norm problem while guaranteeing a low-rank solution. We include numerical tests for both exact and noisy cases. In all cases we assume that knowledge of a target rank is available.
△ Less
Submitted 14 August, 2016;
originally announced August 2016.
-
Computing the generators of the truncated real radical ideal by moment matrices and SDP facial reduction
Authors:
Fei Wang,
Greg Reid,
Henry Wolkowicz
Abstract:
Recent breakthroughs have been made in the use of semidefinite programming and its application to real polynomial solving. For example, the real radical of a zero dimensional ideal, can be determined by such approaches as shown by Lasserre and collaborators. Some progress has been made on the determination of the real radical in positive dimension by Ma, Wang and Zhi. Such work involves the determ…
▽ More
Recent breakthroughs have been made in the use of semidefinite programming and its application to real polynomial solving. For example, the real radical of a zero dimensional ideal, can be determined by such approaches as shown by Lasserre and collaborators. Some progress has been made on the determination of the real radical in positive dimension by Ma, Wang and Zhi. Such work involves the determination of maximal rank semidefinite moment matrices. Existing methods are computationally expensive and have poorer accuracy on larger examples.
This paper is motivated by problems in the numerical computation of the real radical ideal in the general positive case.
In this paper we give a method to compute the generators of the real radical for any given degree $d$. We combine the use of moment matrices and techniques from SDP optimization: facial reduction first developed by Borwein and Wolkowicz. In use of the semidefinite moment matrices to compute the real radical, the maximum rank property is very key, and with facial reduction, it can be guaranteed with very high accuracy. Our algorithm can be used to test the real radical membership of a given polynomial. In a special situation, we can determine the real radical ideal in the positive dimensional case.
△ Less
Submitted 5 November, 2016; v1 submitted 1 June, 2016;
originally announced June 2016.
-
Local Nonglobal Minima for Solving Large Scale Extended Trust Region Subproblems
Authors:
Maziar Salahi,
Akram Taati,
Henry Wolkowicz
Abstract:
We study large scale extended trust region subproblems (eTRS) i.e., the minimization of a general quadratic function subject to a norm constraint, known as the trust region subproblem (TRS) but with an additional linear inequality constraint. It is well known that strong duality holds for the TRS and that there are efficient algorithms for solving large scale TRS problems. It is also known that th…
▽ More
We study large scale extended trust region subproblems (eTRS) i.e., the minimization of a general quadratic function subject to a norm constraint, known as the trust region subproblem (TRS) but with an additional linear inequality constraint. It is well known that strong duality holds for the TRS and that there are efficient algorithms for solving large scale TRS problems. It is also known that there can exist at most one local non-global minimizer (LNGM) for TRS. We combine this with known characterizations for strong duality for eTRS and, in particular, connect this with the so-called hard case for TRS.
We begin with a recent characterization of the minimum for the TRS via a generalized eigenvalue problem and extend this result to the LNGM. We then use this to derive an efficient algorithm that finds the global minimum for eTRS by solving at most three generalized eigenvalue problems.
△ Less
Submitted 23 December, 2015;
originally announced December 2015.
-
ADMM for the SDP relaxation of the QAP
Authors:
Danilo Elias Oliveira,
Henry Wolkowicz,
Yangyang Xu
Abstract:
The semidefinite programming (SDP) relaxation has proven to be extremely strong for many hard discrete optimization problems. This is in particular true for the quadratic assignment problem (QAP), arguably one of the hardest NP-hard discrete optimization problems. There are several difficulties that arise in efficiently solving the SDP relaxation, e.g.,~increased dimension; inefficiency of the cur…
▽ More
The semidefinite programming (SDP) relaxation has proven to be extremely strong for many hard discrete optimization problems. This is in particular true for the quadratic assignment problem (QAP), arguably one of the hardest NP-hard discrete optimization problems. There are several difficulties that arise in efficiently solving the SDP relaxation, e.g.,~increased dimension; inefficiency of the current primal-dual interior point solvers in terms of both time and accuracy; and difficulty and high expense in adding cutting plane constraints.
We propose using the alternating direction method of multipliers (ADMM) to solve the SDP relaxation. This first order approach allows for inexpensive iterations, a method of cheaply obtaining low rank solutions, as well a trivial way of adding cutting plane inequalities. When compared to current approaches and current best available bounds we obtain remarkable robustness, efficiency and improved bounds.
△ Less
Submitted 16 December, 2015;
originally announced December 2015.
-
Facial Reduction and SDP Methods for Systems of Polynomial Equations
Authors:
Greg Reid,
Fei Wang,
Henry Wolkowicz,
Wenyuan Wu
Abstract:
The real radical ideal of a system of polynomials with finitely many complex roots is generated by a system of real polynomials having only real roots and free of multiplicities. It is a central object in computational real algebraic geometry and important as a preconditioner for numerical solvers. Lasserre and co-workers have shown that the real radical ideal of real polynomial systems with finit…
▽ More
The real radical ideal of a system of polynomials with finitely many complex roots is generated by a system of real polynomials having only real roots and free of multiplicities. It is a central object in computational real algebraic geometry and important as a preconditioner for numerical solvers. Lasserre and co-workers have shown that the real radical ideal of real polynomial systems with finitely many real solutions can be determined by a combination of semi-definite programming (SDP) and geometric involution techniques. A conjectured extension of such methods to positive dimensional polynomial systems has been given recently by Ma, Wang and Zhi.
We show that regularity in the form of the Slater constraint qualification (strict feasibility) fails for the resulting SDP feasibility problems. Facial reduction is then a popular technique whereby SDP problems that fail strict feasibility can be regularized by projecting onto a face of the convex cone of semi-definite problems.
In this paper we introduce a framework for combining facial reduction with such SDP methods for analyzing $0$ and positive dimensional real ideals of real polynomial systems. The SDP methods are implemented in MATLAB and our geometric involutive form is implemented in Maple. We use two approaches to find a feasible moment matrix. We use an interior point method within the CVX package for MATLAB and also the Douglas-Rachford (DR) projection-reflection method.
Illustrative examples show the advantages of the DR approach for some problems over standard interior point methods. We also see the advantage of facial reduction both in regularizing the problem and also in reducing the dimension of the moment matrices. Problems requiring more than one facial reduction are also presented.
△ Less
Submitted 1 April, 2015;
originally announced April 2015.
-
Noisy Euclidean distance realization: robust facial reduction and the Pareto frontier
Authors:
Dmitriy Drusvyatskiy,
Nathan Krislock,
Yuen-Lam Voronin,
Henry Wolkowicz
Abstract:
We present two algorithms for large-scale low-rank Euclidean distance matrix completion problems, based on semidefinite optimization. Our first method works by relating cliques in the graph of the known distances to faces of the positive semidefinite cone, yielding a combinatorial procedure that is provably robust and parallelizable. Our second algorithm is a first order method for maximizing the…
▽ More
We present two algorithms for large-scale low-rank Euclidean distance matrix completion problems, based on semidefinite optimization. Our first method works by relating cliques in the graph of the known distances to faces of the positive semidefinite cone, yielding a combinatorial procedure that is provably robust and parallelizable. Our second algorithm is a first order method for maximizing the trace---a popular low-rank inducing regularizer---in the formulation of the problem with a constrained misfit. Both of the methods output a point configuration that can serve as a high-quality initialization for local optimization techniques. Numerical experiments on large-scale sensor localization problems illustrate the two approaches.
△ Less
Submitted 26 August, 2015; v1 submitted 24 October, 2014;
originally announced October 2014.
-
Projection methods in quantum information science
Authors:
Yuen-Lam Cheung,
Dmitriy Drusvyatskiy,
Chi-Kwong Li,
Diane Pelejo,
Henry Wolkowicz
Abstract:
We consider the problem of constructing quantum operations or channels, if they exist, that transform a given set of quantum states $\{ρ_1, \dots, ρ_k\}$ to another such set $\{\hatρ_1, \dots, \hatρ_k\}$. In other words, we must find a {\em completely positive linear map}, if it exists, that maps a given set of density matrices to another given set of density matrices. This problem, in turn, is an…
▽ More
We consider the problem of constructing quantum operations or channels, if they exist, that transform a given set of quantum states $\{ρ_1, \dots, ρ_k\}$ to another such set $\{\hatρ_1, \dots, \hatρ_k\}$. In other words, we must find a {\em completely positive linear map}, if it exists, that maps a given set of density matrices to another given set of density matrices. This problem, in turn, is an instance of a positive semi-definite feasibility problem, but with highly structured constraints. The nature of the constraints makes projection based algorithms very appealing when the number of variables is huge and standard interior point-methods for semi-definite programming are not applicable. We provide emperical evidence to this effect. We moreover present heuristics for finding both high rank and low rank solutions. Our experiments are based on the \emph{method of alternating projections} and the \emph{Douglas-Rachford} reflection method.
△ Less
Submitted 24 July, 2014;
originally announced July 2014.
-
Coordinate shadows of semi-definite and Euclidean distance matrices
Authors:
D. Drusvyatskiy,
G. Pataki,
H. Wolkowicz
Abstract:
We consider the projected semi-definite and Euclidean distance cones onto a subset of the matrix entries. These two sets are precisely the input data defining feasible semi-definite and Euclidean distance completion problems. We classify when these sets are closed, and use the boundary structure of these two sets to elucidate the Krislock-Wolkowicz facial reduction algorithm. In particular, we sho…
▽ More
We consider the projected semi-definite and Euclidean distance cones onto a subset of the matrix entries. These two sets are precisely the input data defining feasible semi-definite and Euclidean distance completion problems. We classify when these sets are closed, and use the boundary structure of these two sets to elucidate the Krislock-Wolkowicz facial reduction algorithm. In particular, we show that under a chordality assumption, the "minimal cones" of these problems admit combinatorial characterizations. As a byproduct, we record a striking relationship between the complexity of the general facial reduction algorithm (singularity degree) and facial exposedness of conic images under a linear mapping.
△ Less
Submitted 8 January, 2015; v1 submitted 8 May, 2014;
originally announced May 2014.
-
Eigenvalue, Quadratic Programming, and Semidefinite Programming Bounds for a Cut Minimization Problem
Authors:
Ting Kei Pong,
Hao Sun,
Ningchuan Wang,
Henry Wolkowicz
Abstract:
We consider the problem of partitioning the node set of a graph into $k$ sets of given sizes in order to \emph{minimize the cut} obtained using (removing) the $k$-th set. If the resulting cut has value $0$, then we have obtained a vertex separator. This problem is closely related to the graph partitioning problem. In fact, the model we use is the same as that for the graph partitioning problem exc…
▽ More
We consider the problem of partitioning the node set of a graph into $k$ sets of given sizes in order to \emph{minimize the cut} obtained using (removing) the $k$-th set. If the resulting cut has value $0$, then we have obtained a vertex separator. This problem is closely related to the graph partitioning problem. In fact, the model we use is the same as that for the graph partitioning problem except for a different \emph{quadratic} objective function. We look at known and new bounds obtained from various relaxations for this NP-hard problem. This includes: the standard eigenvalue bound, projected eigenvalue bounds using both the adjacency matrix and the Laplacian, quadratic programming (QP) bounds based on recent successful QP bounds for the quadratic assignment problems, and semidefinite programming bounds. We include numerical tests for large and \emph{huge} problems that illustrate the efficiency of the bounds in terms of strength and time.
△ Less
Submitted 18 November, 2014; v1 submitted 20 January, 2014;
originally announced January 2014.
-
Extreme point inequalities and geometry of the rank sparsity ball
Authors:
D. Drusvyatskiy,
S. A. Vavasis,
H. Wolkowicz
Abstract:
We investigate geometric features of the unit ball corresponding to the sum of the nuclear norm of a matrix and the $l_1$ norm of its entries --- a common penalty function encouraging joint low rank and high sparsity. As a byproduct of this effort, we develop a calculus (or algebra) of faces for general convex functions, yielding a simple and unified approach for deriving inequalities balancing th…
▽ More
We investigate geometric features of the unit ball corresponding to the sum of the nuclear norm of a matrix and the $l_1$ norm of its entries --- a common penalty function encouraging joint low rank and high sparsity. As a byproduct of this effort, we develop a calculus (or algebra) of faces for general convex functions, yielding a simple and unified approach for deriving inequalities balancing the various features of the optimization problem at hand, at the extreme points of the solution set.
△ Less
Submitted 19 January, 2014;
originally announced January 2014.
-
Explicit Sensor Network Localization using Semidefinite Representations and Facial Reductions
Authors:
Nathan Krislock,
Henry Wolkowicz
Abstract:
The sensor network localization, SNL, problem in embedding dimension r, consists of locating the positions of wireless sensors, given only the distances between sensors that are within radio range and the positions of a subset of the sensors (called anchors). Current solution techniques relax this problem to a weighted, nearest, (positive) semidefinite programming, SDP, completion problem, by us…
▽ More
The sensor network localization, SNL, problem in embedding dimension r, consists of locating the positions of wireless sensors, given only the distances between sensors that are within radio range and the positions of a subset of the sensors (called anchors). Current solution techniques relax this problem to a weighted, nearest, (positive) semidefinite programming, SDP, completion problem, by using the linear mapping between Euclidean distance matrices, EDM, and semidefinite matrices. The resulting SDP is solved using primal-dual interior point solvers, yielding an expensive and inexact solution.
This relaxation is highly degenerate in the sense that the feasible set is restricted to a low dimensional face of the SDP cone, implying that the Slater constraint qualification fails. Cliques in the graph of the SNL problem give rise to this degeneracy in the SDP relaxation. In this paper, we take advantage of the absence of the Slater constraint qualification and derive a technique for the SNL problem, with exact data, that explicitly solves the corresponding rank restricted SDP problem. No SDP solvers are used. For randomly generated instances, we are able to efficiently solve many huge instances of this NP-hard problem to high accuracy, by finding a representation of the minimal face of the SDP cone that contains the SDP matrix representation of the EDM. The main work of our algorithm consists in repeatedly finding the intersection of subspaces that represent the faces of the SDP cone that correspond to cliques of the SNL problem.
△ Less
Submitted 29 January, 2010;
originally announced February 2010.
-
Sensor Network Localization, Euclidean Distance Matrix Completions, and Graph Realization
Authors:
Yichuan Ding,
Nathan Krislock,
Jiawei Qian,
Henry Wolkowicz
Abstract:
We study Semidefinite Programming, \SDPc relaxations for Sensor Network Localization, \SNLc with anchors and with noisy distance information. The main point of the paper is to view \SNL as a (nearest) Euclidean Distance Matrix, \EDM, completion problem and to show the advantages for using this latter, well studied model. We first show that the current popular \SDP relaxation is equivalent to kno…
▽ More
We study Semidefinite Programming, \SDPc relaxations for Sensor Network Localization, \SNLc with anchors and with noisy distance information. The main point of the paper is to view \SNL as a (nearest) Euclidean Distance Matrix, \EDM, completion problem and to show the advantages for using this latter, well studied model. We first show that the current popular \SDP relaxation is equivalent to known relaxations in the literature for \EDM completions. The existence of anchors in the problem is {\em not} special. The set of anchors simply corresponds to a given fixed clique for the graph of the \EDM problem. We next propose a method of projection when a large clique or a dense subgraph is identified in the underlying graph. This projection reduces the size, and improves the stability, of the relaxation. In addition, viewing the problem as an \EDM completion problem yields better low rank approximations for the low dimensional realizations. And, the projection/reduction procedure can be repeated for other given cliques of sensors or for sets of sensors, where many distances are known. Thus, further size reduction can be obtained. Optimality/duality conditions and a primal-dual interior-exterior path following algorithm are derived for the \SDP relaxations We discuss the relative stability and strength of two formulations and the corresponding algorithms that are used. In particular, we show that the quadratic formulation arising from the \SDP relaxation is better conditioned than the linearized form, that is used in the literature and that arises from applying a Schur complement.
△ Less
Submitted 14 December, 2006;
originally announced December 2006.
