We consider statistical procedures for feature selection defined by a family of regularization problems with convex piecewise linear loss functions and penalties of L1 nature. Many known statistical procedures (e.g. quantile regression and support vector machines with L1 norm penalty) are subsumed under this category. Computationally, the regularization problems are a special family of parametric linear programming (LP) problems, which are known as `parametric cost LP' or `parametric right-hand-side LP' in the optimization theory. Exploiting the connection with the LP theory, we lay out general algorithms, namely, the simplex algorithm and the tableau-simplex algorithm for generating regularized solution paths for the feature selection problems. Furthermore, by utilizing the structural traits of the relevant LP problems, we simplify the tableau-simplex algorithm for fast anticycling computation. The significance of such algorithms is that they allow a complete exploration of the model space along the paths and provide a broad view of persistent features in the data. The implications of the general path-finding algorithms are outlined for a few statistical procedures, and they are illustrated with numerical examples.

Meet the speaker in Room 212 Cockins Hall at 4:30 p.m. Refreshments will be served.