arXiv subjects

Ufuk Topcu

Publications and source records attributed to Ufuk Topcu.

At least 343 records · Page 19Linked to original sources

arXiv2016

Synthesis of Shared Control Protocols with Provable Safety and Performance Guarantees

We formalize synthesis of shared control protocols with correctness guarantees for temporal logic specifications. More specifically, we introduce a modeling formalism in which both a human and an autonomy protocol can issue commands to a robot towards performing a certain task. These commands are blended into a joint input to the robot. The autonomy protocol is synthesized using an abstraction of possible human commands accounting for randomness in decisions caused by factors such as fatigue or incomprehensibility of the problem at hand. The synthesis is designed to ensure that the resulting robot behavior satisfies given safety and performance specifications, e.g., in temporal logic. Our solution is based on nonlinear programming and we address the inherent scalability issue by presenting alternative methods. We assess the feasibility and the scalability of the approach by an experimental evaluation.

cs.RO↗

arXiv2016

An Automaton Learning Approach to Solving Safety Games over Infinite Graphs

We propose a method to construct finite-state reactive controllers for systems whose interactions with their adversarial environment are modeled by infinite-duration two-player games over (possibly) infinite graphs. The proposed method targets safety games with infinitely many states or with such a large number of states that it would be impractical---if not impossible---for conventional synthesis techniques that work on the entire state space. We resort to constructing finite-state controllers for such systems through an automata learning approach, utilizing a symbolic representation of the underlying game that is based on finite automata. Throughout the learning process, the learner maintains an approximation of the winning region (represented as a finite automaton) and refines it using different types of counterexamples provided by the teacher until a satisfactory controller can be derived (if one exists). We present a symbolic representation of safety games (inspired by regular model checking), propose implementations of the learner and teacher, and evaluate their performance on examples motivated by robotic motion planning in dynamic environments.

cs.FL↗

arXiv2015

Strategy Synthesis for Stochastic Rabin Games with Discounted Reward

Stochastic games are often used to model reactive processes. We consider the problem of synthesizing an optimal almost-sure winning strategy in a two-player (namely a system and its environment) turn-based stochastic game with both a qualitative objective as a Rabin winning condition, and a quantitative objective as a discounted reward. Optimality is considered only over the almost-sure winning strategies, i.e., system strategies that guarantee the satisfaction of the Rabin condition with probability 1 regardless of the environment's strategy. We show that optimal almost-sure winning strategies may need infinite memory, but epsilon-optimal almost-sure winning strategies can always be finite-memory or even memoryless. We identify a sufficient and necessary condition of the existence of memoryless epsilon-optimal almost-sure winning strategies and propose an algorithm to compute one when this condition is satisfied.

eess.SY↗

arXiv2015

Optimal Temporal Logic Planning in Probabilistic Semantic Maps

This paper considers robot motion planning under temporal logic constraints in probabilistic maps obtained by semantic simultaneous localization and mapping (SLAM). The uncertainty in a map distribution presents a great challenge for obtaining correctness guarantees with respect to the linear temporal logic (LTL) specification. We show that the problem can be formulated as an optimal control problem in which both the semantic map and the logic formula evaluation are stochastic. Our first contribution is to reduce the stochastic control problem for a subclass of LTL to a deterministic shortest path problem by introducing a confidence parameter $delta$. A robot trajectory obtained from the deterministic problem is guaranteed to have minimum cost and to satisfy the logic specification in the true environment with probability $delta$. Our second contribution is to design an admissible heuristic function that guides the planning in the deterministic problem towards satisfying the temporal logic specification. This allows us to obtain an optimal and very efficient solution using the A* algorithm. The performance and correctness of our approach are demonstrated in a simulated semantic environment using a differential-drive robot.

cs.RO↗

arXiv2015

Safety-Constrained Reinforcement Learning for MDPs

We consider controller synthesis for stochastic and partially unknown environments in which safety is essential. Specifically, we abstract the problem as a Markov decision process in which the expected performance is measured using a cost function that is unknown prior to run-time exploration of the state space. Standard learning approaches synthesize cost-optimal strategies without guaranteeing safety properties. To remedy this, we first compute safe, permissive strategies. Then, exploration is constrained to these strategies and thereby meets the imposed safety requirements. Exploiting an iterative learning procedure, the resulting policy is safety-constrained and optimal. We show correctness and completeness of the method and discuss the use of several heuristics to increase its scalability. Finally, we demonstrate the applicability by means of a prototype implementation.

cs.SE↗

arXiv2015

Distributed Charging Control of Electric Vehicles Using Online Learning

We propose an algorithm for distributed charging control of electric vehicles (EVs) using online learning and online convex optimization. Many distributed charging control algorithms in the literature implicitly assume fast two-way communication between a distribution company and EV customers. This assumption is impractical at present and raises privacy and security concerns. Our algorithm does not use this assumption; however, at the expense of slower convergence to the optimal solution. The proposed algorithm requires one-way communication, which is implemented through the distribution company publishing the pricing profiles of the previous days. We provide convergence results of the algorithm and illustrate the results through numerical examples.

math.OC↗

arXiv2015

Convex Optimal Uncertainty Quantification

Optimal uncertainty quantification (OUQ) is a framework for numerical extreme-case analysis of stochastic systems with imperfect knowledge of the underlying probability distribution. This paper presents sufficient conditions under which an OUQ problem can be reformulated as a finite-dimensional convex optimization problem, for which efficient numerical solutions can be obtained. The sufficient conditions include that the objective function is piecewise concave and the constraints are piecewise convex. In particular, we show that piecewise concave objective functions may appear in applications where the objective is defined by the optimal value of a parameterized linear program.

math.OC↗

arXiv2015

Computational methods for stochastic control with metric interval temporal logic specifications

This paper studies an optimal control problem for continuous-time stochastic systems subject to reachability objectives specified in a subclass of metric interval temporal logic specifications, a temporal logic with real-time constraints. We propose a probabilistic method for synthesizing an optimal control policy that maximizes the probability of satisfying a specification based on a discrete approximation of the underlying stochastic system. First, we show that the original problem can be formulated as a stochastic optimal control problem in a state space augmented with finite memory and states of some clock variables. Second, we present a numerical method for computing an optimal policy with which the given specification is satisfied with the maximal probability in point-based semantics in the discrete approximation of the underlying system. We show that the policy obtained in the discrete approximation converges to the optimal one for satisfying the specification in the continuous or dense-time semantics as the discretization becomes finer in both state and time. Finally, we illustrate our approach with a robotic motion planning example.

eess.SY↗

arXiv2015

Optimal control in Markov decision processes via distributed optimization

Optimal control synthesis in stochastic systems with respect to quantitative temporal logic constraints can be formulated as linear programming problems. However, centralized synthesis algorithms do not scale to many practical systems. To tackle this issue, we propose a decomposition-based distributed synthesis algorithm. By decomposing a large-scale stochastic system modeled as a Markov decision process into a collection of interacting sub-systems, the original control problem is formulated as a linear programming problem with a sparse constraint matrix, which can be solved through distributed optimization methods. Additionally, we propose a decomposition algorithm which automatically exploits, if exists, the modular structure in a given large-scale system. We illustrate the proposed methods through robotic motion planning examples.

eess.SY↗

arXiv2015

Value of forecasts in planning under uncertainty: Extended version

In environments with increasing uncertainty, such as smart grid applications based on renewable energy, planning can benefit from incorporating forecasts about the uncertainty and from systematically evaluating the utility of the forecast information. We consider these issues in a planning framework in which forecasts are interpreted as constraints on the possible probability distributions that the uncertain quantity of interest may have. The planning goal is to robustly maximize the expected value of a given utility function, integrated with respect to the worst-case distribution consistent with the forecasts. Under mild technical assumptions we show that the problem can be reformulated into convex optimization. We exploit this reformulation to evaluate how informative the forecasts are in determining the optimal planning decision, as well as to guide how forecasts can be appropriately refined to obtain higher utility values. A numerical example of wind energy trading in electricity markets illustrates our results.

math.OC↗

arXiv2015

Correct-by-synthesis reinforcement learning with temporal logic constraints

We consider a problem on the synthesis of reactive controllers that optimize some a priori unknown performance criterion while interacting with an uncontrolled environment such that the system satisfies a given temporal logic specification. We decouple the problem into two subproblems. First, we extract a (maximally) permissive strategy for the system, which encodes multiple (possibly all) ways in which the system can react to the adversarial environment and satisfy the specifications. Then, we quantify the a priori unknown performance criterion as a (still unknown) reward function and compute an optimal strategy for the system within the operating envelope allowed by the permissive strategy by using the so-called maximin-Q learning algorithm. We establish both correctness (with respect to the temporal logic specifications) and optimality (with respect to the a priori unknown performance criterion) of this two-step technique for a fragment of temporal logic specifications. For specifications beyond this fragment, correctness can still be preserved, but the learned strategy may be sub-optimal. We present an algorithm to the overall problem, and demonstrate its use and computational requirements on a set of robot motion planning examples.

cs.LO↗

arXiv2014

Pareto efficiency in synthesizing shared autonomy policies with temporal logic constraints

In systems in which control authority is shared by an autonomous controller and a human operator, it is important to find solutions that achieve a desirable system performance with a reasonable workload for the human operator. We formulate a shared autonomy system capable of capturing the interaction and switching control between an autonomous controller and a human operator, as well as the evolution of the operator's cognitive state during control execution. To trade-off human's effort and the performance level, e.g., measured by the probability of satisfying the underlying temporal logic specification, a two-stage policy synthesis algorithm is proposed for generating Pareto efficient coordination and control policies with respect to user specified weights. We integrate the Tchebychev scalarization method for multi-objective optimization methods to obtain a better coverage of the set of Pareto efficient solutions than linear scalarization methods.

cs.RO↗

arXiv2014

Differentially Private Distributed Constrained Optimization

Many resource allocation problems can be formulated as an optimization problem whose constraints contain sensitive information about participating users. This paper concerns solving this kind of optimization problem in a distributed manner while protecting the privacy of user information. Without privacy considerations, existing distributed algorithms normally consist in a central entity computing and broadcasting certain public coordination signals to participating users. However, the coordination signals often depend on user information, so that an adversary who has access to the coordination signals can potentially decode information on individual users and put user privacy at risk. We present a distributed optimization algorithm that preserves differential privacy, which is a strong notion that guarantees user privacy regardless of any auxiliary information an adversary may have. The algorithm achieves privacy by perturbing the public signals with additive noise, whose magnitude is determined by the sensitivity of the projection operation onto user-specified constraints. By viewing the differentially private algorithm as an implementation of stochastic gradient descent, we are able to derive a bound for the suboptimality of the algorithm. We illustrate the implementation of our algorithm via a case study of electric vehicle charging. Specifically, we derive the sensitivity and present numerical simulations for the algorithm. Through numerical simulations, we are able to investigate various aspects of the algorithm when being used in practice, including the choice of step size, number of iterations, and the trade-off between privacy level and suboptimality.

math.OC↗

arXiv2014

Integrating active sensing into reactive synthesis with temporal logic constraints under partial observations

We introduce the notion of online reactive planning with sensing actions for systems with temporal logic constraints in partially observable and dynamic environments. With incomplete information on the dynamic environment, reactive controller synthesis amounts to solving a two-player game with partial observations, which has impractically computational complexity. To alleviate the high computational burden, online replanning via sensing actions avoids solving the strategy in the reactive system under partial observations. Instead, we only solve for a strategy that ensures a given temporal logic specification can be satisfied had the system have complete observations of its environment. Such a strategy is then transformed into one which makes control decisions based on the observed sequence of states (of the interacting system and its environment). When the system encounters a belief---a set including all possible hypotheses the system has for the current state---for which the observation-based strategy is undefined, a sequence of sensing actions are triggered, chosen by an active sensing strategy, to reduce the uncertainty in the system's belief. We show that by alternating between the observation-based strategy and the active sensing strategy, under a mild technical assumption of the set of sensors in the system, the given temporal logic specification can be satisfied with probability 1.

eess.SY↗

arXiv2014

Abstractions and sensor design in partial-information, reactive controller synthesis

Automated synthesis of reactive control protocols from temporal logic specifications has recently attracted considerable attention in various applications in, for example, robotic motion planning, network management, and hardware design. An implicit and often unrealistic assumption in this past work is the availability of complete and precise sensing information during the execution of the controllers. In this paper, we use an abstraction procedure for systems with partial observation and propose a formalism to investigate effects of limitations in sensing. The abstraction procedure enables the existing synthesis methods with partial observation to be applicable and efficient for systems with infinite (or finite but large number of) states. This formalism enables us to systematically discover sensing modalities necessary in order to render the underlying synthesis problems feasible. We use counterexamples, which witness unrealizability potentially due to the limitations in sensing and the coarseness in the abstract system, and interpolation-based techniques to refine the model and the sensing modalities, i.e., to identify new sensors to be included, in such synthesis problems. We demonstrate the method on examples from robotic motion planning.

eess.SY↗

arXiv2014

Probably Approximately Correct MDP Learning and Control With Temporal Logic Constraints

We consider synthesis of control policies that maximize the probability of satisfying given temporal logic specifications in unknown, stochastic environments. We model the interaction between the system and its environment as a Markov decision process (MDP) with initially unknown transition probabilities. The solution we develop builds on the so-called model-based probably approximately correct Markov decision process (PAC-MDP) methodology. The algorithm attains an $\varepsilon$-approximately optimal policy with probability $1-δ$ using samples (i.e. observations), time and space that grow polynomially with the size of the MDP, the size of the automaton expressing the temporal logic specification, $\frac{1}{\varepsilon}$, $\frac{1}δ$ and a finite time horizon. In this approach, the system maintains a model of the initially unknown MDP, and constructs a product MDP based on its learned model and the specification automaton that expresses the temporal logic constraints. During execution, the policy is iteratively updated using observation of the transitions taken by the system. The iteration terminates in finitely many steps. With high probability, the resulting policy is such that, for any state, the difference between the probability of satisfying the specification under this policy and the optimal one is within a predefined bound.

eess.SY↗

arXiv2014

Distributed Load Balancing with Nonconvex Constraints: A Randomized Algorithm with Application to Electric Vehicle Charging Scheduling

To schedule a large number of EVs with the presence of practical nonconvex charging constraints, a distributed and randomized algorithm is proposed in this paper. The algorithm assumes the availability of a coordinator which can communicate with all EVs. In each iteration of the algorithm, the coordinator receives tentative charging profiles from the EVs and computes a broadcast control signal. After receiving this broadcast control signal, each EV generates a probability distribution over its admissible charging profiles, and samples from the distribution to update its tentative charging profile. We prove that the algorithm converges almost surely to a charging profile in finite iterations. The final charging profile (that the algorithm converges to) is random, i.e., it depends on the realization. We characterize the final charging profile---a charging profile can be a realization of the final charging profile if and only if it is a Nash equilibrium of the game in which each EV seeks to minimize the inner product of its own charging profile and the aggregate electricity demand. Furthermore, we provide a uniform suboptimality upper bound, that scales O(1/n) in the number n of EVs, for all realizations of the final charging profile.

math.OC↗

arXiv2014

Differentially Private Convex Optimization with Piecewise Affine Objectives

Differential privacy is a recently proposed notion of privacy that provides strong privacy guarantees without any assumptions on the adversary. The paper studies the problem of computing a differentially private solution to convex optimization problems whose objective function is piecewise affine. Such problem is motivated by applications in which the affine functions that define the objective function contain sensitive user information. We propose several privacy preserving mechanisms and provide analysis on the trade-offs between optimality and the level of privacy for these mechanisms. Numerical experiments are also presented to evaluate their performance in practice.

math.OC↗