Covers the state of the art in automatic differentiation theory and practice. Intended for computational scientists and engineers, this book aims to provide insight into effective strategies for using automatic differentiation for design optimization, sensitivity analysis, and uncertainty quantification.
The Fifth International Conference on Automatic Differentiation held from August 11 to 15, 2008 in Bonn, Germany, is the most recent one in a series that began in Breckenridge, USA, in 1991 and continued in Santa Fe, USA, in 1996, Nice, France, in 2000 and Chicago, USA, in 2004. The 31 papers included in these proceedings re?ect the state of the art in automatic differentiation (AD) with respect to theory, applications, and tool development. Overall, 53 authors from institutions in 9 countries contributed, demonstrating the worldwide acceptance of AD technology in computational science. Recently it was shown that the problem underlying AD is indeed NP-hard, f- mally proving the inherently challenging nature of this technology. So, most likely, no deterministic “silver bullet” polynomial algorithm can be devised that delivers optimum performance for general codes. In this context, the exploitation of doma- speci?c structural information is a driving issue in advancing practical AD tool and algorithm development. This trend is prominently re?ected in many of the pub- cations in this volume, not only in a better understanding of the interplay of AD and certain mathematical paradigms, but in particular in the use of hierarchical AD approaches that judiciously employ general AD techniques in application-speci?c - gorithmic harnesses. In this context, the understanding of structures such as sparsity of derivatives, or generalizations of this concept like scarcity, plays a critical role, in particular for higher derivative computations.
Principles and Techniques of Algorithmic Differentiation, Second Edition
Author: Andreas Griewank
This title is a comprehensive treatment of algorithmic, or automatic, differentiation. The second edition covers recent developments in applications and theory, including an elegant NP completeness argument and an introduction to scarcity.
A survey book focusing on the key relationships and synergies between automatic differentiation (AD) tools and other software tools, such as compilers and parallelizers, as well as their applications. The key objective is to survey the field and present the recent developments. In doing so the topics covered shed light on a variety of perspectives. They reflect the mathematical aspects, such as the differentiation of iterative processes, and the analysis of nonsmooth code. They cover the scientific programming aspects, such as the use of adjoints in optimization and the propagation of rounding errors. They also cover "implementation" problems.
The calculation of partial derivatives is a fundamental need in scientific computing. Automatic differentiation (AD) can be applied straightforwardly to obtain all necessary partial derivatives (usually first and, possibly, second derivatives) regardless of a code?s complexity. However, the space and time efficiency of AD can be dramatically improved?sometimes transforming a problem from intractable to highly feasible?if inherent problem structure is used to apply AD in a judicious manner. Automatic Differentiation in MATLAB using ADMAT with Applications discusses the efficient use of AD to solve real problems, especially multidimensional zero-finding and optimization, in the MATLAB environment. This book is concerned with the determination of the first and second derivatives in the context of solving scientific computing problems with an emphasis on optimization and solutions to nonlinear systems. The authors focus on the application rather than the implementation of AD, solve real nonlinear problems with high performance by exploiting the problem structure in the application of AD, and provide many easy to understand applications, examples, and MATLAB templates.
The proceedings represent the state of knowledge in the area of algorithmic differentiation (AD). The 31 contributed papers presented at the AD2012 conference cover the application of AD to many areas in science and engineering as well as aspects of AD theory and its implementation in tools. For all papers the referees, selected from the program committee and the greater community, as well as the editors have emphasized accessibility of the presented ideas also to non-AD experts. In the AD tools arena new implementations are introduced covering, for example, Java and graphical modeling environments or join the set of existing tools for Fortran. New developments in AD algorithms target the efficiency of matrix-operation derivatives, detection and exploitation of sparsity, partial separability, the treatment of nonsmooth functions, and other high-level mathematical aspects of the numerical computations to be differentiated. Applications stem from the Earth sciences, nuclear engineering, fluid dynamics, and chemistry, to name just a few. In many cases the applications in a given area of science or engineering share characteristics that require specific approaches to enable AD capabilities or provide an opportunity for efficiency gains in the derivative computation. The description of these characteristics and of the techniques for successfully using AD should make the proceedings a valuable source of information for users of AD tools.
This book provides the first practical guide to the function and implementation of algorithmic differentiation in finance. Written in a highly accessible way, Algorithmic Differentiation Explained will take readers through all the major applications of AD in the derivatives setting with a focus on implementation. Algorithmic Differentiation (AD) has been popular in engineering and computer science, in areas such as fluid dynamics and data assimilation for many years. Over the last decade, it has been increasingly (and successfully) applied to financial risk management, where it provides an efficient way to obtain financial instrument price derivatives with respect to the data inputs. Calculating derivatives exposure across a portfolio is no simple task. It requires many complex calculations and a large amount of computer power, which in prohibitively expensive and can be time consuming. Algorithmic differentiation techniques can be very successfully in computing Greeks and sensitivities of a portfolio with machine precision. Written by a leading practitioner who works and programmes AD, it offers a practical analysis of all the major applications of AD in the derivatives setting and guides the reader towards implementation. Open source code of the examples is provided with the book, with which readers can experiment and perform their own test scenarios without writing the related code themselves.
Numerical programs often use parallel programming techniques such as OpenMP to compute the program's output values as efficient as possible. In addition, derivative values of these output values with respect to certain input values play a crucial role. To achieve code that computes not only the output values simultaneously but also the derivative values, this work introduces several source-to-source transformation rules. These rules are based on a technique called algorithmic differentiation. The main focus of this work lies on the important reverse mode of algorithmic differentiation. The inherent data-flow reversal of the reverse mode must be handled properly during the transformation. The first part of the work examines the transformations in a very general way since pragma-based parallel regions occur in many different kinds such as OpenMP, OpenACC, and Intel Phi. The second part describes the transformation rules of the most important OpenMP constructs.
The goal of the Encyclopedia of Optimization is to introduce the reader to a complete set of topics that show the spectrum of research, the richness of ideas, and the breadth of applications that has come from this field. The second edition builds on the success of the former edition with more than 150 completely new entries, designed to ensure that the reference addresses recent areas where optimization theories and techniques have advanced. Particularly heavy attention resulted in health science and transportation, with entries such as "Algorithms for Genomics", "Optimization and Radiotherapy Treatment Design", and "Crew Scheduling".
Despite its name, automatic differentiation (AD) is often far from an automatic process. often one must specify independent and dependent variables, indicate the derivative quantities to be computed, and perhaps even provide information about the structure of the Jacobians or Hessians being computed. However, when AD is used in conjunction with a toolkit with well-defined interfaces, many of these issues do not arise. They describe recent research into coupling the ADIC automatic differentiation tool with PETSc, a toolkit for the parallel numerical solution of PDEs. This research leverages the interfaces and objects of PETSc to make the AD process very nearly transparent.
We list several potential problems that arise in differentiating parallel programs and present solutions for each of them. Some of the issues concern the correctness of the generated code, whereas others concern performance. While many of the issues have analogues in sequential programs, the solution is often quite different. In addition, some new concerns arise that are unique to parallel programs.