Distributed optimization on the base of AMPL modeling language and everest platform

Distributed Optimization on the Base of AMPL Modeling Language and Everest Platform doi 10 1016/j procs 2016 11 037 Peer review under responsibility of organizing committee of the scientific committee[.]

Procedia Computer Science Procedia Computer Science 101, 2016, Pages 313 – 322 YSC 2016 5th International Young Scientist Conference on Computational Science Distributed optimization on the base of AMPL modeling language and Everest platform Sergey Smirnov, Vladimir Voloshinov, and Oleg Sukhoroslov Institute for Information Transmission Problems of the Russian Academy of Sciences, Moscow, Russia sasmir@gmail.com, vladimir.voloshinov@gmail.com, sukhoroslov@iitp.ru Abstract Optimization modeling in science and industry requires the use of state-of-the-art software and high-performance computing resources A common problem faced by researchers is how to integrate related software and leverage available computing resources in a distributed environment The paper presents an approach for solving this problem based on unifying remote access to optimization software via RESTful web services and using AMPL (A Mathematical Programming Language) to describe scenarios of computing with optimization models This approach is implemented in AMPLX toolkit that enables modifying any AMPL script to solve problems by a pool of distributed solvers The toolkit is based on Everest platform that is used to expose optimization tools as web services and run these tools across distributed resources The proposed approach and AMPLX toolkit have been verified by a number of decomposition algorithms including branch-and-bound algorithm for a special nonlinear optimization problem Keywords: distributed computing, web services, REST, optimization, algebraic modeling languages Introduction Application of optimization models in science and business requires usage of state-of-the-art software and high-performance computing resources For a decades research community has focused on standalone tools such as solvers and algebraic modeling languages (AML) interpreters designed for a single computing server Despite the considerable progress in the area of distributed computing since 2000’s, integration of optimization tools into a distributed problem solving environment remains an important challenge and is far from completion The existing solutions are either too complex to be used by an unskilled researcher, or too limiting in ability to seamlessly incorporate all available computing resources In this paper we present an approach to deploy optimization systems in a heterogeneous computing infrastructure addressing these problems It is based on providing remote access to optimization tools via unified RESTful web services [1], flexible mapping of services to distributed computing resources and leveraging AML as a convenient domain-specific language Peer-review under responsibility of organizing committee of the scientific committee of the 5th International Young Scientist Conference on Computational Science © 2016 The Authors Published by Elsevier B.V doi:10.1016/j.procs.2016.11.037 313 Distributed optimization on the base of AMPL and Everest Smirnov, Voloshinov and Sukhoroslov We use AMPL (A Mathematical Programming Language) [2], one of the most popular AMLs familiar to optimization experts The proposed AMPLX toolkit is an open source extension of AMPL enabling to execute any optimization modeling algorithm that is presented as an AMPL script in a distributed environment The use of AMPLX consists in modifying of AMPL script in accordance with simple rules in order to exchange data with remote solvers via the regular AMPL interpreter If the AMPL script includes a phase of solving a number of independent optimization subproblems they will be solved in parallel by a pool of remote solvers AMPLX relies on Everest platform [3, 4] to expose optimization tools as RESTful web services and flexibly deploy and run these tools across distributed resources, such as standalone servers, computing clusters and grids The computational performance of AMPLX environment can be modified by adding or removing resources transparently to users In contrast to other AML interpreters, such as GAMS [5] or Xpress-Mosel [6, 7] we not know any other attempt to run AMPL in a distributed environment except rather restrictive NEOS-Kestrel utility [8] The paper is structured as follows Section provides an overview of optimization software Section discusses related work Section briefly overviews Everest platform and its relevant features Section presents basics notions of the AMPLX toolkit Section describes example of AMPLX usage Section concludes and discusses future work Optimization Tools Optimization modeling is an important tool for scientific research for more than 50 years In many cases the solution of optimization problems is not a main goal of researches but is a part of a more complex data processing workflow Therefore researchers have no incentive to spend much time doing low-level programming or tuning of numerical optimization methods In contrast, they need an easy to use, unified approach to solve such problems by reusing existing optimization software, e.g solvers The heterogeneity of optimization software and available computing infrastructure is one of the reasons complicating the wide practical usage of optimization models in a distributed computing mode, especially at a pilot phase of research There is a lack of tools that allow researchers to efficiently use existing software in distributed computing infrastructures while keeping the established optimization modeling practice Here we briefly overview common features expected from state-of-the-art solvers and other optimization software Firstly, it must accept canonical, mathematical programming (MP), formulation of optimization problems [9] As to semidefinite, optimal control and other advanced formulations, we not touch them here since they often may be reduced to MP-form So, further we consider the following optimization problem (with continuous and integer variables): fo (p, x) → min, x∈RN , x= (xc , xz ) , xc ∈RNc , xz ∈RNz x fi (p, x) ≤0 (i∈I) , gj (p, x) =0 (j∈J) , x∈M (p) (1) Above: fo (p, x) is an objective function to be minimized with respect to a finite number of constraints, inequality and equality, represented respectively by two collections of functions indexed by sets I and J (for inequality and equality constraints) fi (p, x)(i∈I) and gj (p, x)(j∈J) We knowingly distinguish variables (x) and parameters (p) of a problem Parameters p remain unchanged during search of decision variables x This separation of variables and parameters is very useful in practice As a rule, the preparation of optimization models to pass them to a solver takes a lot of time Moreover, very often researchers should compare optimal solutions for 314 Distributed optimization on the base of AMPL and Everest Smirnov, Voloshinov and Sukhoroslov a number of different sets of parameters (the so called “parameter sweeping”, “what-if” and case studies scenarios) The optimization software should support various classes of optimization problems (1) based on types of fo , fi and gj functions (on x), e.g linear programming (LP), quadratic programming (QP), “general” non-linear programming (NLP) and so on Another typical requirement for optimization software is to support state-of-the-art numerical optimization methods and their “concomitant characters” The established algorithmic and theoretical practice is based on first and second derivatives of problems functions, dual variables (αi for inequality constraints, βj for equalities) and Lagrange function ∇x fo (p, x) , ∇x fi (p, x) , ∇ fo (p, x) , ∇2xx fi (p, x) , ∇2xx gj (p, x) , x gj (p, x) , ∇xx αi fi (p, x) + βj gj (p, x) , ∇x L ( , x) , ∇2xx L ( , x) L (α, β, p, x) = fo (p, x) + i∈I (2) j∈J Note that the above formulations are used even for discrete problems with integer variables E.g the well-known branch-and-bound algorithm for mixed-integer problems uses relaxed subproblems when some of integer variables are considered as continuous The correct programming of (1) and (2) for passing to a solver via its low-level API may be very hard and time consuming, especially for complex optimization models The problem complicates more when solvers are used in a distributed computing environment Fortunately, the state-of-the-art solvers together with algebraic model languages and systems (AML/AMS) [10] provide a convenient workaround to that The most well-known examples of AMLs are GAMS (General Algebraic Modeling System), AMPL (A Modeling Language for Mathematical Programming), Xpress-Mosel, FICO Xpress Optimization Suite and Zimpl Except some syntactic differences all these systems have the following common features: - symbolic notation of optimization problems including indexed sets of parameters, variables, functions, constraints, formulas for objective and constraints functions etc.; - separation of symbolic notation and data values that define a particular instance of a problem (including numerical values of parameters, multi-dimensional sets of indices, etc.); - automatic differentiation (for NLP) for the first and second derivatives; - support of “duality” as an access not only to the values of decision (primal) variables, but also to Lagrange multipliers (dual variables) corresponding to an optimal solution; - presence of an AML interpreter to run high-level algorithms in the form of AML scripts that perform solution of one or multiple (probably dynamically generated) MP-problems; - seamless integration with solvers via open API (even for commercial AMLs) The general scheme of AML usage presented in Figure1 consists of the following steps The user prepares the symbolic description of the problem as a text file (*.mod) and specifies the values of symbolic sets and parameters in a data file (*.dat) Both files are processed by an AML interpreter to create the so called stub file (*.nl in AMPL) representing the problem in a form readable by AML-compatible solvers An AML-compatible solver returns the problem solution as a file (*.sol) which contains primal/dual variables and may be loaded back by an AML interpreter (e.g for generation of another problems and/or outputs) All these steps and additional processing of input data and solutions may be described in a special AML script which may be executed by an appropriate AML interpreter In case of Xpress-Mosel [6] the above general scheme is slightly modified: model, data and Mosel script are compiled into a binary executable (*.bim file) to be run by Mosel interpreter The use of AML is crucial at preliminary phases of any R&D project when structure of optimization model, data and general computing scheme are not established and may be chosen before diving into low level programming, e.g because of performance issues 315 Distributed optimization on the base of AMPL and Everest Smirnov, Voloshinov and Sukhoroslov Figure 1: The general scheme of AML usage Related Work The porting of optimization software into distributed computing environments has been addressed by several projects in the last decade In this section we provide a brief overview of these attempts and compare them with AMPLX, presented in Section NEOS Server, https://neos-server.org/, is a free web-based service for solving numerical optimization problems via remote access to almost 60 solvers for various types of problems NEOS provides an XML-RPC based Kestrel utility [8] to send problems to the hosted solvers Kestrel may be used from any correct AMPL script as a blocking remote call that does not allow to efficiently handle a number of independent problems in parallel AMPLX avoids this shortcoming by asynchronous processing of independent problems by a pool of available solvers Moreover, NEOS user is tied to the dedicated infrastructure and can not add new computing resources, solvers, etc In contrast, an administrator of the AMPLX system can add/remove computing resources dedicated for solvers at any time transparently for potential users COIN-OS project [11] proposes a number of specifications for data representation and network exchange enabling to deploy distributed system of optimization web services The so called OSxL standards are used to describe optimization problems, their solutions, solver options, optimization services registry etc These standards are based on XML, SOAP and WSDL specifications underlying the so called Big Web Services technologies These technologies became not so widely used as they were decade ago due to their complexity and proliferation of RESTful web services The OSxL standards also provide a few capabilities to describe complex (nondeterministic, iterative etc.) algorithms based on optimization models By reusing the established AML, AMPLX surpasses COIN-OS in usability: if user knows AMPL he must not learn more than 15 OSxL dialects to describe optimization models and computing scenarios GAMS Grid [5] is a GAMS extension for distributed computing It targets pools of machines controlled by a single resource managaer such as Condor or Sun Grid Engine Owing to Everest capabilities, AMPLX supports the use of ad-hoc computing infrastructures consisting of multiple standalone servers, clusters controlled by different batch systems and even resources of grid infrastructures Moreover, as it can be seen from the NEOS Server access statistics , GAMS language loses to AMPL on the breadth of usage in the recent years Pyomo [12] is an open source software package that supports a diverse set of optimization capabilities for formulating, solving, and analyzing optimization models Pyomo provides an http://www.neos-server.org/neos/report.html 316 Distributed optimization on the base of AMPL and Everest Smirnov, Voloshinov and Sukhoroslov open source alternative for AMPL by enabling symbolic description of optimization problems and computing scenarios on Python language involving AMPL-compatible solvers and AMPL data files Pyomo also provides distributed computing capabilities based on Pyro, a Python library for building distributed applications The main disadvantage of Pyomo is much (often 10 times) worse performance when generating AMPL stubs compared to the native AMPL interpreter, particularly for large (multi-dimension) mathematical programming problems Some important AMPL features, e.g import of user defined C/C++ functions into models, are currently at a beta release phase Nevertheless, Pyomo can compete with AMPLX if researchers are willing to save some funds by avoiding purchasing of a licensed AMPL interpreter and are ready to migrate from AMPL to the Python language Mosel-Xpress [6, 13, 7] is a commercial system included in the Xpress Optimization Suite environment developed by FICO company It is the most advanced among similar toolkits for distributed optimization modeling In 2010 FICO presented a new, substantially upgraded version of Mosel language and interpreter that, among other features, provided native support for distributed computing Users can program distributed computing algorithms based on the event driven programming paradigm directly in the high-level Mosel language In contrast to other interpreted AMLs, Mosel-Xpress has to be compiled into intermediate byte-code running by a special virtual machine The system supports two protocols for exchanging data between parallel computing activities: SHMEM (shared memory) and MEMPIPE (inter processor) Both protocols are low-level and require a homogeneous computing infrastructure such as a server or a computing cluster FICO provides Mosel-Xpress in a form of a private cloud service deployed on the internal computing infrastructure In contrast to Mosel-Xpress, AMPLX is open source and supports a wider range of ad-hoc computing infrastructures Everest Overview AMPLX is based on Everest [3, 4], a web-based platform for building computational web services This section contains a brief overview of this platform Everest provides users with tools to quickly publish and share computing applications as services The platform also manages execution of applications on external computing resources attached by users In contrast to traditional distributed computing platforms, Everest implements the Platform as a Service (PaaS) model by providing its functionality via remote web and programming interfaces A single instance of the platform can be accessed by many users in order to create, run and share applications with each other The platform is publicly available online to all interested users [4] Applications are the core entities in Everest that represent reusable computational units that follow a well-defined model An application has a number of inputs that constitute a valid request to the application and a number of outputs that constitute a result of computation corresponding to some request Upon each request Everest creates a new job consisting of one or more computational tasks generated by the application according to the job inputs The tasks are executed by the platform on computing resources specified by a user The dependencies between tasks are currently managed internally by applications The results of completed tasks are passed back to the application and are used to produce job outputs or new tasks if needed The job is completed when there are no incomplete tasks are left The described model is generic enough to support a wide range of computing applications Users can publish applications via provided generic application template that makes it possible to avoid programming The template supports running arbitrary applications with command-line interface and produces a single task corresponding to a single command run 317 Distributed optimization on the base of AMPL and Everest Smirnov, Voloshinov and Sukhoroslov Recent developments made it possible to dynamically add new tasks or invoke other applications from a running application via the Everest API This enabled users to create and publish complex many-task applications such as workflows or AMPLX toolkit presented in this paper Everest also provides a built-in generic application for running parameter sweep experiments An application is automatically published as a RESTful web service [1] This enables programmatic access to applications, integration with third-party tools and composition of applications into workflows The platform’s web user interface also generates a web form for running an application via web browser An application owner can manage the list of users that are allowed to run the application Instead of using internal computing infrastructure, Everest performs execution of application tasks on external resources Users can attach computing resources to the platform and flexibly bind these resources to applications The platform implements integration with standalone machines and clusters by using a developed program called agent The agent runs on the resource and acts as a mediator between it and Everest enabling the platform to submit and manage computations on the resource The platform also supports integration with other types of resources, such as the European Grid Infrastructure Everest manages execution of tasks on remote resources and performs routine actions related to staging of task input files, submitting a task, monitoring a task state and downloading task results The platform also monitors the state of computing resources and uses this information during job scheduling It is possible to run an application on multiple resources In this case the platform performs dynamic scheduling of application tasks across the specified resources From this point of view Everest can be seen as a multitenant metascheduling service Principles and architecture of AMPLX toolkit The goal of AMPLX development is to run any AMPL script by the standard interpreter communicating with remote solvers installed in distributed computing environment Namelly: 1) all MP problems either predefined or dynamically composed during running of the script will be sent to remote solvers; 2) set of independent subproblems will be solved in parallel by a pool of computing resources whose configuration might be changed transparently for users Besides software components AMPLX defines a set of rules for modifying an AMPL script in accordance with AMPLX requirements Figure illustrates AMPLX architecture and principles of its functioning (regardless of computing workflow) AMPLX script is executed by the standard AMPL interpreter AMPLX includes a number of small files amplx_*.amp (a kind of macroses) containing AMPL operators for: setting a few AMPLX parameters ( amplx_*); writing AMPL stubs for a set of optimization problems; interacting with dedicated Everest application solve-ampl-stub via Everest Python API This application acts as a front-end for a pool of solvers (SCIP, COIN-CBC, IPOPT etc.) covering basic types of mathematical programming problems (LP/MILP/NLP/MINLP) These solvers are installed on heterogeneous resources (standalone servers, clusters, clouds etc.) attached to the solve-ampl-stub by the Everest agent Figure provides an example of AMPLX use On top of the figure excerpts of some AMPL script are presented At the beginning solver (IPOPT here) is chosen, then after some data preparation there is a for-loop where a set of independent subproblems are being prepared and solved in turn Below the same excerpts modified in accordance with AMPLX rules are shown First of all, an initialization AMPLX macros amplx_globals.amp is called This macros declares a set of parameters that will be used by AMPLX For example, amplx_solver and amplx_solver_opts_file are used to set solver type and solver’s options file passed 318 Distributed optimization on the base of AMPL and Everest Smirnov, Voloshinov and Sukhoroslov Figure 2: AMPLX architecture Figure 3: Example of AMPL to AMPLX conversion to the solve-ampl-stub by Python subsystem of AMPLX Next, the original loop is replaced with three parts: 1) loop for creating stub files for the set of subproblems (without solving) by using amplx_probName parameter, set of subproblem names amplx_probSet and amplx_write_problem_stub.amp macros; 2) passing of created stubs to the solve-ampl-stub application for parallel solving (results of optimization will be returned from solvers and written to the local folder as a set of *.sol files); 3) processing of solutions in the same way as has been done in the original loop The source code of AMPLX toolkit is available at http://gitlab.com/ssmir/amplx The system may be used either directly at the host with installed AMPL or remotely via run-amplx Everest application, https://everest.distcomp.org The demo AMPLX scripts for DanzigWolfe decomposition can be found at http://distcomp.ru/~vladimirv/restopt/amplx/dw/ 319 Distributed optimization on the base of AMPL and Everest Figure 4: X-Ray diffraction (on nuclei and electronic shells) Experiment and modeling (solid line) Smirnov, Voloshinov and Sukhoroslov Figure 5: Neutron diffraction (on nuclei only) Experiment and modeling (solid line) Example of AMPLX usage The toolkit have been already used to solve several real optimization problems In this section concern one of such problems related to Carbonaceous Structure Identification as Inverse Optimization Problem An algorithm has been developed for structure identification of amorphous carbonaceous nanomaterials with a joint x-ray and neutron diffraction data analysis, using the data on the chemical composition of the sample from other diagnostics [14, 15] Optimization criteria are to minimize a gap between experimental and model data for X-Ray and neutron diffractometries (we use three independent criteria): K m  n      z Xr  + (1−K) z Ne  j k j=1 k=1 n   m   2 2 zjXr + (1−K) zkNe j=1  k=1    K max zjXr  + (1−K) max zkNe  K j=1:m k=1:n −−−−−−−−−−−−−→ (criterion L1 ) ; −−−−−−−−−−−−−→ (criterion L2 ) ; −−−−−−−−−−−−−→ (criterion L∞ ) ; − (zXr ,zNe ,x,y,a,A,c,B) (zXr ,zNe ,x,y,a,A,c,B) (zXr ,zNe ,x,y,a,A,c,B) (3) with the following common constraints: Xr (qi ) − zjXr = Sexp N  i=1 N  Ne (qk ) − zkNe = Sexp N  i=1 xi + a = A, Xr Xr xi ·SiXr (qj ) − a·SC (qj ) − A·Simpur (qj ) (j = : m), i=1 N  i=1 Ne Ne yi ·SiNe (qk ) − c·SC (qk ) − B·Simpur (qk ) (k = : n), yi + c = B, xi A = yi B (i=1 : N ), a A = c B (4) (xi , yi , a, A, c, B)  Let’s comment (3, 4) without going into details (see [14] for them) Input data are as follows: Xr Ne Sexp (qi ) and Sexp (qk ) are the experimental X-ray and neutron scattering intensities, respectively (see Figures 4, 5); j and k are indices of discrete values of scattering vector modulus q for Xray and neutron data; m and n are the total numbers of scattering vectors for X-ray and neutron data; N is the total number of nanostructures selected for optimization; SiXr (qj ) and SiNe (qk ) are X-ray and neutron model scattering intensities curves calculated for the i-th carbon 320 Distributed optimization on the base of AMPL and Everest Figure 6: Successive solution of L1 , L2 , L∞ Smirnov, Voloshinov and Sukhoroslov Figure 7: Concurrent solution of L1 , L2 , L∞ Xr Ne Xr Ne nanostructure; parameters SC (qj ), SC (qk ), Simpur (qj ) and Simpur (qk ) are similar ones relating to the amorphous carbon medium (C) and to all the impurities in the samples Presence of last equalities in (4) (bilinear by variables) makes all three problems (L1 , L2 , L∞ ) non-convex Therefore a direct application of nonlinear local solver, e.g IPOPT, may return solution that is not a global optimum It was the reason of developing a special branch-andbound algorithm to find an approximate solution of three problems with predefined accuracy (relative gap to unknown optimal objective value) This algorithm has been implemented as an AMPLX script involving the use of NLP and LP solvers to solve auxiliary bound subproblems with relaxed to linear last equalities in (4) The algorithm has been published in https: //everest.distcomp.org/apps as the optBnB_Nlp-2-amplx Performance of distributed computing scenarios depends mainly on that how subtasks are dispatched over available processors Everest Python API enables to analyze quality of that dispatching We provided AMPLX with a kind of trace visualization tool (based on Everest Python API and http://jkff.info/software/timeplotters/ This tool helps to evaluate effectiveness of a scenario and to conceive possible improvement Returning to the problem of this section, Figure shows an execution trace of AMPLX run corresponding to the case when all three optimization problems are solved in turn The execution of approximately 60 subtasks took 25 minutes, while no more than solvers of 16 available ones worked in parallel Figure shows a similar trace for a run corresponding to a simultaneous execution of three algorithm sessions for L1 , L2 , L∞ respectively In this case, due to a more efficient use of available resources, all three jobs took about 10 minutes with 13 solvers run in parallel Conclusion We have presented AMPLX, an open source extension of AMPL enabling the execution of any optimization modeling algorithm presented as an AMPL script in a distributed environment It became a promising tool that is already using for development of decomposition, many-task, algorithms for global and discrete optimization problems For example, it is used for intensive testing of local elimination algorithm for discrete optimiztion problems with sparse quasi-block constraints structure [16] Every execution of the algorithm for problems with 100 variables and from 50000 to 100000 constraints leads to solving of about 1000 MILP subproblems As to AMPLX services deployed on Everest platform, we plan to significantly increase the capacity of computing infrastructure associated with solve-ampl-stub application Future 321 Distributed optimization on the base of AMPL and Everest Smirnov, Voloshinov and Sukhoroslov work will also include the use of recent Everest developments to implement an exchange of information between solvers running within an AMPLX session We are also investigating the development of a fully open source solution similar to AMPLX on the base on Pyomo [12] Though Pyomo generates AMPL stubs slower than the native AMPL interpreter, it allows to parallelize the process of stub generation which is rather time consuming for large optimization problems Acknowledgments This work is supported by the Russian Science Foundation (project No 16-11-10352) References [1] Leonard Richardson and Sam Ruby RESTful web services ” O’Reilly Media, Inc.”, 2008 [2] Robert Fourer, David M Gay, and Brian W Kernighan AMPL: A mathematical programming language Thomson/Brooks/Cole, 2003 [3] O Sukhoroslov, S Volkov, and A Afanasiev A web-based platform for publication and distributed execution of computing applications In Parallel and Distributed Computing (ISPDC), 2015 14th International Symposium on, pages 175–184, June 2015 [4] Everest [online] http://everest.distcomp.org/ [5] Michael R Bussieck, Michael C Ferris, and Alexander Meeraus Grid-enabled optimization with gams INFORMS Journal on Computing, 21(3):349–362, 2009 [6] Xpress-mosel user guide [online] http://www.fico.com/en/wp-content/secure_upload/ Xpress-Mosel-User-Guide.pdf [7] Y Colombani and S Heipcke Multiple models and parallel solving with mosel xpress optimization suite white paper Technical report, FICO, 2014 http://www.fico.com/en/latest-thinking/ white-papers/multiple-models-and-parallel-solving-with-mosel [8] Elizabeth D Dolan, Robert Fourer, Jean-Pierre Goux, Todd S Munson, and Jason Sarich Kestrel: An interface from optimization modeling systems to the neos server INFORMS Journal on Computing, 20(4):525–538, 2008 [9] C.A Floudas and P M Pardalos (Eds.) Encyclopedia of Optimization 2-nd Edition Springer & Business Media, 2009 [10] J Kallrath (Ed.) Algebraic Modeling Systems: Modeling and Solving Real World Optimization Problems, volume 104 Applied Optimization, Springer Science & Business Media, 2012 [11] Robert Fourer, Jun Ma, and Kipp Martin Optimization services: A framework for distributed optimization Operations Research, 58(6):1624–1636, 2010 [12] W.E Hart, C Laird, J.-P Watson, and D.L Woodruff Pyomo–optimization modeling in python, volume 67 Springer, 2012 [13] Susanne Heipcke Xpress–mosel:multi–solver, multi–problem, multi–model, multi–node modeling and problem solving In Algebraic Modeling Systems, volume 104, pages 77–110 Springer, 2012 [14] V.S Neverov, V.V Voloshinov, A.B Kukushkin, and A.S Tarasov Algorithm of amorphous carbonaceous nanomaterial structure identification with a joint x-ray and neutron diffraction data analysis, 2014 http://arxiv.org/abs/1301.3418 [15] A Kukushkin, V Neverov, N Marusov, I Semenov, B Kolbasov, V Voloshinov, A Afanasiev, A Tarasov, et al Few-nanometer-wide carbon toroids in the hydrocarbon films deposited in tokamak t-10 Chemical Physics Letters, 506(4):265–268, 2011 [16] Oleg Shcherbina Graph-based local elimination algorithms in discrete optimization In Foundations of Computational Intelligence Volume 3, pages 235–266 Springer, 2009 322 .. .Distributed optimization on the base of AMPL and Everest Smirnov, Voloshinov and Sukhoroslov We use AMPL (A Mathematical Programming Language) [2], one of the most popular AMLs familiar to optimization. .. features Section presents basics notions of the AMPLX toolkit Section describes example of AMPLX usage Section concludes and discusses future work Optimization Tools Optimization modeling is an... for optimization; SiXr (qj ) and SiNe (qk ) are X-ray and neutron model scattering intensities curves calculated for the i-th carbon 320 Distributed optimization on the base of AMPL and Everest