AggregatorX - software design

This document explains how (and to some degree why) the package AggregatorX is designed and used. The main purpose is to enable users to efficiently extend and contribute to the code.

Introduction

Usage

Before diving into the details let us give a breif description of the motivation of the software and a typical workflow.

The main motivation for AggregatorX is to create a piece of software that simplifies the analysis of flexible energy resources and their participation in multiple energy and balancing markets. By analysis we here mean optimization models that try to optimize the scheduled energy flow according to some defined profit. The idea is that most flexible energy resource and markets have many similar features and the software automates all the manual work of setting up the the optimization model (variables, constraints, objective function) based on a high level description of the system under study.

Work flow

Here is a short description of atypical the work flow, hopefully to illustrate the simplicity of setting up and running an optimization model:

Describe your system of interest in a JSON file. This file is refered to as the system description. This file can have any name but typically called system.json and we will use this as a name in the following.
Import the AggregatorX package, using AggregatorX.
Build the system using aggregator = buildaggregatorx("system.json"). What this does is to create objects of AggregatorX types with the number and types of objects and their parameters based on the content of system.json. The aggregatorX objects are stored in the variable aggregator as a dictionary.
Create and run an optimization of the system with optimizeaggregator(aggregator, optimizer). This creates a JuMP model based on the information in the aggregator object. It then tries to optimize the model using the solver referred to by optimizer.

Some words (of wisdom)

A conceptual and mathematical description of what the software does is provided in the document tex\mathematical-description.tex. It is probably a good idea to read this document first to understand to what the software tries to acheive, before diving into the nitty-gritty of the software design.

There are also some design-choices that might be somewhat non-intuitive (hopefully they will gradually become intuitive, otherwise it was probably a bad design-choice...). We provide a list of these here in an attempt at explnation (and justification) of the choice. You can skip this on a first read and refer back to it when necessary.

The objective sees to maximize profits. Any revenue (e.g. selling energy) should therefore be positive, and costs (e.g. buying energy) should be negative.
Parts. You will see later that there is a hierarchy of different types in the system (e.g. Resources, Markets, etc.). To refer to all of them we will use the term parts (also in code). Hence, anything that is a subtype of AggregatorXAny is a part (you can think of all the parts that make the clockwork tick...)

Some users may also not be familiar with the Julia programming language or the JuMP modelling language. We therefore provide a list of some terms that are used in the following that might be unfamiliar for these readers. Again, this may be skipped on a first read, and refered back to when confusion sets in.

VariableRef. This is a Julia type defined by JuMP. An instance of this type is a reference to an internal JuMP optimization variable. I.e it provides access to the optimization variable from the scope where the VariableRef is defined. A Variable ref is returned by JuMP macros that create variables.

Software design

OK, let's take a deep dive into the nitty gritty (Oh boy, I can't wait...)

File structure

Let us first point to where you can find things.

AggregatorX.jl

This is where the main module is defined. It only contains a list of function which the module exports (available when using the package) as well as an include statement for all the files where all the other code as been organized.

AggregatorXComponents.jl

The type system is often an essential part of a piece of Julia software and important for software design. This file describes the hierarchy of new abstract types defined in AggregatorX as well as all the conrecte types (i.e structs) that may be instantiated. These concrete types typically represent physical (e.g. battery) or conceptual (e.g. a market) objects in the system we want to study. When you want to create a new type to represent some new physical object, you start here by defining the necessary fields that describe the characteristics of the object. The system description you write provide the parameter values that go into these types during a particular run of the software.

AggregatorXExceptions.jl

Defines specialized exceptions that are used by the package to provide detailed error information.

BuildAggregatorX.j

This file defines the function buildaggregator(). What this function does is to take the path of the system description as an argument and return a dictionary that contains instances of the various components defined in AggregatorXComponents.jl. It basicall translates what you have defined in your system description to the internal data structure of AggregatorX. The function first parses the system description file into individual components. Each component has a type field in the JSON file. The software translates this to an AggregatorX type using a dictionary (called typetable) which translates string type descriptions to a Julia type. It then calls build_aggregatorx_object() with the type as an argument. This function call dispatches to different functions depending on the type argument, which contains the appropriate code for that given type. All the instantiated objects are grouped in a dictionary which is returned by buildaggregator(). We will refer to this dictionary as the aggregator object/dictionary.

AggregatorXConstructors.jl

This file contains the definitions of all the different build_aggregatorx_object() functions called from buildaggregator(). Each of these functions takes the data from the system description file and instantiates the AggragatorX objects with the appropriate fields with data from the system description. Each function returns the instantiated object.

OptimizeAggregator.jl

This file contains the single function optimizeaggregator(). This function does two things. First it sets up the optimization problem based on the information in the aggregator object. This is done by repeated calls to set_optimization_variables(), set_optimization_constraints() and set_objective() which respectively defines the variables, constraints and objective function in a JuMP model. Finally it calls optimize!(model) to try to optimize the model and return the (hopefully) feasible solution.

AggregatorXMethods.jl

This is the meat and potatoes of the software. This is were the JuMP optimization model is set up. There are specialiced functions for each type of component. The most complex part of the software is how the constrains are set up.

plz_solve_my_problem.jl

This friendly file solves whatever problems you might have:D. Just kidding, it is just a script that contains all the commands needed to run the whole optimization procedure. All you need to do is changed the system description file.

Software design

Location of system description and data files

AggregatorX needs some data to work with. The system that you wish to analyse is stored in a *.json file. This is assumed to be located in the working directory of the current Julia instance. However the global variable SYSTEMDIR might be set using the function setsystemdir(directory) where directory is relative to julia working directory

The system description might load parameters from other filese rather than hardcoding them directly. The location of these files might changed by changing the global variable DATADIR using setdatadir(directory) where the directory is releativ to SYSTEMDIR, ie. wd + systemdir + directory.

The AggregatorX type hierarchy

The AggregatorX software defines a set of new abstract and concrete types (The concrete types are akin to classes in other OO languages. Abstract types cannot be instantiated, but can be used for dispatching on functions). All the information about the system under analysis is stored in instances of these types.

There is a hierarchy of abstract types. Not all levels of the hierarchy are in use, but has been implemented to provide conceptual overview of the types and with the idea that a sensible hierarchy will make extending the code easier in the future.

At the top is AggregatorXAny, which is extended by three abstract types Components, Groupsand TimeStruct.

Components represent concrete elements in the system, either physical objects (batteries, EV charger, connections) or markets. It is the parent of five abstract types Resources, Markets, Node, Grids and AbstractConnection. Resources are components that represent flexible energy resources, conceptually they are either generation/loads or storage units. Marketsare components where energy is bought or sold and typically adds terms to the objective function. Grid componets represent limitations or cost due to local distribution grids. Node represents a distribution point of energy among resources and markets. AbstractConnection represents connections between the various components.

One can visualize the Components as the nodes of the graph that represent the system. Conceptually one can think of the components as entities that exchange energy.

Groupsare collections of resources that can participate in balancing markets. The groups keep track of reserved capacity and activated energy of the resources in the group and can sell this to various balancing markets.

TimeStruct represents the timestep used in the system.

The aggregator dictionary

The aggregator object is a dictionary with string keys that refer to particular abstract types: TimeStruct, Market, Resource, Node, Grid, Connection. The key points to a vector of instances of concrete types that are subtypes of these abstract types. This dictionary holds all the information about the system.

Keeping track of the energy

In general, each component has a dictionary power that reprsent the power flowing from the component. Each key in the dictionary is the id of a component that is connected to an output from the component, and the value is a vector of VariableRef that. Currently the node component is the only component that has multiple outputs, but the dictionary structure is used to maintain some unity in implementation and for flexibility in future versions that might require additional flexibility.

Components

Resources

Markets

Concrete market subtypes must implment at least the following fields:

type - type of market, ie the concrete type
power - the power flowing to or from the market (Dict(Integer->Vector(VariableRef)))
price - the cost of energy
resource - the resource (component) the market is connected to (note this is different from source in other componets which is the input of the component. Resource can be both input and output depending on wether energy is bought or sold in the market.)
sign : 1 if the market is a source of energy, (-1) if the market is a sink of energy.
id - A unique identifier.

The framework assumes that a market is only connected to a single component.

The concrete market objects are stored in a vector of type market. This vector is stored in a dictionary value in the aggregator object

The markets do not in general have to impose any real constraints. However, for markets that are sinks of energy (energy is sold to the market) the power is constrained to be equal to the output power of the connected resource (this is just a convenience implementation, having the power to the market available in the market object is more convenient than having to query the power in the resource connecte to the market)

Node

A node is a lossless transmitter of energy between componets.

A node has the following fields

power A dictionary where the key is a target component and the value is a vector of VariableRef that represents the flow of energy to each resource
sources A vector of integers that represent the id of the components that send energy to the node.
id

In the JSON file the node simply needs to set and id (and connections).

Buildaggregator calls constructor for each node in json list. The constructor resolves connections and sets power and sources.

Optimizeaggregator sets the inputs equal to the outputs. No new variables are needed.

Grids

Connections

Simplify determines which components are connected, i.e. can exchange energy. For balancing markets, the Group object defines which resources are connected to the different markets.

Groups

TimeStruct

Making new components

An important feature of AggregatorX is that new components can be added to represent physics, markets, resources or behaviour that are not possible with the existing library of components. This section describes the steps that are necessary for adding new components to the software. It simultanously suggest a best practice workflow where some things are not absolutly necessary but will decrease likelihood of errors and make the software and ecosystem easier to maintain.

Start with a mathematical description, preferably in the style of the Mathematical description of AggregatorX.
Add new type description.
- First write a test that creates an object of the struct (In the following, always start each step by writing a test. Much less likelihood of logical errors and much quicker to remove simple typos).
- Implement the new struct/type in AggregatorXComponents.
- Add type to the export list of the package and run the test.
Add constructor for the new type
- Write a test that creates the object. For this you need a system description that includes the new component. The test should then call buildaggregator() using the system description.
- Write a method in AggregatorXConstructors that returns the initalized object (make sure it will be called from buildaggregator - check calling signature) and test it.
Add methods to set variables, constraints and objective terms.
- Make a test for each method. The type of test will depend on the particular component and method content.
- Implement method and test it.
Make test cases
- One or more test cases that implements the component and with analytical results
- Focus on edge cases
- Test large systems

Type description

The first step is to define the type as a mutable struct in the AggregatorXMethods file. Making the concrete type a subtype of a meaningful parent is important. Some methods dispatch on the parent type. The necessary field types will vary somewhat between the types. All components must have an id field.

The type must be added to the export list of the package in AggregatorX.jl

The "type" key in the JSON file refers to this concrete type and used to dispatch the correct constructor.

Make constructor

Methods to define optimization problem

Testing

The script runtests.jl runs a series of test that can be used to verify system behavior after changes have been maded. It uses system descriptions from the test-systems folder.

Working with the solution

Here are some tips for how to look at the output from the optimization. It primarily a review of key JuMP commands

The JuMP model is stored in the variable model. This object contains all the results from the optimization.

Before doing anything else, first check that an optimum has been found is_solved_and_feasible(model). If that is the case then information about the solution can be found with solution_summary(model).

Next you would probably like to acces the values of the variables in the optimal solution. To do this you need a reference a VariableRef object that points to a particular variable in the model. These reference are stored in the the different components. There are several ways to get this. One quick way is if you know the id of the component (see the system description). You can then use get_component(id, aggregator) which returns a reference to the component. The component typically has a field (e.g. power) which stores a VariableRef object. Letting the VariableRef object be called ref, you can then use value(ref)to get its value. Usually one gets a Vector of VariableRef and you have to use the broadcasting operator in Julia ., value.(ref).

Another way is to access the variables by name. You can find these from all_variables(model) that lists all variable names. To get a reference to the variable you can use variable_by_name(model, "name")