

Tasks¶

Tasks are the primary workhorse of AO's backend.

Task Types¶

Docker (`task:docker`)¶

Docker tasks will be run within a docker container and use stdin and stdout to provide input and capture output. This allows users to use any environment necessary to run analysis and perform tasks.

Docker Documentation

API (`task:api`)¶

A thin wrapper to provide an easy way to integrate custom API calls into AO's system.

Task Detail¶

The task detail page provides the capability to create and edit tasks. Tasks are instatiated with an event and create a task run

Task Selector¶

The task selector is responsible for determining where and when tasks can be run.

type filter: Task runner will attempt to prefix match on this type

ex. asset:gen will allow this task to show up for all asset:gen types and their children
task contract: Used to filter tasks in projects or scenarios. Certain steps will require a specific contract and this location denotes which contract it fulfills. This will be given more robust support in the future.

ex. tc:resource:solar requires inputs lat, lng, and will output a resource file
ignore type: optionally ignore certain tasks for the task runner
queue: which queue to place the task into

Task Parameters¶

Task parameters allow you to add custom parameters to tasks, perform any mapping on event objects, and provide validation details for events.

Parameter Config¶

Parameter config is used to configure custom parameters to be used within your task.

This parameter config adds a parameter for min and max acres used in a parcel search

acre_max:
  formtype: number
  key: acre_max
  name: Max Acres
  order: 20
acre_min:
  formtype: number
  key: acre_min
  name: Min Acres
  order: 10

Parameter Map¶

The parameter map is used to transform the event object before it is passed to the task.

Transform a data object to sync down to workspace

The following example adds a new field to the event object under the property ao_syncDown.

ao_syncDown is an array of item refs that are to be downloaded and stored on the workspace with a given filename. An example array item is

{
    rn: 'team/<team>/file/<fileid>',
    filename: 'test.json'
}

The maps are applied from top to bottom, so the bottom have highest preference and will overwrite previous values.

This example has 3 possible names for the file, a default file name, the name of the data item, and finally a filename specified directly on the event object

The data property map will pass through the data key from the event input

yaml

data:
  value: data
syncDownFileA:
  fixed: default-name.data
  key: 'ao_syncDown[0].filename'
syncDownFileB:
  key: 'ao_syncDown[0].filename'
  value: data.parameter.name.name
syncDownFileC:
  key: 'ao_syncDown[0].filename'
  value: filename
syncDownRn:
  key: 'ao_syncDown[0].rn'
  value: data.rn

Validator Config¶

Validator config is used to determine whether a task is ready to be run.

Validate parameters for either a radius search or a wkt_search

This example has two options it can validate. If either succeeds, the task can be run.

The first looks to ensure there is a lat, lng, and buffer that is used as a radius

The second looks for a wkt geometry and a buffer.

yaml

radius_search:
  checks:
    - get: lat
      name: Latitude
    - get: lng
      name: Longitude
    - get: buffer
      name: Buffer
  description: 'Require Lat,Lng and Radius'
  name: Radius Search
wkt_search:
  checks:
    - get: wkt
      name: WKT Geometry
    - get: buffer
      name: Buffer
  description: Require WKT Geometry
  name: WKT Geometry Search

Environment Variables¶

Custom environment variables can be added to the docker container and used within the arguments for starting the container.

Rest API¶

Used to configure API calls

url: Url for the call
checkurl: An endpoint to query before calling the endpoint to check for specific information
querystringmap: A config block to map the input event to the query string

This is only found for type task:api

This will be updated to be more versatile in the near future

Runtime¶

The runtime describes the environment in which your task is run and is mapped to Docker input.

arguments: The arguments to pass into the entry point
entry point: Entry point into your script
docker image: Docker image to use for task
docker options: Options to add to docker

This is only found for type task:docker

Docker options is not implemented currently for security concerns

Workspace¶

The workspace determines which filesystem is mounted and accessible through docker.

filter type: Used to filter tasks. Should be in task selector parameters
workspace: Which workspace to mount.

This is only found for type task:docker

File¶

This is a temporary parameter that allows for an easy way to change and update files for a given task. Eventually the file system will be better integrated into the task developer.

filename: Filename to store file
contents: Python file contents

This is only found for type task:docker

Temporary parameter that is to be deprecated by 2019 pipeline

Event¶

This is a temporary parameter that allows for an easy test event for a given task. Eventually there will be a way to save, copy, and list custom events for use in building tasks.

data: JSON event

Temporary parameter that is to be deprecated by 2019 pipeline

Sample Events¶

This sample event will download a specific file item to the workspace and save as test.bmp

json

{
  "ao_syncDown": [
    {
      "filename": "test.bmp",
      "rn": "team/demo-v3/file/-LHK7ldtBTKc5fqIe-zQ"
    }
  ]
}

Notes¶

Notes can be provided in markdown in the parameter group Notes. It will be formatted and displayed in the task summary.

notes: notes provided as markdown

Input

Parsed to HTML

Task Run¶

Task runs are an instatiation of a task with an event.

Output¶

Output of a task run.

The output object can be used to create items within AO. Currently, the task manager can create file, data, asset, and project types.

Create a project with the task manager

An output ref will be created upon succesful creation of the item

json

{
"project": [
      {
        "tid": "demo-v3",
        "item": {
            <project item>
        }
      }
    ]
}

Manager Logs¶

Container Logs¶

Output¶

Output Refs¶

When items are created from a task, an output ref is created that links to the new item.

Tasks¶

Task Types¶

Docker (task:docker)¶

API (task:api)¶

Task Detail¶

Task Selector¶

Task Parameters¶

Parameter Config¶

Parameter Map¶

Validator Config¶

Environment Variables¶

Rest API¶

Runtime¶

Workspace¶

File¶

Event¶

Sample Events¶

Notes¶

Task Run¶

Output¶

Manager Logs¶

Container Logs¶

Output¶

Output Refs¶

Docker (`task:docker`)¶

API (`task:api`)¶