# Taskgraph Overview Taskgraph builds a directed acyclic graph, where each node in the graph represents a task, and each edge represents the dependency relationships between them. See Taskgraph's [graph generation documentation] for more details. ## Decision Task The decision task is the first task created when a new graph begins. It is responsible for creating the rest of the task graph. The decision task for pushes is defined in-tree, in `.taskcluster.yml`. That task description invokes `mach taskcluster decision` with some metadata about the push. That mach command determines the required graph of tasks to submit, then calls the Taskcluster API to create the tasks. :::{note} `mach taskgraph decision` is *not* meant to be invoked by humans. Instead, follow [this guide] (prepending `mach` to all commands) to invoke Taskgraph locally without submitting anything to Taskcluster. ::: (kinds)= ## Kinds Generation starts with "kinds". These are yaml files that denote groupings of tasks that are loosely related to one another. For example, in Gecko there are `build` and `test` kinds. Each kind has its own directory under {searchfox}`taskcluster/kinds` which contains a `kind.yml` file. For more information on kinds, see Taskgraph's [kind documentation]. For a list of available kinds in `mozilla-central`, see the {doc}`kinds reference `. ## Loader Next, a "loader" is responsible for parsing each `kind.yml` file and turning it into an initial set of tasks. Loaders will always parse kinds in an ordering satisfying the `kind-dependencies` key. See Taskgraph's [loader documentation] for more details. (transforms)= ## Transforms After the initial set of tasks are loaded, transformations are applied to each task. Transforms are Python functions that take a generator of tasks as input, and yields a generator of tasks as output. Possibly modifying, adding or removing tasks along the way. See Taskgrpah's [transforms documentation] for more details on transforms, or the {doc}`transforms section ` for information on some of the transforms available in `mozilla-central`. ## Target Tasks After transformation is finished, the [target_tasks] module filters out any tasks that aren't applicable to the current {doc}`parameter set `. ## Optimization After the "target graph" is generated, an optimization process looks to remove or replace unnecessary tasks in the graph. For instance, a task may be removed if the push doesn't modify any of the files that could affect it. See Taskgraph's [optimization documentation] for more details, or the {doc}`optimization strategies ` available in `mozilla-central`. ## Task Parameterization A few components of tasks are only known at the very end of the decision task -- just before the `queue.createTask` call is made. These are specified using simple parameterized values, as follows: `{"relative-datestamp": "certain number of seconds/hours/days/years"}` : Objects of this form will be replaced with an offset from the current time just before the `queue.createTask` call is made. For example, an artifact expiration might be specified as `{"relative-datestamp": "1 year"}`. `{"task-reference": "string containing "}` : The task definition may contain "task references" of this form. These will be replaced during the optimization step, with the appropriate taskId for the named dependency substituted for `` in the string. Additionally, `decision` and `self` can be used a dependency names to refer to the decision task, and the task itself. Multiple labels may be substituted in a single string, and `<<>` can be used to escape a literal `<`. `{"artifact-reference": "...."}` : Similar to a `task-reference`, but this substitutes a URL to the queue's `getLatestArtifact` API method (for which a GET will redirect to the artifact itself). (taskgraph-graph-config)= ## Graph Configuration There are several configuration settings that are pertain to the entire taskgraph. These are specified in {file}`config.yml` at the root of the taskgraph configuration (typically {file}`taskcluster`). The available settings are documented inline in {searchfox}`taskcluster/gecko_taskgraph/config.py`. (taskgraph-trust-domain)= ## Action Tasks Action Tasks are tasks which perform an action based on a manual trigger (e.g clicking a button in Treeherder). Actions are how it is possible to retrigger or "Add New Jobs". For more information, see Taskgraph's [actions documentation]. (target-tasks-1)= [actions documentation]: https://taskcluster-taskgraph.readthedocs.io/en/latest/howto/create-actions.html [graph generation documentation]: https://taskcluster-taskgraph.readthedocs.io/en/latest/concepts/task-graphs.html [kind documentation]: https://taskcluster-taskgraph.readthedocs.io/en/latest/concepts/kind.html [loader documentation]: https://taskcluster-taskgraph.readthedocs.io/en/latest/concepts/loading.html [optimization documentation]: https://taskcluster-taskgraph.readthedocs.io/en/latest/concepts/optimization.html [target_tasks]: https://taskcluster-taskgraph.readthedocs.io/en/latest/concepts/optimization.html [this guide]: https://taskcluster-taskgraph.readthedocs.io/en/latest/howto/run-locally.html [transforms documentation]: https://taskcluster-taskgraph.readthedocs.io/en/latest/concepts/transforms.html