All Collections
Execute Pipelines
Advanced: Executable Wrappers
Advanced: Executable Wrappers

For advanced configuration of parameterized pipelines

Gwenn Berry avatar
Written by Gwenn Berry
Updated over a week ago

About Executable Wrappers

Executable Wrappers are a one-time setup configuration that controls how Miqa interacts with your component. Wrappers allow you to "generically" pass runtime-relevant information, such as workflow parameters, dataset properties, and other configurations to the component to parameterize the execution.

Diagram

Description automatically generated

In your initial pipeline setup, you or the pipeline administrator likely created the only Executable Wrappers you will ever need for that pipeline's steps. To view the Executable Wrappers associated with your workflows, check out the Dynamic Wrapper Configuration section of the Workflow overview page.

Tip: you can quickly get to the Workflow overview page for any execution by clicking the Workflow link in the Details card of the Execution page.

Dynamic Wrapper Configuration

You will see three elements in the table corresponding to each step of the workflow: the Step details, the Dynamic Config, and the Effective Config. These are described below.

Step

The step name is listed in the first column of the table. Clicking on this name opens the Step overview page, which shows both the Component and the Executable Wrapper. From the step view, you can add or edit the Executable Wrapper in use by using the edit button or the tooltip button.

Dynamic Config

The Dynamic Config section shows the configuration to translate dynamic variables from the Executable Wrapper for this particular workflow version. Different workflows that use the same components may have different Dynamic Configs while sharing the same underlying components and Executable Wrappers. The Dynamic Config follows the same format and variable types as the Executable Wrappers (described below) and can be edited as JSON directly from this view.

Effective Config

The Effective Config section shows the resulting config values that will be passed to the executor at runtime. It combines the information of the Executable Wrapper and Dynamic Config to populate default and constant variables as well as sourcing instructions for dynamic runtime variables.

Creating Executable Wrappers

You can create new executable wrappers for new components, or new use-cases for existing components, from the Executable Wrapper Management view. Wrappers are defined by a JSON configuration and wrapper type. Most users should use the API Wrapper type and choose Auto-Generate Wrapper from Config. For more advanced use cases, contact your Magna Labs support team.

Wrapper Variables

Wrappers are comprised of a list of variables that may be used in executions (e.g. execution templates or scripts) and the variable types, optionally provide default values and instructions on where to source these variables from. Variables that are sourced dynamically at runtime (e.g. by properties of the Dataset being run) are designated as "dynamic" and are further configured. Named attribute, parameter strings, and docker/component property variables are automatically sourced based upon the Component ParameterSets used in the workflow version that is being executed.

Variable Types

The main types of variables and their configurations are described below.

  • Dynamic

    • Dataset attributes and input files

    • Configured by Workflow hookup

  • Component Parameters

    • Can set defaults in wrapper and/or component

    • Can override in Component Parameters (--> Workflow Variant)

    • Types: named_attribute, params_string

  • Component Version

    • Docker

  • Execution

  • Environment

Type

Required

Optional

Description

docker

default

Get docker key from component/version instructions

instructions_item

key

default

Get item by key from component/version instructions

static

default

Static parameter

source_attribute

key

default

Get a property of the input datasource by key

file_by_identifier

identifier

returntype (full*, bucket, key, keybase)

Get file associated with the input datasource by identifier

params_string

default

Get params string from workflow variant

named_attribute

key

default

Get named attributes from workflow variant

outdir

Get output directory location for execution

execid

Get unique identifier for execution (for posting execution status)

servername

Get Facet server name (for posting execution status)

See also:

Did this answer your question?