Any variables experiment

[pd93]

Note

This experiment has been released and is now generally available since Task v3.37.0. This issue is kept for historical purposes only. For up-to-date information, you can read our blog post or check the variables documentation.

Context

This experiment attempts to solve the problems originally described by #140.

Currently, all Task variables are strings. You can define a variable as a YAML string, bool, int, float or null value. However, when task interprets it, it will be decoded as a string regardless of the input type.

For example, the following Taskfile will always output "foo" even when BOOL is false because the variable is being interpreted as a string and an if statement evaluates to true when the string is not empty:

version: 3

tasks:
  foo:
    vars:
      BOOL: false
    cmds:
      - '{{ if .BOOL }}echo foo{{ end}}'

Lists are also interpreted as strings and this has led to some annoying workarounds in the codebase. For example, the for feature is great for running a task multiple times with different variables. However, if you want to loop over an arbitrary list of strings, you have to define the list as a delimiter separated string and then split it by specifying the delimiter in the for statement. For example:

version: 3

tasks:
  foo:
    vars:
      LIST: 'foo,bar,baz'
    cmds:
      - for:
          var: LIST
          split: ','
        cmd: echo {{ .ITEM }}

This could be simplified if we supported variables as lists:

version: 3

tasks:
  foo:
    vars:
      LIST: [foo, bar, baz]
    cmds:
      - for:
          var: LIST
        cmd: echo {{ .ITEM }}

Proposal

We propose to change the type of internal Task variables to any (otherwise known as an empty interface{} in Go). This will allow users to define variables as any type they want, and Task will interpret them properly when used in tasks. The following types should be supported:

• string
• bool
• int
• float
• array
• map

Adding support for these types is relatively simple by itself. However, there a few changes that will be needed to make the rest of Task's features work nicely with the new variable types:

• for should support iterating over arrays (and maybe maps?) feat: support looping over map variables #1436
• sh needs to be removed and replaced with a new syntax for dynamically defined variables (see backwards compatibility)
• ...

Backwards Compatibility

The current implementation of Task variables allows for dynamic variables to be specified by using the sh subkey. For example:

version: 3

task:
  foo:
    vars:
      CALCULATED_VAR:
        sh: 'echo hello'
    cmds:
      - 'echo {{ .CALCULATED_VAR }}'

Running task foo will output the following:

task: [foo] echo hello
hello

Since we are adding support for map variables, this syntax will conflict and can no longer be supported. Instead, we should detect string variables that begin with a $ and interpret them as a command to run. For example:

version: 3

task:
  foo:
    vars:
      CALCULATED_VAR: '$echo hello'
    cmds:
      - 'echo {{ .CALCULATED_VAR }}'

If a user wants a string variable that starts with $, they can escape it with a backslash: \$.

Removing the sh subkey will break any Taskfiles that currently use it and it is possible that the new syntax will also break existing taskfiles that have variables that start with $. For this reason, the functionality in this proposal will stay as an experiment until at least the next major version of Task.

[task-bot]

This issue has been marked as an experiment proposal! 🧪 It will now enter a period of consultation during which we encourage the community to provide feedback on the proposed design. Please see the experiment workflow documentation for more information on how we release experiments.

[andreynering]

Thanks for opening this issue!

Some small comments:

• CLI_ARGS is intentionally a string, so it's as easy as possible to delegate arguments like yarn {{.CLI_ARGS}}. We should keep it as a string for this use case, but we can set an additional var which would be an array.
• There's an irony of suggesting $ as a prefix for variables, because that's how Task detected commands in the beginning, but it was eventually deprecated in favor of sh: 🙂. We would be returning to its origins. Perhaps there's a better than way than that prefix, but I'm out of ideas at the moment.

[pd93]

• CLI_ARGS is intentionally a string, so it's as easy as possible to delegate arguments like yarn {{.CLI_ARGS}}. We should keep it as a string for this use case, but we can set an additional var which would be an array.

Ok, that makes sense. I'll remove this section for now. We can revisit down the line if necessary. This reduces the amount of breaking changes.

• There's an irony of suggesting $ as a prefix for variables, because that's how Task detected commands in the beginning, but it was eventually deprecated in favor of sh: 🙂. We would be returning to its origins. Perhaps there's a better than way than that prefix, but I'm out of ideas at the moment.

I don't think I knew about Task in the days before sh, but I actually agree. I'm not convinced by this syntax and it seems like it could cause problems/confusion. However, I'm not entirely sure what a good alternative is either. I suggested this approach as there appeared to be some consensus in #140. However, the comments there are quite old, so perhaps a fresh think about this is in order.

There is always the option of not supporting maps initially. This would actually mean that we could release this without a major version bump as I don't think there would be any remaining backward-incompatible changes.