Controllable pre/post task arguments

[bitprophet]

I.e.:

@task
def pretask():
    pass

@task(pre='pretask')
def realtask(arg):
    pass

This will fail because at the bottom of Executor.execute() we build a task list and then give all of them the CLI arguments. I.e. pre/post tasks are given the same input as the "main" task.

The API we would need to fix this is probably:

@task
def pretask():
    pass

@task(pre={'pretask': []})
def realtask(arg):
    pass

Or, with a pretask that has an interesting signature:

@task
def pretask(myarg):
    pass

@task(pre={'pretask': ['foo']})
def realtask(arg):
    pass

Would be equivalent to calling pretask('foo') prior to calling realtask().

Allowing a dict for keyword args would also make sense; however the full implementation would want both an args list and a kwargs dict, which would then both need wrapping in a tuple or list. That gets pretty fugly pretty fast:

@task(pre={'pretask': ([], {'kwarg': 'value'}})

Another possibility (this is actually what nate-pycon on IRC originally thought of) is to reuse the parser and say e.g.:

@task(pre=['pretask --kwarg=value'])

However I think even the most complex pure-Python version above is preferable, because:

• it's not sufficient to be the ONLY way of doing this, because forcing a serialize/deserialize step when pure Python is much faster/easier;
• if we make it just another alternative, that's almost definitely crossing my "too many ways to do it -> confusing and overly complex code" threshold

[bitprophet]

See also #120 and #123 though they're only related in that we might as well fix them when we're in this part of the code.

[bitprophet]

Should also make sure that the default behavior (i.e. @task(pre=['justthetaskname']) or @task('pretaskhere') calls the subtask with zero arguments.

[xarg]

There is also a possibility to avoid using any args, kwargs in the decorator (I would actually prefer this one).

@task
def pretask(myarg):
    pass

@task('pretask')
def realtask(arg):
    pass

In order to pass myarg it should be supplied at the run time. Like so:

inv realtask --arg 123 --myarg 456

If you really need to execute a task from within another task with arguments, then it would be better to just call that task directly:

@task
def realtask(arg):
    execute('pretask', 123, 'abc', kwarg1=345, kwarg2='hello'))
    ...
    execute('posttask', 123)

I also find the above preferable than the form below:

@task(pre={'pretask': ([123, 'abc'], {'kwarg1': 345, 'kwarg2': 'hello'})}, post={'posttask': ([123])})
 def realtask(arg):

[patcon]

Forgive me the silly question, but why use execute() over simply running the function directly?

[xarg]

@patcon execute would preserve the context in which the task is run. However this function doesn't exist in invoke. I took inspiration from fabric's execute: http://docs.fabfile.org/en/1.8/api/core/tasks.html?highlight=execute#fabric.tasks.execute

[patcon]

Ah gotcha. Haven't gotten into contexts yet, but that makes sense :)

[matleh]

Is there another benefit besides deduplication of pre-tasks to using

@task('pretask')
def realtask():
    pass

as opposed to

def realtask():
    pretask()

?
Because, if that is the only benefit, wouldn't it be possible to have a run_task method on the Context which only runs the given task if it has not been run during that invocation? That would render the pre parameter unnecessary and also make the parameter passing between tasks and their pre-tasks explicit.

Any of the suggested solutions break as soon as a pre-task depends on one of the params of the real-task....

[bitprophet]

@humanfromearth Some thoughts:

• Yes, executing (either directly or via some analogue to execute()) pre/post tasks "manually" within the task body can work in simple cases - totally true.
• However, there are use cases where users will want the task body executed >1 time, while the pre/post tasks only run 1 time - that use case doesn't work well with the above.
  • Counterpoint: continue using execute() as is done with Fabric 1, with a top level task running once, and internally calling pre tasks, another "real" main task, and then post tasks.
  • However, it still feels like something the core API should be able to handle, if it can be done in a non crazy fashion.

---

Unrelated thought I had while rereading my original description: we do not need to do the complex args-and-kwargs version of the API, because all "args" can always be called as "kwargs" instead (there's no such thing as a truly positional-only argument in Python.)

So we could possibly implement that API as just ("just"...) two levels of dicts:

@task(pre={'othertask': {'somearg': 'someval'}})
def maintask(): pass

I think that this level of complexity is also a good argument for allowing additional decorators besides @task as it would allow making this less complex/ugly. E.g.:

@pre('othertask')
@task
def maintask(): pass

@pre('othertask', arg='val') # equiv to othertask(arg='val')
@task
def maintask2(): pass

Food for thought.

[bitprophet]

@matleh See my above response, it addresses your question as well to some degree (re: why not call pre-tasks within task bodies.)

[bitprophet]

I am working on this now, FTR.

[bitprophet]

Refactored somewhat and at least fixed it so pretasks are given no args by default.

Kicking around the ideas above about the best api for this: https://gist.github.com/bitprophet/16a273d07772207e6f9d

[bitprophet]

So far people on IRC seem to be leaning towards option 2, i.e. @pre, which is also what I had been leaning towards.

@shazow also reminded me that we don't need a decorator to get call semantics, i.e. this is possible:

@task(pre=[call(positional, kw=arg), ...])

Will need to think more & get more feedback.