sub-commands for task manipulation¶

Note

Not all options/arguments are documented below. Always check doit help <cmd> to see a complete list of options.

Let’s use a more complex example to demonstrate the command line features. The example below is used to manage a very simple C project.

DOIT_CONFIG = {'default_tasks': ['link']}

# map source file to dependencies
SOURCE = {
    'main': ["defs.h"],
    'kbd': ["defs.h", "command.h"],
    'command': ["defs.h", "command.h"],
    }

def task_link():
    "create binary program"
    OBJECTS = ["%s.o" % module for module in SOURCE.keys()]
    return {'actions': ['cc -o %(targets)s %(dependencies)s'],
            'file_dep': OBJECTS,
            'targets': ['edit'],
            'clean': True
            }

def task_compile():
    "compile C files"
    for module, dep in SOURCE.items():
        dependencies = dep + ['%s.c' % module]
        yield {'name': module,
               'actions': ["cc -c %s.c" % module],
               'targets': ["%s.o" % module],
               'file_dep': dependencies,
               'clean': True
               }

def task_install():
    "install"
    return {'actions': ['echo install comes here...'],
            'task_dep': ['link'],
            'doc': 'install executable (TODO)'
            }

help¶

doit comes with several commands. doit help will list all available commands.

You can also get help from each available command. e.g. doit help run.

doit help task will display information on all fields/attributes a task dictionary from a dodo file accepts.

list¶

list is used to show all tasks available in a dodo file. Tasks are listed in alphabetical order by default, but –sort=definition can be specified to sort them in the order in which they appear in the dodo file.

$ doit list
compile : compile C files
install : install executable (TODO)
link : create binary program

By default task name and description are listed. The task description is taken from the first line of task function doc-string. You can also set it using the doc attribute on the task dictionary. It is possible to omit the description using the option -q/–quiet.

By default sub-tasks are not listed. It can list sub-tasks using the option –all.

By default task names that start with an underscore(_) are not listed. They are listed if the option -p/–private is used.

Task’s file-dependencies can be printed using the option –deps.

status¶

If you would like to know if a task would actually be executed, the option -s/–status can be used to display the task’s status . It is one of:

R: run

U: up-to-date

I: ignored

This is an alternative to dry-run or preflight feature provided by some tools.

info¶

You can check a task meta-data using the info command. This might be useful when have some complex code generating the task meta-data.

$ doit info link

link

status  : up-to-date

file_dep:
  - command.o
  - kbd.o
  - main.o

targets:
  - edit

Note that if the task is not up-to-date the reason a task is not up-to-date.

$ doit info link

link

status  : run

 * The following file dependencies have changed:
    - main.o
    - kbd.o
    - command.o

forget¶

Suppose you change the compilation parameters in the compile action. Or you changed the code from a python-action. doit will think your task is up-to-date based on the dependencies but actually it is not! In this case you can use the forget command to make sure the given task will be executed again even with no changes in the dependencies.

If you do not specify any task, the default tasks are “forget”.

$ doit forget

Note

doit keeps track of which tasks are successful in the file .doit.db.

–disable-default¶

If your default tasks are expensive, you can avoid accidentally forgetting all of your default tasks by setting forget_disable_default = True in doit.cfg.

You can explicitly forget default tasks with --enable-default.

–all¶

Use doit forget --all to forget all tasks.

clean¶

A common scenario is a task that needs to “revert” its actions. A task may include a clean attribute. This attribute can be True to remove all of its target files. If there is a folder as a target it will be removed if the folder is empty, otherwise it will display a warning message.

Note

The targets’ removal order will be the reverse of their lexical ordering. This ensures that files in a directory are removed before the directory irrespective of their order in the targets array.

The clean attribute can be a list of actions. An action could be a string with a shell command or a tuple with a python callable.

If you want to clean the targets and add some custom clean actions, you can include the doit.task.clean_targets instead of passing True:

from doit.task import clean_targets

def simple():
    print("ok")

def task_poo():
    return {
        'actions': ['touch poo'],
        'targets': ['poo'],
        'clean': [clean_targets, simple],
        }

You can specify which task to clean. If no task is specified the clean operation of default tasks are executed.

$ doit clean

By default if a task contains task-dependencies those are not automatically cleaned too. You can enable this using the option -c/–clean-dep. If you are executing the default tasks this flag is automatically set.

Note

By default only the default tasks’ clean are executed, not from all tasks. You can clean all tasks using the -a/–all argument.

If you like to also make doit forget previous execution of cleaned tasks, use option –forget. This can be made the default behavior by adding the corresponding cleanforget configuration switch:

DOIT_CONFIG = {
    'cleanforget': True,
}

dry run¶

If you want check which tasks the clean operation would affect you can use the option -n/–dry-run.

When using a custom action on dry-run, the action is not executed at all if it does not include a dryrun parameter.

If it includes a dryrun parameter the action will always be executed, and its implementation is responsible for handling the dry-run logic.

def my_cleaner(dryrun):
    if dryrun:
        print('dryrun, dont really execute')
        return
    print('execute cleaner...')

def task_sample():
    return {
        "actions" : None,
        "clean"   : [my_cleaner],
    }

ignore¶

It is possible to set a task to be ignored/skipped (that is, not executed). This is useful, for example, when you are performing checks in several files and you want to skip the check in some of them temporarily.

def task_create_file():
    for i in range(3):
        filename = "file%d.txt" % i
        yield {'name': filename,
               'actions': ["touch %s" % filename]}

$ doit
.  create_file:file0.txt
.  create_file:file1.txt
.  create_file:file2.txt
$ doit ignore create_file:file1.txt
ignoring create_file:file1.txt
$ doit
.  create_file:file0.txt
!! create_file:file1.txt
.  create_file:file2.txt

Note the !!, it means that task was ignored. To reverse the ignore use forget sub-command.

auto (watch)¶

Note

Supported on Linux and Mac only.

auto is provided through the package doit-auto1 plugin. To install it:

$ pip install doit-auto1

auto sub-command is an alternative way of executing your tasks. It is a long running process that only terminates when it is interrupted Ctrl-C. When started it will execute the given tasks. After that it will watch the file system for modifications in the file-dependencies. When a file is modified the tasks are re-executed.

$ doit auto

Note

The dodo file is actually re-loaded/executed in a separate process every time tasks need to be re-executed.

callbacks¶

It is possible to specify shell commands to executed after every cycle of task execution. This can used to display desktop notifications, so you do not need to keep an eye in the terminal to notice when tasks succeed or failed.

Example of sound and desktop notification on Ubuntu.

Contents of a pyproject.toml file:

[tool.doit.commands.auto]
success_callback = """
    notify-send -u low -i /usr/share/icons/gnome/16x16/emotes/face-smile.png "doit:   success"; aplay -q /usr/share/sounds/purple/send.wav
    """
failure_callback = """
    notify-send -u normal -i /usr/share/icons/gnome/16x16/status/error.png "doit:  fail"; aplay -q /usr/share/sounds/purple/alert.wav
    """

Contents of a doit.cfg file:

[auto]
success_callback = notify-send -u low -i /usr/share/icons/gnome/16x16/emotes/face-smile.png "doit:   success"; aplay -q /usr/share/sounds/purple/send.wav
failure_callback = notify-send -u normal -i /usr/share/icons/gnome/16x16/status/error.png "doit:  fail"; aplay -q /usr/share/sounds/purple/alert.wav

`watch` parameter¶

Apart from file_dep you can use the parameter watch to pass extra paths to be watched for (including folders). If paths are folders their sub-folders will not be watched unless these sub-folders are also part of the given extra paths. The watch parameter can also be specified for a group of “sub-tasks”.

import glob

def task_xxx():
    """my doc"""
    LIST = glob.glob('*.xyz') # might be empty
    yield {
        'basename': 'do_x',
        'name': None,
        'doc': 'docs for X',
        'watch': ['.'],
        }
    for item in LIST:
        yield {
            'basename': 'do_x',
            'name': item,
            'actions': ['echo %s' % item],
            'verbosity': 2,
            }

tabcompletion¶

This command creates a completion for bash or zsh. The generated script is written on stdout.

bash¶

To use a completion script you need to source it first.

$ doit tabcompletion > bash_completion_doit
$ source bash_completion_doit

zsh¶

zsh completion scripts should be placed in a folder in the “autoload” path.

# add folder with completion scripts
fpath=(~/.zsh/tabcompletion $fpath)

# Use modern completion system
autoload -Uz compinit
compinit

$ doit tabcompletion --shell zsh > _doit
$ cp _doit ~/.zsh/tabcompletion/_doit

hard-coding tasks¶

If you are creating an application based on doit or if you tasks take a long time to load you may create a completion script that includes the list of tasks from your dodo.py.

$ my_app tabcompletion --hardcode-tasks > _my_app

dumpdb¶

doit saves internal data in a file (.doit.db by default). It uses a binary format (whatever python’s dbm is using in your system). This command will simply dump its content in readable text format in the output.

$ doit dumpdb

strace¶

This command uses strace utility to help you verify which files are being used by a given task.

The output is a list of files prefixed with R for open in read mode or W for open in write mode. The files are listed in chronological order.

This is a debugging feature with many limitations.

can strace only one task at a time
can only strace CmdAction
the process being traced itself might have some kind of cache, that means it might not write a target file if it exist
does not handle chdir

So this is NOT 100% reliable, use with care!

$ doit strace <task-name>

reset-dep¶

This command allows to recompute the information on file dependencies (timestamp, md5sum, … depending on the check_file_uptodate setting), and save this in the database, without executing the actions.

The command run on all tasks by default, but it is possible to specify a list of tasks to work on.

This is useful when the targets of your tasks already exist, and you want doit to consider your tasks as up-to-date. One use-case for this command is when you change the check_file_uptodate setting, which cause doit to consider all your tasks as not up-to-date. It is also useful if you start using doit while some of your data as already been computed, or when you add a file dependency to a task that has already run.

$ doit reset-dep

Warning

reset-dep will NOT recalculate task values and result. This might not be the correct behavior for your tasks!

It is safe to use reset-dep if your tasks rely only on files to control its up-to-date status. So only use this command if you are sure it is OK for your tasks.

If the DB already has any saved values or result they will be preserved otherwise they will not be set at all.

Donate

OpenCollective

Paypal

sub-commands for task manipulation¶

help¶

list¶

status¶

info¶

forget¶

–disable-default¶

–all¶

clean¶

dry run¶

ignore¶

auto (watch)¶

callbacks¶

`watch` parameter¶

tabcompletion¶

bash¶

zsh¶

hard-coding tasks¶

dumpdb¶

strace¶

reset-dep¶

Donate

OpenCollective

Paypal

sub-commands for task manipulation¶

help¶

list¶

status¶

info¶

forget¶

–disable-default¶

–all¶

clean¶

dry run¶

ignore¶

auto (watch)¶

callbacks¶

watch parameter¶

tabcompletion¶

bash¶

zsh¶

hard-coding tasks¶

dumpdb¶

strace¶

reset-dep¶

`watch` parameter¶