Command line interface - run¶

A general doit command goes like this:

$ doit [run] [<options>] [<task|target> <task_options>]* [<variables>]

The doit command line contains several sub-commands. Most of the time you just want to execute your tasks, that’s what run does. Since it is by far the most common operation it is also the default, so if you don’t specify any sub-command to doit it will execute run. So $ doit and $ doit run do the same thing.

The basics of task selection were introduced in Task Selection.

python -m doit¶

doit can also be executed without using the doit script.

$ python -m doit

This is specially useful when testing doit with different python versions.

dodo file¶

By default all commands are relative to dodo.py in the current folder. You can specify a different dodo file containing task with the flag -f. This flag is valid for all sub-commands.

$ doit -f release.py

doit can seek for the dodo.py file on parent folders if the option --seek-file is specified.

as an executable file¶

using a hashbang¶

If you have doit installed on /usr/bin use the following hashbang:

#! /usr/bin/doit -f

using the API¶

It is possible to make a dodo file become an executable on its own by calling the doit.run(), you need to pass the globals:

#! /usr/bin/env python3

def task_echo():
    return {
        'actions': ['echo hi'],
        'verbosity': 2,
        }

if __name__ == '__main__':
    import doit
    doit.run(globals())

Note

The doit.run() method will call sys.exit() so any code after it will not be executed.

doit.run() parameter will be passed to a ModuleTaskLoader to find your tasks.

from IPython¶

You can install and use the %doit magic function to load tasks defined directly in IPython’s global namespace (more).

returned value¶

doit process returns:

0 => all tasks executed successfully

1 => task failed

2 => error executing task

3 => error before task execution starts
(in this case the reporter is not used)

DB backend¶

doit saves the results of your tasks runs in a “DB-file”, it supports different backends:

dbm: (default) It uses python dbm module. The actual DBM used depends on what is available on your machine/platform.

json: Plain text using a json structure, it is slow but good for debugging.

sqlite3: Support concurrent access (DB is updated only once when process is terminated for better performance).

From the command line you can select the backend using the --backend option.

It is quite easy to add a new backend for any key-value store.

DB-file¶

Option --db-file sets the name of the file to save the “DB”, default is .doit.db. Note that DBM backends might save more than one file, in this case the specified name is used as a base name.

To configure in a dodo file the field name is dep_file

DOIT_CONFIG = {
    'backend': 'json',
    'dep_file': 'doit-db.json',
}

verbosity¶

Option to change the default global task verbosity value.

$ doit --verbosity 2

failure-verbosity¶

Option to control if stdout/stderr should be re-displayed in the end of of report. This is useful when used in conjunction with –continue option.

$ doit --failure-verbosity 1

output buffering¶

The output (stdout and stderr) is by default line-buffered for CmdAction. You can change that by specifying the buffering parameter when creating a CmdAction. The value zero (the default) means line-buffered, positive integers are the number of bytes to be read per call.

Note this controls the buffering from the doit process and the terminal, not to be confused with subprocess.Popen buffered.

from doit.action import CmdAction

def task_progress():
   return {
     'actions': [CmdAction("progress_bar", buffering=1)],
}

dir (cwd)¶

By default the directory of the dodo file is used as the “current working directory” on python execution. You can specify a different cwd with the -d/–dir option.

$ doit --dir path/to/another/cwd

Note

It is possible to get a reference to the original initial current working directory (location where the command line was executed) using get_initial_workdir().

continue¶

By default the execution of tasks is halted on the first task failure or error. You can force it to continue execution with the option –continue/-c

$ doit --continue

single task execution¶

The option -s/--single can be used to execute a task without executing its task dependencies.

$ doit -s do_something

parallel execution¶

doit supports parallel execution –process/-n. This allows different tasks to be run in parallel, as long any dependencies are met. By default the multiprocessing module is used. So the same restrictions also apply to the use of multiprocessing in doit.

$ doit -n 3

You can also execute in parallel using threads by specifying the option –parallel-type/-P.

$ doit -n 3 -P thread

Note

The actions of a single task are always run sequentially; only tasks and sub-tasks are affected by the parallel execution option.

Warning

On Windows, due to some limitations on how multiprocess works, there are stricter requirements for task properties being picklable than other platforms.

reporter¶

doit provides different “reporters” to display running tasks info on the console. Use the option –reporter/-r to choose a reporter. Apart from the default it also includes:

executed-only: Produces zero output if no task is executed

json: Output results in JSON format

zero: display only error messages (does not display info on tasks being executed/skipped). This is used when you only want to see the output generated by the tasks execution.

$ doit --reporter json

custom reporter¶

It is possible to define your own custom reporter. Check the code on doit/reporter.py … It is easy to get started by sub-classing the default reporter as shown below. The custom reporter can be enabled directly on DOIT_CONFIG dict.

from doit.reporter import ConsoleReporter

class MyReporter(ConsoleReporter):
    def execute_task(self, task):
        self.outstream.write('MyReporter --> %s\n' % task.title())

DOIT_CONFIG = {'reporter': MyReporter,
               'verbosity': 2}

def task_sample():
    for x in range(3):
        yield {'name': str(x),
               'actions': ['echo out %d' % x]}

It is also possible distribute/use a custom reporter as a plugin.

Note that the reporter have no control over the real time output from a task while it is being executed, this is controlled by the verbosity param.

check_file_uptodate¶

doit provides different options to check if dependency files are up to date (see file_dep (file dependency)). Use the option --check_file_uptodate to choose:

md5: use the md5sum.

timestamp: use the timestamp.

Note

The timestamp checker considers a file is not up-to-date if there is any change in the the modified time (mtime), it does not matter if the new time is in the future or past of the original timestamp.

You can set this option from command line, but you probably want to set it for all commands using DOIT_CONFIG.

DOIT_CONFIG = {'check_file_uptodate': 'timestamp'}

custom check_file_uptodate¶

It is possible to define your own custom up to date checker. Check the code on doit/dependency.py … Sub-class FileChangedChecker and define the 2 required methods as shown below. The custom checker must be configured using DOIT_CONFIG dict.

from doit.dependency import FileChangedChecker

class MyChecker(FileChangedChecker):
    """With this checker, files are always out of date."""
    def check_modified(self, file_path, file_stat, state):
        return True
    def get_state(self, dep, current_state):
        pass

DOIT_CONFIG = {'check_file_uptodate': MyChecker}

output-file¶

The option –output-file/-o let you output the result to a file.

$ doit --output-file result.txt

pdb¶

If the option --pdb is used, a post-mortem debugger will be launched in case of a unhandled exception while loading tasks.

get_initial_workdir()¶

When doit executes by default it will use the location of dodo.py as the current working directory (unless –dir is specified). The value of doit.get_initial_workdir() will contain the path from where doit was invoked from.

This can be used for example set which tasks will be executed:

### README
# Sample to test doit.get_initial_workdir
# First create a folder named 'sub1'.
# Invoking doit from the root folder will execute both tasks 'base' and 'sub1'.
# Invoking 'doit -k' from path 'sub1' will execute only task 'sub1'
##################

import os

import doit

DOIT_CONFIG = {
    'verbosity': 2,
    'default_tasks': None, # all by default
    }


# change default tasks based on dir from where doit was run
sub1_dir = os.path.join(os.path.dirname(__file__), 'sub1')
if doit.get_initial_workdir() == sub1_dir:
    DOIT_CONFIG['default_tasks'] = ['sub1']


def task_base():
    return {'actions': ['echo root']}

def task_sub1():
    return {'actions': ['echo sub1']}

codec_cls¶

doit uses the json package to serialize and deserialize values returned by python-actions. If required an alternate encoder/decoder pair may be specified so that return values which are not JSON serializable, such as Python class instances, may be provided as return values from python-actions.

Check dependency.py ‘s JSONCodec interface.

minversion¶

minversion can be used to specify the minimum/oldest doit version that can be used with a dodo.py file.

For example if your dodo.py makes use of a feature added at doit X and distribute it. If another user who tries this dodo.py with a version older that X, doit will display an error warning the user to update doit.

minversion can be specified as a string or a 3-element tuple with integer values. If specified as a string any part that is not a number i.e.(dev0, a2, b4) will be converted to -1.

DOIT_CONFIG = {
    'minversion': '0.24.0',
}

Note

This feature was added on doit 0.24.0. Older Versions will not check or display error messages.

automatic regex for delayed task loaders¶

When specifying a target for doit run, doit usually only considers usual tasks and delayed tasks which have a target regex specified. Any task generated by a delayed task loader which has no target regex specified will not be considered.

By specifying –auto-delayed-regex, every delayed task loader having no target regex specified is assumed to have .* specified, a regex which matches any target.

Donate

OpenCollective

Paypal

Command line interface - run¶

python -m doit¶

dodo file¶

as an executable file¶

using a hashbang¶

using the API¶

from IPython¶

returned value¶

DB backend¶

DB-file¶

verbosity¶

failure-verbosity¶

output buffering¶

dir (cwd)¶

continue¶

single task execution¶

parallel execution¶

reporter¶

custom reporter¶

check_file_uptodate¶

custom check_file_uptodate¶

output-file¶

pdb¶

get_initial_workdir()¶

codec_cls¶

minversion¶

automatic regex for delayed task loaders¶