Jobman

Jobman: A meta-scheduler for cluster computing.

Features

  1. Ingest jobs from directories.
  2. Distribute jobs to multiple workers.
  3. Run jobs on local and cluster resources.

Quickstart

  1. Ensure Requirements:

    $ python --version
# Should be >= 3

  2. Install:

    Option A: pip install:

    Option B: git clone:

    1. Clone:

      git clone https://github.com/aspuru-guzik-group/jobman jobman_repo

    2. Make an alias to the standalone jobman script:

      $ alias jobman="$PWD/jobman_repo/standalone-jobman-entrypoint.sh"

  3. Sanity check:

    $ jobman sanity_check
Hello Jobman!

  4. Generate default config:

    $ jobman generate_config > my_jobman_cfg.py

  5. Initialize dbs & directories:

    $ jobman initialize
Initialized.

  6. Create a basic job dir:

    $ jobman generate_echo_job_dir \
  --job_dir_path="job_dirs/inbox/my_job_dir" \
  --message="my message!"
Created job_dir at 'job_dirs/inbox/my_job_dir'

  7. Run the job’s entrypoint manually:

    $ job_dirs/inbox/my_job_dir/entrypoint.sh
my message!

  8. Run the job via Jobman:

    $ jobman --cfg=my_jobman_cfg.py tick --nticks=5
$ find job_dirs -maxdepth 3
job_dirs
job_dirs/completed
job_dirs/completed/my_job_dir
job_dirs/completed/my_job_dir/entrypoint.sh
job_dirs/completed/my_job_dir/job_spec.json
job_dirs/completed/my_job_dir/JOBMAN-COMPLETED
job_dirs/completed/my_job_dir/JOBMAN.ENTRYPOINT.sh
job_dirs/completed/my_job_dir/stderr
job_dirs/completed/my_job_dir/stdout
job_dirs/failed
job_dirs/inbox
job_dirs/queued

  9. Run Jobman as a Daemon:

    @TODO!

Getting Started