JAWS Commands

Warning

Before these commands will work, you need to set up everything: See Quickstart Example for full setup instructions.

Example submitting a workflow:

jaws submit <WDL> [inputs json] <SITE>

Usage: jaws [OPTIONS] COMMAND [ARGS]…

Run a command with --help to see it’s options.

  jaws --help

  Usage: jaws [OPTIONS] COMMAND [ARGS]...
  JGI Analysis Workflows Service

  Options:
    --config TEXT     JAWS config file
    --user TEXT       User config file
    --log TEXT        Log file
    --log-level TEXT  Logging level [debug|info|warning|error|critical]
    -h, --help        Show this message and exit.

Commands:
  * admin        Administrator Commands
  cancel       Cancel a running workflow; prints whether aborting was...
  cancel-all   Cancel all active workflows for the user.
  download     Force download of failed tasks in a run.
  get-user     Get current user's settings.
  health       Display the current system status and health.
  * history      List past workflow runs submitted by the user.
  info         JAWS version and info.
  inputs       Generate inputs template (JSON) from workflow (WDL) file.
  list-sites   List of all available compute sites for workflow execution.
  * log          View the state transition log for a specific workflow.
  * purge        Remove the cromwell-execution directory for a run.
  * queue        List of the user's currently running workflows.
  resubmit     Resubmit a run (at same compute-site).
  * status       Check the current status of a specific workflow run.
  * submit       Submit a run for execution
  * tasks        Get detailed information about workflow tasks.
  * teams        Manage user teams and permissions.
  * update-user  Update current user's settings.
  validate     Validate a WDL using miniwdl.

The commands with an asterik have additional options. Expand the below commands to see them.

Examples

See status of the JAWS services

Not all sites are represented here yet.

jaws health
{
    "Central": "UP",
    "RabbitMQ": "UP",
    "defiant-Site": "UP",
    "dori-Site": "UP",
    "jgi-Site": "UP",
    "nmdc-Site": "UP",
    "nmdc_tahoma-Site": "UP",
    "perlmutter-Site": "UP",
    "tahoma-Site": "UP"
}

To run a wdl

To run a workflow, you will need two files: a WDL file (.wdl) and an inputs (.json) file. For the latter, you can generate one from scratch by running jaws inputs <WDL>, which will print out a template based on the WDL, you just need to fill out the values.

A simple example template would look like:

{
  fq_count.fastq_file: File
}

To submit a run

jaws submit my.wdl my.json dori

# output looks like
{
  run_id: 7235,
}

Include a tag for your run to help keep track of things. jaws status will display the tag.

jaws submit --tag 'some useful info' my.wdl my.json dori

Run with Cromwell’s call-caching off. Call-caching will allow you to re-run JAWS submissions without re-running tasks that completed successfully. However, this is not always desirable and you can turn caching off, as follows:

jaws submit --no-cache my.wdl my.json dori

Monitor your Run

The examples are in order of verbosity.

# overview of the whole WDL
jaws status 7235

or

# the times each stage started, e.g. queued, running, etc.
jaws log 7235

or

# the times each task entered each stage
jaws tasks 7235

Understanding the Stages

These are the possible states, in order, that a JAWS run passes through.

created:            The Run was accepted and a run_id assigned.
upload queued:      The Run's input files are waiting to be transferred to the compute-site.
uploading:          Your Run's input files are being transferred to the compute-site.
upload failed:      The transfer of your run to the compute-site failed.
upload inactive:    Globus transfer stalled.
upload complete:    Your Run's input files have been transferred to the compute-site successfully.
ready:              The Run has been transferred to the compute-site.
submitted:          The run has been submitted to Cromwell and tasks should start to queue within moments.
submission failed:  The run was submitted to Cromwell but rejected due to invalid input.
queued:             At least one task has requested resources but no tasks have started running yet.
running:            The run is being executed; you can check `tasks` for more detail.
succeeded:          The run has completed successfully.
failed:             The run has failed and will not be transferred unless you run the 'download' command.
complete:           Supplementary files have been written to the run's output dir.
finished:           The task-summary has been published to the performance metrics service.
cancel:             Your run is in the process of being canceled.
cancelled:          The run was cancelled by either the user or an admin.
download queued:    Your Run's output files are waiting to be transferred from the compute-site.
downloading:        The Run's output files are being transferred from the compute-site.
download failed:    The Run's output files could not be transferred from the compute-site.
download inactive:  Globus transfer stalled.
download complete:  Your Run's output (succeeded or failed) have been transferred to the team outdir.
download skipped:   The run was not successful so the results were not downloaded.
fix perms queued:   The fix permissions request has been queued.
fix perms complete: The permissions of the outputs have been changed.
fix perms failed:   The permissions of the outputs could not be changed.
sync complete:      Inode metadata sync delay complete.
slack succeeded:    Notification of run completion was sent via slack.
slack failed:       Notification of run completion could not be sent via slack.
post succeeded:     Notification of run completion was POSTed to webhook URL.
post failed:        Notification of run completion could not be POSTed to webhook URL.
done:               The run is complete.

Get current or old history of jobs owned by you

# get list of your currently running jobs
jaws queue

# view history of your jobs for last 7 days on site Dori
jaws history --days=7 --site=dori

Debugging

The errors.json file is a catch-all command for viewing errors. It is created by JAWS after a run has completed and it will be transfer to the teams output directory.

The errors.json file should be capturing errors from:

cromwell

the WDL tasks

HTCondor backend

Slurm

Sometimes there is additional information in the files created by cromwell:

script.submit - contains the commands used by cromwell to run the script file.
script        - the commands representing cromwell boilerplate stuff in addition to your commands from the wdl-task.
stderr        - the stderr from running script.
stdout        - the stdout from running script.
stderr.submit - the stderr from submission script.
stdout.submit - the stdout from submission script.

These files will be transferred to the team’s output directory solely for tasks that have failed, for the purpose of debugging.

Getting your output

The final outputs from the workflow will be moved to your team’s directory. If a run fails, the resulting final outputs will still be transferred. Furthermore, for any tasks that fail, the complete Cromwell execution folder for that task will be copied over.

jaws status <RUN_ID> display the output_dir for a specific run.

Specialty Commands

This command uses the miniwdl as a linter for your WDLs. You would use this when developing a WDL.

jaws validate my.wdl

The --user flag allows someone to use a different jaws token` than the default. This way, you can have a token representing a user like ‘rqc’ or ‘jaws-admin’ with certain permissions. Then multiple people from a group can use this token to have access to certain files.

jaws --user <~/jaws.conf> <command>

JAWS Deprecated Commands

In this section, you’ll find a compilation of commands deprecated in the previous releases. For each command, we have also provided guidance on finding its current alternative. If you want more details about each release, please check the Release notes.

jaws task-log and jaws task-summary
- These two commands have been merged and into a new command, jaws tasks <RUN_ID>.
- For backward compatibility, jaws task-log will be reporting the output of jaws tasks.
jaws get
- As we transition to copying the output files to the JAWS Teams directory, we are deprecating out the jaws get command. To ensure backward compatibility, this command will remain functional for a few more cycles, serving solely to copy the final workflow outputs.
- jaws status <ID> will provide the path to final path for the output files. Please check output_dir. Then, you can find the expected file tree structure:
```
/<TEAM PATH>/<RUN_ID>/<Cromwell_ID>
```
jaws outfiles, jaws outputs, jaws metadata and jaws errors
- All these commands were deprecated as part of the effort to refactor jaws metadata command (that was using cromwell metadata). As users are submitting large workflows (> 10k tasks), cromwell metadata became too expensive to query and would sometimes timeout. Instead, we now wait for the run to finish and then write some associated reports (e.g. errors, outfiles, and outputs) to disc. These json files are written to the run’s execution directory and transfer to the Team’s directory.
- How to find the report files:
```
ls /<TEAM PATH>/<RUN_ID>/<Cromwell_ID>

<workflow>.wdl
<inputs>.json
errors.json
metadata.json
output_manifest.json
outputs.json
summary.json
tasks.json
```