============= JAWS Commands ============= .. role:: bash(code) :language: bash .. warning:: Before these commands will work, you need to set up everything: See :doc:`Quickstart Example ` for full setup instructions. Example submitting a workflow: :bash:`jaws submit [inputs json] ` Usage: jaws [OPTIONS] COMMAND [ARGS]... --------------------------------------- Run a command with :bash:`--help` to see it's options. .. code-block:: text jaws --help Usage: jaws [OPTIONS] COMMAND [ARGS]... JGI Analysis Workflows Service Options: --config TEXT JAWS config file --user TEXT User config file --log TEXT Log file --log-level TEXT Logging level [debug|info|warning|error|critical] -h, --help Show this message and exit. Commands: * admin Administrator Commands cancel Cancel a running workflow; prints whether aborting was... cancel-all Cancel all active workflows for the user. download Force download of failed tasks in a run. get-user Get current user's settings. health Display the current system status and health. * history List past workflow runs submitted by the user. info JAWS version and info. inputs Generate inputs template (JSON) from workflow (WDL) file. list-sites List of all available compute sites for workflow execution. * log View the state transition log for a specific workflow. * purge Remove the cromwell-execution directory for a run. * queue List of the user's currently running workflows. resubmit Resubmit a run (at same compute-site). * status Check the current status of a specific workflow run. * submit Submit a run for execution * tasks Get detailed information about workflow tasks. * teams Manage user teams and permissions. * update-user Update current user's settings. validate Validate a WDL using miniwdl. The commands with an asterik have additional options. Expand the below commands to see them. .. dropdown:: admin :color: info :animate: fade-in .. code-block:: text Usage: jaws admin [OPTIONS] COMMAND [ARGS]... Administrator Commands Options: -h, --help Show this message and exit. Commands: add-team Create a new user team. chown-team Change the team's owner. create-user Create new user and get JAWS OAuth access token. deactivate-user Mark user inactive. del-team Delete a user team disable-sites Turn off site(s) ex) jaws disable-sites cori jaws... enable-sites Turn on site(s) ex) jaws enable-sites cori jaws... queue-wait List compute Sites' estimated queue-wait times (in... update-token Update user's token. .. dropdown:: history :color: info :animate: fade-in .. code-block:: text Usage: jaws history [OPTIONS] Print a list of the user's past runs. Options: --days INTEGER history going back this many days; default=1 --site TEXT limit results to this compute-site; default=all --result TEXT limit results to this result; default=any --tag TEXT List runs from single tag; default=False --wdl TEXT Filter runs by wdl file name; default=any --json-file TEXT Filter runs by json file name; default=any --user TEXT Filter runs by user; default=any --verbose Return more fields; default=False --all List runs from all users; default=False -h, --help Show this message and exit. .. dropdown:: log :color: info :animate: fade-in .. code-block:: text Usage: jaws log [OPTIONS] RUN_ID View the log of Run state transitions for the workflow as a whole. Options: --fmt TEXT the desired output format: [text|json|tab]. -h, --help Show this message and exit. .. dropdown:: purge :color: info :animate: fade-in .. code-block:: text Usage: jaws purge [OPTIONS] RUN_ID remove cromwell-execution directory for a specific run Options: -h, --help Show this message and exit. .. dropdown:: queue :color: info :animate: fade-in .. code-block:: text Usage: jaws queue [OPTIONS] List of user's current runs Options: --site TEXT limit results to this compute-site; default=all --all-users, --all List runs from all users; default=False --fmt TEXT the desired output format: [text|json|tab] -h, --help Show this message and exit. .. dropdown:: status :color: info :animate: fade-in .. code-block:: text Usage: jaws status [OPTIONS] RUN_ID Check the current status of a specific workflow run. Options: --verbose Returns ids that are useful for debugging if you have admin privileges -h, --help Show this message and exit. .. dropdown:: submit :color: info :animate: fade-in .. code-block:: text Usage: jaws submit [OPTIONS] WDL_FILE [INPUTS]... SITE Submit a run for execution. Options: --tag TEXT identifier for the run. --no-cache Disable call-caching for this run. --quiet Don't print copy progress bar. --sub TEXT Subworkflows zip (optional; by default, auto-generate). --team TEXT User-team to associate with this run. --webhook TEXT Deprecated. Please use `jaws status` or `jaws log` to retrieve updates. If provided, JAWS will POST to this URL when Run completes. --forcequeue Submit Run to an unavailable site. --overwrite-inputs Overwrite input files that already exist in JAWS cache. -h, --help Show this message and exit. .. dropdown:: tasks :color: info :animate: fade-in .. code-block:: text Usage: jaws tasks [--fmt FORMAT] Get detailed information about workflow tasks for a specific run. Arguments: - `run_id`: The ID of the workflow run to get task information for (required) Options: - `--fmt FORMAT`: Specify the output format. Options are: - `txt`: Human-readable table with space-aligned columns (default) - `json`: JSON format with header and data arrays - `tab`: Tab-separated values, machine-parseable - `csv`: Comma-separated values, machine-parseable Examples: # Get tasks for run 12345 in default text format jaws tasks 12345 # Get tasks for run 12345 in JSON format jaws tasks 12345 --fmt json # Get tasks for run 12345 in tab-separated format jaws tasks 12345 --fmt tab # Get tasks for run 12345 in CSV format jaws tasks 12345 --fmt csv CSV Format (csv) Example: - Comma-separated values with proper quoting for fields containing spaces: TASK_DIR,JOB_ID,STATUS,QUEUE_START,RUN_START,RUN_END,QUEUE_MIN,RUN_MIN,CACHED,TASK_NAME,REQ_CPU,REQ_GB,REQ_MIN,CPU_HRS,RETURN_CODE,INPUT_SIZE_B,OUTPUT_SIZE_B call-alignment,4249181,succeeded,"2025-05-12 10:34:42","2025-05-12 10:34:49","2025-05-12 10:34:56",0,0,False,bbtools.alignment,1,1,10,0.0,0,3727,1222884 call-samtools,4249182,succeeded,"2025-05-12 10:35:21","2025-05-12 10:35:29","2025-05-12 10:35:31",0,0,False,bbtools.samtools,1,2,10,0.0,0,4823,22847 - Column Descriptions - **TASK_DIR**: The task directory name within the workflow - **JOB_ID**: The Cromwell job ID for this task - **STATUS**: Current status of the task (e.g., succeeded, failed, running) - **QUEUE_START**: Timestamp when task entered queue - **RUN_START**: Timestamp when task started running - **RUN_END**: Timestamp when task finished running - **QUEUE_MIN**: Minutes spent in queue - **RUN_MIN**: Minutes spent running - **CACHED**: Whether the task was called from cache (True/False) - **TASK_NAME**: The full task name (fully qualified) - **REQ_CPU**: Requested CPU cores - **REQ_GB**: Requested memory in GB - **REQ_MIN**: Requested runtime in minutes - **CPU_HRS**: Actual CPU hours used - **RETURN_CODE**: Exit code from the task execution - **INPUT_SIZE_B**: Input size in bytes - **OUTPUT_SIZE_B**: Output size in bytes .. dropdown:: teams :color: info :animate: fade-in .. code-block:: text Usage: jaws teams [OPTIONS] COMMAND [ARGS]... Teams Commands Options: -h, --help Show this message and exit. Commands: add-site Grant team access to a site (restricted) add-user Add a user to a team get-site-path Get a team's site config list List all teams list-sites List a team's sites. members List the users associated with a team. my-teams List the teams to which you belong remove-site Remove team access to a site (restricted) remove-user Remove a user from a team set-site-path Configure a team's folder at a jaws-site (restricted) .. dropdown:: update-user :color: info :animate: fade-in .. code-block:: text Usage: jaws update-user [OPTIONS] Update existing user in JAWS Options: --email TEXT Your email address. --name TEXT Your full name. --slack_id TEXT Your slack Member ID. --slack_webhook TEXT Your slack webhook. -h, --help Show this message and exit. Examples -------- **See status of the JAWS services** +++++++++++++++++++++++++++++++++++ Not all sites are represented here yet. .. code-block:: text jaws health { "Central": "UP", "RabbitMQ": "UP", "defiant-Site": "UP", "dori-Site": "UP", "jgi-Site": "UP", "nmdc-Site": "UP", "nmdc_tahoma-Site": "UP", "perlmutter-Site": "UP", "tahoma-Site": "UP" } **To run a wdl** ++++++++++++++++ To run a workflow, you will need two files: a WDL file (`.wdl`) and an inputs (`.json`) file. For the latter, you can generate one from scratch by running :bash:`jaws inputs `, which will print out a template based on the WDL, you just need to fill out the values. A simple example template would look like: .. code-block:: text { fq_count.fastq_file: File } **To submit a run** +++++++++++++++++++ .. code-block:: text jaws submit my.wdl my.json dori # output looks like { run_id: 7235, } Include a tag for your run to help keep track of things. :bash:`jaws status` will display the tag. .. code-block:: text jaws submit --tag 'some useful info' my.wdl my.json dori Run with Cromwell's call-caching off. Call-caching will allow you to re-run JAWS submissions without re-running tasks that completed successfully. However, this is not always desirable and you can turn caching off, as follows: .. code-block:: text jaws submit --no-cache my.wdl my.json dori **Monitor your Run** ++++++++++++++++++++ The examples are in order of verbosity. .. code-block:: text # overview of the whole WDL jaws status 7235 or # the times each stage started, e.g. queued, running, etc. jaws log 7235 or # the times each task entered each stage jaws tasks 7235 **Understanding the Stages** ++++++++++++++++++++++++++++ These are the possible states, in order, that a JAWS run passes through. .. code-block:: text created: The Run was accepted and a run_id assigned. upload queued: The Run's input files are waiting to be transferred to the compute-site. uploading: Your Run's input files are being transferred to the compute-site. upload failed: The transfer of your run to the compute-site failed. upload inactive: Globus transfer stalled. upload complete: Your Run's input files have been transferred to the compute-site successfully. ready: The Run has been transferred to the compute-site. submitted: The run has been submitted to Cromwell and tasks should start to queue within moments. submission failed: The run was submitted to Cromwell but rejected due to invalid input. queued: At least one task has requested resources but no tasks have started running yet. running: The run is being executed; you can check `tasks` for more detail. succeeded: The run has completed successfully. failed: The run has failed and will not be transferred unless you run the 'download' command. complete: Supplementary files have been written to the run's output dir. finished: The task-summary has been published to the performance metrics service. cancel: Your run is in the process of being canceled. cancelled: The run was cancelled by either the user or an admin. download queued: Your Run's output files are waiting to be transferred from the compute-site. downloading: The Run's output files are being transferred from the compute-site. download failed: The Run's output files could not be transferred from the compute-site. download inactive: Globus transfer stalled. download complete: Your Run's output (succeeded or failed) have been transferred to the team outdir. download skipped: The run was not successful so the results were not downloaded. fix perms queued: The fix permissions request has been queued. fix perms complete: The permissions of the outputs have been changed. fix perms failed: The permissions of the outputs could not be changed. sync complete: Inode metadata sync delay complete. slack succeeded: Notification of run completion was sent via slack. slack failed: Notification of run completion could not be sent via slack. post succeeded: Notification of run completion was POSTed to webhook URL. post failed: Notification of run completion could not be POSTed to webhook URL. done: The run is complete. **Get current or old history of jobs owned by you** +++++++++++++++++++++++++++++++++++++++++++++++++++ .. code-block:: text # get list of your currently running jobs jaws queue # view history of your jobs for last 7 days on site Dori jaws history --days=7 --site=dori **Debugging** +++++++++++++ The :bash:`errors.json` file is a catch-all command for viewing errors. It is created by JAWS after a run has completed and it will be transfer to the teams output directory. The :bash:`errors.json` file should be capturing errors from: 1. cromwell 2. the WDL tasks 3. HTCondor backend 4. Slurm Sometimes there is additional information in the files created by cromwell: .. code-block:: text script.submit - contains the commands used by cromwell to run the script file. script - the commands representing cromwell boilerplate stuff in addition to your commands from the wdl-task. stderr - the stderr from running script. stdout - the stdout from running script. stderr.submit - the stderr from submission script. stdout.submit - the stdout from submission script. These files will be transferred to the team's output directory solely for tasks that have failed, for the purpose of debugging. **Getting your output** +++++++++++++++++++++++ The final outputs from the workflow will be moved to your team's directory. If a run fails, the resulting final outputs will still be transferred. Furthermore, for any tasks that fail, the complete Cromwell execution folder for that task will be copied over. :bash:`jaws status ` display the :bash:`output_dir` for a specific run. **Specialty Commands** ++++++++++++++++++++++ This command uses the `miniwdl `_ as a linter for your WDLs. You would use this when developing a WDL. .. code-block:: text jaws validate my.wdl The :bash:`--user` flag allows someone to use a different :bash:`jaws token` than the default. This way, you can have a token representing a user like 'rqc' or 'jaws-admin' with certain permissions. Then multiple people from a group can use this token to have access to certain files. .. code-block:: text jaws --user <~/jaws.conf>